Crossroads of Social Science: The Icpsr 25th Anniversary Volume [1st ed.] 0875860915, 9780875860916, 0875860907, 9780875860909, 9780875862705

Citation preview

Crossroads of Social Science

Crossroads of Social Science: The ICPSR 25th Anniversary Volume Heinz Eulau Editor

Agathon Press New Yor k

0 1989 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Agathon i'rcss Agathon Press is iiii imprint 01'

ALGORA ~'UBLISMINC; zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCB

222 Riverside Drivc. zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGF New Y o k h'Y 10023

AN Rights Reserved

No portion of this book may be reproduced by any process, stored in a retrieval system, or transmitted in any form, or by any means, without the express written permission of the publisher.

Printed in the United States

Library of Congress Cataloging-in-Publication Data Crossroads of social science. Bibliography: p. Includes index. 1. Social sciences - Research - Congress. I. Eulau, Heinz, 1915. 11. Inter-university Consortium for Political and Social Research. H63.A3C76 1989 300l.72 88-7421 ISBN 0-87586-090-7 ISBN 0-87586-091-5 (pbk.)

Contents About the Contributors Preface

vii

ix

Introduction: Crossroads of Social Science Heinz Eulau

1

1 , Toward Cumulative Knowledge: Theoretical and Methodological Issues Hubert M. Blalock, JK

15

Comment Progress in Research Methodology: Theory, Technology, Training Herbert E Weisberg

38

2. A Strange Case of Identity Denied: American History as Social Science Allan G. Bogue

51

Comment Time Is on the Historian’s Side Eric Monkkonen

87

3 . When Social Science Findings Conflict Philip E. Converse Comment On Conflict Resolution: Right Makes Right Karl Taeuber

4. What We Know, What We Say We Know: Discrepancies Between Warranted and Unwarranted Conclusions Norval D.Glenn Comment Causal Inferences: Can Caution Have Limits? Susan Welch

91

113

119

141 V

5 . Research Life as a Collection of

Intersecting Probability Distributions Warren E. Miller

147

Comment Innovation: Individuals, Ideas, and Institutions Ivor Crewe

161

Infrastructures for Comparative Political Research Max Kaase

166

Index

vi

175

About the Contributors Hubert M. Blalock, Jr. is Professor of Sociology and Adjunct Professor of Political Science, University of Washington, Seattle. His research interests are in applied statistics, causal modeling, theory construction, conceptualization and measurement, and race relations. His most recent books are Conceptualization and Measurement in the Social Sciences (1982) and Basic Dilemmas in the Social Sciences (1984). He is currently working on a book dealing with power and conflict processes. He is an Associate Director of ICPSR. Allan G. Bogue is the Frederick Jackson Turner Professor of History, University of Wisconsin, Madison. His books include The Earnest Men: Republicans of the Civil War Senate (1981) and Clio and the Bitch Goddess: Quantification in American Political History (1983). A member of the National Academy of Sciences, he has also served as president of several professional associations. He is currently completing The Congressman’s Civil War: An Institutional Reconnaissance. Philip E. Converse is the Robert Cooley Angel1 Distinguished Professor of Sociology and Political Science, University of Michigan, Ann Arbor, and Director of the Institute for Social Research at the university. His books include The American Voter (1960), The Human Meaning of Social Change (1972), The Quality of American Life (1976), and Political Representation in France (1986). Ivor Crewe is Professor and Chair of the Department of Government, University of Essex, England. A former Director of the ESRC Data Archive and Co-director of the British Election Studies, he is currently coeditor of the British Journal of Political Science. Recent books include Decade of Dealignment (1983), British Parliamentary Constituencies (1984), Electoral Change in Western Democracies (1985), and Political Communications: The General Election Campaign of 1983 (1986). Heinz Eulau is the William Bennett Munro Professor of Political Science, Emeritus, Stanford University. A fellow of the American Academy of Arts and Sciences, he edits the journal Political Behavior, published by Agathon Press. He serves as an Associate Director of ICPSR. His latest book is Politics, Self and Society (1986). He is working on a book in the methodology of micro and macro analysis. vii

Norval D. Glenn is Professor of Sociology and Research Associate in the Population Research Center, University of Texas, Austin. He is editor of the Journal of Family Issues and serves on the editorial boards of Public Opinion Quarterly, Journal of Marriage and the Family, and Social Indicators Research. He is an Associate Director of ICPSR. Max Kaase is Professor of Political Science and Comparative Social Research, University of Mannheim, Germany. A former Executive Director of ZUMA, his research interests include empirical democratic theory, political participation, voting behavior, mass communication, and social science methodology. He is the author of Political Action: Mass Participation in Five Western Democracies (1979). Warren E. Miller is Professor of Political Science, Arizona State University, Tempe. He is Director of the National Election Studies, Center for Political Studies, University of Michigan. A former President of the American Political Science Association, his most recent books are Leadership and Change: The New Politics and the American Electorate (1976) and Parties in Transition: A Longitudinal Study of Party Elites and Party Supporters (1986). Eric H. Monkkonen is Professor of History, University of California, Los Angeles. He has published books and articles on U. S. criminal justice history, historical methods, and urban history. His most recent book is America Becomes Urban: The Development of U S . Cities and Towns, 1790-I980 (1988). Karl Taeuber is Professor of Sociology, University of Wisconsin, Madison, and a member of the University’s Institute for Research on Poverty and its Center for Demography and Ecology. His current research is on trends in racial segregation in housing and schooling in the U. S. He is currently serving as chair of the Council of ICPSR. Herbert E Weisberg is Professor of Political Science, Ohio State University, Columbus. A former member of the ICPSR Council, his research interests include voting behavior, Congressional politics, and research methods. He has served as coeditor of the American Journal of Political Science and has edited two volumes: Controversies in American Voting Behavior (1976), and Political Science: The Science of Politics (1986). Susan Welch is the Carl Happold Professor of Political Science at the University of Nebraska, Lincoln. A member of the ICPSR Council, her recent books are Women, Elections, and Representation (1987) and Urban Reform and Its Consequences (1988). viii

Preface

T

he papers brought together here were presented on the occasion of the twenty-fifth anniversary of the Inter-University Consortium for Political and Social Research, held in Ann Arbor, Michigan, in November, 1987. In the instructions to the contributors no attempt was made to harness them to a particular theme or suggest a specific topic. Knowing their interests and experience as well as their diverse and distinguished roles in the development of the social sciences and the Consortium, we were confident that they would have something worthwhile to say on an occasion that called for reflection on the successes and failures of the social sciences over the last few decades as well as on their future. As reading of the chapters in this volume will show, of concern is of course that sector of the social sciences that is devoted to empirical research, theory and method - the sector cutting across the various disciplines that defines the mission of the Consortium as the country’s and perhaps the world’s leading center for archiving quantitative data and for utilizing these data through instruction in relevant theories and methods. Needless to say, it was our hope that the contributors would note the contributions of the Consortium to the development of the social sciences and its relation to broader trends in the various disciplines. However, the Consortium is not the “object” of these essays or in any sense their main focus. Rather, the focus is on generic problems, difficulties, and dilemmas in the social sciences that the contributors are uniquely qualified to articulate. Each of them has been intimately involved in the development of one or another discipline in the last thirty years or so; each has made significant contributions to that development in many ways; each has a personal perspective on accomplishments and failures, promises and needs, continuities to be cultivated and opportunities to be seized. We have brought the anniversary essays together in this volume because, we feel, the experiences and thoughts of their authors can be beneficially shared by the larger social science community and deserve the community’s reflective consideration. Jerome M. Clubb Executive Director, ICPSR ix

Introduction

Crossroads of Social Science Heinz Eulau

T

he title given to this introduction and, by editorial chicanery, to this collection of essays as a whole is not intended to be a hyperbolic metaphor of the imagination. Let us consider an alternative formulation that, in fact, was first suggested in the search for a covering title, “Social Science at the Crossroads.” The image conveyed is considerably more sensational. It would suggest that, here and now, social science as a collective enterprise has come to some intersection of roads where a crucial decision must be made as to the direction in which to move-forward or backward, left or right. The image is one of crisis, possibly conflict, and, depending on a participant’s or observer’s mood, of optimism or pessimism. I do not believe that this image is serviceable as a model to characterize the current state of affairs in social science or that it has ever fit any prior phase or stage in the development of social science. There are, of course, several other models of social science as well, but I believe that a single viable model has yet to be invented. If there is much disagreement as to what social science “looks like” or should look like, it is in part due to what I take as a “fact”- that not all “social scientists” are social scientists. There are those who, in their attempts to define social science, take recourse to philosophies of science, but philosophies of science, it seems to me, are the business of philosophers and not of scientists. Much as the method of “thick description” differs from the methods of quantitative analysis pursued by the contributors to this volume, there is much wisdom in the reminder of the thick-descriptionist anthropologist Clifford Geertz (1 973) that “if you want to understand what a science is, you should look in the first instance not at its theories or its findings, and certainly not at what its apologists say about it; you should look at what the practitioners of it do.” (p. 5 ) . I

2

Eulau

The contributors to this volume are all well-known practitioners reflecting in one way or another on what they as individuals or as members of a social science discipline are doing. There is no lack of apologetic, critical, or programmatic statements in their diagnoses, but they come to these statements as practitioners and not as philosophers. All of them speak from experience in doing theory-driven empirical research, not from the lofty perch of some comprehensive vision that, more likely than not, is built on feet of clay and likely to crumble when the next vision comes along. Yet, there is much heterogeneity in these essays. Reviewers of volumes such as this collection almost invariably complain that the symposium resembles more a potpourri than a symphony, or that the editor has failed to “integrate” the various contributions into some overarching framework. As I shall argue, there are many crossroads where social scientists meet, move, halt, or collide, and the contributors to this symposium, while having more in common than not, also differ in their particular interests or concerns. It cannot be otherwise. If today the word progress is no longer used with as much abandon as it was a hundred or even fifty years ago, we still speak of “advances” in methods or “cumulation” in theory as if there were a single and straight road ahead. This, perhaps, has to be a necessary stance to give any meaning at all to the enterprise, even as the “goal” of a social science appears to be more distant and less translucent than it once was to the founders of modern social science and to early generations of social scientists. Nevertheless, paradoxical though it may seem, there is something almost inevitable in the “progressive” vision. On one hand, as we must talk of advances and cumulation in order to move on toward the goal of a science of social behavior and human relations, we also find that the more we learn or know about the human condition, the less confident we seem to be because the goal proves to be so elusive. In the process of reducing ignorance, increasing knowledge seems to produce more new problems than are solved. Yet, on the other hand, it is this very condition that drives us to go on, so that we can speak in good conscience of disciplinary “frontiers” or “cutting edges” as something more than symbolic terms that find their way into our proposals to funding agencies intent on “promoting” what they think we should be doing. I do not mean this in any critical sense at all: the scientific faith in an advancing and cumulative body of knowledge is the very ruison dPtre for our doing any serious work in social science at all. The contributors to this volume certainly share this faith even as they locate themselves at different crossroads of social science. Individual social scientists, “schools” of social science, specializing

Crossroads of Social Science

3

social science disciplines, or “invisible colleges” within and across the disciplines, in order to give direction to their work and be self-conscious about what they are doing, must locate themselves in time and space. However, I suggest, time and space have implications for the collective behavior of the community of social scientists and the corporate enterprise of social science that are different from those for the behavior of the single social scientist. A single person cannot be at more than one place at one point in time, though he or she can, of course, imagine being at different places at different times; my concern here is not with the scientific imagination of the single scholar but with real-world conditions. Now, “real time” is the same for the individual and the collectivity. As I write this, it is February 1, 1988, 10:23 A . M . , and this is the same time (making allowance for global time zones) for all of my colleagues, that is, for the collectivity of scholars as a whole. However, the members of this collectivity are widely dispersed-that is, they are located not at one crossing of the roads but at many crossroads; and these crossroads are manifold and diverse. Some are like the major intersections of super highways in metropolitan areas, with overpasses and underpasses, bridges and tunnels, four lanes in one direction only and turnoffs to right or left. Others are like the lonely intersections in quiet rural landscapes with wide vistas and travelers only occasionally encountering each other. And there are many other configurations between the extremes. Is it surprising that, at one time or another, we make false turns and take the wrong road? But this is not the point. The point is that different scholars or groups of scholars find themselves at very different crossroads, though they are there at the same time, and their choices of direction in which to travel are constrained by the spatial environment in which the choices must be made. For the collectivity of social scientists as a whole, this means that the territory to be explored and how to explore it are a labyrinth. As I put it many years ago, referring to my own discipline of political science, its members “are riding off in many directions, evidently on the assumption that if you don’t know where you are going, any road will take you there” (Eulau, 1959, p. 91). So, as individual social scientists or as collectives, organized in disciplines, subdisciplines, specializations within and across disciplines, we find ourselves at different crossroads at the same time. The essays presented in this book reflect some of the diversity of concerns of their authors as they locate themselves at different crossroads of social science in terms of their own experiences in traveling along diverse routes. Blalock presents a comprehensive map of new directions, or of old directions lost sight of, and of the many roadblocks at the intersections of modern social science. Bogue, as

4

Eulau zyxwvutsrqpo

behooves the historian, retraces some of the roads his discipline has taken since the beginning of the century and points to some of the (false) turns it has made, and he describes the moves that some historians are making toward what is now called social science history. Converse directly addresses the problem of what can be done and what is, in fact, being done if and when heavy traffic at some crossroads makes for collisions and requires new signs or signals. Glenn is also concerned with a generic crossroads issue: the discrepancy between what we may really know and what we say we know, and how we present our knowledge to ourselves and to the public. Miller, finally, in a very personal document, recalls some of his own travels as a scholar and research administrator, some of them well planned but subject to contingencies at unexpected crossroads that not even a highly goaloriented social scientist could possibly foresee. And six commentators either elaborate and confirm or take exception to these diagnoses. A volume like this, prepared at a particular time in the development of social science, reflects the mood of the time. What is our current mood? How does it come about, and how does it affect our sense of where we are? It is not uncommon to characterize our moods as being either optimistic or pessimistic as to the possibility of a genuine social science. It seems to me that while descriptions of this sort probably contain some validity, they are predicated on a false dualism. I shall say something more about this later on but here proceed on the assumption of the dualism’s validity. There are, I believe, two sources of our changing moods, from optimism to pessimism and back again. One is immanent in the social science enterprise itself. As our many crossroads make for traffic jams, we often seem to be standing still and not getting ahead very much. When the traffic again flows more easily, pessimism about ever getting to where we want to go turns into optimism until we face a next set of obstacles. However, I suspect that our changing moods are rooted less in social science itself, precisely because we are at different crossroads at the same time, and more in the general temper of society in given periods of time. Science itself, of course, influences societal moods, but I am inclined to believe that the preponderance of evidence points to the primacy of societal effects on scientific moods. This is not to say, of course, that societal temper is the “cause” of scientific advances and discoveries. Surely, nineteenth-century scientific findings concerning evolution were not the “result” of the “spirit of progress” that had animated the humanistic enlightenment of the previous century. However, equally surely, the intellectual heritage of the age of enlightenment greatly facilitated the acceptance of science and its discov-

Crossroads of Social Science

5

eries -along with, perhaps needless to say, the arrival and growth of industrial capitalism. One need only compare this favorable nineteenth-century “climate” with the climate surrounding the earliest modern scientists - a Copernicus, a Brahe, a Kepler, a Galileo, or even a Hobbes-whose trials and tribulations are evidence of the inhospitable intellectual milieu maintained by a Church-imposed Aristotelianism. Early social science, coming into its own toward the turn of the century, was the beneficiary of the confluence of societal and scientific moods. Its modest accomplishments notwithstanding, it could afford to be optimistic and “progressive,” even though the notion of progress itself soon became a problem that still leaves social science in a quandary. Indeed, there is the charge that the social science of the 1920s and 1930s was more “scientistic” than scientific as, following World War I and the Great Depression, societal optimism gave way to a more pessimistic outlook. Although “advances” had undoubtedly been made up to this time, there was no longer the kind of optimism concerning the future that had inspired an earlier generation. By the end of the Depression and just before World War 11, Robert S. Lynd (1939) delivered a scathing indictment of social science for asking the wrong questions that, as the evident cycles of optimism and pessimism grew shorter, has been repeated time and again in our own time. In fact, this shortening of cycles, if it is permissible to speak of cycles at all, seems to be the characteristic feature of post-World War I1 social science, to the point, I shall argue, where it makes little sense to think of optimism and pessimism as a duality at all. The roads and crossroads of contemporary social science are too many and too diverse to describe the mood of social science in such simple terms. Optimism and pessimism, it seems to me, are not affective opposites but mutually entailed like the poles of a magnetic field. It is not that one begets the other in an endless cycle of changing moods but that, in our era, they coexist just as action and reaction coexist. What the authors of the essays share is a healthy regard for the multiplex nature of the social science enterprise, the high roads as well as the low of social-scientific developments, the obstacles to advance but also the promises of occasional return, depending on a sense of both destination and realistic opportunities. It is not just difficult but impossible to call them either optimists or pessimists because, it seems to me, they see social science not as a set of methodical routines to be administered but as a set of theoretical problems to be solved. Whatever their personal views of the role of social science vis-a-vis societal needs, demands, or expectations, they strike me as, above all, pragmatists in

6

Eulau zyxwvutsrqpo

the sense of John Dewey’s notion of “creative intelligence” rather than in the sense of some mindless positivism that is empirically empty and, more often than not, ideologically soiled. In characterizing the authors as pragmatic I do not mean to imply that they all are adherents of some particular version of pragmatic philosophy in the technical sense. Rather, I read them as possessed of a pragmatic sensibility that recognizes limits on degrees of freedom and a self-reflective mood that experiences our age as one of both enchantment and disenchantment. They bear witness to the marvels of scientific and technological “progress” in both the hard sciences and the soft, so-called, but they are not insensitive to many dysfunctional and even regressive consequences of scientific development. (It is in this respect, I think, that Karl Marx was the first social scientist of our era.) In short, the mood of our time in social science is as complex as the phenomena that social science makes its own to explore. The dualism of optimism and pessimism has little, if any, purchasing power at the crossroads of contemporary social science. As Kaase notes in his comment, our critics sometimes suggest that social scientists are reinventing the wheel. But this analogy takes us only so far and not further, for it neglects that the wheels being reinvented are differently structured and serve new functions. The analogy’s implication is, of course, that not just the mood but the phenomenon of social science itself is cyclical. Does the notion of cycles have no validity at all, then? Or are cycles themselves related in some unilinear fashion? My own favorite developmental image of social science (which I physically demonstrate in lectures by bringing the tool along) is modeled on a simple kitchen instrument. It is the cone-shaped, elastic-metal, and spirally constructed egg beater, with a handle at one end and spirally widening circles as one moves to the other end, which expands or contracts depending on the amount of pressure one exerts in beating the egg. In this image or model, we do move in circles, seeming to return to the same point on the lateral dimension but doing so at different points on the vertical dimension, depending on whether we move up or down the elastic spiral structure. What is “up” or “down.,” in turn, depends on how we hold the egg beater and look at it. We can look at the tool by turning the handle down, all the better to be able to beat down on the expanding mixture (in our case, of social science research). Or we can look at the tool by turning the handle up with the widest circle of the spiral also up, allowing us to imagine an ever expanding universe (of research activities). My own inclination is to view the development of social science in the latter mode, but being ever mindful of the instrument’s elasticity, spiral structure, and reversibility.

Crossroads of Social Science

7

Given this observational standpoint, it is possible to be especially sensitive to the question of cumulative knowledge. In some sciences, cumulation means that the “new” supersedes the “old,” so that the old is, at best, of antiquarian interest. One does not often find references to Galileo or Newton in the research reports or even the textbooks of physics. By way of contrast, references to Darwin or Mendel are not uncommon in the writings of biologists, though by no means overwhelming more recent theories or discoveries. This is not so in the literature of social science. Our research articles and books abound in references to “classical” theories or more recent but “outmoded” investigations, so that the enterprise looks more “additive” than cumulative. Whatever may be genuinely cumulative stands alongside the merely additive. This situation, I suppose, is due to the everchanging and rapidly changing complexities of social existence that do not permit the kind of “crucial experiment” which, at least for a time, is at the base of current knowledge until overcome, in a cumulative process, by a new experiment. We therefore cannot claim, even with the calculus of probability in mind, that a hypothesis has been falsified at a level of confidence at which we can safely discard it or consider it “overcome” by subsequent research findings. So we carry along, in our intellectual baggage, the wisdom of a Plato, the classifications of an Aristotle, the observations of a Machiavelli, the deductions of a Hobbes, the insights of a Hume, the constructs of a Marx or Weber, the sensitivities of a Mead, the imagination of a Lasswell, and so on. And we do so not only because confirmation or falsification is elusive but also because, in our insecurity, we may find in the classical theories some suggestion still worthy of inquiry or some explanation of what appears to be a new phenomenon. It has always seemed to me, therefore, that the war between the “ancients” and the “moderns”- those who insist on the universality of old, enunciated “truths,” and those who would discard these truths because they see them as obstacles to fresh inquiry-is something of a false war. Both sides, it seems, seek a hegemonic victory that cannot be had. There is much talk in the literature on social science, including some of the essays in this volume, of giving our students proper training in the most advanced methods of social science but also in what is called theory construction. But how does one learn about how to construct a theory? We seem to return, more often than not, to one or another classical writer precisely because, in the study of human affairs, the issue of what constitutes “good” theory is problematic and elusive. The price we pay is costly: given our uncertainties, we do not abandon a natural law theorist like Locke, or a dialectition like Hegel, or an empiricist like Durkheim, or a choice theorist like Smith, or an

8

Eulau zyxwvutsrqpo

organicist like Spencer, or an action theorist like Parsons, or a functionalist like Malinowski. We return to the classics selectively, of course, but it is precisely our selection that is the issue when we talk about theory construction. For by what criterion are we to select? As far as I know, there is no consensual answer. The difficulty involved accounts, in part at least, for the once-upon-atime popularity of Thomas Kuhn’s notions of “paradigmatic change” and “scientific revolution.” That Kuhn’s theory was so popular at a time, in the 1960s, when “revolution” was much in the air and seemed to define the mood of many social scientists is not the issue here, and that Kuhn may have been wrong or, if right, may have been misunderstood or misinterpreted is also not the issue. What concerns me is that Kuhn was evidently read by many colleagues and students, whether themselves “revolutionary” or not, as justifying intellectual shortcuts that they were all too ready and willing to follow. For if, as they interpreted Kuhn, paradigmatic change is revolutionary, the theory seemed to free them from having to carry the intellectual ballast of the past. Their error was, of course, the convenient assumption that revolutions have no antecedents and always constitute a new beginning that is not constrained by the past. No social science can be built on such an erroneous and superficial assumption, and it was abandoned as quickly as it appeared. (Needless to say, perhaps, I do not imply that today’s students should not read Kuhn: to understand our current mood and situation makes reading him mandatory- his work is not obsolete but has become a classic in its own right.) The moodiness of moods I can best illustrate by reference to my own discipline of political science. Only a few years after one (then future) president of the American Political Science Association (APSA) had announced victory for the “behavioral movement” (Dahl, 1961), another president called for a “new revolution” in the name of something he called “postbehavioralism” (Easton, 1969); and exactly ten years later, another president asserted that most prior behavioral research had really been “prebehavioral” (Wahlke, 1979). What these meanderings suggest is either that professional memories are very short, or that professional visions are distorted, or that dreams of the future are altogether unrealistic. If a discipline’s professional leaders can be so utterly at variance, what can we reasonably expect from students, who are inclined to think that the latest is always the best? I am reminded of the contrary wisdom of the late Harold D. Lasswell, who never mistook the present for the future. In a letter written on the occasion of the publication of the International Encyclopedia of the Social Sciences in 1958 (on which I served as associate editor for political science), he wrote, “I

Crossroads of Social Science

9

should like to hear what your present reflections are on the way to conduct an enterprise of this kind. After all, this may not be the last encyclopedia.” For Lasswell, the best was yet to come. “The best is yet to come” might well be the motto that could be inscribed on a flyleaf of this book and, beyond it, on the work of the InterUniversity Consortium for Political and Social Research (ICPSR), whose contributions are celebrated in many of its pages. As one compares the programmatic statements of the APSA presidents with the work carried out by the ICPSR in exactly the same period of social science development, one must be impressed by a degree of continuity-in-change that has kept the goal - a science for social science - clearly in mind without confusing a possible future for the present. Although there have been significant changes in the consortium’s doings, on both the archival and the instructional sides, there has been no self-proclaimed revolution, such as so often entices intellectual movements. If the best was yet to come, it required more than an idea, critical as the idea for a sharing of social science data was to make a beginning. So the consortium may have begun as an intellectual-social movement, but the movement would surely have failed without organization. As Warren Miller, the consortium’s founder and first director, vividly narrates, organizational directions are nevertheless often the result of contingencies and unexpected consequences that defy rational planning but present new opportunities. How to seize these opportunities in a creative manner without becoming their slave is the art of statesmanship, which is as critical in the world of science as in the world of politics. Not all the siren calls of foundations or government agencies necessarily or inevitably work in the best interests of a science. Nevertheless, I must take exception to the statements of those humanistic critics of behavioral science who deplore what they call the “opportunistic” stance of social scientists, as if seizing opportunities and bending them to one’s purposes were somehow a criminal act- a kind of “betrayal of the intellectuals.” It is a curious argument and incorrect. As far as the ICPSR is concerned, there have probably been opportunities not seized that equal those seized or sought, but they are not reported and therefore remain unknown. As a participant in the consortium’s cours d’honneur from the beginning, I have always been impressed by the fact that despite changing council and staff personnel, the ICPSR has managed to serve an expanding and heterogeneous research community without falling into the trap of those fads and foibles that plague other sectors of social science. Miller’s, and also Crewe’s and Kaase’s, stories of contingencies pertain

10

Eulau zyxwvutsrqpo

not only to infrastructural but also to intellectual opportunities. (For some reason, critics of an “opportunistic” social science do not find unexpected intellectual encounters opportunistic.) These intellectual opportunities are more difficult to capture in retrospect because they are more of a personal than of an organizational sort. Organizations may have memories, but they do not think. Yet, the thinking of individuals has consequences for the organization. I am persuaded, after many years of observation, that the contributions made to the consortium by its council members do not lie in their pouring over annual budgets or infrastructural aspects of the organization, but in the ideas on scientific direction they bring with them and call to organizational attention. Most directly, these ideas work their way into the ICPSR’s summer instructional program, which is sometimes described, wrongly I think, as “training in methodology.” It involves, admittedly, training in some and often advanced methods of social-scientific inquiry and statistical techniques but, only in a limited way, education in the logos of method. It is refreshing to find that in their papers here Glenn and Converse pick up some fundamentals of methodology as that term should be properly used. Both are really concerned with the problem of what we should make of our findings when they seem to tell us either less or more than the methods employed to make the findings permit. Glenn reminds us that even if our inferences from our statistical analyses are not so verbally inaccurate as to violate the calculus of probability, even these inferences can and should be kept in harness by such “side information” (including presumably “qualitative” evidence) as may be legitimately brought upon them. Converse, in turn, calls our attention to meta-analysis, which seeks to cope with conflicting findings not just through verbal interpretation but through that oldest of scientific techniques: classification. In our quest for ever more sophisticated mathematical or statistical modeling and analysis, it seems that we have increasingly lost sight of the problems of conceptualization and operalization, as Blalock - himself a leading figure in causal, nonexperimental analysis - again mentions in his paper here as he has on numerous other occasions. To this I would add the art of observation, including under the rubric the kind of interviewing done in survey research. It seems to have become a neglected theme in the methodic training of our students. I want to say a few things about it because it involves the consortium in an indirect way. Although I can only call it a hunch, my impression is that the very success of the ICPSR in making data widely available and training students in their analysis has engendered the lack of interest in the art of observation that I consider a shortcoming in current graduate instruction. Of course, I

Crossroads of Social Science

11

do not “blame” the consortium for this in the sense of holding it “responsible.” However, given the ease of access to machine-readable data for secondary analysis, we have failed to train recent generations of students in one of the fundamentals of social science research: the sensitive interaction with and observation of our “subjects,” who, after all, are not just “data” but human beings. As I read articles and books in social science, I find the writers talk about correlation or regression coefficients as if these coefficients could walk, talk, think, feel, and so on. The result is a kind of reification of persons and personification of statistics that makes human beings into skeletons - indeed, at the aggregate level, “structures”- without flesh and blood. It also makes for dreary reading. Some kind of field work that brings the student into direct contact with respondents and thus sensitizes him or her to the brittle nature of the survey instrument should be an essential ingredient of social science research training. We may not be able to capture a respondent’s hesitation or quizzical look or mistaken understanding of a question as he or she locates on a point of a 5- or 7-point scale, or as he or she responds with more or less enthusiasm to a question about a political candidate, party, or issue. Are what Converse has called “nonattitudes” the real things or artifacts of the survey instrument? Are what may appear to be random answers really random or a function of question location on the instrument? We still do not know enough about these and many other problems, but I believe that the student will be more aware of them by learning how to observe and interview. And this learning can and must be fed back into the design of a study and the construction of a questionnaire. I have come across colleagues who construct questionnaires or items for a questionnaire who have never interviewed a single person in the field (just as there are interviewers who are not properly informed about the theoretical objectives of the questions they ask). It is truly a deplorable situation. Yet, observation, often unfortunately called data collection, is what connects conceptualization to measurement. Just as we cannot observe without proper concepts as tools, so concepts need to be enriched by what we observe. Theorists, and even some of those who call themselves empirical theorists, give the impression and sometimes even defend the view that armchair speculation about what the world looks like is sufficient for the task of conceptualization and “theory building,” as it is grandiloquently called. Perhaps this is possible if the imagination can more than adumbrate the complexities that social science must deal with, at the levels of individual persons, small corporate units, or large aggregates and collectivities. My own experience is that observation and conceptualization, going hand in

I2

Eulau zyxwvutsrqpo

hand, yield better measures and measurement at all levels of analytic complexity than does purely theoretical modeling in the deductive mode, which, as soon as it comes in touch with the real world, is likely to relax some of its assumptions and then discovers too late that the data needed to specify the model correctly have not been collected. Unless we expose ourselves and our students to the real world, social science will be no less formalistic than those branches of philosophy and mathematics whose proper business, indeed, is the building of formal (“positive”) ideational models. It is not the kind of social science which, I believe, the consortium has been created to pursue. All of this may sound like old-fashioned stuff, but I could probably cloak it under the label of new empiricism. Nothing is so beguiling in social science as the word new. The “new history” described by Bogue in his essay is only one of “new beginnings” in that staid and oldest of social science disciplines. I have survived or am surviving the “new functionalism,” the “new institutionalism,’’ the “new political economy,” the “new Marxism,” or, more recently, the “new corporatism.” What the word new really tells us, however, is that the new phenomena attacked by the new approach are not so new, after all, and that the new approach is a revival of some “old” ways of seeing and dealing with things. As my spiral model of social science predicts, there is a good deal of revivalism in social science, as there is in religion, and this is hardly something to cheer about, even though the new apostles are unwilling to acknowledge it, out of either ignorance or sheer perversity. What new signals is one’s being at the frontier or cutting edge, at least for a while; but the euphoria lasts only until it ossifies and needs to be transcended by the rediscovery of some other old remedy for our unsolved difficulties or intransigent dilemmas. The same kind of self-deception accompanies the word interdisciplinary. Who still dares to write a research proposal that does not include this magic word? But what does it mean to be interdisciplinary? The answer strikes me as self-evident: we have erected disciplinary barriers that in only limited ways correspond to whatever reality may be “out there.” It is equally easy to understand why, in the face of the social complexities brought by modernity, social science would embark on the division of labor that gave us anthropology, economics, political science, psychology, and sociology. But the division of labor, in social science as in society at large, creates new problems of its own and, as a result, itself becomes a topic of inquiry. Paradoxically, there seems to be no other way to come to grips with complexity but to break it down and specialize in various parts and parts of parts. And, also paradoxically, in dividing the labor, the complexity that

Crossroads of Social Science

13

social science seeks to cope with also disappears from the intellectual horizon, and the whole is lost in the analysis of the parts. As cracks and gaps appear and we cannot puzzle out what is going on in the parts, we become interdisciplinary and discover something that seems new but is really quite old: psychologists had to “discover” that there is something “social” about individual human beings, and sociologists had to discover that there is something “psychic” about collectivities - a “new” discipline, social psychology, emerges. Similarly, economists had to discover (actually, rediscover) that politics has something to do with markets, and political scientists had to discover that the “state” is not some self-standing structure but that the economy is closely linked to its existence - a “new” discipline (actually, not so new), political economy, emerges. So the division of labor in social science entails its evident polar opposite, a fusion of labor, but we rarely seem to be aware that the “polarity principle” is at work. We often seem to think that a problem is solved when two or more scholars from different disciplines come together and seek to tackle it in an “interdisciplinary manner.” More often than not, these interdisciplinary teams, so-called, are mariages de convenance that simply look at the problem from the perspective of their individual members’ home disciplines. Only rarely emerges from these efforts a genuinely “new” shared or integrated theoretical frame of reference that is not dominated by one or another exclusively disciplinary orientation. I could illustrate this at length but forgo the exercise. What I find objectionable is not this failure but the confusion of being interdisciplinary or of doing interdisciplinary work with the product of such efforts. If, as individual disciplinarians, we participate in an “interdisciplinary effort,” we should be clear about what our own discipline expects of us in such participation. I happen to believe that what makes a discipline a discipline is what it seeks to explain: the explicandum (or dependent variable). What is interdisciplinary, then, cannot entail the object of interdisciplinary analysis but can only refer to the subjects (or independent or intervening variables) we bring to bear on the explanation of the object. To be interdisciplinary requires that we school ourselves in the theories, methods, and substance of the disciplines on which we call for explanation. This requires much learning and hard work. In my own time, I have known only very few truly interdisciplinary social scientists, who, in themselves, represented simultaneously the division and the fusion of labor: the late Harold Lasswell, the late Talcott Parsons, the late Theodore Newcomb, and the late David Potter. I shall avoid mentioning the living. All of this is easier said than done. As I contemplate the many cross-

14

Eulau

roads of social science, being interdisciplinary in orientation seems to require more than the traffic can bear. If I compare what I had to study as a graduate student fifty years ago with what we should expect from the best of our graduate students today, it truly boggles the mind. As I reflect on what I have written so far (and there are many other themes not mentioned), and as I read what Blalock and Weisberg tell us about proper training in contemporary social science, I can only think of “overload.” Is it not too much to expect that a graduate student should “master” both the classical theories of social science and the most advanced statistical techniques as well as, possibly, some esoteric language that enables him or her to do field work in China or sub-Saharan Africa? Does not something have to be jettisoned? I wish I had an answer, but I have not. If there were an answer, perhaps the problem of continuity and discontinuity in social science might disappear. I doubt that this will happen precisely because we do not know what to jettison. As long as theories divide us, and I think they always will, the only recourse we have is to methods of scientific investigation that constitute the bonds uniting us. The essays in this volume confirm this belief, as do the efforts of the ICPSR to give us the data we need and to guide us in making good use of them as we cross the roads into the future. References Dahl, Robert A . (1961). The behavioral approach in political science: Epitaph for a monument to a successful protest. American Political Science Review 55 (December): 763-772. Easton, David. (1969). The new revolution in political science. American Political Science Review 63 (December):1051-1061. Eulau, Heinz. (1959). Political science. In Bert F. Hoselitz (ed.), A Reader’s Guide to the Social Sciences. Glencoe, IL: The Free Press, pp. 89-127. Geertz, Clifford. (1973). The Interpretation of Cultures. New York: Basic Books. Lynd, Robert S. (1939). Knowledge for What? The Place of Social Science in American Culture. Princeton: Princeton University Press. Wahlke, John C. (1979). Pre-behavioralism in political science. American Political Science Review 73 (March): 9-31.

Chapter 1

Toward Cumulative Knowledge: Theoretical and Methodological Issues Hubert M. Blalock, Jr.

I

n very broad terms, and in one sense, a major objective of many social scientists is the creation of a cumulative body of knowledge that provides an increasingly adequate theoretical understanding of general social processes and institutional structures. There is another sense in which knowledge inevitably accumulates, if one means by this the piling up of miscellaneous pieces of factual information about specific events that have occurred at particular times and places. Such an accumulation of facts is likely to result in an information overload; and there follows the necessity of discarding all but the most recent or otherwise most interesting facts, unless there is an accompanying theoretical or interpretive system capable of organizing and prioritizing such a body of factual materials. In my judgment, there are far too many areas of social science that are becoming rich in facts and poor in theories, so that the cumulative process - in the first sense that I have used the term - has been disappointingly slow and erratic. What are the necessary ingredients of such a cumulative process? I believe there are at least five, which I shall mention very briefly at the outset and then selectively elaborate on in later sections. First, there needs to be an understanding of the logic, the basic principles, and the limitations of the scientific method, as it pertains to the peculiar problems of the social sciences. There are many traps, dilemmas, or trade-offs, methodological artifacts, advantages and disadvantages of alternative research designs, implicit assumptions required by data analysis strategies, and untestable assumptions needed to make the intellectual leap between factual information and causal interpretations. I believe we have made substantial progress in under- zyxwvutsrq 15

16

Blalock

standing many of the basic issues involved in applying the scientific method to the social sciences, and we have been able to borrow very successfully from one another as well as from more advanced disciplines. But there remain most certainly some gaps in our methodological understanding, and many more in the training we provide to our students. Disciplinary blinders are also evident. Second, a variety of critical measurement and conceptualization problems arise in each discipline. In some instances, reasonably general principles can be and have been formulated that facilitate borrowing both across disciplines and across problem areas within a specific field. The literatures on scaling and on measurement errors in multiple-indicator models are cases in point. But measurement problems are much more peculiar to each discipline, and the comparability of measures across settings is difficult to assess and achieve. We therefore seem to have sidestepped or postponed consideration of a wide variety of measurement-conceptualization problems that, until they are resolved, will continue to place very real upper bounds on the achievement of a cumulative body of theoretical knowledge. Indeed, the lack of attention given to conceptualization and the generalizability of our basic concepts are a factor contributing to the tendency to concentrate heavily on the collection of dated factual information and to proliferate subspecialties with little or no common theoretical ties. Third, our theories need to be far more complex and explicitly formulated than they are at present. Theories in economics and psychology are, in general, more precise and abstract than those in sociology, cultural anthropology, and political science, but they are also more delimited in terms of the number and kinds of explanatory variables they encompass. Often, verbally stated theories are complex enough but are so unclearly specified that purported tests are extremely simplistic, sometimes to the point of involving a few bivariate relationships, perhaps with one or two control variables thrown in. There is still a strong tendency for rival schools to consider only very simplistic versions of theories that are posed as alternatives rather than as being complementary. As a result, theorists talk past one another, and students are encouraged to take sides in disputes, rather than to reformulate more inclusive explanatory theories. Also, since theories of measurement error are rudimentary (if they exist at all), it remains possible to reject negative findings on the grounds that a proper test of the theory has not been made. Indeed, theories are often not stated in such a way that they imply falsifiable predictions, so that there is an accumulation of large numbers of somewhat different theo-

Toward Cumulative Knowledge

17 zyxwvutsrqpo

ries that, even when ignored for a decade or more, can readily be revived with a very modest new twist or perhaps a slight change in terminology. Fourth, for data collection technology to improve steadily, there must be the necessary funding and social support from outside the social science community. Here, our progress has been very uneven. Survey research technology has improved considerably, as has the number of large-scale surveyresearch projects in selected substantive areas. But certain kinds of variables continue to be either poorly measured and difficult to conceptualize, or they remain beyond the reach of either survey or experimental research technology. In particular, the social sciences are weak on the measurement of many kinds of actual behaviors, systematic observation techniques, and an ability to study past and current contextual effects, and to collect and analyze longitudinal data in which measurement-error complications abound. Often, analyses fail to take past experiences and expectations of the future into account. The growing methodological literature on social networks has been very suggestive, but thus far, it has not resulted in the kinds of datacollection efforts needed to extend the number of substantive problems to which network analyses can be applied. Finally, we have not created the undergraduate and graduate training programs needed to bring the majority of practicing social scientists sufficiently close to the “cutting edges” of methodological advances. We continue to attract relatively unprepared students as undergraduate majors and do not offer enough sequences of courses involving prerequisites, so that students will encounter increasingly demanding intellectual challenges in their more advanced courses. This practice continues into many of our graduate programs, with the result that technical training remains weak and students are not encouraged to undertake serious theoretical or conceptual tasks. A least-common-denominator effect then carries over to our scholarly literature, resulting in overly simplistic data analyses and theoretical interpretations. In the sections that follow, I shall use these five topic areas in order to highlight a few issues that I believe have been relatively neglected in at least several, if not all, of our disciplines. In doing so, I am deliberately overemphasizing areas of neglect or weakness in comparison with the many areas in which I believe we have made genuine progress. Understanding Basic Methodological Principles

As suggested, I believe we have gained considerable knowledge concerning the general logic of the scientific method, as well as a number of

18

Blalock zyxwvutsrqpo

more specific principles useful in enabling us to select among alternative designs, measurement strategies, and data analysis procedures. In gaining this understanding, we have, of course, been able to borrow very substantially from many other disciplines outside the social sciences. Our ability to communicate to our colleagues many of the more technical arguments, however, has lagged considerably behind the so-called cutting edge of this methodological knowledge, and there are still many disciplinary blind spots and faulty practices that are often based as much on considerations of research expediency as on lack of understanding. For example, although it has become very obvious to me that in nonexperimental research our theories must be sufficiently complex to build in numerous complicating factors, present practice, in sociology at least, is to test much simpler theories involving no more than five to ten variables, with little or no attention given to auxiliary measurement theories or the explicit statement of untested assumptions. In other words, current practice does not match our methodological understanding. From a methodological perspective, one of the most glaring defects is the relative inattention given to the implications of omitted variables in terms of the possible biases they may produce and the misleading interpretations then given in our causal explanations of important social phenomena. All investigations must necessarily omit many kinds of variables from consideration, variables that then become implicit contributors to the disturbance terms in our equations. But such omissions are selective and depend on disciplinary biases or blind spots, as well as on the limitations of whatever mode of data collection has been used. To admit that omissions are inevitable in any given piece of research is one thing. But to claim or imply that their impact can be ignored on theoretical or methodological grounds, as well as for research expediency, is quite another. Without more complete theoretical models that contain the unmeasured as well as the measured variables, one will, of course, not be in a position to say anything very definite about the costs we are paying, or about the biases we are creating, whenever we are forced by expediency to ignore their impact. There has been an overwhelming tendency in sociology-and, I believe, in all of the other social sciences as well-to assert, in effect, that variables that have not been measured can safely be ignored in one’s causal explanations. Readers and critics, however, are quick to note the deficiencies. Where this then results in elaborations on the theory and efforts to collect additional data, a self-correcting cumulative process is set in motion. Where disciplinary blinders or very general data gaps exist, however, it may be a very long time before certain omissions are detected.

Toward Cumulative Knowledge

19

As an illustration, survey research tends to be rather weak in tapping early experience variables, previously held attitudes, and contextual influences. A few of these are indexed by so-called background variables that are easily recorded, such as the respondent’s age, sex, race, and religious preference; the father’s occupation; and the region or community of birth. Many of these background variables are treated as nominal scales and are introduced into the analysis atheoretically as control variables. If differences are found, the investigator is likely to infer “sex effects,” “race effects,” or “religious effects.” The experience variables that intervene and that may be the critical causal factors influencing subsequent attitudes or behaviors are simply neglected, as are temporal sequences or interactions with contextual variables at different points in time. Since several background factors may be linked to some of the very same experience variables, the theoretical picture becomes further clouded, with a resulting tendency for the analyst to report the data in a descriptive, atheoretical manner. Others, however, may then interpret identical findings by invoking very different kinds of unmeasured explanatory mechanisms. Some may read discrimination into race differences, whereas others interpret them in terms of minority subcultures or even biological causes. Without measures of the numerous omitted variables, one is virtually free to select whatever explanatory mechanism is in vogue at the moment or is most compatible with the interpreter’s intellectual or ideological biases. Certain implications of causal modeling for research designs have also not been adequately studied, much less having worked their way into the applied research literature. Sampling designs are often based on the assumption of fixed populations, ignoring distinctions between independent and dependent variables, to say nothing of making allowances for nonrecursive relationships. A respondent’s location in space is treated as a “given,” as, for example, when one uses a cluster or area sample design or when delineating the spatial or organizational boundaries of a target population. Yet one may later attempt to make inferences about interrelationships among variables that may have operated prior to decisions to migrate, to apply for work in a specific industry, or to be self-selected into a given social environment. In a recursive system of equations, it is not legitimate to introduce controls for variables assumed to be dependent on the variables being interrelated. Thus, if X , and X , are independently operating causes of Y and are thus uncorrelated with each other, a control for Y will produce an uninterpretable association between them. More generally, if one designates a causal ordering among variables by using subscripts, so that an Xicannot cause an X j with a smaller subscript, then in studying the relationships

20

Blalock

among the first k variables, one should not introduce, as controls, any X s with subscripts greater than k.’It may not be recognized, however, that either the variances or the covariances in such subsequent variables may be inadvertently manipulated in one’s research design - with essentially the same kinds of misleading “findings.” (Blalock, 1985). For example, suppose one deliberately oversamples extreme cases on a dependent variable, perhaps because such extremes are empirically rare or because convenient lists are available only on such dependent variables. One may wish to find out why persons develop a particular illness, become drug addicts or prostitutes, or do exceptionally well in school. Such persons do not appear in large enough numbers in a random sample, and so they may be oversampled or may be matched with others whose behavior is at the opposite extreme or perhaps with so-called normal control groups. A study is then made of the supposed causes of the differences between these two categories of persons. But in sorting out the causes, it may not be recognized that the particular variables that have been explicitly selected for investigation are likely to be confounded with many others that have not been recognized or that could not be measured in the study in question. This is the opposite of “controlling” for the dependent variable, and interrelationships among independent variables are also distorted in peculiar ways. Again, consider the simple case where there are two independent causes of Y , with both XI and X , having positive effects on Y . If one oversamples extremes on Y , one will also tend to oversample cases that are high on both independent variables (to get high Y values) or else low on both, thereby producing a positive association between them.] Now if, say, the investigator is focusing on XI as a cause of Y , being unaware of the operation of X,, the influence of the latter variable will be credited to the independent variable of interest, and the researcher will be led to exaggerate its impact. Clearly, many other variables may be at work, making it impossible for the investigator to locate and measure variables that have become confounded with XI. In effect, a faulty design will, in this instance, amplify the distortions produced by specification errors in the theory, in much the same way that aggregation biases become amplified when one uses macrolevel data to infer microlevel processes. (Hannan and Burstein, 1974). So far as I am aware, cluster sample designs have not been examined from a similar perspective. We take location in space (or in work settings) not as a dependent variable but, by implication, as a cause of the other variables under investigation. Yet we often ask questions about how the respondents got that way prior to their migration, or how their religious orientations may have been linked to schooling, parental influences, or per-

Toward Cumulative Knowledge

21

haps the job-selection process. In simply taking current location as given, it may not be recognized that we are, in effect, “manipulating” variables that belong somewhere toward the middle of a recursive model. If, say, a locational decision or workplace decision appears as X , in a twelve-variable recursive system, and if our design strategy has involved a manipulation of even the variance in A’,,to say nothing of covariances with prior variables, then our inferences concerning the prior six variables may be incorrect. And, more practically, if we have omitted any of these variables, even a wellformulated theory will not be very useful in enabling us to anticipate possible biases unless we also know how they are related to the decision to locate. The important point is that these spatial or locational decisions need to be incorporated explicitly into the theory in order to assess the biases likely to result. We need to be especially cautious in using either cluster or stratified sampling in instances where spatial location or the stratifying criteria may be dependent variables. A nuts-and-bolts discussion of sampling, as applied to supposedly fixed populations, is far too simplistic. A similar concern will be discussed in the next section in connection with self-selection into social contexts. Progress in Conceptualization and Measurement

The measurement and conceptualization of our most important variables have also proceeded at an uneven pace. Whereas psychologists and educational psychologists have generally been more concerned about problems of reliability and validity than have other social scientists, they have not displayed a corresponding concern about issues of measurement comparability or generalizability across diverse settings. Within sociology and cultural anthropology - and, I believe, political science as well - there has been a greater interest in cross-cultural generalizability, but measurement procedures have tended to be more ad hoc and dictated by data availability than by careful conceptualization. Economists have made use of so-called proxy variables, with little or no concern about the auxiliary measurement theories needed to assess the measurement errors involved. And since we have been unable to borrow extensively from the biological sciences and statistics in connection with measurement issues, progress has, in general, been much slower than in the case of our use of improved data analysis procedures. Particularly in applied survey research, empirically inclined social scientists have tended to focus on what very quickly become dated pieces of information about particular political issues or about attitudes toward specific kinds of persons or policies, or about choices among health plans,

22

Blalock

occupations, religious denominations, or other nominally scaled options. As a result, findings are reported and propositions stated at a rather low level of abstraction that makes comparisons across social systems or time periods rather difficult. This tendency to stick very close to one’s data may be supported by professional norms that warn against overgeneralizing one’s findings, as well as by pressures to report findings to the study’s sponsors and to grind out a new proposal for funding. There may possibly be a tendency to believe that sophisticated data analyses can, in some sense, compensate for this inattention to generalizability and to the need for more abstractly formulated theories. In any event, we have, in many disciplines, a sharp split between the empiricists, who do quantitative research, and the theorists, who develop theories that are inattentive either to this research or to the need for careful conceptualization of the important variables or to concepts in these theories. There remains the very considerable gap between theory and empirical research that was evident in my own student days, when Merton’s classic papers (1949) on the interplay of theory and research were read by nearly every graduate student in sociology. One of the problems we face, organizationally, is that careful theoretical conceptualization requires considerable effort, extensive exploratory work at the pretest stage of research, and continued refinements and reassessments. Even studies that attempt to assess reliability are relatively rare in sociology, and the concept of validity is as messy and as carelessly used as ever. Although the technology of multiple-indicator analysis is now well developed and is becoming increasingly feasible in terms of computer costs, the kinds of norms regarding measurement assessment that exist in, say, educational psychology have not yet seeped into sociology and political science. Of course, the more indicators one has for a given set of variables, the less the amount of additional information that can be collected in a fixedlength interview. There is thus a very real trade-off between the need to measure variables carefully and to allow for consistency checks, and the number of such variables one can include in any given study. We seem to have been opting for large numbers of poorly measured variables in most of our social surveys. Indeed, survey research seems to me to be strong on sampling methods and data-collection technology, but weak on conceptualization and measurement. In macrolevel research, our problems stem from a different source: our almost total dependence on others to collect and compile our data. I do not believe there is yet a sufficient awareness of the critical compli-

Toward Cumulative Knowledge

23

cations that arise in data analysis whenever measurement errors are substantial in comparison with real variation in the “true” values of whatever variables are being studied, There is, for example, considerable lip-service agreement that in many circumstances, longitudinal research is essential and ideally superior to cross-sectional studies. But there is not the corresponding recognition that, since real changes are often rather small as compared with differences across cases, reducing measurement errors becomes all the more crucial in panel or time-series designs. When combined with misspecified lag periods, even random measurement errors can totally invalidate the inferences that one might naively make by examining what appear to be “change” scores in panel designs. And this says nothing about nonrandom measurement errors produced by repeated measurements on human subjects, whose memories imply that notions such as independent replications will be almost completely misleading in same-subject designs. My own observation is that those who extol the advantages of longitudinal studies in connection with causal inferences also become conveniently ignorant of the measurement-error complications to which such designs are peculiarly sensitive. If we are serious about longitudinal research in settings where there are relatively small changes in our most important variables, then we must pay far more attention to these measurement-error complications. This is in addition to any problems of measurement comparability over time, problems that may indeed be less serious than those that plague analysts who wish to use cross-sectional, comparative data. One of the main points to stress in this connection is that conceptualization and measurement problems are best handled during or before data collection, rather than being handled later, during the analysis stage or, worse still, being ignored altogether. Far too many write-ups of empirical research begin with discussions of measurement imperfections but then proceed to data analyses that presuppose that all variables have been perfectly measured. We are just kidding ourselves, of course, if we then take these data analyses at face value. With the growing number of courses, texts, and journal articles on categorical data analyses, there may be a corresponding neglect of ordinal and ordered-metric data, as well as metric and nonmetric scaling procedures oriented to improving our aspirations with respect to using data at higher levels of measurement. Regrettably, also, there has been a neglect of measurement-error problems created by decisions to collapse into a small number of categories in order to simplify categorical data-analysis procedures. If, for instance, one collapses occupational categories so as to make interpretations manageable, the rationale underlying such collapsing decisions may go

24

Blalock

unexamined. Presumably, similarity judgments must be made on the basis of assumptions about the unidimensionality of the criteria being invoked, but readers may never be told what these are. If the number of unordered categories remains small, named categories (e.g., countries or religious denominations) may be retained, whereas such a procedure would be manifestly unwieldy with fifty or more categories. In the former case, conceptualization problems may simply be ignored, making generalizability next to impossible. What concerns me most in this connection is that we appear to have lowered our scientific aspirations with respect to striving to come closer and closer to interval and ratio-level measurement. It is as though we believe we may safely postpone difficult conceptualization and measurement problems merely because sophisticated analysis tools have been invented to handle categorical data. I have no quarrel with the statistical tool per se; my concern is that it may distract us from tackling the job of improving measurement quality. If we simultaneously use categorical data analyses and ignore measurementerror complications, we may succeed in misleading an entire generation of quantitative social scientists. And we may direct them even more than at present toward straightforward data reporting, rather than toward careful theoretical interpretations of their findings. Finally, I want to make a few observations about contextual effects and cross-level analyses and the measurement-conceptualization problems that are peculiar to them. Once more, we seem to have a reasonable understanding of the major methodological problems involved, thanks to the technical literature on aggregation that has been developed within the econometrics tradition, as well as to some thoughtful discussions of contextual effects derived from selected fields of political science and sociology.2 We have difficulty in applying this literature, however, because of gaps in our empirical data and because of inadequate conceptualizations as to the causal processes involved. For example, we recognize that individuals will be influenced by a variety of contextual variables appropriate to different contextual units: friendship cliques, neighborhoods or voluntary associations, classrooms or schools, or entire villages or even regional characteristics. Critical in this connection are the definitions of boundaries for these contexts, along with the recognition that such boundaries will differ for each individual and will overlap as well as be nested (Blalock, 1984). Unfortunately, individuals may be influenced by normative pressures within several different contexts, as well as by inconsistencies or incompatibilities among them. They may have self-selected themselves into certain contexts but may have been born into

Toward Cumulative Knowledge

25

others, and the temporal processes involved may be diverse. Simply putting in a group mean on the independent or dependent variable of concern may provide one with a crude estimate of a “contextual effect,” but interpretations of such “effects” will depend on either explicit or implicit assumptions about contextual boundaries, time periods, and self-selection. Unfortunately, the kinds of supplementary data needed to choose among rival alternative contextual explanations are rarely collected. Nor can we expect this to occur on a regular basis until more careful conceptualization has taken place. So debates will continue as to whether there are true peer-group “effects ,” or whether the individual has merely selected a group of peers similar to himself or herself. Normative influences will be difficult to disentangle from so-called frog-pond effects (Firebaugh, 1980), and so forth. And if a person is embedded within a subgroup of some larger group, we will not know whether comparisons are being made with other subgroup members, with members of the entire group, or perhaps between subgroup and total group means (Blalock and Wilken, 1979). If some groupings are more salient to one actor than to another, we may not have collected the data necessary to decide how to model the contextual-effects process for these different individuals. I believe that many of the problems we face here are basically conceptual in nature, with measurement decisions being dependent on a much better understanding of the diversity of possibilities that one may expect to encounter. There are similar conceptual problems in connection with aggregation and disaggregation, as indicated by a series of relatively recent discussions in the sociological and political science 1iteratu1-eq3 At issue here is the problem of “consistency” across levels, as formulated by econometricians, and of aggregation biases and how they may be affected by the role played by the criterion used in aggregation. Instead of aggregating according to a carefully conceived theory as to what constitute homogeneous units that may safely be aggregated, many social scientists are forced to use data that have been aggregated by others, usually on grounds of convenience or expediency. In particular, we often use data aggregated by geographic proximity (states, counties, census tracts, voting precincts, or city blocks). Or individuals may have been aggregated by place of work or school attended. What may seem to be a straightforward “empirical” decision may very well mask a whole series of implicit theoretical assumptions regarding how the aggregating criterion enters into one’s theoretical model. For example, one may be aggregating by a criterion that is partly independent and also partly dependent if individuals have located themselves in space or jobs through selective migration.

26

Blalock zyxwvutsrqpo

This may not appear to be a “measurement” problem until one asks whether one’s empirical results are expected to depend on the decision as to what aggregated unit to use, as, for instance, whether a county or a state, or a classroom, a school, or an entire school d i s t r i ~ tThen . ~ it becomes important to specify one’s assumptions regarding self-selection and the homogeneity of group “members” in terms of the similarity of the structural parameters in their equations. Persons aggregated together may indeed be homogeneous with respect to their properties (say, of income or education), but not necessarily with respect to how they will behave if a change is produced. And if the criterion for aggregation is dependent on some of the variables in a recursive system, estimates of causal parameters involving equations for prior variables may be severely biased. As Hannan and Burstein (1974) have noted, any specification errors appearing in the microtheory are likely to be amplified in the macrotheory.

The Complexity of Our Theories In most empirical tests, we find that whenever we use four or five explanatory variables in our models, the explained variance is disappointingly small, even reaching the vanishing point. Of course, this may be attributed, in part, to measurement errors or restricted ranges of variation in our independent variables. But the problem lies elsewhere, namely, with the possibility that adequate explanations of most social phenomena will require twenty or more independent variables and will involve more than simple linear, additive relationships. Furthermore, “dependent” variables are likely to feed back to affect some of our explanatory variables, particularly if one is studying repeated behaviors, such as test or job performances, delinquent acts, or social interaction patterns. Especially in instances where we focus on relatively powerless actors, such as young children or minorities, there is a tendency to treat such behaviors as strictly dependent, rather than as part of a larger set of endogenous variables. But children do affect the behaviors of parents and teachers, as we must reluctantly admit! Sometimes highly complex implicit theories are contained in our lengthy verbal discussions of social phenomena, such as peasant revolts, family tensions, or bureaucratic structures or processes. When empirical studies are developed to “test” these theories, however, a few very simplistic bivariate propositions are stated to “represent” these theories. If they are then rejected or found to lack explanatory power, critics of the research may cite obvious inadequacies in the empirical test, while hanging onto their favorite theories. In contrast, those who wish to reject the theories may, in

Toward Cumulative Knowledge

27

effect, conduct highly inadequate and overly simplistic empirical investigations by ignoring the subtleties of the theory. Either way, one is likely to end up with weak explanatory power in the statistical sense of that term. Nearly all of us believe that social reality is complex, yet we subscribe to the notion, derived from a philosophy of science literature based heavily on the physical sciences, that theories should be parsimonious. We are all familiar with the early debates over whether or not the earth is the center of the universe, and with the necessity of building unwieldy complexities upon other unwieldy complexities in order to salvage such an ethnocentric theory in the light of increasingly embarrassing empirical evidence. From this, it seemed to follow that simple theories are to be preferred over more complex ones. But what if these simpler theories don’t work? One strategy is to search for alternative simple theories, under the assumption that eventually a highly satisfactory one will be found. Thus we tend to formulate theories as alternatives to one another, as though one must be “true” and the others “false.” One is then tempted to set up straw men (seldom women) so as to demolish one’s “opponent’s” simple theory by simplifying it still further, or by conducting a weak test of its predictions. Rival “schools” thus flourish and are difficult to dissolve, partly because of the vested interests that have been created in the process. Simplistic labels, such as world system theorists and modernization theorists, or conflict theorists and functionalists, are then attached to the rival schools, and we are off and running to another prolonged debate. If such rival schools happen to coincide with disciplinary boundaries, there may be a nondebate and an increasing rigidity of disciplinary biases and blind spots. An overconcern about psychological reductionism among sociologists is a case in point. As some social scientists have suggested to me in personal conversations, if not in print, it may be that a focus on a dependent variable of applied interest is to produce a more eclectic orientation, since it may lead to interdisciplinary efforts to throw into the hopper almost any explanatory variable that “works.” In contrast, where the focus is on a small set of explanatory variables - say, status or power - there may be a greater vested interest in these few variables, as “opposed” to rival explanations offered by members of other schools. Whether or not this hypothesis has any validity, the trick is to turn highly eclectic empirical investigations into more systematic theories that are both explicitly formulated and clearly explicated and that are also capable of being expanded as additional complexities are needed. There are two commonly employed approaches endemic in sociology, at

28

Blalock

least. One is to formulate theories with little or no thought of their testability, leaving basic concepts undefined or explicated in such a way that linkages with empirical indicators will be highly problematic. The other is to specify a relatively simple theory - say, by means of a causal diagram - but to confine the theory to variables that can be rather easily measured or that happen to be available to the investigator in terms of data already collected. Unmeasured variables thus have no place in the latter types of theories but are predominant in the former. As previously emphasized, the obvious intermediate but more difficult strategy is to formulate more complex theories carefully enough to distinguish between those important variables that are likely to remain unmeasured, for the present, and those that can be treated either as directly measured or as measured with specifiable sources of measurement error. With the full and more complex model explicitly specified, one can systematically examine its implications in those instances in which certain variables need to be treated as unmeasured in any given piece of research. This, then, makes it possible to make the necessary simplifications, based on theoretical and methodological principles, rather than on mere convenience. Perhaps the omitted variables are assumed to intervene between several others in a simple recursive model. But perhaps there are feedbacks involving these unmeasured variables, so that it would be much more risky to omit them without at least rudimentary efforts to measure them (Blalock, 1982, Chap. 5 ) . Our linear additive equations with constant coefficients are also overly simplistic, not only in terms of the linearity restrictions but also in the use of constants as multipliers of each of the independent variables. Whenever we treat regression coefficients as constants, we impose homogeneity assumptions to the effect that all actors respond in the same way to changes in independent variables. Such assumptions may be somewhat plausible in delimited settings, but they are much less so when one attempts to generalize the theory to more diverse populations or settings. Thus the more general we wish our theories to be, the less reasonable it is to irnpose such homogeneity assumptions in the form of constant coefficients. The logical implication, then, is that the “constants” in our equations should be treated as variables, to be explained in their own right. But since these “constants” appear as this means that more complex multiplicative terms will multipliers of our A’,, appear in our theories, providing a rationale for the systematic search for statistical interactions. Once more, an obsession with parsimony may come into conflict with other scientific objectives - in this instance, the desirabil-

Toward Cumulative Knowledge

29

ity of increasing the generalizability or scope of our theoretical propositions by examining homogeneity assumptions that often remain implicit or hidden in our data analyses. Also, as we attempt to generalize our theories we must leave open the possibility that the auxiliary measurement theories needed to link our conceptual apparatus to measured indicators may also have to be altered or made more complex. For example, an inequality measure that serves as an indicator of discrimination in one setting may require several more controls in a second setting. Thus both our substantive and our measurement theories may need to become increasingly complex if we wish to move in the direction of increased generalizability. It is small wonder that many social scientists prefer to stick closely to time- and place-specific research, leaving it to “others” to add up the diversity of findings that result from such studies. Such “others,” unfortunately, must become increasingly venturesome. Finally, our more complete theories should also contain dynamic features, including attempts to specify the variables that affect rates of change or lag intervals. In the distributed lag models of economists, for example, one may take present values of Y - say, a behavior- as a function of a series of X values, but with coefficients that may decay exponentially as one goes back in time. This then allows for memory factors that may not all be identical for diverse actors, some of whom may have shorter memories than others. In retrospective research, of course, we become increasingly skeptical of the measurement accuracy of all but very recent events, meaning that our measurement-error theories concerning long-term memories might have to become quite complex and justifiably suspect. But if social processes depend upon memories of past events, perhaps it is the present perceptions of these events that are the important variables. If not, and if we also distrust retrospective reports, there may be no substitute for rather lengthy longitudinal research. If we simply omit the variables that are most difficult to measure, however, we may get our research reports out, but this does not imply that we shall thereby produce a cumulative body of research findings and generalizable theories that are not at the same time misleading. Organization to Collect Better Data

A call for more complex theories involving large numbers of exogenous

and endogenous variables, multiple indicators to assess measurement errors, and dynamic models allowing for lags and feedbacks is obviously also a call for more adequate data. But are we well enough organized and intellectually

30

Blalock zyxwvutsrqpo

coordinated to make large-scale data-collection efforts more than a huge wasted effort? I believe a number of things must occur before and during the process of data collection if substantial headway is to be made. First, we all recognize the existence of major gaps in our knowledge, as well as the strengths and weaknesses of our most important research designs and data-collection capabilities. What needs to be accomplished, however, is sustained progress in finding better and more efficient ways to fill these gaps and to hitch together information collected in different ways. Here, we seem to be making very slow progress. Nor are we organized for the purpose. Consider survey research, which obviously is most efficiently accomplished by a small number of large data-collection agencies or consortia. It is well known that survey research is weak in a number of respects, being far stronger on sampling designs than on conceptualization and measurement, tending to treat respondents as though they were atomized individuals, and failing to tap actual as contrasted with reported behaviors. Survey research in the form of panel designs is capable of providing certain kinds of overtime data, provided that frequent day-to-day information is not required over a prolonged period. But panel studies are not only extremely expensive but also subject to severe constraints in terms of the attenuation of cooperative respondents and the costs involved in tracking highly mobile individuals or families. All of this creates substantial gaps in the information that has been obtained by this method. For example, the very large data-collection efforts on individual students and school characteristics that resulted in the Coleman Report (Coleman et al., 1966) and a series of follow-up studies have been severely criticized because they did not adequately tap processes occurring at the classroom level, among peers or teachers, or within family settings. The rather superficial school characteristics that were obtained from school principals turned out to have very weak explanatory power in terms of the measured performance levels of students of various ages. Many kinds of seemingly critical interaction patterns within schools, classrooms, and cliques could not be measured or combined in the same studies with the more “objective” measures of school facilities and resources. Ex post facto critiques of this sort are always possible and may, if taken seriously, result in the collection of supplementary information. What is most needed, however, are more carefully coordinated efforts before investigators have settled on a single data-collection instrument that is anticipated to be highly inadequate for certain kinds of needed data. Hurry-up studies mandated by Congress are sometimes necessary and may serve in a negative sort of way to redirect our energies to the collection of very different kinds

Toward Cumulative Knowledge

31

of information. But how much more efficient and ultimately less costly it would be, in terms of both money and contentious scholarly disputes, if a more carefully coordinated data-collection effort could follow a series of more diverse but individually less ambitious pilot projects. Given that survey research currently represents our most efficient method of large-scale data collection, it seems wise to pose the question of how data collected by other means can be grafted onto survey data, so that a greater diversity of variables can be attached to each respondent, and so that responses can be aggregated to provide better measures for macrounits. We could then supplement “global” macromeasures with those based on average member scores, with measures of homogeneity, and even with those that enable one to attach differential weights according to the members’ relative power, their network positions, or their external contacts. It also seems advisable to find better ways to combine reasonably inexpensive systematic observations of behaviors or participation rates with survey data. Unfortunately, our data-collection capabilities and methodological knowledge have advanced unevenly, making it all the more difficult to combine information in such an ideal fashion. The technology connected with, say, a systematic observation of behaviors does not seem to have advanced much beyond the classic work of Bales (1950) and his associates some thirty to forty years ago. Network analysis approaches are becoming increasingly sophisticated, and a few network analysts, such as Burt (1983), have grappled with the difficulties involved in applying them to more open network situations of the sort that would be most amronriate to social survevs of the general population. Certainly, in elite studies, one can well imagine a fruitful wedding of survey and network approaches, with units of analysis being either individual respondents and their positions in networks, or pairs, triads, or larger sets of individuals, and with scores attached to these sets being derived, in part, from surveys. There is also no inherent reason that studies of “school effects” cannot be supplemented by data collected within individual classrooms, including network studies shedding light on peergroup processes. What is essential, if complex theories are to be tested, is that such pieces of information be joined in the same study. If one investigator studies variables U K and W and another X , r; and Z , huge intellectual leaps will be required to “add up” or assess such a series of disjointed findings. Metaanalyses of the sort discussed by Converse will be difficult to conduct (see Chapter 3). All of this is much easier said than done, of course. I do not believe it reasonable to expect a move in this direction from our major survey research

32

Blalock

institutes, partly because of their vested interests in that particular mode of data collection, and also because their present staffs are not equipped for the task. The necessary impetus may have to come from clusters of social scientists whose interests focus on a single institution, such as the family, or on important social processes, such as power and dominance, exchange and equity, or mechanisms of allocation and discrimination. It is perhaps more likely that coordinated efforts will occur as a result of federal funding for the study of some major social-problem area, such as crime and delinquency, child and spouse abuse, or employment and discrimination. The highly ambitious National Election Studies effort provides us with an important “case study” of such a cooperative endeavor. Regardless of the nature of the initial impetus, what I believe to be most critical is that the effort be sustainable over a sufficient period of time so that careful planning, exploratory data collection, and conceptualization and measurement efforts precede, by at least several years, any major datacollection efforts. And, of course, the implication is that such work should be conducted by our very best social scientists, attracted to the enterprise because of its potential scholarly value, rather than as a quick source of funding or individual publication. At some point, it will also be necessary to create ongoing and sustainable research efforts, including a relatively permanent staff of technical specialists who can be rewarded professionally in spite of the likelihood that they themselves will not be in a position to produce their own publications from the ongoing research. Here we must face up to the free-rider problem, as well as territoriality on the part of those who are responsible for the dayto-day details and who will be in charge of collecting the actual data. If data are to be made available to a large body of scholarly users, why should these users waste their own valuable time in the data-collection process? But if those who are primarily responsible for planning and executing the study have first rights to the data, then their own peculiar preferences and datacollection biases may prevail. Or certain universities and their faculties may gain at the expense of others. Perhaps we have by now accumulated enough practical wisdom, based on actual experience, to enable us to anticipate such problems and to devise effective ways to overcome them and to get the ball rolling in a growing number of general problem areas of interest to intellectually diverse clusters of social scientists. I leave it to others to tell us how to accomplish this sort of miracle, given our current very limited organizational structures and intellectual networks. All I can say is that I believe that cumulative progress

Toward Cumulative Knowledge

33

will depend very heavily on our ability to get our individualistic acts together in a sustainable manner. Some Implications for Graduate Training

Over the thirty-five-year period that I have been in sociology, at least, I have perceived very few changes in the quality of graduate training, apart from the introduction of specialized methodology courses beyond the level of two semesters of statistics. I have been disappointed that, in spite of Sputnik and considerable talk about the need for more technical training in the social sciences, entering graduate cohorts are no better prepared for technical courses than they were when I entered graduate school in 1950, or when I began teaching in 1954. There are always a few who have had mathematical or technical science backgrounds, but this was also true when I entered the field. We can probably count on nearly everyone’s having had an undergraduate statistics course, whereas this may have been somewhat more rare in the 1950s. But most such courses remain at a very low level, especially when they are required of all majors. Attitudes toward taking technical courses are somewhat more positive among current graduate students than they were in my day, and it is recognized that in order to get journal articles published and to keep abreast of certain, but not all, bodies of literature, one must be able to understand the most current fads. These may have shifted from Guttman scaling to loglinear analysis, or from factor analysis to path analysis, but again, I do not perceive the degree of change that, at one time, I hoped would have come about by now. Perhaps my experiences are unique, and my 1950s peers at North Carolina very atypical, but in any event, I have not witnessed anywhere near the improvements in training programs that I believe are necessary to keep pace with “cutting-edge” developments in the field. If these observations and experiences are not too far off base or unique to sociology, one must ask why the situation has not changed over the course of a third of a century. In large part, I believe this is because our recruitment policies, the available pool of entering graduate students, and our undergraduate programs have remained basically static. Sociology still attracts students with lower aptitude scores than nearly all other disciplines, with several of the other social sciences not doing much better in this respect. Our undergraduate curricula are still “horizontal” in the sense of containing very few sequences of courses that build upon one another and that have demanding prerequisites. A student who completes introductory sociology is eligible to take a wide variety of subsequent courses, in any convenient

34

Blalock

order. Our majors, who have presumably taken more sociology courses than nonmajors, are seldom at a distinct advantage in “advanced” courses, and they have often put off taking their methods, statistics, and theory courses until their senior year. What is perhaps even more true today than thirty to forty years ago, we then recruit our graduate students primarily from this pool of poorly trained undergraduates, finding it necessary to devote the first two years of graduate training to what is basically remedial work. A student who makes it through two semesters of “graduate” statistics is then too exhausted, or demoralized, to pursue the subject further, nor is he or she required or highly motivated to do so. Perhaps I am exaggerating, but when one compares the level of demandingness in, say, sociology and statistics departments, the differences are striking. Although I have been speaking primarily from the perspective of the technical, quantitative side of sociology, I believe that much the same can be said of theory training in our discipline. I see no evidence that our current students are any better informed about major theorists than were students in my day, though the theorists studied may have changed. Most important, in my view, I do not see them constructing theories or using them in a sophisticated way in their own empirical research. Conceptualization efforts may even have deteriorated since the early 1950s. At least when I ask students to discuss what they perceive to be the most sophisticated conceptualization efforts in their own fields of interest, a surprising number refer to the same literature I was reading back in the 1950s! I do not see any solution to this problem unless there is a very serious effort on the part of our graduate programs to raise their standards substantially and to insist on a much stronger undergraduate background than we now provide our honors undergraduates. Yet what department can afford to be so highly selective under present circumstances? If we were not improving standards in the early 1970s, when the supply of incoming students was greater than the real demand for Ph.D.’s, where are the present incentives to change? What we have been doing in sociology, at least, is to supplement the American applicant pool with increasing proportions of foreign students, particularly those coming from Asian countries and having high quantitative GRE scores. We also increasingly provide applied sociology programs, which, though perhaps emphasizing techniques of data analysis, hardly ever stress a comparable program in theory, theory construction, or conceptualization and measurement. Thus our basic standards have not shifted upward, though the nature of our graduate student composition may have changed very slightly. There

Toward Cumulative Knowledge

35

simply has not been an incentive system in place that would have encouraged us to do otherwise. The quantity of Ph.D.’s produced has remained more important than their quality. Again, this may be an exaggeration, but not much of one when we look carefully at the GRE scores of recruits to our field or when we attempt, at the hiring stage, to appraise the newly minted Ph.D.’s who come to us with superlative letters of recommendation and one or two publications but little else. We need to introduce, very intentionally, into both our undergraduate and our graduate training programs a series of tough intellectual challenges of the sort that, say, a mathematics or physics major receives. The exact nature of these challenges may be less important than their distribution throughout the student’s academic career. In the junior year, perhaps, students may be required to take their methods, statistics, and theory courses, so that they will then be prepared during their senior year to write a serious theoretical paper or to undertake an individual research project, or, ideally, both. Better undergraduates, including all honors majors, should be encouraged, if not required, to take what we normally consider graduate seminars or somewhat more advanced methods and statistics courses. At various points during the graduate student’s intellectual development, a series of rather specific intellectual challenges should be presented. These should include not merely a set of exam or course hurdles, but well designed tasks that, when accomplished, assure us that the student is capable of careful theoretical and methodological work. This would include efforts to apply the thinking of “classic” theorists to contemporary research topics, to grapple with problems of generalizability and the comparability of measurements across a variety of settings, and to construct theories of considerably greater complexity than those required by their ow‘n delimited research. These more complex theories should prove useful in enabling the student to formulate aprogram of research, rather than undertaking a series of disjointed efforts. And finally, students should receive more advanced training in those specific technical areas most likely to be relevant to their own subdisciplines, as well as extensive work in at least one other social science discipline. In my view, unless students are put through the “intellectual wringer,” so to speak, and learn to see the value of careful and systematic thinking, we will merely continue to perpetuate the existing state of affairs - as we have done over at least my own academic career. I am not very optimistic, however, that as a collectivity we will indeed undertake this kind of serious intellectual reappraisal of our existing curricula and teaching practices. Perhaps as individual scholars, we have a better chance of making a more

36

Blalock

substantial contribution to the needed effort. The goal of working systematically and self-consciously toward a cumulative body of knowledge should guide our instructional programs as well as our research. Notes 1 . With a control for Y , the relationship between X I and X , would be negative. 2. For discussions of contextual effects see Davis, Spaeth, and Huson (1961), Erbring and Young (1979), Farkas (1974), Firebaugh (1980), Hauser (1974), Przeworski (1974), and Sprague (1976). 3. For discussions of aggregation and disaggregation in addition to those of Hannan (1971) and Hannan and Burstein (1974), see Firebaugh (1978), Goodman (1959), Hammond (1973), Irwin and Lichtman (1976), Langbein and Lichtman (1978), and Robinson (1950). 4. See the debate concerning school effects between Bidwell and Kasarda (1975, 1976), Alexander and Griffin (1976), and Hannan, Freeman, and Meyer (1976).

References Alexander, Karl L., and Griffin, Larry J. (1976). School district effects on academic achievement: A reconsideration. American Sociological Review 41: 144-152. Bales, Robert F. (1950). Interaction Process Analysis: A Method for the Study of Small Groups. Reading, MA: Addison-Wesley. Bidwell, Charles E., and Kasarda, John D. (1975). School district organization and student achievement. American Sociological Review 40:55-70. Bidwell, Charles E., and Kasarda, John D. (1976). Reply to Hannan, Freeman and Meyer, and Alexander and Griffin. American Sociological Review 41:152-160. Blalock, Hubert M. (1982). Conceptualization and Measurement in the Social Sciences. Beverly Hills, CA: Sage. Blalock, Hubert M. (1984). Contextual effects models: Theoretical and methodological issues. In Ralph H . Turner and James F. Short (eds.), Annual Review of Sociology, Vol. 10, pp. 353-372. Palo Alto, CA: Annual Reviews. Blalock, Hubert M. (1985). Inadvertent manipulations of dependent variables in research designs. In Hubert M. Blalock (ed.), CausalModels in Panel and Experimental Designs, Chap. 5. Hawthorne, NY Aldine. Blalock, Hubert M., and Wilken, Paul H . (1979). Intergroup Processes: A MicroMacro Perspective. New York: Free Press. Burt, Ronald S. (1983). Applied Network Analysis. Beverly Hills, CA: Sage. Coleman, James S., et al. (1966). Equality of Educational Opportunity. Washington: U.S. Government Printing Office. Davis, James S., Spaeth, Joel L., and Huson, Carolyn. (1961). A technique for analyzing the effects of group composition. American Sociological Review 26:2 15-225. Erbring, Lutz, and Young, Alice. (1979). Individuals and social structure: Contextual effects as endogenous feedback. Sociological Methods and Research 7:396-430. Farkas, George. (1974). Specification, residuals, and contextual effects. Sociological Methods and Research 2:333-363. Firebaugh, Glenn. (1978). A rule for inferring individual-level relationships from aggregate data. American Sociological Review 43: 557-572.

Toward Cumulative Knowledge

37

Firebaugh, Glenn, (1980). Groups as contexts and frog ponds. New Directions for Methodology of Social and Behavioral Science 6:43-52. Goodman, Leo A. (1959). Some alternatives to ecological correlation. American Journal of Sociology 64:610-625. Hammond, John L. (1973). n o sources of error in ecological correlations. American Sociological Review 38:764-777. Hannan, Michael T. (1971). Aggregation and Disaggregation in Sociology. Lexington, MA: Heath-Lexington. Hannan, Michael T., and Burstein, Leigh. (1974). Estimation from grouped observations. American Sociological Review 3 9:374-392. Hannan, Michael T., Freeman, John H., and Meyer, John W. (1976). Specification of models for organizational effectiveness: A comment on Bidwell and Kasarda. American Sociological Review 41:136-143. Hauser, Robert M. (1974). Contextual analysis revisited. Sociological Methods and Research 2:365-375. Irwin, Laura, and Lichtman, Allan J. (1976). Across the great divide: Inferring individual level behavior from aggregate data. Political Methodology 3:411-439. Langbein, Laura I., and Lichtman, Allan J. (1978). Ecological Inference. Beverly Hills, CA: Sage. Merton, Robert K. (1949). Social Theory and Social Structure. Glencoe, IL: Free Press. Przeworski, Adam. (1974). Contextual models of political behavior. Political Methodology 1:27-61. Robinson, William S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15:351-357. Sprague, John. (1976). Estimating a Boudon type contextual model: Some practical and theoretical problems of measurement. Political Methodology 3:333-353.

Comment

Progress in Research Methodology: Theory, Technology, Training

Herbert F. Weisberg zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFED

Would you realize what Revolution is, call it Progress; and would you realize what Progress is, call it Tomorrow.

Victor Hugo, Les Miserables, zyxwvutsrqponmlkjih Cosette, Book 1, chap. 17.

H

ubert Blalock has been concerned with the cumulation of knowledge in the social sciences and some of the problems related to it. My own concern here is the particular topic of methodological progress. How does such progress occur in the social sciences? The textbooks on research methods give little hint of how new scientific methods are developed. The usual “reconstructed logic” of the history of science treats ,the developments of method largely as having occurred because they were necessary, instead of reflecting on the conditions that lead to their occurrence. Yet any proper understanding of the actual “logic-in-use” in methodological innovation admits that such innovations require special conditions and infrastructures. The argument that I shall make here is that progress in social science research methods is dependent on three conditions: theory, technology, and training. Theory provides the basis for methodological progress, pointing to areas of inquiry that provide the opportunity for progress. Technology guides that progress, as progress is always limited by the available technology. Training fosters methodological awareness and thus encourages further progress. 38

Comment

39

Methodological Progress and Theory

Philosophy of Science All that is human must retrograde if it does not advance. Edward Gibbon, Decline and Fall o f the Roman Empire, chap. 71

The first element of progress in research methods is theory within a substantive context. Rarely do new methods emerge full blown outside of a substantive nexus. The substantive theoretical structure from which the innovations take off provides the setting for the progress that occurs. Theory is especially relevant in delimiting what is considered appropriate data. Data do not exist outside a theory, for it is the theory that calls attention to the appropriate data. Particularly relevant here is Kaplan’s (1964) discussion of “invisible data” as “those which are recognized as data only conjointly with the acceptance of the theory explaining them” (p. 134). It is theory which makes us realize what constitutes data and what data should therefore be collected. The dominant approach in the social sciences during the past 25 years has been empirical. Yet the empiricism of contemporary social science has not been atheoretical. It is embedded in a rich behavioral theory, with rational choice and evolutionary approaches providing additional theoretical perspectives which are simultaneously competitive with and complementary to that behavioral theory. Theory has been an important element in the ICPSR’s approach to data acquisition. First, the consortium came out of political science and, specifically, out of the political behavior movement. Certainly the organization has benefited as its horizons expanded past political science and political behavior studies, but the consequences of the original orientations remain noticeable. After all, the underlying behavioral theory emphasizes the importance of individual attitudes and cognitions. Survey research on the mass public was seen as an ideal means of measuring those individual attitudes and cognitions, and the ICPSR worked closely with the Center for Political Studies and the Survey Research Center in the design, execution, and storage of data from such surveys. Similarly, given its underlying theoretical orientation, the consortium has devoted its resources to facilitating the collection of elite data and to making those data available to a wide community of scholars. The dominant behavioral theory had historical and longitudinal implications, and the ICPSR has worked with relevant scholars to develop a social science history community and to foster longitudinal research. Thus,

Weisberg zyxwvutsrqp

40

the consortium has played a major role in promoting progress in social science methods because of these theoretical connections. The dominant behavioral theory has coexisted at times in the consortium’s history with other theoretical approaches, such as the rational choice perspective. The competition involved in such coexistence of alternative theories is healthy. Any one theoretical approach will consider some data relevant while ignoring other potential data; a second theoretical approach will suggest that new sources of data are relevant. Different theories will suggest different emphases in data generation, data collection, and data holdings, so the influence of multiple theories in the social sciences should usefully serve to increase the data holdings of the consortium. Theory may seem less relevant now that the consortium has expanded past a single discipline. Yet, progress in research methods can come only out of a theoretical connection. The theoretical direction may change over the years, but to sever the theoretical connection would stymie progress. The consortium will have to wrestle with the dilemma of how to retain a theoretical direction while operating across disciplinary lines. It must choose between being an atheoretical data library and continuing to have an active role in shaping social science research directions. Some changes in theoretical perspectives are appropriate and likely, but a theoretical nexus is essential for maintaining the consortium’s leadership role. If we do not continue to advance, we will retrograde as surely as the Roman Empire that Gibbon described. Data Analysis Progress is our most important product. (From a General Electric advertisement)

Progress in theory-driven data analysis over the past 25 years is most evident in the causal realm, where we have adopted a view of causality that has led to the development of techniques for determining the relative weights of alternative causal paths in both recursive and nonrecursive systems. Also, we have adopted a view of dimensional structure and classification that has led to refinements in cluster analysis and factor analysis, to a new emphasis on confirmatory factor analysis, and to the development of nonmetric multidimensional scaling. And we have adopted dynamic perspectives that have led to the development of techniques for modeling time series data and for estimating intervention effects. Data analysis a few decades ago consisted of only the mundane use of a very small number of tools, even when those tools had a poor fit with our

Comment

41

theories and data. Our methods often did not fit our rich substantive understanding of what was being studied, but that simplification was usually ignored. Today we are trying to incorporate into our data analysis both structural models of causality and measurement models, with special provisions being made for a realistic interpretation of the limitations imposed by the quality of measurements and research designs. Such techniques as logit and probit procedures for limited dependent variables, covariance-structuremodeling programs such as LISREL and EQS, and transfer function analysis epitomize this increased recognition that we need not oversimplify our methods when we study complex substantive processes. At the same time, I would caution against an evidently growing orthodoxy in the area. As the methodologists concentrate more and more of their efforts on adapting sophisticated approaches to analyzing our data, we are in peril of becoming complete captives of the linear model. Data analysis should be more than just another special case of the linear model. Other methodological approaches should continue to be emphasized. Also, methodological progress is unfortunately beginning to outstrip the capability of substantive specialists to understand the methodological work. If this were just an instance of subject-matter specialists’ needing training in some new methods, I would not be concerned. Instead, it is more a case of methodologists’ losing their concern with remaining intelligible to their colleagues in the substantive fields. And that is a loss that I strongly regret. If more value is placed on being esoteric than on being communicative, it is a loss for social science. It is important to be at the “cutting edge” in methodology as much as in any substantive field; yet we must not go to the cutting edge just to be at the cutting edge. It is important for methodologists to be able to communicate with one another in full jargon, but it is equally important for them to communicate with substantive scholars in more intuitive terms. Otherwise, we will be left with a very unsatisfying state of methodology where progress exists only for the sake of progress. Progress should not be our only important product. Methodological Progress and Technology Computer Utilization pity this busy monster, manunkind, not. Progress is a comfortable ease.

dis-

e. e. cummings, one times one, 14.

If the first condition for methodological progress is theory, the second

42

Weisberg zyxwvutsrqpo

is technology. Progress is inevitably linked to technology. There has always been a close bond between research methodology and the state of technology. One prime area of such a link for modern social sciences is computer utilization. Over the past quarter century, we have witnessed a remarkable revolution in the speed of calculation without a concomitant sacrifice in accuracy. In the early days of computing equipment, punched data cards were put into machines which could be wired to perform some calculations. Then programs were written to perform the desired analyses on large computers, with delays of several days between the submission of the job to the computer and the return of the output. Now, there are a few standard large-scale dataanalysis program packages which offer a wide range of statistical techniques and which can often provide virtually instantaneous output, sometimes on small computers on the researchers’ own desks. As a result of the computer revolution, the nature of data analysis in the social sciences has changed considerably. Procedures for univariate statistics, cross-tabulation measures of association, correlation, regression, and factor analysis had already been devised by 1962, but they became more convenient to compute with the development of high-speed computers. Indeed, some analysis techniques devised in the first half of the twentieth century were originally viewed as impractical except for small numbers of variables. For example, factor analysis was more important as a theory than as a practical procedure, since it would take a week for a large research staff to compute a factor analysis by hand, with a single computation error rendering that time wasted. The early factor-analysis literature devised several different factor solutions, some of which were developed only because they could be computed more easily than more theoretically satisfying solutions. By contrast, sophisticated forms of factor analysis can be evaluated instantaneously on modern-day computers, Similarly, many of the early cluster-analysis solutions were not theoretically pleasing, but they were devised as approximations of more appropriate factor-analysis or clustering procedures. Modern-day computer technology largely obviates the need for such concessions to practicality. The data collection process is also becoming computerized. The most notable case is the use of computer-assisted telephone interviewing (CATI) in the survey research arena. CATI can guide the interviewer through complicated branching sequences in the interviews, can demand the correction of inconsistent data entries during the data entry process, and can provide a continuous tallying of results so that early trends can be spotted. The computer can be useful in other approaches to data collection as well, such

Comment

43

as the computerized content analysis of documentary information. Also, today’s lap-top computers permit researchers in the field to enter their observations directly into a computer’s memory. While the impact of the computer revolution on the social sciences has been dramatic, there is at the same time some need for caution. First, we are still limited by the perennial problem in computer analysis: “Garbage in, garbage out.” The quality of the results depends on the quality of the input. Increasing the amount of computer analysis does not improve the quality of the analysis if the quality of the data is poor. Second, we are overly limited by the available computer software. Most “new” computer programs reinvent the wheel rather than extending it. Many microcomputer statistics programs are not much better than the mainframe statistics programs available a decade ago. We are moving toward more interactive analysis, but many microcomputer statistics programs are not even interactive. We need new ideas for analysis programs in the new microcomputer environment, rather than just the reprogramming of mainframe program packages for smaller machines. Third, we still do not take full advantage of the possibilities for graphics. It is fairly easy to have microcomputers provide graphic displays of our data before proceeding to a complete analysis. But graphics procedures are often separate from, rather than integral to, the statistical applications. Graphic displays can alert us to pathologies in the data, but we still use them as an afterthought because computer programs do not provide them automatically. The original Apple I1 and Macintosh computers fully incorporated graphics into their design, but the inclusion of graphics in statistical software has been handicapped by the confusing array of graphics and nongraphics monitors for the IBM P C line of microcomputers. With IBM’s movement to PS/2 technology, we may be on the verge of the acceptance of a graphics orientation for all microcomputers, an orientation that will facilitate the incorporation of graphics into our statistical applications. Fourth, we are too quick to accept flashy computer applications as valid because they are done on the computer. Thus, telephone interviewing nowadays must be done with state-of-the-art CATI systems using randomdigit-dialing (RDD) sampling procedures, even though it is still possible to take really good samples by much less technologically advanced procedures. Unfortunately, it is hard for a good survey operation to explain to a potential client why CATI and RDD are not necessary, so survey firms struggling to receive contracts must invest in such systems regardless of whether they make sense in particular applications. Fifth, we have allowed computer use to be dominated by word process-

44

Weisberg zyxwvutsrqp

ing. Everyone agrees that microcomputers are a wonderful invention, but statistical analysis on microcomputers has not kept up with word-processing applications. In other words, we are not taking full enough advantage of the power of our computers. This can be viewed benignly as a case of people who would not otherwise need computers finding that they can be used in writing, but I feel it is more than that. Word-processing programs have become simpler and cheaper, while statistics programs remain complicated and expensive. We have to make statistical programs as easy to use as wordprocessing programs and then see if the full power of the machine will be harnessed by more social science users. Our microcomputer usage need not be limited to statistics and wordprocessing. The spreadsheet and database applications that have become so prevalent in the business world would also be useful for social scientists, except that we do not sufficiently recognize their utility. The microcomputer revolution should be producing a revolution in social science computer use, but that is not occurring. We were relatively advanced in social science computer use by 1977; since then, business offices have taken great strides in their computer usage, while our computer use has remained amazingly steady (outside of word processing). This is not to say that progress in the computer field has stopped. There is a sense in which it has stalled. We can now do much more in this area than we used to, but we do not give sufficient attention to determining the next set of priorities. And we are once again about to be overtaken by technological advance. CD-ROM and optical storage technologies will soon make it possible for the most massive data collections to be stored on a very small medium which can be analyzed on a small microcomputer. For example, the entire set of American national election studies will probably fit on a single CD-ROM disk that can be accessed by a personal computer. Obviously this will be very convenient for many researchers, but so far, we have not seen a growth of software to take full advantage of this advance. We are going to need statistical programs which can retrieve variables across studies, do graphic analysis along with statistical operations, perform interactively, and use easier human interfaces. The consortium has been at the forefront of technological progress in the computer area. It has stored its data in machine-readable form since the beginning, and it converted its codebooks to machine-readable form very early. Its programming staff participated in some of the earliest development of social science programs, program documentation, program packages, and interactive programs. The entering of data into the computer, the ordering of data by member schools, and the sending of data to member

Comment

45

schools have become effectively automated over the years. Indeed, one of the remarkable cost-containment stories of the past decade is how the consortium was able to keep its fees stable during a period of rapid inflation and greatly increasing demands on its resources by improving its computer resources. A major issue for the consortium in the coming years will be to confront directly how its data resources will be utilized in the next computer environment. Being able to put large parts of its data resources on small data-storage devices will provide opportunities for data analysts but will simultaneously challenge a membership organization’s ability to control its own data-dissemination process, Technology is required for progress, but technological progress can also constitute a threat. Progress may be comfortable, as Cummings observed, and our computer progress may be too comfortable for our own welfare. Research Design

All progress is based upon a universal innate desire on the part of every organism to live beyond its income. Samuel Butler, Notebooks, Life

Social science progress is also related to the technology of research design. As new designs become possible, our research becomes progressively more sophisticated. We have seen progress in the technology of research design over the years as we better understand the distinctions between different types of designs. The role of experiments is better understood. The possibilities of quasi experiments are appreciated. The varieties of survey research designs have been more fully explored. It is important that we try out more different designs. It is too easy to repeat designs that have been used in the past, without recognizing the potential value of innovation. In the political analysis realm, the National Election Studies have experimented with a variety of designs to study change over the election year and to focus on the presidential primary season. Even so, we still need more experimentation. Surveys of the mass public should be more routinely supplemented by surveys of the relevant elites. When a study goes into the field, its questionnaire can be made available to other scholars who wish to use it, so that, for example, an in-depth study in a single state could use many of the same questions as are asked in national studies. The research-and-development work preparatory to surveys should include work in the social psychologist’s laboratory to understand better the answers which respondents are giving.

46

Weisberg zyxwvutsrqpo

Of course, innovation in research design is inevitably limited by financial resources. Research design is always a set of compromises between theoretical requirements and the practical limitations on time and money. Under such circumstances, methodological innovation and trying out new research designs can seem like superfluous luxuries. Yet, testing new research designs is the lifeblood of empirical research; without it, stagnation is likely to occur. Even with scarce resources, one must be willing to try out new research designs so that future studies can benefit. Perhaps the ultimate warning to be given in the design area is to remind data analysts that the data are only as good as the design. Too often, we analyze archived data without going back to wonder about the quality of the research design. Data on computer tape look so official that we forget where they came from and how they were generated. As more and more data are made machine-readable, the possibilities for the misuse of data grow rapidly. Of course it is important to make data convenient to use, but it is equally important to demand quality control of the data. Regardless of the politics involved, there are some data which an archive should not accept and should not distribute. In this light, one of the newest developments in science policy affecting the archiving field is a mixed blessing. For years, data archives have worked hard to obtain agreement by principal investigators to archive their data for public access. Now, the National Science Foundation (NSF) is beginning to require the research projects that it supports to send their data to archives. On the one hand, this requirement should make data access easier for other scholars. On the other hand, it makes it more difficult for archives to maintain quality control. Data archives may be inundated with data sets of questionable quality, in addition to being flooded by small, idiosyncratic datasets. The consortium has prospered during a period in which having more data has meant having better data, but the two may no longer be synonymous. In this new era, the consortium may now have to make decisions on what data to seek, what data to accept, and what data to save. Seeking large new bodies of data for its collection entails staffing as well as computer costs. The consortium has done well in finding foundation sponsorship for the costs associated with acquiring important new collections of data, but it may not always be so fortunate. In particular, the possibility of NSF-sponsored data collections being routinely shipped to the consortium raises the question of what limits there are to the organization’s appetite for data. Finally, the consortium will have to decide whether it should permanently

Comment

47

keep all data that it is given, or whether some sunset procedures are appropriate for data that are rarely, if ever, used. It is fortunate for an organization to have to face the problems associated with success rather than those associated with failure, but that success can also breed potential problems. The challenge of trying to live beyond its income has resulted in progress for the consortium, as Samuel Butler would have predicted, but the consortium should still consider what limits are appropriate for its data collections. Methodological Progress and Training

Graduate Education Not to go back is somewhat to advance, and men must walk, at least, before they dance. Alexander Pope, Imitations of Horace, Epistle I,

bk. I, I. 53

The third condition for progress in research methods is training. Graduate students must be taught the latest methods. Indeed, they must be given a sense of excitement about those new methods so that they will be interested in developing new ones themselves and in transmitting them to another generation of students. Social-science-research training was in its infancy a quarter of a century ago. A few social-science statistics books were available, most notably Blalock’s Social Statistics, but few social scientists were prepared to read or teach from such books. Progress in social science methodology required refining the training in methods for graduate social-science students. From its inception, the consortium took a lead role in this effort. Its summer training program has taught research methods to some 6,000 students. From the beginning, the consortium’s summer program has had to decide on the proper mix of teaching the basics of research methods and teaching “cutting-edge” work. At the beginning, almost everything was cutting-edge because so few schools were teaching research methods. Soon, the summer trainees were teaching methods on their home campuses, and the summer program moved to cutting-edge teaching modules in the areas of dimensional analysis, causal analysis, and dynamic analysis. The balance between the basics and the cutting-edge work has shifted back and forth over the years. Most recently, the consortium has instituted a series of new cutting-edge courses and has thus been maintaining its leadership in the field, while also retaining the basic courses for students who need them. Moreover, one-week intensive short courses on new research methods add to the ability of the program to retain its relevance in the continued retooling of

Weisberg zyxwvutsrqp

48

social scientists. Indeed, past students in the program now teaching throughout the nation and even past instructors in the program are returning to Ann Arbor for these one-week retooling efforts. The consortium always faces the question of its most useful role in teaching research methods. The summer program affords a unique opportunity to offer advanced courses that could not gather a sufficient enrollment on most single campuses. Not all schools need this cutting-edge methods training, but the summer program can retain a sense of excitement for its instructors only if it can have a real commitment to cutting-edge efforts. Perhaps someday its very success will render the summer program unnecessary, but for the moment, current enrollments are at record levels. With my apologies to Alexander Pope, the consortium played a crucial role in its early years in teaching social scientists how to walk through elementary research methods; as it achieves maturity, the consortium should play an equally important role in teaching social scientists the dance of cutting-edge data analysis. General Education

If there is no struggle, there is no progress. Frederick Douglass, from John W. Blassingame, Frederick Douglass: The Clarion Voice.

Finally, I want to say something about the implications of all of these developments for general education. E. D. Hirsch has written of the importance of “cultural literacy” that all citizens of a country should share some important knowledge (when some wars were fought, who some authors were, and so on), so that references to this knowledge in the media, the arts, and letters will have common meaning. There is a simple logical extension of these ideas to cover “quantitative literacy.” Citizens today are bombarded with statistics. Not only are they confronted with political poll results in the media with ever-increasing frequency, but they are shown all sorts of official statistics. Unemployment rates, cost-of-living indices, crime rates, trade surpluses and deficits, government budget data, and lists of best-selling records and most popular movies and television shows represent only some of this statistical information. Citizens must be able to deal with some simple statistical data reports. “Quantitative literacy” should be part of the general education for college students. As colleges revise their undergraduate curricula, they should be adding a requirement for a course explaining how to analyze data. Of course, I would not expect every citizen to be able to conduct an analysis-of-

Comment

49

variance or to perform a causal analysis. But the concepts of means, medians, variances, and correlations are important. The logic of hypothesis testing by way of significance tests, causal arguments and the notion of spuriousness, and scientific versus nonscientific sampling should be understood more generally. The ordinary citizen should understand when to believe numbers and when someone is trying to lie with statistics; he or she should understand when a medical study reported in the newspaper is reasonable and when such a study has used a poor sample with poor controls. In short, the time is ripe for data analysis procedures to move down one level in our educational system. As an example, in early 1987, a special curriculum committee on general education at Ohio State University reported and included in its description of an educated person: Of equal importance [to writing!] in the development of logical and critical thinking is the ability to make intelligent and sophisticated responses to problems involving quantitative and statistical data. Students should be able to discern the truth and fiction inherent in quantitative presentations used for planning, budgeting, or other matters affecting the general public. They should have a command of mathematical concepts and methods adequate for contemporary life and should understand arguments based on statistical data, surveys, and polls.

I have served on the committee developing a general education program to implement those goals, and this second committee is including in its report a requirement for a data analysis course for all of our B.A. and B.S. students: Every student should have an understanding of arguments involving quantitative data. In courses satisfying this requirement, mathematical skills should be reinforced in discussions of data gathering, presentation, and analysis. Graphical presentation and descriptive statistics should be emphasized. Examples should be drawn from areas such as polling, sampling, and the significance of measurement error. The uses and misuses of data should be discussed and elementary probability theory mentioned. Also a student should solve some exercises and problems with the assistance of a microcomputer.

This plan has not yet been adopted and may well be amended before it is instituted, but it demonstrates the importance of moving quantitative literacy into general education. The consortium has long had an interest in developing undergraduate instructional materials which use its data resources, particularly as shown in the SETUPS packages prepared in cooperation with the American Political Science Association. The consortium can play a lead role in developing

50 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Weisberg zyxwvutsrqp

educational materials for general education in quantitative literacy and should be actively advocating the inclusion of such quantitative literacy courses in the general education curriculum for the undergraduate and perhaps next for the high school student. This may entail the struggle of which Frederick Douglass spoke, but such a struggle is required for continued progress. Conclusion

Theory, technology, and training are three essential conditions for progress in social science research methods. The history of the consortium from 1962 through 1987 is, in large part, a saga of the interplay of the three in promoting such progress. In line with the quotation from Victor Hugo that opens this paper, the consortium has been a major actor for progress in research methods over the past quarter of a century, adopting a revolutionary stance toward achieving a vision of the social science of tomorrow. References Kaplan, Abraham (1964). The Conduct of Inquiry. San Francisco: Chandler.

Chapter 2

A Strange Case of Identity Denied: American History as Social Science Allan G. Bogue

istorians disagree both about what they do and about what they should do. Do they rethink the thoughts of their subjects or merely describe past behavior? Are they best visualized as storytellers, detectives, the compilers of informal legal briefs, moral critics, or the ghost writers of the past? How many of them would agree with the definition of historical writing given by George Dangerfield, a winner of the Pulitzer Prize for history: “a combination of taste, imagination, science and scholarship; it reconciles incompatibles, it balances probabilities; and at last it attains the reality of fiction, which is the highest reality of all”? (New York Times, Jan. 6 , 1987). If modesty and distaste for the obscure deter most historians from enlisting under that banner, they also reject the functional view of the historian as mere flack, caught so nicely in the light-hearted commemoration of the foibles of Edinburgh which ends:’ And send it good historians to clear whatever blots M a y rest upon the memory of Mary, Queen of Scots!

We could indeed continue rehearsing at length the different conceptions of role and mission held by historians, and we could complicate the story still more by examining the views of these same historians on the methods and evidence that they consider appropriate to their tasks. The plain fact is that historians, quite legitimately, may have different objectives, and their evidence, methods, and completed work reflect this simple truth. At the same time, as self-appointed guardians of the time dimension of analysis, they have a good deal more in common in their use of sources and their evaluation of evidence than is sometimes admitted. Historians have wasted 51

52

Bogue

much time and ink in trying to draw internal and external boundary lines, identifying traitors, and exaggerating the difference in approach and methods among themselves and between them and the members of the related disciplines. Since the emergence of the university as we know it, some historians have considered themselves social scientists. It is the social science dimension in history, its place in modern scholarship, and, more specifically, its relation to the history and mission of the Inter-University Consortium for Political and Social Research (ICPSR) during the last generation that I wish to consider here.

I Most of the men who established the American Historical Association in 1884 believed that they were developing a science of history in the Rankean tradition.2 Albert Bushnell Hart was honored by election to the presidencies of both the American Historical Association and the American Political Science Association. Speaking to the historians as their president in 1909, Hart (1910, pp. 232-233) defined the ideal scientific history. The members of a “genuinely scientific school of history,” he explained, would “remorselessly examine the sources and separate the wheat from the chaff . . . critically balance evidence . . . dispassionately and moderately set forth results.” Just as Darwin had spent twenty years accumulating and analyzing evidence before venturing a generalization, so with the historian: “History, too, has its inductive method, its relentless concentration of the grain in its narrow spout, till by its own weight it seeks the only outlet. In history, too, scattered and apparently unrelated data fall together in harmonious wholes; the mind is led to the discovery of laws; and the explorer into scientific truth is at last able to formulate some of those unsuspected generalizations which explain the whole framework of the universe.” Not all of Hart’s listeners would, perhaps, have echoed his every syllable, but such views were widely prevalent in the small community of university historians at the beginning of the new century. But unanimity on the methods, objectives, and content of scientific history never existed.3 Emergent critics within the discipline called for a “new history” and were most notably represented by James Harvey Robinson of Columbia University. Robinson urged his colleagues to develop a history that would better allow Americans to understand the present and that should be enrolled under the banner of social reform. In 1912, he collected various papers that he had prepared during the previous decade or so under the title, The New History, Here he argued that the objective of the historian should be to promote understanding of the present and to provide guidance for the fu-

History as Social Science

53

ture; that history should be broadened beyond the narrow confines of the political, diplomatic, and military realms; that the insights and methods of related disciplines should be used; and that the general perspective should be broadly genetic. Robinson entitled one of his chapters “The Allies of History” (pp. 70-100) and presented examples of the ways in which related disciplines had provided perspectives, data, and methods that allowed historians to provide accounts far superior to earlier treatments of the same subjects. He urged colleagues to use anthropology, archaeology, psychology, research in comparative religions, political economy, and sociology. These he called the “newer social sciences” (p. 83), in contrast, we assume, to that older social science, history. Not all historians were so welcoming in their views of social science. In his presidential address to the American Historical Association in 1908, George Burton Adams (1909) maintained that history was under attack from political science, geography, political economy, sociology, and psychology. “For more than fifty years,” he noted, “the historian has had possession of the field and has deemed it his sufficient mission to determine what the fact was, including the immediate conditions which gave it shape. Now he finds himself confronted with numerous groups of aggressive and confident workers in the same field who ask not what was the fact . . . but their constant question is what is the ultimate explanation of history, or, more modestly, what are the forces that determine human events and according to what laws do they act?” (pp. 223-224). On the other hand, a founding father of American sociology, Albion W. Small concluded that historians were “not really finding out what mattered most . . . causal principles . . . but were largely occupied with trivialities” (Hayes, 1927, pp. 155-156). The historians’ science was not a science of social theory, or laws, but a mere systematic method. Despite criticism, the scientific historians did not completely abandon the search for overarching rules or laws. As late as 1923, Professor Edward P. Cheyney (1924) submitted six historical laws to those listening to his presidential address at the meeting of the American Historical Association: (1) the law of continuity; (2) the law of impermanence; (3) the law of interdependence; (4) the law of democratic tendencies; (5) the law of necessity for free consent; and (6) the law of moral progress. Although eliciting an “ovation unprecedented,” Cheyney’s laws were a mixture of truism, metahistory, and unwarranted good intentions (Nichols, 1968, p. 95). They had little long-run impact on historical practice in the United States. There was, however, historical research being done, during the first thirty years of the twentieth century, that transcended the mere method of

54

Bogue

the scientific historians and that we can in good conscience call social science. Much of this work was done by Frederick Jackson Turner and his students, and other major contributors included Charles A. Beard and the scholars whom he had influenced. From a variety of sources, including the political economy of Achille Loria, ’hrner formulated major hypotheses and derivative theoretical statements to the effect that American democracy derived its peculiar characteristics from the influence of the frontier, and second, that the competing economic and social interests of the various physiographic regions and subregions of the United States shaped national policies to a significant degree. In both cases, Turner applied ingenious, although inadequate, methods of analysis, while at the same time bringing a rhetorical intensity to his argument that greatly contributed to its attractiveness. In its analytical methods, Turnerian research unfortunately stuck fast at the level of simple tables and comparative dot-map analysis. During his later career, Turner seems to have been unaware that social scientists in various departments at the University of Chicago were using statistical methods that would have been highly useful in addressing the research problems that he and his students sought to solve. (see Turner, 1920, 1932; Benson, 1960, pp. 1-91; Billington, 1973, pp. 108-131, 184-232, 364-390, 444-471; Jensen, 1969a). A student of Robinson who was exposed to academic Marxism while sojourning at Oxford, Charles A. Beard published a stirring contribution to American political science and history in 1913, A n Economic Interpretation of the Constitution. Here he presented a cluster of related hypotheses: the Constitution was an outgrowth of a conservative counterrevolution, it was an economic document reflecting the policy preferences of major economic interests, it was approved in undemocratic fashion, and so on. Beard derived much of his evidence from a collective biography of the members of the Constitutional Convention, an impressively systematic exercise. Criticized emotionally by those who preferred to believe the Founding Fathers to be pure patriots, Beard’s study of the framing of the constitution was a tour de force that uniquely expressed the progressive outlook and the frustrations roused in the minds of early twentieth-century reformers by the efforts of defenders of the status quo to shelter behind the protective clauses of the federal Constitution (see Benson, 1960, pp. 95-228). More than any other historians of their era, lbrner and Beard articulated and shaped the so-called progressive interpretation of American history. Theirs was a past in which economic forces or interests contended for control - “economic power secures political power,” wrote the young Turner- and, in their view, the ordinary people, despite exploitation by

History as Social Science

55

plutocrat and corporation, advanced painfully toward a fuller enjoyment of their democratic rights. During the 1920s, Charles and Mary Beard wrote the progressive interpretation into a great survey text, The Rise of American Civilization (1927), which was still being assigned in American history survey courses during the late 1940s. Both Turner and Beard were men of great personal charm and persuasiveness. Turner never forsook the academy; a progressive martyr at Columbia, Beard became, in effect, a free-lance author of texts and treatises, a municipal consultant, and a gentleman dairy farmer (see Beale et al., 1954; Nore, 1983). When Howard W. Odum edited American Masters of Social Science in 1927, five of the nine biographical sketches in the book described the contributions of historians, although two of this group worked the borderlands of constitutional law between history and political science. Apparently the social scientist, Odum, was prepared to include history as a full-fledged partner in the social science enterprise. During the same year, Edward C. Hayes, a sociologist, edited Recent Developments in the Social Sciences and delegated Harry E. Barnes, a committed new historian, to survey recent developments in history. Here again history was included among the social sciences.4 Barnes argued that practicing historians were committed to seven general interpretations: the great man theory, the idealistic, the scientific and technological, the economic, the geographical, the sociological, or the “collective psychological.” But in general, he noted, “the older type of historian either clings to the outworn theory of political causation, or holds that historical development is entirely arbitrary and obeys no ascertainable laws” (p. 397). He classified Beard among the economic historians and placed Turner in the geographical camp. Barnes maintained that the historian who wished to “write intelligently about the history of society” must understand sociology, political science, jurisprudence, and economics. “Nothing,” he wrote, was “more humorous and tragic than the insistence by the historian upon intensive training in paleography, diplomatic, lexicography and the principles of internal and external criticism in the effort to secure accurate texts and narratives, and the co-existent ignoring of adequate training in the only group of studies which can make it possible for the historian intelligently to organize and interpret his material” (p. 403). It was, he said, a “scandalous state of affairs.” Thus Barnes’s ideal historian was one who was thoroughly trained across the range of social sciences needed to understand society. Such a scholar would presumably have been, in Barnes’s eyes, a social scientist; but the conventional historian of the time was, in his estimation, something less.

56

Bogue

Another datum of the 1920s highlights the somewhat anomalous position of history within the social sciences. During 1923 and 1924, Charles E. Merriam was organizing the Social Science Research Council (SSRC) to serve as the voice of social science in dealing with foundations and other granting agencies. When he invited the American Historical Association (AHA) to participate in the new organization, the AHA president, Charles H . Haskins, rebuffed him, noting that the literary and archaeological constituency in the historical association might have little interest in the objectives of the SSRC. He raised the issue of funding, fearing perhaps the possibility of levies on the constituent societies or the potential competition which the new agency might provide for the recently organized American Council of Learned Societies in its search for funds. Shortly, however, with a more cooperative president at the helm, the AHA joined the ranks of the SSRC (see Karl, 1974, pp. 123-133; Nichols, 1968, pp. 116-123). The developments in history subsequent to 1930 are best illuminated if we briefly examine two presidential addresses delivered to the American Political Science Association (APSA), the organization with which the American Historical Association shared four presidents during the first third of the twentieth century. In 1925, Charles E. Merriam spoke to his assembled colleagues on “Progress in Political Research” (see Merriam, 1926). A power within his discipline and in social science at the University of Chicago, a practicing progressive politician, and an early prototype of the academic entrepreneur, Merriam spoke in terms that, in retrospect, were highly prophetic. The recent tendency in political research, he believed, had been the trend “toward actual observations of political processes and toward closer analysis of their meaning . . . in contrast to a more strictly historical, structural, and legalistic method of approach.” He suggested the possibility of offering courses “in many categories of political action as legitimate” as the formalized categories of that time: the use of force in political situations, propaganda, conference processes, and leadership, among other examples. “Some day,” he forecast, political science might “begin to look at political behavior as one of the essential objects of inquiry.” Government, he argued, was “fundamentally based upon patterns of action in types of situations.” Merriam suggested to his colleagues that they were “on the verge of significant changes in the scope and method of politics, and perhaps in the social sciences as a whole” (ibid). He applauded the appropriate application of statistics in social science research. Two advances appeared inevitable: “One is toward greater intensity of inquiry . . . the other is toward closer integration of scientific knowledge centering around political relations.”

History as Social Science

57

Merriam thought it likely that there was to be “closer integration of the social sciences,” since “neither the facts and the techniques of economics alone, nor of politics alone, nor of history alone, are adequate” to the “analysis and interpretation” of many social problems. He asked, “Is it possible to build up a science of political behavior, or in a broader sense a science of social behavior with the aid of” new developments in other sciences? “Perhaps not” as yet, he admitted but he foresaw “interesting possibilities .” A year later, Charles A. Beard delivered his presidential address to the APSA, “Time, Technology, and the Creative Spirit in Political Science” (see Beard, 1926). He challenged his listeners to prepare their students for the changing world that they must face, given the remorseless play of time and technology upon society. But he catalogued a “heavy burden of acquired rights and servitudes” that impeded “creative work” in political science. There was a tempting accretion of statutes, judicial decisions, and the like that obscured the scholar’s understanding of the “stern realities of life.” A “second great incubus . . . is the baggage provided by the professional historian.” The circumstances of academic life, Beard believed, also worked against adventurous, creative thinking in a myriad of ways. Finally, he argued that the “monoculous inquiries” then “generally praised and patronized’’ were harmful, narrowing “the vision while accumulating information” (ibid., pp. 6-8). Although Beard was prepared to approve and even extend “research in detailed problems with reference to specific practical ends . , . still with respect to large matters of policy and insight there are dangers in overemphasis .” He maintained that “research under scientific formulas in things mathematically measurable or logically describable leaves untouched a vast array of driving social forces for which such words as convictions, faith, hope, loyalty, and destiny are pale symbols - yielding to the analysis of no systematist .’, Excessive emphasis upon inductive methods, he warned, discourages “the use of that equally necessary method- the deductive and imaginative process.” To Beard, the solution to disciplinary problems lay less in drafting research agendas than in modifying the scholar’s environment to give the widest latitude to “inquire and expound - always with respect to the rights and opinions of others.” Of more specific recommendations he had few. “Something pertinent,” he believed, might be derived from “a study of the factors that have entered into the personality of each great thinker in our field.” But of Americans, only the authors of The Federalist and perhaps John Taylor would qualify for inclusion. He endorsed practical experience in politics and argued that “no small part” of the profession’s

58

Bogue

“intellectual sterility” could be attributed to “the intense specialization that has accompanied over-emphasis in research.” Work in the cognate fields, particularly economics, would be beneficial, and he reminded his hearers of Buckle’s “profound generalization . . . that the philosophy of any science is . . . on its periphery where it impinges upon all other sciences” (Beard, 1926, pp. 8-11). But in this address, reservations about present practice set the tone; Beard obviously did not share the vision of the future that Merriam presented. In his presidential address to the APSA, Beard, as we have seen, described the work of the professional historians as being a “great incubus” and indeed charged that the historian “in mortal fright lest he should be wrong about something . . . shrinks from any interpretation.” He suggested that history as a discipline was permeated with the “philosophy of Alice in Wonderland” (1926, pp. 6-7). One may wonder why Clio should have been willing to draw such an asp to her bosom, but when the American Historical Association created a Commission on the Social Studies at the end of the 1920s, Beard became a member and played an extremely important role in its work. He wrote the two volumes that defined the nature and the appropriate role of the social sciences and exerted a major influence in shaping the group’s final report (see Karl, 1974, pp. 186-200). While the commission was at work, Beard became president of the American Historical Association, and his presidential address and his volumes published under the aegis of the commission can be regarded as essentially complementary. In Part VII of the Commission’s Report, The Nature of the Social Sciences, Beard (1934b, pp. 53-58) provided a short history of Western historiography. During the mid-nineteenth century, he noted, historians committed to a secular approach became disenchanted with the “partisan, class, and nationalist uses” of history and, influenced by the “collecting, analyzing, sifting, and arranging” of “‘facts”’ in the natural sciences and the discoveries supposedly based on the process, they proclaimed themselves to be “scientific.” In Beard’s words, such historians said, “‘Let us clear our minds of purposes and conceptions, let us get the facts - the raw facts - and let us draw from them only the conclusions which their intrinsic nature will allow.”’ This was the prelude to a great efflorescence of activity in the collection and editing of historical materials and in the writing of history. These practitioners, argued Beard, believed that a “great synthesis of world history would emerge” and that “some genius would discover the ‘laws’ of all history as Darwin had presumably disclosed the ‘laws of evolution.’” Alas, wrote Beard, neither was to emerge. By the early twentieth century, continued Beard, the scientific his-

History as Social Science

59

torians were themselves under fire, accused no less of writing to their own agenda than their predecessors, and of‘ unwarrantedly accepting the physicist’s theories of causation. He challenged them to explain why, if they could so well explain past events, they could not predict the future. Even their “basic assumption” that there were “‘raw facts,”’ which, if appropriately assembled would “automatically and inexorably suggest or dictate their own conclusions in all cases,” was untrustworthy. Disconcertingly too, observers discovered subjective elements in the scientific historians’ much admired model of objective analysis, the research of the physicist (Beard, 1934b, pp. 56-58). Beard sought to describe the emerging “school of contemporary historiography.” Its practitioners rejected the “‘chain of causation’ idea long dominant in physics,” as well as the concept of society as organism “borrowed from biology.” The new-wave historian would “concede” “the utility of the empirical method of fact-finding and [insist] upon its use in intellectual operations” but would maintain that it is an “instrument of thought” and not a “machine which automatically manufactures truth, guidance, correct conclusions, and accurate understanding of the whole past as it actually was.” Though rejecting the “closed perfection” of the scientific historians, the contemporary historian recognized the need for the skilled consideration of “occurrences, records, documents, and established particularities of fact.” Eschewing the determinism of physics, they hold that “scholarly and empirical research . . . may, in a large measure, disclose the conditions which made possible past occurrences, institutions, revolutions, and changes.” Rejecting the possibility of discerning precise cause and effect in history-asactuality, the new-style historian recognized the possibility of accident or contingency “in which great efforts of human design and will seem ‘to make history’” (ibid., pp. 59-60). From this line of argument, Beard suggested, it followed that “the central consideration of modern historiography becomes the relation between ideas and interests in history-as-actuality, the relation between individual and mass thought and action on the one side and total environment on the other. . . , An ideal written history would present certain conditioning realities and forces in their long perspective . . . the drama enacted by the human spirit within the conditioning, but not absolutely determining, framework of the material world.” Now, Beard continued, with “the determinism of empirical history” cast off, the historian could once more reintroduce the “role of personality in history.” In this history, there was room for ideas, for “will, design, courage, and action, for the thinker who is also a doer.” Although the historical actor

60

Bogue

is also influenced by conditioning circumstances, “by understanding the conditioning reality revealed by written history as thought and description, by anticipating the spirit of the coming age, he may cut new paths through the present and co-operate with others in bringing achievements to pass.” Thus “biography is restored to history” (ibid., p. 61). In his presidential address to the American Historical Association in 1933, Beard maintained that historiography was in crisis. Historians “must cast off their servitude to the assumptions of natural science and return to . . . history as actuality.” They must accept the fact that each member of the discipline works within a unique set of conditioning influences or frame of reference. If historians wished to place history within an overarching system, their options were limited: history as chaos, history as cyclical in nature, and history as linear and directional. “The historian who writes history, therefore, consciously or unconsciously performs an act of faith, as to order and movement, for certainty as to order and movement is denied to him by knowledge of the actuality with which he is concerned.” Should historians abandon the scientific method? “The answer,’’ he said, was “an emphatic negative.” “[Tlhe empirical or scientific method . . . is the only method that can be employed in obtaining accurate knowledge of historical facts, personalities, situations, and movements.” But “the historian is bound by his craft to recognize the nature and limitations of the scientific method and to dispel the illusion that it can produce a science of history embracing the fullness of history, or of any large phase, as past actuality” (see Beard, 1934a, pp. 221-222, 226-227). Beard’s relativist vision of history left the scientific historian with only an empirical method, which, though seemingly necessary, could deliver far less to its practitioners than had once been thought. His alternative challenge to historians to make their act of faith and set their work within the great designs of history seems to echo his pronouncement that individuals could “by anticipating the spirit of the coming age . . . cut new paths through the present and cooperate with others in bringing achievements to pass.” To some, such a definition of role appeared to reduce the task of the historian to that of mere propagandist. Certainly Beard’s formulas revealed a scholar who had little interest in directing historians down that path toward a rigorous science of human behavior of the type that Charles E. Merriam forecast in his presidential address to the American Political Science Association in 1926. Part of the contrast between the messages of Merriam and of Beard seems to lie in the differing levels of analysis involved. When Beard thought

History us Social Science

61

of historical laws, he conceived of sweeping generalizations like those propounded by Cheyney (1924). In 1931, when the work of the AHA commission must have been very much in his mind, Beard joked that he had been able to discern only four historical laws at work in the affairs of humankind, Of these, the first was, “Whom the gods would destroy they first make mad,” and the last ran, “When it gets dark you can see the stars.” He did not think in terms of middle-range social theory or the bounded generalizations that were to be so vital a part of behavioral analysis. But some of the brightest and most influential young American historians of the 1930s idolized Beard, thronged around him at professional meetings, and welcomed letters from “Uncle Charley.” If one historian must be chosen to symbolize the trends of American progressive historiography during the years between 1930 and 1946, it should be Beard. The appearance in 1946 of Social Science Research Council Bulletin 54, Theory and Practice in Historical Study, marked a historiographic turning point. A letter from Beard initiated the institutional sequence leading to the preparation and publication of the report, and he was a major contributor to it. The criticism that greeted the bulletin’s publication forecast a new era in the evolution of history as social science (see Beale et al., 1954, pp. 251-252; Social Science Research Council, 1946). Merriam also served on the Commission on the Social Studies and was so frustrated by the final report that he refused to sign it. His differences with Beard appear to have become obsessive. During the early 1930s he prepared a lecture series dedicated to the proposition that the Constitution, far from being Beard’s conservative product of counterrevolution, was a radical document. Some historians caught the vision of behavioral analysis during the 1930s, but like many political scientists, most ignored the gate that Merriam sought to swing open. In the view of later behavioral historians, their predecessors of the 1930s and 1940s followed the standard of the “pretender,” rather than that of the true Prince Charles (Karl, 1974, pp. 189-200; Nichols, 1968, pp. 92-109). The historian’s task that Beard defined in The Nature of the Social Sciences (1934b) was both challenge and charter of liberties for those interested in developing intellectual history. Through the 1930s and early 1940s, the emphasis in American history swung significantly toward the history of ideas and of the social aspects of American life. That literary sector of the profession to which Haskins had referred many years earlier gained in strength. The belief that history was more appropriately considered a humanististic discipline than a social science came to dominate the profession.

62

Bogue

II zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA If the years prior to World War I1 show ample evidence of interrelationships between history and the social sciences, as well as indications of varying opinions on what those linkages should be, the 1950s were years in which closer ties were again advocated and forged. Some historians, of course, proclaimed themselves to be humanists; others announced that they practiced neither art nor science, but simply history (see Vaughn, 1985, pp. 149-157). Some well-known practitioners developed a dual persona: when describing the flow of unique historical events and human action in the past, they performed as historians, and when concerned with regularities or recurrent phenomena, they confessed to writing social science (see Malin, 1958). In their approach, other historians of the postwar years might most aptly be termed waders. They happily introduced social science hypotheses in explanation of historical phenomena but showed little interest in providing the rigorous mechanics of test or proof that social scientists might have demanded. Richard Hofstadter (1955) perhaps best illustrated such an approach at that time, and his imaginative efforts were rewarded with a Pulitzer Prize for The Age of Reform, an influential monograph describing the progressive reform movement in the United state^.^ In contrast to the waders, other historians might be labeled imrnersionists- eagerly accepting theoretical models from social science and trying to use them in a systematic and rigorous fashion. Social science influences worked sweeping changes in field after field of American history during the 1950s and 1960s, significant developments occurring initially in economic and political history. Seminal conferences took place in both of these subdisciplines in 1957. At Williams College in that year, the National Bureau of Economic Research and the Economic History Association jointly sponsored a symposium on the growth of income and wealth in the United States. Taken in conjunction with several other landmark papers of about that time, the presentations at Williams College suggested that a major reorientation might be under way in the study of American economic history. At the time, the state of knowledge and method in American economic history was best exemplified in well-written descriptions of American economic institutions, business leadership, and government economic policy. Although such work was not devoid of motivational assumptions, or explanatory theory, it did not provide specific tests of economic theory or identify the relative importance of the various factor contributions to historic outcomes; the statistics were essentially descriptive.6

History as Social Science

63

The young scholars of the Williams College meeting did not abandon the general topics of the institutional historians - they too were concerned with economic growth, economic institutions, and government policies. But utilizing neoclassical economic theory (for the most part), accumulated time series (along with others that they developed), and econometric methods (particularly regression analysis), they posited economic models in which they were able to specify elasticities, and factor shares, and to make precise attributions of causal influence. Sometimes residuals served as the basis for estimates, and the use of so-called counterfactuals produced great consternation. “HOW much trust can one put in an estimate derived from taking the difference in output between the real-life economy of the past and an imaginary one?” demanded critics.7 Their numbers swelling speedily with new recruits, the new economic historians rapidly developed a remarkable esprit de corps and enthusiasm that was fostered by the development of a yearly “Cliometrics” meeting at Purdue University (subsequently migrating to Wisconsin and beyond). Attendance at one of the Purdue meetings during the early 1960s was an experience that the first-timer would not soon forget. The econometric historians worked a veritable reconstruction of the economic history landscape in the space of a generation. They substantially revised our understanding of causal relationships, the functioning of the economic system, and various equity considerations. Today we view the history of American slavery, the development of major industrial sectors, public land policy, the currency and credit systems, the growth of cities, and many other aspects of American economic history quite differently from the institutional economic historians. The many books and articles of the new economic historians verified what both theorists and institutional historians had assumed from their different vantage points: rational economic decision-making explained most developments in American economic history (see Fogel and Engerman, 1971; Davis et al., 1972; National Bureau of Economic Research, 1960, 1966). This literature also provided us with a greatly enhanced understanding of the richness of the source materials available, as well as an abundant harvest of quantitative indices and time series that will continue to serve historians in the future. No one economic historian served uniquely as a leader of the econometric history movement. Slightly senior in professional advancement to most of his colleagues in the movement and editor of the Journal of Economic History at a crucial period in the development of the new persuasion, Douglas C . North perhaps came closest to filling that role. Although we

64

Bogue

cannot designate the work of one individual as being preeminently influential, Robert W. Fogel showed an intuitive flair for drawing attention, respect, and sometimes scorn to that part of the new economic history agenda on which he chose to work. His research on the contributions of railroads to economic development and the place and significance of slavery in America drew the jovian bolts of infuriated critics as no other of his peers (Fogel, 1964; Fogel and Engerman, 1974). None questioned the fact that the new economic historians were social scientists, and the apparent rigor of their approach stood as an example and a spur to imitation in other areas of history, although these practitioners were located almost solely in departments of economics. The meeting of historians on the politics of the early national period at Rutgers University in 1957 was less elaborately planned and structured than that of the economic historians at Williams College. Personal statements of research interest and activity and informal discussion provided the fare for the political historians. But Richard P. McCormick’s report to the funding agency, the Social Science Research Council, was in retrospect a notable document, discussing and urging the adoption of new research emphases, methodologies, and institutional strategies that soon became highly apparent within the profession (see McCormick, 1957). Lee Benson (1957, 1960, 1961, 1971) played a major role in the discussions at Rutgers and provides the prime example of an immersionist in political history. Trained in history at Brooklyn College and at the graduate schools of Columbia and Cornell Universities, Benson found congenial the admonitions of his doctoral adviser, Paul W. Gates, to work diligently in the related disciplines. At the conclusion of his doctoral work in the early 1950s, Benson received a postdoctoral training grant from the Social Science Research Council that enabled him to carry his interdisciplinary interests still further. During the mid-l950s, he developed ties with Paul F. Lazarsfeld and the researchers of the Bureau of Applied Social Research at Columbia University. In this young historian, Lazarsfeld sensed a scholar who might well assist in making history more rigorous, systematic, and problem-oriented than the flaccid discipline that this social scientist believed it to be. Lazarsfeld’s encouragement and modest subsidization, added to support from the Rockefeller Foundation and the Center for Advanced Study in the Behavioral Sciences, assisted Benson in producing a monograph on the use of hypotheses, interpretations, and systematic analysis in American political history, another on the ideas and methodologies of Turner and Beard, and a substantive study of the politics of the Jacksonian period that was to influence strongly the writing of a generation of American political historians.

History us Social Science

65

History’s different objectives, Benson believed, involved the provision of entertainment, of a sense of identity, or of normative values, and he hoped also that history could be written as social science, a goal that had not yet been achieved. He was to be a unique catalyst of the so-called new political history. Associating with the social scientists of the Bureau of Applied Social Research during the mid-l950s, Benson adopted their folkways. He discussed his developing research on the nature of politics in Jacksonian America widely and circulated working papers. The approach and substantive content of his book The Concept of Jucksoniun Democracy: New York us u Test Case was much less a surprise on its publication in 1961 than might otherwise have been the case. But as no one book in the new economic history, Benson’s study of the Jacksonian Era forecast the revisionist agenda of the next twenty years in American political history. Here was a rejection of the Beardian or progressive thesis that the clash of competing economic interests or forces largely shaped political outcomes. Within the heterogeneous society of the United States, a broader range of causal variables might be found in play, argued Benson, in differing strengths and combinations at various times. In the Jacksonian Era, according to Benson, ethnoculturalreligious group affiliations were better predictors of party affiliation and electoral choice than economic interest. Benson’s comparisons were quantitative in nature, and quantitative analysis, of a rather elementary sort, was another major characteristic of his analysis. Benson found the theoretical rationale for the new causal explanations in Robert K. Merton’s discussions of reference group behavior. Almost as an aside, Benson suggested that the coalescent parties of the Jacksonian period might well be viewed as the receptacles of social groups of two basic cultural orientations - Puritan and non-Puritan, a dichotomy that some later scholars elaborated in detail. Benson also noted an element of periodicity in the history of the American political parties, a tendency for recurrent reorganization or reorientation that he believed to be analogous to business cycles. During the twenty years after the publication of Benson’s book, dozens of books and articles showed the impact of its argument-modifying, amplifying, and rejecting, but at the same time working within the general framework articulated there.9 Although the major themes and emphases of a “new” political history are clearly evident in the report on the Rutgers conference and in Benson’s writings during the second half of the 1950s, it was Samuel P. Hays who succinctly identified what was afoot in the title “History as Human Behavior” (1960), under which he published a short hor-

66

Bogue

tatory article, describing, elaborating, and endorsing various of the approaches being discussed in the network of historians developing around Benson. The new political history was an aspect of the behavioralism that swept to dominance in the social sciences during the years after World War 11.10 In an era when “history from the bottom up” became the cry, the study of American political elites and legislative activity was less popular than investigation of electoral activity, but a substantial number of historians tried to use the quantitative methods of political scientists and sociologists in charting the legislative behavior of individuals and groups, and others looked for the explanation of elite political behavior by compiling extensive collective biographies or exploring the utility of modernization theory. In sum, the practitioners of the new political history created a very different landscape from that depicted by the progressive historians of the 1940s and 1950s, richer both in detail and in theoretical conceptualization, and much more social-scientific in tone than the publications of the previous generation. The future of the new political history appeared to be bright during the mid-1960s. The American Historical Association had recognized Benson and a core group around him as the association’s Ad Hoc Committee on the Quantitative Data of American Political History. At Benson’s suggestion, Warren E. Miller, the director of the new Inter-University Consortium for Political Research (ICPR), invited their cooperation in the gathering of basic political data series. Grant monies for committee work flowed to it through the parent association and the ICPR, and it sponsored conferences and seminars appropriate to its mission. Members of the committee discussed their vision and their substantive research before large audiences at association meetings or in other academic settings. Benson pursued the twin tasks of proselytization and organization with apostolic fervor. Within approximately a decade of its organization during the early 1960s, his committee had become a standing committee of the AHA with an international focus and had also spawned the Social Science History Association. But the period during which the new political history was a new or major attraction was relatively brief. When in 1964 Stephan Thernstrom published the monograph Poverty and Progress: Social Mobility in a Nineteenth-Century City, he touched off an explosion of interest in urban and community studies that particularly focused upon the phenomenon of social mobility. Meanwhile, the community study approach was being applied in the study of colonial society as well (Greven, 1970).11 Rising on such foundations, a veritable boom began in

History as Social Science

67

social history during the years 1965-1975 that far outclassed the activity in political history of the early and mid-1960s. This involved the relatively rigorous urban social history of the kind pioneered by Thernstrom and the authors of several quantitative colonial-community studies and, on one flank, the still more technical and quantitative research of a small cadre of demographic historians. But historians of ethnic and other minority groups and women also claimed the title of social historian, as did students of the family and scholars investigating a variety of American social institutions. Chroniclers of such research cite advances in our understanding of the psychology of childhood, of the historical development of the nuclear family, of the factors associated with crime in society, and of changes in social mores, as well as an enhanced understanding of social mobility. Proponents of a working-class approach have reshaped the history of American labor. There has been variation in the degree to which the social historians, broadly defined, have been influenced by the social science literature. Controversy developed as to whether social history must demonstrate its scientific character by the use of quantification, and growing numbers of scholars argued for the negative. Even so, much of the social history of the late 1970s and the 1980s displayed an analytic dimension that was not apparent in the prebehavioral days.12 When compared with their predecessors of the 1930s and 1940s, or the more traditional post-World War I1 practitioners, the new economic, political, and social historians of the 1950s and 1960s shared some characteristics. Their papers and publications were much more analytical in tone than those of conventional narrative historians. At their best, the latter dug deeply and analyzed their evidence carefully but did not interrupt their narrative story line with an explication of models or a consideration of rejected alternative explanations. The new historians focused on problems and themes and tended to argue their case with both themselves and their predecessors. They were, in general, much more interested in the grass roots, the city block, and social masses than in elite figures. Societal processes attracted them more than did discrete events or colorful individuals. These scientific revisionists were self-conscious in their use of political and social theory, conceptual frameworks, and behavioral models derived from the related social sciences. Their sources dictated the use of statistical techniques and new methods of analysis. Indeed, “systematic analysis” became a key phrase. Sacrificing breadth for depth, they were particularly drawn to the use of the case study. Within the history discipline initially, there was a tendency to see quantification as the major difference between the new and the old history. It did provide the most obvious contrast, but there was much more to the reorien-

68

Bogue

tation of a significant sector of the discipline than the analysis and presentation of quantitative evidence. The fundamental issues were broader: How fully and how well were the theory and methods of social science to be integrated into the discipline and to what degree was the historian to contribute to a rigorous social science literature? Hypothetically, at least, as some historians came to agree, publications in social-scientific history might be both truly scientific and minimally quantitative. Not all historians enlisted under the banners of change during the thirty years after 1955, some preferring to follow conventional practices and to publish narrative accounts of political, intellectual, and social phenomena. In political history, historians of ideas mounted a major investigation of the concept of republicanism in American national history. Biographers retained their popularity, and few of them felt it essential to qualify as psychohistorians. Conservative historians rallied in criticism of the new persuasions: “numbers were not literature,” they could not be used to address the really important problems, their intoxicated devotees could have better used their energy elsewhere, and so on. Although the new historians believe that they have helped to created a broader, more diverse, and more engaged discipline, critics dismiss quantification as a fad, emphasize the splintering and lack of communication within the profession, and bemoan the excessive emphasis that they believe has been placed upon insignificant or excessively specialized subjects. By the late 1970s, eminent members of the profession were calling for a return to synthetic narrative history, and one of them contracted with a major press to edit a new multivolume survey of American history in that style. And critics from a revitalized left joined in the criticism of history that belabored the reader with sterile numbers and that used techniques that were too clumsy to catch the changing processes of social class formation (see Barzun, 1974; Bailyn, 1982; Genovese and Fox-Genovese, 1976; Stone, 1981; Wilenz, 1982; Woodward, 1982). Some of the criticism of the new histories came from within. Indeed, the movement has been characterized by intense self-criticism and technical disagreements that have often been misunderstood by conventional practitioners. Seldom stressed has been the fact that the new historians restored a concern with method and an insistence upon the evaluation of source materials that was perhaps even more far-ranging than had been the case in the earlier heyday of scientific history. During the late 1960s and early 1970s, a substantial number of books and articles appeared, explaining the application of quantitative methods and social science theory in history. Among the various journals begun to serve as vehicles of the new histories, one was devoted solely to such matters.13 Various summer programs were created,

History as Social Science

69

dedicated to improving the quantitative skills of history graduate students and faculty, and by the late 1960s, courses in quantitative methods began to appear in history department offerings. These developments immeasurably increased the methodological sophistication within the history profession. Behavioral historians in good standing also noted that the excessive penchant for case studies and the tendency to deal with relatively narrow topics over short time spans contributed to a literature that was excessively compartmentalized and uncoordinated.14 But in all of the American fields of the new histories, the behavioralists did try to utilize social science theory. If they were much more consumers and testers of the theory than its inventors, they were, nonetheless, essentially social scientists in their approach. Few social scientists are basic theory builders, and few social science areas have generated highly successful predictive models. But it is also true that in most areas of the new history, the theory in play was inadequately developed, and opportunities to ask and answer additional questions of theoretical interest were often disregarded.

111 The concept ofparadigm has been both used and abused during the last generation, to the point where an effort to apply it in an explanation of change in so disparate a discipline as history is ill-advised. The term is now so burdened with special meanings that the expression “research orientation” is probably preferable. Unquestionably, a very considerable revision of method, substantive thrust, and - to some degree - purpose took place within a substantial sector of the historical discipline between 1945 and the early 1970s. A number of stimulating or facilitating developments were apparently involved (see Kuhn, 1962, 1977; Gutting, 1980; Hollinger, 1973; Wise, 1973). One aspect of the situation involved the coming of a post-World War I1 generation into the graduate schools, including a considerable component of veterans, somewhat older, more serious, more worldly-wise than students in the preceding years of war and peace, and probably less willing to accept professorial pronouncement as holy writ. This cohort’s background of social experience was markedly different from that of its predecessor of the 1930s. The major themes of progressive historiography failed to capture the realities of American society for these scholars of the GI Bill. The historiographic mission and rules formulated by Beard and other progressive historians during the 1930s seemed less satisfactory as well. These ideas and precepts appeared in penultimate form in SSRC Bulletin 54 in 1946, and that publication drew sharp criticism as well as praise. The way

70

Bogue

was being prepared for new departures. Negative reevaluations of the research of the previous generation characterize changing disciplinary research orientations. Beard’s attack on Franklin Delano Roosevelt’s war diplomacy had lost him part of his historian constituency. And during the 1950s, three young scholars subjected Beard’s An Economic Interpretation of the Constitution (1913/1935) to critical scrutiny. In each case, this classic of progressive historiography was found wanting in both conceptualization and method (Social Science Research Council, 1946; Brown, 1956; McDonald, 1958; Benson, 1960). Even less satisfying to some was the call for a history that emphasized dramatic narration. When Samuel Eliot Morison called upon the members of the American Historical Association to revitalize their narrative obligations, Thomas C. Cochran (1948) stressed the shallowness of conventional history and, using political history as an illustration, suggested that the practice of presenting our political past within a framework of presidential administrations was sterile. He argued that the presidential synthesis should be replaced by a state-centered, sociologically informed political history. And even in the Beardian years, historians had made outstanding contributions to social science that would serve as models for the future.15 In addition, there were seminar masters in the graduate faculties of the time who believed that the related disciplines could assist their graduate students in understanding and explaining the past. Professors like Benson’s doctoral advisor at Cornell, Paul W. Gates, urged their advisees to avoid internal doctoral minors and to take minors instead in economics, agricultural economics, and anthropology. Such work convinced many graduate students that, if there were not historical laws, there certainly were exciting behavioral hypotheses and methods of validation available in such disciplines that would allow them to proceed beyond the qualified role and indeterminate judgments prescribed for them in SSRC Bulletin 54 (1946). There was irony involved in this development because the social scientists, for the most part, were notably ahistorical or even antihistorical in outlook, equating historical,analysis with the past failures of social science to deliver an adequate science of society. But not all social scientists of the 1950s were hostile to history, even if they deplored its current softness; some were prepared to give history graduate students the ideas and methods necessary to change their discipline. The trend toward social science history also benefited from the great surge in graduate enrollments and the expansion of graduate education during the 1960s. Thousands of young scholars sought viable thesis and

History as Social Science

71

dissertation topics, and the novelty of the new histories attracted many. It was a time also when college and university history departments were expanding; additional budget lines could be conveniently assigned to the new specialties. Some discussions of paradigm shift give the impression that the process takes place within closed systems, that the actors involved move in a world of free or even equal choices. Actually, the material context is highly important, and external facilitative agencies may greatly influence the attractiveness of particular research areas and problems and may significantly shape disciplinary agendas. (“I am working in demography now,” a respected social science historian remarked recently. “That’s where the big bucks are in research these days.”) Throughout the first several decades of the twentieth century, history as a discipline benefited greatly from the fostering nurture of the Carnegie Institution. J. Franklin Jameson headed its Department of Historical Research and was history’s point man in Washington, guiding various bibliographical and research projects and providing essential continuity in the governance of the American Historical Association. This happy alliance of foundation and discipline ended in 1928. But within this general perspective, the relations between the Social Science Research Council and the discipline of history have been of considerable importance. Thus far, we do not have a full-scale analysis of the contributions of the SSRC to the development of historical studies in the United States, but that agency contributed greatly to the development of the social science sector of American history. It was, in part, SSRC funding that enabled a committee of the American Historical Association of the early 1930s to prepare a major report on appropriate directions that historical research should take. Historical Scholarship in America: Needs and Opportunities is still a thoughtproducing document. Many more positive contributions were to flow from the relationship between the SSRC and scholars in history. But historians were not always content in the camp of social science. Early in its history, the SSRC established a program of fellowship assistance and grants-in-aid for individual scholars within the social sciences. And some historians believed that worthy applicants failed in competition for such funding because their projects were “narrow, technically historical” rather than “interdisciplinary social-science oriented” (Nichols, 1968, pp. 120-121). In 1934, the executive secretary of the AHA proposed to argue this position aggressively, but the historian members of the SSRC board of directors worked to defuse the issue. Definitions of fair share in such situations are subjective, but historians of many different types and specialties

72

Bogue

benefited from the SSRC fellowship programs for both senior and junior scholars until the programs expired during the late 1960s and early 1970s (see Sibley, 1974; Karl, 1974). A historian, Roy F, Nichols, headed the council’s Committee on the Control of Social Data, which surveyed the appropriate holdings of the newly opened National Archives. A conference of 1937 under council aegis examined the utility of local history, and its recommendations led to the preparation of an authoritative publication by Donald D. Parker, Local History, How to Gather It, Write It, and Publish It (1944). During the late 1930s the SSRC sponsored a series of conferences in which major contributions to social science writing were discussed, including the thesis-oriented historical monograph The Great Plains (1931), written by Walter P. Webb. The conference organizers requested another historian, Fred A. Shannon (1940), to prepare a critique of Webb’s book in order to focus the discussion, and the resulting council Bulletin is still cited occasionally as an illustration of rigorous textual criticism. Attracting much more discussion was the series of monographs on historical research methods prepared under the direction of council committees on historiography and, subsequently, historical analysis. The first, already mentioned, was Council Bulletin 54: Theory and Practice in Historical Study, which appeared in 1946. Eight years later, another council committee on historiography produced Bulletin 64: The Social Sciences in Historical Study (1954), noting that a “negative emphasis had seemed to predominate” in the earlier publication and suggesting that its monograph was to be a “more positive type of report.” This committee, headed by Thomas C. Cochran, was not without its own doubts and qualifications, but in general, its report reflected confidence that there were approaches and methods current in the social sciences that historians should understand and could perhaps adapt to their own uses. The committee members stressed in conclusion that “theory and practice in history as social science are still in an experimental, exploratory stage. The nonhistorical social sciences offer, to those willing to learn, a wide range of concepts, hypotheses, and theories, many of them firmly based on careful empirical research. But to the historian this represents not a collection of finished products ready for use but a source of raw material which demands the development of proper techniques of exploitation before it can be made to produce something useful and valuable.” Bulletin 64 was a careful and sympathetic account of current approaches, concepts, and methods in social science that Cochran and his colleagues believed might be used to revitalize history. They did not argue

History as Social Science

73

that history, or a part of it, was a social science but in effect argued that it could be. In 1963, an SSRC Committee on Historical Analysis published Generalization in the Writing of History, a volume that revealed some of the discipline’s ambivalence concerning its role and methods. The chairman and volume editor, Louis Gottschalk, essentially a narrative historian and the author of a much used and conventional manual of historical method, emphasized in his introduction that history displayed “some of the characteristics of science, some of art, and some of philosophy.” He stressed the importance of imagination, although the “historian ought not to be wholly artistic, of course,’’ in bridging the gap between “history-as-actuality” and “knowable history.” In fixing upon the concept of generalization as the major focus of a collection of essays, the group chose a unifying theme that was, as Aydelotte, both committee member and contributor, noted, “slippery and evasive.” Although the contributors were distinguished historians and submitted thoughtful essays, the volume as a completed work seemed to stand outside the main thrust of the behavioral movement in history. The council did not provide funding for its publication. In the process of developing its mission, however, Gottschalk’s committee had approved council funding of several historical conferences, including the Rutgers meeting, which we see today as clearly foreshadowing the emergence of the new political history. There is much other evidence of the SSRC’s supportive posture toward history as social science during the 1950s and 1960s. Many of the young historian behavioralists of the time obtained postdoctoral cross-disciplinary training or research fellowships from the council. The council supported a pilot study on the feasibility of collecting extended time series of local-level electoral data in machine-readable form for general use. An SSRC staff member attended the meetings of the AHA Quantitative Data Committee, and as interest in the use of quantitative data spread, the council contributed, in part, to the support of a seminar during the summer of 1965 on the analysis of electoral and legislative behavior under the auspices of the InterUniversity Consortium and the AHA Quantitative Data Committee. Shortly thereafter, the council joined with the Center for Advanced Study in the Behavioral Sciences to establish the Mathematical Social Sciences Board. That body, in turn, organized a history advisory subcommittee, chaired by Robert W. Fogel, that began an active program of conferencing and publishing historical research, which combined substantive contribution with illustration of useful mathematical and statistical methods. During the late

74

Bogue

1960s, the council was also a joint sponsor of the Survey of the Behavioral and Social Sciences, and its history panel produced History as Social Science (1971) under the leadership of David S. Landes and Charles Tilly, historians who, in contrast with Louis Gottschalk, affirmed that history could be social science. The historian behavioralists of the 1950s and 1960s in the history discipline were uniquely the protCgCs of the Social Science Research Council. Despite the increasingly ahistorical character of the social sciences in these decades, the historian representatives on the SSRC board were apparently sufficiently persuasive and their social science colleagues broad-minded enough to condone the allocation of vital resources to making part of the history discipline, at least, a social science enterprise. That situation changed during the early 1970s. The foundation grants that had for many years enabled the SSRC to maintain fellowship programs of unique value for both mature and novice historians with interdisciplinary interests now were exhausted. The council staff failed to find other funding for such purposes; the agendas of the private foundations had apparently changed. The efforts of a new president to reorient the council’s policies and tap federal funding ended with his resignation. His successor, in turn, sought to move the council in new directions, and these did not include an interest in furthering the cause of behavioral history. Although the historian representatives on the council were allowed to hold a series of planning conferences in the mid1970s, the president resolutely opposed their recommendation that a special committee be established to assist in furthering the agenda of social science history. Key social scientists on the council bowed to the executive will. The SSRC had disowned its children in social science history.16 This is not to say that those children were thrown completely defenseless into a cruel world. At about this time, the American Historical Association upgraded the status of its ad hoc Quantitative Data Committee to that of a standing committee. Members of the ad hoc committee had already taken the initiative to organize the Social Science History Association, whose annual meetings rapidly became a most exciting forum for the presentation of interdisciplinary historical research. Its journal joined other new periodicals as publication outlets for behavioral history (Bogue, 1987). Encouraging also was the fact that the executive director and the council of the Inter-University Consortium for Political and Social (at about this time) Research (ICPSR) continued to be a source of material and moral support for history as social science. As that organization got under way during the early 1960s as a mechanism designed to facilitate the storage and dissemination of data generated in the Survey Research Center’s electoral

History as Social Science

75

polling activity, its agenda lengthened rapidly under the imaginative leadership of Warren E. Miller. Lee Benson was then resident in Ann Arbor and enthused at the possibility of developing a coordinated effort to collect a machine-readable body of electoral and legislative data which would eliminate the need for every researcher to consult and abstract from the original sources. He urged Warren Miller to create a historical division within ICPSR’s archive for this purpose. Miller recognized the merits of the proposal and raised foundation backing in amounts that surprised even him. The precedent was thus established for a supportive consortium posture toward historical data and analysis that has remained to this day, when historical series constitute a very substantial part of the ICPSR archival holdings. Since almost the initial summer institute for historians of 1965, summer courses dealing with the analysis of historical quantitative data have been a feature of the ICPSR summer program. From its establishment during the late 1960s, a consortium’s historical data advisory committee has provided assistance in the identification of appropriate historical data sets. During the formative years of the Social Science History Association, its executive director was also the executive director of the ICPSR.I7 Agencies supportive of scholarly research and somewhat apart from the university and college proper have appeared in various forms. There are private foundations, with their idiosyncratic - and sometimes eccentric agendas. We have broker agencies like the ACLS and the SSRC, whose leaders and staffs try to educate the foundation policymakers into spending their money as the brokers think best; but they sometimes also take the money and scurry to oblige when foundations prove willing to place the administration of programs in their hands. We can also identify bootstrap agencies whose directors try to draw resources from their own clientele. The famous WARF (Wisconsin Alumni Research Foundation) has been a premier bootstrapper, turning the returns from patents emanating from university research back to a university research committee for distribution to other university researchers. In the years after 1945, new federal patrons of research appeared - most notably the National Science Foundation, the National Endowment for the Humanities, and the National Institutes for Mental Health. The facilitative agencies have, to an important degree, influenced the course of social science history, sometimes positively, sometimes negatively. Within this universe, the ICPSR has been a modest hybrid- part bootstrapper, part broker-and always well attuned to its constituency. Of it we can say confidently that its influence on the social science dimension in history has always been positive. l 8

76

Bogue

IV zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Shelves of books and bound sets of periodicals that begin in the 1960s or the 1970s, changes in the graduate history curriculum, the meetings of scholarly groups that did not exist in the 1940s-all suggest that the renewed liaison with social science in the 1950s and 1960s had a considerable impact upon the discipline of history. A second generation of social science historians has emerged, who resolutely advocate a rigorously analytical history, as in the methodological contributions of J. Morgan Kousser (1980, 1984). But the souring of the late 1960s and the academic depression of the 1970s were inhibiting. Members of the profession who maintained their dedication to traditional narrative history continued to find followers. Some of them maintained that the foundations had misguidedly and disproportionately stimulated the social science element in history (see Hamerow, 1987). There was backlash, to some degree, within the movement as well from those who found middle-range hypotheses too confining and quantitative data analysis too unrewarding. In social history, and to some degree in political history as well, some scholars declared their commitment to research designs derived from anthropology or Marxism in preference to rigorous quantitative analyses on the models of political science or sociology. In 1984, the editor of the journal Historical Methods devoted two issues to papers delivered at a recent conference on “Statistics, Epistemology, and History.” Although the symposium had not been organized ostensibly to evaluate the success and status of the new histories, some participants considered such issues, and their evaluations were mixed. Eric H. Monkkonen (1984) forthrightly addressed the critics who had charged that quantitative history had not lived up to expectations, was inaccessible, and had failed “to change our understanding of the past ,” He concluded that the charges were “groundless.” The challenge of the quantitative historians was to produce research “which shows us how to deal with social problems in their historical context” (p. 92). They had done so. Social science history could make a substantial contribution to the making of wise public policy. Other participants as well regarded the development of social science history with satisfaction (see Jensen, 1984, p. 117). Not all of the contributors reacted so positively. Nancy Fitch (1984) belabored the behavioral historians for inappropriate or alchemical methods and myth-making propensities. Jerome M. Clubb and Maris A. Vinovskis (1984, p. 261) deplored the slowness with which quantitative analytical offerings had spread within the history curricula. Most shocking of all the symposium papers was that of Lee Benson (1984). During the 1950s and early 1960s, none had surpassed him in scorn for the methodological sloppi-

History as Social Science

77

ness of conventional history, in enthusiasm for a behavioral approach to the discipline, and in finding the levers of power necessary to implement and support far-flung organizational activity to facilitate change in the discipline. Now, in the 1980s, social science in general and especially in history was still, he believed, in a low and unsatisfactory state of development. Earlier conceptions of science and social science had been badly flawed. Particularly, Benson faulted the failure of social scientists, including himself, to recognize the pervasive and potentially crippling results of accepting versions of the mistransference fallacy - “the illegitimate transformation of similes into identities” (p. 118). The methods used by social science historians, he went on to argue, were frequently inappropriate because they had been designed for other purposes in the natural and physical sciences and could not be applied with any confidence to the data and problems of social science or social science history. Benson confessed that even his own tremendously influential book The Concept of Jacksonian Democracy (1961) had been flawed by his failure to recognize the mistransference fallacy. He was not prepared to surrender, however, or to retreat to old ways of doing history. “Social scientists,” Benson (1984) maintained, “need to be very wary of methodological strategies and criteria borrowed from the natural sciences. Complexity, not simplicity, is the human condition. Methods that require and/or exalt parsimony tend strongly to caricature social reality” (pp. 128-129). So what do we conclude from this account? The last generation has seen a strong social science element within the discipline of history. And the impact of its presence has not been confined to its practicing members. A surprising amount of historical literature that seemingly represents another historical tradition is found, on examination, to draw significantly upon the work of social science historians. Conversely, if we keep a policy focus in mind, well-done history of a quite traditional sort may be invaluable. No conscientious policymaker in the area of natural resources management can ignore Paul W. Gates’s tour de force of the late 1960s, A History of American Public Land Law Development (1968). Various conventional definitions that are often used by social scientists and historians in drawing definitional boundaries cannot today stand close examination. Social scientists, some say, are primarily concerned with, the further development and testing of a body of deductive theory about society. Historians, they explain, don’t do that. If they present generalizations, these are inductively derived and poor stuff into the bargain. Continuing the indictment, social scientists, it is argued, use their theory to predict social developments and to ground social policy. And historians don’t do that,

Bogue

78

either. The fact is, however, scientists do proceed inductively sometimes, and they often ride hunches. Conversely, social science historians may develop explanatory models deductively. If universal laws of human behavior are beyond them, bounded generalizations frequently are not, and they make them. As to prediction, even the economists, those physicists of society, have not mastered that game, and Monkkonen’s findings about past patterns of crime may be as useful in policy development as some elegant economic models. Indeed the issue of prediction may be little more than a red herring. Donald N. McCloskey (1986, p. 65) recently taunted a panel of eminent economists by arguing that members of their discipline merely tell stories, just as historians do. This line can be taken too far, however. On the one hand, it is a gross oversimplification of what some historians do, and on the other, it downplays the fact that much social science has been more rigorous and theory-oriented than work on comparable subjects in history. We argue, therefore, that the enterprise of the historians who see themselves as being social scientists is much less different from that of their opposite numbers in the related disciplines than either humanistic historians or some social scientists have believed. And it has been an ironic frustration that at the very moment when a more impressive social science presence was developing among historians than ever before, unique institutional impediments were raised against it. None apparently spoke for history as social science when the disciplinary obligations of the National Science Foundation (NSF) were enunciated during the 1950s and 1960s. Those historical scholars with membership cards in that fraternity slipped in behind the sheltering cloaks of more ostentatiously scientific colleagues in the behavioral disciplines. This development is not, perhaps, difficult to explain. The NSF was organized when social science in history was coming to the end of its period of Beardian self-doubt and when the members of the new persuasion just emerging had high hopes but little clout either in Washington or in the executive offices of foundations (see Klausner and Lidz, 1986, pp zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA 186-264). During the 1960s, the U.S. Congress set history within the province of the National Endowment for the Humanities (NEH). As a result, social science historians were placed at the mercy of panels whose historian members considered themselves humanists, and who voted humanities up and social science down when evaluating grant applications. The disaster was not completely unmitigated. NSF staffers expressed sympathy on occasion, and their programs or project grants sometimes benefited quantitative historians peripherally. The National Endowment for the Humanities staff has tried sometimes to be broad-minded, as have panel members; both the NEH I

History as Social Science

79

fellowship and the NEH research tools programs have sometimes benefited behavioral historians. But the definitional screening criteria currently being applied by some NEH referees and panels probably would have denied aid to such a landmark institutional study as Gates’s (1968) great history of the development of the public land law system had he turned to them for fellowship support. Occasionally, members of the new persuasion found succor at the National Institutes of Mental Health or the National Institute of Education (now eliminated), but they were few indeed in number. In plain fact, the social science historians were akin to refugees; they had institutional homes, but in the wide world of foundation and government granting agencies, they found none willing to recognize their national origins consistently. Even the Social Science Research Council, as we have seen, tacitly cast them out. During the last generation, only the ICPSRwhether under the leadership of Miller, Richard I. Hofferbert, or Jerome M. Clubb, and despite the changing nature of its council membership-has consistently pursued policies that foster the cause of an improved and more powerful social science history. Now, as social science increasingly emphasizes the development of dynamic models and adopts approaches that utilize both long-run and short-run variables, the basic wisdom of the ICPSR’s commitment to the archiving of historical data series will become steadily clearer. Social science historians urgently need the executives and policy boards of several major granting agencies to recognize that historians range in orientation, objectives, and methods from humanistic to rigorously socialscientific. Foundation executives should understand that the potential contribution of historians to a more powerful array of social sciences justifies special treatment and programs. Interdisciplinary training fellowships at both predoctoral and postdoctoral levels and a program of fellowships in support of policy-oriented historical research would be invaluable. Meanwhile, the administrators of those few programs in which fellowships are dispensed to historians as social scientists should ensure that their history fellows will indeed be sincerely committed to social science. We must admit, however, that the fault for the problems of social science history does not rest completely in our stars, that is, the external context of indifference or hostility within which the historian behavioralists have labored. Much of the problem lies within history itself, that is, in the insistence of some practitioners that everyone in the profession should define their objectives and methods in the same way, that only good narrative history can handle the really important problems, and that social science history exposes its followers to unique conceptual, methodological, and

80

Bogue

source problems. These positions are mischievous and wrong, and the best way to refute them is to produce good social science history. Once these truths are generally recognized inside and outside the historical profession, the task of making part of the history discipline a useful and respected sector of the social sciences will be a good deal easier. Notes 1. For varying points of view see Conkin and Stromberg (1971), Handlin et a]. (1954), Lichtman and French (1978), and the authors included in Vaughn (1985). A more detailed discussion of representative figures appears subsequently in the text. The verses are from Guiterman (1946, p. 110). 2. Ausubel (1950), Higham (1963), Kraus (1953), and Wish (1960) provide general surveys of the modern historical profession. 3. Hart’s successor, Frederick Jackson Turner, had rebelled spectacularly against the dicta of his east coast mentors (see Turner, 1893), and in another two years, the president of the association, Theodore Roosevelt (1913), was to renew an old theme by pleading for history as literary art. 4. Odum’s masters were John William Burgess, Lester Frank Ward, Herbert B. Adams, William Archibald Dunning, Albion Woodbury Small, Franklin Henry Giddings, Thorstein Veblen, Frederick Jackson Turner, and James Harvey Robinson. 5 . Saveth (1964) brings together some of the most notable illustrations of the genre. 6. Taylor (1961) and Gates (1960) provide excellent illustrations of the approach and the high quality of the series. 7. Among others, Davis (1968) and North (1977) provided useful interpretive surveys of the “New Economic” or Cliometric history. For a recent retrospective view, see W. N. Parker (1987). 8. As submitted, the McCormick Report was a remarkable forecast of things to come, but some recommendations were deleted in the publication process. 9. VanderMeer (1978) and Curry and Goodheart (1984) sketch major trends and provide appropriate bibliography. My summation of the rise of the so-called new histories is a reassessment of the developments discussed in Bogue (1983, pp. 51-78, 113-135; 1986). Detailed bibliographical references appear in those publications. 10. See Hays (1960). For Hays’s more recent views, see his “Politics and Social History: Toward a New Synthesis,” in Gardner and Adams (1983, pp. 161-180). For a description and critique of the developments in political science that so influenced the new persuasion in political history see Eulau (1963, 1969). 11. Thernstrom was to cite Curti’s (1959) study of democracy in a frontier Wisconsin county as having influenced his choice of topic and methodology, and Curti has explained that he was, in part, led to undertake that study because of the criticism directed at the Social Science Research Council (1954). 12. Stearns and the other contributors to Gardner and Adams (1983) note major trends in the various subareas and provide excellent bibliographical references. See also Zunz (1985). There has been some tendency to suggest that the new social history in the United States has been essentially an effort to apply the methods of the French Annales school to the study of American topics. This is a very misleading oversimplification. 13. Benson’s methodological contributions of 1957 and 1961 were very important. Berkhofer (1969) produced the new historians’ first manual. Landes and Tilly (1971) and their colleagues of the History Panel of the Behavioral and Social Sciences Survey argued for history as social science. McClelland (1975) and Murphey (1973) laler published important contributions on the use of theory, and Aydelotte (1971) provided a moderate and well-

History as Social Science

14. 15. 16.

17.

18.

81

argued plea for the use of quantitative methods. Dollar and Jensen (1971) and Floud (1973) produced introductions to statistics for historians. Manuals o n the use of computers were provided by Shorter (1971) and Clubb, Austin, and Traugott (1972). Hays and his colleagues at the University of Pittsburgh initiated the periodical now known as Historical Methods: A Journal of Quantitative and Interdisciplinary History, in which the emphasis has been consistently methodological. The Journal of Interdisciplinary History and Social Science History have published both substantive and method-theory articles, as have other social history journals. Bogue, Clubb, and Flanigan made these points concerning political history in 1977, but the argument applied equally well to other provinces of the new histories. Although they did not define themselves as social scientists, both Webb and Malin made important contributions to American social science. Here I reflect my personal observations while a member of the board of the SSRC during the mid-1970s. Although the council would- with some reluctance - continue to administer a foreign area fellowship program, no particular emphasis was placed upon history as social science in that activity. These developments can be followed in the ICPSR. The idea of collecting a body of basic data for general use had long been in Benson’s mind, perhaps since, as a graduate student, he had observed the energetic collecting program under way in Cornell University’s Collection of Regional History. In 1951, he submitted a memorandum urging the collection of basic data relative to economic regionalism to the Committee on Research in American Economic History. While at the Bureau of Applied Social Research during the mid-l950s, he made a similar argument for the collection of basic political data. For interesting recent information on the role of the foundations in the social sciences, see the two reports (1986a, b) listed under U.S. Congress in the bibliography.

References Adams, George B. (1909). History and the philosophy of history. American Historical Review 14:221-36. Ausubel, Herman. (1950). Historians and Their Craft: A Study of the Presidential Addresses of the American Historical Association. New York: Columbia University Press. Aydelotte, William 0. (1971). Quantification in History. Reading, MA: AddisonWesley. Bailyn, Bernard. (1982). The challenge of American historiography. American Historical Review 81: 1-24. Barnes, Harry E. (1927). Recent developments in history. In Hayes, Recent Developments in the Social Sciences, pp. 328-415. Barzun, Jacques. (1974). CIio and the Doctors: Psycho-History, Quanto-History and History. Chicago: University of Chicago Press. Beale, Howard K., et al. (1954). Charles A . Beard: A n Appraisal. Lexington: University of Kentucky Press. Beard, Charles A. (1935). A n Economic Interpretation of the Constitution. New York: Macmillan. (Originally published, 1913.) Beard, Charles A . (1926). Time, technology, and the creative spirit in political science. American Political Science Review 21: 1-11. Beard, Charles A . (1932). A Charter f o r the Social Sciences in the Schools. Part I: Report of the Commission on the Social Studies. New York: Charles Scribner’s Sons.

82

Bogue

Beard, Charles A. (1934a). Written history as an act of faith. American Historical Review 39:219-231. Beard, Charles A . (1934b). The Nature of the Social Sciences in Relation to Objectives of Instruction. Part VII: Report of the Commission on the Social Studies, Am. Historical Association. Beard, Charles A., and Beard, Mary R. (1927). The Rise of American Civilization. New York: Macmillan, 2 vols. Benson, Lee. (1957). Research problems in American political historiography. In Mirra Komarovsky (ed.), Common Frontiers of the Social Sciences. Glencoe, IL: Free Press. Benson, Lee. (1960). Turner and Beard: American Historical Writing Reconsidered. Glencoe, IL: Free Press. Benson, Lee. (1961). The Concept of Jacksonian Democracy: New York as a Test Case. Princeton: Princeton University Press. Benson, Lee. (1971). Middle period historiography: What is to be done. In George A. Billias and Gerald N. Grob (eds.), American History: Retrospect and Prospect. New York: Free Press. Benson, Lee. (1984). The mistransference fallacy in explanations of human behavior. Historical Methods 17:118-131. Berkhofer, Robert F., Jr. (1969). A Behavioral Approach to Historical Analysis. New York: Free Press. Billington, Ray A. (1973). Frederick Jackson rimer: Historian, Scholar, Teacher, New York: Oxford University Press. Bogue, Allan G. (1983). Clio and the Bitch Goddess: Quantification zyxwvutsrqponmlkji in American Political History. Beverly Hills, CA: Sage. Bogue, Allan G. (1986). Systematic revisionism and a generation of ferment in American history. Journal of Contemporary History 21:135-162. Bogue, Allan G. (1987). Great expectations and secular depreciation: The first ten years of the Social Science History Association. Social Science History 11: 329-342. Bogue, Allan G., Clubb, Jerome M., and Flanigan, William H . (1977). The new political history. American Behavioral Scientist 21:201-220. Brown, Robert E. (1956). Charles Beard and the Constitution. Princeton: Princeton University Press. Cheyney, Edward P. (1924). Law in history. American Historical Review 29:231-248. Clubb, Jerome M., Austin, Erik W., and Traugott, Michael W. (1972). Computers in History and Political Science. White Plains, N Y IBM Corporation. Clubb, Jerome M., and Vinovskis, Maris A. (1984). Training and retraining in quantitative methods of social research. Historical Methods 17:255-264. Cochran, Thomas C. (1948). The ‘Presidential Synthesis’ in American History. American Historical Review 53:748-759. Conkin, Paul K., and Stromberg, Roland N. (1971). The fferitage and Challenge of History. New York: Dodd, Mead. Curry, Richard O., and Goodheart, Lawrence B. (1984). Encounters with Clio: The evolution of modern American historical writing. OAH Newsletter 12 (May): 28-32. Curti, Merle. (1959). The Making of an American Community: A Case Study of Democracy in a Frontier County. Stanford, CN: Stanford University Press.

History as Social Science

83

Davis, Lance E. (1968). “And It Will Never Be Literature”-The new economic history: a critique. Explorations in Economic History 6:75-92. Davis, Lance E., et al. (1972). American Economic Growth: A n Economist’s History of the United States. New York: Harper & Row. Dollar, Charles M., and Jensen, Richard J. (1971). Historian’s Guide to Statistics: Quantitative Analysis and Historical Research. New York: Holt, Rinehart & Winston. Eulau, Heinz. (1963). The Behavioral Persuasion in Politics. New York: Random House. Eulau, Heinz, ed. (1969). Behavioralism in Political Science. New York: Atherton. Fitch, Nancy. (1984). Statistical fantasies and historical facts: History in crisis and its methodological implications. Historical Methods 17:239-254. Floud, Roderick. (1973). A n Introduction to Quantitative Methods for Historians. Princeton: Princeton University Press. Fogel, Robert W. (1964). Railroads and American Economic Growth: Essays in American Econometric History. Baltimore: Johns Hopkins University Press. Fogel, Robert W., and Engerman, Stanley I,. (1971). The Reinterpretation of American Economic History. New York: Harper & Row. Fogel, Robert W., and Engerman, Stanley L. (1974). Time on the Cross. Boston: Little Brown, 2 vols. Gardner, James B., and Adams, George Rollie, eds. (1983). Ordinary People and Everyday Life: Perspectives on the New Social History. Nashville: American Association for State and Local History. Gates, Paul W. (1960). The Farmer’s Age: Agriculture, 2815-1860. New York: Holt, Rinehart & Winston. Gates, Paul W. (1968). A History of Public Land Law Development. Washington: U.S. Government Printing Office. Genovese, Eugene D., and Fox-Genovese, Elizabeth. (1976). The political crisis of social history: A Marxian perspective. Journal of Social History 10:203-220. Gottschalk, Louis, ed. (1963). Generalization in the Writing of History: A Report of the Committee on Historical Analysis of the Social Science Research Council. Chicago: University of Chicago Press. Greven, Philip J. (1970). Four Generations: Population, Land, and Family in Colonial Andover, Massachusetts. Ithaca, NY Cornell University Press. Guiterman, Arthur. (1946). “Edinburgh.” In David McCord, The Pocket Book of Humorous Verse. New York: Pocket Books. (Originally published, 1939). Gutting, Gary, ed. (1980). Paradigms and Revolutions: Appraisals and Applications of Thomas Kuhn’s Philosophy of Science. Notre Dame, IN: University of Notre Dame Press. Hamerow, Theodore S. (1987). Reflections on History and Historians. Madison: University of Wisconsin Press. Handlin, Oscar, et al. (1954). Harvard Guide to American History. Cambridge: Harvard University Press. Hart, Albert B. (1910). Imagination in history. American Historical Review 15:232-233. Hayes, Edward C. (1926). Albion Woodbury Small. In Odum, American Masters of Social Science, pp. 155-156.

84

Bogue

Hayes, Edward C., ed. (1927). Recent Developments in the Social Sciences. Philadelphia: Lippincott. Hays, Samuel P. (1960). History as human behavior. Iowa Journal of History 58 :193-206. Hays, Samuel P. (1983). Politics and social history: Toward a new synthesis. In Gardner and Adams, Ordinary People and Everyday L f e , pp. 161-179. Higham, John. (1965). History: Professional Scholarship in America. Baltimore: Johns Hopkins University Press. Himmelfarb, Gertrude. (1984). Denigrating the rule of reason: The “new history” goes bottom-up. Harper’s 268. Hofstadter, Richard. (1955). The Age of Reform: From Bryan to RD.R. New York: Knopf. Hollinger, David A. (1973). T. S. Kuhn’s theory of science and its implications for history. American Historical Review 78:370-393. Inter-University Consortium for Political and Social Research. (1962-). Annual Reports. Jensen, Richard. (1969a). History and the political scientist. In Lipsett, Politics and the Social Sciences, pp. 1-28. Jensen, Richard. (1969b). American election analysis. In Lipsett, Politics and the Social Sciences, pp. 226-243. Jensen, Richard. (1984). Six sciences of American politics. Historical Methods 17:108-117. Karl, Barry D. (1974). Charles E. Merriam and the Study of Politics. Chicago: University of Chicago Press. Klausner, Samuel Z., and Lidz, Victor M., eds. (1986). The Nationalization of the Social Sciences. Philadelphia: University of Pennsylvania Press. Kousser, J. Morgan. (1980). History QUASSHed: Quantitative social scientific history in perspective. American Behavioral Scientist 23:885-904. Kousser, J. Morgan. (1984). The Revivalism of Narrative: A Response to Recent Criticisms of Quantitative History,” Social Science History 8:133-149. Kraus, Michael. (1953). The Writing of American History. Norman: University of Oklahoma Press. Kuhn, Thomas S. (1962A970). The Structure of Scientific Revolutions. Chicago: University of Chicago Press. Kuhn, Thomas S. (1977). The Essential Tension: Selected Studies in Scientific Tradition and Change. Chicago: University of Chicago Press. Landes, David S . , and Tilly, Charles, eds. (1971). History as Social Science. Englewood Cliffs, NJ: Prentice-Hall. Lichtman, Allan J., and French, Valerie. (1978). Historians and the Living Past. Arlington Heights, IL.: AHM Publishing. Lipsett, Seymour M., ed. (1969). Politics and the Social Sciences. New York: Oxford University Press. Malin, James C. (1935). The turnover of farm population in Kansas. Kansas Historical Quarterly 4:339-372. Malin, James C. (1948). Certainty and history. In James C. Malin, Essays on Historiography. Ann Arbor, MI: Edwards Brothers. Malin, James C. (1958). The historian and the individual. In Felix Morley (ed.), Essays on Individuality. Philadelphia: IJniversity of Pennsylvania Press.

History as Social Science

85

McClelland, Peter D. (1975). Causal Explanation and Model Building in History, Economics, and the New Economic History, Ithaca, NY: Cornell University Press. McCloskey, Donald N. (1986). Economics as a historical science. In William N. Parker (ed.), Economic History and the Modern Economist. New York: Basil Blackwell. McCormick, Richard P. (1957). Conference on early American political behavior. SSRC Items 11:49-50. McDonald, Forrest. (1958). We the People: The Economic Origins of the Constitution. Chicago: Merriam, Charles E. (1926). Progress in political research. American Political Science Review 2O:l-13. Monkkonen, Eric H. (1984). The challenge of quantitative history. Historical Methods 17:86-94. Murphey, Murray G. (1973). Our Knowledge of the Historical Past. Indianapolis: Bobbs-Merrill. National Bureau of Economic Research. (1960). Trends in the American economy in the nineteenth century. Studies in Income and Wealth, vol. 24. Princeton: Princeton University Press. National Bureau of Economic Research. (1966). Output, employment, and productivity in the United States after 1860. Studies in Income and Wealth, vol. 30. New York: Columbia University Press. Nichols, Roy F. (1968). A Historian’s Progress. New York: Knopf. Nore, Ellen. (1983). Charles A . Beard: A n Intellectual Biography. Carbondale: Southern Illinois University Press. North, Douglass C. (1977). The new economic history after twenty years. American Behavioral Scientist, 21: 187-200. Odum, Howard W., ed. (1927). American Masters of Social Science: A n Approach to the Study of the Social Sciences Through a Neglected Field of Biography. New York: Henry Holt. Parker, Donald D. (1944). Local History, How to Write It, Gather It and Publish It. New York: Social Science Research Council. Parker, William N. (1987). Historical introduction. In Peter Kilby (ed.), Quantity and Quiddity: Essays in U S . Economic History. Middletown, CT: Wesleyan University Press. Robinson, James Harvey. (1912). The New History: Essays Illustrating the Modern Historical Outlook. New York: Macmillan. Roosevelt, Theodore (1913). History as literature. American Historical Review 18: 473-489. Saveth, Edward N. (1964). American History and the Social Sciences. New York: Free Press of Glencoe. Shannon, Fred A. (1940). Appraisal of “The Great Plains”: A Study in Institutions and Environment (Critiques of Research in the Social Sciences, Vol. 3.) New York: Social Science Research Council. Shorter, Edward. (1971). The Historian and the Computer: A Practical Guide. Englewood Cliffs, NJ: Prentice-Hall. Sibley, Elbridge. (1974). Social Science Research Council: The First Fgty Years. New York: Social Science Research Council.

86

Bogue

Social Science Research Council (1946). Bulletin 54. Theory and Practice in Historical Study: A Report of the Committee on Historiography. New York. Social Science Research Council (1954). Bulletin 64. The Social Sciences in Historical Study: A Report of the Committee on Historiography. New York. Stone, Lawrence. (1981). The revival of narrative: Reflections on a new old history. In Lawrence Stone, The Past and the Present. Boston: Routledge & Kegan Paul. (Originally published, 1979.) Taylor, George R. (1961). The Transportation Revolution: 1815-1860. New York: Rinehart. Thernstrom, Stephan. (1964). Progress and Poverty: Social Mobility in a NineteenthCentury City. Cambridge: Harvard University Press. Turner, Frederick J. (1920). The significance of the frontier in American history. In Frederick J. Turner, The Frontier in American History. New York: Henry Holt. (Originally published, 1893). Turner, Frederick J. (1932). The Significance of Sections in American History. New York: Henry Holt. U.S. Congress, House of Representatives, Committee on Science and Technology, 99th Congress, Second Session. (1986a). Research Policies for the Social and Behavioral Sciences: Science Policy Study Background Report No. 6. U.S. Congress, House of Representatives, Committee on Science and Technology, 99th Congress, Second Session. (1986b). The Role of the Behavioral Sciences: Science Policy Study -Hearings, Vol. 1 1. VanderMeer, Philip R. (1978). The new political history: Progress and prospects. Computers and the Humanities 11: 265-278. Vaughn, Stephen. (1985). The Vital Past: Writings on the Uses of History. Athens: University of Georgia Press. Webb, Walter P. (1931). The Great Plains. Boston: Ginn & Company. Wilenz, Sean. (1982). On class and politics in Jacksonian America. Reviews in American History 10:45-63. Wise, Gene. (1973). American Historical Explanations. Homewood, IL: Dorsey Press. Wish, Harvey. (1960). The American Historian: A Social-Intellectual History of the American Past. New York: Oxford University Press. Woodward, C. Vann. (1982). A short history of American history. New York Times Book Review, August 8:3, 14. Zunz, Olivier. (1985). The synthesis of social change: Reflections on American social history. In Olivier Zunz, Reliving the Past: The Worlds of Social History. Chapel Hill: University of North Carolina Press.

Cornment

Time Is on the Historian’s Side Eric Monkkonen

F

or quantitative social science history, Allan Bogue has presented a gloomy and dismal narrative, one with Dickensian twists and turns. In this narrative, two leading young heroes turn into villains of the worst kind, those who recruit their troops, lead them deep into enemy territory, and then in the midst of battle change sides. One of his story’s villains, Charles Beard, author of the justly famous An Economic Interpretation of the Constitution of the United States, abandoned the professional history associations and led the effort in the late 1920s to rid political science of what he called the “incubus” of history. The other young hero, Lee Benson, in the late 1950s and early 1960s helped the history profession toward a rapprochement with its offspring social science children, only to turn on the whole family, blaming on them all a failure of mission, a failure to reform a society sadly in need of reform. With intellectual leaders like these, quantitative social science history needs no enemies. Not only has quantitative history been disavowed by some of its prominent insiders, Professor Bogue shows how it has been denied the fiscal support which would better enable its enterprise, bring to it prestige, and provide for the appropriate training of students. In contrast, he emphasizes how in their successful shedding of history, the other social sciences found national funding homes which left history to become, legally, a humanities. Probably few historians, data archivists, or archivists care very much what label their enterprise is given, yet Bogue’s point is important. For this national funding split has left quantitative historians legally homeless, denied shelter in the humanities because their work uses quantitative data and shunned by their own intellectual progeny, the other social science disciplines, which have only recently escaped from the “nightmare” of historical research. Neither scientists nor humanists, quantitative historians have had 87

88

Mon kkonen

to compete in scholarly worlds where other, seemingly different, goals dominate. That they have done so at all is a tribute to their ability to clothe their real activities with the vocabulary and trappings of alien enterprises. There is only one bright ray of hope in Professor Bogue’s tale of intrigue and woe: the ICPSR. In the context of his larger argument, historians are merely gate crashers in the organization, and we can interpret his account as the biographies of gate crashers and as vignettes of the consequences of accident and contingency. But this one hopeful part of his account raises an obvious question: How have quantitative social science historians managed to continuously cajole this one group into recognizing us? The reason a particular mode of historical research has found a home in the ICPSR has to do with the unusual origins and organizational structure of the ICPSR itself. First, the ICPSR represents a virtual institutional and intellectual miracle, a notion confirmed by Warren Miller’s narrative. Consider, for instance, the vast sums which have gone into other, national, forms of archiving, archiving conducted for arguably similar goals and done with an eye toward posterity and social science research: the presidential libraries, called by some of their users our “national treasures.” Whatever their glories, in them we have a costly effort of information collecting which contrasts dramatically with the ICPSR: they are nationally funded, not driven by the research community as is the ICPSR, and do not strive to be at the advances of human understanding. Rather, they preserve the records of one aspect of our political system, creating the presidential paper analogues to ancient tombs. Perhaps because the ICPSR is an institution whose constituents insist on the advancement of the social sciences, it is far less parochial than the presidential libraries and it continues to serve as a national locus for empirical social research. The presidential libraries, in contrast, while nationally funded, seem to be intensely local monuments, even if the locales are sometimes a bit prickly about accepting such glories. Presidents have partisan supporters who are more than happy to glorify their past leaders and partisan opponents who hope to mummify them. Social science does not have such fervently motivated support. For instance, it is almost impossible to imagine starting, today, an organization to which colleges and universities would contribute actual sums of money for the purpose of archiving the fruits of social science. It is even harder to imagine such an activity daring to propose such an archive holding data of historical importance. Yet that is what the ICPSR has done, and this simple fact alone stands as a lively monument to social research.

Comment

89

Colleges and universities continue to fund this enterprise for intellectual reasons, in order to support and encourage systematic social research, in order to facilitate the empirical investigation of social and political systems across nations, both backward and forward in time. In other words, the serious conduct of social research, as defined by those who are doing it, drives the organization. Quantitative history, then, has found its home in the ICPSR for the best possible reason: because a community of scholars deems the work important. Like so many historical stories which, when examined closely, have no satisfactory conclusion, Professor Bogue’s has a most untidy ending and a highly inconclusive conclusion. It is not therefore a fable, but I wish to treat it like one and draw from it several morals. First, we know that at least some adversity is good for the soul; this has meant that quantitative historians have had to define themselves and their tasks in an environment which has never allowed them to be complacent. Those who have never been rejected become arrogant, a problem unlikely to trouble our field. Second, social science historians enter the final decade of the twentieth century unencumbered by national definitions of who they are and what they do. They are privileged to let ideas and fundamental questions define their topics, their agendas, and their relevant universe of discourse. Third, there is, of course, much to be done to improve historical research and the training of historians in appropriate methods. It is also clear, to me at least, that the nonhistorically oriented social sciences have much to learn from the historians’ approaches, and what we learn from Bogue’s narrative of fits and starts is that we must not be impatient, that the complicated intellectual adventure we are all participating in will take longer than our institutional predecessors ever thought. Professor Blalock says things have not improved very much in sociological training or research in the past thirty-five years. He is very ambitious. They have improved in historical research: we can now recover much more of the historical past than we could have begun to hope for thirty-five years ago. But it is excessive to expect that in a few years we can move ahead on all fronts of human knowledge so as to outdo the previous three millennia. Given the size, complexity, and ambiguity of the task we have set ourselves, we must not get discouraged by slow progress. Every social scientist knows that the scales of X and Y axes on a graph control the visual illusion of the slope-in this case, the progress of socialscientific knowledge. A little more humility and different scales are in order, I believe, in the social sciences. It might help the mood of those who despair at the lack of progress to stay motivated, knowing that we are all part of an

90

Mon kkonen

enormous effort, the payoff from which we will not see soon. Note that this is a caution not to relax, but to be patient. Cooperation and intellectual daring have never been easy. As long as we push at the edges of new forms of knowledge, accumulate the research advances of the past, and support the techniques of the future, we can keep building our social-scientific understanding of the world. Finally, every year longer that the ICPSR continues to archive data means that much more history for the historians to work with: time is on the historians’ side. Quantitative historians have every reason to be optimistic, for we have the ICPSR, a world of data professionally processed and archived, a network of data librarians who are capable of providing information in a form that can be analyzed beyond the wildest dreams of thirty-five years ago. I read the ICPSR “Guide to Resources and Services” much as a catalog of opportunities and potential projects.

Chapter 3

When Social Science Findings Conflict Philip E. Converse

A

half century ago, when the new empirical social science was in its salad days, there was not much doubt as to the general strategy to be pursued when one had questions about the nature of complex social affairs. The thing to do was to carry out a study to learn the answer. Indeed, this was the hallmark of the emergent positivism which was challenging a long tradition of armchair speculation about human affairs. It invited a new breed of social scientists to engage in systematic interaction with the real world, precisely to learn “the answer’’ to the question at hand. The age of innocence lasted longer in some of our social science disciplines than others. But sooner or later, as we have moved from the very first study of a given phenomenon to the second or third studies thereof, we have everywhere learned that no single study produces the answer about anything. What it produces at the very best is one answer, which, if we are not too skeptical, we might be willing to see as a legitimate member of a family of answers to whatever our question is, a family which, in principle, has properties like variance. As relevant studies multiply, the final insult is the discovery that this variance exceeds anything which can be written off as garden-variety noise like sampling variability. Two answers which are highly significant by conventional criteria, yet which are opposite in sign, have a certain humbling effect on all of us and overstimulate those detractors of social science convinced that we can never get any answers straight to begin with. Of course, the original belief in the single answer was naive even in its period. Most social scientists, feeling like poor cousins, have tended to assume that natural scientists come away from reality probes with estimates of important parameters that are invariant to several digits. But this is not true: under many circumstances, such estimates show startling variability, 91

92

Converse

the reasons for which are often quite unclear (see, for example, Turner and Martin, 1984, Vol. 1, pp. 14-17, or the Particle Data Group, 1976). Recognizing the likelihood that our modest methods will reveal only mere families of variant answers, rather than some single and monolithic Truth, is not equivalent to a rejection of the faith that out in some Platonic empyrean, a single Truth exists to be estimated. It is not inconsistent to maintain this faith while coping with the grubby fact thal we see such a Truth only as through a glass darkly, where the glass not only fuzzes our view but can actively distort it. Moreover, all of this was recognized nearly two centuries ago when Parson Bayes proposed a theorem which could be developed into a thinking person’s guide to what should be done about such a murky view of reality: how we should integrate over past experience, including informal impressions as well as the results of more formal trials, experiments, and systematic observation; and more particularly how we should update our view of these outcome probabilities in the face of new information from further trials, experiments, or observation. This austere view of the rational progress of inquiry is somewhat less widely useful than it might appear, for it is rather uncommon that shaggy prior experience can be compellingly summarized in the way the calculus presumes. But as an ideal, the Bayesian view has great appeal, since it not only recognizes the practical variability of messy answers derived from intercourse with the real world but suggests forcefully that there is something forward-looking to be done about it. When we have only two studies of a phenomenon which show sharply discrepant results, we are often treated to titillating journal debates of the general form “Our data are better than yours.” This is a natural consequence of a mind set which assumes that any single study, if done properly, should yield up exactly the right answer, so that if there are two variant answers, at least one of them must simply be wrong. As time wears on, however, and further studies begin to generate a richer sense of a full distribution of outcomes, such early reactions begin to look sophomoric. It is clear that results show some kind of variance. Such variance in results should set the juices flowing for the “scientist as detective.” Instead of arguing, on necessarily vague grounds, that one answer is correct and the others somehow badly flawed, why not adopt a more capacious view of the matter, attempting to account in a more systematic fashion for this unexpected but now self-evident variability in answers? The rest of this chapter assumes that such efforts at a systematic decoding of results is not only a worthwhile activity but a vital one at the current

When Findings Conflict

93

stage of social science. We plan to review developments dedicated to this goal over the past decade that have taken place in quadrants of the social sciences which are remote and unfamiliar to many of us who use the resources of the Inter-University Consortium for Political and Social Research (ICPSR). These developments often proceed under the name meta-analysis or research synthesis (Glass, 1976; Glass, McGaw, and Smith, 1981; Light and Pillemer, 1984). Before we proceed, we should perhaps recognize in passing that there is not full agreement that dissecting the variability in our results is worthwhile. One dissenting position argues that this variability in assessments arises from such a complex welter of interactions between methodological artifacts and nonrobust substantive signals that any effort to decipher these residues will be at least unrewarding and perhaps pointless. This is not an absurd position. Fortunately, however, we do not have to make an either-or judgment, deciding whether these residues are completely meaningful or totally meaningless. As usual, the safe and sane view is that they are likely to lie somewhere between these extremes. Surely, enough positive findings have already emerged under the banner of meta-analysis to give the lie to any argument that these cross-study variations in outcomes have no meaning whatever. On the other hand, we can maintain interest in studying such residues while recognizing that some or even much of the variance will be unrewarding, because it is mere noise or, as may amount to the same thing philosophically, is the joint product of such an enormous manifold of petty perturbations that efforts to decipher it are a losing game. The more important point is that we have a scientific and citizenly obligation to pay more serious attention to these residues of variance than we customarily have. There is something which is patently vapid about designing the lOlst study of a particular phenomenon without a good deal of prefatory attention to why such a study is necessary at all. That is, why have the first 100 studies not already converged upon some satisfying result? To sidestep this question, which is one of several that meta-analysis is dedicated to answer, is, on the face of it, a cost-benefit disaster, given the expensiveness of any serious research in both high-talent time and pure dollars, often public ones. Therefore we have a citizenly obligation to pursue this question with some zeal. The scientific obligation is at least as obvious, if less crass. It is easy to argue that some tactic like meta-analysis is crucial in the cumulativity of all of our social science inquiries. Doing science is a matter of pushing ever upward on the slope toward more powerful and more robust generalizations. Social science has suffered some disillusionment in the past fifteen years as

94

Converse

it has come to realize that where complex social phenomena are concerned, this slope is a good deal steeper than met the eye a generation ago (Fiske and Schweder, 1986). But if the point is to arrive at generalizations which are less and less time-bound, culture-bound, or method-bound, then clearly one tactic that rapidly recommends itself is an exploitation of past investments in empirical inquiry with an eye to seeing why they are not more robust than our experience with them suggests. This is exactly what meta-analysis is about. The Development of Meta-Analytic Concerns

Let us begin our discussion by reviewing the more specific context in which serious ideas about meta-analysis began to take shape historically. It would not have been hard to predict where, across the broad waterfront of the social sciences, meta-analytic ideas were most likely to spring up. They should emerge where the need was greatest. Unpacked, this means where the number of studies aimed at the same target is ballooning most rapidly, with discrepant results. We might also have predicted that concern for meta-analysis would arise in methodological contexts where a discrepancy of results was least to be expected but was found to occur nonetheless, as well as where the achievement of considerable precision in answers had special urgency by whatever external criteria. Such hindsight predicts as well as hindsight usually does. The epicenter of meta-analytic thinking has been in the realm of medicine, with special roots where controlled studies are being carried out to ascertain human responses to alternative medical treatments, such as (most obviously) new forms of potential medication and their dosage ranges, but also a wide variety of more complex “treatments,” such as walk-in operations versus hospital stays and the like. Many of the major medical innovations spawn substantial numbers of controlled experiments; and it is clear that getting answers which are reliable within pretty small tolerances is, in this context, uncommonly urgent - indeed, sometimes a matter of life and death. Finally, since the textbooks tell us that the preferred road to robust results which are causally incisive is the controlled experiment with randomized assignment to test and control groups and other features like the “double-blind design,” it is at least interesting, if not downright depressing, that such standardized probes of reality can still return conflicting results. Being the more unexpected, these discrepancies plead the more eloquently for explanation. Of course, this medical context is remote from the concerns of many of our readers here, and some may not code it as social science at all. There are secondary fountains of meta-analysis which are a little more recognizable to

When Findings Conflict

95

us, such as the context of educational research. Here again, it is often true that numerous studies are brought to bear on the evaluation of a limited number of educational “treatments,” thereby generating a keener feeling of discrepant results than in many other areas. Similarly, some early enthusiasm for meta-analytic thinking has escaped from the experimental context entirely, invading areas of naturalistic observation where random assignment is impossible. Even here, however, the context of treatment and the assessment of effect is by far the most common, with studies of such phenomena as recidivism under competing arrangements for prisoner parole and release, or differential rates of hiring in response to alternative job retraining programs. The Two Research Cultures

The two great cultures of empirical research are, of course, the experiment and naturalistic observation. In the case of the social sciences, most substantive concerns can be addressed only poorly, if at all, by the experimental method. Our raw materials, therefore, tend to be bodies of systematically collected, large-scale observations on historically significant populations of the type which has been assembled with such care in the archives of the ICPSR. In this research culture, the natural-science analogues are less physics and chemistry, and more astronomy, geology, and meteorology - the equally observational sciences. Within the context of observational research culture, conflicting results seem to be attributed a slightly different flavor than in the case of experimental research. Usually, for example, there are some ready explanations to allay concern. Sometimes such conflicts can be written off as being due to differences in cultural meanings or in the subpopulation examined. If results have been collected from arguably the same culture and population, it is rare that they have been collected at exactly the same time, and perhaps results differ because “things have really changed.” We are, it seems to me, rather more glib and casual about these discrepancies than we should be; and of course, an emphasis on meta-analytic thinking obliges us to be more attentive to and systematic about them than is customary. But does such thinking even have the same place in our research culture as it obviously must have within a more experimental tradition? I think it does, although we shall see that certain allowances must be made as we translate back and forth between the two cultures. It is of some interest in this regard that the last few decades of empirical research in both traditions have tended to diminish somewhat the sense of distance that may be felt between them. Once upon a time, in some experi-

96

Converse

mental areas like parts of psychology, there was a conviction on the part of many investigators that the only properties and phenomena worthy of their time were the human universals. Once you are dealing with human universals, examining any one human being is equivalent to examining any human being. Historicity of effects poses no problem. Sampling strategies and the other imperatives of external validity are thereby made irrelevant. But how was it known that the phenomena being observed were, in fact, universal? Sad to say, mainly by little more than assertion and appeals to common sense. This is not the kind of reasoning that remains overpowering as the discovery is progressively made that experimental results seem to wander around in a distressing degree with replications over varying times, places, and subpopulations. Discoveries of this kind have created a substantial wave of disillusionment and have produced a deflating sense that perhaps henceforth these experimental probes of alleged universals should be carefully subscripted by time, place, and subpopulation, and stored up for appropriate meta-analyses. None of these remarks are intended to imply that the classic logical differences between controlled experiments and uncontrolled observation are losing their force. That, of course, is not true. But it is true to observe that, in a degree that was much less evident a generation ago, all of usobservers and experimentalists alike- share some of the same curses of inquiry; and these are curses which meta-analysis purports to address. An Issue of Uniqueness

Before turning to some of the specific contributions which I think the meta-analytic “movement” can make to our more observational work, 1 would like to note that some colleagues with whom I talk express antipathy to this thrust which ranges from stony indifference to derision. Perhaps the most frequent complaint is that “meta-analytic thinking” is a ridiculously pretentious label for what should be, and usually is, any serious scientist’s first reflex when confronted with results that conflict. There is a good deal of truth to this complaint. As we are assaulted by independent results that should converge but do not, we all get a little “meta-analytic” in our minds, groping for unmeasured variables that might have mattered in one study but not the other, or seeking to locate differences in procedure that will help to relieve our feelings of dissonance. We may even be more strenuously meta-analytic without knowing it. In the spring of 1987, I fell to talking with my colleague Leslie Kish about meta-analysis, and he recounted to me that in one section of his new book on research design (Kish, 1987), he had worked out some optimal methods for pooling results

When Findings Conflict

97

across independent studies. The book had been batched off and returned in galleys before he realized that this passage about the niceties of pooling across studies was a very central part of the statistical end of meta-analysis. He was able to make appropriate obeisances to the new vocabulary, although he had realized the importance of the topic and had worked out the details without knowing that there was such a rubric as meta-analysis. This is not an uncommon reaction. Like the joyous discovery by Molikre’s M. Jourdain that he had been talking prose all his life, all of us have been doing meta-analytic thought as well. I do not think that for the most part, self-conscious practitioners of this new art make claims of novelty which are overblown and offensive, and my own descriptions can at many points be very brief because much is already familiar to all of us. What is significantly new, however, is the strenuousness, rigor, and discipline of the meta-analytic inspiration. Most of us who see a conflict in results do our “meta-analyses” by muttering “That’s probably because . . .” and passing on. Real meta-analysis requires a good deal more labor, and of the systematic sort we call science rather than guesswork. Things are not left at “probably . . .”; instead, the hypothesis ginned up to relieve our dissonance must be subjected to much more exacting tests, preferably by assembling a more extensive array of relevant studies. Indeed, there is developing something of a canon of good practice as to the conduct of metaanalyses which has some length and bite (cf. Sacks et al., 1987). Parts of this canon are geared to the experimental culture and do not transport well to observational research. Some parts, however, are well worth our attention. Meta-Analysis and Differences in the Two Cultures

While meta-analytic thinking has historically been centered in the experimental culture, it would be incorrect to imagine that it has been somehow confined there. Associated technologies have been evolving to help with the growing number of instances in which the same reasoning is applied to observational data. However, it is useful didactically to treat meta-analysis in the form it tends to take within the experimental culture, for purposes of comparing its potentials with those to be found in naturalistic observation. There are, of course, a considerable number of differences between these two research cultures which have a bearing on the strength or incisiveness of the meta-analyses which can be carried out. We shall focus here on two which seem most important. It happens that one difference favors metaanalyses for the experimental culture, whereas the other favors observational data. In a nutshell, the experimental culture seems to provide rather more

98

Converse

numerous studies on the same topic than are customarily found for observational studies, a source of considerable strength. On the other hand, observational studies can profit from the fact that they are usually much richer informationally than are experimental studies. Let us examine these differences more closely. The Study Base of Meta-Analyses

Some of the first thrusts toward more careful meta-analysis seem to have been touched off in the early 1970s because of exasperation with the uninformative nature of many research reviews published in areas where a great deal of experimentation had taken place. The general observation that Studies A , B, C, and D had shown one direction of result, but Studies E, F, and G had yielded disconfirmations, did not seem very illuminating as a summary of the state of understanding. To be at all edifying, a truly good reviewer must look more closely at the competing studies, differentiate them by quality, inspect what detailed procedures have been used, and the like-in short, do a formal meta-analysis. It should be clear, of course, that what the meta-analyst does is secondary analysis of other people’s primary research, in very much the same sense that the consortium archives are assembled to invite secondary analysis of observational data. But it should also be pointed out that the nature of the meta-analyst’s common access to the research materials is very different. Users of the archive have access to the original investigator’s raw data. Indeed, they may access a data set and massage it strenuously without ever bothering to read whatever research reports the original investigator turned out. The meta-analyst’s contact with the investigation being reviewed is typically limited to whatever gets recounted in the research report. This is, of course, a major limitation, as we shall see, and one many meta-analysts would like to see broken down. But there are rather good reasons that it has not been the norm to archive experimental data. Chiefly, the issue is one of data bulk. The great glory of the experiment is that it is a very sharply honed and focused probe of nature, to be deployed when vague exploration is over and alternatives have been winnowed down to a very few. This means that the data actually to be collected are not at all profuse. Moreover, because of research traditions and the general nature of these experimental studies, they usually involve an N of subjects which, from the perspective of observational studies, seems very sparse. Some studies may report from as few as ten or twenty subjects; studies with more than 100 subjects are usually considered large.

When Findings Conflict

99

Indeed, the completed data for a very elegant experiment in a true “critical test” mode, conducted over thirty subjects randomly split between treatment and controls, might actually consist of little more than thirty numbers representing observations for the dependent variable. These observations can even be listed in a small research report, and their aggregate moments of distribution can be marvellously miniaturized. To be sure, many experiments are more complicated, with multiple treatments and even multiple dependent variables. To be sure, from time to time squabbles break out over whether raw experimental data have been competently analyzed in the parent research report, thereby raising the question of outside access to the raw materials. To be sure, the lack of a norm for archiving raw materials in the experimental culture generates other dysfunctions. It might have been healthier for science, for example, had Cyril Burt been expected to put his raw observational data into a public archive just as soon as he had made them up. But these are quibbles; and the bare fact of the matter is that there is little sense of a need to archive such data because the good experiment is explicitly tailored to answer a single question or a tight family of such questions. And this much answer-which is the total fruit of the enterprise - by traditional assumptions can be exhaustively reproduced in the research report, even a very short one. This is a totally different world, of course, from the typical observational data set in the consortium archives containing 600,000 observations over 400 variables on a sample of 1,500 respondents. The bottom line of all of this is, of course, that with the research report being the chief point of access for the normal meta-analysis in the experimental culture, the first step in such work is pure library research, a sifting of journal articles and other sources to see what “relevant” studies have been reported. And while meta-analysis has developed sophisticated means of pooling differential N’s of individual subjects across a set of studies, the most common N o f reference is typically the Nof relevant studies canvassed. How does the meta-analyst know just what studies are and are not relevant? Obviously, there are strict and loose constructions as to similarity between studies, and how demanding the investigator is about the similarity criterion can often change the N of studies to be meta-analyzed by a considerable factor. Moreover, it is not necessarily true that multiple studies which purport to be exact replications of one another make up an ideal treasure trove of past work. A moment’s thought about the beauties of the “multitrait, multimethod” approach popularized by Campbell and Fiske (1959)

100 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Converse

should suffice to convince us that meta-analysts, while needing studies aimed at the same target, take delight in finding studies which approach the same target but from the vantage point of very different procedures. In point of fact, the question as to the criteria for the inclusion of studies in the pool for meta-analysis turns out to be one of the most nettlesome subjects to be encountered, and one we shall largely sidestep here. Canons of acceptable practice for such inclusion are still being developed (see Sacks et al., 1987). Once the pool is defined, however, the analyst continues with the study as the primary unit, in effect filling out coding sheets with various attributes of the study as an aggregate process. If such studies in the experimental tradition are limited in their subjects and in their variables, they surely are not limited in their number, and this, of course, is a source of power for meta-analytic inferences. One scholar, setting out to do a research synthesis of studies on the effects of treatment for juvenile delinquency, located some 7,300 bibliographic items involving research on the topic. A second winnowing still left about 2,100 reports of primary research that seemed potentially eligible for an actual metaanalysis. A third winnowing that discarded reports containing inadequate descriptions of the research involved was still expected to leave 700-900 independent studies which could meaningfully contribute to the more exacting synthesis of results.' This instance is, unquestionably, an outlier to the high side in the annals of meta-analysis. Nevertheless, analyses over 200 or 300 studies are not uncommon, and some seasoned hands propose that there is not much point in organizing a full meta-analysis if there are fewer than thirty relevant independent studies available. Numbers like this are, of course, rather mind-boggling for scholars from the observational tradition; and it is possible to argue that the observational data sets for most topics remain too few for meta-analytic lore to be of great use. This risks, however, being a rather short-sighted view in several senses. For one thing, it would not do to underestimate the number of observational studies that are available in the archives for many of the more popular research areas. Certainly, the separate studies of elections, for example, must necessarily number in the scores and perhaps the low hundreds. One might remonstrate that we arrive at numbers this high only by including an amazing variety of studies from different times and places, as well as studies of vastly different quality, ranging from the careful sampling and the intensive interviewing of the large National Election Studies at one extreme and the

When Findings Conflict

101 zyxwvutsrqp

hasty overnight media polls that sample with high levels of substitution at the other extreme. While this may seem an uninstructive hodge-podge, it is not entirely different from the opportunistic mix of studies which permits meta-analysts in the experimental culture to amass their impressive study totals. Moreover, some of the variation which creates the sense of hodge-podge is positively welcomed. Thus, for example, it is seen as highly useful to identify studies of clearly differential quality, such as those experiments with randomized assignment to treatment and control groups as opposed to those lacking it. For many purposes, of course, one would want to restrict the pool of studies to be meta-analyzed to some subset more homogeneous than the set of all election studies. But even here, the number of studies need not be embarrassingly small. For example, Kiewiet and Rivers (1985) carried out what amounts to a kind of meta-analysis of poll results bearing on the popularity of presidential candidates that were published during the 1984 campaign. Counting the repeated polls carried out by each of about ten major national firms, they were able to deal with results from about 100 studies and, among other things, arrived at rather telling evidence of biases in estimate which persistently characterized the various polling houses. Thus observational work has, in some areas at least, generated a larger quantity of studies than meets the eye, and this is especially true if work of poor quality is as welcome as that tuned to higher standards. At the same time, it can be argued that the pressure toward a substantial number of studies, such as an informal minimum of thirty, is rather less clear for observational studies of the kind housed in the ICPSR archives than it would be for studies in the experimental tradition. This is so because, as we have already noted, if experimental studies are typically more numerous than one expects of observational studies, they are also usually smaller and hence much less rich in information per study. Contrasts in Information Content p e r Study

We have already described several types of informational richness which distinguish most archived observational studies from the normal run of experimental studies which figure in formal meta-analyses. Thus such observational studies, being less narrowly focused and more exploratory, usually register a much larger number of variables for each subject - probably, on the average, an order of magnitude more. Under some circumstances, this proliferation of variables may not offset a limitation on study numbers. In the degree that a meta-analysis focuses

102

Converse

upon study-level characteristics, such as whether the interviewing was carried out by telephone or in person or what the aggregated means were of the various properties of each study sample, inferences from the meta-analysis might succumb to the kind of difficulty routinely encountered by scholars who deal with parameters characterizing nations. Many national-level parameters are available, but the number of nations in the world is so limited that there is a problem of degrees of freedom: any national value on a dependent variable can be completely “explained,” albeit in a vacuous way, by unique combinations of the vast surplus of independent variables that may be compiled. However, most attention will be given to variation for which the individuals rather than the studies are the appropriate unit of analysis. And here, of course, it is important that archived observational studies usually draw data from more subjects-again, typically an order of magnitude or two more- than customarily figure in experimental studies. It is important as well that subjects in these observational studies tend to be much more heterogeneous in many demographic respects than experimental studies usually recruit. Because one of the main virtues extolled for meta-analyses in the experimental tradition is the fact that pooling across studies increases the statistical power for subgroup analysis (Sacks et al., 1987, p. 450), it can be seen that the larger N’s and sharper natural heterogeneity encountered in most observational studies constitute a direct offset to the need to muster a very large number of independent studies. Indeed, some of this richness is already being exploited in some “pooled cross-section” analysis designs familiar from observational work, strategies which are, of course, entirely meta-analytic in their spirit. Archived observational studies are richer in information in still another sense, which can be helpful even where study numbers are limited and studylevel characteristics are used. The difference here arises from the fact that meta-analysis in the experimental culture does not get closer to the actual research process than the published research report. Certain design specifications are so central to any assessment-case numbers form a good instance - that manuscripts cannot pass peer review without including them. However, this consensual core for study description is quite small, and the meta-analyst is often vitally interested in design variations which some research reports bother to mention and other reports do not. Thus the code sheet for studies either is pretty brief (being limited to a few core parameters) or, if more interesting, has expanded in such a way that data are completely

When Findings Conflict

103

missing for some studies on some of the most interesting diagnostic variables. Such batches of missing data not only reduce significantly the effective N of studies in the pool but also can obscure the appropriate N of individual subjects in various key comparisons. It is obviously possible for the meta-analyst to try to contact the original authors for such fugitive information, and upon occasion, such special efforts may be carried out in crucial instances. But since thirty studies of almost anything will be dispersed over a dozen years and usually much longer, it is not practical to get beyond the research report in many instances: rates of nonresponse to inquiries routinely run out to half or three-quarters of requests. Archived observational data are quite another matter. For one thing, some characteristics of study procedure that may not break the surface of its research reports are explicit or can be reliably inferred once the raw data can be examined. These include some “cover-sheet” or other administrative variables rarely exploited in reported analyses. More important still are the extensive study descriptions which are compiled for materials given top-class archiving and which are an integral part of the documentation routinely disseminated to users. These descriptions cover a good deal of ground of the kind that journal editors like to abbreviate rather than demote to appendices. Moreover, they represent kinds of information that investigators donating major observational studies to the archive feel obliged to report; and if they do not, they are likely to be queried for missing specifications by the archive staff. Once we are past a period of catch-up with ancient studies, this querying process has been taking place while the original investigator remains readily locatable and still remembers some of the details of the study procedure which are nowhere written down or inferrable. Thus a grid of much more detailed and reliable procedural documentation is preserved for observational studies than can be counted on where experimental studies are concerned. Since one of the pressures toward setting a high minimum requirement for study numbers in the experimental case is the likelihood that substantial missing data may be encountered for some of the most interesting discriminanda between studies, it is of some value that such overcompensation is much less likely to be necessary in the observational case. In sum, for a variety of interlocking reasons, all of which are encompassed by the phrase “greater informational richness,” it seems likely that meta-analyses on observational data can survive a somewhat smaller number of studies in the pool to be analyzed.

104 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Converse

The Generic Progress of a Meta-Analysis

With these key differences between the two research traditions in mind, we can turn to a brief description of the stages through which a normal meta-analysis might proceed. The fact that “speaking prose” is customary for all of us means that we can indeed abbreviate this treatment, since much of what would be done is quite familiar. We can focus, however, upon some of the growing lore of meta-analysis that systematizes beyond the obvious. One common narrow purpose of a meta-analysis is to arrive at the most defensible aggregated number, frequently the size of some treatment effect, given a set of studies which estimate the effect with somewhat different results. A broader purpose, and the one which most interests us here, is to assess the role of various predictors in accounting for the variance in any parameter estimate across a portfolio of relevant studies. In both modes, the process must start with some decisions as to what basic pool of studies is to be included in the meta-analysis. Experience has shown these decisions to be less obvious than it might appear, and it has driven home the point that under some circumstances, the nature of these decisions actually shapes, in distressing degree, the conclusions which are reached. Therefore the canons of good practice require a very extended search for potential studies, as well as a careful accounting of the reasons that some of them may be dropped from consideration. Moreover, it is obviously desirable, if at times difficult, to avoid taking account of the final results of a study in forming decisions as to whether or not the study should be included in the pool to be meta-analyzed. To avoid bias, such decisions should depend on the relevance of goals and procedures, rather than a peek at the outcomes. This is merely one of several points at which formal meta-analysis insists on a discipline of replicability which has traditionally been lacking in efforts to synthesize research literatures to summarize the “state of the art.” Once a pool of relevant studies has been defined with clearly replicable criteria, a second stage of preparation involves the coding of study-level attributes. Some of these attributes, such as case numbers or the presence of random assignment to control groups, are objective and unequivocal. Others, however, often including some of the most interesting characteristics diagnostically, may leave more latitude to reader interpretation. Such attributes must be evaluated by more than one judge, so that reliability and hence replicability limits can be established. At later stages of the meta-analysis, procedures may vary in some degree according to the breadth of our purposes. If we limit our sights to what

When Findings Conflict

105

we have designated as the narrowest purpose of a meta-analysis - the establishment of a central tendency for some key parameter, usually an effect magnitude, across a set of studies that estimate it - then once the studies have been selected and coded the chief issue is the formal procedure employed to integrate over the study data to reduce the variation to some kind of average. Truly primitive research reviews in a style familiar to all of us abdicate this task of integration and merely cite some studies which confirm a certain proposition and others which fail to confirm it. As study numbers mount, this kind of exposition becomes unmanageable, and the urge to count something arises. The next most primitive procedure is a kind of “vote counting” under a “one study, one vote” decision rule. Thus, for a review of 30 studies, we might be informed that 18 can reject the null hypothesis that no effect exists and can do so at the .05 level; that 9 more fail to reject it; and that the final 3 show significant results, but in the perverse direction. It is self-evident that integration procedures as primitive as this throw away important information left and right, such as differential study sizes and actual effect magnitudes. Although it is less obvious, Hedges and Olkin (1985) have shown that such vote counts are distressingly likely to lead to a conclusion that no real effect is present when, in fact, it is; and worse yet, the probability of this kind of a wrong conclusion actually increases as more and more studies become available to be counted! Arriving at more defensible grand summaries of key magnitudes over disparate studies has thus received a great deal of attention among metaanalysts. Some of the thinking here is attempting to adapt to inadequate published results. How are we to integrate across studies, many of which publish only significance-test information, such as p values? Rosenthal (1978) discusses nine ways of proceeding on such limited outcome information to arrive at more reasonable overall estimates. Some of this detail is of limited interest if we have access to the actual raw data generated by related studies. And when we do, the proper means of cumulating or pooling studies to arrive at grand summaries of observed effects are, in some cases, obvious. However, statistical complications arise very easily in this context, as when the portfolio of studies available have used different complex sample designs. As the sheer number of relatable studies increases, and it is doing so everywhere, more defensible methods of combination are being examined (Kish, 1987, pp. 183ff). At the same time, the narrow goal of arriving at a more stable effect parameter by some form of averaging over discrepant studies, although it has surely been a popular pursuit, is only a special case of what meta-

106 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Converse

analysis is about and is not necessarily the most powerful or interesting application. It is usually more to the point to take the variability of quantitative outcomes over such studies and to attempt to account for it in a classic and familiar fashion, often with study-level characteristics ignored by the original investigators because these features were constants for each of their single studies taken alone. It does not take much to justify the wisdom of understanding as much as one can of the variability in the observed results before leaping to an average. As we have mentioned, one attribute differing across studies which is a favorite workhorse in meta-analyses is study quality. In the experimental culture, greater quality can be associated with such features as randomized assignment, “double-blind” procedures, and the like. In the observational culture, greater quality may be associated with strict probability sample designs and any number of costly control procedures aimed at improving data validity. Suppose, now, we discover as one product of a meta-analysis that a key effect parameter takes significantly different values in highquality studies and in low-quality ones. Do we still find it reassuring to beef up our estimates by averaging over all studies, better and poorer alike? Actually, for many meta-analyses, studies perceived to be of questionable quality are jettisoned from the pool before the actual serious analysis begins. Indeed, this reason for discard is one of the most common, along with inadequate published information. On the other hand, if quality can be rated reliably, it is of some interest to use it as an explicit variable, since our vague sense of the state of truth in particular areas often fails to sort a babel of results into proper quality piles. By keeping all sorts of reasonable studies in view, for example, Glass and Smith (1979) were able to show that the intuitively expected inverse relationship between class size and student achievement was clearer in well-controlled (random-assignment) experiments than in poorly controlled ones. This kind of information is precious. Once we begin to account for cross-study variance in results, we are back in rather familiar waters, at least with respect to staples of procedure. Meta-analysis merely invites us to think more systematically about studylevel features that can deflect magnitudes up or down and then, when hunches are formed, to subject them to a test as rigorous as our pool of related studies permits. Even here, there is something of a further ordering of stages. Naturally, meta-analytic results are most exciting when a revealed source of variability in apparently coordinate results is substantively or theoretically illuminating. This kind of finding can be very helpful in pushing theory up the slope toward greater generality. Not surprisingly, however, a substantial fraction of

When Findings Conflict

107

the determinants of variability in results turns out to lie in differences in nitty-gritty procedures, such as variant operationalizations. There is not much point in noodling through extensive theoretical revisions to account for conflicting results until this kind of pure methods underbrush is cleared away. Yet even differences which trace clearly to methods, by showing circumstances under which the phenomenon appears more-or-less clearly and robustly, can often yield up clues which turn out to be very helpful in theory development. Publication Bias

We have abbreviated our review of the normal course of meta-analyses on the grounds that we all “speak prose,” meaning that once a meta-analytic frame of mind is generated, and a specific array of competing study results is in hand, we have little trouble, for the most part, in mentally filling in what has to happen next. At the same time, the meta-analytic tradition has begun to get serious and systematic about some facets of the game of inquiry which have simply been left undeveloped in other contexts. Let us examine one illustration of such special facets, the question of “publication bias.” Most of us who have engaged in hands-on work near the leading edge of one or another finger of inquiry, however specialized, are well aware that inside knowledge of a phenomenon being developed by the “hands-on” workers in the invisible colleges rarely matches up very well with the understanding of the same phenomenon that an outsider might distill from even a diligent and insightful reading of all of the recent literature on the topic. Since meta-analysis received its first impetus among scholars commissioned to review much reported research in order to arrive at some reasonable fix on the actual state of knowledge for a given topic, any such disjunctures are of the most vital concern. A more careful examination of the ways in which publication bias distorts the apparent state of a field may, in due time, lay bare a number of intricacies. However, some of the major findings already in hand from systematic work on the subject are rather gross and obvious, a matter which means that there is hope that some defensible adjustments of the outcomes of publication reviews will take account of this form of bias. Even these most obvious findings knife into the privacy of the workways of those of us in the knowledge trade in a degree which we may find embarrassing. Logically, publication bias is the result of two stages of filtration between what investigators encounter in reality probes and what winds up in print. The first stage is a matter of self-censorship. Investigators rarely

108 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Converse

bother to write up more than a very small fraction of what they experience as glints and radiations from the empirical world with which they are working. The second stage encompasses all of the selection processes which determine the fractions - again very small - of manuscript product that actually see the light of day in broadly circulated print. There are, of course, links between these two stages, inasmuch as investigators’ motivations to write up findings are powerfully influenced by perceptions of what an editor will be willing to print. However, the two filters are not identical, since other factors can also affect what an investigator will select for writing up, and since investigator guesses about publishability can be wrong. Nevertheless, there is one line of strict agreement between the two filters, which produces a single large “main effect.” This is that published results tend to be sharper or clearer than the general run of results on the same subject experienced but not even written up by investigators in the field. For one thing, investigators paw through many results and are motivated to shape for print only those that have a relatively clear “message.” zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJI has pointed out that most social science Writing in this context, n f t e (1977), theory is sufficiently imprecise, and subject to such a large number of plausible rival operationalizations and notions of conditional variation, that the number of model specifications that may be subjected to actual estimation is usually very large. Indeed, Tufte guessed that in many areas pivoting on regression analysis, “perhaps 50 to 300 regressions are computed for every regression published.” The elaboration of the configurations of the possible assumptions associated with analyses of the LISREL type simply multiply further the “trials” which can be reviewed. It is obvious, of course, that what emerges from all of this informal searching is likely to be the most sharply structured configuration of results that the data can be made to yield up. Such a search can, in some degree, be healthy, as when it can be shown that the results sharpen systematically in response to theoretically diagnostic variations in the organization of the analysis. But the vagueness of much of our theory does mean that many analytic variants differ from one another arbitrarily, as best the investigator can see. And this gives a rich field for the kind of ransacking which makes a mockery of the common tools of statistical inference, although we publish such evaluative data all the same. This is at the level of the investigator seeking results “worth publishing.” There is probably an added screen at the publication end of the process, with editors and reviewers being more captivated by sharp results than by marginal findings, other things being equal.

When Findings Conflict

109

The outcome of both stages is that the published results are unrepresentatively sharp as reflections of reality. Since the home turf of meta-analysis has been studies of treatment effects, this means simply that the published literature tends to overstate the sizes of such effects. However, this bias can naturally be generalized to all forms of quantitative results, including those commonly produced from naturalistic observation. The actual demonstration of such effects is harder to come by than it may appear at first glance. To proceed in this direction, the meta-analyst should, of course, compare a summation of findings from a proper sample of published results with the same summation from a proper sample of unpublished results. But ensuring a proper sample of published results requires great care in itself. And what does it even mean to talk of “a proper sample of unpublished results”? This is a set which is undefined and probably undefinable. In practice, meta-analysts consider any relevant unpublished results worth their weight in gold for the comparisons they can offer. Meta-analysts also give a good deal of attention to results outside professional journals, in outlets such as government reports, where numbers must be set down by contract, pretty or not. Interestingly enough, meta-analysts enthuse over the world’s corpus of doctoral dissertations as providing some foil for the data published in professional journals. This is so not only because dissertations tend to be derivative and hence replicative of other’s findings. They also tend, unlike a lot of unpublished research which is carelessly done, to be meticulously carried out and explained in painful detail, given the kibitzing, actual and anticipated, of faculty advisers along the way. None of these substitutes for unpublished work are ideal, but they tend to help reveal publication bias effects. It should not be lost on secondary analysts of information from naturalistic observation that in this regard, large data archives are rich quarries of “unpublished data” and might well be used to that effect. Despite the inherent problem in demonstrating publication bias, a number of studies using different methodologies make it pretty clear that the publication process does indeed sift toward sharp results (Greenwald, 1975; Rosenthal, 1978; Smith, 1980). Of course, what constitutes a “sharp” result may differ at differing stages of inquiry into a given phenomenon. In the early stage, the search is on for strong effects and relationships. Later on, when certain strong relationships have come to be expected, failures to detect them, hence suggesting disconfirmation of the received wisdom, begin to have a good deal of publication appeal to investigators and editors alike.

I10 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Converse

If the premium at this stage is not to reject the null hypothesis, it is, of course, most impressive if the nonrejection is itself clear and sharp, rather than being shrouded in a weak but insignificant “trend in the expected direction.” Tufte (1977) has therefore predicted what he calls a “zone of boredom,” which should be essentially evacuated in published results in later stages of research. He then reviews some ninety-five values of R 2 published in a dozen studies of the relationship between short-run economic fluctuations and electoral support for incumbents. The observed values are, as expected, sharply bimodal, with a cluster near zero and a distribution of “clear” positive findings beginning around .40. The “zone of boredom” in this instance runs from about .17 to .36. This largely evacuated zone could be expected to shift downward in applications with disaggregated data where the Ws are larger and individual-level variance is captured. What all this means, of course, is that publication tends to select toward results that fall at the tails of the true distribution of all results, and that it underrepresents the boring middle ground, although in actuality this ground may be the most heavily populated. This kind of stroboscopic, “now-yousee-it, now-you-don’t’’ characteristic of published social science findings is a source of dismay to practitioners and a delight to outside detractors convinced that social scientists can’t arrive at permanent meaningful knowledge in any event. While the examination of publication bias is humbling, we can ill afford to ignore it. A more sophisticated view of what it does to our subliteratures is much to be desired, and work in this direction is in progress within the meta-analytic endeavor. Conclusion

It may as well be recognized, in conclusion, that examinations of conflicting social science results are frequently humbling for many more reasons than oddities of publication bias. As we have intimated, for example, many assessments of social effects are pushed around in an embarrassing degree by what seem to be no more than small and innocuous variations in study procedures. It would be marvelous if the effects we study were more invariant under changes in context; but if this is not the case, a greater theoretical respect for the power of the situational in human behavior needs to be cultivated. Such respect is relevant to work in both the experimental and the observational research cultures. More troubling still, perhaps, is the degree to which areas of ideological polarization seem to resist solution by resort to empirical test. Even in settings of the controlled experiment, social scientists can, in some degree,

When Findings Conflict

111

find what they want to find. For example, one meta-analysis of many social psychological studies of sex differences in influenceability showed that the gender of the investigator was one significant predictor of differences in the published results (Eagly and Carli, 1981). On a more explosive issue still, Wortman and Bryant (1985) report on a very extended attempt to use meta-analysis to arrive at a consensus in the face of conflicting social science results from studies on the effects of school desegregation on the achievement of black students. The National Institute of Education convened a panel of experts to do a more systematic review of these studies. This panel included members with very different prior views concerning the efficacy of desegregation for black achievement, There was general agreement that there was some positive effect of desegregation, but there were wide differences on whether the “effect size” was substantial or trifling numerically. Even in a meta-analytic mode, disagreement continued among the panelists with regard to what studies a proper synthesis should include, as well as what control groups were most relevant, and the like. As a result, the panel disbanded “with their initial views intact” (Wortman and Bryant, 1985, p. 315). However humbling all of these lessons may be, it is surely more constructive to confront them head-on than to continue to do research with our heads in the sand as though single studies, if only done with sufficient care, should generate “the” proper answers. When social science results conflict, as they usually will, we should not ignore the problem or try to dispel it with glib excuses about the likely reasons, as though we were on momentary vacation from the requirements of scientific proof. The notion of metaanalysis has developed as a means of “keeping us honest” in these regards. Notes 1. Mark W. Lipsey, The Claremont Graduate School, personal communication, 1987.

References Campbell, D. T., and Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin 56:81-105. Eagly, Alice H., and Carli, Linda L. (1981). Sex of researchers and sex-typed communications as determinants of sex differences in influenceability: A metaanalysis of social influence studies. Psychological Bulletin 90: 1-20. Fiske, Donald W., and Schweder, Richard A., eds. (1986). Metatheory in Social Science. Chicago: University of Chicago Press. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher 6: 3-8.

112

Converse

Glass, G. V., and Smith, M. L. (1979). Meta-analysis of research on class size and achievement. Educational Evaluation and Policy Analysis 1:2-16. Glass, G. V., McGaw, B., and Smith, M. L. (1981). Meta-Analysis of Social Research. Beverly Hills, CA: Sage. Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin 82: 1-20. Hedges, Larry V., and Olkin, Ingram. (1985). StatisticalMethods for Meta-Analysis. Orlando, FL: Academic Press. Kiewiet, R., and Rivers, D. (1985). The economic basis of Reagan’s appeal. In J. Chubb and P. Peterson (eds.), The New Direction in American Politics. Washington: Brookings Institution. Kish, Leslie. (1987). Statistical Design for Research. New York: Wiley. Light, Richard J., and Pillemer, David B. (1984). Summing Up: The Science of Reviewing Research. Cambridge: Harvard University Press. Particle Data Group. (1976). Reviews of particle properties. Reviews of Modern Physics 48:Sl-S20. Rosenthal, Robert. (1978). Combining results of independent studies. Psychological Bulletin 85:185-193. Sacks, Henry S., Berrier, Jayne, Reitman, Dinah, Ancona-Berk, V. A., and Chalmers, Thomas. (1987). Meta-analyses of randomized controlled trials. The New England Journal of Medicine. February 19, 1987, pp. 450-455. Smith, M. L. (1980). Publication bias and meta-analysis. Evaluation in Education 4:22-24. Tufte, Edward R. (1977). On the published distribution of R2: Consequences of selection of models and evidence. Unpublished manuscript. Turner, Charles F., and Martin, Elizabeth. (1984). Surveying Subjective Phenomena. New York: Sage. Wortman, Paul M., and Bryant, Fred B. (1985). School desegregation and black achievement: An integrative review. Sociological Methods and Research 13: 289-323.

Comment

O n Conflict Resolution: Right Makes Right Karl Taeuber

P

rofessor Converse admonishes that conflict among findings is to be expected. The scholar reviewing the literature should not cry in despair but should view disparities as the melody in a meta-analytic ode to joy. Seeming contradictions in empirical inquiry may, upon metaexamination, yield a welcome boost up the slippery slope of generalization about complex social phenomena. This clarion call to embrace meta-analysis is ably presented, with philosophical persuasion abetted by compelling examples. But as Professor Converse acknowledges, there is strong resistance; many of us have trouble responding to the call with open arms. The joyful perspective on the rewards of sifting the detritus of previous studies contrasts with the somber teaching of experience that talus slopes are treacherous. Far more abundant than Professor Converse’s successful examples are the introductory paragraphs of journal articles that recount the flaws in design and execution that render prior studies unfit as a basis for sound theory. Pursuing the mountain metaphor, consider the headline news some months ago proclaiming that K-2, not Mt. Everest, was the world’s highest peak. In the pre-TV days of my youth, when vicarious adventure was to be found in books, I read scores of mountain-climbing sagas and thrilled to the lure of the ultimate challenge, Mt. Everest. Years later, I can understand, as a sociologist, recent efforts to use an indigenous name for the peak, but can I be philosophical about the reassessment of its relative height? Of course not. I waited with complete assurance for the next headline to come along, and it did, reclaiming for Everest its rightful title. Sorry, Professor Converse, but in this instance, right prevailed. K-2 indeed! I13

114

Taeuber

As I scramble among the foothills of social science, I retain the youthful inclination to think in terms of right and wrong. In my own encounters with conflicting results, particularly instances where my results have conflicted with those of others, is there any doubt that I was right and they were wrong? Isn’t one good analysis worth more than any number of poorer studies? The heights of K-2 and Everest are being more accurately determined with new technologies, not with the meta-analyses of older triangulations. When results conflict, an old-fashioned critique to determine which study had the best methodology may be quicker and more reliable than a new-fashioned multivariate analysis of information from all the studies. Why do I cling to the idea of one right and many wrong answers, rather than a family of estimates, each holding an aspect of truth? In confessional mode, I shall delve into my past to find the roots of this resistance to the meta-analytic call. As youthful reveries with mountaineering sagas gave way to the rigors of professional training, I encountered methods courses. Research design, at least in the old days, had no elements of the stochastic or the multimethod approaches. But Professor Converse and others overcame the cultural obstacle of early training in the right-wrong school; there must be other roots. Although I sometimes try to hide it when in the company of historians, political scientists, and other mainstream ICPSR types, I must confess to being a demographer. As a demographer, I was brought up believing in the one true source of knowledge, the decennial census. When I learned the demographic catechism, the printed table reigned supreme. When a new census volume arrived in the mail, it brought a host of new tables, never before seen or analyzed. If analysts of the 1940 census had studied a table of occupation by income by age, and the 1950 census brought us a similar table but with an additional cross-classification, by race, that was heaven. We could do the first analysis of racial patterns in the occupation by income by age relationship. If anyone else studied the same topic, they used the same data. Any conflicting results came from differing methods. We thrived on debating, in pre-meta-analysis style, which method wielded the key to truth. In the meta-analyses that Professor Converse holds out as models, the separate studies are based on separate data sets. In clinical trials of a medication, for example, many independent investigators may report their own experience with patient samples of varying size. In the domain of observational studies, there may be dozens of preelection polls with considerable overlap in question content. As I write, the news media report a metaanalysis that pooled 101 studies to obtain large-sample estimates of the links between neurotic personality traits and various illnesses. In these examples,

Comment

115

numerous data sets are viewed as replicated studies. Meta-analysis may examine sampling and nonsampling explanations of the distribution of study results or may pool selected results to obtain greater significance or power. Professor Converse suggests the need for about 100 experimental studies, or a slightly smaller number of observational studies, before metaanalysis can function. Too small a number of studies severely hinders the utility of meta-analysis. With only a few observations, unknown and quirky distributions can’t be described well, and multivariate interpretations aren’t practical. Much scholarly work occurs in domains that seem to offer little possibility for meta-analysis. I’ve mentioned the decennial censuses. A few years ago, when several of us persuaded the National Science Foundation to put several million dollars into the creation of public-use microdata samples from the 1940 and 1950 censuses, it was precisely because these would be unique data sets. One of the documents supporting the proposal was a compendium of topics and studies that had never been done and could be undertaken only if these data sets were created. Many expensive studies and their data sets are effectively nonreplicable. The Middletown studies and other large-scale cross-sectional studies resemble the censuses; there may be decades of subsequent secondary analysis and, in some instances, massive restudies, but these do not fit the metaanalysis rubric. The few repeated large-scale cross-sections, such as the monthly Current Population Survey, are more suited to temporal analyses than to meta-analyses. Other unreplicated data sets are the increasingly popular longitudinal studies, such as PSID, NLS, and SIPP, and the extravagantly expensive experiments such as income maintenance and housing allowance. When findings from two or more studies of one of these data sets conflict, what do we do? We are likely to act in PMA (pre-meta-analysis) style. We examine the selection of variables, the operationalizations, the measurements, the models, the concepts, and the interpretations. We get caught up in many of the issues discussed in the chapters by Professors Glenn and Blalock. We dispute on the grounds of the appropriateness and inappropriateness of design, analysis, and execution. We argue in terms of right and wrong, good and bad. We may consider competing or compensating biases, and we may seek to blend information from two or more studies, but we rarely proceed very far toward a meta-analytic perception of a distribution of results. Pre-meta-analysts share with meta-analysts the fundamental canon of science that results should be reproducible. One test of reproducibility is

116

Taeuber

replication. Different studies of the same topic should yield approximately the same results. In some domains of inquiry, gathering a new set of data is relatively feasible. Small clinical trials, classroom studies, some surveys, some organizational analyses, some experimental studies, and selected other studies may yield enough near-replications to meet the applicability criteria for meta-analysis. These are grist for Professor Converse’s mill, and he has provided us with nourishing samples. He extols the economy of metaanalysis. It shares with other secondary analyses the great virtue that the costs of data collection have already been incurred and that further analysis is relatively inexpensive. But shouldn’t this cost-benefit analysis be carried farther? Isn’t there a sense in which the formal multiple-study meta-analysis comes too late? Why should such large numbers of studies accumulate? In principle, the literature reviews that fill up the early pages of proposals and articles take heed of the growing body of previous work. If each successive researcher practiced good research methodology, each succeeding study would be designed not merely as another random member of a family of studies, but as a contribution to understanding specific features of the distribution of prior results. Do not multiple unintegrated replications represent an expensive failure of the scientific process? Do we face a paradox that the spread of meta-analytic ideas could lead to reducing the possibilities for the implementation of meta-analyses? What would happen if the 150th, or 73rd, or 22nd, or 5th researcher conducted not an isolated replication, but a study designed to resolve disparities in findings. Shouldn’t funding agencies or other scientific gatekeepers decide earlier in the process that a few good studies, even if harder and more expensive, would be a cheaper and quicker way to make progress than awaiting the cumulation of scores of lesser studies and an eventual meta-analysis. Secondary analyses of prior work should not await the cumulation of numbers sufficient for full-scale meta-analysis. Replicators should be conducting their own mini-meta-analyses of prior studies. Funders, editors, mentors, and colleagues should be more insistent that this be an expected and rewarded activity. Collective steps can be taken to improve the standards for citing data sources, documenting data manipulations, and reporting methods of analysis. We must make prior work more accessible and meaningful to those contemplating follow-up research. The first criterion of reproducibility of results is verification. Using an investigator’s own data, others should be able to reproduce the results. I think this is a much underused mode of review of articles submitted for

Comment

117

publication, as well as an often neglected first step in the examination of conflicting results. People do make mistakes, they take shortcuts, they don’t communicate clearly with their programmers, they trust their employees, and they substitute hope for tedium in checking their own work. Sometimes, as the history of science repeatedly documents, investigators cheat. In many domains of social science study, confusion and a poor understanding of the concepts, the procedures, the data bases, and the analytic models are to be expected. When I work with the data files from a census or a large social survey, I am confronted with questionnaires and code books running into hundreds of pages. Sampling design, fieldwork procedures, and initial data cleaning, coding, and editing activities are extraordinarily complex and incompletely documented. I’m suspicious of my own ability to know what I’m doing. When a set of results represents the product of a research staff and many stages of data processing, can I know that I’ve done what I’ve think I’ve done? How can I be sure that the final product bears a close resemblance to what I intended? It’s unreasonable to assume that no undetected and uncorrected mistakes were made. Procedures for independent verification are essential. Professor Converse presents an entrancing view of the enormous potential value of developing the tools of rigorous meta-analysis and applying them more widely and creatively. I’ve presented a more somber view, with two main concerns. First, the applicability of fully developed meta-analysis is restricted by the limited availability of domains of inquiry yielding scores of closely linked studies. Second, I caution that error, nai‘vete, and poor study design continue to account for much conflict in findings. It is a sad commentary on the organization of the scientific enterprise that meta-analyses often find indicators of study quality to be key explanatory variables. A more mundane application of the standard scientific criteria of the evaluation of prior work and the design of new studies should allow quicker and cheaper detection of error and resolution of conflict. Awaiting the accumulation of dozens of flawed studies is terribly inefficient. Of course, if this accumulation has already occurred, meta-analytic tools can be a valuable and efficient aid in detecting the problems and resolving the conflicts. Professor Converse’s idea of a distribution of research results and my concern with the role of error are linked by a joint emphasis on science as a cumulative enterprise. We are in full agreement that the accumulated body of prior work deserves increased utilization by social science researchers. I have four recommendations for improving research and enabling a quicker understanding of conflicting findings:

118 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Taeuber zyxwvutsrqp

1. Meta-analysts and other disputatious scholars should do more straight

verification as well as reanalyses and replications of prior studies. 2. Replicators should have a meta-analytic model in mind, should review prior studies, and should seek study designs that add information about the distribution of results. 3. All investigators should promptly deposit their data and full documentation of study design, procedures, and data files in an appropriate archive. 4. The community of social science investigators and archivists should work vigorously to promote and improve archiving.

To facilitate reanalyses, replications, and strategic extensions of already-reported studies, the collective structure, organization, and financing of data archiving need to be strengthened. While Professor Converse’s examples of successful meta-analyses are testimony to the potential benefits of archiving, actual payoffs are all too rare and often much delayed. I think the rate of scholarly return on investment in data archives can be raised. I challenge the community of archivists to take an increasingly active role in promoting a greater knowledge of, and easier access to, the storehouse of studies already produced at great expense and with enormous commitments of scholarly time and energy. Professor Converse chose the title, “When Social Science Findings Conflict,” not “If Social Science Findings Conflict.” An improved understanding of conflicts, a quicker and less costly resolution of disputes, and increased scientific cumulation are possible. Social science archives have a key role to play.

Chapter 4

What We Know, What We Say We Know: Discrepancies Between Warranted and Unwarranted Conclusions Norval D. Glenn

layperson who examined a few issues of the leading social science journals published in the United States might well be impressed with the apparent increment to knowledge about cause and effect contributed by the research reported in those issues. He or she would find numerous statements about causation, impacts, influence, and determination, and many if not most of the statements would not be tentative. It would appear that many of the authors are certain that they have demonstrated with precision what affects what, and in what direction and to what degree. What the lay reader unschooled in the logic of causal inference would not know is that hardly any, if any, of the untentative conclusions about causation in the social science journals are warranted by the evidence. Rather, the evidence for the conclusions generally ranges from “persuasive but not conclusive” to “tenuous and suggestive only.” Some of the articles that contain categorical statements about causation also contain tentative statements and/or cautions that the evidence is not conclusive. But in other articles, there is no indication that the author has any doubts about the validity of his or her causal assertions. Uncritical readers of both kinds of articles are likely to be deceived, and the authors of the latter articles have apparently deceived themselves. Of course, laypersons rarely read social science journals, but many persons who do read the journals are likely to be misled by unwarranted

A

119

120

Glenn

conclusions about cause and effect. These persons include textbook writers, teachers, popularizers of social-scientific findings, synthesizers of socialscientific findings, graduate students, and, of course, social science researchers. And since textbook writers and social science popularizers (and probably teachers as well) have a strong tendency to translate tentative conclusions into untentative ones, the social sciences are represented to students and to the public as knowing a great deal more about cause and effect than they do know. Equally important, many social scientists, including many active researchers, seem to share the illusion that we know more than we do know. An amoral social scientist might take delight in the false image of social science presented to the public, on the grounds that if there were a general awareness of our lack of conclusive evidence about cause and effect, public support for social science probably would not be as strong as it is. However, social scientists’ pretensions to certain knowledge about causation do not always work to our advantage. Rather, among the more sophisticated outsiders, such as the physical scientists, the pretensions tend to undermine respect for the social sciences. And lack of respect from these sophisticated outsiders can affect the allocation of resources within universities and by private foundations and federal funding agencies, such as the National Science Foundation. Furthermore, contradictory dogmatic conclusions by different social scientists that are publicized by the media probably tend to undermine public confidence in social science. Any tendency for social scientists to delude themselves about what they know about causation probably has even more serious negative consequences; for instance, it almost certainly impedes the acceptance, if not the development, of means of overcoming the limitations of quasi-experimental research (Achen, 1986), and premature feelings of certainty would seem to be akin to dogmatism in their inhibiting effect on the openness and flexibility needed for scientific progress. Finally, the most compelling reason for social scientists to refrain from claiming knowledge they do not have is ethical rather than pragmatic. We owe to those who support us, and to those who will use and be influenced by the products of our efforts, honesty in the claims we make about our evidence and knowledge. The validity of the thesis of this chapter, as that thesis is briefly summarized in the preceding paragraphs, is apparently not self-evident (though it seems so to me), since there is, so far as I can tell, no literature within the quantitative tradition that deals directly with a discrepancy between warranted and stated conclusions in the social sciences. There is, moreover, a

What We Know, What We Say We Know

121

literature that urges social scientists to use fewer “weasle words’’ and to be less tentative in their conclusions.’ I must therefore try to make a case for my thesis in the pages that follow. The Need for Tentativeness

The research design that can provide the most nearly conclusive evidence concerning causation is the randomized experiment, that is, the kind of research in which subjects are randomly divided into an experimental group and a control group, and a stimulus or treatment, the effect of which is being investigated, is administered to the former but not to the latter. Any difference between the two groups in the outcome variable, or in its change scores, at the end of the experiment is the estimate of the effect of the stimulus or treatment. Even under the best of conditions, the magnitude of the effect is estimated, not precisely measured, since the indicated effect is subject to random error. Furthermore, there may be substantial error in the measurement of the outcome variable and, depending on the nature of the research, in the administered variable as well, and experimental effects may be commingled with reactive effects of the research (Cook and Campbell, 1979). At least occasionally, experimental research fails to accurately estimate effects for other reasons, as when the last measurement of the outcome variable is taken too early to detect delayed effects of the administered variable. Hence, even in the reports of randomized experiments in the social and behavioral sciences, conclusions about cause and effect from the findings of one study should be stated tentatively, and usually, some tentativeness is warranted even when the conclusions are based on a large body of corroborative evidence. Most research in the social sciences is not randomized experiments and thus can provide less nearly conclusive evidence of causation. Although we could use randomized experimentation more than we have so far, severe ethical and practical constraints make it inconceivable that such research will become the usual or dominant kind in the social sciences (Achen, 1986, pp. 7-11). Most of our research has been and will continue to be quasiexperimental, in which we use statistical controls and causal modeling to estimate the effects of possibly important causal variables on an outcome variable. Just how adequate quasi-experimental research is as a substitute for randomized experimentation is controversial, but at best, it is more complicated, requires more theoretical understanding on the part of the researcher, requires the measurement of more variables, and thus is more susceptible to the effects of measurement error. Clearly, in quasi-experimen-

122

Glenn

tal research, it is more likely that something will go wrong so that the estimates of effects will be inaccurate, and it is especially more likely that the estimates will be wrong in terms of direction as well as magnitude, that important effects will be missed, or that effects will be estimated to exist when they do not. Thus, even the researchers who are most sanguine about the adequacy of quasi-experimental research cannot very well argue that the conclusions derived from it should be stated other than tentatively. A rather substantial literature published during the past few years indicates that quasi-experimental research, as it has usually been conducted, has more serious limitations than most of its users have realized (e.g., Campbell and Erlebacher, 1975; Boruch, 1976; Lieberson, 1985; Achen, 1986). For instance, Lieberson deals at length with several ways in which he thinks social researchers have gone wrong, including especially their failure to deal with unmeasured selectivity. Although the reviews of the Lieberson book have tended to be rather negative (e.g., Costner, 1986; Berk, 1986; Campbell, 1987),2the reviewers have not challenged the author’s claim that the ways in which social scientists have been using controls and causal modeling have serious limitations. Rather, the reviewers say or imply that there is little that is new in the book, and that the shortcomings of conventional research procedures discussed in it are already well understood by advanced methodologists. For instance, Berk (1986) faults the book for not referring to “the many lucid and accessible expositions of selectivity and specification error” but concludes that “there is much to be said for Lieberson’s observation that the causal modeling done by most sociologists is fatuous . . .” (p. 464). Achen (1986), in a discussion more solidly based in the statistical literature than Lieberson’s, essentially agrees with Lieberson when he explains why matching and regression fail in quasi-experimental research when there is unmeasured selectivity into categories or levels of the treatment (independent) variable. This condition may exist whenever persons select their values on the independent variable or when someone else selects it for them, so there may be unmeasured selectivity except when the independent variable cannot be affected by human volition. Clearly, the problem of unmeasured selectivity is pervasive in social research. Thus, there is considerable agreement among sophisticated methodologists that many of the causal inferences made by social scientists in recent years are very likely incorrect.3 In the past few years, methods have been developed that in some circumstances can at least partially overcome the limitations of the conventional procedures, and several of the new methods are explained in a lucid book by Achen (1986). These methods are not panaceas for the problems of quasi-experimental research, however; they cannot be mechanically applied

123 What WeKnow, What WeSay WeKnow zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPON

to arrive at truth. Rather, they all require considerable theoretical sophistication and apriori knowledge (as well as technical expertise) to be used properly, and thus, they are highly susceptible to failure due to errors in judgment by the researcher. Furthermore, in certain circumstances (which are not rare), the methods are not applicable. These considerations, along with obdurate measurement problems (Duncan, 1984), mean that responsible and honest quasi-experimental researchers will not be able, in the foreseeable future, to state their causal conclusions with certainty. The message here is not one of despair. When there is proper sensitivity to the limitations of quasi-experimental research, and when social scientists are willing to consider the evidence from alternative methods (qualitative as well as quantitative), the evidence for cause and effect can often be highly persuasive, if not conclusive. Certainty may be an illusive goal never to be reached, but the cumulative evidence from studies conducted with different methods may often bring us so close to certainty that the difference between the goal and the attainment is of little practical importance. For some readers, hope for eventual “virtual certainty” may not be enough. The need for certitude seems to be strong among many social scientists, as well as among many, if not most, laypersons, and among social scientists, such a need can be vocationally incapacitating. Persons with little ability to tolerate tentativeness and probabilistic knowledge probably should not be social scientists. The main disagreement with my claim of the need for tentativeness in virtually all conclusions about causation in the social sciences is, so far as I can tell, disagreement not about what we know but about whether or not we should claim to know more than we do. Curiously, some social scientists, as well as some outside critics of their writings believe that a reluctance to make bold causal statements leads to poor writing. For instance, in his otherwise delightful and useful book entitled Writing for Social Scientists (1986), Becker writes: Sociologists’ inability o r unwillingness to make causal statements similarly leads to bad writing. David Hume’s essay Concerning Human Understanding made us all nervous about claiming to demonstrate causal connections, and though few sociologists are as skeptical as Hume, most understand that despite the efforts of John Stuart Mill, the Vienna Circle and all the rest, they run serious scholarly risks when they allege that A causes B. Sociologists have many ways of describing how elements covary, most of them vacuous expressions hinting at what we would like, but don’t dare, to say. Since we are afraid to say that A causes B, we say, “There is a tendency for them to covary“ or ”They seem to be associated.”

124

Glenn

The reasons for doing this bring us back to the rituals of writing. We write that way because we fear that others will catch us in obvious errors if we do anything else, and laugh at us. Better to say something innocuous but safe than something bold you might not be able to defend against criticism. Mind you, it would not be objectionable to say, ”A varies with B,“ if that was what you really wanted to say, and it is certainly reasonable to say, ”I think A causes B and my data support that by showing that they covary.” But many people use such expressions to hint at stronger assertions they just don‘t want to take the rap for. They want to discover causes, because causes are scientifically interesting, but don‘t want the philosophical responsibility. (pp. 8-9).

I show below that the social scientists who publish in some of the major American social science journals generally are not unable or unwilling to make causal statements; they are demonstrably unafraid to say that A causes B. Becker would presumably approve. However, is it laudable to take “philosophical responsibility” for making bold, untentative causal inferences when we lack adequate evidence to support them? I think not, but Becker seems to think that the making of strong causal statements will bring forth the evidence needed to qualify them, and he terms the qualifications that make causal statements tentative “bullshit qualifications” that result from cowardice: A real qualification says that A is related to B except under certain specified circumstances: I always shop for groceries at the Safeway unless it’s closed; the positive relationship between income and education is stronger if you are white than if you are black. But the students, like other sociologists, habitually used less specific qualifications. They wanted to say that the relationship existed, but knew that someone would, sooner or later, find an exception. The nonspecific, ritual qualifier gave them an all-purpose loophole. If attacked, they could say they never said it was always true. Bullshit qualifications, making your statements fuzzy, ignore the philosophical and methodological tradition which holds that making generalizations in a strong universal form identifies negative evidence which can be used to improve them. (ibid., p. 10).

In this passage, Becker is criticizing the writing of graduate students, and I am referring to the writing of mature professionals, but that does not account for the differences in our views. It seems to me that Becker is criticizing the students for a reluctance to say they know what they do not know. To me, his position is similar to criticizing a person for having a reluctance to lie based on a fear of being caught in the lie. In any event, more completely socialized social scientists seem generally to lack the timidity about making causal statements that Becker found to be so reprehensible in his students - and the difference is presumably the result of the socializa-

What WeKnow, What WeSay WeKnow

125

tion. If so, it seems to me that socialization into the social sciences tends to produce irresponsibility and di~honesty.~ Of course, there is a difference between causal statements identified as personal opinion or as theoretical statements and those represented as being based on empirical evidence. It is only the latter that concern me here. And of course, any statement, tentative or not, should not be unnecessarily “fuzzy” or unclear (although just what Becker means by “fuzzy” is itself unclear). My final major disagreement with Becker is my belief that the causal statements that occur frequently in the social science literature do little, if anything, to “identify negative evidence which can be used to improve them.” Rather, the reader who takes them seriously will think that no additional evidence is needed, since conclusive evidence already exists. Strong causal assertions in an oral debate usually evoke strong rebuttals, but that such statements allegedly based on sound empirical evidence and published in social science journals will usually evoke rebuttals is not evident. And even if rebutting and refining evidence is eventually forthcoming, important personal and policy decisions may, in the meantime, be based on the strong causal assertions. Therefore, the issue of honesty versus “boldness” has important ethical implications as well as implications for the progress and growth of the social sciences. The implications for “good” versus “bad” writing are rather trivial in comparison. The Lack of Tentativeness in Causal Statements in Reports of Social Research

It might seem to be easy to examine a representative sample of social science publications to see whether or not most conclusions about causation are stated with appropriate caution and tentativeness. The task is complicated, however, by some ambiguity concerning what is and what is not a causal statement. This ambiguity grows largely out of a long tradition (an unfortunate one in my opinion) in the statistical literature of using the word effect in a noncausal sense, so that in some publications one may see references to both “causal” and “noncausal” effect^.^ This usage, of course, is inconsistent with the usual dictionary definition of effect, and to persons unfamiliar with the statistical literature, the term noncausal effect is an oxymoron. For instance, Webster’s New Collegiate Dictionary (1981 edition) gives as the first meaning of effect “something that inevitably follows an antecedent (as a cause or agent),” and none of the other meanings is noncausal in the statistical sense. The synonyms listed for effect are result, consequence, event, issue, and outcome, and the shared meaning element

Glenn

126

given for effect and its synonyms is “a condition or occurrence traceable to a cause.” If authors who used effect in a noncausal sense always made that clear, there would be no ambiguity, at least to readers familiar with the noncausal sense of the word, but often they do not. Frequently, authors who could be referring to noncausal effects slip into using the word in a clearly causal sense, which indicates that they have trouble in their own minds in maintaining the distinction between causal and noncausal effects. In causal modeling, the word effect would seem always to be used in a causal sense, unless the author specifies otherwise, but then there is the very crucial difference between an effect in the statistical model and an effect in the real world (in the real target population). An effect in the statistical model is an estimate of an effect (in a causal sense) in the real world. Many authors fail to maintain a clear distinction between model and real-world effects. For instance, they may begin by discussing effects in the model and then slip into talking about effects in the real world without using the appropriate modifiers to make their conclusions tentative. Sometimes the context of a statement about an effect makes it fairly clear that a model effect is referred to, although the statement by itself would seem to be a statement about the real world. However, statements in different paragraphs from any context that would indicate that they refer to a model are likely to be interpreted by readers as conclusions about the real world, and no other part of a paper can provide the appropriate context for a statement about an effect that appears in an abstract or a summary or a conclusions section. Almost always, statements about effects in an abstract or a concluding section of an article should be made tentative by the use of such words as estimated or apparent. Frequently, however, no such qualifying words are used. Consider the following statements from abstracts of articles in recent issues of leading American social science journals: In this paper, we use time series data to illustrate the long-time negative effects of American direct investment on post World War II economic growth in Canada. (American Sociological Review, 1986) With data from 15 Indian states, in this study I demonstrate that political capacity, defined as the ability of government to penetrate society and extract resources, has a more significant-though indirect-effect on fertility behavior than does (American Political Science Review, 1987) level of economic development ,

..

Bahr’s traditional model was partially supported for the United States, where aging has a slight effect on church attendance. A similar trend was found for

What We Know, What We Say We Know

127

Holland for religious affiliation. The aging process in Japan has a significant effect on religious belief. (American Journal of Sociology, 1987)6 Routine activitieshfestyle variables have relatively strong direct mediational effects on individuals’ risk of property victimization but not for violent victimization. (American Sociological Review, 1987) The analysis uses the nationally representative data collected in Australia in 1983 to show that the child-rearing role and differing attitudes toward work do not account for the greater religiousness of women. By contrast, the lower rate of female work force participation is an important explanatory factor. Possible reasons for this effect are discussed. (American Sociological Review, 1987) Using prejudice toward blacks as the outcome measure, an analysis of national survey data for the years between 1972 and 1985 indicates that (a) consistent with Wirth’s and Stouffer’s arguments, urbanites and non-Southerners are more racially tolerant than their nonurban and Southern counterparts; (b) contrary to some previous research, the net effects of urbanism on tolerance have increased over time while region effects have decreased; the effects on tolerance of urban to nonurban migration confirm Wirth‘s notion of the permanence of urbanism’s (American Sociological influence but not Stouffer’s culture-shock hypothesis Review, 1987)

...

The complete abstracts are not reprinted above, but in no case does the omitted portion make clear that the effects referred to are in a statistical model and are only estimates of effects in the real world. In all of these cases, the model effects may be substantially different from the real-world effects because of such possible shortcomings in the analyses as model misspecification and error in the measurement of the variables. Authors who write about model effects as though they were making conclusions about the real world may give the defense that sophisticated readers will know that the word effect in a report of quantitative research almost always refers to an effect in a statistical model. This is a weak defense, because social science journal articles are read by many persons who are not sufficiently sophisticated to know that causal statements do not mean what they seem to say.’ Furthermore, there is evidence that many of the authors are themselves thinking about real-world effects rather than model effects, since they also use, without being tentative, causal terms that have no statistical meaning. There is no widely accepted convention for using such words and terms as influence, bring about, impact, and determinant to refer to parameter estimates from statistical analyses, but such words and terms are used generously in reports of the findings of social research.

Glenn

128

Consider the following examples, which again are from the abstracts of articles in recent issues of leading social science journals: Possession of multiple role-identities (up to 6 in Chicago, 8 in New Haven) does significantly reduce stress in both samples . (American Sociological Review,

. .

1986)* Different news sources have different effects. News commentators (perhaps reflecting elite or national consensus or media biases) have a very strong positive impact, as do experts . (American Political Science Review, 1987)9

. .

The influence of economic conditions on budgetary outcomes is strong but varies considerably across spending categories. There is no evidence of a political business cycle. Political variables exert a modest influence on the budgetary outcomes examined . . (American Political Science Review, 1987)

.

The analysis confirms that congressmen are purposeful actors, but it also shows that different interests incite participation on different issues and that motivational effects vary in predictable ways across legislative contexts . . (American Political

.

Review, 1987) There is substantial variation in factors affecting expectations about premarital residential independence. Young men more than young women, those with more parental resources, those who expect to marry at older ages, and those who do not have ethnic and religious ties that link them to their parental home until marriage expect to live independently. Religious, racial, and ethnic differences interact in complex ways with gender and socioeconomic status to influence expectations about premarital residential independence. (American Sociological

Review, 1987) Moreover, suburban functional scope influenced status change indirectly through population growth. (American Sociological Review, 1987) O n the other hand, individual SES has a negative influence on subsequent court referrals independent of both self-reported delinquency and police records .

. .

(American Sociological Review, 1986)

Third, child influence on parental attitudes are relatively strong and stable across age groups, while parental influence decreases with age, although the exact pattern of influence varies by attitude domain. (American Sociological Review,

1986) We find that sense of worth i s affected both by level of rewards and by explanations of socioeconomic standing, and that explanations and evaluations play a small role in conditioning judgments of socioeconomic fairness. (American Socio-

logical Review, 1987)

What We Know, What We Say We Know

129

Controlling on earlier traditional-egalitarian orientations, the transition to parenthood exerts effects that vary by the color and marital status of the respondent. For white married respondents, becoming a parent has a clear and traditionalizing influence. For unwed black women, the opposite effect is observed-parenthood leads to more egalitarian attitudes. (American Sociological Review, 1987)'O Our findings show homicides for total populations and whites to be influenced by both poverty and regional differences (Social Forces, 1986)

...

This study examines the influence of occupational characteristics on the early retirement of men. The results indicate that there is some age-grading of occupational "attractiveness" such that occupational characteristics gain or lose their direct salience for retirement depending on the age of incumbents. In addition, when the nature of work is controlled, the influence of pension coverage declines, suggesting that past research may have overestimated the influence of pecuniary benefits. (Social Forces, 1986)" The results show that the scarcity of employed black men increased the prevalence of families headed by females in black communities. In turn, black family disruption substantially increases the rates of black murder and robbery, especially by juveniles (American Journal of Sociology, 1987)

...

All of these quotes contain unwarranted conclusions about causation, since in no case is the evidence conclusive for the cause-and-effect relationship that is asserted to exist. Obviously, it is not hard to find such statements in the social science literature, but just how common are they? To estimate their frequency, I examined all of the papers reporting quantitative causal analyses published in 1985, 1986, and the first half of 1987 in the American Sociological Review, the American Journal of Sociology, Social Forces, the American Political Science Review, the Journal of Politics, and the American Journal of Political Science. I found at least one statement of causation that I considered unwarranted in more than two-thirds of the articles, and about half of them had several such statements. Untentative assertions about cause and effect were somewhat more common in the political science than in the sociology journals but were about equally common in papers reporting research with simpler methods (such as ordinary least-square regression) and in those reporting research with more complex techniques (such as structural equation models). The exact frequencies are not important; persons who agree that almost all statements about causation based on social-scientific research should be tentative would disagree with my judgments about some of the statements,

130

Glenn

and the statements that I classified as unwarranted vary greatly in how far they deviate from the kind of statements that I think should have been made. For instance, the evidence for some of the statements based on the findings of panel or time-series studies is quite persuasive, so that a fairly, though not completely, confident conclusion about causation is warranted. In some cases, there are cautions in the same paper that the evidence is not conclusive, even though I judged that the immediate context did not make clear that a categorical assertion of cause and effect was not intended. Some of the conclusions are based on tests of carefully specified causal models, while others are not. One of the most serious kinds of unwarranted causal inferences is that based on a bivariate relationship. Such inferences are quite rare in articles in the leading social science journals (I found no clear-cut case of one in the six journals I examined), but they are much less rare in textbooks and journalistic treatments of social science findings. To locate an example, I needed only to turn to the edited version of a short paper 1 recently wrote for Psychology Today (Glenn, 1987). In it, I reported that the relationship (zero-order) of marital status to reported happiness declined substantially in the United States from 1972 to 1986 according to data from national surveys. In the original version, I pointed out that it has not been established that being married has, on the average, contributed to being happy, although such a causal link is likely. In the published version (for which I take responsibility, since I approved the editing over the phone), the report of the decline in the strength of the bivariate relationship is followed by the following sentence: “Why is marriage no longer the source of happiness it once was?” This, of course, implies that a one-time positive effect of being married on happiness is not in doubt, whereas the strong relationship between the two variables may have been spurious or may have resulted from the selection of happier persons into marriage and of unhappier ones out of it. Untentative causal conclusions based on bivariate relationships are fairly common in journalistic reports of social research and in textbooks, and often the primary sources-the reports of research or syntheses of research findings - incline journalists and textbook writers to make such conclusions. A conspicuous example is Jessie Bernard’s treatment of “his” and “her” marriages in her influential book The Future ofMarriage (Bernard, 1972). Bernard reviewed the findings of several studies that showed that certain symptoms of psychological distress were more common among married than unmarried women, but that no such relationship existed, or that it was reversed, among men. Furthermore, some of the symptoms were found to be more frequent among married women than among married men.

What We Know, What We Say We Know

131

These differences may be explained by (1) differential effects of marriage on men and women; (2) sex differences in the selectivity into and out of marriage; or (3) a combination of these two kinds of differences. There are reasons for expecting both kinds of differences, but the reasons are perhaps more obvious and clear-cut for expecting the sex differences in selectivity into and out of marriage that would account for the findings. There is a cultural ideal that in regard to a number of quantitative characteristics - including height, amount of education, intelligence, and earning power-the husband should be superior to the wife. Perhaps largely due to this ideal, the selectivity of males and females into marriage has been different in several respects, and it would hardly be surprising if it were found to be different in terms of mental health as well. Bernard considers the possibility of such a sex difference in selectivity, but then she dismisses it in a rather cavalier fashion: We have gone out of our way to pay our respects to the selective factors in explaining the grim mental-health picture of wives precisely because we do not consider them of great importance. For, actually, they have slight weight compared to marriage itself since, sooner or later, practically everyone marries. We are now free, therefore, to explore whatever it might be about marriage itself that could also contribute to an explanation. (1972, p. 36)

Bernard goes on to conclude that marriage tends to produce psychological distress among women but that it tends to prevent the distress among men. However, the fact that “practically everyone marries” is not sufficient reason for ruling out the selectivity explanation, because there can be selectivity out of marriage as well as into it, and characteristics that only delay marriage rather than prevent it from ever occurring may affect the data on the frequency of symptoms of psychological distress by sex and marital status. Clearly, no dogmatic conclusion about how marriage differentially affects men and women is warranted on the basis of the evidence presented by Bernard. Nevertheless, in the few years following the publication of Bernard’s book, that marriage is good for the mental health of men but bad for that of women was reported in many textbooks as proven fact. After a literature critical of Bernard’s conclusion appeared, textbook writers generally became more careful about making conclusions about any difference in the average effect of marriage on the mental health of men and women, but a few untentative conclusions based on Bernard’s evidence can be found in recent textbooks. For instance:

Glenn

132

In general, the effects of marriage are beneficial for husbands and harmful for wives. Women are more attracted to marriage than men but once they are in it end up getting less out of it than men . (Zinn and Eitzen, 1957, p. 238)

. .

Yet studies have shown that women, despite their greater faith in marriage, generally gain less from it than men. Jessie Bernard (1972), who has compared the physical and mental health of single and married Americans, notes that as a group married men are healthier than single men, whereas married women have a higher percentage of health problems than single women. (Popenoe, 1983, p. 182) Jessie Bernard (1972), a prominent sociologist, has suggested that there are actually two marriages: his and hers. In other words, husbands and wives often experience marriage very differently. Bernard points out that married women have higher rates of depression and illness than single women, while married men experience fewer depressions and illnesses than single men. These differences stem at least partly from the traditional sex roles we learn .; wives are socialized to meet the emotional needs of their husbands rather than focusing on their own needs. (McCubbin and Dahl, 1985, p. 155)

..

Another serious kind of unwarranted causal conclusion is the negative one, the conclusion that a certain kind of effect does not exist in the real world. Especially serious, but not common in the journal issues I examined, is the negative conclusion that seemingly reflects the author’s being oblivious to Type I1 errors, as when a dogmatic conclusion of no effect is based on the fact that the statistic that estimates the effect is slightly short of being statistically significant .12 Other kinds of negative untentative causal conclusions are fairly numerous in the journals, and they, too, are serious, because when a less than huge effect exists in the real world, the odds are against its being discovered by any one quasi-experimental study. This is true not only because of Type I1 errors but also because of the crudeness of measurement of most variables in social research and the high probability that important causal variables will not be taken into account at all. Consider the following examples from recent issues of leading social science journals: Popular presidents tend to have positive effects, while unpopular presidents do not . . (American Political Science Review, 1987)

.

However, during the 1970s, population growth disappeared as a determinant of status change . (American Sociological Review, 1987)

..

But identity summation does not consistently reduce gender or gender by marital status differences in distress . , . (American Sociological Review, 1986)

What We Know, What We Say We Know

133

We conclude that non-family living affects some but not all dimensions of global attitudes. (American Sociological Review, 1986)

Some dogmatic causal conclusions are unusually serious because the evidence they are based on is ambiguous for reasons other than the usual dangers of specification, measurement, and sampling errors. This is the case, for instance, when the effects of different variables are likely to be confounded with one another. Such a situation exists when one of the independent variables in an analysis is a linear function of two other independent variables, so that if the different variables all have linear effects, those effects will be confounded with one another. The potential for the confounding of effects in such a situation is known as the identtfication problem (Blalock, 1966; Fienberg and Mason, 1985). One example of the identification problem is in research designed to estimate the effects of vertical mobility. Vertical mobility is a linear function of stratum of origin and stratum of destination; stratum of destination is a linear function of stratum of origin and vertical mobility; and stratum of origin is a linear function of stratum of destination and vertical mobility. It follows that the researcher cannot place all three variables in a regression equation (or use them in any other kind of analysis) to estimate their separate effects on a dependent variable. If one tries to do that, without making some identifying restriction to break the linear dependence among the independent variables, the computer program will not run. If one makes an arbitrary identifying restriction, and if the effects are entirely or largely linear, the computer program will run, but the estimates of effects will not be meaningful. Another example is cohort analysis, in which researchers conceive of separate effects of age, period, and cohort, even though any one of those variables is a linear function of the other two (Fienberg and Mason, 1985). Since 1973, a method introduced by Mason, Mason, Winsborough, and Poole (1973) has often been used as a mechanical solution to the identification problem (that is, as a way to get the computer program to run), and most of the users have ignored the authors’ caveat that the method is applicable only when all effects are nonlinear.13 The inherent ambiguity of cohort data when the effects are linear is illustrated by the hypothetical data in Table 1. The table is a standard cohort table, in which different cohorts can be compared at each date by reading down the columns, in which different cohorts at the same age level can be compared by reading across the rows, and in which each cohort can be traced through time by reading down and across the diagonals, from upper

Glenn

134

Table 1. Hypothetical Data Showing the Pattern of Variation in a Dependent Variable Predicted by Pure Linear Cohort Effects, a Combination of Linear Age and Period Effects, or Various Combinations of Linear Cohort, Age, and Period Effects Age 20-29 30-39 40-49 50-59 60-69 70-79

1940

1950

1960

1970

1980

50

45 50 55 60 65 70

40 45 50 55 60 65

35 40 45 50 55 60

30 35 40 45 50 55

55

60 65 70 75

left to lower right. It follows that the variation in the values of the dependent variable in each column could reflect cohort or age effects, or both; the variation in each row could reflect cohort or period effects, or both; and the variation in each cohort diagonal could reflect age or period effects, or both. Since the values are constant in each cohort diagonal of Table 1, the simplest and most obvious interpretation is that the data in the table reflect pure linear cohort effects. However, there are alternative interpretations, the possibilities being as follows. 1. The table reflects pure linear cohort effects, whereby each successive ten-year cohort that has reached adulthood has had a 5-point lower value on the dependent variable than the next older cohort. 2. The table reflects a combination of linear age and period effects, whereby ten years of aging has an effect on the dependent variable of + 5 but the passage of ten years of time has an effect of - 5 , thus keeping the values for the dependent variable constant within each cohort.14 3, The table reflects a combination of linear cohort, age, and period effects, whereby each successive ten-year cohort that has reached adulthood would have had a 2-point lower value on the dependent variable than the next older cohort in the absence of age and period effects, ten years of aging has an effect on the dependent variable of + 3, and the passage of ten years of time has an effect of - 3. Other combinations of cohort and of equal and opposite age and period effects could have the same empirical results. It should be obvious (although it has not been to many otherwise sophisticated researchers) that if one has only the data in Table 1 and no a priori knowledge of the magnitude and direction of age, period, and

What We Know, What We Say We Know

135

cohort effects on the dependent variable, no statistical analysis can identify the effects reflected in the table. Rather, any reasonable conclusions about the nature of the effects must be based partly on what Converse (1976) has called “side information,” that is, information derived from sources other than the cohort table.15 Any mechanical attempt to separate the effects statistically (such as applying the Mason et al. (1973) method with a minimum identifying restriction) may result in estimates that lack even a crude correspondence to the real-world age, period, and cohort effects.16 To come to dogmatic conclusions about effects from such an analysis is an egregious mistake, and unfortunately, it is a mistake frequently made by social scientists who encounter the identification problem in their research. An example is found in the lead article of a recent issue of the American Journal of Sociology (Sasaki and Suzuki, 1987), which reports a study in which the authors estimated age, period, and cohort effects from data on church attendance and other indicators of religious commitment in three countries. The method they used is more complicated than the Mason et al. (1973) method, but it is unnecessary to describe it in detail to show that it is inadequate. It makes no use of theory or of side information; rather, the identification problem is “overcome” by the purely arbitrary assumption that “successive parameters change gradually” (p. 1063). According to the authors, the method “can provide a satisfactory explanation for the data almost automatically” (p. 1063). In other words, the method is used in a completely mindless and mechanical fashion. It may provide meaningful estimates when all effects are nonlinear, but the issue of the linearity or nonlinearity of the effects is not addressed by the authors, and most of their data fall into an approximately linear pattern. Nevertheless, they state many dogmatic conclusions about age, period, and cohort effects: From 1958 to 1973, historical period had negative effects on religious belief (p. 1069)

...

..

Strong age effects. indicate that the aging process tends to influence the Japanese people to become progressively more religious, with the younger Japanese being potential religious believers. (p. 1069) Japan, however, shows the strongest age effects on religious belief among the three countries examined, with the aging process tending to influence people in Japan to become progressively more religious. (p. 1073)

The authors of another recent article in a leading sociological journal are similarly incautious in their conclusions about effects on the basis of an only slightly less mechanical approach to dealing with the identification

136

Glenn

problem, in this case as it arose in a study of mobility effects (Brody and McRae, 1987). The authors used the Mason et al. (1973) method and made the assumption that the effects for immobile and slightly upwardly mobile persons were the same (zero), their reasoning being that since the identifying restriction could hardly be a gross distortion of reality, their estimates of effects would be approximately correct, However, their reasoning is wrong unless all effects were nonlinear, which is unlikely. On the basis of their estimates, they arrived at the highly dubious conclusion that both upward and downward mobility lead to improved mental health. Although they confess to be puzzled about the reasons for the estimated effects, they do not consider that their estimates of effects might be wrong. In their words, “At least now we are confident that mobility affects men’s sense of competence, their level of anxiety, their affective states, and their satisfaction with various facets of their lives” (p. 222). Of course, their confidence is misplaced.17 In research in which the identification problem is encountered, conclusions should be quite tentative even if a variety of side information is used to help interpret the data, and it is my impression that authors who bring side information to bear on their data do usually keep their conclusions appropriately tentative. If so, it is hardly surprising that those researchers who know that side information is needed also generally know that dogmatic conclusions are not warranted. However, it is ironic that the less adequate the evidence is, the more confident the conclusions tend to be. My purpose in this section of the chapter has been to demonstrate that unwarranted causal conclusions abound in the social science literature but that their seriousness varies greatly, since the evidence supporting them ranges from highly persuasive to so weak that not even tentative causal conclusions should be based on it. Recommendat ions

If, as I claim in this chapter, social scientists very often make unwarranted causal conclusions and their doing so has undesirable consequences for the social sciences and raises ethical questions, then changes are needed in the way in which social science findings are typically reported and interpreted. A necessary condition for such a change is improvement in the training and socialization of social scientists in graduate school, where all too often students get the impression that the way to truth is through the mating of a good computer program with a good data set. The place of judgment calls, subjectivity, and theoretical insights in even the most rigorous of quantitative research is often not emphasized enough, and students

What We Know, What We Say We Know

137

often do not become sufficiently sensitive to the inherent limitations of quasi-experimental research.18 Few social science graduate programs provide systematic training in writing and in the reporting of the results of research, but they should do so, and the training should include a strong emphasis on the proper use of causal language. In that training, the goal of clear, straightforward writing should not be given priority over the ethical responsibility of the researcher not to claim to know what he or she does not know. A few social science departments offer courses for journalism students on the reporting of social science research findings, and those offerings should be expanded. It is not too much to expect a good reporter to know that “correlation does not equal causation” and that no one study in the social sciences can establish with certainty what causes what. Finally, social science journals should publish guidelines for the use of causal language and should edit out unwarranted causal statements in the papers they publish. Most fledgling social scientists learn to write for the journals primarily by reading the journals, so that faulty reasoning and misleading terminological conventions will tend to be self-perpetuating until editors take action to stop them.

Notes 1. A notable example, which I discuss below, is in a book on writing for social scientists by Howard S. Becker (1986). 2. For generally favorable commentary on the Lieberson book, see Arminger and Bohrnstedt (1987) and Singer and Marini (1987). 3. An awareness of mistakes made by social scientists in the past should have a sobering effect on those who do social research today. We may avoid making the same mistakes, but can we be sure that we are not making other, as yet undetected, errors? 4. Here I use “tend to,” one of the phrases disliked by many critics of social scientists’ writing. However, my sentence would not convey the meaning I intend if that phrase were omitted, because I know that the socialization does not always produce irresponsibility and dishonesty. Indeed, it is rare that any influence studied by social scientists always produces a certain effect, and if it sometimes but not always does so, the use of “tends to” is an accurate and concise way to describe what happens. 5 . A “noncausal effect” is a statistic of a kind that is often used as an estimate of a causal effect (e.g., a regression coefficient) but that, in the case in point, is used only as a measure of association. 6. The authors mean that in the statistical model, the statistic that estimates the age effect is statistically significant. However, since this statement seems to be a conclusion about the real world, significant seems to mean “important” rather than “statistically significant.” 7. These persons include students, journalists, and many textbook writers, among others. 8. The phrase “in both samples” is a clue that the author is referring to model effects rather than real-world effects, yet reduce is not a word that would, according to any widely accepted convention, be used to describe a statistical model. 9. In this case, the evidence for real-world effects from the well-conducted study reported in the article is quite persuasive, but it is not conclusive.

138

Glenn

10. Again, the evidence for real-world effects is strong, since the researchers had longitudinal data that enabled them to deal with some selectivity effects. But again, the evidence is not conclusive. 11. The reference to statistical controls and the decline of influence after a variable was controlled is a pretty strong clue that the author is talking about a statistical model. However, the word influence is not a statistical term. 12. Of course, a Type I1 error is the failure to reject the null hypothesis of no difference or no association when an association exists in the population. 13. When the method is used, age, period, and cohort are each converted into a set of dummy . . variables, and the linear dependence among the three is broken by contraining the effects of two or more categories of one or more of the variables to be equal to or to bear some other specified relationship to the other@). Usually, the effects of two adjacent age levels, periods of time, or cohorts are assumed to be equal. That the equal-effects assumption applied to two adjacent categories of one of the variables is unlikely to be a gross distortion of reality apparently accounts for the seductiveness of the method. Many users apparenlly do not realize that even if the assumption is precisely correct, the method cannot meaningfully separate truly confounded effects (those that are linear). 14 One might think that such a combination of age and period effects is improbable, but it is not especially. Consider the case of aging and conservatism. If the conventional wisdom that growing older tends to make people more conservative is correct, then during a period when the general change is in a liberal direction, the opposing age and period influences may approximately offset one another within aging cohorts, leaving them neither appreciably more nor less conservative. For another plausible example, see Glenn (1981). 15. The way in which the side information can be used varies, but in almost all cases, it can be used to assess the credibility of estimates from statistical analyses. For instance, when the dependent variable is amount of formal education, applying the Mason et al. method and assuming that the effects of two adjacent cohorts are equal will lead to estimates that show marked negative age effects. Since we know that an individual’s amount of education can never decrease and that neither selective mortality nor selective immigration or emigration leads to a decline in education within aging cohorts, we know that the estimates cannot be correct. In other cases, the side information may indicate that one kind of effect is absent (or is very nearly constant across categories), in which case the other two variables can be entered into the analysis and their effects estimated in a straight-forward manner. For an illustration of the use of side information in a cohort analysis, see Glenn (1981). 16. For illustrative data showing how inaccurate the estimates of effects may be, see Glenn (1981). 17. It is misplaced not only because the method used cannot definitively separate origin, destination, and mobility effects but also because the authors assume that origin, destination, and mobility affect psychological functioning recursively. In fact, there may be important effects of psychological functioning on mobility and thus on destination. 18. If all social science graduate programs were to provide the training I recommend, a good many students with a high need for certitude might decide not to become social scientists. If so, both those individuals and the social sciences would benefit.

References Achen, Christopher H. (1986). The Statistical Analysis of Quasi-Experiments. Berkeley: University of California Press. Arminger, Gerhard, and Bohrnstedt, George W. (1987). Making it count even more: A review and critique of Stanley Lieberson’s Making I t Count: The Improvement

What WeKnow, What WeSay WeKnow

139

o f Social Theory and Research. In Clifford Clogg (ed.), Sociological Methodology: 1987, Vol. 17. San Francisco: Jossey-Bass. Blalock, Hubert M., Jr. (1966). The identification problem and theory building: The case of status inconsistency. American Sociological Review 31 :52-61. Becker, Howard S . (1986). Writing for Social Scientists: How to Start and Finish Your Thesis, Book, or Article. Chicago: University of Chicago Press. Berk, Richard A. (1986). Review of Stanley Lieberson, Making It Count. American Journal of Sociology 92:462-465. Bernard, Jessie. (1972). The Future of Marriage. New York: World Publishing. Boruch, Robert F. (1976). On common contentions about randomized field experiments. In G. V. Glass (ed.), Evaluation Studies Review Annual, Vol. 3, Beverly Hills, CA: Sage. Brody, Charles A , , and McRae, James A. (1987). Models for estimating effects of origin, destination, and mobility. Social Forces 66:208-225. Campbell, Donald T.,and Erlebacher, Albert. (1975). How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful. In Elmer L. Struening and Marcia Guttentag (eds.), Handbook o f Evaluation Research, Vol. 1. Bevery Hills, CA: Sage. Campbell, Richard. (1987). Review of Stanley Lieberson, Making It Count. Social Forces 65:905-906. Converse, Philip E. (1976). The Dynamics of Party Support: Cohort-Analyzing Party Identification. Beverly Hills, CA: Sage. Cook, Thomas D., and Campbell, Donald T. (1979). Quasi-Experimentation. Chicago: Rand-McNally. Costner, Herbert L. (1986). Research methodology: Pedagogy, criticism, and exhortation. Contemporary Sociology 15:537-540. Duncan, Otis Dudley. (1984). Notes on Social Measurement: Historical and Critical. New York: Russell Sage. Fienberg, Stephen E., and Mason, William M. (1985). Specification and implementation of age, period, and cohort models. In W. M. Mason and S. E. Fienberg (eds .), Cohort Analysis in Social Research: Beyond the Identification Problem. New York: Springer-Verlag. Glenn, Norval D. (1977). Cohort Analysis. Beverly Hills, CA: Sage. Glenn, Norval D. (1981). Age, birth cohort, and drinking: An illustration of the hazards of inferring effects from cohort data. Journal o f Gerontology 36:362-369. Glenn, Norval D. (1987). Marriage on the rocks. Psychology Today, October, pp. 20-21. Lieberson, Stanley. (1985). Making It Count: The Improvement o f Social Research and Theory. Berkeley: University of California Press. Mason, Karen O., Mason, William M., Winsborough, Hal H . , and Poole, W. K. (1973). Some methodological issues in cohort analysis of archival data. American Sociological Review 38:242-258. McCubbin, Hamilton, and Dahl, Barbara Blum. (1985). Marriage and Family: Zndividuals and Life Cycles. New York: Wiley. Popenoe, David. (1983). Sociology (5th ed.) Englewood Cliffs, NJ: Prentice-Hall.

140

Glenn

Sasaki, Masamichi, and Suzuki, Tatsuzo. (1987). Changes in religious commitment in the United States, Holland, and Japan. American Journal of Sociology 92: 1055-1076. Singer, Burton, and Marini, Margaret Mooney. (1987). Advancing social research: An essay based on Stanley Lieberson’s Making It Count. In Clifford Clogg (ed.), Sociological Methodology: 1987, Vol. 17. San Francisco: Jossey-Bass. Zinn, Maxine Baca, and Eitzen, D. Stanley. (1987). Diversity in American Families. New York: Harper & Row.

Comment

Causal Inferences: Can Caution Have Limits? Susan We Ich

N

Orval Glenn has offered an important reminder of the limitations of social science methods in verifying causal relationships and of the tendency of some to ignore these limitations in reporting their research. His well-chosen examples from our leading journals provide a cautionary note to us all in our own scholarly writing as well as a warning in evaluating the work of others. His suggestions for more attention in graduate training to the proper use of causal language are certainly appropriate and timely. What I will explore in this comment are the conditions under which causal generalizations may sometimes be overdrawn in a useful, perhaps even necessary, way. Without denying Glenn’s proper admonitions, there are relevant issues that may shed light on why statements about causality are overdrawn, and we can see that sometimes such generalizations are appropriate. Inappropriate Causal Inferences

Most of the examples Glenn provides are illustrations of applying badly or misapplying the techniques, or rules of thumb, we know or claim to know about causal inferences. There are at least two reasons for this shortcoming. One may simply be ignorance. A good example is offered by Glenn in describing the mechanical use of a methodological procedure purporting to disentangle the effects of age, period, and cohort effects with apparently no understanding of the assumptions underlying that method. As data manipulation methods increase in sophistication, novice users may understand only the “how to” of the method without appreciating its limits and assump- zyxwvutsrqp 141

142

Welch

tions. Alas, as has long been lamented, computers will usually crank out solutions even if the assumptions on which the numbers are based are totally erroneous. Another reason for inappropriate causal inferences in scholarly journals is carelessness or inattention. In these cases, the problem is not that as methodologists we do not know the proper terminology of covariation rather than causation, since in graduate school most of us did learn that correlation is not causality. Rather, through oversight, sometimes we forget or ignore these rules. Thus, some may draw inappropriate conclusions about causality in their own work, even though they might very well recognize the inappropriateness of such statements in reviewing and evaluating work by others. Of course, the responsibility for allowing such statements to find their way into print rests primarily with the carelessness or ignorance of the author, but reviewers and journal and book editors must share some blame. After all, critical evaluations of the examples offered by Glenn should have noted the inappropriate causal statements. Thus, it appears that collectively, we need reminders to keep us alert to the limitations of our methods. A second set of examples of the inappropriate use of causal language comes from popularizers, textbook writers, and popular journalists who overgeneralize findings or ignore causality entirely. Sometimes careless popularizers may take carefully limited generalizations and make them global, with no acknowledgment of the difference and perhaps no realization of it. When summarizing research done at a particular time, other popularizers may ignore research that has refined and sometimes refuted it at a later time. When Otherwise Inappropriate Causal Assertions Must Be Made

But sometimes causal statements and overgeneralization do not stem from simply ignoring what we know or from textbook writers and popular journalists trying to oversimplify reality; rather, they stem from the juxtaposition of the limitations in our methods and the requirement that social scientists provide statements about social reality to an audience of policymakers, concerned citizens, or students. The limitations of our methods are clear. More often than not, we rely on nonexperimental methods or quasi experiments to test hypotheses. From these research designs, generalizations are drawn which do not warrant causal inferences. Making them in the context of scholarly articles cannot be condoned. Nonetheless, under some circumstances, such inferences must be made. These circumstances arise when we, as social scientists, are called

Comment

143

upon to present the results of our expertise to others seeking to understand a particular social phenomenon or to reach some conclusion about appropriate public policy. If we, as social scientists, are to be relevant to anyone but ourselves, we must sometimes be forced to draw generalizations that would not be appropriate in scholarly writing. We often cannot wait for an improvement in the data or an increase in the sophistication of our methods in order to offer some opinion about the effectiveness of a policy or the cause of some social problem. When social science research is relevant to important public issues, it is incumbent upon us to offer advice or information based on the best research that we currently have, even if it is not perfect. Of course, it should go without saying that such information should be offered only in conjunction with information about the limitations on what we know. In providing such information, it is obvious that we are leaving ourselves open to criticisms that we are going beyond both our data and our methods. Nevertheless, we must provide the best information we can. If we do not, if we stand aside awaiting improvements in our data and methods and more certain causal evidence, we cannot assume that others, who have even less information than we, are going to be equally reticent. After all, the absence of systematic evidence from social scientists does not mean that an issue or policy will be ignored; rather, generalizations will be drawn about that issue or policy on much more limited and less systematic evidence than ours. For example, the Coleman studies, concluding that busing is useful because integrating black schoolchildren with white schoolchildren helped the former's academic achievement without hurting the latter, were flawed. Yet, offering that conclusion and the evidence on which it was based for public and scholarly debate was preferable to reaching conclusions about busing only on the basis of ideology, prejudice, or one's own idiosyncratic experiences. Moreover, such generalizations, when widely publicized, offer an appealing target for other researchers, who may be able to improve upon the original design, to specify more carefully the conditions under which a given generalization is valid, and in other ways to move the state of knowledge forward. This is certainly an important part of the scientific enterprise.

The Problem of "Popularizers" While some popularizers, such as textbook writers and journalists, seem blithely unaware of the causal inferences that can be appropriately made from quasi- or nonexperimental research, others are aware but make unwarranted inferences anyway. This, too, seems defensible under certain

144

Welch

conditions. For example, as a recent author of a textbook on American government, I struggled with the dilemma of, on the one hand, qualifying practically every sentence where research findings are concerned and, on the other, writing in a way that makes political science findings sound as meaningful and important as, in fact, I think they are. I do not think one can write a readable text if almost every finding is qualified, as it technically should be, with such phrases as “on the whole,” “generally speaking,” “under certain conditions ,” “has a tendency to,” and “appears to cause.” Seven hundred pages of this would rightfully lead a student to conclude that social science is not a discipline that deals much with the real world, a world where people have to make judgments about actual events. This, of course, does not excuse those authors who continue to portray certain findings as true that have long since been shown to be flawed. Nor does it mean that authors should wrongfully convey a sense of certainty about findings that are anything but certain. However, freshmen can probably be introduced to uncertainty and conflict in social science in more fruitful ways than by the repetitive use of qualifications. One way is by discussions of important issues on which social scientists disagree, with some explanations about the methodology, data, and ideology that have led to those disagreements. Another way is to examine important issues about which social science knowledge is particularly weak and to offer some explanations of why more progress in research on the issue has not been made. A third way is to accompany the discussion of findings, such as those of Jessie Bernard about “his” and “her” marriages that were cited by Glenn, with explanations of the methods used to generate these findings. A cogent discussion of the problem of separating the real effects of marriage from the effects of selectivity in and out of marriage could reveal more to the student about social science methods and their limitations than simply ignoring the flawed findings altogether, or casting them in a very tentative light. Some Final Thoughts

The low esteem in which social scientists are sometimes held by those in the “harder” sciences does not have much to do with the problems discussed here. After all, the medical journals are filled with quasi experiments based on sample sizes so small that no social scientist would touch them. And the results of these experiments are often portrayed, at least by medicine’s own popularizers, as definitive when, in fact, they are far from it. In addition, the reputation of research in the physical sciences has recently suffered because of a series of revelations about research findings that are entirely fabricated. Therefore, while social science clearly has a long way to go to

Comment

145

improve its theory, methods, and analysis, I hardly think we need be embarrassed among our peers from other disciplines. This does not negate the main thrust of Norval Glenn’s argument, that many of us are too eager to overlook the flaws in our methods and the limits on our generalizations and causal inferences. His chapter is useful because it allows us, perhaps even forces us, to stop and think about not only how far we have come in our methods and analysis techniques but also how far we have to go so that our methods will enable us to deal with the issues we would like to resolve. Glenn offers three practical suggestions for improving graduate education and the social science training of journalists and for establishing disciplinary guidelines for the use of causal language in our scholarly journals. Another suggestion I would offer deals with the impact of new and ever more sophisticated methods on our ability to understand their limitations. One way to deal with the gap between increasingly sophisticated methods and the ability of many to use these methods appropriately is to increase the number of methods “popularizers” (for want of a better term) in our field. There is a gulf, probably growing, between those methodological sophisticates who have designed solutions to a number of methodological problems, such as those inherent in quasi-experimental designs, and those people with substantive interests who might fruitfully use those methods. Sometimes novices (as well as those who are not such novices) either misuse the sophisticated methods or continue to explore hypotheses using inappropriate tried and supposedly (but not necessarily) true methods. The ICPSR’s summer program is one attempt by the social science disciplines to close this gap. But clearly, more needs to be done, as our sophisticated methods escalate faster than our understanding of either the methods or the issues.

Chapter 5

Research Life as a Collection of Intersecting Probability Distributions Warren E. Miller

Thoughts on the Improbable

A

mong those of you who have suffered with me the trials and tribulations of proposal writing and money raising, there will be many who recognize my conviction that the events that constitute life, at least the life of the research administrator, form a series of intersecting probability distributions. It is difficult to alter the shape of even a single distribution, changing the mean probability of receiving a grant or reducing the variance among repeated proposals written to fund a given project. And a very few of you may already have heard my occasional conclusion that some of the crucial events in the history of the development of political behavior research were so improbable that they probably really didn’t occur at all. For example, the first major effort of The Inter-University Consortium for Political and Social Research (ICPSR) to organize a systematic collection of nonsurvey data was facilitated by one such totally improbable occurrence. The context for the event was the somewhat more predictable set of circumstances in which the Ford Foundation in the 1950s and 1960s had developed the admirable habit of funding research in the behavioral sciences. Through a thoroughly explicable set of circumstances, Angus Campbell and I had used the good offices of Malcom Moos, then an assistant to the president of the Ford Foundation, and were lunching with a foundation program officer who might have an interest in salvaging what was then known as the Lord Collection, a set of election returns and maps for congressional elections going back to the time of the country’s founding. The collection had originally been assembled as a Works Progress Administra147

148

Miller

tion effort to employ indigent lawyers during the depths of the Depression. They had painstakingly copied election returns from otherwise fugitive records, and their tabulations, along with an accompanying set of maps of congressional districts, had for many years languished in the attic of the Butler Library at Columbia University. As I explained the situation to the foundation officer, I concluded with a plea for funding to bring the data, if not the maps, into ready access for scholars for whom the materials would be an invaluable resource for the study of political history. Despite my youthful enthusiasm and nodding approval and support from Campbell and Moos, our lunch companion’s reaction consisted essentially of a restrained comment to the general effect, “I can’t imagine our board supporting it, but let me think about it”; in any event, he said, we should not plan to ask for more than $200,000 under any circumstances. My rejoinder was immediate; “Fine, I will put together a proposal for that amount, and I am sure it will be enough to get the job done.” The luncheon ended, however, on this note: “Don’t bother to write a proposal; it’s not likely to get support. I will explore the matter and get back to you if the prospects seem promising.” Thus far everything was thoroughly predictable, including the “Don’t call us, we’ll call you” conclusion to our meeting. Time passed. I am not even sure that I remembered to wonder about a date for reaction from the foundation. However, eventually, on a Wednesday afternoon, I answered a call from the same foundation program officer, who, with a mixture of embarrassment and petulance, asked if I could send him another copy of our proposal inasmuch as he had apparently misplaced his copy. In some bewilderment, I reassured him that he had not lost the original because, pending a “go-ahead” signal from him, we had not even submitted a written sequel to the luncheon meeting of many months ago. I was then told in no uncertain terms that I clearly had been negligent because, as promised, the problem had been discussed formally and, as expected, had been enthusiastically received, and here he was, now literally a week away from the foundation’s board meeting, and he had no proposal to distribute. I assured him that he would have a proposal on his desk by Monday morning, and I asked for guidance as to format, length, and other technical details. I was then told the proposal need not be elaborate, just to make sure that I accounted for the planned expenditure of the $500,000-that was the figure, wasn’t it? I agreed without hesitation that his luncheon table figure had been $ 5 00,000. We asked. We received. And I presume the optimism which carried me

Research Life and Probabilities

149

through the next two decades had its confirmation, if not its birth, with that $300,000 error of memory. An error of that magnitude in favor of the folks with the white hats was so rare that it has not been repeated in my presence in all the intervening years. Another event which, at the time, seemed equally improbable was the proximate cause for the birth of the consortium and the death of the oftencited but never presented Miller/Stokes volume on representation. The story is no shorter, but it is equally to the point. The idea for the consortium, in at least its rudimentary form, had been with me for at least two years before I was invited to spend a year at the Center for Advanced Study in the Behavioral Sciences at Palo Alto. However, despite the overwhelming plausibility of the idea, I had been totally unable to find any funding for the organizational start-up costs. In the meantime, local colleagues in the Economic Behavior Program within the Survey Research Center had responded very positively to the ideas in the various memoranda that I had circulated and the various ineffectual proposals that had been approved for my submission. They picked up the basic notions of the consortium and carried them to the Ford Foundation, and I left for Palo Alto with the double burden of knowing that the Ford Foundation had granted $350,000 for the purpose of transforming the Survey of Consumer Finances, by then already a 15-year time series of microeconomic data, into a machine-readable archive for use by the national community. The use of the data would be facilitated by a summer program of training in survey research methods and data analysis. Convinced that economics and not political science was to be the intellectual locus for the grand new social-science resource that I had in mind, I departed for the center free of all obligations other than those associated with completing the data analysis for our representation study. No sooner had I arrived at the center than I received word that an almost forgotten inquiry to the Stern Family Fund had produced a $30,000 grant with which to launch the Inter-University Consortium for Political Research. Quite contrary to the letter, but I think not the spirit, of the guidelines for activity at the center, my year there was spent not in data analysis nor the production of scholarly manuscript, but in the artful composition of letters that would ultimately result in the organizing meeting that we are now celebrating. Once again, an improbable grant led to what many close friends thought to be another improbability: the formation of what came to be the Inter-University Consortium for Political Research. (Explicit recognition that all significant social research was not necessarily political research occurred only much later, as ICPSR made manifest in the title the

150

Miller

grand imperial designs of the founding of ICPR.) The idiosyncratic nature of that beginning is possibly reflected in the fact that that was the first and only funding I or we ever received from the Stern Family Fund. As a third example of the improbable event that leads to critically important consequences, I am reminded of the origin of the Center for Political Studies, which has been so important in the fostering of the consortium and the development of a whole series of research programs, not the least of which is reflected in the National Election Studies. In the mid-l960s, when ISR was temporarily housed in the old brewery on Fourth Street, the wise counsel of Angus Campbell restrained us from joining in what was to be known as Project Camelot. Camelot, as a few of you may remember and as more of you may have heard, was one of the early tokens of the CIA’S interest in basic social research intended to foster a warmer appreciation of Latin politics. Despite a commitment to comparative research, of which I will say more in a moment, our involvement in data collection in Latin America waited for almost another decade. In 1969, with Camelot still fresh in many memories, the highly improbable event took place: the late Kalvin Silvert was able to persuade his colleagues within the Ford Foundation (once again) to commit a large amount of money to an attempt to export to Latin American scholars an example of the workways associated with our American studies of mass political behavior. An emphasis on training in modern research techniques was not surprising; the foundation had been promoting the behavioral sciences in Latin America and elsewhere for a number of years. It was surprising, however, to have large funds committed to the collection of data in Latin America under the leadership of North American academics. A million dollars for North Americans to do political research under the very noses of a set of governing generals was not an expected aftermath of Camelot. Nevertheless, it was that commitment by the Ford Foundation that permitted us to create the Center for Political Studies and sustain the ongoing research interests of many who are still senior colleagues in the center, while redeeming our pledge to provide both training and indigenous data resources for Latin American scholars. O n Individuals and Institutions

The theme of the improbable could be extended exquisitely in detail while recounting many aspects of our Latin American experience, but I have gone on perhaps too long with the appearance of wide-eyed bewilderment at the strokes of fortune that so unpredictably shaped the course of events surrounding the Center for Political Studies and the ICPSR. Let me turn

Research Life and Probabilities

151 zyxwvutsrqp

instead to what I think is a much more important theme: the interdependence of individuals and institutions, Although it is the institutions that do much to define the means and the variances in the distribution of relevant events, it does not necessarily follow that individuals provide only the idiosyncratic outliers. The first example of the important interdependence is provided by the institution that was, and is, the Social Science Research Council (SSRC), and by the individuals who did so much to shape the course of political science in the immediate period after World War 11. Pen Herring, honored by the American Political Science Association in September as the James Madison Lecturer, more than any other single person, should be credited with providing timely support for the type of research which broadly characterizes the activities of those of us who exploit the resources of the consortium. Under Herring’s leadership, two SSRC committees were established: the Committee on Comparative Politics and the Committee on Political Behavior. The former committee provided the impetus for systematic empirical studies of comparative politics under the leadership of such people as Almond, Ward, and Pye. The Committee on Political Behavior was led by such figures as Campbell, Dahl, Heard, Key, Leiserson, and Truman. The impact of these committees on political science is crudely reflected in the extraordinary proportion of their members who were subsequently chosen as presidents of the American Political Science Association. A less personal and equally important documentation is provided by a roster of the studies that came into being because of one or another of the SSRC committees. On the side of American politics, the first major mass election study in 1952 was funded by the Carnegie Corporation in a grant to the Social Science Research Council. The University of Michigan Survey Research Center, under the leadership of Angus Campbell, was, in turn, the council’s chosen instrument for that first and most important of our studies on mass electoral behavior. Four years later, the role of the committee was less direct, and yet it was clearly their enthusiasm for an extension of the work reported in The Voter Decides that led the Rockefeller Foundation to underwrite the 1956 study that became essential to the creation of The American Voter. It was the council’s Political Behavior Committee that also funded the pioneering work of Wahlke, Eulau, Ferguson, and Buchanan, which produced The Legislative System. Although the Committee passed up the opportunity to fund what doubtless would have been a pioneering study of the role of media advertising in politics, proposed by one Warren Miller, it redeemed its reputation for good judgment by funding him in 1958 for the

152 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Miller

study of congressional candidates and the representation of their constituents’ policy demands which is now (or at least once was) known as the Miller/Stokes study. Even without going beyond this very limited personal and parochial listing of the investments made by the SSRC and its committee, there is no gainsaying the importance of the leadership initiatives made possible by the institutional context provided by the Social Science Research Council. However, one of the next intersections that I would mention connects the same SSRC and its committees’ programs support for behavioral research to the origin of ICPSR. Herb Weisberg, in his excellent review of the consortium summer program contained in the Spring 1987 issue of PS, notes that the summer program of the consortium was preceded by two small summer seminars sponsored by the SSRC Political Behavior Committee and hosted by the Survey Research Center in 1954 and 1958. The seminarians, although perhaps not as illustrious as a group as the SSRC committee itself, included many of the next generation of the discipline’s leaders, such as Bob Lane, Allan Sindler, and Heinz Eulau. Whatever the function of the summer seminars in eventually providing a model for the consortium’s summer program (so well described by Herb in the exact spirit of these remarks), it was the aftermath of each seminar that provided the inspiration for the organization of the consortium. The two experiences made it clear to me that the intensive training in methods and techniques of data analysis which could be accomplished in eight weeks could make a massive difference in the development of the research skills of individual scholars. It also became clear that without technical facilities for the manipulation of data at the home school, the individual scholar was unlikely to be able to exploit this training. With few exceptions, seminar members had returned to Yale or Antioch or North Carolina to discover that there was no infrastructure in place to support their continued use of the skills and the substantive insights produced by the Michigan experience. From this observation came the notion that there must be a new kind of institutional support for scholarship based on the exploitation of empirical materials, the behavioral scientist’s equivalent of the traditional scholar’s library and reference service. It is important to note that the original idea for the consortium was to join the experience with the SSRC summer seminars to another idea which had been promoted in the early 1950s by one of the most creative and important leaders of social science in our generation: Stein Rokkan. With the assistance of one York Lucci, and with the sponsorship - once again - of the Ford Foundation, Rokkan had produced a commissioned report in 1957

Research Life and Probabilities

153

on the feasibility of creating an archive of survey data to facilitate crossnational comparative research. In the late 1950s, the swiftly developing technology of the computer promised new capabilities for the tasks of archiving and disseminating data. Given what we had observed to be widespread interest in the Michigan Election Studies, the Rokkan vision of a new kind of resource for the modern behavioral science research scholar was easily married to the theme of specialized training programs in the methods of research and data analysis for political scientists. In the first instance, the idea for the consortium was very much defined by the interests of political scientists doing research on contemporary political behavior. However, the unique contribution of a number of individual intellectual leaders to the development of consortium activities changed the nature of the consortium in a rather unpredictable way. Their impact on the organization was made possible by support from the apparently ubiquitous Social Science Research Council. On the one hand, with SSRC support, Dean Burnham, then a promising protege of V. 0. Key, Jr., spent a year with us in Michigan. He first promoted and then demonstrated the possibility of retrieving historical county-level election data. His efforts were extended by the novel and ambitious organizational ideas provided by Lee Benson, then a professor of history at Wayne State. Lee’s beginning career at Columbia University had included a close association with the Bureau of Applied Social Research, led by Paul Lazarsfeld, and hence a commitment to the systematic analysis of data for the purpose of testing empirical theory. With his own inimitable brand of optimism, Benson proceeded to help the consortium organize a national network of colleagues in history to join a partnership for the retrieval of heretofore fugitive data. Thus, it was the consortium that supplied the institutional support; it was the Bensons, and then the Bogues, Allens, and Clubbs, who provided the intellectual inspiration for defining the historical archives; and it was the National Science Foundation that provided the money. It should be mentioned that Jerry Clubb’s interests in gerontology and criminal justice and library science were long preceded by a commitment to improving the work of his fellow historians. Indeed, the first of Jerry’s signal administrative contributions to modern social science was provided while his role was that of director of the consortium’s Historical Archives. In that role, he was aided and abetted, as well as sometimes haunted and plagued, by colleagues such as Benson, Allen Bogue, Jim Graham, and Bill Aydelotte. But whatever the ambience of the relationships, it was the interaction of many individuals and many institutions that promoted this phase of the intellectual growth of the consortium. In the beginning, the American

I54 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Miller

Historical Association was less than helpful, although their reluctance to accept the new breed of “quantitative” historians as a legitimate part of the discipline may actually have spurred the rebels into more aggressive organizational activity than would otherwise have occurred. On the positive side, the National Science Foundation provided funding support that was absolutely crucial to the archival as well as the instructional program development of the consortium. Out of the efforts to develop historical data relevant to many different intellectual disciplines came the creation of yet another institution that has since taken its place in providing a crucial forum and source of intellectual support for pioneering scholars. It is the Social Science History Association. Although the SSHA has no organic relationship to the consortium, its founders, including Howard Allen, who was Jerry’s predecessor in the Historical Archives and is now the executive director of SSHA, were all individuals who had been caught up in the consortium’s promotion of the development of social science research resources that were historical in their temporal span of coverage, but not at all limited to the substantive interests of the academic political scientist. The point of this truncated review of consortium developmental history is not to chronicle the evolution of the archival resources but to point out that many of the major expansions of the archival facility can be traced directly to the innovative leadership of particular individuals and to the facilitating role played by almost as many different institutions. Subsequent research efforts may, at the stage of publication, bear the names of individual authors, but the scholarly output that can be attributed to consortium resources exists because of the interdependent roles of individuals and institutions that have been central to the creation of those resources.

The Larger World Any attempt to reconstruct major components of the intellectual activities associated with the consortium and the Center for Political Studies must give some attention to our many involvements beyond the continental boundaries of the United States. I have already mentioned the Ford Foundation’s support in extending both our training and our research activities to Latin America. There have been many other international connections that have preoccupied both consortium and center personnel over the years. In the beginning, it was the existence of the Survey Research Center and the presence of Angus Campbell that created many of the personal contacts that ultimately produced ongoing interinstitutional cooperation. First in the early 1950s and then stretching into the 1960s, foreign colleagues such as

Research Ltfe zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA and Probabilities 155 David Butler from Oxford, Stein Rokkan from Bergen, Henry Valen from Oslo, Jorgen Westerstahl from Gothenburg; George Dupeux from Bordeaux; Max Kaase and Hans Dieter Klingemann from Mannheim; Rob Mokken from Amsterdam, and Perti Pessonen from Tampere- all found their way to Ann Arbor, and all carried back with them a commitment to implement “our” kind of research in their national settings. At least in part because of these personal contacts with individuals who turned out to be national leaders, there is now an extended series of academically designed election studies in virtually every country of Western Europe. The cast of principal investigators has changed, with some notable exceptions, but even more, the institutional commitments have changed. Individual entrepreneurial effort is still essential, but it is now supplemented by supporting - or demanding - constituencies, and by more-or-less regularized channels of support. In recapturing many of the exciting times of the first three decades following World War 11, it is easier to remember the successes than the failures, and even our small corner of the political behavior enterprise has had its full share of frustrations and disappointments. Some of these have come as a consequence of unexpected developments that upset well-laid plans and diverted energies from expected courses of action. So, for example, the MillerBtokes study on representation never came forth. After virtually a decade of promising a major research report, it became clear that our respective commitments had given way to other apparently more pressing demands. But if we don’t have Representation in America, we do have Political Change in Britain and the British Election Studies, with the superb resources they have presented to students of British politics and comparative politics alike. And we have the American National Election Studies, and we have the consortium. All of these activities were competing for attention at the same time, and there quite apparently were not enough hours, days, months, and years to make it possible to get everything done, but a four-forone trade-off isn’t all that bad. Nevertheless, I would note in passing, and quite impersonally, one major loss that resulted from the nonpublication of the potentially innovative study of representation in Congress. The analyses completed for the MillerBtokes volume dealt almost exclusiveiy with the representational consequences of such phenomena as candidate recruitment, the committee system of the House, members’ contacts with constituents, the role of presidential leadership, politicians’ careers and career aspirations, and the role of party in government. An analysis of political behavior, supported largely by survey research methods and focused on the analysis of manifestly political

156

Miller

variables, might have extended political scientists’ vision of what the nature and scope of research that blends individual and institutional data could be. Such an example did not appear early enough to demonstrate, in the crucial formative period, the important contribution which the logic of survey research could make to the study of politics and government. Happily, more recently a large part of the promise in the 1958 study design has now been redeemed in the Converse/Pierce demonstration that the elements of the research processes that are combined to form survey research can be applied with profit to the study of electoral processes, even in a non-American setting. For me, the ultimate excitement of their magnificent volume, Political Representation in France, is found in their demonstration that large-scale survey research can deal subtly, precisely, and insightfully with political institutions as well as political actors. One intimation of the early promise of the 1958 Miller/Stokes study of representation lies in the fact that because of the initiatives taken by several senior colleagues in the Center for Political Studies, essentially the same study design was subsequently implemented in at least 11 different countries - largely Western European, but including Japan, Brazil, Australia, Canada, and India as well. The existence of those studies certainly contributed to the continued intellectual excitement associated with comparative research. On the other hand, we were never successful in consolidating the full set of studies. The ultimate consequence of a full decade of foreign involvement by CPS scholars was a series of independent successes, but a grand opportunity lost. The importance of the institutionalization of research support was never made more evident than by the limited publication record of these many data collections. Each national collection was a complex affair, including data from both elite and mass populations, usually populations of constituents and their representatives, and each data collection often involved a combination of individual data and contextual public-record data. Each collection was extraordinarily expensive both in terms of the investments of time made by the principal investigators and in terms of the sheer monetary cost. Inasmuch as every single one of the collections involved extended extensive collaboration between a member of our Center for Political Studies staff and one or more scholars abroad, the strain and dislocation of international travel was often a major unaccounted element of cost. Most of the studies eventually produced at least some limited presentation of results integrating the mass and elite collections. Some, as in the case of the works of Converse and Pierce, or McDonough or Barnes, constituted major publications. In other instances, however, the publication output

Research Life and Probabilities

157

consisted of a limited number of journal articles, usually reporting data on the electoral behavior of the masses. In no instance did any combination of collaborators come together to do the systematic, tightly comparative analyses the studies were designed to permit. As with the nonappearance of the Miller/Stokes volume, the nonappearance of one or more major works capitalizing on the analysis of institutional differences captured by the various representation studies played at least some role in shifting attention away from the promise offered by the conception of these studies. As I have suggested above, these failures provide a striking commentary on the fragility of institutional support for serious cross-national, crosscultural comparative research in our time. In doing so, they add to my list of illustrations of how hard it is to change the world-even the world of social research in a time of revolutionary change. Some reincarnation of the old SSRC committee on comparative politics might have produced a different outcome. As it was, neither the existing private foundations nor the National Science Foundation could be persuaded to provide the incremental, marginal costs that would have transformed the absolutely staggering investment in 11 comparable studies into an effort that might have reshaped one of the major fields of the discipline of political science. Out of prolonged experience with our series of comparative research ventures comes clear evidence that established and committed institutional support is indispensable for the conduct of potentially significant research into the workings of major societal institutions. The same experience, however, also makes it clear that even the biggest of big science enterprises is crucially dependent on individual scholarship for the creation of a scholarly product.

ECPR/I C PR As a final event about which to organize recollections of the interdependence of intellectual contributions and institutional infrastructures, let me turn to a major success story, the formation of the European Consortium for Political Research (ECPR). Although the American counterpart provided an initial model, the evolution of the ECPR took the form it did because of the unique presence of a few individuals as well as because of environmental support exemplified by the existence of what was then still ICPR. A complete reconstruction of first causes would be too complex for this chapter. Suffice it to say that one important institution essential to the creation of the ECPR was the Department of Government at the University of Essex, with its extraordinary collection of scholars, including Tony King,

158

Miller

Ivor Crewe, Ian Budge, and most crucially, at least in the beginning, Jean Blondel. Blondel and the colleagues at Essex were not alone in their enthusiasm for bringing the advantages of the ICPR closer to home. Other colleagues and institutions, including Wildenmann in Mannheim, Scheuch in Cologne, and Daudt in Amsterdam, were also ready (often because of support from the American Ford Foundation) to organize like-minded social scientists from Western Europe. It was Blondel, however, who pursued the task of creating the organization of which he ultimately became executive codirector. Reverting again to my emphasis on the importance of such unpredictable events as the appearance of extraordinary individuals suited and inclined to leadership, it should be noted that the ECPR came into being only when Stein Rokkan was persuaded that it would provide another vehicle for his vision of the social science of tomorrow. Rokkan’s chosen instrument had been the International Social Science Council. With the infinite patience so necessary in attending to details of international academic diplomacy, Rokkan had made both the council’s Standing Committee on Comparative Research and its Standing Committee on Data Archives significant elements in the infrastructure of the European, if not the worldwide, development of an empirically based social science. In the meantime, summer training programs, modeled at least in part on the Michigan experience, had moved as ad hoc occurrences to a variety of sites, including Amsterdam, Cologne, and Essex. Interest in the behavioral sciences had captured attention, even in Geneva and Paris. Archival development had moved ahead at Cologne under the inspiration of Erwin Scheuch, at Amsterdam under the direction of Martin Brouwer, and at Essex with Ivor Crewe. Crewe’s investment in archival development, the commitment of Tony King and Ian Budge to methodological training, and Jean Blondel’s recognized career as a student of comparative politics ultimately made Essex the organizational center for the ECPR. Solving a host of political problems with the creation of ingeniously peripatetic annual meetings of the ECPR, which were extraordinarily open to the presentation of papers on both substance and method, the secretariat, the archive, and the training program for the European consortium have since been firmly associated with the University of Essex. In the earliest days, the Essex Summer Program was staffed, at least in part, by imports from the ICPK, as Lutz Erbring, Michael Traugott, Don Stokes and others made their contributions to the new institution. The ICPR also provided institutional support through a limited but direct subsidy of the new organ-

Research Life and Probabilities

159 zyxwvutsrqp

ization. More generally, this celebration of the 25th anniversary of the ICPSR is also, inevitably, the celebration of the host of personal and institutional connections that have brought the ECPR and the ICPSR not only members in common but accomplishments shared, as well. Without diminishing the many bilateral relationships enjoyed by those of us associated with the Center for Political Studies, the two international institutions provide a valued and special setting for a host of common interests. The appearance of new research institutions may signal changes in the determinants of the events shaping intellectual progress and, most importantly, may create new probability distributions for other proximate causes of intellectual development. We are celebrating the continued success of one institution that has made more than just a noticeable difference in many of the events which make up intellectual life as we know it. In doing so, we are inevitably reminded of the unique contribution of extraordinary individuals who are, by definition, way out on the extremes of distributions of talent and energy. This is not the occasion for further singling out particular individuals from their place among the many whose leadership efforts have created and sustained the institution, but it is our collective good fortune that the consortium has been maintained as a dependable influence on a multitude of research activities, providing a new definition of the modal design for research infrastructures, and altering the probabilities of success of the research undertakings of social scientists around the world. The pleasure of the occasion is properly shared by all of us who have shared in the growth and maturation of the consortium. In an era in which the largesse of the Ford Foundation and the leaders of the Social Science Research Council are no longer sources of strength for the social science research enterprise, ICPSR fills much of the void with collective goods. They are there for our individual or joint use because the official representatives, the council members and council chairs, have, over the years, joined with executive directors and staff to maintain the relevance of the organization. But they have not only shaped the organization and its resources; they have collectively participated in establishing new norms of conduct for the research enterprise. In deciding what I could say on this occasion that would be interesting, and perhaps helpful, to those who were not around 25 years ago, I sought advice in a number of quarters. I will close with my response to one of the more insightful suggestions: “Tell them what makes you most proud of ICPSR.” Given the rich complexity of the organization, it was a provocative suggestion. But my ultimate response doesn’t have much to do with organ-

160

Miller

izational or intellectual complexity, nor with technological sophistication. I am proud of the role of ICPSR in creating new standards of professionalism for our research endeavors. Ours is a world in which the emphasis on creativity, originality, insight, and imagination can lead, and has at times led, to all manner of idiosyncratic, selfishly egocentric and antisocial, less-than-civil behavior. Our world, a world propelled by academic scholarship - whether to be practiced or to be taught- can be easily sullied by theft in the form of plagiarism, deceit in the form of a falsification of the evidence, self-aggrandizement through claiming rewards for the work of colleagues, particularly junior colleagues, or just plain fraud as one argues with rhetoric and the persuasive arts a case that logic supported by evidence would deny. The consortium stands for openness -openness of access to the facts. It is impossible to steal what is in the public domain. It is impossible to sustain falsification of evidence if the real evidence and tests of the evidence are in the public domain. It is more difficult, if not impossible, to argue against the evidence if others have access to the evidence. And I am virtually certain that Jerry Clubb, an old if not former Socialist, was attracted to the consortium because of its social egalitarian spirit: it is an organization that serves a meritocracy in which inequality is not the inequality of opportunity but the inequality of ability. Without having experienced the world of yesteryear, it may be difficult to appreciate the changes of the last 25 years. And it may not be necessary, but it may be useful in quieter moments than this to imagine our world bereft of the material goods and the social norms that the consortium maintains. Personally, I am indeed proud of what this organization has accomplished, and I am optimistic about its future. But I, and I hope you, feel a particular sense of pride and satisfaction in the role of the ICPSR in creating both new opportunities for excellence and new standards of professionalism in the conduct of social research.

Comment

Innovation: Individuals, Ideas, and Institutions lvor Crewe

w

‘arren Miller’s bitter sweet, celebratory but serious review of the ICPSR’s history is a timely reminder above all else that intellectual endeavor is, and always has been, a collective enterprise. Even in the “primordial” era-before the computer, the jet plane, and the sample survey - when social research was conducted by the individual scholar in the library, it relied on collective institutions founded on the norms of reciprocity and public access. The institutional expressions of these values were the small, multidisciplinary, self-governing college; the research library, which created public knowledge out of private intellectual labor; and the scholarly association, which organized and regulated the wider dissemination of that knowledge. The old structures, however, proved unable to cope with the revolution in the social sciences; and the ICPSR was born out of that failure. Colleges and departments could not afford the elaborate infrastructure of support required for the large-scale quantitative research which perhaps only one of their members wished to undertake; they lacked the expertise to offer courses on data collection and analysis; their libraries were too slow to incorporate new media for information; and the established professional associations were not equipped to bring together, at least in the early years, the scattered body of isolated individuals interested in quantitative research. The ICPSR is a monument of innovative institutional responses to the needs of a whole national academic community: the summer training program; the data archive; the software package OSIRIS; and the ICPSR as an academic association in its own right. To paraphrase the advertisement for a well-known lager: the ICPSR reaches parts of the academic body politic that 161

162

Crewe

other institutions cannot reach. It has adapted traditional research norms to modern conditions. And it has done so not only on a national scale, for the United States, but by inspiring parallel institutions abroad. Let me dwell briefly on the importance of the ICPSR to the formation of the European Consortium for Political Research (ECPR) and of the Center for Political Studies (CPS) to the development of systematic cross-national political research in Western Europe. If anything, Warren Miller underrates the impact of the ICPSR. It is difficult nowadays to appreciate the primitive state of empirical research and comparative politics only twenty-five years ago in Western Europe. “Comparative government” meant the study of “some foreign governments”invariably Britain, France, the United States, and the Soviet Union - preferably one at a time. The smaller European democracies were treated, as Chamberlain described Czechoslovakia, as “far off countries about which we know little”; cross-tabulations were thought a risque adventure in statistical sophistication; and the study of institutions frequently consisted of the armchair impressions and anecdotes of the brandy-swilling professor. At the same time, there was no such thing as a European political science. Barriers of language, ideology, and sheer parochialism prevented much personal contact across national frontiers; indeed, there was probably more traffic across the Atlantic than across the Rhine or the Channel. Thus the first products of European collaboration or contact with the CPS made an enormous impact in their respective countries. Political Change in Britain, for example, transformed the study of electoral behavior in Britain - its impact has lasted to this day- and similarly the early election studies by Bo Sarlvik in Sweden, Henry Valen in Norway, and Erwin Scheuch and Rudolf Wildenmann in Germany set new and enduring benchmarks of research. The ECPR is the most important organizational development in European political science since World War 11, and it owes much to the idea of the ICPSR. I stress idea because the ECPR’s actual structure and activities (such as the workshops and the European Journal for Political Research) have developed along slightly different lines; moreover, some of the ICPSR’s features had their origin in Europe. For example, data archiving was already well established at the Zentralarchiv in Cologne and has never been a formal part of the ECPR. But the idea of a consortium of political science departments, pooling their resources and subscribing to common services so as to create a European academic community, was truly original and was inspired by the example of the ICPSR (and, it must be added, by the Ford Foundation’s willingness to support it). In terms of the depth of the departmental

Commen t

163

resistance that had to be overcome, the ECPR was, if anything, an even more audacious piece of institution building than the ICPSR. The ICPSR and the ECPR are both institutional landmarks in the postwar landscape of academic political science. They have benefited the research careers of countless political scientists, especially a younger generation, offering data, networks, and collaborative opportunities not available to their predecessors. How did such an institution come to be built? Warren Miller’s wry answer bears the hallmarks of a weather-beaten fund-raiser. All that is required, he writes, is (1) periodic strokes of extraordinary good fortune; (2) exceptionally generous foundations led by unusually far-sighted idealists; and (3) individuals of outstanding talent, drive, and vision. That’s all. In the absence of all three cherries clicking into row on the fruit machine, there’s no jackpot; and things stay the same. It is right to emphasize the crucial role of a few individuals. Great buildings depend on fortunate happenstance, and on generous patrons with almost unlimited reserves of faith, but most of all on inspired architects. Where does the inspiration come from? First, the key individuals are major scholars in their own right - architects who are also master craftsmen. Their scholarly contribution to comparative politics would have been internationally recognized even if they had not devoted themselves to creating research institutions. In fact, their drive was fueled by the need to invent new structures to accommodate their model of comparative political science, usually in the tangible, practical form of an unwieldy data collection. For example, Stein Rokkan’s life-long study of comparative political mobilization and his autobiographical sensitivity to center-periphery divisions led to his concentration on aggregate data, on the standardization of ecological units, and on electoral cartography- now the core of the Norwegian Data Services’ activities. Jean Blondel’s concept of a truly comparative politics, based on standardized indicators from all states, led to the ECPR and its workshop structure-and now, even more ambitiously, to plans for a world institute of comparative politics based in Lausanne, Switzerland. Creative institution-building is not just a matter of administrative abilities and diplomatic skills. It is also a matter of intellectual vision. Second, these intellectual ambitions are usually accompanied by political and idealistic impulses - although not of a narrow political-party kind. Warren Miller mentions Jerry Clubb’s conception of the ICPSR as a practical expression of academic socialism. A principled internationalism lies behind the achievements of Stein Rokkan and Jean Blondel: both of them

164

Crewe

saw their work as a breaking down of irrelevant, restrictive national boundaries. Indeed, it is not a coincidence that when the ECPR was founded in the early 1970s, the European Movement was at its most optimistic and idealistic. Nor is it an accident that many of the key figures in the founding of the ECPR had personal histories that were truly cross- or multinational (e.g., Hans Daalder and Richard Rose, as well as Blonde1 and Rokkan). In the ICPSR, Warren Miller and Jerry Clubb, in particular, have been noticeably internationalist and far-sighted in their dealing with European members. The system of national federated membership is their initiative, not an invention of the Europeans, and in retrospect, it can be seen as a resourceful and generous way of overcoming disparate funding traditions to provide European scholars with access to the ICPSR’s resources and services. I still remember with what effect Jerry gently chided squabbling representatives of various European archives at a meeting of the International Federation of Data Organisations at Louvain-la-Neuve for taking a contractarian rather than a communitarian view of their reciprocal rights and obligations. Third, on both sides of the Atlantic, there was the human and perfectly respectable desire to beat the old academic establishment by setting up rival and superior centers of excellence - and by defining new forms of excellence. Resistance and inertia from the entrenched authorities are usually spurs to greater effort, as Warren Miller mentions in reference to the pioneer collectors of historical data. Again, it is surely significant that the ICPSR is based at Michigan, not on an Ivy League campus; that its European cousins are in Cologne and Mannheim, not in Heidelberg; at Essex and Strathclyde, not Oxford or Edinburgh; at Bergen, not Oslo. These three characteristics of the institution builders are mentioned in the context of Warren’s sobering comments about the disappointing outcome of the CPS-promoted projects on elite-mass studies of representation and about the “fragility of institutional support for serious collaborative comparative research.” Of course, one immediately recognizes the problem. Drumming up funds for systematic multinational projects is the most Herculean of entrepreneurial tasks; survey research, in particular, is hideously expensive, and the additional overhead costs of coordination prove too much for most foundations. Alternative strategies, such as reliance on domestic funding for a cross-national program with an identical core of data, suffer from the fate of the convoy, progressing no faster than the slowest member, as Sam Barnes and Max Kaase learned only too well in their Political Action program. Yet the picture is not quite as gloomy as Warren Miller implies. The

Comment

165

ECPR has produced a forum for genuinely systematic and rigorously comparative research, as the work of Newton and Sharpe on local policy outputs (inspired by Rich Hofferbert) and of Ian Budge on party manifestos illustrate. ZUMA at Mannheim has spearheaded comparative studies of middlelevel party elites and elections to the European Assembly. And the department of political science at the European University Institute at Florence, successively led by the founding fathers of the ECPR, has inspired new comparative work on party systems. These exceptions aside, Miller is nonetheless right to point out the scarcity of cross-national research, the decline since the heady days of the 1960s and 1970s, and the refusal of the foundations to come forward with support. But the reason does not entirely lie with the refusal of Lady Luck to smile, or with the absence of visionaries in the foundations, or with a sudden dearth of energetic and talented academics. It also reflects a conservative period of consolidation after the quantitative and survey revolutions of the 1960s and an absence of new perspectives. Good ideas have not failed to find data and financial support: think of Ron Inglehart’s use of the Eurobarometers for his continuing work on postindustrial cleavages. But this has been the exception, not the rule. To resuscitate cross-national studies, fresh approaches are obviously needed. There is every reason to believe that the ICPSR can respond to the challenge. Perhaps Political Representation in France will spark enthusiasm for rigorous, exact empirical work on the public impact of variously shaped political institutions, rekindling the interest of the foundations. That would be a fitting beginning to the next quarter century of leadership from the ICPSR in the development of innovative, professional political research.

Comment

Infrastructures for Comparative Political Research M a x Kaase

I

t was in September 1965 that I first visited Ann Arbor and set foot on the premises of the Institute for Social Research. I came as a fellow of the American Council of Learned Societies, hungry to find out about America and the mecca of political behavior studies, the Survey Research Center and the colleagues - Campbell, Converse, Miller, Stokes, Barnes, and Jennings- who over the ensuing years became collaborators and friends. Flashbacklike recollections of these days are still there, for instance, o f Don Stokes, teaching in the ICPSR summer program and conveying the image of the poor data analyst in front of a pile of contingency tables who urges, “Data speak to me.” I did not then know what he was talking about. And I recall as one o f my big frustrations at the time the realization that what had taken me roughly a year in data analysis for my doctoral dissertation at home in Germany could have been done on the ISR computer in about a week of working time. The Story of ZUMA and GESIS in West Germany

Warren Miller has told us about the unlikely, the implausible, the things that just could not happen but happened anyway, Similar improbabilities mark my own experience in West Germany, some of them clearly related to stimuli originating from the Survey Research Center and the ICPR. Returning to Ann Arbor for a second time in 1969, my colleague Uwe Schleth and I came to participate in the first data confrontation seminar ever held. Its topic was the comparative analysis of elections. The data were longitudinal official election returns and census statistics, covering about

166

Comment

167

fifteen countries. The seminar was locally organized by the ICPR in collaboration with the late Norwegian scholar Stein Rokkan, the indefatigable organizer and intellectual leader in cross-national research (Rokkan, 1969; Clubb, 1970). While, in retrospect, I am a little hesitant to characterize the seminar as an overwhelming success, I recall being so impressed by then with the importance of infrastructural support for good empirical research that I returned to West Germany convinced that the social sciences there could not blossom without an infrastructure similar to the one that I had encountered in Ann Arbor. As a start, we stole personnel, and we stole software from Ann Arborthe consortium’s OSIRIS program seemed almost like a wonder to many of us who had just begun to learn how to work with computers. And we received much good advice from the people at ICPR when we began, around 1970, to design a center like the SRC in Ann Arbor. Needless to say, because of the vast differences in the American and German university systems, our center could not be modeled exactly after the establishment in Ann Arbor. After three years of planning, and with the never-tiring persistence of Rudolf Wildenmann, ZUMA- Zentrum fur Umfragen, Methoden and Analysen (Center for Surveys, Methods, and Analyses) - came into being on January 1, 1974. I served as ZUMA’s first director from then until the end of 1979. The senior professional positions at ZUMA were filled by a group of distinguished colleagues: Hans-Dieter Klingemann, now at the Free University of Berlin; Franz Urban Pappi, now at the University of Kiel; and Erich Weede, now at the University of Cologne. Is it just by chance that each of us attended the ICPR summer seminar years earlier? I believe not. From 1974 through 1986, ZUMA was fully funded by the German equivalent of the American National Science Foundation: Deutsche Forschungsgemeinschaft (DFG). The funding followed annual reviews of ZUMA’s work by independent referees and occasional on-site visits. In its final DFG-funded year, 1986, the ZUMA budget was more than $2 million, and this was entirely institutional funding. From its start, ZUMA was assigned two main tasks, which are still its tasks today: (1) to assist scholars in West Germany in doing good empirical research and (2) to conduct basic research in the social sciences. Since 1974, quite a few American colleagues have been visiting professors at ZUMA, and we expect many others to come in the future. ZUMA has established firm ties with several American universities, but ICPSR played an important intellectual role, especially in the founding and development of ZUMA. Among those social scientists from abroad asked by the

168

Kame zyxwvutsrqpo

German Research Foundation about strengths and weaknesses in the initial concept of ZUMA were Warren Miller, Jean Blonde1 of Essex, and Karl W. Deutsch of Harvard. Much earlier, in 1960, Gunter Schmdders of the University of Cologne had founded the Zentralarchiv fur Empirische Sozialforschung, a data archive which he soon handed over to Erwin K. Scheuch, and which was a natural partner to ICPSR. Indeed, very soon exchanges of ideas and, later, of data occurred between the two archives. To this day, the Zentralarchiv carries the German national membership in ICPSR. And in the mid-l970s, the ICPSR, the Zentralarchiv, and ZUMA even joined efforts and resources in launching a concrete archival project: the German Electoral Data Project provides American scholars of comparative political behavior (and, of course, scholars elsewhere as well) with extensive survey data for each West German general election. Richard I. Hofferbert, Miller’s successor as director of ICPSR, was especially instrumental in making this project a success. Until 1986, both ZUMA and the Zentralarchiv were funded by “soft money” that was “there” but could also be withdrawn on short notice. Fortunately, at the end of 1986, the Gesellschaft Sozialwissenschaftlicher Infrastruktureinrichtungen (Association of Social Science Research Infrastructure Institutes, or GESIS) was founded in Mannheim and was guaranteed permanent institutional funding of roughly $7 to $8 million per year. GESIS consists of these components: (1) the Zentralarchiv at Cologne, including the Center for Historical Social Science Research; (2) the Social Science Information Center at Bonn, which documents German-language social science literature and research projects, and which also holds two large data banks accessible on-line from the United States; and (3) ZUMA, including three important smaller centers - a Center for Micro Data; the ALLBUS, the German equivalent of the NORC General Social Survey, a joint venture of ZUMA and the Zentralarchiv (so far operating biennially, ALLBUS will be available on an annual basis in 1988, including the German election study); and a Center for the Study of Social Indicators (formerly directed by Wolfgang Zapf of the University of Mannheim, now president of the Wissenschaftszentrum in Berlin). To the best of my knowledge, GESIS is unique in the Western world. Its responsibility to the West German social science community - and, indeed, the world social science community - is considerable. Cross-National Political Behavior Research

Understandably, in view of its geographical, professional, financial, and intellectual situation, the ICPSR is mainly concerned with the social

Comment

169

sciences in the United States. However, the ICPSR maintains strong ties to data archives outside the United States, as it should, through the imaginative construction of national memberships within the organization. This international linkage is also important because, useful as the distribution of data is to the community of scholars for secondary analysis, such data will be available for distribution only if there is primary research; and the ICPSR makes access to the original data possible. For example, since 1984, two books have been published that are concerned with the comparative analysis of electoral behavior based on survey data (Dalton, Flanagan, and Beck, 1984; Crewe and Denver, 1985). Both are excellent contributions (see Kaase, 1987), but analysis could be much better still if truly longitudinal and comparative studies were available. There are some such studies, like the American National Election Studies, the German Electoral Data Project, the Essex Studies of British elections, and studies in Holland and Sweden. But as the editors of the two volumes just mentioned point out, not enough countries are covered to allow a better job along comparative lines. What other studies are there? The still insightful and valuable fivenation Civic Culture Study of the late 1950s (Almond and Verba, 1963) is widely referenced, There is the comparative study of conventional political participation and political equality in seven countries, with data collected around 1970 (Verba, Nie, and Kim, 1978). There is the Political Action Project, a study of conventional and unconventional political participation in eight democracies made around 1974 (Barnes, Kaase, et al., 1979) and its continuation in three countries around 1980 (Jennings and van Deth, 1988). There is the Comparative Values Survey (Harding and Phillips, 1986), and there is, as Miller has pointed out, the Comparative Representation Study of the Michigan group, with many individual publications (of which the latest and most spectacular is Converse and Pierce, 1986) but without a comprehensive synthesis. On a smaller scale, with fewer countries and differing scopes, there are other comparative studies, but by and large, the assessment of a conference on cross-national research, held in October 1977 at the Center for Political Studies in Ann Arbor, still seems to hold (Kaase and Miller, 1978): there was much talk about the many structural problems facing cross-national research, and there was some fear that cross-national research, especially of the longitudinal kind, would become an extinct species. Nevertheless, some qualification of this general appraisal concerning the lack of such studies is in order. Other data sources, less expensive than surveys, are becoming increasingly available. There is the comparative study of party manifestos

170

Kaase

directed by Ian Budge at Essex (Budge et al., 1987). There are comparative studies of national budgets and state expenditures, in particular the giant project on the development of the Western welfare states directed by Peter Flora of the University of Mannheim (Flora, 1982, 1986, 1987). There are various comparative studies based on historical political data, and on data collected, through and in extension of the Yale Political Data Program, by Charles Taylor and his collaborators and predecessors (Taylor and Jodice, 1983). As a final example, there is the comparative study of corporate power networks by Frans Stokman and his associates (Stokman, Ziegler, and Scott, 1985). Most of the studies are done outside of the United States. This might well indicate that there is in the United States a decreasing interest in comparative politics, a concern strongly expressed only recently by the Council for European Studies in New York. I am not familiar enough with the current status of comparative political studies in the United States to be able to assess the validity of this concern; and I should perhaps better talk about a lack of interest in European studies, as some colleagues have suggested to me. However, if my hunch is correct and there is in the United States a decline of comparative political studies on a large scale, then the ICPSR might be the right place to think about this problem. After all, ICPSRconnected scholars like Angus Campbell, Warren Miller, Richard Hofferbert, Samuel Barnes, and Ron Inglehart, to name just a few, have created an intellectual heritage of social science cosmopolitanism that I find waning in American comparative scholarship today. If we return briefly to the study of mass publics, it is symptomatic that the Political Action Project had to turn to the German Volkswagenwerk Foundation to obtain funding for fieldwork in America in connection with the second wave of its work. And as another example, starting in 1970 and regularly after 1974, there have been the biennial Eurobarometer studies commissioned by the European Community under the supervision of JeanJacques Rabier (since 1987, Karl-Heinz Keif) and Ron Inglehart. Not surprisingly, in view of what I have said about the state of comparative studies in the United States today, one often finds footnotes in reports based on these European data regretting that the United States does not have a comparable time-series collection for many of the Eurobarometer indicators indicators at the core of the contemporary studies of mass publics. The Power and Frailty of Survey Research

In a piece called seminal in our professional jargon, Philip Converse (1970) once made an interesting observation based on data from the

Comment

171

1956-1958-1960 panel election study. Converse found high aggregate stability in the distribution of certain political attitudes over time, although there were sensationally high rates of turnover in the responses of individual respondents. The black-and-white Markov chain model which he fitted to the data indicated almost perfectly that, to a large extent, random attitude change seemed to be producing these results. I know of no political-attitudes-and-behavior panel study conducted since in which similar observations could not be corroborated. It is not surprising, then, that in an analysis of the 1972-1974-1976 NES panel, Converse and Markus (1979) raised the question whether panel studies are really worth their cost. I cannot deal with this delicate problem here in any detail; some of my concerns are discussed by Hubert Blalock (see Chapter 1). Let me just say that I have been increasingly skeptical about a strictly methodological approach to this problem. It seems to me that we have failed to reconceptualize this problem of aggregate stability and disaggregate instability as one of sociological theory (see Kaase, 1986). Could it not be that we constantly miss the social processes behind the formation and expression of individual beliefs, attitudes, and behavioral reports or intentions? Could it not be that instead of disorder at the level of individuals, there is order? And is this apparent disorder not an artifact of the way in which we collect our data? Why address these questions at this time and at this place? Well, the twenty-fifth anniversary of the ICPSR’s founding is an occasion not only for commemorating but also for looking ahead. The ICPSR and other archives are the repositories of survey data from representative samples of some population. This, however, is the type of data mostly collected from atomized individuals in hopefully atomized social situations; but the data are therefore atypical of the real-life settings in which people live and interact. The Columbia school of electoral research long ago spoke of the “breakage effect” (Berelson, Lazarsfeld, and McPhee, 1954). John Sprague and Robert Huckfeldt have recently reminded us again of the enormous importance of social context. As they show in one example, changes in individual political orientations follow, on first sight, exactly the Converse attitudenonattitude pattern, whereas on a second look, these seemingly random distributions reveal a clear pattern of contextual impact by respondents’ social environment (Huckfeldt, 1983, 1986; Huckfeldt and Sprague, 1987). I do not know how far this kind of contextual analysis will take us. It addresses, however, a problem of survey research that deserves the most serious attention, which otherwise might turn scholars and funding agencies away from survey research. As a community of scholars, we have, over the decades, established the sampling frames, the methodology, and the nor-

172

Kaase

mality of cross-sectional survey research. We have recognized that the social sciences must deal not only with structures but also with processes. This recognition is slowly bearing fruit in the form of an increased emphasis on longitudinal studies. These studies, as I mentioned earlier, create new methodological problems of their own. They also face into the wind in terms of funding. There is a definite need for continuity in sampling, questions, and questionnaire format, and this also means a continuity in theoretical approach. Our studies always face the query by funding agencies of whether this type of research is truly innovative. In this connection, I recall a comment by Peter de Janosi of the Ford Foundation in New York in 1971, when he welcomed me as a proud holder of a Ford fellowship. We talked about my research agenda and about the social sciences in general. At one point, he looked me straight in the eye and said, “You know, Dr. Kaase, there is something I will never understand: Why do social scientists always try to reinvent the wheel?” In various academic roles, I now come into regular contact with that other part of the academic world, the “hard” sciences; and I now understand better than I did then, in New York, what de Janosi had in mind. So our studies are slowly becoming cumulative as well as longitudinal, and the ICPSR or scholars related to it have had some impact on these developments. It has been most important that over the years, young scholars have been socialized into an intelligent and reflective use of modern data analysis techniques; and the ICPSR summer training program has been a successful vehicle of this socialization. One next step for the consortium to take, which might turn out to be of the same or an even larger impact on the development of the social sciences, is to trigger dynamic multilevel research designs, with all of the theoretical, methodological, and practical problems involved. I am convinced that this is how the micro-macro puzzle can be solved. Epilogue

This has been an occasion for paying some intellectual dues, to the late Angus Campbell and to all other colleagues and friends in the United States. European social science owes them much. But I have seen, over the years, some of the important personal and structural networks of comparative research that existed earlier between American and European scholars weaken or falter. This is a situation of grave concern to me. Warren Miller has recalled the role of the ICPSR in the establishment and success of the European Consortium for Political Research, and Ivor Crewe addressed this relationship in some detail. I want to emphasize that it

Comment

173

is through the annual ECPR workshops that comparative political research in Europe has come into bloom. Particularly the ECPR’s joint workshop sessions give young scholars something not available at the usual professional meetings: true intellectual exchange in the context of concrete research, and sufficient time for a proper exchange. In closing, let me just say that I understand and share Warren Miller’s pride in the achievements of the consortium, and I sincerely hope that it will maintain its important role in facilitating excellent social science research in the United States and in the many other countries with which it is linked through personal contacts and organizational arrangements. References Almond, Gabriel A , , and Verba, Sidney. (1963). The Civic Culture. Princeton, NJ: Princeton University Press. Barnes, Samuel H . , Kaase, Max, et al. (1979). PoliticalAction: Mass Participation in Five Western Democracies. Beverly Hills, CA: Sage. Berelson, Bernard R., Lazarsfeld, Paul F., and McPhee, William N. (1954). Voting: A Study of Opinion Formation in a Presidential Campaign. Chicago: University of Chicago Press. Budge, Ian, et al. (1987). Ideology, Strategy and Party Change: A SpatialAnalysis of Post- War Election Programmes in 19 Democracies. Cambridge, G.B. : Cambridge University Press. Clubb, Jerome M . (1970). Ecological data in comparative research: A report on a “Special Data Confrontation Seminar.” UNESCO Reports and Papers in the Social Sciences, No. 25. Paris: UNESCO. Converse, Philip E. (1970). Attitudes and non-attitudes: Continuation of a dialogue. In Edward R. Tufte (ed.), The Quantitative Analysis of Social Problems. Reading, MA: Addison Wesley. Converse, Philip E., and Markus, Gregory B. (1979). Plus Ca change , . . : The new CPS Election Study Panel. American Political Science Review 73:32-49. Converse, Philip E., and Pierce, Roy. (1986). Political Representation in France. Cambridge: Harvard University Press. Crewe, Ivor, and Denver, David, eds. (1985). Electoral Change in Western Democracies: Patterns and Sources of Electoral Volatility. London: Croom Helm. Dalton, Russell J., Flanagan, Scott C., and Beck, Paul A., eds. (1984). Electoral Change in Advanced Industrial Democracies: Realignment or Dealignment. Princeton, NJ: Princeton University Press. Flora, Peter, ed. (1986). Growth to Limits: The Western European States Since World War IZ. Berlin, New York: de Gruyter, 4 vols. Flora, Peter, et al., eds. (1982, 1987). State, Economy and Society in Western Europe, 1815-1975: A Data Handbook, Vols. 1, 2. Chicago: St. James Press. Harding, Stephen, and Phillips, David. (1986). Contrasting Values in Western Europe: Unity, Diversity and Change. London: Macmillan. Huckfeldt, Robert R. (1983). The social context of political change: Durability, volatility, and social influence. American Political Science Review 77:929-944.

174

Kaase

Huckfeldt, Robert R. (1986). Politics in Context: Assimilation and Conflict in Urban Neighborhoods. New York: Agathon Press. Huckfeldt, Robert R., and Sprague, John. (1987). Networks in context: The social flow of political information. American Political Science Review 81:1197-1216. Jennings, M. Kent, and van Deth, Jan, eds. (1988). Continuities in Political Action: A Longitudinal Study of Political Orientations in Three Western Democracies. In preparation. Kaase, Max. (1986). Das Mikro-Makro-Puzzle der empirischen Sozialforschung: Anmerkungen zum Problem der Aggregatstabilitat bei individueller Instabilitat in Panelbefragungen. Kolner Zeitschrift f u r Soziologie and Sozialpsychologie 38(2):209-222. Kaase, Max. (1987). On the meaning of electoral change in democratic polities. Political Studies 35:482-490. Kaase, Max, and Miller, Warren E. (1978). A special report: A conference on crossnational research in the social sciences. Comparative Research 8( 1):2-7. Rokkan, Stein. (1969). Data confrontation seminars: A new device in comparative research. Scandinavian Political Studies 4~22-29. Stokman, Frans N., Ziegler, Rolf, and Scott, John, eds. (1985). Networks of Corporate Power: A Comparative Analysis of Ten Countries. Oxford: Polity Press. Taylor, Charles L., and Jodice, David A . (1983). World Handbook of Political and Social Indicators, Vol. 2: Political Protest and Government Change. New Haven; CN: Yale University Press. Verba, Sidney, Nie, Norman H., and Kim, Jae-on. (1978). Participation and Political Equality: A Seven-Nation Comparison. Cambridge, G.B.: Cambridge University Press.

Author Index Achen, C. H., 120, 121, 122 Adams, G. B., 53, 80 Alexander, K. L., 36 Almond, G. A., 169 Ancona-Berk, V. A. (Sacks et al.), 97, 100,102 Arminger, G. 137 Austin, E. W., 81 Ausubel, H. 80 Aydelotte, W. O., 80

Campbell, D. T., 99, 121, 122 Carli, L. L., 111 Cheyney, E. P., 53, 61 Clubb, J. M., 76, 81, 167 Cochran, T. C., 70, 72 Coleman, J. S., 30 Conkin, zyxwvutsrqponmlkjihgfedcbaZYXWVUT P. K., 80 Converse, P. E., 135, 169, 170-171 Cook, T. D., 121 Costner, H. L., 122 Crewe, I., 169 Curry, R. O., 80 Curti, M., 80

Bailyn, B., 68 Bales, R. F., 31 Barnes, H. E., 55 Barnes, S. H., 169 Barzun, J., 68 Beale, H. K., 55, 61 Beard, C. A., 54, 55, 57-60, 61 Beard, M. R., 55 Beck, P.A , , 169 Becker, H. S., 123-124, 137 Benson, L., 54, 64-65, 70, 76-77, 80 Berelson, B. R., 171 Berk, R. A., 122 Berkhofer, R. F., Jr., 80 Bernard, J., 130-131, 132 Berrier, J. (Sacks et al.), 97, 100, 102 Bidwell, C. E., 36 Billington, R. A., 54 Blalock, H. M., 20, 24, 25,28, 133 Bogue, A. G., 74, 80, 81 Bohrnstedt, G. W., 137 Boruch, R. F., 122 Brody, C. A., 136 Brown, R. E., 70 Bryant, F. B., 111 Budge, I., 170 Burstein, L. 20, 26, 36 Burt, R. S., 31

Dahl, B. B., 132 Dahl, R. A , , 8 Dalton, R. J., 169 Davis, J. S., 36 Davis, L. E., 63, 80 Denver, D., 169 Dollar, C. M., 81 Duncan, 0. D., 123 Eagly, A. H., 111 Easton, D., 8 Eitzen, D. S., 132 Engerman, S. L., 63, 64 Erbring, L., 36 Erlebacher, A , , 122 Eulau, H., 3, 80 Farkas, G., 36 Fienberg, S. E., 133 Firebaugh, G., 25, 36 Fiske, D. W., 94, 97 Fitch, N., 76 Flanagan, S. C., 169 Flanigan, W. H., 81 Flora, P., 170 175

Author Index

I 76 Floud, R., 81 Fogel, R. W., 63, 64 Fox-Genovese, E . , 68 Freeman, J. H., 36 French, V., 80 Garner, J. B., 80 Gates, P. W., 77, 79, 80 Geertz, C., 1 Genovese, E. D., 68 Glass, G. V., 93, 106 Glenn, N. D., 130, 138 Goodhart, L. B., 80 Goodman, L. A . , 36 Gottschalk, L., 73 Greenwald, A. G., 109 Greven, P. J., 66 Griffin, L. J., 36 Guiterman, A . , 80 Gutting, G., 69 Hamerow, T. S., 76 Hammond, J. L., 36 Handlin, O., 80 Hannan, M. T., 20,26, 36 Harding, S . , 169 Hart, A. B., 52 Hauser, R. M., 36 Hayes, E. C., 53, 5 5 Hays, S. P., 65-66, 80 Hedges, L. V., 105 Higham, J., 80 Hofstadter, R., 62 Hollinger, D. A., 69 Huckfeldt, R. R., 171 Huson, C., 36 Irwin, L., 36 Jennings, M. K., 169 Jensen, R., 54, 76, 81 Jodice, D. A , , 170 Kaase, M., 169, 171 Kaplan, A . , 39 Karl, B. D., 56, 58, 61, 72 Kasarda, J. D., 36

Kiewiet, R., 101 Kim, J., 169 Kish, L., 96, 105 Klausner, S. Z., 78 Kousser, J. M., 76 Kraus, M., 80 Kuhn, T. S., 69 Landes, D. S., 74, 80 Langbein, L., 36 Lasswell, H. D., 8-9 Lazarsfeld, P. F., 171 Lichtman, A . J., 36, 80 Lidz, V. M., 78 Lieberson, S., 122, 137 Light, R. J., 93 Lipsey, M. W., 100 Lynd, R. S . , 5

McClelland, P. D., 80 McCloskey, D. N., 78 McCormick, R. P., 64, 80 McCubbin, H., 132 McDonald, F., 70 McGaw, B., 93 McPhee, W. N., 171 McRae, J. A , , 136 Malin, J. C., 62, 81 Marini, M. M., 137 Markus, zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQ G. B., 171 Martin, E., 92 Mason, K. O., 133, 135, 136 Mason, W. M., 133, 135, 136 Merriam, C. A . , 57-58 Merton, R. K., 22 Meyer, J. W., 36 Miller, W. E., 169 Monkkonen, E. H., 76 Murphey, M. G., 80 National Bureau of Economic Research, 63 Nichols, R. F., 53, 56, 61, 71 Nie, N. H., 169 Nore, E., 5 5 North, I>. C., 63-64, 80

Author Index

177

Odum, H. W., 5 5 Olkin, I., 105

Stromberg, R. N., 80 Suzuki, T., 135

Parker, D. D., 72 Parker, W. N., 80 Particle Data Group, 102 Phillips, D., 169 Pierce, R., 169 Pillemer, D. B., 93 Poole, W. K. (Mason et al.), 133, 135, 136 Popeno, D., 132 Przeworski, A , , 36

Taylor, C. L., 170 Taylor, G. R., 80 Thernstrom, S., 66, 67, 80 Tilly, C., 74, 80 Traugott, M. W., 81 Tufte, E. R., 108, 110 I n n e r , C. F,, 92 Turner, F. J., 54, 80

Reitman, D. (Sacks et al.), 97, 100, 102 Rivers, D., 101 Robinson, J. H., 52-53 Robinson, W. S., 36 Rokkan, S., 167 Roosevelt, T., 80 Rosenthal, R., 105, 109 Sacks, H. S., 97, 100, 102 Sasaki, M., 135 Saveth, E. N., 80 Schweder, R. A., 94 Scott, J., 170 Shannon, F. A . , 72 Shorter, E., 8 1 Sibley, E., 81 Singer, B., 137 Smith, M. L., 93, 106, 109 Social Science Research Council Bulletin, 61, 69, 70, 72, 73, 80 Spaeth, J. L . , 36 Sprague, J., 36, 171 Stokman, F. N., 170 Stone, L., 68

U.S. Congress, House of Representatives, Committee on Science and Technology, 81 van Deth, J., 169 Vandermeer, P. R., 80 Vaughn, S., 62, 80 Verba, S., 169 Vinovskis, M. A . , 76 Wahlke, J. C., 8 Webb, W. P., 72, 81 Wilenz, S., 68 Wilken, P. H., 25 Winsborough, H. H . (Mason et al.), 133, 135, 136 Wise, G., 69 Wish, H . , 80 Woodward, C. V., 68 Wortman, P. M., 111 Young, A., 36 Ziegler, R., 170 Zinn, M. B., 132 Zunz, O., 80