Applications of Methods of Evaluation [Reprint 2020 ed.] 9780520321908

Citation preview

APPLICATIONS OF METHODS OF EVALUATION Four Studies of the Encampment for

Citizenship

APPLICATIONS OF METHODS OF EVALUATION Four Studies of the Encampment for Citizenship BY

HERBERT H. HYMAN, CHARLES R. WRIGHT, and TERENCE K. HOPKINS

U N I V E R S I T Y OF C A L I F O R N I A P R E S S B E R K E L E Y AND LOS A N G E L E S 1962

U N I V E R S I T Y OF C A L I F O R N I A P U B L I C A T I O N S IN C U L T U R E AND S O C I E T Y EDITORS: W . A . L E S S A , R I C H A R D C E N T E R S , R . T . M O R R I S , R . H . T U R N E R

Volume 7

U N I V E R S I T Y OF C A L I F O R N I A P R E S S B E R K E L E Y AND L O S A N G E L E S CALIFORNIA -OCAMBRIDGE UNIVERSITY PRESS LONDON, ENGLAND

©

1 9 6 2 B Y T H E REGENTS OF T H E UNIVERSITY OF C A L I F O R N I A PRINTED IN T H E UNITED STATES OF A M E R I C A

PREFACE of action research, Kurt Lewin once commented that "research that produces nothing but books will not suffice." Yet, here we are about to present a book on action research. It is not because we reject the good advice. Lewin's remark was in the context of urging research that would serve the goals of rational social action, and he stressed the need to accompany planning and action with evaluation of their effectiveness. The evaluation is the objective measure of success or failure, the source of new knowledge, and the guide to improved action. This is what research should do, not merely produce books. To encourage the alliance, to join the two—evaluation and action—is not easy. But it is equally difficult to develop the principles, methods, and procedures of evaluation research. It is research that must be done under difficult conditions, and upon concepts that are generally obscure, confused, or complex. Any book that would serve to clarify the problems in such research and develop the method certainly works in the service of larger action goals, and it is in this spirit that we present it. Our book is a case study of research conducted in the years 19551959 to evaluate a particular social action program, the Encampment for Citizenship. The many ventures in character education of youth may well profit from the example to be presented. Beyond this, the method of evaluation will have formal similarities, no matter what type of program is involved. We were fortunate in having extended opportunity over these years to elaborate instruments and procedures, to see whether they stood the test of time and repeated application, to compare different designs and solutions for particular problems. The presentation of all this in great detail may be of value in furthering the ideal of evaluation of planned programs of social action, and in improving the quality of evaluations in practice. Our appreciation is expressed to the Encampment for Citizenship and its staff for their invitation to conduct the research. We must note our admiration for their courage in trusting a venture that was so dear to their hearts to appraisal by so cold and heartless a method, and we thank them for unstinting aid without interference. The special cooperation of Algernon D. Black, education director; William G. Shannon, former executive director; and Saal D. Lesser, the present executive director, is noted with thanks. Support for the research was provided by a grant from the SchwarzI N A DISCUSSION

[v]

vi

Preface

haupt Foundation, plus grants-in-aid from the Ford Foundation, and the Columbia Council for Research in the Social Sciences. Research assistance by Roslyn Menzel, Carolyn Dexter, and Sanci and John Michael is gratefully acknowledged. We are grateful to Dr. Henry Riecken, who contributed unpublished material from his evaluation of volunteer work camps. All the authors were members of the Bureau of Applied Social Research, Columbia University. In 1956 Professor Wright joined the faculty of the University of California at Los Angeles, but he continued his participation as one of the codirectors of the research. The research facilities of the Bureau, technical advice by its staff, administrative help, and support in preparation of the manuscript contributed much to our work. TT TT „ ±i. ±1.

XI.

C. R. W. T. K. H.

CONTENTS PAKTI I. Principles of Evaluation Introduction Conceptualization and measurement of the objectives of the program and other unanticipated relevant outcomes Conceptualizing the objectives of the program Conceptualizing unanticipated consequences Research design and the proof of the effectiveness of a program . . . . Controlling for extraneous sources of change by experimental designs . Controlling for effect of repeated testing Workable alternatives to the classic control-group design Research procedures and the reduction of error Quality of response Biases resulting from nonresponse Problems of index construction and the proper evaluation of effectiveness . Weighing effectiveness in the light of restricted ceilings for change . . Weighing effectiveness in terms of individual changes versus net changes Weighing effectiveness by combining discrete aspects of change . . . Weighing the amount of effectiveness and tests of significance . . . . Understanding the findings on effectiveness Describing the program Describing the subjects Differential effects among contrasted types of subjects The contribution of different aspects of the program Inquiry into the processes by which the program produces effects . . .

3 3 6 9 12 17 17 29 37 53 54 59 63 64 66 68 69 72 74 79 81 84 85

PART I I I I . Who Are the Campers? Introduction Diversity within the total Encampment A portrait of the subgroups Campers' social values, attitudes, and opinions Social values on the first day of camp Campers' conceptions of social problems Initial opinions and attitudes on social issues The camper and the rest of society I I I . The Summer's Experience Introduction The nature of the Encampment The educational curriculum The communal life The summer as seen by the campers Expectations, fulfillments, and disappointments Reactions to the formal program

[vii]

89 89 93 95 96 98 104 107 116 119 119 120 120 122 123 123 124

viii

Contents

Reactions to the nondidactic features of the Encampment Disruptive elements during the Encampment and some of their determinants Salient elements in the Encampment from the campers' view Campers' self-estimates of change IV. Immediate Changes Created by the Encampment Introduction Begions of change Changes in salient social attitudes and opinions Changes in basic values Changes in action orientation Changes in cognition of social problems Changes in campers' perceived relationship with the rest of society . . Democratic conduct during the summer Susceptibility to prestige symbols An experiment in resistance to propaganda Would there be improvement without an Encampment? Summary interpretation V. The Dynamics of Change During the Encampment Introduction Exploring factors responsible for changes in campers' attitudes during the Encampment—didactic and communal aspects The contribution of other factors Disruptive experiences and change Bace and ethnicity as factors governing change The ecology of the California Encampment: preliminary exploration . Processes underlying changes during Encampment VI. The Return Home Introduction The study of the 1955 group Perception of the home community Informal relationships after the return home Attempts to apply Encampment principles within the home setting . . Subjective reports of changes after the Encampment VII. Stability of Effects within the Home Setting and Its Determinants . . Introduction Begions of change Salient social opinions and attitudes within the home setting Action orientation in the home setting Cognition of social problems Perceived relationship with the rest of society Differential patterns of stability as related to ethnicity and social setting . Stability of effects as related to social support VIII. Effects in the College Setting Introduction College and the Encampment—the groups studied The college subgroup The control group

125 129 133 135 137 137 138 138 143 146 147 151 154 158 158 161 166 167 167 170 179 179 181 182 183 191 191 194 196 201 203 208 210 210 211 211 212 214 215 217 222 226 226 227 229 230

Contents The findings Campus political life Academic life The value of a college education Conclusion I X . Long-Range Effects Introduction The quality of the evidence The old Encampment—forgotten episode or living memory? The intervening years Regions of change Salient social opinions and attitudes—four years later Changes in cognition of social problems Alienation and anomie Long-range impact on conduct Change as an abrupt or erosive phenomenon Stability and change by race, social context, and history Stability in relation to post-Encampment contacts and reference groups . X . Long-Range Effects Among Alumni of Earlier Encampments Introduction Procedural difficulties and error Response error Sampling problems and nonresponse The findings and their agreement with the 1955 follow-up

ix 241 241 250 251 259 262 262 264 267 269 270 270 273 274 276 283 286 290 297 297 307 307 309 313

APPENDIXES

A. Attitude Scales Used in Measuring Effects of Encampment for Citizenship B . Questionnaires C. The Use of an Index of Effectiveness to Correct for Ceiling Effects . . . D. Turnover Tables and Tests of Significance

325 336 379 381

INDEX OF N A M E S

395

TABLES AND

CHARTS

TEXT TABLES 1-1. Cumulative effects of retesting on changes in opinion during 1955 Encampment 38 1-2. Number of applicants to the Encampment and sources of attrition . . 39 1-3. Influence of the time interval on the estimate of change due to uncontrolled factors, 1955 50 1-4. Nonresponse in studies of immediate effect 60 1-5. Comparison of nonrepliers and repliers to follow-up six weeks after 1955 Encampment 62 2-1. Social characteristics of the four Encampments 92 2-2. Criteria used in evaluating occupations by 1955 campers and nation's young persons 100 2-3. Comparison between campers and nation's high school seniors: political apathy 104 2—4. Comparison between campers and nation's high school seniors: opinions on civil rights 108 2-5. Comparison between campers and nation's high school seniors: opinions on civil liberties 110 2—6. Summary view of campers' opinions and attitudes scores at beginning of 1955 Encampment 114 3—1. Disruptive experiences during each Encampment 130 4—1. Summary of changes in salient social opinions and attitudes during each of four summer Encampments 140 4—2. Comparison of change in certain scores for 1955 campers and Eiecken's and Dentler's subjects . . . 142 4-3. Stability of campers' vocational values during each of four summer Encampments 144 4-4. Friendships within and between ethnic groups of campers . . . . 156 4-5. Susceptibility to prestige symbols in four Encampments 160 4-6. Campers' "natural" improvement during six weeks preceding the Encampment, 1955 and 1957 162 4-7. Changes in college campers, 1957 and 1958, compared with changes in control group of college students 164 5-1. Relation between changes in attitudes and difficulty in following didactic program 172 5-2. Eelation between campers' changes in attitudes and amount of previous formal education 174 5-3. Eelation between campers' changes in attitudes and defection from didactic program 176 5-4. Eelation between campers' changes in attitudes and defection from communal life 178 5-5. Eelation between change in attitudes and initial orientation toward education and community life 180 5-6. Eelation between campers' image of staff members and changes in vocational goals 188 [xi]

xii

Tables and Charts

6-1. Summary of campers' desires and attempts to apply Encampment principles in society 205 6-2. Campers' reactions to witnessing an undemocratic practice in society 207 7—1. Influence of home community on changes produced during 1955 Encampment 218 7-2. Summary of differential changes within the home setting as related to ethnicity (1955 data) 220 7-3. Differential effects upon return to the home community as related to Southern residence (1955 data) 222 7—4. Application of Encampment in action as related to potential or faceto-face interaction with other campers (1955 data) 224 8-1. Basic characteristics of college panel 229 8-2. Control group versus Encampment-experimental group: initial expectations and motives 235 8-3. Control group versus Encampment-experimental group: initial differences on various attitude scales 236 8-4. Control group versus Encampment-experimental group: sex, age, major subject, and occupational goal 238 8-5. Participation 242 8-6. Tolerance on the campus 245 8-7. Tolerance of left-wing campus political groups 246 8-8. Interpersonal relations 248 8-9. Opinions on frequency of discussion of controversial issues in the classroom 249 8-10. Academic life 252 8-11 Commitment to a college education: commitment scale scores . . . . 254 8-12. Commitment to a college education: various single indicators . . . . 255 8-13. Comparison of campers' and other young persons' attitudes toward college 257 8-14. Value of a college education: degree of value 258 8-15. Kind of value placed on a college education 260 9-1. Persistence of Encampment experience 268 9-2. Long-term stability and change in post-Encampment views . . . . 271 9-3. Long-term post-Encampment conduct and vocational goals . . . . 278 9-4. Post-Encampment political participation of ex-campers according to their current agreement with Encampment norm that political activity is important 282 9-5. Patterns of short- and long-term stability and change for sixty-five campers measured at four points in time, 1955-1959 284 9-6. Differential effects on ex-Campers of a combination of "favorable" or "unfavorable" social contexts 288 9-7. Stability and change in opinions as related to post-Encampment reference-group support 294 10-1. Credibility of retrospective reports of events as related to time away from Encampment 308 10-2. Losses in sampling of alumni from official mailing list of addresses . 309 10-3. Losses in sampling of alumni due to nonresponse 311

Tables and Charts

xiii

10-4. Indices of motivational involvement in the Encampment for samples obtained from various alumni years 313 10-5. Long-range stability of effects as revealed by the alumni design and the direct follow-up 314 10-6. Political behavior subsequent to the program as revealed by the alumni study and the direct follow-up 318 10-7. Vocational goals and careers as revealed by the alumni study and the direct follow-up 319 10-8. Decline in alumni's interaction and the Encampment reference group with years of separation 320 A P P E N D I X TABLES

C - l . "Effectiveness" in the areas of civil rights (discrimination) and tolerance, using the Hovland-Lumsdaine-SheflSeld index of effectiveness . . D - l . Changes during the Encampment: turnover from Wave I I to Wave I I I D-2. Changes over the six weeks before the Encampment: turnover from Wave I to Wave I I D-3. College campers compared with the control group of college students, changes from spring to fall, 1957 and 1958 combined: turnover from Wave I to Wave IV D-4. Stability of gains in the six weeks after the Encampment in 1955: turnover from Wave I I I to Wave IV D-5. Stability of gains in the four years after the 1955 Encampment: turnover from Wave I I I to Wave V D-6. Comparison of each camper's attitude toward civil liberties at beginning and end of encampment

380 383 388 389 390 391 392

CHARTS

1. Conceptualization and measurement of the objectives of the Encampment and other unanticipated consequences (major studies) 2. Schematic representation of experimental designs 3. Schema for observing the 1955 Encampment on a time-sampling basis . .

18 52 78

CHAPTER I

PRINCIPLES OF EVALUATION INTRODUCTION THROUGHOUT HISTORY, men have pooled their energies to change some state of affairs and to achieve some common social goal. Some of these efforts succeed, but many fail. Usually, only the historical record documents whether the effort was a success or a failure. Sometimes the lessons of history are swift and dramatic and clear in their meaning. We may learn the hard way, but, nevertheless, we learn that an action is ineffective. At other times, history is not a very good teacher. The verdict of success or failure may come too slowly, after much energy has been wasted and our follies have gone too far. Disaster is not a teacher to be welcomed. Even when the scale of an action program is minor, economy is to be welcomed. At still other times, the lesson of history is not so clear. The outcome of a course of action may be ambiguous; its meaning may be complicated and perhaps misleading. Is there not a more efficient and intelligent guide to evaluating social action? The question becomes all the more compelling as the scale of efforts is enlarged. Ours is an age of action programs, where large organization and huge expenditures go into the attempted solution of every conceivable social problem. As a small assortment of these, but not in order of importance, consider programs of social casework; criminal rehabilitation; treatment of delinquents; dissemination of opinions and knowledge on a national and an international scale; training of workers, foremen, supervisors, and executives; adult education; health information and practices; technological assistance; cultural exchange; reduction of intergroup conflict; mental hygiene; general education; adjustment of the aged; training of soldiers; and treatment of alcoholism.1 There is all the more need today for rational management of such enterprises, for a spiral, to use Lewin's figure, "of steps each of which is composed of a circle of planning, action, and fact-finding about the result of the action."2 Evaluation refers to the procedures of fact finding about the results of planned social action, which in turn move the spiral of planning ever upward. It is the proper methodological accompaniment to rational action. Done well, the accompaniment is almost musical in its appeal. Men have, of course, always attempted to find out the facts about the results of their actions. The theme has al1 Adapted from an unpublished memorandum by Dr. Henry Eiecken on the uses of evaluation. ' Kurt Lewin, Resolving Social Conflicts (New York: Harper, 1948), p. 206.

[3]

4

Applications of Methods of Evaluation

ways been with us; it is the performance that is different. As Klineberg remarks, evaluation should be restricted to fact-finding methods which yield evidence that is "objective, systematic and comprehensive," and should be distinguished from other forms of assessment which do not satisfy these criteria.8 The problem is to develop the general method which satisfies these criteria, and to abstract the principles that underlie the method. But more than this is needed. The principles and the method must take on a specialized form suited to the particular circumstances of the action program to be evaluated. Effective application of the principles to the particular case, special modifications of procedure within the general framework—this is what must be learned. It is to this end that our work is presented. It consists of two parts, a general statement of principles of evaluation, and a detailed exemplification of the principles in a series of evaluative studies conducted in one action program. Even if the program studied were a rare and exotic one, the presentation would still have the virtue of showing the translation of the method into operative research. From one point of view, however, the program is one of a large class. In their quest for a better world, many agencies put their faith in youth, and a variety of programs designed for young persons are in operation. Riecken notes that one special variety, programs of work service, accounts "for more than one hundred projects annually in the United States, sponsored by more than thirty different organizations."1 Other types of youth programs abound, all intended to promote growth, wisdom, competence, character development. The Encampment for Citizenship—EFC—takes on its own special form, but is in the spirit of this larger general endeavor. It brings together each summer about a hundred young persons from many parts of the United States, diverse in creed and race, but united in their dedication to a better world, their strong and serious interest in social problems, and their potentialities for future leadership roles in our society. For six weeks these young persons live together in a community permeated, in principle, by an atmosphere of equality, presumably enriching their experiences, and being stimulated by a varied educational program of lectures, workshops, discussion groups, and field trips, all designed to increase skills in democratic living and inculcate values apposite to an ideal world. Each such Encampment is regarded as a pilot project in democratic education to convey the facts that one hundred persons are but a tiny 3 O. Klineberg, "The Problem of Evaluation," Internat. Soc. Sci. Bull., VII (1955), 347. 4 H. W. Riecken, The Volunteer WorTc Camp: A Psychological Evaluation (Cambridge: Addison-Wesley Press, 1952), p. 1.

Principles of Evaluation

5

army in the service of so great a cause, that the ultimate testing ground is the remote future, and, finally, that this is an ongoing endeavor— imperfect, perhaps, but changing by benefit of practice in such a difficult undertaking. Over fifteen summers have now passed since the initiation of this experiment. More than fifteen hundred young men and women have come into the ranks of the Encampment and have returned to their respective homes. In 1955, in preparation for the tenth Encampment, the sponsors felt the time had come for a scientific evaluation to be undertaken. The responsibility for this evaluation was entrusted to the Bureau of Applied Social Research of Columbia University, and was assigned to the authors, who enjoyed complete freedom of inquiry. The evaluation of the 1955 program was the beginning of a long sequence of studies. In 1957 and 1958 the program was again evaluated, making a total of four studies, because in 1958 two similar Encampments were in operation. The persistence of effects was subject to a number of different studies. The 1955 campers were sent questionnaires at their home communities six weeks after their departure from the Encampment and were restudied some four years later. The experiences of alumni of Encampments before 1955 were reconstructed by a special method, thus providing evidence on campers removed from exposure for periods as long as nine years. These many studies in evaluation provide the case material to be presented in subsequent chapters. The technical features of such inquiries are exceedingly complex, and it was natural that a specialized research organization should be requested to undertake the task. Further, the requirements of objectivity and impartiality demanded that outsiders conduct the inquiry without any constraint upon their activities or conclusions. The complex nature of evaluation, however, requires the participation of the agency which is open to scrutiny. Whereas the researchers are best able to develop scientific methods for evaluating effectiveness, the ultimate goals to be achieved are defined by the action agency. The wisdom of these goals cannot be ascertained by scientific inquiry. The achievement of goals is measured indirectly by translating them into a series of measurable, component, or subsidiary objectives which in totality combine to represent the larger goals. This translation was primarily the responsibility of the research workers, after consultation with the officials of the Encampment. All the studies address themselves to the prime problem in evaluation: to provide objective, systematic, and comprehensive evidence on the degree to which the program achieves its intended objectives plus the

6

Applications of Methods of Evaluation

degree to which it produces other unanticipated consequences, which when recognized would also be regarded as relevant to the agency. It is easy to state the problem of evaluation, but it is difficult to develop the method for its solution. The method, in our formulation, has five major phases, each of which involves a body of principles. The translation of the phases and of the abstract principles into actual research will be presented in fullest form in subsequent chapters. Frequent reference to our studies and to other literature will be incorporated into the present discussion of the methodology of evaluation research. Only in this way, by its utility and applicability to a real problem, can the appropriateness of the method be demonstrated and real understanding be fostered. CONCEPTUALIZATION AND MEASUREMENT OF THE OBJECTIVES OF THE PROGRAM AND OTHER UNANTICIPATED RELEVANT OUTCOMES

Although there are basic principles and specific rules to guide all five phases of evaluation, the obscurities in the phase of conceptualization defy the principles most of all. The design of proof is a classic problem in science; the properties of good measuring instruments are clearly stated; the corresponding procedures are laid out and the knowledge of the past has been codified. In the four other phases the evaluator, like any investigator, follows a well-marked trail. By contrast, conceptualizing any phenomenon in the prelude to research is an act of mind, of the imagination. How does one codify principles and rules for effective use of imagination? There are some useful guides, to be sure, but our ultimate hopes should not be too high. In scientific inquiry where the investigator chooses what he wants to study, the same difficulties are faced, and the counsel provided is not very encouraging: "No rules can be stated for 'hitting upon' relevant hypotheses."5 In a recent work on the methodology of social research, which places special emphasis on the phase of concepts and indices and the appropriate formal procedures, Lazarsfeld and Rosenberg ask the basic question of how units and variables are formed, and they describe the beginning in terms that emphasize the difficulty. "The first step seems to be the creation of a rather vague image." They ask, "How does one 'think up' a number of indicators?" They talk of "this speculative phase of the task."" The evaluator is in the same fix as any curious investigator, with one exception that makes his task both easier and harder. The ordinary investigator perhaps has to be more imaginative, as he originates his inquiry 5 M. R. Cohen and E. Nagel, An Introduction to Logic and Scientific Method (New York: Hareourt, Brace, 1934), p. 202. 8 P. F. Lazarsfeld and M. Rosenberg, eds., The Language of Social 'Research (Glencoe, 111.: Free Press, 1955), pp. 15-16.

Principles of Evaluation

7

and does not have the benefit of the hints to conceptualization inherent in the intended objectives of the program. But if it turns out that he is not imaginative in conceptualizing his problem, he can always drop it and turn to something else. The evaluator is guided by the objectives set down, but at the same time his imagination is disciplined, perhaps harshly, by the requirement to conceptualize in manageable terms the preestablished objectives of the program. The cosmic way in which many social action programs state their objectives makes the task no easy one, but the evaluator cannot escape it and turn to more manageable problems. Consider the gifts of mind needed to conceptualize for evaluation a program of adult education which has the defined aim of "development . . . of personal abilities and aptitudes, and the encouragement of social, moral and intellectual responsibility within the framework of local, national and world citizenship.'" Riecken, when called upon to evaluate the work-camp program of the American Friends Service Committee, experienced similar ambiguities, plus some additional ones. After studying the documents describing the program, he reported: "The grave difficulty is that we have been unable to discover in these writings a simple, clearly and comprehensively stated set of aims that will meet with the universal endorsement of the directors of the program."" The many difficulties suggested—the breadth of the thing subsumed under a particular objective, the multiple objectives encompassed by many programs, the ambiguity inherent in any or all of the objectives as stated, and the disagreement as to the objectives—are characteristic of many programs and are enough to stagger the imagination of the evaluator. Even when a program has a seemingly simple, unambiguous, unitary objective, the ease of conceptualization and subsequent measurement is often deceptive. Alternative definitions of the program's objective come to mind. Meyer and Borgatta, in evaluating a program designed to rehabilitate the mentally ill, note that "although the most general and most objective criterion of success for a person who has been hospitalized for mental illness is his permanent success in avoiding reinstitutionalization, lesser as well as greater degrees of 'health' or 'adjustment' suggest themselves as appropriate criteria if measures can be determined for them." Neglecting for the moment the snare confronting the evaluator in the innocent word "permanent," they list some of the alternative definitions of success: "A second measure of recovery may be defined in terms of the ability to govern one's emotional affairs ' E. M. Hutchinson, quoted in K. M. Miller, "Evaluation in Adult Education," Internat. Soe. Sci. Bull., VII (1955), 431. 8 Op. cit., p. 27.

8

Applications of Methods of Evaluation

independently.... A third measure of recovery may be the inclusiveness of a person's social relations.... A fourth measure of recovery may be economic independence, and a fifth the reality of general orientation. ... A sixth measure of recovery may be one based on self-attitudes."9 These many kinds of problems confronted us. The Encampment states that its goals are "to prepare young Americans for responsible citizenship and citizen leadership, to educate them in the meaning of democracy... and to train and equip them in the techniques of democratic action." Elsewhere, a sampling of goals includes training in "freedom with responsibility," the provision of "information and clarification on current issues," the reduction of "confusion," "helplessness," and "apathy."10 In order to evaluate the effectiveness of the Encampment, it is a prerequisite that such objectives be subject to measurement. But to subject them to measurement we must first be able to translate these profoundly important but nevertheless broad and abstract statements into a series of simpler concepts. What is subsumed under "responsible citizenship"? Under "techniques of democratic action" and "freedom with responsibility"? The series of simpler concepts will then be employed to measure the original objectives. The difficulties are still not done with. What about the unanticipated relevant consequences that might stem from the program? The camper might learn the meaning of democracy, but in the process of boiling down the creed his understanding might be reduced to a provincial or, as we subsequently labeled it, "ethnocentric" image of democracy, allowing for no respect of other forms of the democratic spirit. Suppose his apathy is converted not merely into activity, but rather into a hotheaded and premature activism. Suppose the sense of helplessness disappears, but is replaced by an overoptimistic sense of the ease of social change. Suppose leadership emerges at the price of the formation of a leadership elite which feel alienated from and superior to the mass of people. The problems of evaluation of character education are complicated by the fact that there are no watertight compartments within the character. A change is initiated which may know no bounds, and good and bad consequences may run side by side. Consequently, it would be most misleading if we were merely to evaluate objectives specifically sought by the Encampment without regard for other consequences, toward or untoward. But herein lies a special difficulty: How shall the Rehabilita" H. J. Meyer and E. F. Borgatta, An Experiment in Mental Patient tion (New York: Russell Sage Foundation, 1959), pp. 26-27. 10 Encampment for Citizenship, official brochure (privately printed, 1954).

Principles of Evaluation

9

evaluator anticipate the consequences of social action unanticipated byothers? Granted that certain broad and abstract objectives of the program must be translated into a much larger number of specific concepts, which are then supplemented by other concepts related to unanticipated outcomes, how shall this be done? The principles we shall advance will be of some help in harnessing imagination; but, above all, the gifted imagination is needed. CONCEPTUALIZING THE OBJECTIVES OF THE PROGRAM

A first principle is that some attempt must be made to analyze the kinds of formal entities that are involved, to locate the regions within which the concepts are set. Do the concepts pertain to an individual or to a collective? Or do they pertain to both these regions in that the effects on the individuals are then intended to ramify to others and in turn to organizations, even whole societies? This is a first step toward clarification, for the concepts, procedures, and instruments will differ, depending on the region to be explored.11 Other coordinates may be used to distinguish major regions of concepts. What temporal reference is implied by the program? Are the concepts to be elaborated present states of the individual or of the collective, or future states, or both? In the Encampment program, the region under study is clearly the effect on the individual. B y implication these young persons are supposed ultimately to ramify their influence, but in the perspective of the immediate present and the short-term future, the location of the concepts remains in the individual. Using the first principle, the regions that are located are more properly called continents. Next, within a general region or continent, subregions must be specified. It is not difficult to implement this second principle in evaluations of effects on individuals. Psychology is rich in models of the individual which locate every subregion, even the most subterranean. An obvious and useful principle is to distinguish the subregion of overt conduct from the inner states of the individual. Thus it is immediately clear that the Encampment is engaged in a program of character education. Many of the stated objectives have no connotation of immediate overt conduct. Quite properly, the program is ultimately concerned with the way these young persons will behave when they are adults, in their future performance as citizens, but the means to this end is to develop the character structure that will govern such 11

See, for example, M. B. Smith, "Evaluation of Exchange of Persons," Internat.

Soo. Soi. Bull., VH (1955), 389.

10

Applications of Methods of Evaluation

adult roles. Many of the concepts we started with in the study of the current group of young persons were in this internal region. Attitudes, beliefs, values, ideals, and the like are presumed to be the inner directives to the course of future conduct, and were measured by traditional, acceptable instruments involving batteries of questions which tap these inner states. Once the subregions have been selected, they must be explored. They too are almost subcontinents. How shall the specific concepts be pinpointed? Again, models are available in the literature. Are we interested in the realm of knowledge, or in a system of beliefs, or in attitudes, or in motivations, or in values?" Such classes of concepts are plentiful, and they guide the evaluator toward home base. In conceiving of these subregions, it is useful to invoke a principle of levels or hierarchy. Some aspects of an individual's character are more fundamental than others; some are fixed earlier in life; some are less amenable to change. In our studies, we wished to do justice to this notion of a hierarchy within the total character structure. Could the Encampment or any other brief interlude inserted in the life of a relatively mature person do more than change the peripheral? In our evaluation of the current group of subjects, we elaborated six subregions within the general region of the internal system of the individual, some regarded as more central and others as more peripheral. All six appeared to be related to one or more of the explicit objectives of the Encampment program. The ultimate problem is to elaborate the specific concepts and variables within each subregion that will be measured. The best principle is that there is safety in numbers. We tried initially to be as comprehensive as possible. The Encampment is concerned with a wide array of changes in character. To do justice to the breadth of the program, the number of concepts had to be many. Moreover, not every measuring instrument would work effectively. A single finding might be subject to dispute, whereas many consistent findings would be strong evidence. "What leads or guides are there to the specific concepts at each level within the general region of character? A stated objective sometimes translates itself directly. When it does not, conference with the staff members of a program helps to clarify meaning and intent. For a program that has operated for many past cycles there is an additional and usually abundant resource. Many so-called evaluations make use of tes12 The first four of these were the framework for the evaluations of orientation films conducted by Hovland and his associates. See Carl Hovland, Arthur A. Lumsdaine, and Fred D. Sheffield, Experiments on Mass Communication (Princeton: Princeton University Press, 1949), pp. 33-45.

Principles of Evaluation

11

timonials from alumni, or of files of correspondence. Although, these sources are dubious as evidence for a final evaluation, they provide rich leads for concepts, which then can be subjected to rigorous tests to determine the frequency of their occurrence as effects of a program. A final guide is the body of past literature, either theoretical writings about the region within which the concepts are located, or past evaluations of similar programs. In this respect we were fortunate in being able to profit from the work of Dr. Henry Riecken, who had conducted a pioneering, full-scale evaluation of the work-camp program of the American Friends Service Committee. Many of his concepts and corresponding measuring instruments were adopted in our study, with the result that comparisons can be made between two very similar ventures in character education." In reviewing the research there is a mental oscillation between conceptualizing and the other phases—design of the inquiry, choice of population or sample, and problems of research procedure. Sometimes these technical considerations work to restrain the beautiful flow of concepts, but sometimes they speed it. Where an instrument is ready at hand from a previous inquiry, the concept—even though less relevant— may be incorporated, with the prospect that measurement will be good, since the instrument has already stood the test of past research. More importantly, for reasons to be presented, the incorporation of such concepts and instruments yields comparative data for other groups—a very powerful tool in evaluation. If it does not appear feasible that an instrument can be developed, a concept may be dropped. For example, in the Encampment program attention is given to increasing specialized knowledge and skills and particular intellectual capacities which are relevant to effective citizenship. The serious problems of developing and administering achievement tests led to the decision to omit the area, with one exception. Moreover, it seems very likely that such learning would be readily accomplished. The more crucial question appears to be whether such equipment in knowledge and skills is accompanied by the changes in character which would lead the campers to harness their equipment and ultimately act upon it. One dimension relevant to the Encampment's goals of training in skills, knowledge, and specialized intellectual capacities was included because of its special methodological interest and the challenge it posed. Many psychological variables are amenable to measurement by ques13 Dr. Riecken was most helpful in making available unpublished data which aided in the planning and analysis of our evaluation.

12

Applications of Methods of Evaluation

tionnaire techniques, and this is a standard and efficient apparatus for evaluations such as ours. To measure certain types of capacities—for example, memory, or motor skill—ideally one would wish to test an individual's performance under stimulation on some appropriate experimental task. How to undertake such cumbersome procedures and incorporate them into the routine measurement process is a challenge. It can be accomplished, however, if the questionnaire is treated not merely as a set of questions for collecting information, but as a quasiexperimental vehicle, with many forms into which different levels of stimulation can be inserted and the corresponding responses observed. Chapter iv reports an experimental use of the questionnaire as a device to test whether or not the Encampment trains critical capacity or resistance to prestige symbols as propagandistic devices. In no way are the findings intended to suggest that this region of effects has been studied comprehensively. They merely serve to show that more may be done in the evaluation of capacities through ingenious and inexpensive quasi-experimental manipulation of questionnaires. Research design and choice of population also merge with attempts at conceptualization. In our planning, it was decided to take measurements of the stability of effects after the 1955 group had returned to their homes. In the arena of the home community, the inner region of character is not the only relevant one. Conduct is no longer in the unmeasurable future. Now conduct can occur, and it becomes practical to conceptualize. It was also decided at various points in our studies to extend the research into long-range effects in this group after the passage of several years and to study even older individuals who were alumni of past Encampments. There conduct becomes even more important, and conceptualization and measurement of this seventh region of intended effects had to be incorporated into the studies. CONCEPTUALIZING UNANTICIPATED CONSEQUENCES

The explicit objectives of a program provide a direct lead or a hint which gives the evaluator a start on his conceptualization. "What does the evaluator do when he comes to conceptualization of the unanticipated consequences of a program that may well be relevant to his conclusions? Since the action agency did not anticipate them, how can the evaluator? It is not so difficult as it might appear. Four sources of hints are available. First, for programs that have been in operation before, even though some consequences were unanticipated, they have nevertheless demonstrated themselves in previous cycles. The correspondence

Principles of Evaluation

13

of alumni, the records of the past, are bound to reveal the unexpected, perhaps too late to be taken into account by the planners of that earlier cycle, but not too late to be conceptualized in an evaluation of a new cycle of the program. Second, an unanticipated consequence often is simply an extreme quantitative value of an intended effect, but at the extreme value it is transformed in its meaning. For example, the Encampment program aims to convert the sense of helplessness into the sense of competence. The evaluator must be even more farsighted about the process. One more step, and perhaps a sense of competence is transformed into overconfidence. Third, by distinguishing four types of effects of a program, one arrives at a rather bizarre type of unanticipated consequence, about which the evaluator is given many hints. An agency has anticipated certain effects which it regards as desirable and intends to achieve. These become the explicit objectives. They justify the initiation of the program. There are other unanticipated effects of the program, some of which, if recognized, would be regarded as desirable and others as undesirable. The fourth cell includes those effects which an agency might have originally anticipated as possible effects of its activities, but which are regarded as undesirable. These are the anticipated, unintended objectives. Once these are recognized, the agency obviously tries to incorporate procedures into its program for preventing their occurrence, and may sometimes assume that it is successful. A program is planned, so to speak, to achieve certain things and avert others. The agency may often seek through evaluation to prove that it achieves the desirable goals; it may on occasion neglect to seek proof that it has averted the undesirable, simply because it assumes it is successful in this respect. It is not playing with words to say that in such instances unintended objectives, anticipated at one time, may at a later time become unanticipated consequences, as they are no longer assumed to occur. Such possibilities are revealed readily in reviewing a program with an agency. In the instance of the Encampment program, the agency had been sensitive to the possibility that in adopting a certain system of beliefs about politics, democracy, international relations, intergroup relations, and the like, the camper might enlarge this family of attitudes and adopt an ideology about domestic economic affairs. The Encampment wishes explicitly to avoid this outcome, and the program therefore omits this sphere. Yet, in the nature of psychic processes, it is perfectly possible that the economic and noneconomic somehow merge, and that the Encampment pro-

14

Applications of Methods of Evaluation

duces as an unanticipated consequence an increase in economic radicalism." Or, to take a cue from Riecken, although economic attitudes may not be communicated in the didactic parts of the program, these may be communicated informally by the staff members if their ideologies include not only liberal noneconomic attitudes, but also liberal economic attitudes.15 Certainly this is a hypothesis worthy of testing, and therefore a scale on politico-economic conservatism was included among the instruments. Most agencies do not reject the possibility that an unintended consequence may still occur despite their best efforts to avert it. Although it would not be correct to label such outcomes unanticipated, it would be important to conceptualize and measure changes in these variables in the course of an evaluation. Fourth, the literature provides much guidance in conceptualizing unanticipated consequences. One thing, however, it surely proves. Thoughtful investigators have anticipated the unanticipated, and have systematically measured such consequences from various forms of stimulation and treatment. Riecken's evaluation proves that the task can be done, as do other examples yet to be cited. Because of the parallels in the two programs, we incorporated his concept of alienation, and, using a variety of instruments, measured whether this was an unanticipated consequence of the Encampment program. Even where an earlier investigator did not anticipate a particular outcome from some program of stimulation or treatment, its unexpected occurrence is generally reported after the fact, and the next investigator or evaluator profits. Thus, "boomerang effects" from programs of mass communication were not anticipated in the flush of early optimism about the potency of mass media, but as they cropped up they were reported.16 Hovland and his 14 We do not suggest that there must be a positive correlation between attitudes in the economic and noneconomic realms. Considerable evidence of a generalized pattern of liberalism encompassing both realms has, however, been reported. For a critical review of the early literature see G. Murphy, L. Murphy, and T. Newcomb, Experimental Social Psychology (New York: Harper, 1937), pp. 1027 ff. In the recent period, the most impressive support for the generality of liberal attitudes is presented in T. Adorno et al., The Authoritarian Personality (New York: Harper, 1950). Contrary evidence showing that certain social groups exhibit radical economic attitudes along with conservative noneconomic attitudes is also available. See, for example, G. H. Smith, "The Relation of 'Enlightenment' to Liberal-Conservative Opinion," J. Soc. Psychol., X X V I I I (1948), 3-17. Obviously, special conditions may lead to all forms of patterned relationships between the two spheres, but it was certainly plausible, and worthy of being tested, that the Encampment might produce changes in economic attitudes. 15 Op. cit., p. 30. 16 For an account of unanticipated consequences or so-called boomerang effects of mass communications, see P. F. Lazarsfeld, "Communication Research and the Social Psychologist," in Current Trends in Social Psychology (Pittsburgh: University of Pittsburg Press, 1948), pp. 260-266.

Principles of Evaluation

15

associates incorporated this general notion of a "boomerang" into their planning of the series of evaluations of orientation films produced for American soldiers, and elaborated many specific unanticipated consequences which were systematically measured.17 The four avenues to conceptualizing unanticipated consequences are of considerable help. They do not insure that all the relevant unanticipated consequences will be conceptualized by the evaluator in sufficient time to measure their presence before the program and after the program, in order to establish that they have in fact arisen from the program. As extra insurance, either through direct observation or through "open" questions, information should be obtained after the fact about additional consequences the program may have had for the participants. While such procedures do not provide a base line for systematic measurement of change resulting from the program, the evaluator may qualify his general conclusions in the light of such evidence. No evaluator can be so imaginative as to anticipate all the consequences of a program that might occur at some remote time or place. The lack of such extreme powers of vision is perhaps fortunate. Unanticipated consequences that are exceedingly remote in time or space or are diffused or displaced from the subjects to some other group should not always be incorporated into the reckoning. Opler, for example, in reporting evaluations of UNESCO programs of technical assistance designed to have macroscopic effects on a whole society, remarks: Another question had to do with secondary and tertiary effects. How successful is a project, it was asked, if it solves one problem only to create further difficulties? Would one not value more highly a project that did not create adverse or challenging secondary and tertiary results, or one in which such resultant effects were foreseen and controlled? The ideal situation, it was thought, would be one in which the technically appropriate task has been accomplished without accompanying complicating problems or with only such difficulties as have been anticipated, are controllable, and can be handled together. However, a warning was expressed that a look too far into such future contingencies might paralyze activity, and that some technical jobs are of such immediate and pressing importance that they should be undertaken without too much anticipation of accompanying tensions or intensified problems. It was felt that all plans are sure to have mixed consequences, and that such consequences should be and can be dealt with as they arise.18

Some remote consequences, however, may be germane to an evaluation. One exotic variety of remote but relevant unanticipated consequence has been documented in a number of studies. The target group which had been exposed to a program was not affected as intended. 17

Op. cit., pp. 46-50. Morris E. Opler, Social Aspects of Technical Assistance in Operation, Tensions and Technology Series, UNESCO, no. 4 (Washington, 1954), p. 67. 18

16

Applications of Methods of Evaluation

Other groups were affected although the program was not intended for them. An example is provided by Carlson, who evaluated a mass-information campaign in the state of Mississippi designed to increase knowledge about venereal disease among certain publics and their volunteering for treatment. No such effects were demonstrated, but the surprising finding confronting the evaluator was that in counties exposed to the program the incidence of new cases of syphilis increased. Carlson remarks: "There are sound reasons for believing that much of the new syphilis found in such areas resulted from more vigorous efforts on the part of personnel in the local county health department to locate new pools of infection which they had hithertofore neglected.'™ Before the local health campaigns, public health workers were generally in a state of low morale, brought on by the apparently insurmountable task they faced in trying to eradicate venereal disease in communities where their work was little understood and seldom appreciated by the citizens. This state of low morale generally evaporated as a result of the mass campaigns in the areas. These campaigns, although aimed at all citizens, reacted on the health workers who participated, raising their morale and efficiency to such an extent as vastly to reduce the number of undetected venereal infections in the community. Therefore, viewed from the point of the ultimate objective of the communicator—the reduction of venereal disease—the campaigns were successful although they did not achieve the specific intended goals of increasing public information or voluntary clinic visits. A similar case is reported by Fanshel. A mass-media campaign designed to increase adoption of Negro children yielded the surprising result that only a relatively small number of those couples who responded to newspaper or radio-television publicity went on to adopt a child. Fanshel remarks: Cognizance should be taken of the fact that public informational campaigns help to create an atmosphere within the social agency itself which makes it more alert and responsive to the problem. These publicity campaigns not only inform the community of the crying need for adoptive homes for Negro babies, but they also continuously remind the worker of her need to be creative and resourceful in finding homes for these b a b i e s . . . . The second major contribution of promotional campaigns would appear to lie in the dissemination of information and alertness in secondary referral sources such as clergymen, social agencies, physicians, etc. These campaigns can arouse indigenous community leaders to the problem." 10 R. O. Carlson, "The Influence of the Community and the Primary Group on the Reactions of Southern Negroes to Syphilis" (unpublished Ph.D. dissertation, Columbia University, 1952), pp. 246-247. 20 David Fanshel, A Study in Negro Adoption (New York: Child Welfare League of America, 1957), p. 45 .

Principles of Evaluation

17

The final list of concepts for which measurements of the effect of the Encampment were systematically obtained in our main studies is presented in chart 1. The region or level studied is presented on the left; the specific variables elaborated are then listed along with information on the specific indicators or measuring instruments employed. The unanticipated consequences are shown in a final column adjacent to the level or the specific concept which they parallel. Omitted from the chart are special concepts and variables which were included in particular studies to be described in subsequent chapters. RESEARCH DESIGN AND THE PROOF OF THE EFFECTIVENESS OF A PROGRAM

A program's effectiveness is evaluated by the changes it produces in relevant variables. This is usually demonstrated by systematic measurements taken before and after the subjects are exposed to a program. The changes that are observed, however, cannot be accepted at face value as demonstrating the exclusive influence of a program. Other extraneous factors, concurrent with the program or set in motion by the evaluation itself, may account for some or all of the gross changes observed. Therefore the evaluation must be designed in such a way that the influence of factors other than the program can be eliminated. Two approaches to the solution of the problem should be employed. The first is direct, the second inferential. For certain extraneous factors whose source is identifiable and manipulable, research procedures may be developed to reduce or eliminate such factors before they can intrude. Then, by definition, they cannot influence the results. Discussion of this direct approach is postponed until the third phase. Many extraneous factors, however, cannot be excluded. The approach has to be indirect. Their influence upon the results may be estimated by experimental designs or logical arrangements, and the contribution they make to the gross results can be subtracted after the fact. The second phase deals with this approach in detail. The nature of the extraneous factors, the properties of various research designs for estimating their influence, and the solutions employed in our studies will be presented. CONTROLLING FOR EXTRANEOUS SOURCES OF CHANGE BY EXPERIMENTAL DESIGNS

The basic evaluation of the Encampment involved two sets of measurements, spaced six weeks apart. During that period the campers were exposed to the influence of the program. A t the same time they were also subject to other sources of change. The world of events impinged on them. Perhaps, the campers would have changed in the same degree

Applications of Methods of Evaluation

18

CHART

1

CONCEPTUALIZATION AND MEASUREMENT OF THE OBJECTIVES OF THE ENCAMPMENT AND OTHER UNANTICIPATED CONSEQUENCES (MAJOR STUDIES) Region of concepts

Basic values

Specific concepts and indices related to explicit objectives

Concepts related to unanticipated consequences

Worthy ambitions (free response; norms from Riecken's study) Ingredients of an ideal society (free response; norms from Riecken's study) Ends for which great personal sacrifice would be made (free response) Chief criterion for ranking occupations in society (free response; norms from NORC nationwide study) Evaluation of occupations (battery; national norms, NORC) Personal career goals (free response)

Action orientation

Activity promising greatest personal satisfaction in life (check list; norms from Gillespie and Allport study of youth) Preference for immediate remedial actions versus long-range study of social problems (single item) Preference for action on local, national, or international problems (single item) Action-apathy score (scale; norms on nation's high school seniors) Commitment to act in accordance with role requirements of Encampment (free response and battery)

Cognition of social problems

Optimism about ultimate solution of various problems (battery; norms from Riecken's study) Time perspective on solution of problems (battery; norms from Riecken's study)

Absolutistic views on social causes and cures (scale; six items)

Image of individual citizen's potency in handling social problems (index; five items) Image of group potency in handling social problems (index; five items) Salient social attitudes and opinions

from

Political-economic conservatism (scale; norms from Riecken)

Democratic practices (scale; norms from Riecken)

Political party preference (single item)

Authoritarianism Riecken)

(scale;

norms

Principles of Evaluation

19

CHART 1—Continued Region of concepts

Salient social attitudes and opinions— Continued

Specific concepts and indices related to explicit objectives

Concepts related to unanticipated consequences

Tolerance (scale; norms from national survey by Stouffer)

Ethnocentric image of democracy (scale; nine items)

Civil rights and discrimination (scale; norms on nation's high school seniors) Civil liberties (scale; norms on nation's high school seniors) Constitutional practices (scale; some norms on nation's high school seniors) Group stereotypy (scale) Attitudes on domestic Communism and civil liberties (two items; national norms, NORC) Paramount objectives of democracy (single item)

Perceived relationship with rest of society

Perceived social anomie (scale)

Image of views of average American (two items) Felt alienation between own views and average American's views (scale; five items)

Skills or capacities

Susceptibility to prestige symbols as propagandistic devices (six items; splitballot quasi experiment)

Conduct (covered in follow-up, alumni, and collegiate studies)

Attempts to apply Encampment principles in home community (free response) Attempts to combat undemocratic action in home community (free response) Informal political activity in home community (normsfrom various votingstudies) Nondiscrimination in friendship choices within Encampment (sociometric data) Participation in organizations and community activities (battery) Voting in local, state, and national elections (battery; national norms) Writing to government officials (single item; national norms) Use of leisure time (check list) Present occupation

20

Applications of Methods of Evaluation

if they had lived through a six-week period outside the Encampment. Moreover, they are not adults and presumably are in process of growth. Perhaps, the changes observed are simply the unfolding of normal growth processes.21 Another contribution to change between measurements may derive from the instability that measuring instruments show upon repeated use. There are inherent sources of instability in repeated sets of measurements even when no growth or flux of events or systematic stimulation occurs. A fourth and final set of factors, set in motion by the evaluation itself, requires specialized designs and will be discussed in a separate section. The classic solution which provides an estimate of all three uncontrolled factors in evaluation studies, as in conventional experiments, is the use of a control group of equivalent individuals who are given both the pretest and the post-test, but are not exposed to the particular experience under study. The change in the control group over the same time interval is expressive of the influence of these other factors, and by subtraction the residual influence of the program may be determined. The net change is free of all the three annoying factors that were operative: events, growth, instability of measurement, and, as will be noted below, a fourth operative factor, practice effect from sheer retesting. This classic design has been a model available to all experimenters and evaluators for many decades. It has been prominently displayed and eloquently described, and it provides an automatic, packaged solution to the vexing problem of isolating all annoying extraneous sources of change. A paradox worth pondering is that, although the design has been attractively displayed for sale, very few evaluators have actually "bought" it, and among those who have the product has usually been found deficient. The rejection of this design should not be interpreted as negligence. Often it means that the design cannot be translated into proper practice in evaluation studies because of conditions inherent in the operation of the programs. These conditions usually prevent obtaining an equivalent control group. The control groups that tend to be employed are poor simulations and the results consequently mislead rather than illuminate. A t other times the practical priorities of the action program override the goals of evaluation and bar the evaluator's access to a good control group. In all such instances, slavish insistence on control-group designs is unwise. The purist attitude may defeat any attempt to evaluate planned programs, or it may create false confidence 21 For an unusual example of the contribution of growth, to changes that seemed to be a function of treatment see a study by Willard Olson described in D. G. French, An Approach to Measuring Results in Social Work (New York: Columbia University Press, 1952), pp. 66-67.

Principles of Evaluation

21

in the results of poorly designed evaluations which only simulate an equivalent control group, while undue skepticism is attached to the results of well-designed evaluations which employ reasonable and sound alternatives to the classic design. Sometimes, too, it leads to a misdirection of energies; instead of devoting exclusive attention to obtaining a control group that is likely to be inadequate, the evaluator could develop a variety of inferential and indirect devices for enhancing his ability to impute causal significance to the program. Some extended discussion is called for because the problem is central to research in evaluation. When a large group of individuals is eligible for a program, some could be regarded as experimental subjects and exposed to the program, and others, made equivalent by matching and/or randomization, could be employed as a control group and denied treatment. Even if no practical obstacles to using such procedures exist and the original numbers are large enough, in many fields such as medicine, social work, therapy, or technical assistance the barrier is the ethical imperative not to deny service to those who are in need. Blenkner states the position clearly: "No casework agency is so dedicated to science as to permit it to make a random sort of its applicants, offering help to one half, while merely following up the other half to see what happens to them."22 A solution too rarely employed in such instances is to compare a variety of experimental groups, each one subject to some different form of treatment. Because all are equally exposed to extraneous sources of change, the differential findings are indicative of the relative effectiveness of various types of programs. Such a design, however, may still create some ethical conflicts insofar as certain types of treatment are clearly less advantageous than others. An unusual and elaborate example is provided by an evaluation of the effectiveness of various health and sanitation programs introduced into a number of Egyptian villages. Five very similar villages, all associated with a recently developed health center, received certain basic medical care from the center. One village received, in addition to such care, the full program of installation of wells and latrines, refuse disposal, fly control, preventive medical activities, venereal-disease control, and tuberculosis control. Three other villages received comparative treatments which involved some of these components of the comprehensive program. The fifth village received none of the components but, of course, had the basic care provided by 22 M. Blenkner, "Obstacles to Evaluative Research in Casework," Soo. Casework, X X X I (1950), 98. For a systematic discussion of this and other obstacles to the use of control groups in evaluations of programs of psycho-social treatment see E. Herzog, Some Guidelines for Evaluative Research, U. S. Department of Health, Education, and Welfare, Social Security Administration, Children's Bureau, Publication no. 375 (Washington, 1959), pp. 64-71.

22

Applications of Methods of Evaluation

the health center. Thus, in all five villages which had dire need for improved health conditons, no one was deprived of some care. By appropriate comparisons, the contribution of different programmatic components in the reduction of morbidity and mortality could be assessed.23 A less obvious difficulty confronts the evaluator blessed with a large, eligible target group from which control and experimental subjects may be chosen systematically. In conventional experimentation, the act of measurement is generally simple, quick, and inexpensive because it is limited to a small number of dependent variables which are easily measured in a cooperative, accessible, and clustered group of subjects. Moreover, in experiments, the treatment can hardly be regarded as a reward—if anything, it may be an annoyance suffered by cooperative subjects imbued with some love of science. By contrast, measurement of a control group in evaluation studies may involve a whole array of dependent variables measured in a group of subjects who may live in widely scattered natural conditions. Moreover, the subjects may not be inclined to be especially cooperative, as they must often suffer the measurements and receive nothing in return, certainly not the attractive treatment or program which they need or want. Here the obstacles are not the finding of adequate numbers of controls, nor in denying them service. The obstacles are the cost and the difficulty of measurement itself, and the eliciting of cooperation from those who are aware that they are to be disadvantaged. A study reported by Lippitt and conducted by members of the Research Center for Group Dynamics illustrates the problem. A program of training in community relations was established for intergroup-relations workers in Connecticut. Through lengthy personal interviews, measures were obtained before training and six months after return from training to the home communities. The Research Center is famous for sophisticated methodological work in the design of experiments. Yet no control group of matched workers from the communities was employed; for, as Lippitt informs us, "This procedure had to be ruled out as not feasible . . . because of staff limitations and because of the public relations problems of selecting and getting the cooperation of such a large group to whom we could give no significant service in return." 24 One solution for such problems is to 28 J. M. Weir, I. M. Wassif, F. E. Hassan, S. Attia, and M. A. Kader, "An Evaluation of Health and Sanitation in Egyptian Villages," J. Egypt. Publ. Health Assoc., XXVII, no. 3 (1952), 55-114. 24 E. Lippitt, Training im Community Relations (New York: Harper, 1949), p. 173. Lippitt describes (pp. 173-174) the variety of internal analyses made on the experimental group data to establish confidence that the gross findings observed were caused by the workshop.

Principles of Evaluation

23

conduct the premeasurements on both experimental and control subjects before any individuals have been selected for the program. Then the control-group subjects are no more likely to begrudge cooperation, for they do not yet know they are disadvantaged. Moreover, any bias in response because of motivational factors is equal in all subjects. The advantage of secrecy about who is excluded from a program is, of course, dispelled by the time of the postmeasurements, and any consideration of economy which is required is completely unaffected by the device. Another solution is to disguise the auspices and the purpose of the evaluation, so that it appears as a survey or measurement process unrelated to the program, or so that evaluators, at least, are disassociated from the program agency that denied certain subjects treatment. Thus the special barrier to cooperation and any motivated bias in response are reduced. To the many other gains from the evaluators' being independent of the program agency may be added this further advantage. One other practical problem besets the evaluator seeking an appropriate control group. Many programs do not have a large target group available for treatment. They may have a great scarcity of subjects, owing to the unusual nature of the program or its limited appeal. To discard potential subjects simply to secure a control group is impossible, given their state of need. The issue is not one of an ethical barrier to denying treatment to those who are in need. I t is simply that the program or agency needs most of or all the subjects it can get, and every addition to a control group means one less adherent to fill its ranks. The Encampment, like many similar ventures, illustrates the problem. It seeks a special type of young person and appeals to only a few. This double filtering leaves little surplus from which to constitute a control group. To employ some of the scarce eligible subjects as controls would be to subordinate the program's prime goal of character training to the nicety of experimental design. The problems described thus far are merely obstacles to obtaining any control group at all. Granted that these do not apply, and that the evaluator goes about the task of obtaining a control group; more subtle, but serious, problems arise. The control group should be equivalent to the experimental group in proneness to change, for we use it to estimate how much the experimental group would have changed in response to factors other than the program. If the evaluator could allocate subjects from some common pool to the experimental and control groups through matching and/or randomization, unrestricted by any other requirements, everything would be fine. But this very rarely happens. The

24

Applications of Methods of Evaluation

evaluator rarely has control over the flow of subjects into a program. Selection is governed either by the subject himself when he is favorably disposed to a program, or by the agency which recruits in terms of the wish to have particular subjects and programs joined. In the latter instance, the subjects are regarded as especially appropriate for or predisposed to the program, and the program may consequently have a peculiar clientele. The controls one can find are not likely to be equivalent. If they were, they would have elected to join, or the agency would have recruited them. Then, for the reasons already described, it would be difficult to deny them treatment and to return them to the control group. Bertram Black, the director of Altro, which conducted a rigorous evaluation reviewed below, presents the problem in all its painful aspects: "A control group study is administratively hard to live with. It frequently seemed . . . that the patient who might make best use of Altro was always randomly selected for the control group."® Again and again in the literature one finds sophisticated investigators, who are quite aware of the perfection of a good control-group design, reporting evaluations that omit the use of a control group. The ideal is hard to achieve! For example, in a review of the appropriate designs for evaluating cross-cultural education, Brewster Smith, who played a central advisory role in a series of such studies, stresses the desirability of a comparable control group but notes that the "practical difficulties of gaining access to genuinely comparable groups have made the use of this ideal design a rarity."20 He then mentions a strategy which we shall also stress, that "wisdom and judgment may still be required in distinguishing programme effects from extra-programme influences." He formulates a general policy and approach to the design and interpretation of evaluations: Lest these ideal requirements should discourage needed efforts at evaluation, however, more than a word of qualification is needed.... It is by no means always essential to prove one's case. To the extent that objective proof is important—as it is when an agency's total programme is under outside review—it is less appropriate to relax the evidential requirements. When evaluation is primarily for the benefit of the programme's own administrators, skilled judgment may be substituted for proof at various points in the ideal pattern of evaluation, with great saving in cost and feasibility. The ideal requirements remain a useful reminder of the points at which 26

Meyer and Borgatta, op. cit., p. 4. M. B. Smith, op. cit., p. 391. Elsewhere he notes that control-group design is impossible in studies where the effects examined are societal consequences of having particular nationals trained by cross-cultural education. To eliminate extraneous sources of societal change would mean either to examine the particular society at another historical point when exchanges were not present, but when everything else was the same (patently out of the question), or to compare another society equivalent in every respect except that it lacked exchanges (also out of the question). 26

Principles of Evaluation

25

judgment is being substituted for evidence; they indicate where cautious interpretation is likely to be in order.27

To underscore the fact that the brute conditions of evaluations overwhelm the most alert and sophisticated investigators, one more example may be cited. Riecken, in his comprehensive evaluation of volunteer work camps, which we shall often use as a model, argues most persuasively for the use of control groups. He comments that the introduction of a control group changes the "entire view of the nature of program evaluation" and leads to a fully scientific study.28 Nevertheless, the practical difficulties in obtaining a control group equivalent to the work campers made it impossible for Riecken to achieve the advantages of the ideal design. Without benefit of direct data from a control group, he was forced to estimate by other means how much of the change he detected in his subjects was caused by the work-camp experience.29 At least, an inadequate control group was not allowed to masquerade as the real thing. How deceptive is the apparent simplicity of designing an evaluation that would have an equivalent control group is conveyed by Saenger's study of the effectiveness of the UNESCO pamphlet series on race.30 The original aim was to evaluate the effectiveness of the pamphlets as teaching aids in American high schools and colleges. In 1952, in the initial phase, Saenger scouted a series of communities in New Jersey and New York, discussing the projected study with school superintendents. At the time UNESCO was regarded as controversial, and many school administrators refused to cooperate. In New York City the Board of Education was willing to cooperate, but it imposed the restrictions that no attitude scales making references to ethnic groups could be employed and that the study was to be conducted only in selected schools of "exceptionally high scholastic standards and known for good work in intergroup relations." Because of such restrictions, Saenger made the wise decision to restrict his inquiry to the college level, where he was able to obtain the cooperation appropriate to his needs. A less thoughtful investigator might have used the cooperative high schools and measured a control group of subjects equivalent in various individual characteristics, selected from other schools. The fact that the experi21

Hid. Op. cit., p. 10. He did have evidence, however, that the changes observed did not represent unreliability of measurement, for he had empirically computed the reliabilities of his scales and found them satisfactory. Consequently, this extraneous source of change was controlled. 80 G. Saenger, "The Effectiveness of the UNESCO Pamphlet Series on Race," Intermit. Soo. Soi. Bull., V I (1954), 448-503. 28 20

26

Applications of Methods of Evaluation

mental and control subjects were influenced by a school ideology or atmosphere very different on matters of race would have been an insidious source of incomparability. Choosing experimental and control subjects from within the same schools would have solved this problem, but choosing within small organizations creates the new problem that the control group may become contaminated by communication from the experimental group and thus not yield a fair estimate of changes owing to zero or nontreatment. 81 Insulation from contamination is a condition impossible to meet in programs involving organized groups of people in close interaction. How intractable the problem of obtaining a comparable control group of adequate size can be is illustrated by the Meyer and Borgatta evaluation.32 The details are presented in chapter viii because of their relevance to one phase of our own studies, but their experience is reviewed here because it provides an exquisite account of the general problem. The Altro workshops were planning a program of rehabilitation of posthospitalized mental patients. As their facilities would permit only about forty such patients to be taken per year, and the potentially eligible group was much larger, the requirement of the evaluation that some subjects be treated as controls and denied treatment despite need did not appear arbitrary or improper to the agency. Whether or not an evaluation was conducted, the limited facilities would have required such selection. Because the subjects retained the right not to participate, and the agency retained the right to choose those subjects whom it desired, a double process of screening would be operative upon the initial potential group. Meyer and Borgatta, realizing that the control group would have to be comparable to those finally screened through this double filter, planned to assign experimental and control subjects by a random method, after this normal screening had occurred. The program agency consented to this requirement of random assignment. Thus it might seem that the kinds of difficulties previously mentioned could be solved; however, losses from the original number of cases should first be noted. Six months after initiation of the evaluation, out of 116 potentially eligible subjects, only 32 had survived the initial screening and become actually eligible for the program. This permitted the random assignment of 16 cases to the experimental group and 16 31 The example also suggests that any test of the effectiveness of pamphlets in such experimental schools would not be generalizable, since other schools into which the pamphlets might be insinuated in the future might have a less supportive attitude on matters of race. 82 Op. tit., passim.

Principles of Evaluation

27

to the control group. The randomization, however, became somewhat academic and the comparability of the two groups dubious as a result of another selective process. Of the 16 randomly assigned experimental subjects, only 6 actually entered the program; those not entering mainly defected of their own accord. What would be demanded by the canons of experimental design would be either the random assignment of the final group of 6 to a group of 3 experimental subjects and a group of 3 control subjects—a "small group" experiment with a vengeance—or some method of eliminating from the 16 controls those individuals like the defectors in the experimental group. Clearly, selective losses after the randomization procedure and the general shrinkage in the numbers on whom research findings could be based jeopardized the experimental design, leaving aside the important practical consideration that the shrinkage resulted in the agency's operating well below the capacity it had specified. In the next ten-month period the problem continued: altogether, 39 cases were screened, of whom 19 were assigned to the experimental group, but of whom only 4 actually entered the program. The Meyer and Borgatta study points to one other distinctive feature of an evaluation—in contrast with the usual experiment—which limits the researcher's ability to carry out the conventional design. We have already alluded to such features as the selectivity of the experimental subjects, the scattered locations of the controls, the denial of treatment being a disadvantage, and so forth, but we have not explicitly mentioned the distinctive fact that in most evaluations the treatment is protracted in time, whereas the duration of an experiment is usually short. Granted the initial equivalence of the eligible subjects assigned to treatment and to a control group, the passage of time gives opportunity for the experimental group to terminate the treatment or to defect. The earlier elegant procedures of assignment may be wasted if the subgroup that remains on the scene is biased. The protracted period of an evaluation also is of great significance in the proper interpretation of the meaning of the control group, and we shall return to this feature again. Only in an institutional setting with authority so unlimited that both entrance and exit are beyond the control of the subjects can all these difficulties be prevented. Note the relative ease of implementing a rigorous design in a setting where institutional power over the subjects is complete, self-selection is prevented, subjects are so plentiful that many are expendable, and the duration of the program is so short as to bar any emergent problems. Cooper and Dinerman evaluated the effec-

28

Applications of Methods of Evaluation

tiveness of a motion picture in affecting attitudes of prejudice among high, school students.33 One thousand students in their second year in high school were divided into an experimental and a control group. In regular assembly the experimental group was shown the motion picture whose effects were to be evaluated; the control group was shown a different, innocuous one in their assembly. On the basis of measures from questionnaires administered after exposure, the control and experimental groups were then further refined by excluding certain subjects who were not well matched. Despite such "shrinkage," the evaluators still had an experimental group of 368 subjects and a control group of 491. Of course, such institutional settings may be a mixed blessing to the evaluator of a lengthy action program. When experimental and control subjects are contained within the same organization, albeit large in size, it is perfectly possible that they may contaminate each other, and that the control group is vicariously influenced by the experimental subjects. By contrast, the duration of this experiment was so compressed that contamination was at a minimum. I t should be noted that even in situations where institutional authority is strong other facts may create conflict for the experimenter who plans to follow a rigorous design. An amusing example is provided by Davies, Gross, and Short, who conducted several evaluations of the effectiveness of different methods of college instruction. On the basis of available test scores, 62 students of a college class were to be assigned to three comparable groups receiving different combinations of instruction. Two students were very much retarded in relation to the rest of the class; as there were three groups, this created an awkard situation. A solution which the experimenters did not suggest, for obvious reasons, was to deny instruction to these two. Rather, they "decided to omit them from the statistical analysis, but to carry them along for teaching purposes. [They] . . . were two of the three students who completed the course with a grade of F."34 Another example is provided by the evaluation studies conducted during World War I I on the effectiveness of orientation films. Despite the absolute power of the army, Hovland notes that individual men could not be assigned at random to experimental and control groups, because the natural unit of the company could not be broken apart to create the design, and, if it were, the conditions would have appeared so exotic to the soldiers as to jeopardize the whole experiment. Instead, company units were assigned to the particular 33 E. Cooper and H. Dinerman, "Analysis of the Film 'Don't Be A Sucker': A Study in Communication," Publ. Opin. Quart., XV (1951), 243-264. S1 V. Davies, E. Gross, and J. Short, Experiments in Teaching Effectiveness, State College of Washington, Department of Sociology (Pullman, 1958).

Principles of Evaluation

29

treatments and equivalence of the units had to be cheeked empirically."5 These many difficulties demonstrate that an evaluator may have to rely on internal types of evidence and judgment in imputing causal significance to a program in bringing about the observed changes. When a control group is employed, it should be scrutinized and its comparability checked empirically; the original procedures, no matter how good in principle in producing comparability, may have been undermined by various insidious problems.38 Various devices that control for particular uncontrolled factors piecemeal are an excellent strategy, because taken altogether they may compensate for the lack of the automatic omnibus solution that the conventional control group would provide, and each one alone gives some protection against a false conclusion. Finally, innovations in experimental design may be possible.87 CONTROLLING FOR EFFECT OF REPEATED TESTING

The purpose of various experimental designs is to eliminate the influence of extraneous factors upon the gross changes observed. As noted earlier, certain "natural" factors are operating simultaneously with the program and their contribution must be estimated and subtracted from the gross changes demonstrated by the repeated measurements. Events and growth are natural accompaniments to life, and we shall treat the instability of response that subjects exhibit when they are measured as a feature of human nature, even though it is also a property of the measuring instrument. There is, however, one other class of extraneous factors, set in operation by the evaluation design itself, whose contribution to the changes must be eliminated. In order to measure the change produced by the program, subjects are measured systematically before 35

Op. tit., passim. 36 A number of evaluation studies employ very sophisticated techniques for checking upon the comparability of the experimental and the control groups or develop adjustments to correct for noncomparability through the use of elaborate statistical techniques. See, for example, Meyer and Borgatta, op. ait., pp. 75-76; W. A. Belson, "Learning and Attitude Changes Resulting From Viewing A Television Series, 'Bon Voyage,'" Brit. J. Educ. Psychol., X X V I (1956), 31-38; H. E. Freeman and H. Ashley Weeks, "Analysis of a Program of Treatment of Delinquent Boys," Amer. J. Sooiol., L X I I (1956-57), 56-61. 37 Festinger and Kelley provide an example of an innovation that controls one class of extraneous factors, external events. A program of community activity was instituted in a housing project to remedy certain negative attitudes and was carried out over a period of nine months. Surveys were conducted not only before and after the program, but every two and a half months. Although the major purpose of these interim surveys was to examine the process of change, they served as a control to estimate the influence of external events, for, as noted by the authors, if an event occurred while the program was in progress, they would be able to examine its immediate influence by noting whether or not changes corresponded in time to particular events. L. Festinger and H. Kelley, Changing Attitudes through Social Contact (Ann Arbor: Research Center for Group Dynamics, 1951).

30

Applications of Methods of Evaluation

and after their exposure. The fact that campers had been tested before could by itself possibly account for a change in the second set of measurements, quite apart from the true influence of the Encampment experience. Thus, quizzing individuals on intellectual matters or giving an information test may itself be instructive and thus make them perform better or become more knowledgeable on a second testing. In the opinion or attitude realm, questioning may be stimulating and cause individuals to change their views. This possibility is conveyed dramatically by some amusing observations reported by Cochran: I have heard of f a r m management studies of poor farmers where the list of questions opened the eyes of some of the farmers to financial opportunities that had never occurred to them. I n a few years these f a n n e r s were offering the interviewers jobs. I n the British study of smoking and cancer, the Medical Research Council's scientists became alarmed a t the number of doctors who replied to the original questionnaire by saying, " I have been smoking twenty cigarettes a day, but a f t e r reading this questionnaire I have given up smoking f o r ever." 88

An alternative possibility is that the use of a pretest rigidifies sentiments, if only perhaps for the reason that individuals want to appear consistent in their views and may remember their earlier answers. All these possibilities may be grouped under the heading of practice effects, the changes produced by subjects' having opportunity to practice on a first test. Where the first test precedes some particular program or experience, another artifactual source of change is introduced. As Solomon has noted, the first test not only provides in itself some stimulation or practice, but interacts with the subsequent experience, perhaps modifying the subject's attention to or perception of the subsequent experience.89 Thus, in evaluating a program the combined effect of sheer practice plus the sensitization to the subsequent experience owing to previous testing should be subtracted from the gross findings on change.40 38 W. G. Cochran, "Research Techniques in the Study of Human Beings," Milbanh Memorial Fwnd Quart., X X X I I I (1955), 132. The quotation was brought to our attention by David Fanshel. 38 R. L. Solomon, "An Extension of Control Group Design," Psychol. Bull., X L V I (1949), 137-150. 40 Such effects of retesting may be eliminated altogether by designs that do not employ any premeasurement. Thus, two equivalent subsamples may be drawn f r o m an experimental group; one is tested before exposure, the other a f t e r exposure. I n this situation, retest effects are excluded by definition. A large pool of subjects is required, however, and in an intimate setting the pretesting effect may be communicated and may affect the second group vicariously. Another solution is to have an experimental or exposed group and a control or nonexposed group which, through some trustworthy procedure, are made equivalent, and each of which is tested only once and at a time point a f t e r the program. Here the effects of pretesting are eliminated a t the source, by definition. For an excellent discussion of the advantages and disadvantages of this procedure over the pre- and post-test designs, see Donald T. Campbell, "Factors Relevant to the Validity of Experiments in Social Settings," Psychol. Bull., L I V (1957), 297-312.

Principles of Evaluation

31

There are a number of conventional solutions that have been developed to isolate change owing to retest from the change resulting from the intrinsic effect of a program. The classic experimental design in which an equivalent control group, which does not receive a treatment, is subject to a first test and then a retest separated by the same interval of time as had elapsed for the experimental group provides an estimate of the influence of all other factors, including sheer practice. Whatever retest effect may have occurred applies equally to the control group since it too is tested twice. As we have emphasized, the logical solutions developed in experimentation are rarely practical solutions for evaluation, and this approach became feasible only in one late stage of our studies. Although the use of an equivalent control group measures sheer practice effect, it does not provide an estimate of the effect of pretesting in sensitizing subjects to the subsequent program, because the control group has been pretested but has not been exposed to any particular form of stimulation. Another conventional solution developed to isolate change resulting from retesting from that attributable to the intrinsic effects of a program is to divide the target or experimental group into two equivalent subgroups. In contrast to the usual control-group design, both the groups receive the program or treatment. One group, however, does not receive the pretest. Then, by comparison of the final scores of the two groups, one can estimate the specific influence on the experimental group of testing in producing some distorted magnitude of change, resulting both from sheer practice and from sensitization. The conditions surrounding our evaluation made this latter approach seem unwise. Given so small a total group to start with, we would sacrifice direct evidence of the influence of the Encampment on a sizable number and be left with a very poor basis for generalizing about its effects. In addition, the use of a control group to estimate the practice and sensitization effects has been developed in situations f a r different from the closed and intimate society of the Encampment. Under ordinary conditions no individual needs to realize that he has been singled out for testing, or nontesting. He need never know what has happened to most others. For example, the design was employed in the 1940 voting study of Erie County, Ohio." Of 3,000 persons originally polled, 1,800 were reinterviewed only once (600 on each of three different occasions) and another 600 were repeatedly reinterviewed over a six-month period. But the county contained at the time some 40,000 persons spread 11 P. F. Lazarsfeld, B. Berelson, and H. Gaudet, The People's Choice (2d ed.; New York: Columbia University Press, 1948), pp. 3-5.

32

Applications of Methods of Evaluation

over an area of about 264 square miles. In the Encampment, with its 100-odd persons living in continual and close contact, such selective testing might well have created a hubbub far more damaging to the inquiry than any effects from sensitization of the whole group. Moreover, in this setting, the procedure, had it been employed, would probably have overestimated sensitization effects, as the very process of singling out some campers from the rest would probably have made them especially alert to those Encampment influences measured in the questionnaire. The measurement procedures in this sort of setting must be as unobtrusive as possible. Because a control-group design for estimating practice and sensitization effects would have made them highly visible, this type of design was rejected. What then is the solution in an evaluation study? As we shall argue shortly, the general problems of experimental design must often be dealt with by various approximations, and judgment must often be exercised in weighing conclusions. Reference to the literature would strongly suggest that both practice and sensitization effects are not of serious magnitude. A number of evaluation studies have employed a control group to estimate retest effects of practice and sensitization. Riecken's findings have the most relevance, in that we employed some of his attitude scales, his work campers were a very similar population, and the time interval was about the same as ours. Hisfindingsclearly establish that the pretesting spuriously reduced the amount of change observed, thus providing a conservative measure of the effectiveness of the program." In some manner, the pretesting sensitized his campers negatively, setting up some type of resistance to the subsequent stimulation. Perhaps there is a strain toward seeming consistent which operates with the aid of memory cues. The phenomenon may appear bizarre, but the very same type of negative effect was reported by Solomon in his studies of the influence of pretesting in sensitizing experimental subjects to training in spelling.48 The evidence obtained by Hovland and his associates on retest effects from their evaluations of orientation films used in World War II also suggests that pretesting spuriously reduces the amount of change observed.44 Canter, who applied Solomon's extended controlgroup design to an evaluation of a human relations training program for industrial supervisors, reports findings on the sensitization effects of pretesting for a series of variables. The effects vary in direction, the pretesting enhancing change on some variables and spuriously reducing 42 43 44

Op. tit., pp. 1 0 1 - 1 0 2 . Op. tit., pp. 1 4 4 - 1 4 5 . Op. tit., pp. 3 1 1 - 3 1 3 .

Principles of Evaluation

33

it on others, with perhaps a slight tendency for the reduction effects to be of greater magnitude.46 In an evaluation of the effect of indoctrination about nuclear weapons which accompanied actual A-bomb military maneuvers, Schwartz and Winograd report that retest effects were inconsequential." Wilson and Bonilla analyzed retest effects in the course of an evaluation of the effects of an exchange-of-persons program. The difference in response between those interviewed twice and those interviewed only once averaged less than 1 percentage point." Lana describes an elaborate methodological study of the practice and sensitization effects of pretesting and their relation to the time interval between pretest and treatment and to the time interval and number of post-tests after treatment.48 As the variables were attitude scores and the treatment was a persuasive communication, the study may be regarded as highly relevant to evaluation research. No significant pretest effects were demonstrable whether post-testing was immediate or delayed and whether or not there was cumulative post-testing. Lana also refers to another study, by Piers, in which no sensitization effects on attitude change occurred. So much for the evidence from evaluation studies, where, fortunately, the conclusion seems well warranted that retest effects are not a major source of concern and, if anything, handicap positive findings. Another body of evidence is also relevant. In panel studies, which seek to trace a process of change by reinterviewing while individuals are undergoing some diffuse experience such as a political or mass-media campaign, the use of the control group which does not receive the pretest provides an estimate of the combined effect of sheer practice and sensitization. Glock presents the detailed analysis of such effects for two major panel studies, the Erie County study reported in The People's Choice and a large panel study in Cincinnati. In Erie County interviews were spaced at intervals of one month, but there were seven such interviews, thus providing very powerful, cumulative retest effects. In Cincinnati there were two interviews spaced at an interval of six months. In neither study was there any evidence of any effect on measures of interest in the campaigns, or on reports of increased exposure to informational sources. 45 R. R. Canter, "The Use of Extended Control Group Designs in Human Relations," Psychol. Bull., XLVIII (1951), 340-347. 48 S. Schwartz and B. Winograd, "Preparation of Soldiers for Atomic Maneuvers," J. Soc. Issues, X, no. 3 (1954), 42-52. 47 The findings are briefly mentioned in E. C. Wilson and P. Bonilla, "Evaluating Exchange of Persons Programs," Publ. Opin. Quart., X I X (1955), 20-30. We are grateful to the authors for making the original detailed report of the findings available to us. 48 R. E. Lana, "Pretest-Treatment Interaction Effects in Attitudinal Studies," Psychol. Bull., LVI (1959), 293-300.

34

Applications of Methods of Evaluation

No effects were demonstrable on the crystallization of opinions in Cincinnati, whereas there was such an effect in Erie County, with formerly vague opinions becoming crystallized and formerly undecided voting preferences becoming specific choices. In the Erie County study, the seven waves of interviewing created the possibility of massive, cumulative retest effects and might account for the occasional effects demonstrated. That such retest effects are unlikely to occur despite many waves of repeated interviewing may also be demonstrated by an analysis of a panel study conducted by the Survey Research Center of the University of Michigan. For four successive waves of interviewing on economic matters spread over a year and a half, comparisons with random samples interviewed only once reveal little or no effect attributable to retest." Both in evaluation of some specific program and in social research on a process of change, the variable under study is some stimulus and retesting is an annoying uncontrolled factor arising from the measurement process whose influence we wish to estimate and subtract from the gross findings. By contrast, the same research design had been employed much earlier in psychological experiments where the main variable under study was practice on some test and its effect on subsequent test performance.10 Here the interval between first test and retest was often very short, and was not filled with any particular intervening program or campaign or event which bore any special relation to the items used in tests. Thus, it is purely inquiry into the sheer practice effects of retesting. As this is a component of the combined effects of testing in evaluations, the literature is relevant. Practice effects are demonstrable, but they vary with the type of test material and with the interval between testing. For example, Terman 49 C. Y. Gloek, "Participation Bias and Be-Interview Effect in Panel Studies" (unpublished Ph.D. dissertation, Columbia University, 1952). The Cincinnati study has been described in a number of publications. See National Opinion Research Center, University of Chicago, Beports, nos. 37, 37a. The Michigan findings are reported in M. G. Sobol, "Panel Mortality and Panel Bias," J. Amer. Stat. Assoc., LIV (1959), 52-68. In cultural settings where the interview may be an exotic innovation, panel effects may well be heightened. Neurath reports a rather amusing difficulty which forced him to exclude one Indian village from an elaborate evaluation of farm radio forums in rural India. "In Kondve, S a t a r a , . . . one of the local people made friends with the interviewer. When he learned from the interviewer that in post-broadcast the same questions would be asked as before, he wanted to create a good impression for his village and carried on a little educational campaign, talking over the topics with the group. The interviewer interviewed one at a time in one corner—quite oblivious of the fact that those already interviewed might communicate with those still waiting." J. C. Mathur and Paul Neurath, An Indian Experiment in Farm Radio Forums (Paris, France: UNESCO, 1959), p. 92. 00 For a brief summary of such research and the findings see A. Anastasi, Psychological Testing (New York: Macmillan, 1954), pp. 52-56.

Principles of Evaluation

35

and Merrill report a large-scale study of such effects, based on their standardization of the revision of the Stanford-Binet. 51 The subjects were retested after intervals of only one to a few days, using parallel forms of the test. For the age group of seventeen to eighteen years (most comparable to our subjects), they report a practice effect of only about three I.Q. points. The practice effect of retesting with the same scale had been determined for the earlier version of the test, and a small effect was found to persist. The practice effects of reinterviewing on matters of opinion have been examined by Crespi."2 An interval of three weeks separated the two interviews, which dealt with a wide range of opinions, none of which was timely during the period. In the second interview subjects were instructed not to try to recall their previous opinions, but to express their current views, thus reducing the likelihood that any effect would be in the direction of enhanced stability of original response. No comparable control group of subjects (persons interviewed only on the second wave) was employed, but the nontimely topics made it very unlikely that the changes were the result of events. The consistent finding was a crystallization of opinion among those formerly undecided, reminiscent of the Erie County finding. The net increase in crystallized opinion, over the increase in indecision among those formerly decided, averaged 7 percentage points for the 27 questions examined. The effect of reinterviewing, while consistent, is small in magnitude and must be seen in the context of the brief time interval elapsed and the explicit instructions which operated to reduce any compensating tendency of pretesting to create stability of response. It seems reasonable to conclude, therefore, on the basis of the literature, either that the effect of pretesting is negligible or that it gives a conservative picture of the amount of change produced by the Encampment. Certainly pretesting seems unlikely to enhance artificially the apparent effectiveness of the Encampment. Several lines of evidence, directly taken from the Encampment study, also support this judgment. Direct observation of the campers during the pretesting suggests that it was a rather casual experience for them. After all, as students they had been tested all their lives! Observation during subsequent days also seemed to indicate that they forgot the whole experience rather quickly. Another reason for advancing the view that retest effects were 51 L. T. Terman and M. A. Merrill, Measuring Intelligence (Boston: Houghton Mifflin, 1937), pp. 43-44. 52 L. Crespi, "The Interview Effect in Polling," Publ. Opin. Quart., X I I (1948), 99-111.

36

Applications of Methods of Evaluation

negligible has to do with an unusual feature of the research design. One way to remove this source of change from the comparison of the pre- and post-tests would be to make the pretest not the very first test. If before the pretest proper a preliminary test is given, subjects will already have been practiced by the time they reach the pretest. Then both pre- and post-tests would be responsive to the basic practice in an initial testing and presumably the differences between pre- and postdata would be subject to this artifact only in the degree that practice is cumulative as a function of repeated trials. For reasons to be described shortly, about one-third of the campers had received a shortened version of the basic questionnaire before arrival at the Encampment.53 Whatever gain had occurred from the initial practice was already present in the pretest set of measurements for one-third of the campers. For the other two-thirds, the initial practice effects do not show themselves until the post-test, but the sheer influence of initial practice can be estimated by this procedure and thus be subtracted from the gross findings. This design, while primarily intended for other purposes yet to be described, also provides a direct estimate of the cumulative effect of practice from repeated trials, for we can compare the changes from preto post-tests for the one-third with the changes for the two-thirds that did not have the initial testing. The same comparison provides an estimate of the cumulative effect of testing in sensitizing subjects to the subsequent experience of the Encampment, because the one group had now been alerted by two previous tests, whereas the other group had only been alerted by one previous test. If cumulative practice and sensitization were important, the subgroup of 34 campers should show systematically different amounts of change during the summer in contrast with the other 66 campers. The comparisons are presented in table 1-1 for eleven of the dependent variables used in the evaluation. There is no consistent effect. In five instances, the cumulative effect is to reduce the amount of measured change; infiveother instances the effect is just the opposite—to increase measured change; in one instance there is no effect whatsoever. This summary description even overstates the effects of cumulative testing because in some of the comparisons the magnitude of the difference is negligible, a fraction of a unit on a scale or a per63 This splitting procedure was possible without the problem of contamination mentioned earlier because the subgroup received their questionnaires at their homes and had little or no opportunity to communicate with other members of the Encampment during that period. Observations made during the Encampment suggest that the experience of initial testing was forgotten or taken so casually that it did not appear to have any effect on either group's behavior. Nor did campers of the tested group seem to discuss it with the other campers. Thus any artificial influences of being treated to this special procedure seem negligible.

Principles of Evaluation

37

centage point of difference. Clearly there is no systematic practice or sensitization effect from cumulative testing. Thus for one-third of the Encampment group any effects of testing whatsoever are excluded from the change scores to be presented. By definition the effect of initial testing cannot operate on the changes they show after the Encampment, and no effects of cumulative testing are demonstrable. A final argument might be advanced in support of the conclusion that retest effects are negligible. Unless practice and sensitization effects are a function purely of the diffuse experience of being tested, they should be reduced by subtle or disguised tests, or by obscuring the significant items in a larger questionnaire." The subjects would be less aware of the specific test items and their content, would find them less easy to remember, and would not become so attentive to specific aspects of the program which in more obvious testing would have been made salient. Our questionnaire was lengthy; related attitude items were scattered and buried; and many questions were obscure in their purpose.K WORKABLE ALTERNATIVES TO THE CLASSIC CONTROL-GROUP DESIGN

The history of our studies of the Encampment will provide many illustrations of the problems involved and the strategies that may be employed in controlling for extraneous sources of change. In 1955 the possibility of obtaining a control group of equivalent individuals who were not exposed to the Encampment program seemed insurmountable. Recruits to the Encampment came from scattered sites all over the United States and were recruited through all sorts of informal and nonorganized channels. Theory and past evidence would suggest that individual change is a function of the social supports or barriers to such change and that the control group should come from the same diversity of social contexts. Our findings to be presented throughout the monograph will provide confirmation of this notion. In addition, individuals are self-selected in terms of their ideological predispositions and are filtered by the agency in terms of their potential for leadership. The Encampment group is most select in ideology, but ethnically most 61 Glock's findings (op. eit.) suggest that the effects are not merely the result of the diffuse experience of being questioned, as retest effects were found to be limited to the specific questions used in the pretest. Eesponses on new items introduced into the second wave of the panel were not affected by the experience of being tested before. Studies of practice effects from previous intelligence testing also support the argument. The effects are reduced when parallel forms are employed on the first and second tests rather than the identical form of the test. See Terman and Merrill, op. cit., or Anastasi, op. tit. 53 For a discussion of similar procedures to reduce sensitization to testing see Hovland et al., op. tit., pp. 25-31.

38

Applications of Methods of Evaluation TABLE

1-1

CUMULATIVE EFFECTS OF RETESTINO ON CHANGES IN OPINION DURING 1 9 5 5 ENCAMPMENT Area of opinion and attitude

Civil liberties " A " scale (median) Thrice tested Twice tested " B " scale (median) Thrice tested Twice tested Civil rights Zero discrimination (per cent) Thrice tested Twice tested Tolerance "More tolerant" on scale (per cent) Thrice tested Twice tested Permit peacetime Communist on radio (per cent) Thrice tested Twice tested "Protect the innocent" (per cent) Thrice tested Twice tested Action-apathy (median) Thrice tested Twice tested Absolutism (median) Thrice tested Twice tested Anomie (median) Thrice tested Twice tested Constitutionalism (median) Thrice tested Twice tested Authoritarianism (median) Thrice tested Twice tested

Opinion at start of Encampment

Opinion at end of Encampment

Amount of change

2.5

2.0 2.6

.5 -1.6

Reduces change

2.1

1.2

-

.9

2.5

2.0

-

.5

Increases change

4.2

Cumulative effect of testing

62

82

20

65

80

15

Increases change

88

97

68

86

9 18

Reduces change

79

88

9

53

70

17

Reduces change

97 91

100 89

3 - 2

Increases change

1.3 2.2

1.1 2.0

-

.2 .2

No effect Increases change

7.6

8.0

.4

8.3

8.5

.2

Increases change

2.5

2.1

-

.4

3.1

2.8

-

.3

6.5 7.3

5.5

-1.0

5.8

-1.5

Reduces change

- 3 - 4

Reduces change

40

37

47

43

39

Principles of Evaluation

varied, as will be shown in chapter ii. The counterpart is hard to find. The difficulties of finding individuals equivalent in status, social environment, and initial attitudes, and covering the variety of ethnic groups represented, are staggering. Moreover, to gain the cooperation of a control group that matched the experimental group for the necessary elaborate measurements seemed almost impossible. Consider finding the controls for the handful of Sioux Indian campers, tracking them back to their homes or reservation, and arranging for the tests! The obvious approach to the problem was to use as a control group those young persons who had expressed an intention to go to the EnTABLE

1-2

NUMBER OF APPLICANTS TO THE ENCAMPMENT AND SOURCES OF ATTRITION Group

Applicants Rejected Withdrew Campers a

1955

1957

1958»

Total

135 5 4

150

178 7

463 18

126

125

6 19

26

49

145

396

New York and California combined.

campment, but who for accidental reasons were unable to attend, or who were rejected for reasons unrelated to their attitudes and characteristics—for example, because no openings remained. Such individuals might well be regarded as equivalent, and there would be no problem of finding individuals with such unusual profiles of characteristics, as their names and addresses would be known to us. In 1955 the total number of applicants who withdrew or were rejected for all reasons was f a r too small to be the source for a control group of adequate size, and only some of these were really interested or eligible for the camp. At the time we could not count on continuing the evaluation in subsequent years; but, taking the long view, even banking such controls over a period of several cycles or years to make an ultimate adequate group would not have helped. In every year studied the total number of applicants who did not attend for any reason was very small. The detailed data are presented in table 1-2, which establishes how small a number is involved and the impracticality of setting aside some eligible campers merely for the sake of constituting a proper control group. They are not expendable Is6 w The general approach may well be practical in programs that have a larger number of applicants and a smaller number of slots available. For an example of such a program and this type of control group see Wilson and Bonilla, op. tit., p. 23.

40

Applications of Methods of Evaluation

In 1957 and 1958 it did become possible to obtain a control group of young persons who did not attend the Encampment and who were comparable to one subgroup of campers, the college students. The procedure will be described in detail in chapter viii, and the equivalence of the group assessed, but a number of observations are worth reporting here. The opportunity came about mainly because of a change in recruitment practices, so that recruitment occurred through a much more limited number of organized channels, namely, college administrations. Thus, the practical difficulties of covering a large number of scattered sites to obtain comparable controls in the first place, of locating eligible or interested young persons otherwise hidden in the larger society, and of contacting each one separately at considerable expense were reduced. It should be noted that the control group was comparable to only one subgroup of campers, namely, the college students, but it should be strongly emphasized that this strategy of using a control group that matches a subgroup of the exposed should be encouraged. Although the findings from such a design are limited to the subgroup in question, they nevertheless constitute rigorous proof. Two other aspects of our experience with this design are relevant. Despite the change in recruitment, and the way in which it facilitated finding and measuring a control group, in each year studied the control group obtained was still very small in size. Consequently the conclusions had to be strengthened by the replication of the design in two years and by pooling the two cycles to form one larger experimental and one larger control group. We have already commented on the distinctive fact that evaluations, in contrast with conventional experiments, are protracted in time and on the need this creates to keep an experimental group intact over time. As will be discussed in chapter viii, the protracted inquiry also means that the control group may well become exposed through its own wanderings to experiences much like those the experimental group receives. It cannot be assumed that the control group means a zero treatment or that it means a treatment different from that of the experimental group. It must be established empirically. In 1955 we sought an alternative to compensate for our lack of a conventional control group. A solution that immediately recommended itself was the use of Riecken's evaluation of work campers as a basis for constituting a control group. By incorporating many of his concepts and instruments into the Encampment study, we could compare the change caused by the Encampment with the change caused by workcamp exposure. This latter measure would be expressive of the influence on a very similar group of growth and the flux of ordinary events, and

Principles of Evaluation

41

of the contribution of instability due to measurement, plus the influence of the specialized experience of work-camp life." The use of Riecken's subjects as a control group may appear somewhat bizarre in the light of traditional views about control groups. Although they functioned satisfactorily as a control on growth, extraneous events, and instability of measurements, his young persons were exposed to the program of a work camp. They thus seem more like a comparative experimental group than a control group in the usual sense of a group that receives no special treatment. As Riecken argues most persuasively, however, and as we have suggested, in real social life a control group can never be exposed to zero experience over a protracted period of time; rather it is exposed to some undefined type of experience other than that to which an experimental group is exposed. How much better to be able to define the other type of experience. Thus, Riecken's group truly functions as a control group. It permits us to estimate the residual effects of the Encampment apart from the usual uncontrolled sources of change and over and above the changes that would have occurred if the campers had received the systematic experiences of a work camp. If there are no residual effects when this subtraction is completed, that, of course, does not mean there have been no Encampment effects. It simply means that there are no differential effects of this experience versus another systematic program of character education. If the effects of the program are smaller than observed for the comparative treatment, it again does not mean that there are no effects at all. I t does mean, however, that the relative effectiveness of the program under study is less. If the effects are greater, it is a very dramatic and compelling proof of relative success.68 The comparisons are presented in chapter iv. 67 The work camps studied had a cycle of about three months, rather than six weeks. Thus, in a sense, we are subtracting too large an estimate of the change that would occur for uncontrolled reasons in the Encampment, making our conclusions conservative. Admittedly, there is an assumption that the range, the relevance, and the intensity of events were about the same in 1948 when Riecken made his evaluation and in 1955 when we conducted our first study. 58 The use of a comparative treatment as a control group may seem to some an excessively harsh standard, but it is from one point of view a most reasonable, if tough-minded, standard to employ. Meyer and Borgatta, op. tit., pp. 105-106, present the logical basis for this standard in very persuasive and clear fashion, worth quoting at some length: "The idea of evaluation is usually conceived as examining whether an agency or program has some or no impact on the community by asking whether those who receive treatment fare better than those who do not. In the sense of general social welfare, however, any single evaluative research should be placed in the perspective of comparisons between alternative approaches that might be available to deal with a problem. It is not sufficient to determine that an agency's program is better than neglect. An agency is, inevitably, compared with other agencies whenever it asserts that its services deserve community support. It must therefore justify its existence in terms of efficiency within a competitive economy of

42

Applications

of Methods of

Evaluation

In 1955, when our evaluation of the Encampment was planned and started, Riecken's study was the only major comparative inquiry which had measured the effects of a similar youth program. Since then a second major work which provides additional comparative data has been published. In 1958 Robert Dentler evaluated several youth programs other than the work camps, but also under the auspices of the American Friends Service Committee. In measuring effects, Dentler employed some of Riecken's scales and some of ours, in addition to new ones of his own. Where the same scales have been used, this, of course, provides another comparative treatment which can function as a control group in interpreting the Encampment data, in the ways that have just been noted.™ These comparisons are also presented in chapter iv. Because the use of the comparative treatment was regarded as a demanding, although legitimate, standard, an alternative procedure was developed in the 1955 study. The problem in employing a traditional control group is the difficulty of obtaining a group for "zero" treatment which is equivalent to the campers. They might be matched in many obvious respects and still differ in some crucial but hidden psychological respect. An unusual solution developed in this study solves this problem of equivalence, by definition, and provides a control group to estimate the effects of growth, extraneous events, instability of measurement, and practice effect from pretesting. The campers were exposed to 42 days of an educational program, but the "after" measures contained not only these 42 days of specialized experiences, but also the influence of 42 days of normal living which needed to be separated. Why then not measure changes in the very same individuals during another 42 days of normal life? In abstract form the question we put was: Cannot the experimental group (or portion thereof) be utilized as a control group for itself over some time period other than the experimental period? With such a procedure, matching of experimental subjects and controls presents no difficulty, for the same persons constitute both groups. By determining how much instability there is in the group's attitudes, opinions, or other characteristics during a normal period of time we could then estimate how much of the change manifested during the experimental period exceeds the normal change resulting from the other factors. agencies.. . . The assumption that a service should be provided at any cost if it can be demonstrated to be useful can only be maintained if no other alternatives e x i s t . . . Although limited evaluation studies are worthwhile and necessary in order to judge the impact of individual programs, they would be of greater usefulness if they were part of a systematic program of evaluative studies of alternative services within a community." 68 E. A. Dentler, The Young Volunteers: An Evaluation of Three Programs of the American Friends Service Committee (Chicago: National Opinion Research Center, 1959).

Principles of Evaluation

43

The virtues of such a procedure have, to our knowledge, been generally neglected in evaluation research, although the use of experimental subjects to establish control data on themselves has precedent elsewhere. Subsequent to our development of the procedure, a paper by Selltiz and Barnitz called our attention to a British evaluation which had employed the same procedures as a substitute for a control group to estimate extraneous sources of change."0 James and Tenen tested the influence of personal contact on attitudes toward ethnic groups by having a group of English school children taught by African teachers.81 As Selltiz and Barnitz note, the usual exigencies of evaluation research prevented the use of a conventional control group. The school from which experimentáis and controls ideally could be allocated was too small (150 students) and contamination, no doubt, would have occurred. As the school was unconventional, it would have been difficult to find another school that would have yielded a comparable control group. Therefore, James and Tenen obtained two pretest measures of attitude, using the changes in the interval between them, when no treatment had occurred, "as a standard of comparison or control for the changes that followed upon the visit." After contact with the teachers, 41 of the 43 children had changed favorably, as indicated by ratings based on interviews. In the interval between the two pretests, 40 had been stable in attitude. The evaluation of psychotherapy poses similar fundamental difficulties in obtaining a conventional control group of subjects who would be equivalent in motivation and personality to those undergoing therapy. If the controls were equivalent in these respects, they would themselves either be in therapy or requesting it and there would be ethical barriers to withholding it. To solve this problem, Carl Rogers and his associates in the course of their evaluation of psychotherapy independently developed a design that parallels ours, labeling it the "own-control" group. A subsample of individuals requesting therapy were asked to postpone their therapy for a sixty-day waiting period during which time repeated tests established the magnitude of change due to natural events and growth, against which the effects of subsequent therapy could be appraised. The general utility of the design for evaluation of psychotherapy is demonstrated in detail in their work. The necessity of postponing the therapy during the waiting period, however, sets both prac60 C. Selltiz and E. Barnitz, "The Evaluation of Intergroup Relations Programmes," Internat. Soo. Sci. Bull., VII (1955), 364-375. 61 H. E. O. James and C. Tenen, The Teacher Was Black (London: Heinemann, 1953).

44

Applications of Methods of Evaluation

tical or ethical and technical limitations on the use of the design in evaluating therapy.63 For phenomena that are usually subject to routine record-keeping, the existence of a time series often provides the basis for such a design to be employed in the evaluation of the effectiveness of a program that is initiated. The time series yields evidence on the magnitude of the phenomenon for an interval other than the one during which the program is present, and thus gives an estimate of the influence of extraneous factors. Here, the evaluator even has the luxury of choosing more than one "control period." The evaluation of health measures in reducing infant mortality in Egyptian villages employed this device in addition to the conventional procedure of a control group from another village. This example incidentally suggests that, where the traces of a program do not carry over, the control period chosen may be one after the program has terminated, rather than one preceding its introduction. The evaluators in this instance observed low infant mortality rates during the program of fly control, and remarked that "the argument for the effectiveness of fly control is strengthened by the sudden increase in deaths in Sindbis and Quaranfil with the loss of fly control when 64% of all the infant deaths occurred after loss of fly control."03 In place of the convenient available time series, our procedure simply requires the investigator to collect the necessary data. Similar approaches have been used effectively in other fields. In experimentation on animal learning, the concept of the operant level refers to an empirical measurement of behavior during a normal, preexperimental interval which may then be used as a base line against which to measure the effect on the behavior of a subsequent experimental variable." Similarly, the classic concept of repeat reliability in psychological testing refers to an empirical determination of the stability of instrument performance during some interval against which changes in the measurements obtained on subsequent subjects may be appraised. Elsewhere in the medical sciences, instruments are calibrated to provide some expression of the range of variation in their performance against which variations under other conditions may be interpreted. 05 02 C. Rogers and R. F. Dymond, eds., Psychotherapy and Personality Change (Chicago: University of Chicago Press, 1954). See especially pages 46, 97, 139, and 188-190 for the limitations. 63 Weir et al., op. cit. 64 W. H. Schoenfeld, J. J. Antonitis, and P. J. Bersh, "Unconditioned Response Rate of the White Rat in a Bar-Pressing Apparatus," J. Comp. 4~ Physiolog. Psych., XLIII (1950), 41-48. 65 A. Graybiel, P. D. White, L. Wheeler, and C. Williams, Electrocardiography in Practice (Philadelphia: Saunders, 1952), p. 28.

Principles of Evaluation

45

In our inquiry, the design consisted of an initial measurement of the members of the experimental group six weeks before their arrival at the Encampment—(tj. The second set of measures for control purposes, taken on the first day of the Encampment—(t 2 )—is, of course, also the traditional "pretest" measure on the experimental group: two measures for the price of one, in a sense. Incidentally, this constitutes one practical advantage of the procedure over the traditional control-group design. In the latter, the four sets of measurements are costly, whereas here the research worker economizes on one set. Another practical advantage is that the subjects who provide the control data are not denied access to the program—not an unimportant consideration in the light of our earlier discussion of the settings in which evaluations usually occur. By comparing the changes between times 1 and 2 with those observed between times 2 and 3 we can determine how much of the change is associated with the program and how much might have come about simply from natural instability of the phenomena, test unreliability, practice effect from initial testing, and extraneous events. Obviously such a procedure involves the basic assumption that the magnitude of changes resulting from instability, unreliability, practice effect, and extraneous events will be the same during the six weeks preceding the program as it is during the experimental period. As to instrument unreliability, this assumption seems warranted. Similarly, because the two time periods are consecutive and not of long duration, there is reason to assume that natural instability or growth would not be significantly greater in one period than in the other. When the time intervals are long, or when they correspond to major turning points toward new developmental stages, the assumption of equivalence would be unwarranted. Thus the method is limited in its applicability to certain kinds of subjects or age cohorts and to programs other than those of very long duration. There still remains, however, a wide range of situations for which the required assumption about growth is met. "What of the influence of extraneous events? One should not assume that the outside world always impinges upon subjects with equal force during two time periods. In any particular evaluation, however, it is possible to determine whether any unusual events occurred in either period. Procedures for such determination are essential to the design. For the control period this can be determined by introducing questions at time 2 concerning events and experiences of the preceding six weeks. Answers obtained in 1955 indicated no unusual events in the interval which made that six weeks any different from any other six weeks in the subjects' lives. Admittedly, the results of such a check may not always

46

Applications of Methods of Evaluation

be so fortunate, and this is an unavoidable risk inherent in the method. The risk, however, is not so great as it might seem. This design was employed not only in 1955, but again in our 1957 study of the Encampment, where, as part of standard procedure, questions were again asked about the events of the six-week period before arrival. If we had merely been fortunate that one rare time in 1955, luck would not have broken our way again. In 1957 the same outcome attended our efforts, the evidence being that nothing unusual had occurred. What about the practice effect of initial testing which normally is confounded with the changes during the experimental interval? There is no reason to argue that practice on a test is any different in nature when it is obtained at one time rather than another, unless the subjects are in some unusual state of fatigue, inattentiveness, or reduced motivation so that they cannot benefit from the practice. It should be noted, however, that this design could be employed either to estimate the contribution of practice or to eliminate it. If all experimental subjects receive the initial testing (t t ), the influence of initial practice is already manifest by the second test (t 2 ) and is excluded by definition from the subsequent changes between times 2 and 3. Ideally one would not wish to subtract a factor for it from the gross findings after exposure to the Encampment, because this would be to overcorrect the experimental results. Because the change in the preexperimental period inextricably contains this component of initial practice effect, the method inevitably would subtract too much and make positive findings conservative. Although the literature suggests that practice effects would be small, and the design always works conservatively to handicap positive findings from an evaluation, it would be more desirable if the error in overcorrecting results for initial practice were not operative. The problem is taken care of by a special feature of the design. The total experimental group should be broken into two subgroups of unequal size. The smaller group (about one-third in our study) should be given the initial test and used to estimate change resulting from extraneous factors. Thus, for most of the experimental group, initial practice remains a component of the gross change observed from pretest to post-test (t 2 to t 3 ), and it is proper that its effect be subtracted. We are overcorrecting for practice only in very slight degree, in that for only a small portion of the total group have practice effects already been eliminated. Then, as in the conventional design for testing practice, by comparing the results at the pretest (t 2 ) for the group who had the initial test (t x ) against those who did not, one may obtain an estimate of the sheer effect of practice. If its magnitude is considerable, one may then allow for

Principles of Evaluation

47

the error that would be introduced by overcorrecting for its effect on a small portion of the total group. It might also be argued that practice is cumulative with repeated trials, and that the design introduces cumulative testing effects because one group is subject to three tests. The gross results observed from pretest to post-test are perhaps a function of the cumulative practice operative on the third test and of the cumulative sensitization to the program produced by being tested twice before the experience. The influence of cumulative testing, however, is operative on only a small part of the group and thus is probably of small magnitude; it may be estimated by comparing the change from pretest to post-test for the groups that had been subject to three as against two tests. If the effect of cumulative testing is of any magnitude, one may then allow for its influence having been operative on a small part of the total group. In our study, the effects were reported earlier and were negligible. The fundamental assumption of the design—that the two time periods, preexperimental and experimental, are equivalent from the point of view of the changes that would occur because of growth, events, instrument instability, and practice—seems well protected on grounds of logic and the special safeguards built into the procedure. Granted that some insecurity may still accompany the assumption, the situation is in no way worse than in evaluations that employ conventional control groups. As the many examples presented earlier show, the conditions of evaluation are such that the equivalence of a control group, no matter how carefully selected, finally rests on some judgments or assumptions. There is one major limitation to this alternative method of control. Ideally it requires that all the subjects of the program to be evaluated be known and available to the researcher well in advance of the program, so that the subgroup used for control purposes may be chosen randomly. This is not possible when recruitment to a program occurs right up to the time it begins. Insofar as late recruits differ from those known and available at the time of the initial control measurements, there will be errors in estimating change attributable to nonprogrammatic factors. In the instance of the Encampment, only the early applicants were available for the first test, and all these had to be employed for the control data. For certain characteristics, however, their equivalence to later applicants could be determined empirically. Moreover, an analysis of the factors governing early versus late recruitment suggested that the grounds were usually accidental and probably uncorrelated with the variables under study. In such a situation of late recruitment, one may adopt the rigorous and conservative procedure of comparing the changes

48

Applications of Methods of Evaluation

in the control and experimental interval only for the early subgroup. Then the findings on effectiveness obtained from the design would be limited to that subgroup, but would not be vulnerable to the danger that early and late recruits differed in some relevant respect. In our presentation of evidence in chapter iv, we compare the total Encampment group changes with the changes in the control period for the subgroup, because for the reasons presented we felt confident that early and late recruits were homogeneous. One other procedural difficulty attends the use of this design. The nature of the difficulty and a mode of solution are illustrated by the 1955 study. Although questionnaires were mailed to the subgroup of campers used for the control data in sufficient time to allow for a 42-day interval between measurement and arrival at camp, often they were not returned promptly. Among the 39 campers in the control group, 25 returned their questionnaires 5 weeks or more before the Encampment, but 14 returned their questionnaires less than 5 weeks before the Encampment. Consequently, if uncontrolled factors of growth, instability, or events are dependent on the exact duration of time over which they are operative, we have subtracted too small an estimate of change. In turn, we have overstated the intrinsic influence of the program. To test for this possible source of error, the 39 were divided into the two groups, contrasted in the time interval over which the control data were obtained. The comparison is presented in table 1-3. Differences between these two subgroups other than the time interval may obscure the comparison. For example, the prompt returners may be different in character from the tardy returners. All of the 39 were highly homogeneous, however, and all were very early recruits, suggesting that the problem was not serious. Fortunately, the procedural difficulty does not distort the conclusions based on the design. As reported in chapter iv, the changes for the entire control group of 39 were negligible. Instead, if we take only the 25 cases who meet the requirements of the proper time interval, the changes over the control period are still negligible. In the two instances where change is considerable, in per cent in the "no discrimination" category and in median score on democracy, the changes goes in the direction opposite to that obtained in the Encampment. This implies, so to speak, that without the Encampment they would have declined in support of civil rights and democracy. Therefore the improvement shown after the Encampment is all the more compelling. (See table 1-3.) Another mode of demonstrating that this procedural difficulty cannot have distorted our conclusions is available, namely, to compare the

Principles of Evaluation

49

relative changes for the short time interval versus those for the long time interval. There is no consistent evidence that the 14 who were measured during a time interval shorter than 35 days show generallyless change. In four instances, they show more change (although on two of the variables the differences between the groups are negligible). In three instances, they show less change, but in two of these instances, taking account of sign, they show more improvement than the longinterval group. Thus, there seems no reason to believe that our estimates of the uncontrolled sources of change based on the aggregate of 39 cases is too small. Evidence on the persistence of the effects of the Encampment will be presented in various chapters. Depending on how one defines persistence, an empirical measurement must be postponed for a long or short period of time after the termination of a program. In chapter vii, findings are based on tests of the campers six weeks after they returned to their homes. The tests were regarded as a measure of the stability of the effects despite the impact of community pressures and the intervention of natural forces such as growth. The choice of this time interval was dictated in part by theoretical considerations. It was also dictated, however, by its symmetry with our control data for a 42-day period before the Encampment. Some of the changes subsequent to the Encampment might, for example, represent the instability of measurement. The interpretation of the six-week follow-up thus might be clarified by comparison with the changes for the control period of six weeks. In 1955 the impossibility of employing a conventional control group led to the use of comparative treatments, such as Riecken's work campers, and to the innovation just described, using the campers themselves at another time period to obtain an estimate of change arising from uncontrolled factors. The continuation of our studies into 1957 and 1958, and the accompanying changes in recruitment procedures, provided an opportunity to employ a conventional control group of subjects who were measured over the same interval as our campers, and who were equivalent to the collegiate subgroup within the Encampment. The continuation of our studies also provided automatically a fourth solution to the problem of controlling for extraneous sources of change, which in contrast with conventional designs is peculiarly suitable to the conditions under which most evaluations are conducted. The replication of an inquiry may be regarded as a test of the generality of some earlier finding. If the evaluations of the 1957 or 1958 Encampment did not establish that they were as effective as the 1955 Encampment, one might then regard the earlier findings as valid but

a CD a

o hi

g® o II •ojzr

o H

§

»'S o a.

t

a H

O

o ce

3 0) > t» .a "m ® 2 • 2 "5b i S*a I &.§ o h a. O •S a

co

O &

M a o «

'S"

'S"

i bO1i bO Stm M o V V tí a io

1—1 co cd

> j -ca i3j S1 ó O

I 'cS a

a o

-ö œ a

Mo 3

Principles of Evaluation

51

specific to the particular cycle of the program, perhaps seeking the explanation of the difference in some particular feature of the recruits or staff of that year or execution of the program." When independently obtained findings differ, this may also call into question the validity of the original finding, for it might be that the earlier seeming effectiveness was related to uncontrolled factors, external to the program, staff, and subjects, such as the events occurring during that period or some artifact of the measuring instruments. Admittedly, one often does not know which conclusion to draw, whether to salvage the original finding, although sacrificing its generality, or to scrap it as spurious. Stouffer's comments about the researches on The American Soldier, where the opportunity for a great deal of replication occurred, describe the situation: The safest check . . . is the consistency of replications.... One of the most important lessons learned in the Eesearch Branch experience was that of the risks of generalizing from a pattern of differences revealed in a single small sample.... Even i f differences are in excess of chance, there are other biases which can play havoc with the interpretation.... Much the easiest parts of these volumes to write were those based on relatively sparse data. Where several replications were available, an analytical headache was all too frequently in store. Two studies would show a certain phenomenon; why did not a third? This was the point at which the I B M machines ran overtime. Sometimes the discrepancies stimulated further statistical analysis which led to quite important new ideas later backed by other evidence. Sometimes the discrepancies could not be reconciled by the most detailed and time consuming study. Sometimes the reluctant decision had to be made to stop with mere reporting of the discrepancies.... Obviously left unexplored are numerous tantalizing discrepancies, some of which may, in the perspective of future research, come to seem too important to be treated as perfunctorily as now seems necessary.67

When the replication confirms the original findings of an evaluation, the situation contains no element of ambiguity. Not only does it establish the generality of the finding; it also enhances confidence in the validity of the earlier finding. The uncontrolled factors of events and unreliable instruments would not operate in the same way in different periods. Unstable measurements ought to oscillate in a random way. The flux of events does not always push in the same direction. Thus, a replication of an earlier evaluation on a new group of experimental subjects performs certain functions of a control group, for it permits estimates of the influence of events, natural instability, and unreliability of instruM For a replication of an evaluation that shows that the effectiveness of a program of treatment of juvenile delinquency was not dependent on the original staff member conducting the treatment, see H. Ashley Weeks, Youthful Offenders at Highfields (Ann Arbor: University of Michigan Press, 1958), pp. 54-59. 67 S. A. Stouffer et al., The American Soldier: Adjustment during Army Life (Princeton: Princeton University Press, 1949), pp. 46-47.

oa f-c d re as a a .-a p g^.ai bo a 2 2 s a .a "S § Ifloo' o< iC S.2CD5 C C Q | 03 >

g^a - e 2

2

u,

a o ® m

¡CO » gCi3i 5(3 M ™ S-g $ O.S

O-P fl)flhO «B i s 2 ® a.« ® .2 § cl •S 8"= a ®o o 03 d s d 2 ° ® n . 2 y d - « a> ojjS S3 => 2 • fh 03 ^ "TS r v S ® O 3-S § s.a - a 73 S C3 O .S Kh Mi CQ O «•S So K 03

§ c" 2

>S P.

3 0Q ^ s i ijf i-. a I I « 2 b0 S l f f a i (3 -e w .S t s S i i a> 13

d g

5 Exposure programs

09 I 03 H Q

¿•a bp « 9.3 aam> m £ K S Sb.2tn m hO fl (V

® V hcd I s 02.2

o ^ o.-s^s

(3T3 •JS I s «3 * 0) o U 03l f-t bo »S K e-S. -T3 ®a o Su (3 55 £ a 2 ig g s ••a

§ fti

fl 03 HH O^l 03 a A * h S o s-H """Em" ad b e s K O >® 2 ® M c 2 a =3 o 2 m o a 53 ° aM d »

5 m

M

S

2

«

m i tt, a

— •"S, « » s d"S o .s £ ' 3 •s o S a s " o> 0 , 2 co tO

co CO

M hO

o ja *

:

» id

s u* ®

1

SS

S

«

-m *« O

S

o

M ®

a S

S

® SS 3 o S

® ' M o »

os

.

a

O

9
» O îg

fi o

h£)

^c o ^c o CO CO

co to

>>

® A

"S o

co o

1

5

.

• »H û) ö «3 S £ a 03 fn qj s » o S è a G O

*

c8 £ 0 tí t-i - 5 'S œ r»

s o) 0>

® -ti

©

g

»3 ® m O a 5 a O

_ O > a fe

»

c ^o CO

"S

o

S g i», —

S

^

o

o

3

Ê

o

B

-3 XI

5

a

0

I

e - s S * O 3

P. m O X ©

M g o -H CO o O o, s

«

•a

Who Are the Campers?

Ill

segregation in public schools, swimming pools, and hotels. Theoretically, scale scores could range from zero for persons who favored no discrimination in any of the areas covered to a score of 10 for persons who would discriminate against minority races or nationalities in all the situations. Incredible as it may seem, the median score for campers when they arrived at each of the four Encampments never exceeded 0.9. In fact, only 3 campers out of the total combined group of more than 300 endorsed more than half the discriminatory practices. Fully 61 per cent of the 1955 group expressed no discriminatory attitudes on the scale; similarly high proportions showed no discrimination in the 1957 and 1958 groups. It might be argued that such liberal views on civil rights were an artifact of the ethnic composition of the camp—perhaps the group seemed to have less discrimination than the national norms because it contained such a large proportion of minority group members who might be expected to be more tolerant. It turns out upon investigation, however, that this explanation is not true. There is approximately as much rejection of discrimination among the dominant white segment of the 1955 camp as there is among the Indian and Negro minority. In fact, fully 65 per cent of the white campers scored zero on the discrimination scale, as compared with 63 per cent of the Negroes and 60 per cent of the Indians—differences of negligible size. Differences in the civil rights scores of white and Negro campers were not great in 1957 or 1958, either. At this point the reader might begin to have some doubts as to the adequacy of the items used to measure opinions on civil rights. Surely any set of questions which fails to differentiate among half or more of the persons tested might be suspected as being too lenient or insensitive to be meaningful. It can easily be demonstrated, however, that the items on which this scale is based are far from acceptable to many of the young persons in our country. The items on which the scale is based are identical with questions asked of a national sample of high school students. A comparison of the views of the campers with those of the cross section of high school seniors is contained in table 2-4. Inspection of these findings reveals that, item for item, the campers express opinions on civil rights which are considerably more favorable than the opinions held by their peers in society. There can be no doubt that the Encampment was dealing with an exceptionally liberal group of young men and women. Just as the campers were found to be extremely liberal in their opinions on civil rights, so can they be shown to be quite favorable toward

112

Applications

of Methods of

Evaluation

civil liberties, indeed, much more favorable than the average high school senior. Two separate scales were devised to elicit campers' opinions on issues such as the necessity of trial by jury in criminal cases, freedom of the press, freedom of speech, and the necessity for warrants before searching a person or his home. Scores on these two scales on civil liberties ranged from zero, for persons who supported all the liberties involved, to 10 or 12 for persons who would disregard all. The median score in each of the four Encampments was well below the midpoint on each scale. Only 1 out of every 5 campers made scores that could be interpreted as antilibertarian on these issues. Here, too, as with civil rights, responses from a national sample of high school seniors may be compared with the opinions of the campers in order to give perspective to our findings. These comparisons are summarized in table 2-5. Again it turns out that the Encampment recruits were initially well above the average youth of the nation in their libertarian attitudes. In 1954 a nation-wide survey was conducted by Samuel Stouffer to determine the degree of tolerance which Americans had toward various forms of deviant beliefs and behavior. Items used in that study were incorporated in the questionnaire given to campers. In the nation-wide survey only 31 per cent of the persons sampled scored as markedly more tolerant than the rest of the population; fully 73 per cent of the 1955 campers would have qualified for this "more tolerant" group.13 Similarly high percentages of tolerant persons were found in the 1957 and 1958 Encampments. One of the major findings of the national study of tolerance was that tolerance seems to be a correlate of youth and formal education; that is, by and large, younger people tend to be more tolerant than older people and people of higher education more tolerant than those with less formal schooling. Perhaps the group within the national sample who most approximate the campers is the segment under twenty-nine years of age who have at least completed high school. Within this segment 55 per cent scored as relatively more tolerant on the scale—a considerably smaller percentage of tolerant persons than was found among the campers." Also relevant to the concept of tolerance are two additional questions, 18 S. A. Stouffer, Communism Conformity and Civil Liberties (New York: Doubleday, 1955), p. 51. 14Ibid., p. 93. This percentage is based on persons who followed the news about domestic communism "fairly closely" or "very closely." In view of the campers' demonstrated high interest in civic affairs it seems reasonable to compare them with this "interested" segment of the national population.

Who Are the Campersf

113

concerning the issue of domestic Communism, which were separate from the items used for the tolerance scale. First, campers were asked, "In peacetime, do you think that members of the Communist party in this country should be allowed to speak on the radio?" Second, they were asked, "In your opinion, which of these two is more important: (1) to find out all the Communists even if some innocent people should be hurt, or (2) to protect the rights of innocent people even if some Communists are not found out?" For both of these questions norms are available on the national population and on the segment of that population who have had some college education.15 How did the campers initially answer these two questions? In response to the first question, 59 per cent of the 1955 campers said, "Yes, in peacetime Communists should be allowed to speak on the radio." The magnitude of this expression of tolerance can best be pinpointed by the fact that only 22 per cent of the national population gave this response at the beginning of 1956. Even among the group most approximating the campers, that is, people of college education, only 38 per cent expressed this opinion. This view was somewhat less prevalent in the 1957 and 1958 Encampments, being found among 49 per cent of the 1957 group, 35 per cent of the 1958 New York group and 50 per cent of the 1958 California campers. Even more dramatic evidence of the 1955 campers' tolerance is contained in their answers to the second question.16 Ninety-one per cent of the group believed that it was more important to protect the rights of innocent people even if some Communists were not found out. In con16 National data are from a survey conducted early in 1956, and were made available through the courtesy of the National Opinion Besearch Center. The use of these items in no way implies that an objective of the Encampment program was to increase support of the Communist party in the United States. To the contrary. However, the Encampment was concerned with the development of such democratic attitudes and values as tolerance and support f o r civil liberties, and a research design properly should provide measures of the full range and extent of such attitudes. By including these two items, one could test whether or not the youth would include within the scope of their tolerance individuals and groups as unpopular as avowed domestic Communists. There may well be differences of opinion in evaluating the meaning of "tolerant" responses to such items, arising from the issue whether a truly tolerant and democratic outlook includes such a component attitude. But i t should be stressed that only by supplementing the Stouffer scale by such items could one test general tolerance, however one might decide to define it. The Stouffer scale is so constructed that responses to items referring to atheists, socialists, and Communists are all pooled into one single score, and the internal complexity of a person's total attitude is not amenable to analysis. The two additional items refer exclusively to Communists and various typological groupings could therefore be made of individuals who are generally tolerant but not tolerant toward the Com munist party versus those who are tolerant toward Communists as well as generally tolerant and even of the group which is intolerant generally but tolerant toward Communists. 16 Not asked in 1957 and 1958.

Applications of Methods of Evaluation

114

T A B L E

2-6

S U M M A R Y V I E W O F C A M P E R S ' O P I N I O N S AND A T T I T U D E S S C O R E S AT B E G I N N I N G O F 1 9 5 5 E N C A M P M E N T

S c a l e or i n d e x

Campers' median Bcore

T h e o r e t i c a l range of scale

E X P L I C I T ENCAMPMENT O B J E C T I V E S

Authoritarianism

48

No authoritarianism, 11 All authoritarian, 77

Democratic practices

58

All democratic, 15 No democratic, 105

Group stereotypy

28

No stereotypes, 9 All stereotypes, 63

Constitutionalism

7.1

All proconstitutionalism, 0 AntiConstitution, 14

Discrimination on civil rights

0.8

No discrimination, 0 All discrimination, 10

Action-apathy

1.9

No apathy, 0 All apathy, 6

Anomie

3.5

No anomie, 0 All anomie, 16

Civil liberties, scale "A"

3.8

Most libertarian, 0 Least libertarian, 12

Civil liberties, scale " B "

2.4

Most libertarian, 0 Least libertarian, 10

Individual political potency

8.0

Maximum potency, 10 Minimum potency, 0

Group potency

9.2

Maximum, 10 Minimum, 0

HYPOTHESIZED UNANTICIPATED E F F E C T S

Political-economic conservatism

37

All liberal, 10 All conservative, 70

Absolutism

7.3

No absolutism, 0 All absolutism, 12

Ethnocentric image of democracy

3.4

No ethnocentrism, 0 Most ethnocentrism, 9

Who Are the Campersf

115

trast to the campers, only 30 per cent of the national population and 40 per cent of college-educated citizens expressed this opinion. Table 2-6 presents in summary form the high lights of the attitude scores of the 1955 campers on their day of arrival at the Encampment. It documents precisely the findings outlined above, that the recruits were predominantly nonauthoritarian, democratic, and tolerant in their attitudes. Substantially the same characteristics prevailed in the 1957 and both 1958 Encampments; consequently, detailed data are presented for 1955 only. In three other areas in which the Encampment is not explicitly concerned with producing change, but which have been hypothesized as subject to unintended effects, the campers are clearly a selected group also. In the political-economic sphere they are inclined toward liberalism, somewhat narrow in their conception of democracy, and fairly absolutistic in their analysis of social problems. To simplify the interpretation of these numerical scores for the reader, the minimum and maximum scores possible on each scale are indicated in the second column of the table. These findings demonstrate unequivocally the high caliber of human material with which the Encampment began. Such an extreme group is, of course, to be expected. The aims of recruitment are most explicitly of this character. Moreover, other institutions similar to the Encampment which have been examined in previous studies also established a high degree of self-selection. Candidates for such institutions are usually elite individuals who are predisposed to agree with the objectives and sentiments represented by the institution. Comparisons with Riecken's group indicate that the 1955 campers are somewhat less deviant than his work-campers. On authoritarianism his subjects had a mean score of 41; on democratic practices a mean score of 51; on politico-economic conservatism a mean score of 32." Dentler's findings on subjects participating in other Friends' programs also reveal that the Encampment recruits a slightly less deviant group. His subjects match Riecken's in authoritarianism and democracy and are less authoritarian and more democratic than ours. Dentler employed our civil liberties scale and found his subjects to be slightly more libertarian than ours.18 Ignoring 17 H. W. Riecken, The Volunteer Worlc Camp: A Psychological Evaluation (Cambridge: Addison-Wesley Press, 1952), p. 245. The comparison, is somewhat obscured by the fact that we have computed median scores in contrast to Riecken's use of the mean. The mean, as more sensitive to high numerical values, would tend to be higher than the median. Thus the difference betwen the two sets of recruits is probably even a little larger than is suggested by the magnitude of the figures. M R. A. Dentler, The Young Volunteers: An Evaluation of Three Programs of the American Friends Service Committee (Chicago: National Opinion Research Center, 1959), p. 192.

116

Applications of Methods of Evaluation

these differences, however, all three programs clearly recruit an ideologically deviant type of young person. It is wrong to prejudge the consequences of having a student body which is initially so good. On the one hand, it might be argued that such liberal-minded persons are already attuned to the message of the Encampment and therefore can easily be intensified in their democratic tendencies. On the other hand, it may be that such individuals—already improved in their beliefs and attitudes—find the Encampment experience redundant or tiresome and thereby gain little from the summer. Nevertheless, one need not be discouraged or disturbed if the campers emerge from the summer having simply reinforced and confirmed the same basic democratic beliefs and liberal attitudes with which they started. For, as the evaluators of the volunteer work camps have stated, "if we grant that the democratic outlook on life is desirable, it does no harm to intensify the trait, and to give it support at a time when lifeattitudes are being crystallized." 1 " Certainly, no experiment in democratic education could ask for better recruits than were received at the Encampment. Nor could many face a greater challenge than attempting to improve upon such extremely favorable opinions as confronted the Encampment staff. While one might not anticipate great changes in the democratization of a group already so high-minded as the campers, this does not preclude the possibility of other influences of the Encampment. Such elite young persons may profit in other subtle ways—for example, by reinforcement of their convictions. It is important to measure such possibilities, given the elite composition of the group. In particular, one variable is highly relevant to the problem, namely, the manner in which the camper sees his views in relationship to the views of the rest of society. THE CAMPER AND THE REST OF SOCIETY

How does the camper relate himself to society as he juxtaposes his own views on social issues against the views that prevail? Does he feel that his views differ from those of the average American? Does he feel isolated in his views, or does he see some social support for them among other people? Does he perceive any norms for conduct which are operative in the larger society, or does he feel that he lives in a time of anomie? Given the best intentions in the world, a camper who felt such isolation might be so discouraged as to be unable to act upon his intentions. Perhaps the growing sense that others share his views—if only those within the Encampment—would have a salutary effect. 19

Riecken, op. cit., p. xvi.

Who Are the

Campers?

117

Campers saw themselves as more liberal than the average American in their views on issues such as civil liberties, economic planning, urgency of social reform, race relations, and international affairs. For example, only 6 campers in 1955 believed there was no difference between their views on these topics and those of the average American, whereas almost two-thirds believed they differed on three or more of the issues. The same type of self-image occurred in 1957 and 1958. Few of the campers saw this gap between themselves and the rest of society as very small; about half of the 1955 group felt that the difference in opinion was very great on one or more of the issues. It is evident, therefore, that campers were aware of the fact that their views deviated from those of many members of society. That this self-image is not a false one has already been demonstrated. Granted that campers see themselves as different from the average American, do they see themselves as being relatively alone in their stand on social issues? About 3 or 4 campers in 10, in each Encampment, said that, at best, "only a small minority of people think the way I do about social problems." About 4 or 5 in 10 believed there was a sizable minority of Americans who shared their views. One or 2 in 10 felt that most Americans think as they do. Finally, there is evidence of a certain amount of anomie in the world as perceived by the campers. A scale was devised which summarizes each person's views on the extent to which there are norms of proper conduct in our society which are still recognized and subscribed to by many people. Sample items which could be endorsed or rejected included: "Nobody cares whether you vote or not except the politicians," or "Since 1890 people's ideas of morality have changed a lot, but there are still some absolute guides to conduct." Scores ranged from zero, for persons who saw a complete set of norms operating in the areas tested, to 10 for persons who perceived complete anomie. The median for the 1955 group was 3.5; several persons scored zero, but only 1 camper, out of more than 100, scored 10. Similar findings were obtained in 1957 and at both 1958 Encampments. What interpretation can be put on these numerical anomie scores? Obviously they have little absolute meaning. To persons who have few internalized norms themselves, a view of a relatively normless society should not be particularly distressing. But campers are not of this normless type. As has been documented above, they are, by and large, a highly moral group, altruistic and self-demanding. It occurred to the researchers that for such people to perceive a disparity between their own conscientiousness and an amoral society would indeed be troublesome.

118

Applications of Methods of Evaluation

Therefore it would be a matter of some importance if these feelings of anomie were reduced as a result of a summer's contacts with other young persons of like high-mindedness and social concern. Such an accomplishment, if performed, should surely be credited to the positive side in evaluating the net effects of the Encampment. The awareness of social supports for one's views may well spell the difference between the well-intentioned democrat who remains silent and inactive in the face of apparently overwhelming public apathy or opposition, and the outspoken, active, and potent democratic citizen of tomorrow.

CHAPTER I I I

THE SUMMER'S EXPERIENCE INTRODUCTION F U L L UNDERSTANDING of the changes brought about by the Encampment requires a brief description of the nature of the summer's program and the campers' immediate reactions to it. What the campers were formally exposed to can be outlined quickly and need not be the primary concern of this chapter. More central to an analysis of the Encampment is the way it was regarded and experienced by the campers themselves. It is crucial, however, not to mistake these immediate reactions for evidence of the Encampment's effectiveness or ineffectiveness. The ultimate test of success cannot be whether the participants liked or disliked the summer's events, were happy or unhappy, were thrilled or disappointed, or believed they had changed or not. The final court of evalution must consider only the carefully gathered objective evidence on the degree to which the Encampment achieved its expressed over-all objectives without the cost of many undesirable side effects. The objectives, clearly, went beyond merely providing a pleasant summer; the Encampment is no lark. "Being at the Encampment," campers were told on their first evening at Riverdale, "is a sacred responsibility." The degree to which the Encampment achieved its quite serious objectives is discussed in the remaining seven chapters. The campers' immediate sentiments and reactions are not evidence of effectiveness, but they can establish a proper background for an understanding of the ultimate success or failure of the program. Pleasure or disappointment with the six weeks' investment of time and money, fatigue, traumatic incidents, and other subjective phenomena provided the emotional setting within which the objectives of the Encampment had to be achieved. Therefore, the current chapter, after a brief summary of the formal Encampment program, will describe the summer as seen by the campers. It describes their over-all feelings of satisfaction or disappointment, their images of the Encampment staff members and of one another, their feelings about the formal and informal features of the Encampment, the strains and disturbances they experienced, the salient elements of the Encampment for them, and their own self-analysis as to how much they felt they had changed and how much their future at home would be changed because of the Encampment experience. These latter senti[119]

120

Applications of Methods of Evaluation

ments, while poor evidence of actual change, are perfectly legitimate indicators of experienced satisfaction. The most detailed accounting will be made for the 1955 Encampment, the original group under study. Parallel data are presented for the 1957 and the two 1958 Encampments wherever available. In general, the formal organization and the informal operations of all four camps were similar. Each of the three New York Encampments was held on the campus of the Fieldston School, Riverdale. Each constituted a community unto itself, with campers not only participating in self-government, but also taking responsibility for such chores as cleaning up the grounds and serving on "kitchen police." Although not far from New York City, the Encampment was physically and socially isolated. By contrast, the 1958 California Encampment was less of a self-contained community and less well isolated from the outside world. This Encampment took place at International House, Berkeley, California. In this setting, maintenance and service were provided, and there was little call for campers to do routine chores. Furthermore, even though the campers were living together on several floors of the dormitory, there were summer session students from the University of California, Berkeley, and others living in the building, and the campers formed a community less isolated than that in New York. THE NATURE OF THE ENCAMPMENT

The Encampment combines an education in the principles of democracy with the practical experience of democratic living. These components— the formal education and the informal communal life—interact to produce the ethos of the summer's experience. "Talk is meaningless without the experience of living together," the educational director told the campers in his introductory speech on the first night of the 1955 Encampment. Tacitly implied in the Encampment's structure is the converse working hypothesis that "living together is meaningful only if directed toward some common democratic goals, such as the educational program." Consequently both the formal, didactic elements and the informal, social aspects of the Encampment need to be sketched for our report on how the campers saw their summer. THE EDUCATIONAL CURRICULUM

The core of the Encampment is its educational program. To illustrate, in 1955 this program consisted of three units of study, each lasting two weeks: a unit on political and social problems, one on economic problems, and one on international affairs. Each unit consisted of a series of

The Summer's Experience

121

basic lectures by the faculty and guests, followed by a period of intensive questioning of the speaker by the campers. Then the campers formed small discussion groups of 10 to 12 persons, with a staff member in charge. The purpose of these discussions, which lasted from an hour to an hour and a half, was to give each camper the opportunity to raise questions, explore points made by the lecturer, formulate and express his own views on the subject, hear the views of his fellow campers, and, in general, enrich his understanding of the topic. The combination of lecture and discussion usually occupied four mornings each week, beginning at nine o'clock and lasting until noon. Attendance was mandatory, although roll calls were not taken at the lectures. Three afternoons each week and all day on Thursdays the 1955 campers met in one of the nine workshops available for their choice. There were workshops on civil liberties, civil rights, desegregation, education, farm problems, international affairs, labor, political action, and recreation. Each camper could attend only one workshop during the summer; his choice was made in the first week. On the average each workshop recruited about 12 campers, led by 1 or 2 staff members, although some groups were larger and some smaller. As their titles suggest, each workshop explored intensively some practical and significant topic in which the campers were interested. The latent intention of each workshop was to provide campers with an opportunity to learn how to go about working on such problems, but the manifest goal was a final oral presentation of workshop findings before the rest of the Encampment, supplemented by an extensive mimeographed report. Although membership in some workshop was required, the groups did not always meet as units during the work sessions; occasionally individuals or small groups of campers worked independently on their project assignments during the workshop hours. The three major components of the educational program—lectures, discussion groups, and workshops—were supplemented by a number of ancillary educational features. For example, once a week, for a full day, groups of 1955 campers journeyed to New York City or to some other nearby attraction on field trips. These trips were not mere sight-seeing excursions. They were carefully planned to integrate with the current series of lectures and discussions and thereby to enhance the camper's understanding through providing him with firsthand observations of such institutions as the United Nations, labor unions, and the city government. Attendance on field trips was obligatory. Occasionally there were film forums or other cultural programs in the evenings. An adequate library rounded out the campers' sources of information.

122

Applications

of Methods of Evaluation

THE COMMUNAL LIFE

Of equal importance with the educational program is the communal setting within which it took place. The Encampment, as has been noted, emphasizes its qualities as an "adventure in democratic living." Campers were made aware of the privileges and responsibilities of communal living and self-government. They were told that their teachers likewise would be sharing their summer's experiences and responsibilities—living on the campus away from their families, eating, playing, and learning along with the campers. This was, in fact, exactly what happened. The most formal aspects of the community life were the meals and the scheduled recreation periods. Campers dined together at every meal, selecting one of the several tables, each of which seated about seven persons. A t Riverdale they took turns in serving food to themselves and their mealtime companions and in policing the dining hall. Similarly, campers took part in some sport together four afternoons each week, from three to about five o'clock. There were available, among other sports, tennis, volleyball, swimming, basketball, and softball. Campers were responsible for the equipment and for keeping the sports areas in good condition. In addition to the regular recreational periods there were a number of scheduled entertainment events, such as dances, sight-seeing trips, and beach parties. On Sundays arrangements were made for campers to attend a church of their choosing in the metropolitan area. Confronted with this lengthy array of scheduled educational and social activities for which attendance was obligatory, the reader may begin to wonder whether life was so regimented during the summer that there was no time for informal, personal interaction and activities. He may be assured, however, that there was time. First, there was a good deal of free time over the week ends. Although campers were still obliged (with some exceptions) to attend meals and certain scheduled cultural and recreational events, there were large blocks of time from Saturday noon through Sunday during which campers could visit places of interest in the city, have dates, go on walks, swim, or otherwise do as they pleased. Second, even during the heavily scheduled weekdays there were many moments for informal conversations and activities through which campers could get to know one another. On countless occasions the researchers observed a high rate of interaction and general esprit. Among the groups of three or four who would sit and rest by the great trees while waiting for the evening's program to begin and the hundred

The Summer's

Experience

123

campers assembled at the dining hall and waiting for the doors to open, there was talk, singing, and laughter. Campers were busy, but not regimented. As a community the Encampment was self-governing. That is, campers were free to make their own form of government and to choose their own leaders. In 1955 they elected to govern themselves through town meetings held each Monday night. It was made abundantly explicit, however, at the first meeting of the campers, that this democratic selfgovernment had to operate within the framework of a minimum set of regulations and subject to the higher authority of the Encampment directors and staff. Specifically, campers were told the "ground rules" of the institution—for example, regulations essential to the maintenance of good health and moral standards throughout the summer. A curfew would be enforced to ensure that campers obtained sufficient sleep; no alcohol or gambling would be permitted; campers were free to leave the premises, but were responsible for signing in and out so that officials would know their whereabouts; and, as has been mentioned, attendance was mandatory at meals, educational meetings, and scheduled social activities. In short, it was pointed out to the campers that freedom in government involved self-discipline; anarchy was not to prevail. Much of the routine maintenance of the New York campus was conducted by the campers themselves, through both voluntary and assigned work crews who helped with kitchen police, cleaning the dormitories, and so forth.In addition, campers organized a number of extra activities with the help of the staff, such as an Encampment newspaper and daily news reports over the loud-speaker at dinnertime. T H E SUMMER AS S E E N BY THE CAMPERS

SO much for the formal description of the Encampment. The preceding details should suffice as background on what the campers were exposed to during the summer, and may provide something of the flavor of that experience. The interested reader is referred to the Appendix for a complete statement of the six-week curriculum in 1955. It is now time to let the campers speak for themselves and tell how they perceived their summer experience and how they felt about it. EXPECTATIONS, FULFILLMENTS, AND DISAPPOINTMENTS

Campers were pleased with their summer at the Encampment. Nearly all of them agreed, at the end of each summer, that in general the Encampment had lived up to their expectations. (Only in 1957 did a sizable minority, 20 per cent, register general disappointment.) The majority

124

Applications

of Methods of

Evaluation

of campers even felt that in certain specific respects the Encampment had worked out better than they expected. A small proportion, though generally satisfied, felt that there was some specific way in which the Encampment had not met one or another expectation. When the Encampment exceeded expectations, the reason was not that the camper had initially had low aspirations or some misgivings. On the contrary, most of these instances revealed that the camper found things even better than his initially high expectations. Such discoveries were divided chiefly between gratifications derived from the formal educational program and those derived from various aspects of the communal life at the Encampment. There seemed to be less of a pattern in the occasional disappointments with the Encampment. A few campers felt that the educational program was inadequate in that it provided insufficient information or intellectual stimulation, or was too formal. Others felt that the program was not formal enough or contained too much material. Some people were disappointed because they felt there was not enough opportunity for relaxation and informal social activities. These complaints are not, of course, mutually exclusive; often what disappointed one camper pleased another. As a final sign of satisfaction, almost every camper said that he would recommend the Encampment to a close friend. REACTIONS TO THE FORMAL PROGRAM

Most of the campers enjoyed the formal educational program, felt that they understood what was being said, and were conscientious in their attendance. To illustrate, nearly two-thirds of the 1955 campers said that the didactic features of the Encampment (lectures and discussions) were among the activities they enjoyed most during the summer. Similar reactions were obtained in 1957 and in 1958, California. Only in the 1958 New York Encampment did less than half of the campers rank lectures and discussions among the most enjoyable activities. Conversely, few campers ranked these two activities as among the least enjoyable—4 per cent in 1955, 15 per cent in 1957, 19 per cent in 1958, New York, and 14 per cent in 1958, California. Furthermore, the lectures usually were understandable. In 1955 only 1 out of every 10 campers found the lectures and discussions difficult to follow "a lot of the time." About 6 out of 10 had occasional difficulties; the remaining three-tenths never had trouble. Difficulties were even less frequent in 1957 and 1958. When there were difficulties in following lectures or discussions, they were generally attributed to the vocabu-

The Summer's Experience

125

lary, the concepts, or the strangeness of the topic for the camper. Sometimes, but not often, the summer's heat, humidity, fatigue, noise from low-flying airplanes, and other environmental disturbances conspired to distract the camper and make him lose the thread of the lesson. Ordinarily, roll calls were not taken at the various activities at the Encampment. Yet defections were few. Anonymous self-reports by the campers, frequent observations by the researchers, and special daily reports by cooperating members of the Encampment staff confirmed the high level of attendance at these events. By their own reports, a majority of campers in each of the four Encampments missed no more than three educational activities throughout the entire six weeks. Many of these absences were nonvolitional, for example, because of illness. Obviously, campers backed their verbal enthusiasm about the program with hard currency of faithful attendance. REACTIONS TO THE NONDIDACTIC FEATURES OF THE ENCAMPMENT

The good feelings that campers expressed concerning the didactic elements of the Encampment were matched by their favorable reactions to the informal features of the program. For example, campers enjoyed the various social activities connected with the Encampment. They maintained a high record of attendance at these events, although occasionally they felt quite fatigued. Few campers felt imposed upon by the restrictions on their personal freedom which were necessary in communal living. Some persons encountered traumatic disturbances in their adventure in democratic living, but many of these unpleasant incidents were not connected with the Encampment. Campers had an image of themselves and the staff as dedicated, democratic persons, among whom friendships could easily be formed and who would continue to be good friends long after the summer had passed. Specific likes and dislikes.—In addition to the lectures and discussion groups, campers seemed greatly to enjoy the field trips and informal conversations or "bull sessions." Next in popularity was private exploration of the metropolis, and then the scheduled sports and recreation. Conversely, few persons listed either field trips or informal conversations as activities that they enjoyed least during the summer. (The most unpopular activity was routine chores around the camp.) Campers' enjoyment of informal activities is supported by a high record of attendance. By their own accounts, a majority of campers in all four Encampments missed no more than ten such activities (including the communal meals) during the summer. The strain of the six weeks' work.—Training for citizenship was hard

126

Applications of Methods of Evaluation

work. To illustrate, in 1955 the campers were up at seven-thirty each morning and, after eating breakfast and performing their routine chores around the campus, were ready for their morning lecture at nine o'clock. Their day ended with "lights out" at ten-thirty each night. Each day was tightly packed with assignments, formal activities, and social interaction. Listening, taking notes, discussing points with their neighbors, participating in workshops and communal activities—all took toll of the patience and the energy of the campers. How much strain did the campers feel? At the end of the summer each camper was asked, "How often, at the end of an average day of Encampment activities, did you have the feeling that it had been too much of a strain and a rush for you—almost always, frequently, occasionally, or never?" Very few in any Encampment claimed that they had never felt fatigue during the summer; most said they occasionally felt that way; and a minority frequently felt tired. Despite occasional periods of fatigue, almost no one felt that six weeks was too long a period for the Encampment. In fact, in 1955 about twofifths thought the Encampment had not lasted long enough; the remainder considered it just about right in length. Restraints on personal freedom.—Living under the constant surveillance of the staff and following so rigorous a daily round of educational activities, campers had few outlets for private activities and personal freedom. Given the natural vitality of youth and the especially antiauthoritarian orientation of the campers, these limitations on personal freedom could easily have created an atmosphere of resentment against the Encampment and its staff. This reaction did occur in some instances. About two-fifths of the 1957 and 1958, California, group felt that they had not been given enough personal freedom at the Encampment. On the other hand, most of the 1955 and the 1958, New York, campers (86 per cent and 72 per cent) felt their freedom had been sufficient. Furthermore, the satisfied campers were not simply Spartan individuals. In 1955, more than a quarter of them anticipated a great deal of personal freedom when they arrived at the beginning of the summer; yet they were satisfied. Clearly, then, little resentment at the disciplined life of the Encampment arose as a barrier to the over-all educational work. Traumatic incidents in democratic living.—Even in the most carefully supervised of activities there sometimes occur untoward incidents which upset the participants and impede their learning. In such a unique experiment in interracial, self-governing, communal living, where the pressures of interaction and study are intensified, one might

The Summer's Experience

127

expect many such incidents to occur. From a third to a half of the group in each Encampment reported that something had happened in the summer which disturbed them a great deal.1 Upon inspection of these answers it turns out that many of the reported disturbances were, in reality, intellectual awakenings having nothing to do with unpleasant occurrences within the program. That is, some campers became disturbed by their awakening to the ways in which the world is not ideal, or by coming into contact with new ideas contrary to their own. For example, one camper reported: "Since I came from a place where we don't have the Negro problem, I was indeed upset to find that the colored people face such discrimination." And another stated, "In a local bar I sensed a certain amount of feeling against the Negroes in the group. This was the first experience I had concerning race." Clearly, such intellectual awakenings, traumatic though they may be, can hardly be construed as detrimental to the learning process. A few of the reported disturbances, however, were potentially disruptive of the educational process. Some of them centered in real or imagined incidents of undemocratic behavior within the Encampment, on the part of either the staff or fellow campers. For example, one camper claimed, "Our faculty advisor ran our workshop in an authoritarian manner." Let it be clear that what is at issue here is not whether in fact such incidents occurred, but rather that some campers believed that this undemocratic behavior occurred. (It may be that some campers had forgotten that they were subject to decisions from authorities higher than themselves, as they had been clearly told at their orientation meeting, noted above, and therefore resented any "infringements" of their autonomy. Or perhaps these campers were disturbed by the manner in which some particular decision was executed. It is impossible to tell.) How serious were the consequences of such perceived discrepancies between the prescribed tenets of democracy and the imperfect application of these principles in practice? Such a judgment is difficult to make because so few campers were disturbed in this way. According to their own accounts, only one camper let his traumatic experience raise doubts about the principles being taught at the Encampment; several campers became somewhat less motivated to participate in the program, but others actually became more motivated as a result of their experience; and very few let the experience cast doubt upon the sincerity or abilities of the staff. Campers' image of the staff.—The campers' image of the Encamp1

This information was not obtained for the 1958 New York Encampment.

128

Applications of Methods of Evaluation

ment staff is a particularly essential factor in an understanding of the program's operation. As has been noted, both campers and staff members resided at the Encampment for the entire six weeks. This proximity between educator and pupils made the former particularly visible to the latter as role models for emulation. Obviously, if the campers thought that their leaders were selfish, insincere, or not interested in them as individuals, these sentiments might obstruct the staff's impact and weaken the Encampment's effectiveness. Fortunately such sentiments were rare. Campers were asked what they thought was the main reason that the staff members came to the Encampment. Hardly anyone attributed purely material purposes to the staff, such asfinancialgain or having a pleasant summer near New York. In contrast, the most frequently cited motives were usually purely altruistic: "A desire to help the students and the cause of understanding among people." A minority perceived mixed altruistic and private motives among the staff members: "They came to help us and, if possible, learn some more themselves." Still others felt that, though the staff came primarily for private reasons, these pertained more to some worth-while social goal than to purely crass ends: "They get so much satisfaction out of this sort of work" and "They enjoy feeling that they are working creatively with young people." Likewise campers were impressed by the interest which the staff members took in them as individuals. Most of them felt that the staff was greatly interested in them; others felt that the staff was somewhat interested ; at the most, a meager 3 per cent felt that the staff was very little interested; no one felt that the staff was not at all interested in the campers. It is evident, then, that despite an occasional incident or two, the campers thought very highly of the staff members, who appeared to them as models of altruistically oriented citizens. Campers' images of one another.—Equally as important as the campers' image of the staff is their image of themselves and their fellow campers. When they arrived in June, most of the 1955 campers (85 per cent) expected that they would meet a group of young persons who would have a great deal of interest in social problems. They were not disappointed; at the end of the summer 86 per cent reported that they believed most of their fellow campers did indeed have a high interest in social problems. At the start of each Encampment about 9 out of every 10 campers felt that most of the other campers would be liberal in their views on social problems. Again, they were not usually disappointed. For ex-

The Summer's Experience

129

ample, at the end of the 1955 and 1957 Encampments very few campers (9 per cent and 16 per cent) felt that there was little agreement between their own views and the views of most of the other campers on such social issues as civil liberties, economic planning, race relations, the urgency of social reforms, and the importance of politics." Surrounded by peers of such congenial views, campers found no trouble in making friends. In 1955 and 1957 hardly anyone reported that he had made no real friends during the summer; the average tendency was to have acquired from 6 to 8 new friends. 3 These newly acquired friends were persons with whom campers expressed an interest in keeping in touch after the Encampment. Furthermore, most of the group expressed a desire to keep in touch frequently with some other campers and alumni after they had returned to their homes or to school. The rest of the group, with a few exceptions, said they would like such post-Encampment contacts once in a while. (Discussion of the crossracial character of these friendships will be postponed to chap, iv, but it might be noted in passing that such friendships did cross ethnic lines.) DISRUPTIVE ELEMENTS DURING THE ENCAMPMENT AND SOME OP THEIR DETERMINANTS

In reviewing the campers' reactions to the Encampment thus far, we have touched upon a number of experiences that were potentially disruptive of the learning process. For example, general disappointment with the Encampment or specific disappointment with some part of the educational or social program of the Encampment might have lowered morale and thereby impeded learning. Or some other aspect of Encampment life might have influenced learning—fatigue, trauma, loneliness, feelings of regimentation, dislike for required sports or for the inescapable informal conversations and "bull sessions." Of course, not all reports of strain, tension, and experienced conflict in ideology should be construed as disruptive forces or barriers to learning. It may well be that some of these are simply accompaniments of personal development and fundamental change. Dramatic experiences might even accelerate or activate the process of change. There might, however, have been some disruptive experiences that were more directly related to the educational program itself, such as difficulty in following the lectures, or dislike for studying, reading, or lectures, which would have hindered learning. Finally, for one reason or another, some campers might have been absent from a large proportion of the formal or informal activities during the summer and thereby missed the full Encampment treatment. ' N o t asked in 1958. » N o t asked in 1958.

130

Applications

of Methods

of

Evaluation

Such disruptive experiences were not frequent. (Table 3-1 presents in summary form the frequency of their occurrence.) Of course, it is conceivable that these experiences had more impact than their respective frequencies imply; that is, while each of them might have only minor influence on the learning process during the summer, they might, altogether, have great impact. This hypothesis would seem especially realistic if a sizable number of campers had several of these experiences simultaneously. Such extremity of experience would, presumably, have TABLE

3-1

D I S R U P T I V E E X P E R I E N C E S D U R I N G E A C H ENCAMPMENT

(Percentages) Campers Experience

1956

(N = 100) (N - 106)

General disappointment Encampment did not live up to expectations. Encampment life Disturbed by some incident Frequently felt fatigued Felt there was not enough freedom in daily routine Occasional specific disappointments With communal life With miscellaneous aspects Enjoyed sports and recreation least of all the activities Enjoyed "bull sessions" least Made no friends at the Encampment Felt the staff had very little interest in campers Educational difficulties Occasional specific disappointments with educational program Frequently had difficulty understanding lecture or discussion Enjoyed lectures and discussions least Enjoyed study and reading least Absences Missed more than five educational periods.. Missed more than twenty meals or recreation activities » D a t a n o t available.

1958

1957

N.Y. (N - 81)

Calif. (N = 36)

2

20

5

11

27 22

55 13

a

12

46 33

13

43

22

42

12 11

5 27

7 15

16 33

11 7 3

11 5 0

8 3

19 6

a

a

3

2

2

3

13

19

19

19

11 4 40

8 15 38

5 19 28

0 14 39

15

18

9

19

12

8

4

11

The Summer's Experience

131

been a great handicap to the learning process. Such compounding, however, does not seem to have occurred. An analysis was made of the joint occurrence of three of the disruptive experiences: difficulty in following the lectures and discussions, frequent feelings of fatigue, and some disturbing incident.4 Only 9 campers in 1955 and 3 in 1957 had all three of these experiences during the summer. At the same time there were only 17 campers in 1955 and 5 in 1957 who had none of them. It seems evident that few campers escaped having at least one potentially disruptive experience during the summer, but also that few campers had to bear the brunt of several such experiences. Although the incidence of disruptive experiences was never high in any of the Encampments studied, the fact that the four Encampments differed considerably on the various measures deserves emphasis. Notably, a higher proportion of the 1957 New York and 1958 California campers had disruptive experiences. At the end of our first inquiry in 1955, it seemed that the rarity of disruptive or traumatic reactions was one of the important factors conducive to the Encampment's success. Now, in the light of variations in the number of disruptions in the different Encampments, and the corollary findings (to be presented) that all the Encampments were effective, it seems that the desired attitudinal changes can be brought about even when there is more disruptive experience than was noted in 1955. It might have occurred to the reader that these reported difficulties may be not genuine troubles, but merely verbal manifestations of some underlying dissatisfaction or disgruntlement on the part of a few of the campers. It is a well-known psychological fact that latent dissatisfactions may take any number of overt, seemingly rational manifestations. Therefore it is important to establish whether those campers who reported, for example, that they were fatigued or had difficulty in following lectures were not, in actuality, simply looking (consciously or unconsciously) for excuses to criticize a program with which they were initially unsympathetic. Were the campers who had disruptive experiences simply hostile to the Encampment to begin with? There is no evidence to support such a hypothesis and, indeed, considerable material which indicates that these campers were initially quite attuned to the major values expressed during the Encampment. In fact, in some instances these campers were the elite of the group, insofar as they initially endorsed Encampment principles even more than their fellow campers. For example, 1 This analysis is confined to 1955 and 1957; there were insufficient cases or data for analysis in 1958.

132

Applications of Methods of Evaluation

the 1955 campers who had a relatively high rate of defection from the educational periods (i.e., missed more than four such periods) had a median score of 2.6 on the civil liberties scale when they began the Encampment, whereas those who defected little during the summer began with a score of 4.2—less libertarian. Similarly, 81 per cent of the 1955 defectors began the Encampment with scores among the "more tolerant" group on the Stouffer tolerance scale, as compared with 74 per cent among campers who seldom defected. Or, consider the campers who reported frequently becoming fatigued. Initially they were just as libertarian as campers who only occasionally became fatigued, having a median score on civil liberties of 3.0 as contrasted with 3.8 for the latter. And they were as tolerant as those who did not complain of feeling tired; 77 per cent of the fatigued group qualified for the "more tolerant" scores at the beginning of the Encampment, as compared with 76 per cent of the group who felt little or no fatigue. Likewise, campers who had some disturbing experience while at the Encampment were not "worse" than other campers. Nor were those who reported that they felt there was insufficient freedom in regulating their daily routine during the summer. Nor was the group that met disappointment in their initial expectations about sundry features of the Encampment. And so on. In short, these reported disruptive elements in the Encampment do not come from an initially disgruntled or undemocratic group. The reported difficulties cannot simply be dismissed as the complaints of "cranks." They come from good campers, whose reports must be regarded as sincere. Granted, then, that there were real disruptive experiences during the Encampment, it may be useful to attempt to pinpoint some possible determinants of these experiences. For example, it might be conjectured that the younger campers, or those who had not attended college courses, might have had more difficulties in following the lectures. Or perhaps the women became fatigued more readily than the men. Or perhaps campers with previous experience in other citizenship camps had less difficulty than others. Answers to such questions might be useful either as guides to recruitment or to institutional provisions for reducing disruptive elements in future Encampments. What kinds of campers had troubles? As it turns out, these difficulties do not seem to be functions of any of the more obvious attributes of individuals. By way of illustration, data will be presented on people who reported that they frequently felt that the Encampment was too much of a strain and rush. First, this reaction seems to have little relation to

The Summer's

Experience

133

sex differences. About an equal proportion of men and women felt fatigued. Second, age seems to make little difference. In 1955 fatigue was reported by 20 per cent of those under twenty-one years of age and 28 per cent of those twenty-one or over. Similarly, greater formal education makes little difference; about a quarter of the high school students, as contrasted with 20 per cent of the college students, reported fatigue in 1955. One important correlate was established. It was discovered that those campers who initially had some apprehensions about the major, recurrent, inescapable elements of the Encampment reported greatest fatigue. Among campers who looked forward least to lectures or discussions, sports, and informal conversations—all salient components of the program—subsequent reports of fatigue were more frequent. Most probably this finding simply means that repeatedly facing a situation about which one has misgivings takes its toll in physical strain. Campers who initially had misgivings about parts of the Encampment which were escapable or less recurrent (e.g., private reading, dates) were not more prone to feel the strain of the summer. These data should not be interpreted to mean that campers who reported strain did not subsequently like those features of the Encampment—they liked them as much as the campers who felt no fatigue; but the adjustment required by them because of their initial apprehensions was more exhausting. What can be concluded about these disruptive experiences during the Encampment? Because of their diversity and pervasiveness (few campers escaping at least one such experience) it seems there are no particular causes of these incidents. Rather, each seems to be idiosyncratic. Taken all in all, they appear to be part of the natural adjustment and development to be found within a community of more than a hundred young persons, working hard and living intensely. This is not to say that such experiences may be ignored. It simply means that, so far as the data indicate, there was no particular factor in the various Encampment programs or the kind of person it attracts which accounts for these phenomena or creates an excessive amount of disruption. SALIENT E L E M E N T S IN T H E ENCAMPMENT FROM T H E CAMPERS' VIEW

Data have been presented on what campers liked and disliked about the Encampment, the disturbances which they experienced, their reactions to numerous features of the didactic program and communal life. There remain to be examined two important questions. (1) Of all the stimuli to which the camper was exposed during the summer, which components

134

Applications

of Methods

of

Evaluation

of the program (if any) left the greatest impression upon him? (2) In what ways (if any) did he feel that his Encampment experiences had changed him? At the end of the Encampment the campers were asked whether anything that happened during the summer had a very favorable influence on them." A majority believed that something had. But when they were called upon to describe what had happened their answers revealed a common inability to abstract any specific elements from the total Encampment experience. For example, in 1955 only 8 per cent cited some specific educational, interpersonal, or other type of event (e.g., a lecture, a new friendship). The bulk of the responses repeated in a variety of general ways the camper's feeling that communal life, with a diversified group of young persons, genuinely improved him, but cited no specific events. Frequently, references were made to the value of the interracial contacts made during the summer. Seldom were references made to specifics in the didactic parts of the Encampment. A few illustrative answers from 1955 may dramatize this finding: Camper 242: Getting to know myself better through group living experience. And getting to accept emotionally, not merely intellectually, people of different racial, religious and economic backgrounds. Camper 256: The encampment in general. Camper 255: The fact that people who differ so greatly in background, race, religion, etc., could get along so well together. Camper 129: The experience of living with such a diversity of personalities. Camper 224: I now feel that people with a common goal can live peacefully together and learn to be citizens of a democratic society.

The common pattern in these typical responses is patent. They are composed more of general feelings about the Encampment than of reports on specific happenings during the summer. They emphasize the communal living rather than the didactic program. They focus on the values of contacts with a diversified group, and especially an interracial group. This indicates that, from the campers' point of view, the salient elements of the Encampment were those which constituted the "adventure in democratic living." Additional support for this interpretation may be found in the 1955 campers' retrospective appraisal of their experience, obtained from the mail questionnaire sent to them six weeks after they returned to their homes. Campers were asked, "Now, as you think back to this summer's Encampment, what would you say was the most important experience that you had there?" With six weeks separating them from the Encamp5

Not asked in 1958, New York.

The Summer's

Experience

135

ment, most of the group were then able to isolate some part of the experience as outstanding in their minds. Only 4 per cent still felt that everything that happened was so nearly equally important that they could not single out any one element as more important than the rest. When campers specified the most salient feature of the summer, fully 80 per cent cited the community living. Within this sizable group, about three quarters made specific reference to the diversity of persons with whom they had lived during the summer, and more than half of this latter group made specific mention of the interracial composition of the Encampment. Only 9 per cent of the total group singled out any kind of didactic experience as the most important experience of the summer. The remaining few answers were varied. To conclude, there is strong evidence that the most salient feature of the total Encampment experience was the community life. This does not mean that the educational program was ineffective or unimportant. But it does mean that what was psychologically central to the campers was the group experience in democratic living which they so much enjoyed. CAMPEKS' SELF-ESTIMATES OF CHANGE

No description of the campers' image of their six-week experience would be complete without some evidence on their own appraisals of the impact of this experience upon themselves. Did they feel that the Encampment changed them very much? In what ways? Would the Encampment experience stretch out and influence their activities once they returned home? Would it change in any ways the relationships between themselves and their families, friends, or neighbors? Let it be emphasized here, as earlier in the chapter, that the answers to such questions are not presented as evidence of actual change. Real changes have been measured objectively, and are reported in chapters iv, vi, and viii. Nevertheless, these subjectively felt or imagined changes (whether real or not) are part of each camper's psychological experience during the summer. How did campers think they had changed? In each of the four Encampments a sizable minority (21 per cent to 42 per cent) believed that their Encampment experiences had changed them personally "a great deal." Others (45 per cent to 67 per cent) thought they had changed "a moderate amount." Few (11 per cent to 19 per cent) felt they had changed "hardly at all." When asked, "In what ways do you feel you are a changed person?" most of those who believed they had changed were able to specify how. Indeed, many of the campers were quite articulate in their self-analyses. Four themes

136

Applications of Methods of Evaluation

dominated their notions of change: (1) a greater awareness of and knowledge about social problems; (2) a greater respect for other persons' views and more understanding of people; (3) less prejudiced attitudes in the area of racial and ethnic relationships; (4) a greater interest in social problems, with the intention to engage in activity directed toward solving these problems.8 In summary, campers felt that they had become more aware, knowledgeable, interested, and potentially active in the realm of social problems. Many felt they had become less prejudiced and more respectful and tolerant of others and their views. Did the campers expect their new educational experience to change their primary social relationships after they left the Encampment? Most did not. With some exceptions, most campers did not think that, as a result of their Encampment experience, there would be any change in their relationships with their family, old friends, or neighbors. Of those who expected changes in their family relationships, some thought the relationships would be better, others thought they would be worse, the rest were not clear as to the direction the changes would take. Similarly, when people thought their old friendships would be affected, some felt there would be improvement, others were pessimistic, and some were not certain what the direction would be. The same pattern appeared in predicting relationships with neighbors. To summarize, there is no evidence of great apprehension concerning the major social relationships to which they would return—family, friends, and neighbors. There were even some who anticipated improved social relationships. These are some of the ways in which campers believed they had changed as a consequence of their experiences at the Encampment. In the following chapters we shall see to what extent they actually had been changed in their values, opinions, and social attitudes. • As campers were free to give as many examples of change as they pleased, some of them mentioned more than one of these items, in addition to several miscellaneous changes.

CHAPTER I V

IMMEDIATE CHANGES CREATED BY THE ENCAMPMENT INTRODUCTION THE TIME has come to examine, in detail, data on the actual changes produced by the Encampment during the summer. The analysis consists of juxtaposing a description of the campers' characteristics at the end of the summer with data on these same characteristics at the beginning of the Encampment. Changes that appear between the two times provide evidence of the effects of the Encampment, plus the influence of growth and extraneous events in the six weeks. By subtracting the amount of change attributable to these latter phenomena it is possible to isolate the impact of the Encampment and to determine how well it achieved its immediate goals.1 Objective measurements were made of changes among the campers within six areas of character development, selected and defined as translations of the Encampment's explicit objectives or possible unanticipated effects, as described in chapter i. These six areas are: salient social attitudes and opinions, campers' basic values, action orientation, cognition of social problems, perceived relationships with the rest of society, and conduct. In describing the Encampment's immediate impact, the following procedure is followed. First, precise descriptions are presented of changes in each of the afore-mentioned six areas of character. Most details are given first for the original 1955 study. Then, wherever possible, comparative findings are cited from the three replications: 1957; 1958, New York; and 1958, California. In assessing all these changes we ignore temporarily the influence of growth and extraneous events, and simply treat each change as if it were caused solely by the Encampment. Later the amount of change which might have occurred through growth, and so forth, is described objectively through data obtained from a specially devised control group. One further prefatory comment is necessary. Whenever possible, the reader is provided with comparative data on changes discovered by Riecken in his analysis of the effects of voluntary work camps and Dentler's evaluation of other programs of the American Friends Service Committee. By comparing 1 As the same instruments were employed at both times, the change cannot be attributed to any change in measurement procedures; but the possible influence of instability, unreliability, and other measurement errors must be taken into account in interpreting gross findings on change.

[137]

138

Applications of Methods of Evaluation

these sets of data, one may acquire perspective on the differential effects of the Encampment experiment versus other approaches to character education and citizenship training. This provides, as earlier noted, another solution for the control group." REGIONS OF CHANGE

How much did the campers change in the direction desired by the Encampment? CHANGES IN SALIENT SOCIAL ATTITUDES AND OPINIONS

Data presented in chapter ii have demonstrated that when each Encampment opened most campers were extremely tolerant; favorable toward civil rights, civil liberties, and constitutional practices; democratic, but somewhat ethnocentric in their concept of democracy; relatively free from group stereotypes; liberal in their political ideology and low in authoritarianism. Important changes occurred during the Encampment. At the end of each Encampment the campers were even more tolerant, and more favorable toward civil rights, civil liberties, and constitutional practices. They may have been slightly less authoritarian. They changed little, however, in their group stereotypes and their democratic attitudes, and lost little of their ethnocentric concepts about democracy. At the same time, there is no evidence that as a result of the Encampment they became radical in their political and economic ideologies. These changes in salient social opinions and attitudes are summarized in table 4-1. By way of illustration, a few of the more important findings have been selected for discussion in this section. Perhaps the most striking changes brought about by each Encampment are the improvements in campers' attitudes concerning tolerance toward nonconformist views and behavior, discrimination against or violation of the civil rights of minority groups, and the civil liberties of citizens. Campers' attitudes were extremely democratic on all these issues from the start, yet they became even more so during the Encampment experience. For example, at the start of the 1955 Encampment 74 per cent of the campers made scores on the Stouffer tolerance scale which would have classified them as among the "more tolerant" persons of the country. (It may be recalled that such tolerance scores would have been made by only 31 per cent of the national population, and by 55 per cent of the persons in the country under twenty-nine years of age 2 Minor discrepancies between the data on campers' initial opinions as reported here and those reported in chapter ii are accounted for by the campers who missed the final questionnaire, as has been discussed in chapter i.

Changes Created by the

Encampment

139

who have at least completed high school—the group most closely matching the campers.) When the Encampment ended, 89 per cent of the campers made these high tolerance scores. Similar gains occurred during the other three Encampments. Parallel evidence on the growth and maintenance of extreme tolerance during the summer is provided by the two questions concerning the issue of domestic communism. First, the percentage of campers who would allow the peacetime use of radio broadcasting by a Communist party member increased during all four Encampments—for example, from 61 per cent at the start of the 1955 Encampment to 76 per cent at its end. Second, the percentage of 1955 campers who felt that it was more important to "protect the rights of innocent people even if some Communists are not found out" remained extremely high—92 per cent and 93 per cent at the beginning and end of the summer, respectively. Again it may be recalled that only 38 per cent of the country's college-educated population would have permitted use of the radio, and only 40 per cent would have considered protecting the innocent as more important than finding all domestic Communists. (This question was not asked in other years.) Evidence of dramatic improvements in the campers' attitudes on civil rights and minority-group discrimination may be obtained by comparing the percentage of campers who scored zero on the civil rights scale at the beginning and the end of the Encampment. (A score of zero means that the person disagreed with every discriminatory item in the battery comprising the civil rights scale.) To illustrate, initially 62 per cent of the 1955 campers made a score of zero; at the end of the summer the "zero" group had risen to 81 per cent. Again, these improvements were replicated in 1957 and 1958. Similarly, scores on both scales on civil liberties revealed an improvement in campers' views over the summer. For example, on one scale the 1955 campers changed from an initial median score of 3.7 to a final median of 2.4. On the other scale they changed from a median score of 2.4 to 1.8 over the summer. Though each change may seem small, the two sets of scores confirm each other and lend confidence to the interpretation that civil liberties was one of the areas in which the Encampment was most effective. Similar gains are recorded in 1957 and 1958. These changes compare favorably with Dentler's summer interns, who scored 2.4 on the civil liberties " A " scale at the beginning and 2.0 after their summer program (mean scores).8 ' R. A. Dentler, The Young Volunteers: An Evaluation of Three Programs of the American Friends Service Committee (Chicago: National Opinion Research Center, 1959), p. 106.

140

Applications of Methods of Evaluation TABLE

4-1

SUMMARY OF CHANGES IN CAMPERS' SALIENT SOCIAL OPINIONS AND ATTITUDES DURING EACH OF F O U R SUMMER ENCAMPMENTS* Area of opinion and attitude

Start of Encampment

E n d of Encampment

Direction of change

EXPLICIT ENCAMPMENT OBJECTIVES

Civil liberties " A " scale (median) 1955 1957 1958, New York 1958, California

3.7 4.5 4.4 3.7

2.4 3.4 2.6 3.2

Improvement in all four groups

Civil liberties " B " scale (median) 1955 1957 1958, New York 1958, California

2.4 2.6 2.2 2.5

1.8 2.2 1.5 2.0

Improvement in all four groups

Civil rights Zero discrimination (per cent) 1955 1957 1958, New York 1958, California

62 58 59 72

81 70 85 86

Improvement in all four groups

Tolerance "More tolerant" than average (per cent) 1955 1957 1958, New York 1958, California

74 63 67 81

89 78 88 92

Improvement in all four groups

Permit peacetime Communist on radio (per cent) 1955 1957 1958, New York 1958, California

61 51 35 50

76 67 54 75

Improvement in all four groups

Authoritarianism (median) 1955 1957 1958, New York 1958, California

45 50 49 46

41 46 45 42

Improvement in all four groups

Changes Created by the

Encampment

141

TABLE 4-1—Continued Area of opinion a n d a t t i t u d e

S t a r t of Encampment

E n d of Encampment

Direction of change

EXPLICIT ENCAMPMENT O B J E C T I V E S — c o n t i n u e d

Constitutionalism (median) 1955 1957 1958, New York 1958, California Group stereotypy (median) 1955 1957 1958, New York 1958, California Democratic practices" (median) 1955

6.9 7.1 6.5 7.1 28 30

5.8 7.3 6.1 7.3 25 28

b

b

30

28

58

56

Inconsistent changes

Little change in any group

No change

HYPOTHESIZED UNANTICIPATED EFFECTS

Political-economic conservatism (median) 1955 1957 1958, New York 1958, California

37 40 40 36

35 37 38 37

Little change in any group

Political party preference Democratic (per cent) 1955 1957 1958, New York 1958, California

48 51 44 58

56 54 40 64

Slight changes

Ethnocentric image of democracy (median) 1955 1957 1958, New York 1958, California

3.4 3.2 2.7 2.4

3.0 3.2 2.5 2.6

• N - 100 in 1955, 104 i n 1957, 81 in 1958, New York, a n d 36 in 1958, California. N o t asked. « N o t m e a s u r e d in 1957 a n d 1958.

b

No change or inconsistent small change

142

Applications of Methods of Evaluation

Although less dramatic than the findings just described, data on three other areas are especially worthy of further attention. These are authoritarianism, democratic practices, and political-economic conservatism. On the first there was some change, on the others hardly any. For each of these three areas there are comparative data on the amount of change which Riecken and Dentler found among their subjects over a summer. These data, then, provide a special control by which the TABLE

4-2

COMPARISON OF CHANGE IN CERTAIN SCORES FOR 1 9 5 5 CAMPERS AND RlECKEN'S AND DENTLER's SUBJECTS Attitude

Authoritarianism 1955 campers (median) Riecken's work-campers (mean) Dentler's interns (mean) Democracy 1955 campers (median) Riecken's work-campers (mean) Dentler's interns (mean) Political-economic conservatism 1955 campers (median) Riecken's work-campers (mean) Dentler's interns (mean)

Beginning of program

End of program

Amount of change

45 41 41

41 39 38

- 4 - 2

58

56

- 2

51 49

46 48

- 5 - 1

37 32

35 31

- 2 - 1

38

37

- 1

- 3

Encampment's seeming lack of influence in these areas can be appraised. A comparison between Riecken's and Dentler's data and the data from the 1955 Encampment is presented in table 4-2.* First, it is evident that the small decrease in authoritarianism which appeared within the Encampment over the summer exceeds that which occurred in the comparative groups. The magnitude in all instances, however, is small, especially considering that the range of the scale is from 11 to 77. Therefore, our interpretation that only slight improvement in this personality domain was achieved at the Encampment seems 4 H. W. Riecken, The Volunteer Work Camp: A Psychological Evaluation (Cambridge: Addison-Wesley Press, 1952), p. 248; Dentler, op. cit., pp. 100, 103, 107. Riecken's and Dentler's use of the mean score and our use of the median score might account for some minor differences. As the mean is more responsive to an extreme score, radical changes among a very few individuals could account for a seemingly greater difference in the aggregate than would be shown by the median. Mean and median scores have been rounded in this table.

Changes Created by the Encampment

143

warranted. Perhaps this level of character is not amenable to influence in so short a time as six weeks or a summer. Second, Riecken's subjects made better scores and improved more on the democracy scale than the campers. One possible explanation for this finding is advanced in chapter v, on the social and psychological determinants of change. Dentler's subjects changed little. Finally, none of the groups changed much in their scores on politicaleconomic conservatism. From the Encampment's point of view this is desirable. That is, the Encampment does not have as one of its objectives the indoctrination of its members with any political ideology, either radical or reactionary, liberal or conservative. Nevertheless, there was always the possibility that some such "conversions" might have been an unanticipated by-product of the Encampment program or community ethos. Indeed, the reason for including the scale in the study was precisely that: to determine if such unintended changes occurred. Now it seems that they did not. True, campers were very liberal in their political ideologies, but they started the Encampment already committed to such ideologies and were neither encouraged toward nor detracted from these sentiments over the summer. CHANGES IN BASIC VALUES

On the fundamental level of basic values only limited change was brought about by the Encampment during its six-week summer session. This finding supports the initial hypothesis that the program would have relatively little effect upon such basic components of personal character as values. There were some changes, which will be discussed. Taken all in all, stability prevailed in campers' values as reflected by five indicators: type of vocational goal, chief criterion used in evaluating an occupation, goal considered most worthy of the greatest personal sacrifice, opinions about the most worthy ambitions that a man can have, and opinions about the indispensable features of an ideal society. Stability of values is best illustrated by the types of career goals held by campers before and after the summer's Encampment. These data, presented in table 4-3, are available for all four Encampments under study. The findings are similar for each group. There is no indication of major changes during the summer in the values engaged in campers' selection of their own lifework. For example, at the start of the 1955 Encampment 42 per cent of the campers planned to enter some service-oriented career, such as teaching, medicine, or social work. At the end, 40 per cent had such intentions—no significant change. At the start, 8 per cent were inclined toward politically oriented

144

Applications

of Methods TABLE

of

Evaluation

4-3

STABILITY OP CAMPERS' VOCATIONAL VALUES D U R I N G EACH OF F O U R SUMMER ENCAMPMENTS»

(Percentages) S t a r t of Encampment

E n d of Encampment

Wanted service-oriented career 1955 1957 1958, New York 1958, California

42 52 55 45

40 55 53 44

- 2 No noteworthy change +3 -2 in any group -1

Wanted politically oriented career 1955 1957 1958, New York 1958, California

8 4 5 14

11 4 7 17

+ 3 No noteworthy change 0 in any +2 group +3

Wanted some other kind of career 1955 1957 1958, New York 1958, California

31 29 31 28

31 28 31 23

0 No noteworthy change -1 in any 0 group -5

Gave no occupational goal 1955 1957 1958, New York 1958, California

19 16 10 14

18 14 9 17

- 1 No notewor-2 thy change in any -1 group +3

Goal

Change

« N - 100 in 1055, 104 in 1957, 81 in 1958, New York, a n d 36 in 1958, California.

careers, such as diplomatic service; at the end 11 per cent were so oriented. The remaining choices were for nonservice and nonpolitical careers, and these changed little over the summer. The findings do not necessarily mean that campers did not change their specific occupational goals (indeed about 3 out of 10 believed they had changed them); what the findings do demonstrate is that there were few losses or gains in the general nature of expected lifework, about four-tenths of the campers always intending to go into some kind of occupation which was oriented toward service to mankind, and another tenth planning on careers of political action. Similar stability appears in the chief criterion by which campers judged various occupations. In 1955, about 31 per cent of the campers indicated at the start of the summer that the chief thing they considered

Changes Created by the Encampment

145

in evaluating an occupation as having excellent standing was its service to humanity or its essential value to the community; at the end of the summer 34 per cent cited this criterion. (Comparable data were not collected in 1957 and 1958.) At the start of the 1955 Encampment, 51 per cent of the campers designated some nonprivate goal as the end for which they would be willing to make the greatest personal sacrifice. At the end, 53 per cent gave this response. Among the nonprivate goals listed there was a slight shift from initial concern with peace toward subsequent concern over racial, religious, and social equality, but the change was not great. Somewhat greater gains in attention to nonprivate goals were obtained in the 1957 New York and 1958 California Encampments—10 per cent and 17 per cent, respectively. Only minor changes in the group's values are revealed by responses to the question, "What are the three most worthy ambitions that an individual may have?" At the start of the 1955 Encampment 27 per cent of the answers to this question reflected altruistic and humanitarian values; at summer's end, 36 per cent—a gain of 9 per cent. The gain in such altruistic responses was only 4 per cent in the 1957 Encampment and 11 per cent in the 1958, California Encampment. (This question was not asked of the 1958, New York, campers.) At the same time, obviously "selfish" answers to the question dropped from 20 per cent to 17 per cent in the 1955 Encampment—an improvement of only 3 per cent. Similar results were obtained in 1957 and 1958. Comparison of the above findings with those obtained during Riecken's evaluation of volunteer work camps is possible. Riecken also discovered great stability in young persons' values, with a few small changes like those found among the campers. In response to the question about man's most worthy ambitions, for example, he found a slight tendency for his subjects to answer in more sociocentric or humanitarian terms and less in egocentric and impersonal ones, but the changes were small.5 The proportion of altruistic and humanitarian answers given by his work-campers was 40 per cent at the start and 44 per cent at the end of their summer —a gain of only 4 per cent." Finally, in the 1955 Encampment there were few changes in the kinds of answers given to the question about the indispensable features of an ideal society, with the exception of an increase in the frequency of such answers as freedom of speech, religion, and thought. (The question was not asked in 1957 and 1958.) This increased concern over 5 6

Riecken, op. tit., p. 106. / 6 i d . , p . 239.

146

Applications

of Methods of

Evaluation

civil liberties contrasts with the reactions of Riecken's work-campers, who became slightly less likely to mention civil liberties and more likely to cite equality and democratic government as indispensable features of an ideal society.7 The increase in saliency of civil liberties for campers is a genuine reflection of an important feature of the 1955 Encampment. Civil liberties was "in the air" at the Encampment during the summer; it was a subject of lectures, discussions, and workshops as well as a topic for many of the informal conversations and "bull sessions" among the campers. So much emphasis had its impact on value orientations, as demonstrated here, and on opinions and attitudes as described earlier. CHANGES IN ACTION ORIENTATION

Campers came to the Encampment with a strong proclivity for social and political action, and they retained this orientation throughout the summer. There were even minor signs that these tendencies were strengthened. But the most interesting change was a shift of this action orientation away from international affairs toward the solution of national problems during the 1955 Encampment. On the political action-apathy scale, the median score changed from 1.8 to 1.5 in 1955; from 2.4 to 1.9 in 1957; from 2.1 to 1.4 in 1958, New York; and from 1.9 to 1.2 in 1958, California. (Dentler's subjects changed from mean scores of 1.4 to 1.2 on action-apathy.8) Each of these small changes indicated slightly more political-action orientation. At the same time, a few more campers indicated that, in attacking social problems, they would prefer to put the bulk of their efforts into actions that would result in immediate improvements rather than into longrange scientific study of such problems. On the other hand, there was a slight drop in the percentage of 1955 campers who selected participation in public affairs among the two activities they expected would give them the most satisfaction in their life (from 57 per cent to 52 per cent). (This question was not asked in 1957 and 1958.) Given the motivation to work for the betterment of the social life of mankind, few men have the opportunity to apply their energies equally to local, national, and international problems. Usually some priority must be given to one or another area toward which a person can direct his chief energies. Campers were asked which of these three areas they would prefer if they had a choice. "When the 1955 Encampment began, relatively few (17 per cent) seemed interested in national problems. 7 8

Ibid., p. 242. Dentler, op. cit., p. 104.

Changes Created by the Encampment

147

Many more (38 per cent) were concerned with international problems, and even more (43 per cent) were mainly interested in local community problems. Something changed this pattern of interest during the 1955 Encampment. At the end of that summer fully 28 per cent expressed a major interest in national problems—a group now rivaling in size the group that was interested chiefly in international affairs (31 per cent). The most frequent choice of the campers, however, remained the field of local problems, selected by 40 per cent at the end of the 1955 Encampment.' Such changes did not occur, however, in 1957 and 1958. CHANGES IN COGNITION OF SOCIAL PROBLEMS

The Encampment brought about an increased optimism about the ultimate satisfactory solution of such social problems as race prejudice, war, and economic depression and unemployment. Furthermore, it shortened the time that campers believed would be necessary before solutions could be achieved. At the same time, there is no evidence that this optimism developed at the cost of creating oversimplified notions of the causes and the solutions of social problems. Absolutistic thinking was not a by-product of increasing social optimism about man's power over his problems. Finally, there seems to have been a reassessment of the power that the individual has for solving social problems. A considerable drop in the campers' view on individual political potency occurred during the summer, whereas belief in the potency of the group in these matters did not waver. The greatest increase in optimism about the ultimate solution of a social problem came in the area of race prejudice. To illustrate, when the 1955 Encampment started most campers were already rather optimistic about the elimination of race prejudice, fully 41 per cent feeling certain of this and another 40 per cent feeling that a solution was "fairly likely." But a sizable minority (16 per cent) felt doubtful of the chances and another 3 per cent felt it was "highly unlikely" that a solution to prejudice would ever be found. At the end of the Encampment the size of the group who felt an end to prejudice was "fairly likely" or even "certain" had swelled to 93 per cent. The doubtful cases dropped to 5 per cent and only 1 camper out of the 100 felt that this feat would be "highly unlikely."10 Similar changes occurred during the 1957 and the two 1958 Encampments. Occasional, and less striking, changes occurred in the campers' out0

One per cent did not answer the question. At each time 1 camper did not answer this question, thereby reducing the sum of the percentages reported to 99. 10

148

Applications of Methods of Evaluation

look on the solution of war, cancer, and economic depression and unemployment. For example, the percentage of campers who believed that it was fairly likely or certain that man would find a satisfactory solution for war rose from 44 per cent at the start of the 1955 Encampment to 51 per cent at the end. Similar results were obtained in 1957 and in 1958, New York, but not in 1958, California. Optimism that a solution to cancer would be found was always high in all Encampments (around 90 per cent), rising only among the 1955 group. And belief that a solution to economic depression and unemployment is fairly likely or certain increased during the 1955 and the 1958, New York, Encampments. Thus the Encampment fanned the optimism of youth. But did it fan it too much? As no one knows the "correct" answers concerning man's chances for solving such problems, it is impossible to tell how realistic the campers' optimism might be. Nevertheless, some inkling of their realistic or unrealistic outlook may be obtained from data on how long they felt would be the time before each problem would be solved. There are two items of information of interest here: first, the average camper's time perspective on these problems and how much it changed during the summer, and, second, how many campers believed these were matters of extremely quick solution—say ten years or less. The former datum gives some perspective on whether campers felt these were problems that could be solved during their lifetime, and the latter datum gives a clue as to how much of the campers' optimism might be discounted as a reflection of "starry-eyed" youthfulness. At the beginning of the 1955 Encampment the median time designated by campers for the solution of each of the four problems was as follows: war, 98 years; economic problems, 71 years; race prejudice, 59 years; cancer, 9 years. At the end of the Encampment the judgments were: war, 92 years; economic problems, 66 years; race prejudice, 31 years; cancer, 8 years. Clearly, the campers' opinions on the length of time required for the solution of these problems had decreased somewhat during the Encampment, although not uniformly. The greatest change was in youth's outlook on race prejudice—a finding repeated in both 1958 Encampments, but not in 1957. Only war seemed a problem so great as to fall outside the possibility of solution during the average camper's lifetime, for all four Encampments. Other data on time perspectives in 1957 and 1958 formed no consistent pattern. With few exceptions, only a minority of the campers believed that the problems of prejudice, war, cancer, and economic depression and unemployment would be solved within the next ten years. Such short-time

Changes Created by the

Encampment

149

perspectives were notably more frequent at the end of each Encampment than at its start, except in one instance. The proportion of campers having such short-time perspectives on race prejudice rose in 1955 from 13 per cent at the start to 28 per cent at the end, but not in the other Encampments. Concerning war the short-time group hardly changed during any Encampment. For example, it was 18 per cent at the start and 17 per cent at the end of the 1955 summer. Similarly, the group who thought there would be a quick solution of cancer hardly changed, being 56 per cent at the start and 60 per cent at the end of the 1955 Encampment. And the group believing in a quick end to economic depression and unemployment merely changed from 17 per cent to 19 per cent over the 1955 summer. Thus it appears that, excluding views on race prejudice in 1955, the majority of campers retained rather long-range perspectives on the time required for the solution of pressing social problems. In the flush of youthful optimism about remedying social problems there always lies the danger of too simple an image of the nature of these problems and a haste to apply some ready-made solution. Certainly the increased optimism that the Encampment imparts to its participants would be less than desirable if it contributed to an increase in some absolutistic thinking about these social problems and their solution. To detect whether such untoward effect followed in the wake of the mounting enthusiasm of the campers, the researchers included in the measuring intruments a number of items which subsequently were compounded into a scale of absolutism. Campers' scores on this scale demonstrate clearly that no increase in absolutistic thinking was created during the Encampment. The median score of campers on the first day of the 1955 Encampment was 7.3; at the end of the summer it was 7.6—no significant change. In 1957 absolutism scores changed from 8.0 to 8.3—again insignificant. (The scale was not used in 1958.) At the same time, however, these figures indicate that a relatively high level of absolutistic or oversimplified thinking about social problems existed among the campers initially and throughout the summer. It was neither reduced nor enhanced by the Encampment experience. Campers, then, started with an optimistic view on the solution of social problems and became even more optimistic as a result of the Encampment. They also began with a rather absolutistic view on the nature and the solution of these problems and neither improved nor became worse in this respect. One final aspect of the campers' cognition of social problems remains to be examined: how much power they thought the individual citizen and groups of citizens have for solving

150

Applications

of Methods of

Evaluation

these problems, and the extent to which these views changed over the summer. Campers began the summer believing that both the individual citizen and groups of citizens had a great deal of potency in solving problems such as the prevention of war, reduction of corruption in national government, reduction of corruption in local government, improvement of housing, and improvement of race relations. On the index of individual political potency based on the answers to these issues the 1955 campers began with a median score of 8.1, which is quite high. Similarly, they felt that groups of citizens were potent, making an initial median score of 9.3 on the index. High scores on both scales were also obtained in 1957 and in the two 1958 Encampments. Nor did campers reserve their faith in the power of the individual for local matters only. For example, 51 per cent of the 1955 group felt that the individual citizen could do "a great deal" or "a moderate amount" toward the prevention of war and the reduction of corruption in national government. Likewise, 70 per cent believed that groups of people could do at least a moderate amount in these two areas. During the 1955 Encampment these outlooks changed. There was a considerable drop in the belief in the power of the lone individual to resolve such problems, but no wavering in the attribution of power to individuals operating as groups. The median score on individual political potency dropped from 8.1 at the start of the summer to 6.9 at its close. And the percentage of campers attributing moderate power to the individual in matters of war and national government dropped from 51 per cent to 35 per cent over the summer. On the other hand, the median score on group potency held almost steady, being 9.3 at the start and 9.0 at the end of the Encampment. Likewise, 65 per cent of the campers ended the summer firm in their belief that groups can help prevent war and corruption in government. The lesson that 1955 campers learned is clear. In group action there is strength; in isolated behavior, relative impotence. The finding can be documented in another way. If we construct a typology based on both the belief in individual potency and the belief in group potency, we find at the beginning of the Encampment that 27 per cent of the group are high in the belief that groups are potent, but low in the belief that individuals are potent. By the end of the Encampment, 43 per cent of the campers are classified as this type. Dentler's findings parallel those we obtained for the 1955 Encampment. On the index of individual potency, his interns dropped from a mean score of 6.1 to 5.7 over the summer; at the same time their belief in group potency increased.11 11

Op. tit., pp. 86-87.

Changes Created by the Encampment

151

Such a "lesson" seems to be restricted to the 1955 Encampment; it was not "learned" in 1957 or 1958. On the contrary, there was a slight increase in the scores on the index of individual political potency during the 1957 and both 1958 Encampments. Group-potency scores increased in 1957 and 1958, New York, but not in 1958, California. CHANGES IN CAMPERS' PERCEIVED RELATIONSHIP WITH THE REST OF SOCIETY

Early in the formulation of the study, two hypotheses were entertained about the ways in which the Encampment might affect campers' perceptions of the views of other persons. One hypothesis dealt with an unanticipated and presumably unfavorable consequence: there might be an increase in alienation of the campers from the larger society. Such alienation might occur if a camper came to regard himself as belonging to an elite group, possessing greater social interest and better social attitudes than the average American. The second hypothesis was that the Encampment might perform the favorable latent function of reducing the sense of intellectual loneliness and anomie with which campers began the summer. Such an improvement might occur through the campers' discovery of a new reference group of peers and staff members who shared many of their own sentiments and interests in social problems. If the former consequence occurred—growing alienation from the larger society—its implication would be somewhat changed if, at the same time, the campers developed an increased identification with other campers, a subgroup of that society, albeit a minority group in the society. The hypothesis concerning increased alienation between campers and the larger society had its origin in Riecken's findings on the effects of volunteer work camps. On the basis of a number of tests, Riecken concluded that his subjects had an inaccurate view of the opinions of the "average American" and that they did not improve in accuracy over the summer. Further, he found that during the summer his subjects became harsher in their judgments of the average American and, because they had improved their own attitudes, ended the summer feeling that there was a wider gap between themselves and their fellow citizens than had existed earlier. The undesirability of such perceived differences is argued by Riecken: . . . these developments may lead to a feeling of alienation from the larger society and may make work campers aware that they are members of a selected group. Such awareness is only a step away from smugness, from a sense of being part of a moral elite. And this step is so short and so easy that work-camp leaders should be on their guard to detect and crush moral priggishness." 13

Op. cit., p. 172. See also pp. 102-104.

152

Applications of Methods of Evaluation

Did the Encampment experience also contribute to a self-conscious distinction between self and the "average" American? Our findings neither support nor refute the initial hypothesis about alienation. In fact, there is a slight indication that two processes were going on simultaneously throughout the summer: an increase in felt alienation on the part of some campers, matched by an increased identification between themselves and their fellow citizens on the part of some others. To illustrate, at the start of the 1955 Encampment 41 per cent of the group felt that their views differed from those of the average American on four or five of the following issues: civil liberties, economic planning, urgency of social reform, race relations, and international affairs. At the end of the summer 46 per cent felt that they differed from the average American on four or five of the issues. Comparative figures for 1957 were 23 per cent at the start and 35 per cent at the end of the Encampment. The change, although small, supports an interpretation of growing alienation. On the other hand, at the 1955 Encampment's start 9 per cent of the campers felt that they differed from the average American on only one or on none of the issues. At the end of the summer 17 per cent of the campers felt they differed on only one or on none—an increase of 8 per cent, suggesting the possibility that there was a slight increase in identification with other Americans. A similar finding was obtained in 1957. Granted that an increase in alienation was an unanticipated consequence of exposure to the Encampment or to similar programs, such as the work camps, it does not necessarily follow that this is undesirable in light of the Encampment goals. Riecken tends to emphasize psychological components of alienation, the priggishness or elitism, and sees it as undesirable, as we did initially. It is equally important, however, to ask whether the alienated might not feel a greater compulsion to act upon, to reform, to alter a state of affairs from which they were alienated. If so, the rise of alienation, though unanticipated and unintended, cannot be evaluated as completely undesirable. It would only be undesirable from this point of view if the alienated were less prone to action. Small-scale analysis of the 1955 Encampment data suggests that Riecken's views and our initial views require considerable qualification. At the time of their departure from the Encampment campers were asked about their action intentions after they returned to their communities. On the various indicators examined, the alienated show an intention to act equal to or greater than that of the nonalienated. As will be described in chapters vii and ix, the 1955 group was restudied twice after they returned to their communities, once shortly after return and again some four years later. A

Changes Created by the Encampment

153

variety of questions about actions in behalf of Encampment principles, actions to counter undemocratic incidents, activity in various organizations, and the like were asked. The general finding on all these measures is that the alienated show action tendencies equal to or greater than those of the nonalienated.13 Did the campers become harsher in their estimates of the opinions of Americans, as did Riecken's subjects? In 1955 campers were asked how they thought the average American would answer the two questions concerning domestic communism: Should peacetime Communists be allowed to speak over the radio? Is it more important to protect people or to find out all the Communists even if some innocent people should be hurt? A comparison of the answers given to these two questions before and after the 1955 Encampment yields inconclusive evidence of an increase in harshness of judgment about the average American over the summer." Among the campers who ventured an estimate of the average American's opinions about a Communist radio speech, the percentage who thought the average American would not permit a Communist to speak on the radio in peacetime rose from 69 to 79—a slightly harsher view." On the other hand, the percentage who thought the average American would stress finding "all the Communists even if some innocent people should be hurt" dropped from 62 to 52—a less harsh view." In the 1957 and in both 1958 Encampments campers were asked how they thought the average American would answer the question about a peacetime Communist speaking on the radio. Evidence of some increase in harshness was obtained. Among campers making a judgment, the proportion who thought the average American would not allow such a 13 The assumption that alienation had undesirable consequences was first questioned by Barbara Levenson, whose suggestion led to the detailed analysis described. 14 In 1955 the campers became less certain of how the average American would answer (that is, less willing to stereotype him). At the start of the Encampment 19 per cent of the group stated that they did not know how the average American would answer the question about peacetime Communists on the radio; at the end 25 per cent said they did not know. Likewise, initially 27 per cent said they did not know how the average American would answer the question about protecting innocent persons, as against finding all Communists; at the end of the Encampment these uncertain persons made up 40 per cent of the campers. No such change occurred in the 1957 and 1958 Encampments. The percentage of campers who said they did not know whether the average American would permit a peacetime Communist to speak on the radio was 13 at both the start and the end of the 1957 Encampment; 20 per cent at the start and the end of the 1958, New York, Encampment; and 11 per cent at the start and the end of the 1958, California, Encampment. The other question was not asked in 1957 and 1958. 15 Incidentally, this answer was the more accurate view. National Opinion Research Center data show that in 1954 most Americans (81 per eent) would not permit such use of the radio. Cf. S. A. Stouffer, Communism, Conformity and Civil Liberties (New York: Doubleday, 1955), p. 56. 18 In 1956 the average American (64 per cent) would have answered "find all the Communists," according to surveys by the National Opinion. Research. Center.

154

Applications of Methods of Evaluation

speech increased from 82 per cent to 89 per cent during the 1957 Encampment; from 83 per cent to 89 per cent in 1958, New York, and from 82 per cent to 88 per cent in 1958, California. To summarize, there is no conclusive evidence that the Encampment intensified a feeling of alienation between campers and their fellow citizens, or that it led to a harsher view of the average American on the part of the campers." "What happened with the other hypothesis, that the Encampment might serve to lessen the sense of aloneness and anomie with which campers began the summer? There is considerable evidence that the Encampment was successful in partially dispelling these two feelings. To illustrate, at the start of the summer about a third of the group said they usually felt that, at best, only a small minority of the population thought the way they did about social problems. At the Encampment's end only a fifth of the campers felt so isolated. Although few campers believed that most Americans shared their views, more than half of them ended the summer convinced that at least a sizable minority of Americans thought as they did. Similar findings were obtained in the 1957 and 1958 Encampments. In 1955 the campers declined in their beliefs that the larger society was indifferent to moral considerations governing conduct. Their median score on the anomie scale dropped from 3.5 to 2.3 over the summer—a noticeable improvement. During the 1957 and 1958 Encampments, however, only small changes in anomie scores occurred. DEMOCRATIC CONDUCT DURING THE SUMMER

Occupied, as they were, from morning to curfew with lectures, workshops, discussions, and other features of the Encampment program, campers had few opportunities to translate democratic principles into new conduct. Of course, they could participate in the student government, the newspaper, and the like, but such activities were all part of the institutionalized procedures for training to be better citizens. Most 17 A recent evaluation of the Experiment in International Living, conducted by the Bureau of Applied Social Research under the direction of Robert Somers, also provides evidence that alienation does not increase after exposure to specialized programs of youth training. Under this program small groups of young persons under the guidance of a leader live in a foreign community for a summer, each one living with a foreign host family. Somers establishes that such experience leads to a more favorable image of the host country as indicated by changes in the traits attributed to the "typical father" in that country after the summer experience. A measure of alienation is provided by a parallel question asked before and after the summer about the traits of the "typical American father." The image of the American father showed practically no change. See "Young Americans Abroad," Columbia University, Bureau of Applied Social Research, October, 1959, mimeographed.

Changes Created by the Encampment

155

of the lessons of the Encampment were, and had to be, directed toward improved conduct as a citizen in situations to be faced once the Encampment was over—in facing local community problems, and national and international issues. (The campers' conduct in these areas after their Encampment training is discussed in chapters vi-ix.) Of course, it should be noted that the entire Encampment experience, with its emphasis on community life, provided a daily "experiment" in democratic living and, hence, in applying democratic principles to conduct. But the essence of the community living, as an experience, is so diffuse as to defy precise measurement and so intangible as to have eluded articulation by the campers themselves. (See chap, iii.) Nevertheless, campers engaged in one particular kind of voluntary conduct which reflects their internalization of democratic norms, that is, the selection of new friends across racial lines. An evaluation of these friendships is not so simple as it might seem, and requires some theoretical discussion. How ought the campers to have selected new friends? Obviously, one of the tenets of the Encampment was that racial prejudice is undemocratic. Therefore one might expect interracial friendships to have developed, provided that campers endorsed this democratic principle. The critical issue here is not really whether such friendships were formed, but the extent to which they were formed. Prejudice, by definition, is a two-edged sword. An individual may be prejudiced in favor of a minority group as well as against it, thereby exhibiting prejudice against the majority. Hence it is conceivable that people might be prejudiced either if they tend to select friends across racial lines more than is "normal," or if they make fewer interracial friendships than would be "normal." Clearly, the concept of what is "normal" is at issue here. To provide a democratic standard another principle of the Encampment must be invoked: the value of each person as a human being. Time and again campers were exposed to the philosophy that to be democratic they should evaluate an individual only in terms of his own qualities, not according to his color, religion, or other group stereotype. To be consistent with this philosophy of democracy, campers ought to have selected new friends because they liked them as persons. It would have been just as undemocratic deliberately to make new friends from other racial groups simply because they were members of that group as it would have been to reject such people as friends on racial grounds. How can we tell how "democratic" the campers were in their formation of new friendships'? Obviously, a simple direct question to each camper concerning his motives in friendship formation would not be

156

Applications of Methods of Evaluation

adequate. And to probe such motivations would have required lengthypersonal interviews which might have jeopardized the larger goals of the evaluation and incurred costs excessive in relation to the utility of the data. But there is a way to estimate the part that democratic individuality played in the campers' choice of friends. The method involves comparing the percentage of interracial and intraracial friendships made by members of each racial group. If race does not enter as a determining criterion in campers' choice of friends, then the total pattern of friends selected by any racial group should contain about the same proportion of members of other racial groups as existed in the total TABLE

4-4

FRIENDSHIPS WITHIN AND BETWEEN ETHNIC GROUPS o r CAMPERS

Camper's ethnic

Negro White

Proportion of total Encampment*

Per cent of friends who are Negroes b

Per cent of friends who are Whites1»

1955

1957

1955

1957

1955

1957

25

33

44

58

56

29

63

45

19

31

74

58

» T h e Encampment included a small number from other ethnic groups. Therefore the total, Negro plus white, does not equal 100 per cent. Data exclude foreign campers. b Because some of those chosen as friends b y Negroes or whites came from other ethnic groups, the total of choices does not equal 100 per cent.

camp. For example, since Negroes constituted about 25 per cent of the total 1955 group, then by chance about 25 per cent of the friends selected by white campers should be Negroes, unless radical prejudice is operating. If white campers' friends consisted of markedly less than 25 per cent Negroes, then it might be inferred that anti-Negro prejudice affected the choices. On the other hand, if white campers' friends consisted of a great deal more than 25 per cent Negroes, then prejudice in reverse was operating to produce an overselection. In either of the latter two instances the democratic principle of accepting a person on his own merits would have been violated. What was the interracial pattern of friendships formed over the summer? Campers were asked the name of their best friend at the end of the 1955 and 1957 Encampments. Specific names were obtained from 73 campers in 1955 and 84 in 1957.18 Too few Indians and other nonwhites (but not Negroes) gave responses to permit treating them as subgroups for analysis. Excluding them left 70 Negro and white campers in 1955 and 64 in 1957 for whom the data are adequate. "Many of the remaining group indicated that they had "several" or "many" friends.

Changes Created by the Encampment

157

Negro campers constituted about 25 per cent of the 1955 Encampment and 33 per cent of the 1957 Encampment. If friendships were formed irrespective of the race of the individual, then the amount of in-group choices among Negro campers should account for 25 or 33 per cent of their friends. In fact, Negro campers chose other Negroes as friends somewhat more frequently than would be expected. Similarly, the amount of in-group choices among white campers should account for 63 per cent of their friends in 1955 and 45 per cent in 1957 (the proportion of whites in each Encampment). White campers chose other whites in about the proportions to be expected. Negroes underchose white friends in 1957, but in 1955 in just about the proportion to be expected. White campers named Negro friends in about the expected proportions. The data are presented in table 4-4. What do these findings mean? The evidence points to a high degree of democratic selection of friends. Both groups show patterns of intraracial and interracial friendship close to the expected values if race were not a desideratum. White campers select very nearly the exact proportion of Negro friends that would be expected by chance in both Encampments. They hardly discriminate in friendship against either white or Negro. Negro campers do not quite overcome all the barriers to friendships outside their own racial group. At first thought it might seem that this slight defection from "ideal" conduct by the Negro campers could be explained by their having had less interracial social relations before the Encampment. But there is evidence to refute this interpretation. For example, 60 per cent of the Negro campers in 1955 reported that they had close friends among white persons before coming to the Encampment; only 43 per cent of the white campers had close Negro friends before the Encampment.19 Nevertheless, the Negro campers' choice of white friends in 56 per cent and 31 per cent of the cases must be evaluated against the usual pattern of interracial friendships for the Negro population in the society at large, where nothing approximating this degree of democratic friendship formation exists. All things considered, it appears that campers were to a large extent uninfluenced by racial distinctions when choosing new friends over the summer. Inferentially, they thereby demonstrated their internalization of the democratic norm of accepting each individual as a person in his own right. u There remains the further possibility that these interracial friendships existed at the time campers came to the Encampment. A check was made on this possibility. It turned out that old friendships which had carried into the Encampment were few and did not affect the percentages reported above.

158

Applications of Methods of Evaluation SUSCEPTIBILITY TO PRESTIGE SYMBOLS

One other finding, of particular interest, does not appear in the summary table 4—1. It represents the product of a special experiment embedded in the questionnaires and therefore has been reserved for separate discussion at this time. AN EXPERIMENT IN RESISTANCE TO PROPAGANDA

An important characteristic of the competent citizen, according to Encampment philosophy, is his ability to form his opinions about social issues on their own intrinsic merits. His opinion, ideally, should not be altered by his emotional reaction to persons associated with the issue or to the propagandistic symbols in which the issue is couched. Moreover, the didactic program included a clarification of such propagandistic devices as the use of emotional symbols to manipulate the electorate's views. A special battery of items was invented to test whether campers developed an increased resistance to having their opinions manipulated by "prestige symbols" or traditional political party labels. The question employed asked campers if they personally favored or opposed each of six issues which would probably be political party planks in a presidential campaign. The issues were: establishing universal military training; setting a deadline for desegregation in the schools in the South; granting statehood for Hawaii; increased participation in joint Canadian-American control of uranium deposits; governmental reorganization transferring certain congressional investigatory powers to the Department of Justice; and strengthening the executive branch's immunity from congressional investigations. All these issues were, of course, merely hypothetical at the time of the study and some were outright fictions. All campers were asked the identical questions at the beginning and the end of the summer. But half of the campers were given a form of the questionnaire that attributed these policies to the Democratic party and the other half received a form that attributed the policies to the Republican party. Each camper was to receive the same form at both times. Presumably each camper's opinion on each issue was a composite of two factors: his attitude toward the substantive issue in its own right, and his reaction to the "Republican" or "Democratic" party label. In turn, his reaction to the label presumably results from his feelings about the two parties: the more he prefers the Democratic party, the more likely he is, other things being equal, to respond favorably to policies

Changes Created by the Encampment

159

attributed to that party and unfavorably to policies attributed to the Republican party; and the more he prefers the Republican party, the more likely he is, other things being equal, to respond in the opposite way, unfavorably toward policies attributed to the Democratic party and favorably toward policies attributed to the Republican party. To observe the effects of emotional symbols, we compare the responses of Republican campers receiving "Republican" forms and Democratic campers receiving "Democratic" forms with the responses of Republicans receiving "Democratic" forms and Democrats receiving "Republican" forms. If campers are susceptible to the party symbols, the first group should average more favorable responses to the six statements than the second group. The relative degree of susceptibility is indicated by the magnitude of the difference. If the campers became less susceptible over the summer, therefore, the difference between the responses of the two groups should be less at the end of a summer than it was at the beginning. A schematic presentation may make the procedure clearer. SCHEMATIC DIAGRAM OP EXPERIMENT IN RESISTANCE TO PRESTIGE SYMBOLS

Party preference

Version of questionnaire

Democratic

Democratic

Republican

Republican

Democratic

Republican

Republican

Democratic

Preference in relation t o questionnaire

P e r cent of responses t h a t are favorable a t t h e : S t a r t of Encampment

E n d of Encampment

Same

S

S'

Different

D

D'

Procedure: S minus D equals initial symbol suggestibility. S' minus 1> equals final symbol suggestibility. If S minus D exceeds S' minus D', then campers are less susceptible to the influence of party labels at the end of the Encampment than they were at its beginning.

It is conceivable, however, that these two groups would respond differently, not because of their different reactions to the party labels, but because of their "pure" attitudes on the substantive issues. Those campers whose questionnaire form and party preferences coincide (S) might happen to be more inclined to approve the six statements than those whose forms and party preferences differ (D). It was technically im-

Applications of Methods of Evaluation

160

TABLE

4-5

SUSCEPTIBILITY TO PRESTIGE SYMBOLS IN F O U R ENCAMPMENTS

Encampment

1955

1957

Time

Party preference in relation to schedule

Start

s

N»

Total responses

Per cent favorable responses

D

35 26

210 156

52 50

+ 2

End

S D

31 35

186 210

52 53

- 1

Start

S D

29 42

174 252

49 38

+11

S D

32 39

192 234

48 48

0

S D

25 31

150 186

57 44

+13

S D

20 28

120 168

53 44

+ 9

S D

10 17

60 102

45 48

- 3

S D

16 15

96 90

55 48

+ 7

End 1958, New York Start End 1958, California Start End

Sb - D° (percentages)

» Campers who did not prefer either one of the major parties have been omitted. There were 40 at the start and 35 at the end in 1955; 33 at both times in 1957; 29 at the start and 37 at the end in 1958, New York; and 13 at both times in 1958, California. I n five instances they had the smallest percentage of favorable responses^ and in three they were intermediate to the other two groups. b S — same. • D = different.

possible, before giving out the questionnaires, to match the groups on their party preferences and "pure" attitudes on the six issues. Therefore the two forms of the questionnaire were "randomly" distributed, with every camper having an equal chance of getting either version. In 1955 and 1957, the expedient was adopted of giving one form to campers whose surnames began with A to K and the other form to the rest. At the start of the two 1958 Encampments, the two forms were interleaved, and at the end of the Encampment persons who had initially received the Democratic form were seated separately from the others, who had initially received the Republican form. (In neither instance, of course, were campers aware of the reason for the procedure.) What were the results of this experiment? In three out of the four Encampments studied, some reduction in susceptibility occurred, but

Changes Created by the

Encampment

161

in only one, 1957, was the reduction sizable. In two Encampments, 1955 and 1958, California, there was little initial susceptibility to be reduced. In the latter Encampment the measured difference between the two groups actually increased between the start and the end of the Encampment. Weighing these results, we conclude that the Encampment program had no consistent effect on the campers' susceptibility to prestige symbols, as measured by this experimental device. Table 4-5 summarizes the data. W O U L D THERE B E IMPROVEMENT W I T H O U T A N E N C A M P M E N T ?

Earlier in the chapter the reader was promised that consideration would be given to the amount of change during the summer which could be attributed to factors other than the Encampment, such as natural maturation of sentiments, instability of youthful views or of the measuring instruments themselves, and extraneous events. The time has come to redeem that promise. First, it should be noted that the replicated findings over four independent tests of the effectiveness of the Encampment, 1955,1957,1958, New York, and 1958, California, provide a measure of all these factors except maturation. Instability of the responses of youth and instability arising from the instruments, by definition, should not work to produce changes in the same direction in the four tests. Rather, such instability should produce a pattern of change which is random in direction. For example, if instability accounted for the observed changes, the findings should go one way in 1955, perhaps the other way in 1957, and then reverse themselves again in 1958. Similarly, it is unreasonable to argue that extraneous events in three different calendar years should all work to produce changes in the same direction. One would hardly argue, for example, that the course of race relations in the United States took so even a course between 1955 and 1958 as uniformly to produce about the same magnitude of increased optimism among the campers in these different years.20 The phenomenon of growth or maturation, admittedly, should behave 20 Other classic procedures solve the problem of instability of measurement and obviate the need for controlling such factors by a control-group design. In using scales that have already been established in the literature, such as Riecken's, we know that they have a high repeat reliability. Riecken reports for political-economic conservatism a retest reliability of .81; for his democracy scale, a value of .73; for authoritarianism, a value of .78. These values were obtained from his work-camp participants, a group highly comparable to ours which can thus be generalized to our group. For technical reasons his retest reliabilities are probably conservatively stated. Similarly, the Stouffer tolerance scale, used in our inquiry in truncated form, seems unlikely to be subject to errors of measurement because it satisfies all the requirements of a Guttman scale and gives the same findings when used by two survey agencies on equivalent national samples. Op. cit., passim.

162

Applications

of Methods TABLE

of

Evaluation

4-6

CAMPERS' " N A T U R A L " IMPROVEMENT D U R I N G SIX W E E K S PRECEDING THE ENCAMPMENT, 1 9 5 5 AND 1957

Area of opinion

Scores fi weeks before Encampment

Scores at Start of Encampment

Amount of "natural" improvement

Action-apathy (median) 1955 1957

1.7 2.7

2.3

Little Little

2.5 2.5

None Some

1.5

Anomie (median) 1955 1957

2.4 3.6

Authoritarianism (median) 1955

39

42

None

Civil liberties " A " scale (median) 1957

4.7

3.3

Considerable

Civil rights Zero discrimination (per cent) 1955 1957

80 54

59 50

None None

Constitutionalism (median) 1955 1957

6.6 7.0

6.6 8.0

None None

Democracy (median) 1955

53

56

None

25

25

None

37 41

34 37

Little Little

67 31

74 42

Some Some

95

97

Little

Group stereotype (median) 1955

Political-economic conservatism (median) 1955 1957

Tolerance Permit peacetime Communist on radio (per cent) 1955 1957

"Protect the innocent" (per cent) 1955

Changes Created by the Encampment

163

according to the same process, no matter what calendar year is being investigated. All these groups of young persons were at about the same stage of development and subject to the same basic process of growth. Thus, though replications are a most important type of control-group design, they cannot provide an estimate of the influence of growth. Moreover, there is always the rare statistical possibility that instability would through mere accident produce a pattern of findings which is uniform in direction, and some protection against this "one chance in a million" is desirable. In addition, evidence on all the uncontrolled factors was desired in particular evaluations at the time they were conducted, that is, at the time of the 1955 study or of the 1957 study. Thus other types of controls were developed. One of these will be described now. In order to determine how much change there would be in campers' views under "natural" conditions a special procedure was introduced into the research design. About a third of the group were sent questionnaires approximately six weeks before they arrived at the Encampment. This third of the campers served as a control group on itself and on the rest of the campers in the following manner. Comparisons were made between the views expressed by campers on this mail questionnaire and their opinions at the start of the Encampment. The difference between views expressed at these two times provides a measure of the amount of fluctuation and stability which campers' opinions might be expected to show during a six-week period under ordinary conditions.21 By comparing this amount of "natural" improvement with the amount found under the special conditions of the Encampment it is possible to evaluate how much of the latter change might simply be due to non-Encampment factors. The residual change may be attributed to the impact of the Encampment. Table 4-6 presents the amount of change shown by the special control group of campers during the six weeks preceding the Encampment. It was not possible, of course, to measure all the important variables by the pre-Encampment mail questionnaire, but the items selected cover a range of areas relevant to the evaluation. What do these data reveal? It seems that change in campers' opinions during an ordinary six-week period in their lives resulting from all other major sources of instability is negligible. There are, to be sure, a few exceptions. But only one of these (civil liberties) is sizable, and the others show no consistency in their direction. For example, authori21 The assumptions implicit in this method and the procedure employed to safeguard the assumptions have already been, discussed in chap. i.

H Z

m a P

o

a a 'S-S o S n a

w

S

g

g

S

S m

S

s »

t-

+

++

"5

00 N

rt

o

o

oo

oo

o

t
o «

* a si

IO «3 + +

i l Bs

+

+

CO +

flS S S s I 0 C H&

t-

+

+

co +

r-

+

eo +

+

N

tO

+

8

O

S

i-i eo + i

» s

H Z O O m H O Z

m o

h o

t - 00 IO U3 OS OS

-g o

eo I

+

+

00 O

00

t - 00 « s OS OS ^ ^

^ -t? o PQ

•
> t.>. 3

M MS Sg Sg-jj « 22.3

a

a

S

.2.2 8 E & §

o

tS • • O .9 •

&

|z

-Jm • • • bCO 1Í5 b- 00 k

O

S ® » »

s, s i

O ¿«I ® S J r

©

fc «ci Va 05 Ü S «9 us io d - C i o "5 >ra 03 01 F OS OÍ Ol H H J «-t »-H

iî 03o 0) OQ IO 00 [3 1ÍJU51J5 ^ ~ OS OS OS ^„JrtrHrt

I l s S S I ^ °t- c& ® J3J3 « g g g S S o a a « V © t* jsjs o < ^ «

Dynamics of Change

173

difficulty following the program also improved—from 52 per cent scoring no discrimination at the start of the Encampment to 82 per cent at the end, a gain of 30 per cent. Similar, although not always consistent, changes occurred on other opinions and attitudes and in the other Encampments. Next, a comparison is made of the changes during each Encampment among campers who had received some college education before the Encampment and among those who had not. Presumably, if the didactic element of the Encampment is paramount, campers with less formal preparation (and therefore less initial knowledge and less experience with schooling) should change less. This did not happen (see table 5-2). Campers with less formal education still changed during the summer, and almost as much as those with a college background. To illustrate, in 1958, New York, 60 per cent of the college campers started with no discrimination against minority groups on civil rights; 87 per cent showed no discrimination at the close of the Encampment—a gain of 27 per cent. A t the same time, however, campers without a college background also changed, from 59 per cent scoring no discrimination at the start to 82 per cent at the close of the Encampment—a gain of 23 per cent. A more direct test of the impact of the didactic program is obtained by examining campers who were not exposed to the entire program. Admittedly, the educational program was an ongoing, complex process to which all campers were highly exposed. Nevertheless, given the generally permissive environment of the Encampment, some campers occasionally missed such specific educational events as a lecture, a discussion, a workshop, a field trip, a town meeting, or another scheduled educational event. Sometimes the absences represent voluntary defection; at other times they reflect factors beyond the control of the camper. Consequently, an index of defection from the didactic program is constructed from the campers' reports on the number of times they missed any of the various educational activities.* "Attendance records also were kept by the staff, at the request of the research team. Inspection of the campers' own reports, however, indicates that they are accurate enough for purposes of the index. The magnitudes reported seem reasonable in the light of other information; there is no "round-number" tendency; and very detailed comments suggestive of much thought were frequently inserted by campers answering the questionnaire. As the self-reports also covered absences from communal activities, and the staff's records did not, the former are used here. The index, as used here, assumes that different elements of the educational program are not substitutable for one another and that the total effect of the program cannot be achieved from merely one or a few units of exposure. Both assumptions seem warranted for the analysis of defections from didactic work. For the analysis involving the index of defection from communal life, presented later, the validity of the assumptions is more questionable.

.a O a ® « „ I P. a

s s

"g ja e

coeôci

*

c3 a l i t OS O.Ü

Oí t- 00 iH N (N

«•si

s

w «

0 "F

H

O ¡Z

o m ta "3 .2.2 © MMM 0 -»a ds

a a o O -E » S S Q) te S c £

ssg © © H Q. Q.+» ggS S S g o o g. ^ ® c« ® •s-s s

CS3

2 »

O" H

t - 00 icio g OÏOI H H H J Ph

Q,iC> m O Oí i - l i—I

• :

•

o > fe ©

fe o a © 03 ® •ï ® «

œ tn "5 ® • -tí . . ta£TO I O t"» 0 0 • g1 _ « 5 > g

lOlO ° 0 >O 0 l1 O 0 i) H

,2 ? "oH

:, tso

p£ Ol • r - oo io io ÖS CO OS

>• o

£

«a -

Ph O

:¡z¡

co > 2 t i OC IO IQ ÛJ C i OS O Î

Dynamics of Change

179

ment community. At other times, the prodidactic group changed more. Thus the evidence again supports the hypothesis that both the didactic and the communal features of the Encampment play an important role in the campers' attitudinal changes. Furthermore, the data suggest that the informal communal life is at least as important to the changes as is the didactic program.8 The mutual importance of both educational and communal activities in producing attitudinal changes is also underscored by Riecken in his study of the Friends' work camps. Admittedly, the work camps did not have an elaborate formal didactic program accompanying the communal life, but they did incorporate a variety of educational practices, some of which parallel those of the Encampment. Yet, in his analysis of the major channels by which ideology is transmitted, Riecken concludes that the active participation of the camper is almost essential to the process of change: [The ideology] . . . is transmitted in a fairly subtle fashion, and through actions and relationships to a much greater extent than through written or spoken "propaganda." Both kinds of influence come to bear upon the camper, however, and tend to reinforce each other. Most important of all, nearly all of the channels of ideological communication involve the active, conscious participation of the individual in the process of learning." T H E CONTRIBUTION OF OTHER FACTORS DISRUPTIVE EXPERIENCES AND CHANGES

In chapter iii a variety of disruptive experiences which campers might have encountered during the summer were described. Such disruptions were found to be uncommon, and were often limited to a few campers. As such disturbing events might inhibit the changes desired by the Encampment, either by interfering with the responsiveness to the didactic program or the communal life, or both, it seems plausible that one of the factors accounting for the Encampment's general effectiveness is the relatively undisturbed, pleasant atmosphere pervading the didactic and communal activities. By comparing campers who vary in their exposure to such disruptions, it is possible to test this factor. The extent to which the Encampment is successful in changing the attitudes 8 A similar analysis was made on the basis of campers' ratings of satisfaction with several aspects of the program, at the end of the Encampment. Using the data on satisfactions, prodidactic and procommunal types of campers were identified, and contrasts were drawn between the amount of change in attitudes shown by each. The findings are suggestive of the same conclusion as above, that both the prodidactic and the procommunal groups changed, and neither more consistently so than the other. • H , W. Eiecken, The Volunteer Work Camp: A Psychological Evaluation (Cambridge : Addison-Wesley Press, 1949), p. 77.

g s 0

s s o O o

z
-< ^

a

o

c o +c o

h

n+

c o - +m

- h +o

-h

c q+

7 * °

¿

"z

ì

II

O O

a

o

O

s II

o

¿

¿

+

+

«! O Z H

p< & o M O

1 ra

05

6

ë«

a •a ••3 S a

a 03 M

â

EH

S! O O

o

o

o Ph

o h

> Ö

o Eh

»

Sf

3 ü

S

»

o H

I o O

a O

Effects in the College Setting

237

in response to questions concerning the probable ideological character of the campers and staff members, the overwhelming majority of all four groups saw the Encampment as composed mainly of people holding liberal views on social problems. Again, all four groups were similar in their degree of interest in attending the Encampment: all reported work to be the principal alternative to going to the Encampment, and on the critical question of relative interest, all but one respondent in each group saw the Encampment as at least as interesting a way to spend the summer. This similarity is especially notable in light of the answers to a question (asked only in 1958) concerning the likelihood of their actually attending: the control respondents, it seems, viewed the Encampment as no less attractive than did the campers, even though most of them knew at the time that they would probably not in fact attend. Nor were there important differences in the features of the Encampment that attracted them: most members of all four groups said they looked forward to the community-living aspect of Encampment life,* and most wanted to spend a large part of their time in listening to lectures, attending discussion groups, studying, and serious reading, as opposed to informal conversations and bull sessions, personal exploration of the region around the Encampment, sports, and recreation. As critical, perhaps, to the character of the Encampment as the motivation to attend are the highly liberal attitudes with which the campers as a whole begin a summer. As table 8-3 suggests, no consistent differences separated the two groups, so far as general political attitudes are concerned: campers on the average were as liberal (or actionoriented), as measured by the political action scale; somewhat more liberal, as measured by the political-economic conservatism scale; and somewhat less liberal as measured by the civil liberties and tolerance scales. Thus, the control group respondents would probably have been at least as receptive (or, in some instances, as resistant) to the Encampment's program as the campers were. On the dependent variables in the present analysis, on the other hand, campers seem generally to be rather less liberal than the control-group respondents. The several tables in the later part of the chapter make this quite clear, but in order to illustrate the point, table 8-3 also contains three of the college scales: campers were as action-oriented, measured by the campus action-apathy scale, but they were somewhat less tolerant, as measured by the tolerance-of-campus-organizations scale, and somewhat less committed to a college education, as measured by the commit* Note, however, that the control respondents, on the average, looked forward less to the travel and big-city features than to the didactic program and the intellectual activity, while the reverse is true of the campers.

238

Applications

of Methods of

Evaluation

T A B L E 8-4 CONTROL G R O U P VERSTJS E N C A M P M E N T - E X P E R I M E N T A L G R O U P : S E X , A G E , M A J O R SUBJECT, AND OCCUPATIONAL GOAL Control group Characteristic

Sex Male Female Age 19 years and under 20 years and over M a j o r subject Business and e d u c a t i o n . . . Humanities N a t u r a l sciences Social sciences None Occupational goal Service-oriented Politically oriented Other Not stated

Encampment

1958 (N = 20)

Total (N = 35)

1957 (N = 20)

1958 (N = 30)

Total (N = 50)

7 8

12 8

19 16

6 14

11 19

17 33

11 4

9 11

20 15

9 11

15 15

24 26

2 7 2 4

3 4 4 9

5 11 6 13

6 2 4 8

10 12 3 3 2

16 14 7 11 2

5 5 5

8 3 8 1

13 8 13 1

15 1 3 1

16 2 10 2

31 3 13 3

1957 (N = 15)

ment-to-college scale. (These latter three scales are explained in detail below and in the Appendix on scales.) Here, too, the program would have found at least as receptive an audience among the control respondents as it found among the campers.7 So far as such characteristics as sex, age, major subject, and occupational goal are concerned, the groups are not particularly similar (table 8-4). Among the campers there are many more girls, a few more older students, more students majoring in business or education (mainly the 7 So f a r as the substantive problem of the Encampment's effectiveness is concerned, that the control respondents are as liberal as or more liberal than the campers is fortunate. For if its effectiveness depends in part on the initially high liberalism of campers, as we suspect it does, the even greater liberalism of the control respondents suggests that they would have reacted to the program much as the campers did. The same quality, the control group's greater liberalism, may, however, create measurement problems, so f a r as effectiveness is concerned: if by the scales the control respondents were very liberal, relative to both the campers and the upper reaches of scales themselves, the amount they could measurably change upward would be limited by "ceiling effects," which are artifacts of measuring instruments. The usual type of comparison between the two groups therefore could not be used to indicate the Encampment's effectiveness, f o r the comparison would be biased in the Encampment's favor. In general, the control group is not so liberal, or at least not so much more liberal, that this difficulty actually arises. Appendix C presents a fuller discussion of the problem of ceiling effects.

Effects in the College Setting

239

latter), and, correspondingly, more students aiming toward a serviceoriented career. The nature of the relationship between these characteristics and the impact of the Encampment's program is, however, not at all clear, and the possible significance of these differences is therefore difficult to assess. Differences in the summer's experiences.—The classic view of the control group pictures it as equivalent to the experimental group in all relevant respects except degree of exposure to the test factor. The preceding section has shown the extent to which the first condition is met for two of the more important "relevant respects"—interest in the Encampment and in social and political attitudes. Here it is necessary to assess the extent to which the second condition is met. The general problem has been well stated by Campbell. Traditionally, as he writes, the experimental-versus-control comparison has been described as being made between a group that received a treatment, X, and another group that received no treatment, No-X. He then comments: While this may be a legitimate description of the stimulus-isolated physical science laboratory, it can only be a convenient shorthand in the social sciences, for any No-X period will not be empty of potentially change-inducing stimuli. The experience of the control group might better be categorized as another type of X, a control experience, an X c instead of No-X.®

Applied to the situation at hand, Campbell's point is this: it is not enough to know that the control respondents did not attend the Encampments. Some, possibly, might have attended a Friends' work camp. In order to interpret the control-group findings in the traditional fashion, it is necessary to know that their experience in the interim of several months differed in relevant ways from the campers' experiences. So, it must be demonstrated that our control group did not receive a treatment from life similar in kind or degree to the treatment received by the Encampment group. Characteristics of the Encampment which seems basic to its effectiveness are (1) the removal of the campers from their usual social contexts and their relative isolation while at the Encampment from the wider society's influences; (2) their sustained participation in an educational program designed to alert them to a variety of social problems and to shape their attitudes toward these problems, as well as to impart information; and (3) their membership in a cohesive community whose members share a broadly liberal outlook on the social and political issues of the day. To what extent did the control-group respondents find them8 Donald T. Campbell, "Factors Relevant to the Validity of Experiments in Social Settings," Psychol. Bull. LIV (1957), 306.

240

Applications of Methods of Evaluation

selves in similar situations? For if they, too, are divorced from their usual associations, subject to an intensive educational program, and active in a highly cohesive community, they would not have received a different treatment—even though they did not go to the Encampment— and so could not be used to estimate the Encampment's effectiveness over and above "normal" experience. The control group's histories were obtained at Wave IV. As the following data suggest, the campers and the control-group respondents did in fact receive quite different "treatments" in the course of the summer, and in consequence it is legitimate to compare them in order to gauge the Encampment's effects: Whereas all 50 campers were away from home for at least the six-week period, 25 of the 35 control-group members (11 in 1957 and 14 in 1958) spent most of the summer at home. All 50 campers were engaged in an educational program, but only 2 control respondents in 1957 and 3 in 1958 were in an educational program of any sort (all 5 were in summer school); instead, 14 in 1957 and 16 in 1958 spent the summer working. When asked about the most important experience of their summer, 15 campers in 1957 and 24 in 1958, mentioned something that reflected their having lived closely with others within a definite community; only 2 control respondents in 1957 and none in 1958 gave answers reflecting such an experience. Finally, the campers found themselves in a politically more congenial environment. After the summer all were asked whether they felt themselves to be in agreement with the views held by their associates during the summer on matters of civil liberties, economic planning, urgency of social reform, and the importance of politics. Of the 50 campers, only 6 in 1957 and 3 in 1958 felt themselves to be in "very little" agreement with most other campers on one or more of these topics; whereas, of the 35 control-group students, 8 in 1957 and 11 in 1958 felt themselves to be in "very little" agreement with their summer friends on one or more of these topics. Otherwise put, 5 campers in 1957 and 10 in 1958 felt in "very great" agreement with most other campers' views on four or five of these items, but only 3 control students in 1957 and 1 in 1958 felt this way.

It is, we believe, gross differences of this sort which account for the Encampment's relative effectiveness in changing those attitudes and opinions that it aims especially to alter. The camper, in contrast to the control-group respondent, is drawn into and contributes to a definite community, and in the process his attitudes are modified in the direction of the opinions that are salient in that community. Thus, owing to the Encampment's emphasis on tolerance and on participating in organizations and in discussions with one's friends, it should have an effect on the camper's participation in the political life of his campus. The exact same reasoning, however, suggests that little or no change will occur in

Effects in the College Setting

241

those opinions and attitudes of the campers which are not salient in the day-to-day activities at the Encampment or not directly relevant to the community's sentiments; this seems to be true of attitudes toward a college education and academic work. T H E FINDINGS

The findings are presented under the three headings of "Campus political life," "Academic life," and "The value of a college education," in that order. For, as one moves from the campers' political attitudes and activities and toward the values they place on college education, he moves progressively away from the kinds of norms and values stressed in the Encampment's program and continually made salient by its daily routine, and approaches a realm of norms and values which the Encampment would affect only by some subtle and diffuse process. Consequently, the greatest changes should occur in the campers' political life, and the least changes in their general views on college education. For the most part, this expectation is borne out: the campers became somewhat more activity-oriented, somewhat more willing to have diverse political views expressed on campus (tolerance), and slightly more influential in their relations with friends; in the specifically academic side of college life, only their attitude toward the handling of controversial issues in the classroom seems to have been affected, and this but slightly; in the sphere of values proper no change of any magnitude or consistency took place. CAMPUS POLITICAL LIFE

Three aspects of campus political life were examined: interest and actual participation in campus activities of various sorts, but especially the political; tolerance as exemplified by views of the campus as a place where a variety of political and social beliefs may be expressed; and participation in political discussions with friends. Changes occurred in all three, indicating that the Encampment has an effect in this general area, but the magnitude of the changes is uniformly small. Against this, however, are the findings from the control-group respondents, which show that in a corresponding group of young persons negative changes (judged from the perspective of the Encampment) are not unusual. The Encampment's total effect in this realm thus comprises not only the small positive changes, but also the prevention of negative changes. Nevertheless, in comparison with the previous findings, the effects here are uniformly small. Participation.—The disposition to take part in campus activities is

242

Applications

of Methods

of

Evaluation

measured by a battery of questions similar to that used for the political action-apathy scale, but focused specifically on campus life. Called the campus action-apathy scale, it extends from 0, which indicates an active orientation, to 10, which indicates a relatively apathetic orientation. In each of the three Encampments—1957 and 1958, New York, and 1958, California—a small but persistent change to a lower average score took place, the median dropping from about 2.5 to about 2.0 in all three groups. In table 8-5, this cleavage is shown by the increases in the numbers scoring 0. Though small, the consistency of the findings over the three groups suggests that an actual change probably occurred; and TABLE

8-5

PARTICIPATION Group»

N

Before b

After

Campus apathy scale: number scoring 0

NY-57 NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

13 8 3 15 13

17 11 7 22 11

+4 +3 +4 +7 -2

Less apathy Less apathy Less apathy Less apathy More apathy

Interest in politics: choices for speakers on political topics (per cent)

NY-57 NY-58 Cal-58 EFC-exp Control

e

76 76 78 70 80

75 81 82 79 71

-1 +5 +4 +9 -9

Less interest More interest More interest More interest Less interest

Activities engaged in: average number of types of activity per person

NY-57

50

4.4

3.9

-

NY-58 Cal-58

30 17

4.1 3.6

4.1 3.1

-

.5

EFC-exp

50

4.2

3.9

-

.3

Control

35

4.1

3.7

-

.4

Variable

Diff.

.5 0

Direction of change

Less participation No change Less participation Less participation Less participation

• T h e five g r o u p s are: NY-57: New Y o r k E n c a m p m e n t , 1957. NY-58: New York E n c a m p m e n t , 1958. Cal-58: California E n c a m p m e n t , 1958. E F C - e x p : E n c a m p m e n t - e x p e r i m e n t a l , combining campers f r o m 1957 a n d 1958 w h o were on Waves I (spring) a n d IV (fall). C o n t r o l : Control group, combining control respondents f r o m 1957 a n d 1958. b T h e " b e f o r e " scores for each of t h e t h r e e E n c a m p m e n t s were secured on Wave I I , a d m i n i s t e r e d on t h e first d a y of t h e E n c a m p m e n t . T h e " b e f o r e " scores for t h e E n c a m p m e n t - e x p e r i m e n t a l a n d control groups were secured on Wave I , a d m i n i s t e r e d in t h e spring before t h e E n c a m p m e n t while t h e s t u d e n t s were still in college. 0 T h e base here is t h e total n u m b e r of choices. E a c h r e s p o n d e n t was t o m a k e t h r e e choices, b u t i n some instances t h e r e were a few w h o m a d e only one or t w o . T h e " b a s e " t h u s varies between t h e " b e f o r e " a n d " a f t e r " measures t o a slight extent. F o r NY-57 i t is 148 a n d 147 on t h e before a n d after scores, respectively; for NY-58 i t is 83 a n d 88; for Cal-58 i t is 45 a n d 45; for E F C - e x p it is 144 a n d 150; a n d for Control i t is 105 a n d 102.

Effects in the College Setting

243

as the median of the control group's scores was virtually constant (from 2.2 to 2.1), the change probably reflects the Encampment's influence. Regarding table 8-5, and subsequent tables in this chapter as well, several comments are in order. On the far left is the dependent variable and the particular measure (campus action-apathy scale; number scoring 0). Next come the five sets of respondents used in the analysis, the three Encampments (1 in 1957 and 2 in 1958), and the Encampmentexperimental and control groups. Changes for each group—the difference between the figures in the "before" column and those in the "after" column—are labeled "Diff." The last column simply states the direction of the differences with respect to the meaning of the variable, for sometimes a negative numerical difference indicates an improvement from the Encampment's point of view. Note that, in contrast to most other tables in the monograph, those in this chapter contain actual numbers instead of percentages or even medians, owing to the relatively small size of the base figures (50, 30, and 17 for the three Encampments, and 50 and 35 for the Encampment-experimental and control groups.) Also, whereas changes within the first three groups are from Wave I I (beginning of Encampment) to Wave IV (fall), changes within the last two groups are from Wave I (spring) to Wave IV, for reasons given earlier, and each of the two latter groups includes respondents from both 1957 and 1958 because the patterns of change for both years are similar on most items. (That the base figure is 50 for both the 1957 Encampment and the Encampment-experimental group is sheer coincidence.) Finally, in these tables the patterns of the differences over the five groups are of most interest. Consider, for example, the number scoring 0 on the campus action-apathy scale: in all three Encampments more campers scored 0 on Wave IV than on Wave I I ; moreover, a similar change took place in the Encampment-experimental group from Wave I to Wave IV; however, for the same time period (Wave I to Wave IY) a small decrease occurred in the number scoring 0 within the control group. All differences thus point to the Encampments' having reduced the campers' apathy toward student activities. Interest in politics proper was measured indirectly by asking campers to choose from a list of seventeen speakers and topics the three speakers they "would be most interested in having speak on the topic listed" on their campuses. Three of the speakers were listed with topics that were neither "serious" nor political (e.g., Emily Post on etiquette in dating relationships), four were listed with serious but nonpolitical topics (e.g., Bruno Walter on similarities in the music of Bach and Mozart), and the remaining ten were listed with political topics (e.g., William Knowland

244

Applications of Methods of Evaluation

on the United Nations and national sovereignty). If the campers were neither specifically interested in nor opposed to political speakers, they would choose them, on the average, 10 out of 17 times; that is, about 60 per cent of their choices would be given to one or another of the ten speakers on political topics. Similarly, if their interest in politics increased or decreased markedly, the change should be reflected in the distribution of their choices over the speakers. In all three Encampments they began the summer with an "overchoice" of political speakers; about 75 per cent of the choices were among the ten listed. (See table 8-5.) Minor increases occurred in 1958 and none in 1957. One would have to conclude that the Encampment has little effect, were it not for the control-group findings: whereas the Encampment-experimental group increased the proportion of its choices of political speakers by about 10 per cent, the control group decreased its proportion by a like amount. At best, however, the effect of the Encampment on political interest, as measured by these answers, is not very large. Actual participation was measured as follows. Campers were asked to check on a list of eleven types of student activities (e.g., musical and theater groups, student government committees, team sports) those in which they were taking part. These checks were totaled to give each camper a participation score. This score, it should be remarked, is not the actual number of memberships a camper had; it is, rather, the number of types of activity in which he engaged. I t scores him on both his range of participation and his amount. Although a students' number of memberships is probably not much larger than his number of activities, the index properly stands for the latter, not the former. By this score the degree of actual participation, in contrast to the disposition to take part, was less in the fall than it had been in the preceding spring for the 1957, New York, and the 1958, California, campers and the same at both times for the 1958, New York, campers. I t seems from the control-group data, however, that it is "normal" for student participation to be higher late in the second semester (spring) than early in the first (fall), and so the Encampment is probably not responsible for the drop-off." " It might be noted here that on a question concerning the two most preferred types of activity, those most often named, on the average, were musical and theater groups and student government committees. Next in order were political and public affairs groups; campus religious groups; departmental or preprofessional clubs; and publications (including campus radio stations). Least popular were social clubs, dance committees, and the like; language, literature, and cultural societies; intercollegiate organizations; debating societies; and team sports—in that order. Various changes occurred in the preferences, but none that could be construed as reflecting explicit Encampment aims. As far as general political activities are concerned, the number of mentions for student government committees fell off in all three Encamp-

Effects in the College Setting

245

Tolerance on the campus.—Two general questions were asked about the campers' views on the college campus as a market place of ideas. In one set of questions the camper checked for each of seven student political groups whether it ought or ought not to be allowed on campus, or that he was not sure, or that he did not know enough about the organization to decide. These groups ranged from the right-wing Students for America through the Young Democratic Club to the left-wing TABLE

8-6

TOLERANCE ON THE CAMPUS Variable

Group®

Tolerance of campus organizations scale: number scoring 11 or more

NY-57 NY-58 Cal-58 EFC-exp Control

Interest in diverse political views: choices for "conservative" political speakers (per cent)

NY-57 NY-58 Cal-58 EFC-exp Control

Before b

After

50 30 17 30° 20°

23 16 8 11 10

d

35 41 26 29 36

N

Diff.

Direction of change

29 21 13 20 9

+6 +5 +5 +9 -1

More tolerant More tolerant More tolerant More tolerant Less tolerant

36 41 30 38 19

+1 0 +4 +9 -17

More interest No change More interest More interest Less interest

* See note a, t a b l e 8-5. See n o t e b, t a b l e 8-5. T h e questions in t h i s scale were n o t asked on Wave I (spring), 1957. T h e base here is t h e t o t a l n u m b e r of choices for political speakers, a n d so varies f r o m Wave t o Wave as well as f r o m group t o group. F o r NY-57 i t is 112 a n d 111 on Waves I I a n d IV, respectively; for NY-58 i t is S3 a n d 71; for Cal-58 i t is 35 a n d 37; for E F C - e x p (Waves I a n d IV) i t is 102 a n d 118; a n d for Control (Waves I a n d IV) i t is 84 a n d 72. b 0 d

Young Socialist and Young Communist leagues. The answers were combined to form a "tolerance-of-campus-political-organizations" scale by scoring 2 for each "allowed," 0 for each "not allowed," and 1 for anyother response. This scale ranges in principle, then, from a low of 0 (least tolerant) to a high of 14 (most tolerant), with the actual ranges in the different years from about 5 to 14. By this measure a slight but definite increase in tolerance occurred among all three groups of college campers, as shown in table 8-6. Again, the comparison between the Encampment-experimental group and the control group suggests that an actual change took place and that it is an effect of the Encampment's program. ments; and although expressed interest in political and public affairs groups increased in both 1958 contingents, it decreased in 1957. Interest in campus religious groups decreased in all three; and interest in departmental and preprof essional clubs increased markedly in all three instances.

246

Applications

of Methods

of

Evaluation

An analysis of responses to the individual organizations reveals that, although small increases occurred in the numbers who would allow the right-wing and center organizations to exist on campus, the principal changes occurred in the responses to the two left-wing organizations. The results for those who would allow these two organizations are presented in table 8-7; a corresponding reduction in the numbers who would not allow them also took place. The change reported in table 8-7, it should be pointed out, reflects not TABLE

8-7

TOLERANCE OF L E F T - W I N G C A M P U S POLITICAL G R O U P S

Variable

Group»

N

Before1»

After

Diff.

Number allowing Young Communist League

NY-57 NY-58 Cal-58 EFC-exp0 Control0

50 30 17 30 20

10 5 4 5 6

13 8 8 11 6

+3 +3 +4 +6 0

Number allowing Young Socialist League

NY-57 NY-58 Cal-58 EFC-exp» Control0

50 30 17 30 20

15 10 7 11 11

20 15 10 16 12

+5 +5 +3 +5 +1

» See note a, table 8-5. See note b, table 8-5. • Responses are for the 1058 experimental and control subjects only.

b

some particular ideological impact of the Encampment, but rather its effect on the range of views tolerated on the campus. It happens that, by this measure, the broadening occurs mainly on the "left" side of the political continuum. But this result appears to be a fault of the measure, as the right-wing group seemingly had considerably lower visibility than the left-wing groups. About half of the respondents in each Encampment, both before and after, said of the Students for America organization that they did not know enough about it to decide whether it should or should not be allowed on campus. The next measure of tolerance is more sensitive in this respect—"liberal" and "conservative" alternatives seem to be equally well known—and it shows that the campers had no particular preferences in one direction or the other, either before the Encampment or after. The question about campus speakers may be used to measure not only interest in political versus nonpolitical subjects—as it was—but also the balance in the choices between "liberal" political speakers and "con-

Effects in the College Setting

247

servative" political speakers. If the campers were interested in hearing both points of view expressed on campus, we would expect them to distribute their choices between the two types of speakers approximately in accordance with number of each type. As there are six "liberal" speakers and four "conservative" speakers, the political choices should be split about 3 to 2.50 As it happens, the campers began the summer with their choices split about in this way, and little or no change occurred; the change that does occur moves the group in the direction of a more even split. (See table 8-6.) Once again, if the direction of change exhibited by the control group is an indication of what would have taken place in the absence of the Encampment, the Encampment's effect here is to encourage campers to want to hear other than only "liberal" points of view. The third area of interest is the camper's informal political behavior: whether he talks politics with his friends, the frequency with which he has serious discussions with them, and the part he plays in these discussions. In order to estimate the frequency of discussing politics informally, the campers were asked the simple question, "Have you discussed political issues with anyone in the past week?" In all three groups, more answered "yes" after the Encampment than before. It should be pointed out that this difference is not owing to an increase in the frequency with which they discussed any serious matters informally, because the amount of time they estimated spending in "serious discussion with other students" actually shows a decrease. As is recorded in table 8-8, the control group shows an equal drop in the hours spent discussing serious matters with other students, but, in contrast to the Encampment-experimental group, it has a smaller number having discussed politics on the fall wave than on the spring wave. Moreover, in these political discussions the campers as a whole came to take a more active part. In 1957, though not in 1958, an increase occurred in the number taking at least an "equal share" in these conversations. In all three groups the opinion-leader question elicited more leadership replies in the fall than it had in the spring. These changes in the campers' role in discussion, like most of the behavioral effects of 10 The six speakers classified for present purposes as "liberal" are: Balph Bunche on Arab-Israeli relations, Eleanor Roosevelt on women in politics, Earl Browder on a defense of communism, Walter Reuther on forming a new labor party. Dean Acheson on recognizing the People's Republic of China, and "William O. Douglas on desegregation and the Supreme Court. The four classified as "conservative" are: William Jenner on the Communist threat, William O. Eastland on maintaining segregated schools, William Knowland on the United Nations and national sovereignty, and Francis J. Spellman on Roman Catholic views of church-state relations.

Applications

248

of Methods TABLE

of

Evaluation

8-8

INTERPERSONAL RELATIONS N

Before b

After

Diff.

Direction of change

Number discussing NY-57 politics in past week NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

36 21 11 36 33

41 25 12 42 30

+ 5

+4 +1 +6 -3

More discussion More discussion More discussion More discussion Less discussion

Number spending average of ten or more hours per week in serious discussion with other students

NY-57 NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

16 7 11 16 15

9

7

-7 -2 +1 -7 -8

Less discussion Less discussion More discussion Less discussion Less discussion

Number taking part in discussions as often as or more often than others

NY-57

50

31

40

+9

NY-58

30

19

21

+2

Cal-58 EFC-exp

17 50

9

9

33

34

0 +1

Control

35

28

27

-1

More participation More participation No change More participation Less participation

NY-57

50

8°

16

+8

NY-58

30

5

12

+7

Cal-58

17

3

5

+2

EFC-exp

50

10°

18

+8

Control

35

15c

17

+2

Variable

Number saying their opinions are sought more than are the opinions of others

Group®

5 12 9

More participation More participation More participation More participation More participation

a See note a, t a b l e 8-5. b See note b, table 8-5. ° See chap, viii, n . 11, for t h e t i m e a t which these d a t a were collected.

the Encampment, are not large; but they are large enough in comparison with the control-group data for us to infer that the Encampment had a definite, if small, impact." In general, the results of this analysis support the hypothesis that, owing to the relevance of its norms to this part of college life, the Encompment should have a definite but not an especially large effect. Comu In 1957, the question, concerning whether one's opinions were asked for more or less often than others' opinions was not on Waves I or II. Instead, on Wave IV

Effects in the College TABLE

Setting

249

8-9

OPINIONS ON FREQUENCY OF DISCUSSION OF CONTROVERSIAL I S S U E S IN THE CLASSROOM COMPARISON OF COLLEGE PROFESSORS AND 1 9 5 8 COLLEGE CAMPERS Campers (N - 47)

Professors» (N = 2,451)

P r e f e r r e d f r e q u e n c y of discussion

After

Before

Frequently Honestly, but only when issues arise Avoided if possible D o n ' t know or no answer

27 1 4

Total

6

79% 21 0 0

100%

100%

70% 24 0

68%

100%

COMPARISON OF 1 9 5 8 ENCAMPMENT-EXPERIMENTAL GROUP AND CONTROL GROUP

Preferred frequency of discussion

Frequently Honestly when relevant. . As little as possible N o answer Total

EFC-exp

Control

Wave I

Wave IV

Diff.

Wave I

Wave IV

Diff.

19 11 0 0

24 6 0 0

+5 -5 0

16 4 0 0

11 8 0 1

—5 +4 0

30

30

20

20

a P a u l F . Lazarsfeld a n d Wagner Thielens, J r . , The Academic Mini: Crisis (Glencoe, 111.: F r e e Press, 1958), p . 135.

Social Scientista

in a Time

oj

there were two questions, one retrospective—it asked campers to "think back to last year"—and one current. The data for the regular college panel in 1957 and f o r the Encampment-experimental and control groups, therefore, are in response to retrospective questions and thus subject to the bias of memory factors. I n order to have some estimate of the seriousness of the distortion, both the retrospective and the current questions were asked on Wave IV, 1958. l a the following table the Wave I I responses are compared with the Wave I V retrospective responses f o r t h a t year. Evidently, the distortion introduced through using the retrospective question alone is negligible in this instance. Wave 11-1958 Wave IV-1958 Are you asked Last year in college, your opinions . . . ? were you asked . . . ? More than average person As much as average person Less than average person Total No answer

8 34 5

8 31 5

47 0

44 3

250

Applications

of Methods of Evaluation

pared as to their attitudes before the Encampment, or compared with control-group respondents before and after the summer, campers were favorably influenced in their attitudes toward taking part in campus political activities, their interest in political affairs, the range of viewpoints they were willing to have expressed on campus (tolerance), and the influence they exerted in informal discussions of politics. As will now be shown, however, on the other aspects of college life which were studied, aspects for which its program has considerably less pertinence, the Encampment's impact is virtually negligible. ACADEMIC LIFE

The second topic examined here is the effect of the Encampment on the academic attitudes and actions of the campers. The findings lead us to conclude that there were no effects of any consequence, with perhaps one exception. This exception concerns the handling of controversial issues in classes covering social problems or public affairs. In 1955 some 2,400 social scientists at 165 colleges were interviewed in an effort to assess the impact of McCarthyism on the teaching of the social sciences." Among the questions asked in that study was one about discussing controversial matters in the classroom, and, in modified form, it was also asked in the 1958 phase of the present study." Table 8-9 presents the findings for 13 The principal results of this reasearch are reported in Paul F. Lazarsfeld and Wagner Thielens, Jr., The Academic Mind: Social Scientists in a Time of Crisis (Glencoe, 111.: Free Press, 1958); see pp. 134-140 for the matter under discussion here. 18 Below are the two versions of the question: Original version Encampment study Some claim there hardly exists an area in For college courses that deal with social the social sciences which does not lend problems or public affairs, which of the itself to value judgments—that is, sub- following ways of dealing with the subject to difference of opinion. Now, in ject matter do you think is right? general, for the courses you teach, which emphasis would you lean to?

Such controversial matters should be discussed frequently in undergraduate teaching because of the educational value of such discussions.

Controversial matters should be discussed frequently because of the educational value of such discussion,

One should answer such questions when they come up, but not seek out such discussions.

Controversial matters should be discussed honestly when they come up, but the teacher should not seek out discussion.

In times like these, it is better to avoid the discussion of controversial issues as much as possible.

Both students and teachers should in general try to avoid discussing controversial issues because such discussions have little educational value.

Effects in the College Setting

251

both the college professors and the 1958 college-panel campers. The correspondence between the professors' answers and the campers' is, to say the least, striking. If faculty members are assumed to set the norms about such matters on campuses, that the Encampment succeeded in producing even a slight improvement over the average position of such an authoritative body is noteworthy. Moreover, when seen in the light of the negative change on the part of the control-group respondents, the total effect becomes somewhat more than slight (see table 8-9). In general, however, the Encampment has no impact on the college student's academic life. Campers were asked how many hours they spent in class and in studying during an average week, how much of this time was "used in doing things that do not seem important," and whether most of what they were learning in college was "very worth while." They were also asked how often they cut classes and whether "most of the rules and regulations" at college seemed "useless" to them. Except for the answers concerning rules and regulations, none of the changes was large or consistent over the three Encampments. The implication of the exception is that over the summer the campers became rather more critical of restrictions. As comparable data from the control group were not collected, it is not possible to judge whether this is a "normal" change from spring to fall or one specifically attributable to the Encampment. The findings are given in table 8-10. THE VALUE OF A COLLEGE EDUCATION

The area of values is here divided into two parts: the camper's personal commitment to the goal of a college education, and his general evaluation of college education. Personal commitment.—The camper's personal commitment was assessed both directly and indirectly. In the latter procedure ten conditions were listed under which a college student might leave college. Campers were asked to say, for each, whether it would or would not be a reasonable ground for leaving, or whether they are not sure about it. Presumably, if a person's commitment were high, he would consider few of the reasons for leaving valid; if his commitment were low, he would consider a number of them valid. The statements are listed below. As a matter of interest they are arranged not as they appear on the questionnaire, but according to their "reasonableness" to the students after the Encampment.

252

Applications

of Methods of

Evaluation

T A B L E 8-10 ACADEMIC L I F E Variable

Group®

N

Before b After

Diff.

Direction of change

Number estimatimg they spend an average of forty or more hours per week in class or studying

NY-57 NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

29 13 9 25 18

27 14 8 24 21

—2 +1 -1 -1 +3

Less time spent More time spent Less time spent Less time spent More time spent

Number saying that "a lot" or "some" of their class or study time is used to do unimportant things0

NY-57 NY-58 Cal-58

50 30 17

19 15 8

25 13 8

+6 -2 0

More critical Less critical No change

Number agreeing that most of what they learn in college is "very worth while" 0

NY-57 NY-58 Cal-58

50 30 17

42 23 15

43 24 13

+1 +1 -2

Less critical Less critical More critical

Number saying that most rules at college seem "useless" 0

NY-57 NY-58 Cal-58

50 30 17

7 3 2

11 6 5

+4 +3 +3

More critical More critical More critical

Number estimating they cut on the average no more than three classes per month

NY-57 NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

32 23 12 31 16

38 23 13 36 17

+6 0 +1 +5 +1

More attendance No change More attendance More attendance More attendance

Number saying they cut classes less often than most other students they know0

NY-57 NY-58 Cal-58

50 30 17

30 25 12

32 23 12

+2 -2 0

More attendance Less attendance No change

See note a, table 8-5. See note b, table 8-5. These questions were not asked on Wave I (spring), and therefore no comparison is made between the EFC-experimental and control groups in this instance. B b 0

Effects in the College Setting

253

Ground for leaving college: (Generally considered "reasonable") Is needed to help support his (her) family Is bored by studying, but finds real satisfaction in working with his hands The expense of going to college is too great (Generally considered less "reasonable") Considers his (her) grades too low to warrant continuing in college Is offered a well-paying position in. a reputable company having a good promotion policy Wants to get married and raise a family (Generally considered least "reasonable") Prefers to work full time in order to be financially independent of parents Parents object that college is doing student no good Wants a job helping people who have problems and cannot see how spending two or three more years in studying will be of much use Feels the wide world provides a better education than do colleges

Two measures were constructed from the answers given to this question. Both suggest that a very slight increase in commitment may have occurred among the 1957 and 1958, New York, college campers, but not among the 1958, California, college campers. One measure is the average number of approvals (or disapprovals) per respondent. As it happens, no change in the mean number of disapprovals occurred in any Encampment. But in the two New York groups the average number of approvals declined from 2.9 and 2.6 to 2.4 and 1.9; in California, however, it increased from 2.7 to 3.1. The other measure was arrived at by scoring campers 2 for each "not reasonable," 1 for each "it depends" (or no answer), and 0 for each "reasonable," and combining the results into a single score. Total scores on this commitment scale thus range theoretically from 0 (low commitment) to 20 (high commitment), the actual range being from about 3 to about 17. I n the two New York Encampments the median score moved up slightly, from 10.2 to 11.9 in 1957 and from 11.4 to 12.5 in 1958, in the direction of a somewhat greater commitment. The median for the 1958 California Encampment dropped from 11.9 to 11.5. In table 8-11 these results are reported in terms of the number of respondents scoring 11 or more on this scale. As this table shows, an improvement occurred within the control group also, and so, although some increase in commitment may have occurred over the summer, it probably is not owing to the Encampment's program. (In the control group the number of approvals dropped, from 2.6 to 2.2, and the median score on the commitment scale went up slightly, from 12.2 to 12.9.) The direct assessment of commitment took the form of three nega-

254

Applications

of Methods

of

Evaluation

tively phrased questions concerning the campers' feelings about being at college and one positively phrased question concerning their plans for attending graduate or professional school. Except in 1957, where the number who never thought of taking a few years off from college dropped by a third between the beginning of summer and the fall, no TABLE

8-11

COMMITMENT TO A COLLEGE EDUCATION: COMMITMENT SCALE SCORES

Variable

Average number of items per respondent said to be' 'reasonable" grounds for leaving college

Number scoring 11 or more on the commitment scale

N

Before b

After

Diff.

Direction of change

NY-57

50

2.9

2.4

-.5

NY-58

30

2.6

1.9

-.7

Cal-58

17

2.7

3.1

+ .4

EFC-exp

50

2.8

2.1

-.7

Control

35

2.6

2.2

-.4

Increased commitment Increased commitment Decreased commitment Increased commitment Increased commitment

NY-57

50

17

28

+11

NY-58

30

16

18

+2

Cal-58

17

10

9

-1

EFC-exp

50

28

27

-1

Control

35

21

25

+4

Group»

Increased commitment Increased commitment Decreased commitment Decreased commitment Increased commitment

•b See note a, table 8-5. See note b, table 8-5.

sizable change occurred in the answers to these questions, nor were there any consistent trends, as table 8-12 shows. A very slight indication of an increase in commitment occurred in answer to the question, "Do you ever feel that what you are doing at your college is a waste of time? (Yes, often; Sometimes, but not often; Rarely; Never)," but the same slight indication shows up among the control-group respondents. The conclusion here, then, must be the same as that about academic life: the Encampment has little or no systematic impact. It neither in-

Effects in the College TABLE

255

Setting

8-12

COMMITMENT TO A COLLEGE EDUCATION: VARIOUS SINGLE INDICATOBS Variable

Number saying college is " r a r e l y " or "never" a waste of time

Number saying they "never" thought of taking a few years off from college

Number saying they "never" thought of leaving college altogether

Number saying they "definitely" plan to go to graduate or professional school

Group»

N

Before!» After

Direction of change

Increased commitment No change Increased commitment Increased commitment Increased commitment

NY-57

50

30

32

+2

NY-58 Cal-58

30 17

17 7

17 9

0 +2

EFC-exp

30°

12

15

+3

Control

20°

11

15

+4

NY-57

50

35

24

-11

NY-58 Cal-58

30 17

19 9

19 8

0 -1

EFC-exp

50

31

28

-3

Control

35

21

23

+2

NY-57

50

37

35

-2

NY-58

30

22

19

-3

Cal-58 EFC-exp Control

17 50 35

12 36 30

12 36 26

0 0 -4

NY-57

50

28

30

+2

NY-58

30

13

14

+1

Cal-58

17

8

7

-1

EFC-exp

50

25

29

+4

Control

35

22

21

-1

» See note a, table 8-5. See note b, table 8-5. 0 This question was not asked on Wave I (spring), 1957.

b

Diff.

Decreased commitment No change Decreased commitment Decreased commitment Increased commitment Decreased commitment Decreased comment No change No change Decreased commitment Increased commitment Increased commitment Decreased commitment Increased commitment Decreased commitment

256

Applications of Methods of Evaluation

creases to a measurable extent the campers' personal commitment to securing a college education, nor does it decrease it." Evaluation of a college education.—Two aspects of the general value the campers attributed to a college education were examined. One is the degree of value they place on a college education compared to other kinds of educational experience. The other is the particular features of college which the campers held to be most valuable. As for the first, the degree of value, the answers to the following four questions were used to indicate whether any changes had occurred: About how much schooling do you think most young men need these days to get along well in the world? What about young women—about how much schooling do you think most of them need these days to get along well in the world? I think college does not really equip you for life outside the campus. (Agree, Disagree, Not sure.) College education does more to break down values than to build up ideals.15 (Agree, Disagree, Not sure.)

Before a summary of the results is presented, it should be noted that for three of these items, plus a fourth one, data are available from other studies, and so it is possible to assess whether campers are as unusual in their attitudes toward college as they are in their social and political views. The question about the amount of schooling needed for getting along well in the world was asked by the National Opinion Research Center in 1947 in a nation-wide survey which included 450 young persons (aged 14-20). Of course, very substantial changes have occurred in the intervening decade in the public's attitude toward college education, and a national survey today would probably report somewhat different results. For what they are worth, however, the 1947 data are presented in table 8-13, along with the responses of the college-panel campers. The two questions on college education itself, plus a third used in the section on personal commitment, come from the Cornell Values Survey, a study of 4,585 undergraduate men at eleven institutions of higher learning, and the campers' responses to these may be compared with the responses obtained in that survey. As table 8-13 shows, the 14 Some suggestion that the Encampment has a mixed effect over the whole group occurs in the data, with some campers becoming more committed and others less so. But the evidence is slight; even if it were much more substantial, it still would not alter the conclusion that the Encampment has no systematic impact either one way or the other. 15 It seems reasonable to suppose that for these idealistic young persons a college education would not be valued if they thought it had mainly a negative effect on beliefs.

257

Effects in the College Setting

campers are rather more favorably disposed toward college than either the 1947 sample of young persons or the male undergraduates in the Cornell study, but the differences are not nearly so great as the differences between campers and other groups of young persons on political matters.1* The changes over the summer on the four questions listed earlier are TABLE

8-13

COMPARISON OP CAMPERS' AND OTHER YOTJNG P E R S O N S ' ATTITUDES TOWARD COLLEGE

Question or s t a t e m e n t

About how much schooling do you think most young men need these days to get along well in the world? Per cent saying at least "some college"

1947 national survey® (N = 447)

Students at 11 schools1» (N = 4,585)

Encampment college panel (N = 97)

79

63

I think college does not really equip you for life outside the campus. Per cent agreeing

18

10

College education does more to break down values than to build up ideals. Per cent agreeing

15

9

Most of what I am learning in college is very worth while. Per cent agreeing

71

82

» D a t a f r o m H e r b e r t H . H y m a n , " T h e Value Systems of D i f f e r e n t Classes: A Social Psychological C o n t r i b u t i o n t o t h e Analysis of Stratification," in R e i n h a r d B e n d i x a n d S e y m o u r Martin Lipset, eds.. Class, Status, and Power (Glencoe, HI.: Free Press, 1953), p p . 426-442. T h e figure of 63 per cent is calculated f r o m t h e d a t a presented in T a b l e IV, p . 432. b D a t a f r o m P h i l i p E . Jacobs, Changing Values in College (New Y o r k : H a r p e r , 1957), p . 34.

presented in table 8-14. In all three Encampments most campers began the summer feeling that most young men and women need at least some college education "to get along well in the world," and they felt the same some four months later. In 1957 and in 1958, California, there were a few more feeling this way after the Encampment, but the opposite was true of the 1958, New York, group. Again, in all three groups, most initially disagreed with the statement that college does not equip one for the outside world, and most disagreed with it after the Encampment, although somewhat fewer disagreed in the fall. The same is true for the statement concerning the effects of college on ideals. Clearly, the 18

See chap. ii.

258

Applications

of Methods of

Evaluation

Encampment had no positive effect on the degree of value campers place upon a college education; it may have had a very slight negative effect, a possibility the Encampment-control comparison supports. In order to assess changes in the sort of value campers place upon a college education, they were asked to check from a list of ten alternatives TABLE

8-14

V A L U E OP A C O I X E G E E D U C A T I O N : D E G R E E OF V A L U E Variable

Group»

N

Before"» A f t e r

Diff.

Direction of change

Number saying men need at least some college to get along wellintheworldthese days

NY-57 NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

37 28 12 42 25

40 23 13 35 28

+3 -5 +1 -7 +3

Increased value Decreased value Increased value Decreased value Increased value

Number saying women need at least some college to get along well

NY-57 NY-58 Cal-58 EFC-exp Control

50 30 17 50 35

37 26 10 39 25

38 23 14 37 24

+1 -3 +4 -2 -1

Increased value Decreased value Increased value Decreased value Decreased value

Number disagreeing with statement that college fails to equip one for the outside world 0

NY-57 NY-58 Cal-58

50 30 17

45 23 12

36 18 11

-9 -5 -1

Decreased value Decreased value Decreased value

Number disagreeing with statement t h a t college breaks down ideals 0

NY-57 NY-58 Cal-58

50 30 17

42 26 11

40 23 12

-2 -3 +1

Decreased value Decreased value Increased value

• See note a, table 8-5. See note b, t a b l e 8-5. T h i s question was not asked on Wave I (spring) in either 1057 or 1058.

b 0

the two "most valuable things about a college education." These ten were equally divided between what are usually considered primary values of college (such as broadening a person's knowledge) and secondary values (such as preparing a person for the kind of a job he wants). The full set of statements—here separated into the two categories—is as follows: College education is valuable because it: (Primary values) —broadens a person's knowledge about the world he lives in —enriches one's appreciation of the traditions of thought lying behind presentday beliefs

Effects in the College

Setting

259

—equips one to be a better citizen —develops a person's intellectual abilities —develops and strengthens a person's moral and character traits (Secondary values) —prepares a person for getting the kind of job he wants —gives one time to find out what he wants to do in life —provides the opportunity to meet new and influential people who will be of use to him later in life —fosters the growth of a well-rounded personality" —provides him with a few years of pleasurable respite before having to assume adult responsibilities

Campers were scored as having checked two of the five primary values, only one of them, or none of them. By this method, it seems that the Encampment may have had a slight positive effect on the value orientation toward college, for in all three Encampments a small increase occurred in the number checking two of the primary values or a small decrease took place in the number checking none of them. The same tendency is reflected if the total number of choices for primary values is expressed as a percentage of the total number of choices made: for all three Encampments, 5 to 10 per cent more of the choices in the fall went to the primary values than before the summer. Since the opposite is true of the control group in 1958, it seems possible that in this regard the Encampment may have had a slight positive effect. (See table 8-15). Thus, the measured effects here are both small and mixed. The Encampment seems both to lower somewhat the degree of value placed upon a college education and to strengthen slightly the tendency to evaluate college in terms of its primary goals. It must be concluded, therefore, that the Encampment's program has no systematic effect on the campers' general evaluations of a college education. CONCLUSION

In this chapter data relevant to an assessment of the Encampment's impact on the campers' life at college have been presented. So far as the area of campus politics is concerned, the Encampment, as would be expected, had a generally positive if rather small effect. As for the other two areas investigated, however, academic life and the value of a college education, no sizable or consistent effect is discernible. It seems that for campus life the Encampment has no "overflow" effects. This completes the analysis of the Encampment's short-run effects. " Some may feel that this is not a secondary but a primary value of college. So far as the present anaysis is concerned, however, it makes no difference in the pattern of the results which way this one item is viewed.

O 8 *H s _o « a Z

e o £

o S

5 fe

O

H

e 03

o H

O £

Ih

_

«

a

Effects in the College Setting

261

These have now been assessed for two different kinds of settings to which the campers return, the community and the college campus. Although the effects in the latter setting are limited to one aspect of college life, significant effects nevertheless do persist after the return. Important in demonstrating this are the systematic comparisons between a subgroup of the college campers and a group of college students who were similar to campers, but who did not attend the Encampment. The next two chapters present data on the long-run stability of effects.

CHAPTER I X

LONG-RANGE EFFECTS INTRODUCTION CUMULATIVE EVIDENCE has been presented that the Encampment achieves many of its goals effectively. Studies of the 1955, 1957, and 1958 groups establish that the program has immediate effects; each group changed in a variety of respects by the end of its Encampment. Further, the 1955 study shows that many of these effects persist at least six weeks after the campers have returned to their home communities, where they no longer have the organized moral and social support for Encampment ideas which was provided during the summer. Indeed, ex-campers sometimes confront difficult situations which are hostile to the Encampment views. Nevertheless, some of the Encampment's impact persists over the short term. In this chapter attention is turned to questions about longer-range effects. What is to be gained by extending the evaluation beyond the measurement of immediate impact and short-term results? To a large extent, both the programmatic and scientific objectives of evaluation are already served by the information on impact and short-term effects. Data on attitudinal changes during the summer, for example, clearly demonstrate the important role that the Encampment's communal life plays in affecting the participants. Reexamination of ex-campers after six weeks at home provides the opportunity to detect any abrupt reversals or changes in view. But many important questions in the evaluation of such educational programs as the Encampment remain unanswerable in the absence of evidence from extended research over a longer time span. Long-term study of the Encampment helps us to determine: (1) the eventual stability or instability of attitudes and opinions manifestly changed or reinforced during the Encampment; (2) latent effects, subsequently brought to light; and (3) the presence or absence of effects that had no opportunity to be expressed during the program or shortly thereafter, such as participation in the national political process through voting. There are, of course, other gains from long-range studies.1 1 For example, in the measurement of immediate effects it was necessary to take many precautions against various possible artifacts that might distort the results. To illustrate, the danger that a respondent might be motivated to hide his change in attitude in order to appear consistent with his earlier (pretest) answers was minimized by the use of many items combined into scales to measure each attitude and

[262]

Long-Range

Effects

263

Direct evidence about long-term effects demands inquiries on the experimental group extended over the long span of future time. How long a span depends upon the nature of the effects being considered and the predilections of the critic. Consider first the question of stability of opinions. From one point of view the six-week follow-up might be considered as a sufficiently long period to establish stability. The impact of the community should be dramatic and sudden. The conflict between the norms of the Encampment and those of the larger society would be immediate; six weeks at home should certainly provide ample opportunity for the challenge to be met, or the defeat to occur. On the other hand, perhaps six weeks is too short a time if reversals in opinion do not occur abruptly, but gradually, the slow workings of time wearing away the impression of the Encampment. After such a process of erosion, perhaps only a faint trace remains of the values earlier developed, or they may even disappear completely, leaving the ex-campers with the attitudes and opinions they held before their enrollment in the Encampment. How much time is needed for such erosion to become evident remains problematic, of course. One might defer follow-up inquiries for a year, or for five, ten, or more years, and still not meet the problem. The model of geology might be invoked and the critic might ask for the evidence of many decades to be entered into an analysis of erosion. Aeons of geologic time have their parallel here in the long life span of the young persons who come to each Encampment. Practically, however, a more limited period of time must be taken as a reasonable working definition of "long term." Or consider the question of latent effects. Studies of the impact of motion pictures and other forms of communication designed to change opinions and attitudes have shown that some members of the audience who do not appear immediately to change their views in the direction advocated by the communicator sometimes change their views weeks later. 4 Although some of these sleeper effects might be detected within six weeks, it is fully possible that others take much longer to occur. the jumbling of these scale items throughout the questionnaire. One further advantage of long-term studies is that such problems of bias cannot arise, because the final measurement is so dissociated from the original experience and the pretest situation. 2 For a summary of such studies see Carl Hovland, "The Effects of the Mass Media of Communication," in G. Lindzey, ed., Handbook of Social Psychology (Cambridge: Addison-Wesley Press, 1954), pp. 1062-1103. Evidence suggestive of a sleeper effect has been reported in an evaluation of a program of cross-cultural education involving groups of young Germans studying in the United States, who were retested some months after return to Germany. Certain of the groups continued to change after departure from the program and after their return to Germany. For the general finding and the detailed evidence suggesting that the sleeper effect may be greater for those who change less during the program see J. Watson and E. Lippitt, Learning across Cultures: A Study of Germans Visiting America (Ann Arbor: Institute of Social Eesearch, 1955), pp. 60-61.

264

Applications

of Methods of

Evaluation

Finally, there are intended objectives of the Encampment which simply take a long time to become evident. Such Encampment-encouraged activities as voting, participation in civic affairs, achievement of public-service offices and occupations, for example, are often prerogatives of adults or take years for expression. Even an extended long-term inquiry might be insufficient to detect some such gains—as an instance, political leadership—but others, such as voting, may be observed within a reasonable time. Ideally, then, there is a need for longitudinal studies of ex-campers, extending throughout their life span. Practically, of course, such studies are rarely possible because of the limitations of time, the costs involved, and the losses of respondents over the years. A partial solution to the problem of measuring long-term effects was achieved here by two separate approaches: (1) a follow-up of the 1955 group through a mail questionnaire distributed to them four years later, and (2) a sample survey of alumni from the nine Encampments of 1945-1954, conducted in 1955. Of the two approaches the four-year follow-up is the most valuable and reliable, because data are available on the 1955 group's views at the close of their Encampment as well as four years later. These earlier data provide a base line against which current opinions may be evaluated as gains, losses, or maintenance of opinions and attitudes on such issues central to the Encampment program as civil liberties, civil rights, and tolerance. Such a base line is missing for the other alumni groups. Nevertheless, the analysis of the views expressed in 1955 by nine past cohorts of alumni provided a basis for limited inferential analysis of long-range effects at a stage in the evaluation where such a direct study as the fouryear follow-up was not possible. We shall take up additional methodological points about the two approaches in greater detail later. In the remainder of this chapter we present results from the follow-up on the 1955 group. Then, in chapter x, we turn to the alumni study. T H E QUALITY OF THE EVIDENCE

Inquiries were restricted to the 100 ex-campers who formed the core group for whom data had been collected at both the beginning and the end of the Encampment in 1955. From this group 73 people returned mail questionnaires sent to them in the spring of 1959, nearly four years after they left the Encampment. The absence of 27 ex-campers from the study raises, of course, serious questions about the representativeness of the results. Perhaps those persons who do not respond are more hostile in their views, or have suffered greater instability or reversals in

Long-Range

Effects

265

opinion during the post-Encampment period, than those whose replies are received. Questions of bias are always present in any mail survey. They are always difficult, sometimes impossible, to answer. In this instance, however, it is possible to make some inferences about possible bias, using data about the relative character of the returnees and nonreturnees gathered during previous stages of the evaluation, both at the Encampment and at home, during the six-week follow-up. How much bias is there? First, it should be noted that the amount of potential bias, as reflected in the size of the loss from the panel of 100, overstates the problem. Not all the 27 cases for whom questionnaires were not obtained in 1959 represent volitional nonreturns, perhaps for psychological or personal reasons. Fifteen of the 27 never received questionnaires, 6 being deliberately excluded from the study for a variety of reasons: 1 had died; 3 had returned to the Encampment as staff members sometime between 1955 and 1959, and therefore were excluded; 2 were foreigners, no longer in the universe under study. Thus the redefined universe is 94. The remaining 9 of the 15 never received their questionnaires because their addresses were unknown. Thus only the remaining 12 received questionnaires and did not return them." In examining the relative character of nonreturnees, however, all 27 missing cases will be compared with the 73, to estimate bias. How do the two groups compare? They are, in many respects, quite similar. The most serious difference is in the ethnic and racial composition of each. Negro and Indian ex-campers are underrepresented among the 73 returnees, there being only 15 such minority members among the 73 in contrast to 15 among the 27 nonreturnees. Southerners also are underrepresented and Midwesterners slightly overrepresented among 3 There are certain striking similarities between our follow-up inquiry and one neglected phase of Newcomb's well-known study of Bennington College students. To establish the long-term stability of the changes observed during residence in Bennington, Newcomb conducted a follow-up study in 1939 of students who had been away from Bennington for intervals ranging from one to three years. There were about equal numbers from each of the three alumni groups. We shall have occasion to compare many of his findings with ours, but what is relevant at this point is the similarity in the loss rate. Among all the students who had protracted exposure to Bennington, 83 per cent of those who were mailed questionnaires returned them. Tor the group that had been away from the college for three years, thus corresponding most closely to our EFC group, the response rate was 71 per cent. (T. M. Newcomb, Personality and Social Change [New York: Dryden Press, 1942], pp. 74—75.) For our group, which had been away from the Encampment for about four years, 86 per cent of those who received questionnaires in 1959 returned them. In Biecken's followup of his panel of work campers, to assess stability after an interval of ten months, he obtained a 71 per cent return, comparable to the 78 per cent return from our panel of 94 ex-campers. (H. W. Biecken, The Volunteer Work Camp: A Psychological Evaluation [Cambridge: Addison-Wesley Press, 1952], p. 41.) Thus our losses over four years, by either nonresponses or inadequate addresses, or both, compare favorably with the other studies.

266

Applications of Methods of Evaluation

returnees. But both groups are similar in their average age and sex composition. Attitudinally, the groups were similar at the end of the Encampment in their liberal views toward civil liberties, their general tolerance of deviant social and political views, and in the degree to which they perceived society as lacking in important social norms (anomie). There is some indication, however, that the nonreturnees were slightly less supportive of certain civil rights and privileges. But, despite their occasionally lower rank on such issues at the end of the Encampment, the 27 nonreturnees had not been less receptive to the influence of the Encampment; during that summer they had changed while there as much as or more than the 73 others, in directions congruent with Encampment goals. This means, then, that the 27 are not necessarily hostile or resistant to the Encampment and its norms; they simply started with less liberal views on some issues.4 One critical question to be considered, of course, is whether the missing ex-campers might have been more prone to change their opinions after the Encampment than the 73 others. The real focus of our chapter is not on changes during the summer, but on subsequent stability or change. A partial answer to the question may be obtained through a comparison of the stability and instability of views for the members of each group who also happened to be represented in the short-term follow-up six weeks after the Encampment. Only 3 of the 27 nonreturnees of 1959 failed to return questionnaires during the six-week follow-up in October, 1955. This fact in itself may be taken as evidence that the group was not, at least in the short run, uncooperative or alienated from the Encampment. Eight of the 73 others had not responded on the six-week survey. Thus, comparisons can be made in the stability of opinions after six weeks at home in 1955 between the 65 ex-campers who also were measured in 1959 and the 24 ex-campers missing from the 1959 study. The ex-campers who did not return questionnaires in 1959 were just as stable during the first six weeks after the Encampment as the others. The two groups appeared equally stable in their attitudes toward civil liberties and civil rights, their tolerance toward deviant social and political views, their specific opinions about whether a peacetime Communist should be allowed to speak on the radio, and anomie. Thus the evidence from the short-run study provides no reason to suspect that the 1959 results overrepresent persons who are inherently more stable in their views about topics salient to the Encampment goals. * Newcomb (op. cit., pp. 75-76 and tables 28, 29) observes the same phenomenon. His nonresponding alumni were somewhat more conservative, but changed as much or more during their period of exposure.

Long-Bange

Effects

267

In summary, the results based on 73 cases in the 1959 follow-up will perhaps overrepresent the post-Encampment experiences of white excampers and persons whose immediate post-Encampment views on some topics were more in line with the Encampment norms (and hence more at odds with the larger society) than others. There is evidence that the 73 cases are not biased toward overrepresenting ex-campers who are more cooperative with the evaluation research or more stable in their views, as demonstrated through participation in the six-week follow-up study in 1955. With these points in mind, then, let us examine some of the results of the 1959 study. T H E OLD ENCAMPMENT—FORGOTTEN EPISODE OR LIVING MEMORY?

Exciting and impressive as the summer's Encampment might have been for the participants, it was, after all, a brief encounter of only six weeks' duration. Furthermore, it was an episode in the early part of most campers' young adult years, years marked by much activity and many thoughts and decisions centering on education, future marriages, families, careers, and the like. A serious question for long-term evaluation, then, concerns the extent to which the summer's experience makes a lasting impression on the memory of the ex-campers, so that it is salient rather than forgotten, a source of subsequent contact and reference rather than an isolated youthful interlude. For more than a third of the group the Encampment still seems to be a salient experience. I n 1959, nearly four years after leaving the Encampment, 36 per cent of the ex-campers report that they "frequently" stop and think about the campers and the staff members they knew at the Encampment; another 59 per cent have such thoughts "occasionally"; only 5 per cent "not at all." For many participants, ties of friendship link the Encampment episode to their current activities. Fifty-five per cent feel that they are really close friends today with someone from their 1955 Encampment, nearly a quarter having three or more such close friends. A little more than a third of the group feel that they are really close friends today with one or several Encampment alumni not in the 1955 group. Sometimes a friendship formed during the Encampment carried over until 1959. Eight of 10 ex-campers were able to recall the name of the closest friend they had at the Encampment. Among the group who recalled a friend, more than half had written to him or her within the past year and at least 30 per cent of the friends met together socially or spoke over the telephone, or both. Various forms of informal contacts with fellow 1955 alumni con-

268

Applications of Methods of Evaluation

tmued throughout the four post-Encampment years. In that time, for example, two-thirds of the group had met socially; half had spoken together over the telephone; more than 8 out of 10 had written or received letters, or both; a fifth had worked together on some project. None failed to have some such contact at some time after the Encampment. Fiftynine per cent report attending meetings or reunions with former campers from years other than 1955. The saliency of the Encampment and the persistence of informal soTABLE

9-1

PERSISTENCE OF ENCAMPMENT EXPERIENCE

(Percentages) 1955 ex-campers w h o h a d indicated experience P o s t - E n c a m p m e n t experience

Thought about Encampment Frequently Occasionally N o t at all Contacts w i t h members of 1955 group Received mail from them Wrote to them Met them socially Spoke with them over telephone Worked on project with them

I n first six weeks I n f o u r t h year after E n c a m p m e n t after E n c a m p m e n t (N = 89) (N = 73)

67 30 3

36 59 5

98 93 82 70 39

49 45 34 33 14

cial contacts among members of the 1955 cohort were not so marked during the fourth year after the Encampment as they had been during the first few weeks, of course. But that they were not temporary phenomena is evident from table 9-1, in which data from the 1955 six-week follow-up are contrasted with the results for 1958-1959. Considerable interaction was evident even during the last year, when a third of the ex-campers met socially with other campers; a third spoke together on the telephone; about half corresponded. Only 27 per cent had had no contact whatsoever with their group in the past year.6 6 Dentler's inquiry among former interns in programs of the American Friends Service Committee indicates that about 25 per cent of those who were separated from the program for three to seven years do not report any contacts with other alumni within the past year. E. A. Dentler, The Young Volunteers: An Evaluation of Three Programs of the American Friends Service Committee (Chicago: National Opinion Research Center, 1959), p. 146.

Long-Bange

Effects

269

Extensive interaction with former campers from any alumni cohort within the home community was not equally easy for all the 1955 group. About 4 out of 10 were currently living in communities in which, according to their own estimates, there were, at the most, 1 or 2 Encampment alumni. About a quarter lived in places that had from 3 to 12 alumni in their communities. The rest did not know how many were around. Neither saliency of the Encampment nor actual interaction seems to be completely dependent upon the presence of large numbers of alumni in the home town, however. To illustrate, 46 per cent of the group who believe there are only 2 or less alumni in their community think about the Encampment "frequently," in contrast to 31 per cent among those who say there are 3 or more alumni in the community. Fully 89 per cent of the group in communities with few known alumni were in contact with others from the 1955 cohort during the past year; 46 per cent of them met face to face or talked over the telephone; the other 43 per cent corresponded only. In areas where campers report there are 3 or more alumni, 63 per cent of the current group had some contact with the 1955 alumni; 37 per cent face to face or by telephone, 26 per cent by mail only. Thus, for a fairly large proportion of the 1955 group, the Encampment and its members persisted as a potential reference group, despite seemingly limited current opportunities for easy interaction. Other forms of contact with the Encampment, over which the excamper had less control than his informal social interaction, included visits by members of the staff and literature from Encampment headquarters or staff. A quarter of the group had been visited at least once by a staff member. All ex-campers had received letters or literature from a member of the staff, half of them receiving ten or more such communications between 1955 and 1959. THE INTERVENING YEAES

The four years following the Encampment were full, active young adult years for all the ex-campers. For some of them these years meant passage into adulthood and the assumption of the rights and obligations that go with adult status in our society. After leaving the Encampment almost all of the group returned to school: a few to high school, most to college, a few to graduate or professional schools. Most of them remained students for the greater part of 1955 through 1959, almost half being in school for the full four years. Several entered military service for part of the time. By 1959 only a third were still single and living with their parents;

270

Applications of Methods of Evaluation

about a quarter were single, but living away from home; about a third were married, some with children of their own. Residentially there was much mobility during the four years. More than half had lived in two or more places during that time. For the most part, the residential experience was an urban one. Only 6 per cent of the group, for example, lived in a rural place in 1959; at the same time, half lived in cities of 500,000 or more inhabitants. Each passing year saw a larger percentage of the ex-campers enter the full-time labor force: 12 per cent in 1955, 18 per cent by 1956, 31 per cent by 1957, 41 per cent by 1958, and 51 per cent by 1959. Mostly they entered various professions, only 10 per cent being engaged in manual or lower white-collar work in 1959. The end of schooling, the beginning of careers, new families, new places to live—these and other events, then, mark the highly varied pattern of experiences which intervened between the end of the 1955 Encampment and the follow-up measurement in 1959. How much of the impact of the Encampment on its participants survived such subsequent experiences is the subject to which we now turn. REGIONS OF CHANGE SALIENT SOCIAL OPINIONS AND ATTITUDES

FOUR TEAKS LATER

The 1955 campers, it may be recalled, left the Encampment with "improved" attitudes toward such social issues as support for traditional civil liberties, civil rights for minorities, and tolerance for deviant points of view. Furthermore, after six weeks' exposure to the home community, they still seemed to retain many of their liberal social opinions and attitudes. Could such opinions, however, be sustained over a longer time period, during which the young persons probably would be subjected to many different and perhaps opposite opinions in the larger society while the common ethos of the Encampment became more distant in time and memory? Over the long run, severe loss in liberal attitudes did not occur. Here, as in the measurements of opinions after six weeks' time, four tests of stability were used—repeated measurements on the scales of civil liberties, civil rights, and tolerance, and a single question about whether a Communist should be allowed to make a radio speech in peacetime. Only on the last item was there much change between the views expressed at the end of the Encampment and those opinions held in 1959. Even in the opinion areas where there was some loss over the years, the percentage of the group who expressed liberal attitudes in 1959 did not

Long-Range Effects

271

drop completely to the pre-Encampment level (which, it should be recalled, was always higher than average for persons of this age group). Hence, although life away from the Encampment reduced some of the gains made during the Encampment, it never erased them completely. Let us consider the evidence in greater detail. (See table 9-2.) On civil liberties the 73 campers under study began the 1955 Encampment with a median score of 2.6 and ended it with a score of 2.0. The very slight improvement in the direction of more favorable attitudes of this group toward civil liberties parallels the more dramatic TABLE

9-2

L O N G - T E R M STABILITY AND C H A N G E IN P O S T - E N C A M P M E N T V I E W S E n d of Encampment, 1955 (N = 73)

Area

Four-year follow-up, 1950 (N = 73)

Post-Encampment pattern

OPINIONS AND ATTITUDES

Civil liberties (median) Civil rights Zero discrimination (per cent) Tolerance "More tolerant" (per cent) Permit peacetime Communist on radio (per cent)

2.0

2.1

Stable

84

79

Nearly stable

90

84

Nearly stable

79

70

Drop

92 38

71 60

Much less optimistic

SI 90

41 100

COGNITION OF SOCIAL PROBLEMS

Race prejudice Optimistic about solution (per cent) Number of years needed (median).. War Optimistic about solution (per cent) Number of years needed (median).. Cancer Optimistic about solution (per cent) Number of years needed (median).. Index of group potency (median)

96 10 8.9

95 10 9.0

More pessimistic Stable optimism Stable

PERCEIVED RELATIONSHIP TO SOCIETY

Alienation Feel different from average American on four or five issues (per cent) Anomie (median)

51 2.1

58 2.3

Slight increase Stable

272

Applications of Methods of Evaluation

change for the entire Encampment group presented earlier, in chapter iv. Pour years later, the follow-up group's average score was 2.1— representing no significant loss subsequent to the Encampment. Only a slight regression appeared during the four years in the followup group's attitudes toward civil rights. Back at the start of the 1955 Encampment 59 per cent of this group scored zero, or "least discriminatory" with respect to the civil rights of various minorities; at the end of the summer 84 per cent showed no discrimination—representing a sizable improvement over the summer. Four years later 79 per cent scored zero, indicating a slight reduction over time, but leaving the group still considerably more favorable to civil rights than they had been before the Encampment. Increased tolerance toward deviant social and political opinions was one of the major gains during the summer at the Encampment; four years later, fewer of the group were as tolerant as when they left the Encampment, but, again, not so few as initially. The tolerance items were part of a scale that had been used previously by Samuel Stouffer in a nation-wide study of conformity in 1954. From the scores it is possible to determine what percentage of the group can be classified as relatively more tolerant than the rest of the nation. Seventy-seven per cent of the current group fell into this very tolerant segment of the public when they entered the Encampment; 90 per cent were very tolerant at the summer's end, and 84 per cent in 1959. A specific application of the concept of tolerance is provided by the issue of whether or not someone of such extremely deviant views as an admitted Communist should be allowed to speak over the radio in peacetime. At the start of the Encampment 67 per cent of the group thought this should be allowed; at the end, 79 per cent; four years later, 70 per cent—again, a drop in tolerance, but not completely to the pre-Encampment level. To say that some of the gains made by the Encampment were slightly lost in subsequent years should not obscure the fact that the ex-campers remained a highly liberal and tolerant group, compared to the general population, as is illustrated by the fact that only 31 per cent of the latter (and 47 per cent of adults under 29) scored "relatively tolerant" on the Stouffer scale (in 1954), in contrast to 84 per cent of the current sample of former campers. Whether or not the Encampment's impact is followed by a retention of attitudes developed during the summer is one question—precisely the one for which we have evidence. "Whether or not the various persons involved could have maintained the initially elite opinions and attitudes of their youth during

Long-Bange

Effects

273

their subsequent development from 1955 to 1959 without having had the experience of being members of an Encampment group in 1955 is a separate question, the answer to which cannot be obtained from the evidence at hand. CHANGES IN COGNITION OF SOCIAL PROBLEMS

Dramatic evidence of the impact of the outside world on the gains made during the Encampment was provided earlier in the realm of beliefs about the solution of the problem of race prejudice. During the Encampment, it may be recalled, there was a growth in optimism as reflected by a relatively short time period within which campers felt the problem would achieve resolution. Once returned to the home community, campers, although still generally optimistic, nevertheless increased their estimates of the number of years necessary for solution. At the same time, however, there was no reduction in the general faith that such problems would eventually be solved. Given the local and national events that took place in the four years after 1955—e.g., Little Rock, Clinton—one might expect even greater inroads upon opinions about this issue. This seems to have happened. At the end of the Encampment fully 92 per cent of the group believed that a satisfactory solution to the problem of race prejudice was fairly likely or certain; by 1959 only 71 per cent shared this belief. Furthermore, the solution was believed to be a matter of about 38 years, on the average, at the end of the Encampment, but had risen to 60 years by 1959; and the percentage of the group whose time perspective on the solution was 10 years or less dropped from 28 per cent in 1955 to 11 per cent in 1959. As an alternative to the interpretation that pessimism is a product of specific events in the field of race relations, one might hypothesize that such changes in the perspectives on the solution of race prejudice reflect a generalized process of increasing pessimism among ex-campers. Perhaps pessimism is a function of age or cumulative life experiences. To some extent the evidence supports such a hypothesis of general pessimistic growth, but not unequivocally. (See table 9-2.) Specifically, the views of ex-campers were also obtained on the probabilities of solutions to war, cancer, and economic depression and unemployment. In two of the areas—war and depression—pessimism increased between 1955 and 1959, but never so pronouncedly as in the instance of race prejudice. To illustrate: in 1955, 77 per cent of the ex-campers thought that a solution to economic depressions and unemployment was fairly likely or even certain; by 1959 only 64 per cent felt this way; and the

274

Applications of Methods of Evaluation

average number of years believed to be required for the solution increased from 47 in 1955 to 76 in 1959. Pessimism about the solution to war was fairly high in 1955, but was more so in 1959, the percentage believing a solution to be fairly likely or certain dropping from 51 to 41, and the average time required increasing from about 90 years to 100. On the other hand, ex-campers were consistently optimistic about a solution to cancer—96 per cent in 1955 and 95 per cent four years later; the average time remained about 10 years. In summary, the relatively smaller magnitude of the increase in pessimism about war and depression, plus no increase in pessimism about cancer, suggests that the larger increase in pessimism about race problems stems more from specific national events in that realm than from a general increase in pessimism associated with age. Admittedly, major national events also occurred in foreign policy and the economy between 1955 and 1959, and the impact of these events may also be reflected in the increased pessimism about these two problems. Another aspect of beliefs about social problems was measured by the indices of potency of group and individual action. As time passed, the confidence that the Encampment had instilled in the campers with respect to the power of group action in solving social problems was retained. At the same time, the devaluation of the potency of individualistic action, which came about during the summer, was never undone. To demonstrate, at the end of the Encampment the median score on the index of group potency in the solution of social problems was 8.4; four years later it was 8.5, representing high stability of these beliefs. Furthermore, the proportion of ex-campers who believed that groups could do a great deal or a moderate amount to solve such macroscopic problems as war or corruption in national governments increased from 64 per cent at the close of the Encampment to 88 per cent four years later. On the other hand, the median score on faith in individualistic actions dropped slightly below the already low figure obtained at the end of the Encampment. ALIENATION AND ANOMIE

In the analysis of long-term effects, as in the evaluation of immediate and short-term impact, provisions were made to detect important effects not part of the planned and specified objectives of the program. One kind of officially unanticipated side effect with which we have been concerned is the possible alienation of campers from society through such phenomena as feeling that they hold views greatly different from those of the average person, evaluating the average citizen harshly, and per-

Long-Range Effects

275

ceiving society as lacking in clear norms and moral concern (anomie). Results from measures in this area at the end of the Encampment and after six weeks at home made it seem important enough to be reexamined in the four-year follow-up. After six weeks at home, it may be recalled, there were conflicting indications as to whether ex-campers were becoming more alienated. On the one hand, increased alienation was clearly revealed by the excampers' growing sense of discrepancy between their opinions and those held by the average ciitzen. There was also a harsher view of the average person as less liberal and tolerant than the campers. On the other hand, there was increased uncertainty as to how the average American would respond to such questions as whether a Communist ought to be allowed to talk on the radio in peacetime. And ex-campers increased only slightly in their perception of society as anomic, that is, as possessed of relatively few clear, enduring norms. This mixed, equivocal picture continues in the four-year follow-up. (See table 9-2.) Again, there is evidence of a feeling that the average American holds views that are much less liberal than those of the ex-camper. And this felt alienation increases somewhat with the passage of time. To illustrate, at the end of the Encampment about half (51 per cent) of the group thought their views differed from those of the average American on at least four out of five such subjects as civil liberties, economic planning, the urgency of social reforms, race relations, and international relations. Four years later 58 per cent felt such differences, almost always feeling that the difference consists of less liberal views held by the average American. Only rarely does the ex-camper feel that these differences in viewpoint between himself and the average American are "very small," and about 8 ex-campers out of every 10 feel that the difference is "very great" on at least one of the issues. Furthermore, there was a drop in the percentage who feel they disagree with the average American on only one or none of the issues, from 18 per cent in 1955 to 10 per cent in 1959. On the other hand, there was little change over the years in the excampers' willingness to estimate the average American's opinion about peacetime Communists on the radio. At the end of the Encampment 75 per cent gave their judgment about the average American's views on this subject, and among them fully 80 per cent believed that he would not allow such a speech. In 1959, 79 per cent of the ex-campers estimated the average American's view, and, among these, 76 per cent believed that he would not permit the speech. Thus the degree of "harshness"

276

Applications of Methods of Evaluation

with which the average person is perceived, as reflected by this question, remained about the same throughout the four years. Nor was there any appreciable change in the perception of society as anomic. The average score at the end of the summer was 2.1; in 1959 it was 2.3. LONG-RANGE IMPACT ON CONDUCT

One of the manifest purposes of the Encampment is to encourage the campers to play a more active and effective role as citizens in democracy. Data on post-Encampment conduct, then, are especially pertinent to an over-all evaluation of the program. But the questions about conduct which can be approached through a long-term follow-up study differ from those concerning opinions and attitudes in several ways. One important question in the long-term study of opinions and attitudes is whether the Encampment ideology persists: are the views expressed at the end of the 1955 Encampment still evident in 1959? In a long-term study of conduct, however, the issue of stability or change becomes relatively unimportant because most of the significant activities of adult citizens, such as voting, could not be performed by the young campers before and during the Encampment. Hence there is no 1955 base line for such conduct against which 1959 behavior can be compared. In some instances it is possible to use data on conduct gathered during the six-week follow-up as a quasi base line for evaluating changes in amount of conduct by 1959. Reported attempts at applying Encampment principles in concrete social situations are a case in point. But for most significant forms of citizenship such base lines are not available, and therefore the issue of stability in conduct becomes meaningless. Rather, the critical questions now are whether the excampers actually participated as active citizens and whether such conduct has been influenced by the Encampment. Some answers to the question about participation can be obtained directly on the follow-up study. The fact that four years have elapsed since the Encampment should permit some forms of conduct encouraged by the Encampment to occur. There was, for example, ample opportunity to apply the principles of democratic action and tolerance stressed by the Encampment and for a modicum of political and civic participation to begin. Several national, state, and local elections took place between 1955 and 1959 for which some of the ex-campers were eligible to vote. Adult status also should permit some of them to be involved in voluntary associations treating social problems and civic matters. On the other hand, some socially important behavior will not have had time to develop. I t will take decades, for example, before any

Long-Bange

Effects

277

1955 camper even reaches the age at which he might possibly achieve prominence in politics, government, or other careers affecting the public welfare. Nevertheless, such specific careers may be affected somewhat by the attitudes and motives developed or strengthened during the summer at the Encampment. In the 1959 follow-up, measurements are limited to such manifest current behavior as application of Encampment principles, voting, and civic participation. For some of these actions it is possible to make some comparisons between the conduct of ex-campers and other American young persons, thereby evaluating their role as citizens relative to the average. It is more difficult, however, to learn whether the ex-campers' activities are in any way influenced by the Encampment. To some extent one can make a variety of inferential tests which shed light on the possible influence of the Encampment, but direct causal connections between the Encampment and current behavior are exceedingly rare and difficult to establish. Inferential evidence of influence, for example, would be provided by finding that ex-campers who still feel that they strongly agree with the principles of the Encampment are more likely to be active citizens than ex-campers who are now relatively alienated from the Encampment norms. With these methodological considerations in mind, let us now turn to the available evidence on the ex-campers' conduct between 1955 and 1959. Applying Encampment principles.—Even before attending the 1955 Encampment the group under study had a strong proclivity for social and political action, a tendency that was strengthened during that summer. Yet, it may be recalled, evidence from the six-week follow-up in their home communities showed that few of the ex-campers had engaged in social actions to implement the ideals of the Encampment. Despite the fact that most ex-campers encountered some social situation in which they wanted to apply Encampment principles, only 37 per cent actually tried to do something about such concrete social problems during their first six weeks away from the Encampment. The meaning of this low rate of action was ambiguous at that time. On the one hand, six weeks might have been insufficient time for action on social problems. On the other hand, the impact of the home community and separation from the Encampment might have been already so great as to attenuate social action. The 1959 follow-up affords an opportunity to clarify the issue, for surely four years provide sufficient time for ex-campers to try to apply Encampment ideals in some social situation. Each respondent was asked whether there had been any major in-

278

Applications

of Methods TABLE

of

Evaluation

9-3

L O N G - T E E M POST-ENCAMPMENT CONDUCT A N D VOCATIONAL GOALS

(Percentages) End of Encampment, 1955 (N = 73)

Conduct and goal

Four-year follow-up, 1959 (N = 73)

Post-Encampment pattern

CITIZEN PARTICIPATION

Tried to apply Encampment ideas Voted, among eligibles b In 1956 presidential election In 1958 congressional election In any state election In any local election Recently discussed politics Usually take active part in political discussions Have written to government or public official Belong to at least one voluntary association

37'

41

No change

78 70 67 72 86

High political participation

66

High activity

45 71

Relatively high activity High activity

40 3

No change Slight decrease

High activity

VOCATIONAL GOALS

Desired service-oriented career Desired politically oriented career...

40 12

a b

Data from six-week follow-up, based on 89 cases. The numbers of ex-campers eligible to vote were 36 in 1056, 53 in 1958, 55 in state elections, and 58 in local elections.

stance since leaving the Encampment in which he had tried to apply any of its ideas. Then a description and a date were obtained for the single event which stood out in his mind as most important. (See table 9-3.) Fifty-eight per cent of the ex-campers report attempts at applying Encampment principles or ideas sometime between 1955 and 1959. A few of the reports indicate, instead of any specific event, a general effort at living up to such ideals as tolerance and respect for individual dignity. Discounting such cases leaves 41 per cent of the group who describe a specific event. Comparing the proportion of active ex-campers in 1955 with 1959— 37 per cent versus 41 per cent—shows clearly that additional time does not change the picture. At each time only about 4 out of 10 ex-campers report enterprises, while something in the individuals or in their environment leads the other 6 to remain inactive. It also should be noted,

Long-Range Effects

279

however, that the proportion of actives, while not increasing over time, did not decrease either, even after several years of absence from the Encampment and exposure to other outside influences. About half of the events reported took place in 1958 or 1959, indicating that ex-campers continue to apply Encampment principles as much as three or four years after their departure. (There may have been others in the group who also tried to apply Encampment principles within the past year, but chose to report only about an earlier experience which they considered to be more important.) The incidents occurred in a variety of contexts—military, collegiate, community—but mostly while the ex-camper was a student. Politics and race relations were the most frequently cited areas in which the incidents took place. A few cases will illustrate the variety of such events. Action within the campus setting is illustrated by ex-camper 104, who writes: Throughout the greater part of my career on the college newspaper (19561958) I was active in the battle against membership lists, imposed by the administration on the extracurricular religious and political clubs. I saw these required lists as an affront to the basic civil liberties of those members who preferred to keep their affiliations private. By means of editorials and articles, as well as participating in student government plans, I tried to have these lists made voluntary, rather than mandatory. Unfortunately, I was graduated before I could see the issue through to a satisfactory conclusion.

And by ex-camper 250: I n college—1955-1956—I tried to help break the color barrier (unwritten) f o r a friend at a f r a t e m i t y . I was unsuccessful.

An incident in military service is related by ex-camper 203: I n August of 1957 I was in the Army stationed in . I was in an all-"white" outfit. ( I too am "white.") This situation was no mere coincidence. As soon as a Negro would be placed in our outfit, the top-ranking men would see t h a t he was transferred out. One day, in the presence of the majority of my outfit, I forced an open discussion with the policy-making members of my outfit, about the subject of integration. Shortly a f t e r , very luckily f o r me, my two-year tour of duty was over and a Negro was admitted into our outfit.

Involvement in the local community is illustrated by ex-campers 135 and 246: This occurred this year in an election on the village level. I tried to have people register f o r voting and also look at the other party's platform and people running f o r election regardless of what their party backing was. 1959—provoked investigation by county prosecutor and grand j u r y into charges I made of corruption in local school board.

280

Applications of Methods of Evaluation

Voting, and political and civic involvement.—That ex-campers constitute a very active segment of the political public is demonstrated by the high proportion who voted in different types of elections held between 1955 and 1959: 38 per cent voted in the presidential election of 1956; 50 per cent in the congressional elections of 1958; 50 per cent in state elections, and 58 per cent in local elections. Not all campers were eligible for each type of election, of course, and when the inéligibles are removed from the calculations the proportion of voters increases considerably." Among ex-campers who were eligible to vote in the 1956 presidential election, 78 per cent did so; 70 per cent of the eligible excampers voted in the congressional elections; 67 per cent in state elections; and 72 per cent, in local elections. (See table 9-3.) Compared with political participation by the general public, the rates of voting among ex-campers are quite high. To illustrate, in 1956 about 50 per cent of the electorate voted in the presidential election. Voting in congressional elections is even less. Campbell and Cooper report that there was about a 43 per cent turnout of voters in the 1954 congressional election, which is probably fairly typical of other years.7 Of course, the rate of voting among college-educated persons, who would be more comparable to the ex-campers, is usually higher. On the other hand, young persons generally are less likely to vote than older ones, and the ex-campers are still quite young; only 10 per cent are more than twentyfive years old. The role of active citizen for most campers goes beyond voting and involves them in active discussions about public issues and politics and in occasional correspondence with government or public officials. Nearly 9 out of 10 ex-campers report that they had recently talked about public issues or politics with someone, and in such discussions with friends two-thirds of the ex-campers either take an equal share in the conversation or try to convince the others of their views. Finally, 45 per cent of the group have written at least one letter to some government or public 6 Ineligibility was determined by self-reports of the respondents. Admittedly, some ex-campers may have claimed to be ineligible when in fact they were eligible to vote. It is possible, however, to make one validity check on such reports insofar as ineligibility is a function of age. Thirty-three of the 37 ex-campers who claimed to be ineligible for the 1956 presidential election were in fact under twenty-one years of age at that time. Eleven of the 19 who claimed ineligibility for congressional elections were in fact under twenty-one ; 12 of the 19 who said they were ineligible for state elections were only twenty-one or twenty-two years old in 1959 ; and 8 of the 15 inéligibles for local elections were still only twenty-one or twenty-two years old. Of course, even those ex-campers who were old enough to vote at these various times might have been disqualified for some other reason, such as residential requirements. 7 Angus Campbell and Homer Cooper, Group Differences in Attitudes and Votes: A Study of the 1954 Congressional Election (Ann Arbor: Survey Eesearch Center, University of Michigan, 1956), p. 10.

Long-Range

Effects

281

official. (National surveys show that only about 15 per cent of American adults and about a third of college-educated American adults have ever written such a letter.8) It is, of course, difficult to estimate the extent to which the persons involved would have been politically active if they had not gone to the Encampment. Nevertheless, it is possible to make some limited inference about the connection between the Encampment experience and subsequent behavior. Active citizenship as expressed through voting and other forms of civic participation was definitely one of the ideals stressed by the Encampment. If the behavior derives from the Encampment, the ex-campers who still feel themselves to be in agreement with the Encampment norms should, in turn, play a more active role as citizens than those who now feel alienated. This hypothesis can be tested. (See table 9-4.) Respondents were asked to think back to the way most of the campers felt about the importance of politics during the summer of 1955 and to indicate whether they themselves are now in very great, moderate, or very little agreement with such views. Considering only those excampers who were eligible to vote, in each election there was a slightly higher percentage of voters among those ex-campers who now feel in great agreement with the old Encampment norms about the importance of politics. Similarly, such ex-campers were also more likely to play an active role in political discussions and to write to government officials than those ex-campers who feel only moderately in agreement with the Encampment opinion. Although none of the differences in rates of political participation is large, the fact that they all favor the group currently in agreement with Encampment norms suggests that excampers may have translated the program's encouragement of active citizenship into behavior in the subsequent years. Ex-campers are active also in a variety of voluntary associations. Seven out of 10 belong to at least one club or organization; more than a third belong to three or more such associations, which is exceptional in American society. National studies have indicated that only about 4 out of 10 American adults belong to some voluntary association and less than 10 per cent belong to as many as three. Among college-educated American adults, about 6 out of 10 belong to some association, and nearly 4 in 10 belong to at least two organizations.9 For ex-campers who belong to such groups the incidence of leadership is high: nearly half Data from survey conducted by National Opinion Research Center, 1946. " For examples and evidence from such surveys see Charles E. Wright and Herbert H. Hyman, "Voluntary Association Memberships of American Adults," Amer. Sociol. Bev., X X I I I (June, 1958), 284-294. 8

282

Applications of Methods of Evaluation TABLE

9-4

POST-ENCAMPMENT POLITICAL PARTICIPATION OF E X - C A M P E R S ACCORDING TO T H E I R C U R R E N T A G R E E M E N T WITH ENCAMPMENT N O R M T H A T POLITICAL ACTIVITY I S IMPORTANT

Ex-campers who feel that the agreement between their current (1059) views on the importance of politics and those held by most campera in 1955 is Activity Very great

1956 presidential election Voted (among eligibles) Did not vote (among eligibles) 1958 congressional elections Voted (among eligibles) Did not vote (among eligibles) Any state election Voted (among eligibles) Did not vote (among eligibles) Any local election Voted (among eligibles) Did not vote (among eligibles) Role during political discussions Active talkers Mainly listeners Letters to government or public official Have written Have not written

Moderate or very little

N«

Per cent

N»

Per cent

18 18

89 11

17 17

65 35

30 30

73 27

23 23

61 69

30 30

73 27

23 23

61 39

33 33

76 24

24 24

67 33

37 37

70 30

35 35

57 43

37 37

57 43

35 35

34 66

• For the first four items (elections), N represents number eligible to vote; for the last two items it denotes number of cases.

who are members report that they currently hold an office in one or more of the organizations. But the ex-camper's life is a varied one, with many demands upon his leisure time. Consequently, the time that he devotes to civic and political activities still falls short of his own expectations. Only 12 per cent of the respondents report that participation in civic and political activities is one of the two ways in which they usually spend spare time, while 29 per cent cite such activity as the one to which they would like to devote much more leisure time if it were available. Careers and aspirations.—When they left the Encampment, 40 per cent of the respondents hoped eventually to enter such service-oriented careers as the ministry, teaching, and medical work. Another 12 per

Long-Range Effects

283

cent desired such politically oriented careers as diplomacy, government, and political journalism. The rest had a variety of other career goals. By 1959 only half of the ex-campers have a full-time occupation; most of the rest are students, although a few are in the armed forces, or are housewives, or are unemployed. Twenty per cent of the group currently are in service-oriented careers, and 5 per cent are in political careers. Vocational achievement is, of course, greatly dependent upon time, especially where professional careers are concerned. Therefore it may well be that four years are insufficient for determining the eventual pattern of careers for these ex-campers. One may, however, contrast their current vocational goals with those expressed in 1955 and thus determine the extent to which commitment to service orientation is retained by the group. (See table 9-3.) In 1959, 40 per cent of the respondents had service-oriented career goals, 3 per cent political, thus reflecting a fairly stable pattern over the four years. There was, however, a sizable increase in the percentage of ex-campers who were uncertain about their eventual goals, from 16 per cent in 1955 to 32 per cent in 1959, perhaps reflecting the impact of college on youth. CHANGE AS AN ABRUPT OR EROSIVE PHENOMENON

The four-year follow-up sheds some light on the temporal aspects of post-Encampment changes. One might argue, on the one hand, that any loss in the liberal social opinions and attitudes of the campers would become evident almost immediately upon reentry into the larger society and would therefore have been detected in the six-week follow-up at home. If the opinions remained firm at that critical time, then they might be expected to remain firm over the long run. On the other hand, one might argue that loss of liberal attitudes is a gradual process, the net result of an erosion over years of contact with a less liberal society. Information about the rate of change may be obtained from 65 excampers who were represented in both the six-week and the four-year follow-up studies." Their views on selected subjects at the beginning and at six weeks and four years after the Encampment are presented in table 9-5. The evidence shows clearly that no single pattern of change fits all the phenomena; some opinions remain stable throughout the postEncampment period, others change immediately in the home community, and still others take longer to change. There is, however, one 10 These represent almost all the group for whom long-term data are available in 1959. The 8 missing cases, then, are obviously not persons who strenuously refuse to participate, for they made themselves available after four years. Presumably some temporary event prevented their replying in the six-week follow-up.

iS Ui l "•la a W

-Q 03

-Q 03

S S O O W H

02

.ß 03

S ° o S

i1 Xi 'S

G

O Ih O. 0)

2 ^

o "O -S « «H

T3 O

§ " " -5 g.

•J-o S3 S 8-S»

s-a .2 S V a - a o. O H O

Long-Range Effects

285

theoretically possible pattern that never occurred: abrupt drop-off in liberal views during the immediate return to the home community, followed by a recovery of such views over the long run. To illustrate the variety of patterns, there was long-range stability in the group's views about civil liberties, civil rights, and norms in society (anomie). Tolerance and optimism, on the other hand, seemed stable after six weeks, but had changed by four years. The percentage of extremely tolerant persons in 1959 (as measured by the tolerance scale and the question about allowing a Communist to speak on the radio in peacetime) had dropped from its high point immediately after the Encampment back almost to the pre-Encampment level, and the percentage of persons who were optimistic about the solution of race problems was even lower than before the Encampment. More sudden changes, however, were detected in the realm of alienation. The percentage of campers who felt that their views differed from those of the average American on four or five major issues, which had increased during the summer, became higher when the campers returned home; but then it became no worse in the following years. An interesting point is raised by a seeming paradox in the shortand long-term data on ex-campers' cognition of the social problem of race prejudice. Under the immediate impact of the home community, six weeks after leaving the Encampment most campers retained their optimism that the solution of race problems was a fairly likely event, but they tempered their optimism by increasing the number of years that must pass before this goal is achieved. Over the long run, however, such a temporal adjustment evidently is not enough, for the general expectation that the problem will probably be solved is also eroded by time or events, or both. This raises the more general possibility that erosion of optimism may take this two-step course: first, a retention of optimism through the device of increasing the time perspective about when the goal will be achieved; then, under the erosive forces of time, a reduction in the general optimism about the likelihood of solution. The several patterns of stability and change illustrated here underscore the need for long-range as well as short-range data on effects. It would not have been possible to estimate the long-run patterns from the measures of stability or change made in the short-term follow-up. Losses during the short run, as has been noted, were genuine and enduring. But some opinions that were stable for a short time began to drop later, while others held steady over the longer as well as the shorter period. Hence the value of longitudinal studies for the actual determination of change cannot be overrated in evaluation research.

286

Applications

of Methods of

Evaluation

STABILITY AND CHANGE BY RACE, SOCIAL CONTEXT, AND HISTORY

The general pattern of long-range stability and occasional change in opinions has been illustrated. "We may now examine the extent to which the Encampment influence persisted despite specifically adverse postEncampment conditions. The pressures to yield to the opinions of the larger society, which are often contrary to those expressed during the Encampment, will be greater under some circumstances than others and for some subgroups more than others. Maintenance of Encampment attitudes and their expression in community conduct should be relatively more difficult, for example, for Negroes than for whites, Southerners than Northerners, those who have moved about than those with stable residential histories, those living in communities where young persons have little social power than those living in college communities, and so on. A compelling argument for the long-range influence of the Encampment can be made if even those ex-campers with "unfavorable" postEncampment statuses retain the liberal social views encouraged by the Encampment and participate in citizenship activities. Unfortunately, such analysis is limited by the insufficient number of cases available which fit the relevant types. For example, the 1959 follow-up sample contains only 4 Indians and 11 Negroes; only 15 persons lived in the South for any of the four years, let alone for all that period. Consequently, comparisons among such subgroups are difficult and the analysis is, at best, tentative and limited to instances of unambiguously large or minute changes over time. With these qualifications in mind, let us examine some of the findings. In general, as in the six-week follow-up, there is little evidence of severe loss of Encampment-sponsored opinions and attitudes, even among those subgroups most subject to environments that might be expected to erode the viewpoints during the four years. This is not to say that there were no losses over time, or that some groups did not react differently from others. To be sure, some signs of attrition are evident. But in the main there are few dramatically greater losses even in such subgroups as ex-campers who have lived in the South, those who have been highly mobile, those who are Negroes, and so on. First, consider the long-term stability or change in liberal opinions held by Negro and white ex-campers. Tests were made on civil liberties, civil rights, tolerance, optimism and time perspective on race problems, and vocational goals. In none of these areas is there a substantially

Long-Bange

Effects

287

greater drop in the liberal opinions of Negro ex-campers than in those of the white ex-campers. Some differences occur but they are rare. In 1955, 86 per cent of the Negro campers scored "more tolerant" than average, 71 per cent in 1959, reflecting a change in one ex-camper. Meanwhile the white ex-campers remained fairly stable, 94 per cent scoring "more tolerant" than average in 1955, 91 per cent in 1959. Both groups remain stable in their support of individual civil liberties; both lose some of their optimism about the solution of race problems (especially the white group, who had been more optimistic than the Negroes at the end of the 1955 Encampment). A slightly higher percentage of the Negro campers prefer service-oriented occupations now than then. In the area of conduct, however, a smaller percentage of Negro than white ex-campers report trying to apply Encampment principles in some concrete situation—a finding in keeping with the earlier results from the six-week follow-up. Although many opinions that were developed or strengthened during the Encampment are contrary to the norms of the South, ex-campers who spent up to four years there after the Encampment seem to have sustained their original opinions as well as persons who were never in the South during the four years. For example, they show no differential loss in attitudes toward civil liberties, civil rights, belief in the power of the individual to solve social problems, or permission for a Communist to speak on the radio in peacetime. There is, however, a larger drop in the percentage who are most tolerant—from 87 per cent in 1955 to 67 per cent in 1959 for the Southerners, versus 91 per cent to 86 per cent for the others. Both groups are less optimistic about the solution of race problems than they were in 1955. Ex-campers who had lived in the South were just as likely to have tried to apply the Encampment's ideas in some concrete situation between 1955 and 1959 as those who lived elsewhere. At first glance this finding may seem contradictory to the data in chapter iv which indicated that Southern campers were less likely to apply Encampment principles within the first six weeks of their return home than were others. But no contradiction need be involved. Perhaps the lapse of four years gives relatively equal opportunity to Southerners and others to apply Encampment ideas at least once, despite a more difficult environment in the South. Or, perhaps, some of the ex-campers classified here as Southern were living in some other region when they tried out the Encampment ideas, for only 5 of the 15 Southerners spent all four years there after the Encampment; 4 had been there for two or three years

«s o àS® 91"

:

a a3Sa»

z

faao sa E§

eo O

00 i

ci

* à«" œ^ Sio 9 s 55 O

S

h O a o ¡z 0 a ¡s «! (H 1 9 H m

d H

a O

I 03 o d H

•tí d

'S •o 1T5 d d d co •S W a> 1-H . _ . ® '

a

Ial

a d H

p. o d bo tí

'S

m ® •o i c d o

O ja -a e bC a»

> S

d

tí 5 . d o5 11 O H d - a) c o > > OD CO O 0Q ' S . :G O Ö a a s a

03

a o. a 03 Cí tí

a

03

«

I-I

œ ai o o o o m ra tí d 03 o3 O y ••-! o O •tí « T3« - 3 « .. Tg; fe ® 0) S S fe

tí tí

d d O) o O : »H§ o o »H h h a .. O O* O B S S fe fe "o fe fe O fe Eh O b

•s ^ IO d oí

I«3 O tí H bC d

Long-Range Effects

295

Despite occasional exceptions, the general pattern of the findings is now clear. The opinions and attitudes expressed by the campers at the close of the 1955 Encampment were, on the whole, equally well maintained by ex-campers with or without subsequent contacts with the Encampment or use of the 1955 campers as a reference group. Action, on the other hand, is somewhat more likely to be dependent upon such contacts and supports. This pattern was observed in the studies of both short-term and long-term stability. The relative independence of opinion from subsequent referencegroup support during the four years after the Encampment may seem paradoxical in view of the important role that the group played in the dynamics of opinion formation during the Encampment summer itself. As noted in chapter v, communal living and the incorporation of the campers into the group experience seemed to play a greater part than the formal didactic program itself in shaping opinions and attitudes during the summer. These two findings form an interesting contrast worthy of additional thought, for they provide the basis for several hypotheses concerning the connection between reference groups and individual opinions. They suggest, for example, that the connection may depend upon the functional relations among the individual, his opinions, and the reference groups, as well as upon the larger situational context which determines visibility of opinions. To elaborate, reference groups seemed especially important in producing change in opinions or attitudes during the summer. Perhaps such opinions change because they serve the function of relating the individual to the new group that surrounds him at the time, a group in which the individual's attitudes and opinions are made highly visible and a group into which he would like to be accepted. Under the conditions of life at the Encampment it was rather difficult for sentiments and opinions to remain private. There was, in effect, a relatively thin line between private and public realms during the summer. Both formal and informal pressures worked to change opinions from private to publicly expressed phenomena. Formally, for example, the Encampment encouraged the tapping of private sentiments and the verbalization of opinions through such institutionalized parts of the program as small discussion groups (in which each camper was drawn into a conversation about civil liberties, civil rights, tolerance, or whatever other subject had just been treated by a lecturer), workshops, personal biographies and group reports delivered before other campers, and so on. Informally, the general atmosphere, the close living, the evening "bull sessions," and other elements combined to make talking

296

Applications of Methods of Evaluation

easy and hiding one's views difficult. Expression of opinion then became a necessary condition for acceptance into the group; and verbalization of an opinion is in itself a form of conduct. The process of change of individual opinions toward the group norm is facilitated by the atmosphere, which makes expression of opinions almost mandatory, plus the close community living, which increases the mutual dependence of members of the group. The extent to which the individual is involved in group activities affects his exposure to the group's norms and his susceptibility to change during the summer. Once the new opinions are formed and expressed, however, they become part of the individual's personal viewpoint which can be maintained in subsequent life with as much tenacity as his other private opinions. They can function on their own, without group support, even in a hostile environment, as they are once again private matters. Thus in the ordinary, day-to-day experiences of life at home, in the community, or on the campus, the individual can protect his Encampment norms either by keeping them private, as there is no institutional pressure to make them public, or by selecting new friends and environments that support the liberal views. In either event he is no longer dependent upon the original group of campers who played so important a role in developing or reshaping his pre-Encampment ideas. One might conjecture, however, that the actual or potential psychological support of the original group would again take on significance if the individual were forced to make his opinions known before a hostile larger society, just as group supports increase the probability of playing the visible role of active alumnus who tries to apply Encampment ideas in the larger community. Findings such as those summarized here, then, suggest new hypotheses for future research which promise to increase our knowledge about the distinctive functions of reference groups for opinion formation, or maintenance and change, as well as the conditions under which reference groups are likely to affect opinions and behavior. The testing of such new hypotheses requires new research beyond the current project. We turn now to the second source of information about long-range effects alluded to earlier, the survey of nine classes of alumni.

CHAPTER X

LONG-RANGE EFFECTS AMONG ALUMNI OF EARLIER ENCAMPMENTS INTRODUCTION T H E MAJOR questions which all evaluation research is designed to answer have now been treated. Evidence on the effectiveness of the Encampment in producing intended or other changes was provided in chapter iv for the 1955 group. Evidence on the long-term stability of these changes was presented in chapter ix. For discrete programs, these are the two crucial questions of evaluation. "What, then, is the need for additional evidence from alumni of Encampments before 1955? From the perspective of 1959, there is no need for such evidence. But our researches in evaluation have a history beginning in 1955, and at certain stages in this history the need to use the earlier alumni was acute. Because the problems which we faced are common, if not universal, to all evaluation, it is worth while to review them. Many evaluators will not be fortunate enough to be able to prolong their inquiry by subsequent stages, and the detailed presentation and critical examination of the alumni study may therefore prove valuable to those who will have to limit themselves to the kinds of procedures we employed. Consider the circumstances in 1955 and 1956. The question of immediate effects of the Encampment of 1955 was amenable to study, and, in principle, a satisfactory answer could be obtained quickly. To answer the other crucial question of long-range effects was impossible at the time. Resources were limited and, naturally, the question of immediate effects comes before the question of whether such effects persist. But even though modest resources were available, the postponement of the final conclusions of the evaluation far into the future would have gravely impaired the usefulness of research in planning subsequent Encampments. A compromise solution would be to study the persistence of immediate effects over a short interval of time, as was done in our follow-up of the 1955 group six weeks after they returned home. This at least has the virtue of establishing whether any effects persist at all, and does not postpone conclusions to some remote future date. Whether one may generalize the findings of such research to long spans of time is debatable. In the instance of the Encampment, our speculations advanced in chapter ix suggest that any reversal of effects should be apparent quickly. We had assigned prime influence in reversing the [ 297 ]

298

Applications

of Methods of

Evaluation

effects to the counternorms of the community and the larger society rather than to the slow grinding of time. Yet, as the subsequent empirical comparison of six-week changes versus four-year changes showed, there is no consistent pattern. Some changes occur abruptly within six weeks and then stabilize; other changes begin only gradually and are not revealed by a six-week follow-up. One of the real gains from the sixweek study is this very evidence that, as a compromise solution to the comprehensive measurement of long-term stability, it cannot be trusted. Granted foreknowledge of this finding, all conservative judgments about long-range effects would remain in abeyance from 1955 until 1959. And, if one insisted on an extreme definition of the problem of longrange effects, conclusions might be postponed until 1960, or 1965, or even later—until the time matched one's definition. But the problem gets even worse! For planned programs of social action which are repetitive or cyclical, a single evaluation is not definitive, because the generality of the findings from one cycle may not apply to other cycles. If other cycles of the program operate according to the same plan, with the same staff, upon the same kinds of human material, and are embedded in the same kind of milieu or matrix of events, one might generalize the findings from any single cycle of its operation. Probably the first three—program plan, staff, and subjects—will be fairly stable, as they are within the control of the action organization. 1 Certainly the last factor, the larger milieu within which particular cycles operate, could vary radically simply because events are capricious and beyond the control of the organization. One event could throw an entire cycle out of line.2 Observation of all four factors may create some confidence that they are similar from cycle to cycle and make generalizations plausible. But such conclusions would always be shaky, because the constancy of all the major factors would remain in doubt. Admittedly, the empirical studies of the 1955, 1957, and 1958 cycles of the Encampment presented in the earlier chapters replace such a speculative enterprise and warrant the conclusion that the immediate success of the Encampment is a general finding that holds for different cycles of its operation. These replications, however, could not deal with long-range effects, and in our 1955 1

See chapter ii, where constancy in recruitment of subjects is demonstrated. Constancy of all four factors is desirable if one wishes to generalize to other cycles of a program without having any replications. If replications are to be conducted, however, variations in events are especially desirable. As we noted in chapter iv, these are external to the program and their influence on the changes observed in the subjects should be isolated. The replication may substitute for the usual control group in estimating the extraneous influence of events, for if similar effects occur despite variations in events, one cannot attribute the effects to the events. 2

Longe-Bange Effects among Earlier Alumni

299

circumstances the possibility of conducting the subsequent studies was not even "pie in the sky." Thus, even if a final evaluation were postponed from 1955 to 1959, this would merely provide one test of long-range effects from one cycle of the Encampment program, after a four-year interval. Replication of the long-range study would require a group from a subsequent cycle, which would have to be studied over the same interval of years, thereby postponing the conclusion till 1960 or later and necessitating lavish additional resources. For such reasons evaluation designs that provide indirect, but nevertheless sound, evidence on long-range effects, at low cost and without delay, are urgently needed. Such a design was developed and realized through the study of alumni from the Encampments of 1946 to 1954, conducted in 1955. For any program that is cyclical and has already operated for more than one cycle at the time of the initial evaluation, such a design is applicable and is an efficient solution to the problem of long-range effects. The alumni of the Encampments of 1946 to 1954 were the past history of the Encampment, but they may also be used to represent the future histories of the 1955 cohort. In 1955, the alumni from 1954 had been away from exposure to the Encampment for one year; the alumni of 1953 had had two years in which to change; and the alumni of 1946 had had nine full years subsequent to their Encampment experience. The alumni of 1951, measured in 1955, had had four years in which to change, just as the follow-up of the 1955 group in 1959 was a four-year test of stability. These past cohorts could thus be used to project the future careers of the 1955 group forward ten years in time. The risks involved in this procedure are serious and the assumptions required by such an inferential method may be tenuous. We shall shortly turn to these methodological issues. Despite the hazards, however, it is desirable, for it is the only indirect avenue to any answer to the profoundly important question of the persistence of effects over relatively long spans of time. The unusual benefits of this design are worth emphasizing. Our direct study of long-range effects was limited to an interval of four years. The indirect study serves not merely as a poor man's equivalent of the direct study. It more than doubles the range of observation, carrying us nine years out in time. It also provides much greater refinement at little cost. Any direct study adds only one set of measurements and gives only two points in the process: the immediate effects and the final outcomes—in this instance, four years later. What shape the process takes is lost to

300

Applications of Methods of Evaluation

sight." With the indirect study of many alumni cohorts, one can see whether final outcomes develop gradually or abruptly, or follow an erratic course. Thus the indirect study is an exquisite instrument combining the power of the telescope and the sensitivity of the microscope. Thus far, our argument has been that the alumni study may be used inferentially to measure long-range stability. Given the difficulties, the cost, and the delay entailed in any direct follow-up after a long time interval, to undertake more than one such follow-up is beyond the reach of most investigators. Thus it is that we can present only one direct study of long-range effects on the 1955 cycle. The value of checking the findings from the direct study of one cycle against those obtained from the indirect study of other cycles is obvious, and the use of the alumni design in this supplementary fashion seems desirable. In theory, the alumni design also provides for other cycles a replication of a direct study of immediate effect. I t may be modified so that approximate measures of initial status and status immediately after treatment are incorporated into the instruments. But this use, in our judgment, is incidental to the design and is dubious. I t is incidental because many cyclical programs of social action have a short cycle, the treatment is relatively short, and the interval between successive cycles is also short. Thus, the Encampment is a six-week program that recurs annually. The same feature characterizes many other programs of social action. I t does not require much time or many resources to conduct direct inquiry into immediate effects on new cycles. But no matter how short a cycle and how brief the interval between cycles, the study of long-range effects on any single group of subjects can never be speeded up. The crucial unanswered question for which the alumni design often provides the only solution is that of long-range stability. As a method of studying immediate effects, what is more important is that it is likely to yield distorted evidence and is dubious. We shall demonstrate this shortly. Where one does not have resources to study immediate effects on new cycles of a program, it is perhaps wiser to rely on a single direct inquiry than to place false confidence in the indirect evidence of immediate effects obtained from the study of alumni. No special difficulty attends the measurement of the present status of any alumni group, and the cumulative influence of events and time working upon the immediate changes produced by an original program is exactly what one wishes to know from studies of long-range stability. 3 Combining a short-term follow-up, as in our six-week inquiry, with one longrange study does provide two points and reveals the shape of the process. The shortterm follow-up may be recommended on this ground. It cannot, however, provide the detailed evidence on process that many alumni cohorts provide.

Longe-Range Effects among Earlier Alumni

301

By contrast, immediate effects must all be measured retrospectively in a study based on alumni. The experience itself is buried in the past and may be lost to memory. The status at the end of the program is also buried in the past. The status of any alumni group before the program requires an archaeological dig down to still an earlier level of experience, where the memory of respondents may be even more treacherous. Further, in studying immediate effects, one needs to control extraneous sources of change which operate simultaneously with the program, because the intrinsic effect of the program is precisely what one wishes to know. Only later does one ask whether the changes persist, perish, or grow in the context of other forces. Control-group procedures are beyond the scope of an alumni study. An inquiry deficient in so many respects is no substitute at all for direct studies of immediate effect. Perhaps it is wiser to rely on a single, full-scale, direct study of immediate effects than to place oneself in such jeopardy, and to strengthen the generality of conclusions by observations on the typicality of the particular cycle studied.4 By way of summary, the major uses of the alumni design are to evaluate long-range effects indirectly when circumstances prevent the evaluator from making any direct test, to supplement the findings of direct tests by evidence from other past cycles, to extend the time period within which long-range stability is studied, and to refine knowledge of the process. All these gains are achieved by comparing the present scores of alumni groups who vary in the length of time they have been removed from exposure to the original program and juxtaposing them against the immediate effects at the end of the program. Without this base line, misinterpretations might occur. Thus all alumni groups might be equal in current status, and yet changes might have occurred abruptly in all the groups within a short time interval after leaving the program. Unless the immediate effects were available as a point of reference, the analyst would not know whether there had been complete stability of the original gains or sudden great change which had then stabilized. The fundamental assumption that underlies the design is that the alumni groups that are compared were in point of fact equivalent 4 A memorandum was requested by the research workers on changes and trends over the ten Encampments, 1946-1955. Specific information was requested of the directors as to variations in program, staff, and type of student body recruited. Apart from minor changes in the vehicles used in the didactic program, and some variations in the demography of the group in the very early years, the only variation of possible importance occurred in the 1951 Encampment. During that Encampment the program grew more loosely and inductively out of the early days of informal discussion, in contrast with the more intense and formally organized pattern of other summers. Thus, it seems that the cycle within which our first rigorous evidence was obtained is typical, and this was borne out later by the 1957 and 1958 replications.

302

Applications of Methods of Evaluation

when they left the program. Only on this assumption may we regard the findings as a test of the influence of time on the stability of the original effects. If the assumption were not warranted, the present status of different alumni groups might simply reflect their respective past situations at the point they departed the program, and no safe conclusions could be drawn. How reasonable is the assumption? Although precedent is no guarantee of truth, it should be noted that designs of this very type are frequently reported in the literature of social psychology. "With minor modification appropriate to the features of evaluation, which reflects to our advantage, the design is a classic one. Thus, by way of illustration, Newcomb employed two procedures in studying, perhaps evaluating, the influence of protracted exposure to Bennington College. In the orthodox direct procedure he retested the same group of students as they progressed through college. As the first class to be tested in its freshman year was the class of 1939, the direct test required Newcomb to wait until 1939 to appraise his results. In the indirect test, he administered his scales in 1935 to all four classes in residence. Thus the group that had entered in 1932 was in its senior year in college in 1935 when Newcomb began his research. No delay was entailed in concluding that the influence of the Bennington environment was to reduce conservative ideology. Obviously, the assumption employed is that the different classes were initially equivalent in ideology. The parallel is obvious. In the classic use, initial equivalence is assumed; in the instance of evaluation of long-term stability, immediate effects are assumed to be equivalent. Although there are many similar designs in the literature, the Newcomb study is cited not merely as a precedent, but because it provides an empirical check on the validity of this design and an exact test of the assumption involved in all such designs.5 If the assumption were misleading, and the approach invalid, Newcomb would have drawn false conclusions. Yet the conclusion he drew was confirmed by the direct 5 In another example of such a design, the Cornell values studies, different college years are compared. One of the detailed analyses of these data is available in Norman Miller, "Social Class and Value Differences among American College Students" (unpublished Ph.D. dissertation, Columbia University, 1958). The method is scrutinized methodologically and some of Miller's data are summarized in Allen H. Barton, Studying the Effects of College Education (New Haven: Hazen Foundation, 1959), pp. 56-57, 61. Another illustration is available in current studies of the effect of medical education, where comparisons are made year by year in medical school. See Mary Jean Huntington, "The Development of a Professional Selfimage," in E. K. Merton et al., The Student Physician (Cambridge: Harvard University Press, 1957), pp. 179-187. Another illustration is provided by current research at Vassar on the effects of college education on personality. In addition to comparisons of the four college years, alumni who graduated between the years 1904-1943 are being studied. See J. Soc. Issues, XII, no. 4 (1956), passim.

Longe-Range Effects among Earlier Alumni

303

measurement of the 1939 class as it progressed from the freshman to the senior year." An exact test of the assumption of initial equivalence is also provided through certain comparisons within the direct follow-up studies. By way of illustration, the indirect test conducted in 1939 involves a comparison of the four classes in residence in 1939. As all these classes had also been subject to direct follow-up from the time they entered college, we can retrace each class back to its original freshman scores. The freshman scores of three of the classes are so close as to seem almost miraculous. One of the classes yields an obtained freshman score that is moderately different, but, as Newcomb remarks, this was owing to a deficiency in the sampling of the class at the time of the original freshman score.' The assumption of equivalence seems eminently correct. It should be emphasized, however, that a deficiency in sampling or other routine procedures may destroy the value of the design, as we shall again have occasion to note from our own experience with the alumni design. One aspect of Newcomb's study, like others in the literature, examines the influence of a program presented over time by comparing groups with differing length of exposure and assuming that initial scores before exposure are equivalent, an assumption supported by empirical inquiry. The use of the alumni design simply translates this logic to fit the needs of evaluation. By comparing groups differing in the length of time they are removed from contact with a program, rather than in the length of their exposure, and assuming equivalence at the point they depart, the stability of effects is examined. The minor variation when the design is employed for evaluation has two substantial advantages. In studies of some particular treatment presented over time, whether by direct or indirect means, one is concerned to measure its cumulative influence apart from other factors occurring in the same time period, such as aging and outside events. The comparisons reveal genuine change, but to unravel the determinants calls for additional procedures." As noted, however, in the evaluation of long-range stability one is concerned with the very influence of time and all its accompaniments, and 6 Additional confirmation is provided by the other classes. Although classes before or after the class of 1939 were not tested in each of the four years of their college career, every class beginning with the class of 1937 and ending with the class of 1942 was subject to some repeated testing. The direct examination of their development in successive years also confirms the indirect findings. See T. M. Newcomb, Personality and Social Change (New York: Dryden Press, 1942), p. 23. 7 The exact comparisons are presented in Newcomb, op. cit., table 1, p. 23. 8 Newcomb is able to establish that the Bennington experience is the important factor in a variety of ways. Thus, for example, the changes through the college years at Williams and Skidmore colleges also reflect the influence of events and aging, but in no way match the Bennington changes in magnitude. Ibid., pp. 30-33.

304

Applications of Methods of Evaluation

this is the only factor that is present, because the variable of the program is no longer operative. In using the design to study the influence of a program, there is always the danger of experimental mortality, to use the graphic term suggested by Campbell. As he puts it, "even if the groups were equivalent at some prior time, [they] may differ now not because individual members have changed, but because a biased subset of members have dropped out. This is a typical problem in making inferences from comparisons of the attitudes of college freshmen and college seniors."" To illustrate the point, in studying the influence of exposure to a certain college atmosphere, those who reject its values or who are hostile to the general direction of change endorsed may defect before reaching the senior year. If so, the subgroup that is examined as seniors would thus overstate the aggregate change that would be expected for unselected students going through all four years. Whereas the class in the aggregate would probably have had a freshman score equivalent to that of other classes in their freshman years, the surviving subgroup would probably not have had equivalent freshman scores. If such drop-outs, however, are few in number, for reasons uncorrelated with the phenomenon, such as illness or lack of funds, the problem is minor. When the design is employed with alumni from a program to evaluate long-range stability, the problem of experimental mortality does not exist. There is no defection from life! There is, however, as we shall see, an analogous problem that might be labeled sampling mortality, in that some biased subgroup of alumni may not be available for inquiry. Newcomb's study is most appropriate to our discussion for another reason. He, too, was concerned with the question of the long-range stability of the changes that occurred during exposure to Bennington, conjecturing, as we did, that the students would tend to "revert to home and family standards after graduation." He therefore made followups on three groups of alumni, those from the graduating classes of 1936,1937, and 1938. Having begun his investigations in Bennington in the fall of 1935, he had senior-year scores on all three of these alumni groups and could, therefore, determine the precise magnitude and direction of change by length of time removed from the exposure to Bennington, up to a maximum of three years' separation. To study the process for any time interval longer than three years required that he postpone the follow-ups to a date beyond 1939, as he did not have senior scores on any alumni group before the class of 1936. Because Newcomb ° Donald T. Campbell, "Factors Relevant to the Validity of Experiments in Social Settings," Psychol. Bull., LIV (1957), 300; italics ours.

Longe-Bange Effects among Earlier Alumni

305

was fortunate enough to make three direct tests for graded intervals of time after exposure to the program of Bennington, it is possible to recast his material so that it sheds light on the use of the indirect test with alumni groups. Let us suppose that Newcomb had not been fortunate enough to have senior scores on these groups. He then would have simply compared present scores for alumni at various distances removed in time. On the assumption that they were equivalent at the point of departure, any differences between alumni which varied with the time interval would have been the basis for a conclusion. In point of fact, he did develop one new battery of items which interested him greatly, and which he applied to these three alumni groups for the first time in 1939. For his new battery of items Newcomb concludes that the recent alumni show less conservative scores than the earlier alumni, that "Those longest out of college . . . correspond to present college freshmen."10 Now, by using the other scales, where both senior scores and present scores are available, it is possible to check this conclusion by comparing the directly observable changes with the changes shown merely by the alumni comparison or indirect test. Both approaches involving the old scales confirm the conclusion from the new battery of items: that those with the most time since college are the most conservative, and that some counterprocess is at work in the noncollegiate setting. There are, however, some obscurities in the data, which make the methodological comparisons rather difficult to interpret. In brief, Newcomb also observes in the direct test some unfinished process toward nonconservatism, generated while in college and continuing after departure from college. Those alumni groups longest away show none of this process, but the most recent alumni do exhibit it. At first the development toward less conservatism continues after college; then it is countered and subsequently reversed by progressive exposure to the home setting. Thus, the indirect test using alumni groups is not so sensitive as one would like, but is far from misleading. One test—even a multiple one—does not prove the argument. Our own test of the validity of this approach for evaluating long-range stability will provide additional support. Admittedly, confidence must be mixed with caution whenever this design is applied. We do assert, however, that the conditions of evaluation research are never ideal, that demands for orthodox designs may be impossible to realize, and that reasonable approximations may have to be employed. The danger in comparing nonequivalent groups may also be reduced in various 10 Op. cit., p. 79.

306

Applications of Methods of Evaluation

ways. Subgroups from different alumni years may be matched in various characteristics, reducing the likelihood that they differed initially." Certain alumni years may be excluded when there is reason to believe that recruitment, or staffing, or program had been modified, posing the danger that the particular group would not be comparable to others. Or such an alumni group may be combined with other adjacent years so as to attenuate the influence on the comparison of this undesirable situation.12 There is another assumption implicit in the design which may create some difficulty. "We should like to construe the comparison of early versus recent alumni as representing the cumulative impact of time on the original changes. But the flow of time may be punctuated by unique events. A particular alumni group might be altered by the impact of powerful events, and this would obscure the orderly influence of time interval. Here, again, the comparisons must be manipulated in such a way as to exclude such groups or to combine them with others, and interpretations must be reviewed in the light of such problems." Although these difficulties may well apply to a particular alumni group, it seems unlikely that all or many alumni groups will be afflicted with special circumstances, special experiences during the program, or unique critical events in the subsequent years. In applying this design to long-established programs, the past cycles yield a great many alumni groups which may all be efficiently studied, and such hazards are minimized. The properties of the method—its virtues and its deficiencies—have now been argued, but not proved. We shall now test it by reviewing the findings from our alumni study. The logical advantages may well be offset by difficulties that emerge in the employment of the procedure. 11 An example is available in Norman Miller's analysis of changing values during exposure to college. Comparisons of the different years are made separately for subgroups of students matched in various respects. u Thus, for example, the class of 1936 in Bennington was the first class to be admitted, and may have been unusual in entering a brand-new institution. The faculty did not reach its full membership until 1935. In our study, evidence indicated that the 1951 program was somewhat different from that of other Encampments, and this had to be taken into account in our analysis. " Newcomb notes that the class of 1936 had not only been out of college three years, but had experienced the Spanish civil war after graduation. The classes of 1937 and 1938 had experienced the Spanish war only while in college and only as it was defined by the college community. The 1936 group showed a conservative response on this question which reflected these special circumstances rather than the general influence of their three-year separation from college.

Longe-Bange Effects among Earlier Alumni

307

PROCEDURAL DIFFICULTIES AND ERROR RESPONSE ERROR

Inquiries after long intervals of time to establish the stability of the effects, whether by direct follow-ups of an original group or indirect designs involving alumni, will in most instances have to be conducted by mail questionnaires. The subjects of a program are the convenient captives of the researcher during their exposure. Afterward they scatter. Years later, when the problem of stability must be studied, they may be so scattered that mail inquiries offer the only economical approach that the evaluator can employ. The conditions under which the answers are obtained and the general quality of response cannot be as well controlled as in evaluations of immediate effects on the subjects of a program. But this deficiency is common to all approaches to the measurement of long-range stability. Such sources of error, however, could obscure the general findings. In comparing the direct and indirect methods, our main concern is that errors arising from faulty procedures shall not arbitrarily handicap one approach more than the other and thereby make the test of the intrinsic goodness of the indirect method misleading. In general, the procedures were equivalent. The questionnaire and covering letter used in the alumni study are presented in Appendix B. In both studies, the sponsor was identified as Columbia University. All the mailing was done from the university; the questionnaires were returned to the university. The respondents were assured that their replies would be kept confidential. Certain scales used were carefully disarranged so that the simulation of some consistent and virtuous attitude was made more difficult. Some of the questions were subtle in their intent. With respect to reliability, the use of the many different indices and scales of measurement rather than single questions should have increased the reliability of response. A s some protection, a small-scale pretest of the alumni questionnaire was conducted on a group not to be drawn into our final sample. All these safeguards should have reduced error. One special type of response error would beset the alumni method, if it were employed to replicate studies of immediate effect. The measures of the dependent variables, as they were before and just after the program, would involve retrospective questions back to distant points in the past and would be a severe test of memory. B y definition, in the direct follow-ups of a group, the earlier measurements of immediate effect exist. In our discussion, we assumed that there would be great danger of response error, and a test made in the alumni study does

308

Applications

of Methods of

Evaluation

establish the danger. As a check on the trustworthiness of retrospective reports, a "confusion control" was incorporated. Embedded in a list of events during the Encampments, which the alumni were asked if they remembered, were two fictitious events which nevertheless bore the stamp of verisimilitude. These had never occurred in any of the Encampments. Alumni who reported remembering these nonexistent occurrences might be regarded as doubtful witnesses. On the basis of the findings, we would place more or less confidence in their retrospections. The data presented in table 10-1 were the basis for our placing little credence in questions involving memory. TABLE

10-1

CREDIBILITY OF RETROSPECTIVE REPORTS OP EVENTS AS RELATED TO TIME AWAY FROM ENCAMPMENT

Alumni in our sample from Encampments held in

1946-1948 1949-1951 1952-1953 1954

Per cent who "remember" one or both of the nonexistent events 43 34 25 26

N

65 73 48 31

Response error increases regularly with time interval, and the retrospections of the alumni of the earliest Encampments are definitely untrustworthy. But even among the most recent alumni much caution is needed." The method is peculiarly vulnerable to error of memory when it is employed to replicate studies of immediate effect, and the problem may occasionally arise even when the method is employed to study longrange stability. The dependent variables are not subject to such error of measurement, as they refer to current status. Any hypotheses as to the factors that account for differential stability or change may, however, involve independent variables located in past years or during original exposure to the program. All such measures would be retrospective in an alumni study." In a direct follow-up, only those variables 14 Another test of error owing to retrospection was presented in chapter viii, where it was established for the 1958 group that accurate reports could be obtained. The time interval involved was less than a calendar year. This suggests, as does the regression in our table by distance in time, that the procedure may be employed within very narrow ranges of time. This hardly makes it useful in studies of many alumni groups from long-past cycles. K It should be noted that certain questions might be couched as if they were asking for memory reports of objective reality, but are intended as indices of states of

Longe-Bange Effects among Earlier Alumni

309

that referred to past years would be retrospective; those that referred to experiences during the program would probably have been measured at the original time. To evaluate long-range stability is certainly safe; to establish its causes by such modes of analysis is not so safe. SAMPLING PROBLEMS AND NONRESPONSE

The sampling design for our alumni study was simple. The Encampment office had an official mailing list, ordered by year, containing the names of all alumni, and the list was updated periodically on the basis of information about new addresses. From the total of about 1,000 TABLE

10-2

LOSSES IN SAMPLING OF ALUMNI FROM OFFICIAL MAILING LIST OF ADDRESSES Per cent with wrong address

Number drawn

Per cent with no addresB

193 170

28

2

1949-1951 1952-1953 1954

103 54

9 7 0

5 3 2

15

3

Alumni from 1946-1948

All years

520

names, a systematic sample of 520 names was drawn from the files by taking every other name. This yielded a sample of the various alumni groups from recent and earlier years as adequate in size as our resources permitted. The design created no difficulty, and there would be no bias, in principle, in making comparisons of alumni from the various years. The problem that immediately arose was that 75 of the 520 names selected had to be excluded because no current address was available. Table 10-2 establishes that the missing addresses are much more characteristic of the earlier alumni. Any attempt to make generalizations about effects would be likely to yield biased descriptions of the alumni from the earlier Encampments (1946 through about 1948). In addition, the addresses for another 15 names in the sample were reported by the postal system to be incorrect. The latter small group is uncorrelated with year of Encampment and probably not contributory to any bias. Altogether, 18 per cent of the original sample was not available. Those alumni for whom addresses are not available cannot be in contact with the Encampment; in turn, it feeling, e.g., a question asking an alumnus to describe the type of camper in his year. Here we make no inference as to objective reality, but rather one of psychological reality. The uncertainty of report is part of the process we wish to include.

310

Applications of Methods of Evaluation

seems reasonable that their involvement is probably less and perhaps their ideology and conduct are less in conformity with Encampment philosophy. Thus, any over-all description of effects for all alumni would probably overstate long-range stability. Effectiveness might also be overstated for the earlier alumni, because for them to remain in the current files of addresses is unusual and perhaps correlated with involvement, whereas it is most common for recent alumni to be correctly listed in the files. Thus, tests of the decline in stability with the passage of years would be obscured by the fact that those in whom extreme decline is to be measured, that is, the earlier alumni, are a peculiar subgroup, better insulated against the decline. One might argue that any erosion with time is probably an understatement, as the sample obtained from the early years would be most buttressed against erosion. If such are the findings, there is no problem of interpretation. But suppose we find no erosion? The findings would be ambiguous, for we might feel that, if only we had less motivated individuals in the obtained sample for the early years, erosion would have been demonstrated. The problem does not become acute, however, until we reach alumni separated by more than six years. This view is strengthened by the follow-up of the 1955 group after an interval of four years, when addresses were unknown for about 10 per cent of the eligible group. This corresponds to the losses resulting from missing addresses in the alumni groups from 1949 to 1951, also separated from the Encampment for an interval of four to six years, and to the losses among fairly recent alumni, separated for intervals of two to three years. This is to suggest that no matter which year of Encampment operations we study in its middle or later history as an organization, and no matter whether we study very recent or moderately old alumni, we have lost track of only about 1 in 10." However, the fact that only about 7 out of 10 of the earliest alumni, those separated for intervals of seven to nine years, are still listed suggests that this group available for inquiry is a biased and possibly unusually loyal group. The 430 alumni remaining in the sample, for whom addresses were reasonably current, received the questionnaires through the mails. The returns were very poor at first. Despite follow-up letters of a persuasive nature, final returns were obtained from only about half this group. w In Dentler's inquiry among alumni from various Friends' programs, separated by intervals ranging from one to seven years, he also sustained losses because of inadequate or missing addresses of 10 per cent. The consistency of the finding and the small magnitude of loss from this source is reassuring. R. A. Dentler, The Young Volunteers: An Evaluation of Three Programs of the American Friends Service Committee (Chicago: National Opinion Research Center, 1959), p. 41.

Longe-Range Effects among Earlier

311

Alumni

The distribution of these cases by alumni year is shown in table 10-3." The 49 per cent who did not reply to the questionnaires which must have been delivered to the correct addresses tend to be scattered in about equal proportions throughout the years, although nonresponse is at a maximum among the earliest alumni. The combined factors of inadequate addresses and nonresponse yield a final sample consisting of only 42 per cent of the original 520 names drawn; this is clearly a very poor basis for any estimate of long-range stability for alumni in the aggregateThe losses among the group who received the questionnaire should, of course, not be regarded as indicative purely of lack of interest TABLE

10-3

LOSSES IN SAMPLING OF ALUMNI D U E TO NONRESPONSE

Alumni from

Per cent of non-response in sample of correct addresses

1946-1948.. 1949-1951.. 1952-1953.. 1954

52 50 49 42

All years

49

or motivation. Some may have been away in the army or at college or on trips and the questionnaire might not have been forwarded. Others may have been ill. A number returned the questionnaire after the cutoff date for processing. But wisdom dictates great caution in using the final obtained sample for generalizations, because an unknown number of those who did not reply may be less motivated than the repliers, thereby overstating effectiveness when the repliers are used. Reference to our experience with the 1955 follow-up will underscore the argument. Among the 1955 group who received questionnaires, 86 per cent returned them. By comparison, among the alumni from 1949 to 1951, separated by about the same interval of years, only 50 per cent of those who received questionnaires returned them. Our various attempts at persuasion of the alumni were not so elaborate, but whatever the reason 11 In Dentler's inquiry, lack of funds precluded any follow-up on the initial mailing to alumni. Of those who received his first mailing, 44 per cent returned questionnaires. The response rate, considering the difference in follow-up attempts, is very similar. 76id., p. 41. 18 To give the reader proper perspective, it should be noted that a 42 per cent return is considerably higher than that usually achieved in ordinary mailings. For a summary of early studies on response rate to mail questionnaires see M. Parten, Swrveys, Polls, and Samples (New York: Harper, 1950), pp. 391-396.

312

Applications

of Methods of

Evaluation

for the poorer rate of return, clearly it was a highly selected group who finally responded. Assuming that the 1955 follow-up does not deal with an unusually cooperative cohort, one would have anticipated from the 1949-1951 alumni a mortality of about 15 per cent. Here, over and above this expected mortality, another 35 per cent were noncooperative, suggesting that the remainder who responded were an unusually motivated group. Although this particular alumni inquiry was beset by sampling biases of this magnitude, we may be heartened about its general future use, in the light of the nonresponse rate in the 1955 follow-up study. Clearly, it would be possible to obtain from alumni separated by as long as five years a very high response rate. And the fact that various groups of alumni—from the most recent to those separated for five years—show about the same proportions of loss owing to inadequate address and to lack of response suggests that comparisons of stability as influenced by length of time could be made safely, with the exception of the very early alumni groups. Granted that each alumni group is missing the least motivated members of the original group, so long as the remaining members who are available from each year are equally motivated the comparison by years is unaffected." There is even some empirical evidence that, despite the magnitude of loss in our inquiry and the differentially great losses among the earliest alumni, the various alumni groups remain comparable in their involvement with the Encampment. The greater losses in the early years may simply represent greater mobility,20 and may not have any motivational significance. If the alumni of the early years who returned questionnaires represented an unusually motivated group, we would expect them to show signs of greater involvement. In table 10-4 it is shown, to the contrary, that they exhibit much less involvement; they show greater "psychological distance" from the Encampment than recent alumni, which is exactly what one would w Dentler (op. tit., p. 45) reports the same finding. Nonresponse is distributed equally over his alumni groups contrasted in length of separation, suggesting that generalizations of aggregate effect are compromised, but also that the study of persistence over time is possible. 20 Eesidential mobility is especially high, in the younger age groups in the population. In one year, census estimates indicated that 29 per cent of the group aged 18-19 had changed residence, 43 per cent of those aged 20-24, and 35 per cent of those aged 25-29. After these ages, residential mobility progressively declines. The addresses of all alumni would therefore frequently become out of date, and perhaps the losses from our sample of earliest alumni would simply reflect the cumulative residential mobility with which the official lists had not caught up. (U.S. Bureau of the Census, Statistical Abstract [Washington, 1959], p. 36.) This speculation is supported by the 1955 follow-up, in which a complete history of residential mobility was obtained for the four-year period, 1955-1959. For this group, which would fall into the ages 20-24, 55 per cent had changed residences at least once during the period, and 7 per cent had lived in four or more places.

Longe-Bange Effects among Earlier Alumni

313

expect on grounds of the passage of time alone. Whether a more nearly complete sample ought to have exhibited even more psychological distance is imponderable, but at least we can establish that the earlier alumni are a group not especially involved in Encampment affairs, and that they are less involved than the recent alumni. THE FINDINGS AND THEIR AGREEMENT WITH THE 1 9 5 5 FOLLOW-UP

In examining the influence of time on long-range stability, it seemed desirable to combine adjacent alumni years. Each Encampment cycle yields only a small number of alumni, and the losses due to sampling TABLE

10-4

INDICES OF MOTIVATIONAL INVOLVEMENT IN THE ENCAMPMENT FOB SAMPLES OBTAINED FROM VARIOUS A L U M N I Y E A R S P e r cent of a l u m n i samples exhibiting various characteristics Index 1946-1948 (N - 65)

Agreement between current views and norms of original Encampment Little agreement on civil liberties Little agreement on race relations Had no kind of interaction in past year Report frequent thought about Encampment.

11 6 32 22

1954 1949-1951 1952-1953 (N = 7 3 ) | (N = 48)1 (N = 31)

4 3 34 38

9 10 8 56

7 0 3 74

difficulties made the obtained samples for individual years even smaller. We wished, however, to test the hypothesis that the confrontation of the community would produce an abrupt or rapid loss of immediate effects, and with this aim had conducted a six-week follow-up. To supplement the evidence on this hypothesis the alumni of 1954 were treated separately. If six weeks was not long enough to erode the immediate effects, one year should provide ample opportunity for demonstrating a quick reversal of original success.21 Thereafter, because the passage of time should work by gradual means, we pooled the alumni into three groups: the alumni of 1 9 5 2 - 1 9 5 3 , a relatively recent group, with a maximum of three years of separation; the alumni of 1 9 4 9 - 1 9 5 1 , the intermediate group, with a maximum of six years of separation; and the alumni of 1 9 4 6 - 1 9 4 9 , the earliest group, with a maximum of nine 21 The group separated for one year are also comparable to Kiecken's follow-up study ten. months after the departure of his subjects from the work camps. (H. W. Eiecken, The Volunteer Work Camp: A Psychological Evaluation [Cambridge: Addison-Wesley Press, 1952].) We were interested at many points in our study to compare findings from two rather similar action programs.

> •*» §§ « 11 SR fc P pi ä

o fe

H Q Z

z o M

m H Q

sn a ® s a

Q,lO ©»o _ ¡82« •a--Î

o

M

I a

co •o

O

00

s-s.2 s a £» a 3

o Oí

OS t^

co to

1Í3 00

00 N

M t-

(N Oï

00

co co

s

A"

+

a H

9

'S

+

+

+

« "O 11 0 a E'Ô£ S Is an

1a S«

€=3

M

¡B S P

15

A

+

3 2 i-H

o O5

s."

o co

00

+

W •J m




H

a — a> a °k< .3 ¡»H

fi ^ S J M «Tl o>

3>

S o

z «¡

a o -o 3 -tí ' SO ® ÍO

m a

Pi Ò

z o

0)

x>

p-3

S œ .-S Ú h 3 S M o h Ë t. ®

A

•3 H

®

4) t - S •2 ä —i O

" S

3 .5 'S 03

- s ®> 3 kl O « s * >> 3 . 3 "S .5. •S "S fct œ m a.-I I S 5 " ! Pi

S i fl w O TS '-3 3 .S ~ œ o » œ a

•H ö .s ë • M a O ¡5 sï

S wt a O

3 T3 ® roi• aV > m fl •S » 3 te O O cj S XI « > 1 . 2 "S h » •m S o " 1 S O £ g H

V a

° .2 t» T-l d O -O 3 -S

-

S.B S « f& w J3T5 u Ss d s £ £a •o

3S

œ a) .2 a •o £> sa a a «g 5 S Ï.2 o* 3 •s s E 2 gS .1 ® 'S« . JNü

Hl

I l i

® O ffi m fl

3 o1 C O a o

-2 a> e -2 í S o z

s s s

>> —ja a •-•S'a "•s S -S*3 H.2 M «•S-S S t S ° S. m

Longe-Bange Effects among Earlier Alumni

315

years of separation. The one Encampment where there was some report of differences in official program was that of 1951. By pooling it with two other years, this possible source of difficulty in. the comparison is diluted. Even with these grosser groupings of alumni years, necessitated by the exigencies of the sampling, the smallness of the Encampment enrollment, and the need to attenuate the possible noncomparable elements in the 1951 cycle which might influence the findings, considerable refinement remains. The process can still be examined at four points spaced out in time. Changes over time need an elaborate context for interpretation. Comparison with the status of the 1955 group immediately after exposure is essential as the point of reference for assessing stability. Other possible norms would be the status of the 1955 group at entry into the program; still another point of reference is provided by national norms from the general population or some appropriate segment of it. In chapter ix such standards for evaluating long-range change were employed. By their use, we supplement the mere demonstration of change over time. We reveal not only long-range change, but show whether or not the changes work to cancel all the effects originally produced by exposure, or even to make an originally deviant or elite group indistinguishable from the mass.22 In the tabular presentation of the alumni findings for the different years, we do not present such norms. They were, however, taken into account in our original interpretation of the results. Here our real concern is to compare the findings of the indirect test, using the alumni design, with the findings obtained from the direct follow-up of the 1955 group after four years. In table 10-5 findings based on both approaches are presented for a number of selected variables. Two aspects of the findings are relevant to the validation of the indirect method. A quantitative comparison could be made. The score of the 1955 group after four years could be compared with the score of the 1949-1951 alumni group, which, when measured in 1955, had been separated by an interval of four to six years. Ideally, the 1955 group's current status should be compared only with the 1951 group, but, as noted, the number of cases would have been very few and the 1951 Encampment was peculiar in certain respects. We preferred to sacrifice the exact equivalence of the time interval to avoid the errors that otherwise would have crept into the comparison of the two methods. Because all the measurements of status, direct or indirect, are subject 22 Newcomb, it will be recalled, compared the status of his alumni with freshman norms to see if the entire change was erased.

316

Applications of Methods of Evaluation

to a variety of errors of sampling or response, and because the 19491951 group is not exactly equivalent, a much more meaningful validation of the alumni design is to compare the general conclusions on stability which would be drawn from the two different tests. We shall see whether or not the alumni design is misleading. These results from the alumni study were available in 1955-1956, and comparisons of the changes over time, starting from the base line of the 1955 group's status at the end of the Encampment, were the basis for conclusions reported to the agency. As noted earlier, one of the virtues of the design is that the evaluation of long-range success does not have to be postponed to a remote future date pending the findings of a direct follow-up. "We shall compare the conclusions that were drawn then with those now available from the 1955 follow-up.23 In this manner we shall avoid any retrospective falsification that might creep in now in the light of hindsight and our knowledge of the direct test. On the three measures of attitude in the realms of tolerance and civil liberties, we had concluded that "there is no evidence of any erosion of campers' democratic sentiments as the years go b y . . . . There is no systematic drop in the level of democratic attitudes between the group just graduated from the Encampment and groups of alumni who have been away for one to nine years." These conclusions agree in the main with the findings from the 1955 follow-up and the conclusions therefrom which were presented in chapter ix. The alumni study, however, does not show any drop in tolerance for a "peacetime Communist," whereas the follow-up reveals a moderate drop. On cognition of social problems, we had concluded that "time does alter initial optimism about social problems. Two areas seemed most sensitive to the impact of time [and these were] cognition of race prejudice and war as social problems capable of ready solution." Again, our follow-up conclusions presented in chapter ix agree in the main with these earlier findings. Dentler's findings on long-term stability of effects produced by various Friends' programs agree with the findings of our direct followup and our alumni study. Comparing the scores of the group immediately after exposure to 1958 programs with the average scores of alumni of the Friends' programs separated from one to seven years, Dentler notes that for many of the same attitude scales that we used there is no loss of effect. To the contrary, he observes that his alumni make scores superior to those of his current group.24 He suggests that 23 H. Hyman and C. Wright, "Youth in Transition: An Evaluation of the Contribution of Encampment for Citizenship to the Education of Youth," Columbia University, Bureau of Applied Social Eesearch, March, 1956, mimeographed. 24 Op. oit., passim.

Longe-Bange Effects among Earlier Alumni

317

the long-term process is one not merely of stability, but of continuing political and social maturation. Such subjects "tend to mature politically with impressive speed."55 Although the conceptualization and the measurement of dependent variables or effects of Encampment in the alumni study were parallel to those employed in the 1955 follow-up, the fit is not perfect. The alumni coverage was somewhat shorter because of fear of loss of cooperation. A few of the variables and the instruments employed in the alumni study were dropped from the 1955 follow-up, either because their intrinsic interest had waned or because reflection and analysis led to their rejection. The phenomenon of alienation was complex and elusive to measure, and considerable improvising and change occurred. There is one comparable datum, however, based on the question that asks the camper what view on tolerance toward a peacetime Communist he would attribute to the average American. Here, too, agreement between the alumni findings and the follow-up is high. In the earlier report it was concluded that "there is no sizable systematic increase or decrease in reluctance to stereotype average citizens.... Among those alumni who are willing to express a judgment on the matter, there is no evidence of any systematic 'harshening' or 'softening' of the image of the average citizen." The same conclusions were drawn when this question was employed in the four-year follow-up study. In the 1955 follow-up study various aspects of conduct were measured. As previously noted, the question of change or stability in conduct is not very meaningful, as many forms of conduct, such as voting, for example, cannot be studied in the Encampment setting or require a certain stage of adult development before they can occur. Thus, no base line from which to measure subsequent change is available. Bather, the problems are to determine whether the young persons ultimately exhibit various forms of conduct which are regarded as desirable by the Encampment, and, if possible, to trace the contribution of the program to these adult outcomes. These problems were examined for the 1955 group after a four-year interval. In an evaluation making use of the alumni design, the investigator would naturally be interested in these problems, and we had, therefore, incorporated them into our inquiry. It is instructive to compare the results obtained by the two approaches. As the issue is not one of change over time, we shall simply compare the status of the 1949-1951 alumni in 1955 with the status of the 1955 group in 1959 to see how well the two methods agree. The two groups are separated from their respective experiences in the program by about 26

Ibid., p. 140.

318

Applications

of Methods

of

Evaluation

the same interval, and both are approximately equated in age, an important correlate of political behavior. The data are presented in table 10-6. There is considerable agreement in the findings of the two methods. Comparisons of the two methods are possible for a few other areas of conduct. In both inquiries, voluntary association membership was determined. After four years, 71 per cent of the 1955 group belong to TABLE

10-6

POLITICAL BEHAVIOR SUBSEQUENT TO THE PROGRAM AS REVEALED B T THE ALUMNI S T U D T AND THE DIRECT FOLLOW-UP

Index

Voted, among eligibles, in Any state election Any local election 1954 congressional11

1949-1951 a l u m n i f o u r years later

1955 group f o u r years later

Per cent

N«

Per cent

N*

66 61 70

56 57 61

67 72

55 58

70

53

80

35

75 59

73 73

78 86 45

36 73 73

or

1958 congressionalb 1952 presidential1» or

1956 presidential1" Have talked politics with someone recently.. Have written to government official

» See note o, table 9-4. b T h e s e pairs of elections are t h e f u n c t i o n a l l y e q u i v a l e n t elections for t h e respective g r o u p s of c a m p e r s . F o r example, t h e 1054 congressional election occurred for t h e a l u m n i t h r e e years a f t e r exposure t o t h e E n c a m p m e n t , a n d t h e 1958 congressional election was t h r e e years after exposure for t h e 1955 g r o u p . B o t h groups were a p p r o x i m a t e l y t h e s a m e age a t t h e t i m e of these respective elections.

at least one voluntary association, and 34 per cent belong to three or more associations. Among those who are members, nearly half hold office. For the alumni from 1949 to 1951, the findings are similar. At the end of four years, 84 per cent belonged to at least one association, and 47 per cent belonged to three or more associations. Of those who are members, 42 per cent were officers. In both inquiries, attempts to apply Encampment principles in action were enumerated. As noted in chapter ix, 58 per cent of the 1955 group reported such attempts in the four-year period, and 41 per cent of the attempts actually were meaningful demonstrations of conduct. For the 1949-1951 alumni group, the respective figures are 48 per cent and 40 per cent. After screening the dubious examples of application of Encampment principles, the agreement is very high.

Longe-Bange Effects among Earlier Alumni

319

Vocational goals and actual occupations constitute the final area of conduct for which there are comparable data. At the time they left the Encampment, most members of the group were too young to have actual occupations, and therefore a base-line measure was not meaningful. All that could be demonstrated by the direct follow-up was the proportion who had entered service-oriented or politically oriented careers by 1959, these types of careers being consonant with the Encampment's ideology. As noted in chapter ix, professional careers are essentially a function of age and time, and a single follow-up study after only four TABLE

10-7

VOCATIONAL GOALS AND CAREERS AS REVEALED BY THE A L U M N I STUDY AND THE DIRECT FOLLOW-UP

(Percentages) A l u m n i sample (1955) f r o m t h e following E n c a m p m e n t Index

Current occupations Service careers Political careers Not in labor force (housewives, students, military) Current vocational goal Want service career Want political career

1955 c a m p e r s September, 1955 (N = 73)

1954

1952-1953

1949-1951

1946-1948

(N = 31)

(N = 48)

(N = 73)

(N = 65)

3 0

21 0

26 4

31 3

20 5

78

62

45

32

50

53 10

48 6

41 5

39 5

40 12

1959 (N = 73)

40 3

years is not a very sensitive measure of the ultimate success of the Encampment in fostering such types of careers. In this respect the indirect alumni design is more sensitive, because it carries the career line to a later stage of time and adult development and permits internal comparison for various points in time. As table 10-7 reveals, the 19491951 alumni group and the 1955 group, both taken after a separation of about four years, show about the same proportions who have entered service or political careers. The alumni study, however, provides a much more comprehensive background for interpreting the finding, for comparison of the four groups reveals a progressive contribution of alumni to service-oriented careers as time permits them to enter the labor force. Career goals were studied in 1955 and again in 1959, and the direct study revealed stability in the goal of a service-oriented career, and a

320

Applications

of Methods of

Evaluation

moderate decrease in the goal of a politically oriented career. The alumni study shows somewhat the same pattern, although there are some inversions in the function for the most recent alumni. Both approaches to the measurement of long-term stability may be used to test specific hypotheses as to the factors that account for differential stability or change. In the direct follow-up, amount of change was compared for subgroups varying in their exposure to disruptive environments or their experience of social and psychological support. In the alumni study, each alumni group may be broken down with referTABLE

10-8

D E C L I N E IN ALTJMNI'S INTERACTION AND THE ENCAMPMENT R E F E R E N C E GROXJP WITH Y E A R S OF SEPARATION

(Percentages) Class of alumni 1954 1952-1953 1949-1951 1946-1948

N

Not in interaction in past year

With frequent thoughts of Encampment

31 48 73 65

3 8 34 32

74 56 38 22

ence to similar factors. Then, inferentially, differential change over time may be estimated by comparing early and recent alumni who do or do not have exposure to the particular factor. The number of alumni who exhibit particular disruptive characteristics and combinations of characteristics varies so much in the two studies that only one specific hypothesis was duplicated in the two tests. Both approaches document that the "gains" of the Encampment are not reduced by the disruptive factor of residential mobility. In both studies, a test also was made of the influence of a combination of extremely disruptive conditions on long-range stability. The actual content of the index of disruptive conditions is quite different in the two studies, however, and no direct comparison of the two approaches is possible. Contrasted with hypotheses in the direct and in the alumni study which dealt with factors that would contribute to long-range loss were hypotheses about reference groups and interaction as factors contributing to stability. Parenthetically, it is of interest to recall that interaction with members of the Encampment and its maintenance as a reference group persisted in the 1955 group despite a four-year interval. Thirtysix per cent reported frequent thoughts of the Encampment, and only 27 per cent reported no interaction whatsoever with their group in the

Longe-Bange Effects among Earlier Alumni

321

fourth year after separation. Among the alumni, it is possible to trace interaction and reference-group maintenance over longer ranges of time, in order to see if such cohesiveness can withstand long periods of separation. For the comparable alumni cohort, those from 1949 to 1951, the findings are similar. Thirty-eight per cent report frequent thoughts of the Encampment and 34 per cent report no interaction in the past year. When the process is considered for contrasted groups of alumni separated from one to nine years, one notes a decline in interaction and in the maintenance of the Encampment reference group. The data are summarized in table 10-8. In the follow-up study, the presence or the lack of any type of interaction did not affect the maintenance of the camper's attitudes. The same finding was obtained over many tests from the alumni study. Stability for alumni separated for short or long periods was not dependent on kind or amount of interaction. In the follow-up study there was some evidence that interaction contributed to action consonant with Encampment principles, but the findings were not consistent on all the tests made. The alumni study yielded similar findings. Those who continued to have face-to-face interaction did act more on Encampment principles. In the follow-up study, maintenance of the Encampment as a positive reference group had no effect on the stability of attitudes, but contributed to the likelihood of action. In the alumni study, the findings were equivocal, and no safe conclusions may be drawn. The comparisons of the alumni design and the direct follow-up as tests of long-range stability, and of the factors related to differential stability, have now been presented in summary form. Where the direct follow-up can be incorporated into an evaluation, it is certainly to be preferred. Considering the exigencies that will confront most evaluators trying to assess long-range stability, it is most fortunate that our analysis lends confidence in the validity of indirect designs employing alumni from past cycles of a program.

APPENDIX A

ATTITUDE SCALES U S E D I N MEASURING EFFECTS OF E N C A M P M E N T FOR CITIZENSHIP AUTHORITARIANISM SCORE T H I S SCORE is the sum of the scores of the eleven items below. Subjects responded to these items on a 6-point scale as follows:

+ 1 : 1 AGREE A LITTLE + 2 : 1 AGREE PRETTY MUCH + 3 : 1 AGREE VERY MUCH

- 1 : 1 DISAGREE A LITTLE - 2 : 1 DISAGREE PRETTY MUCH - 3 : 1 DISAGREE VERY MUCH

The sign of the response to the starred item (*) was reversed. The number 4 was then added to each response to avoid negative values. (Occasional unanswered items were scored as 4.) Whenever the respondent had written in two numbers, the first was used. "When large blocks of questions had been left unanswered, the score was called YY unascertainable). The resulting sum ran from a theoretical low of 11 (least authori tarian) to a theoretical high of 77 (most authoritarian). Human, nature being what it is, there will always be war and conflict. Obedience and respect for authority are the most important virtues children should learn. No weakness or difficulty can hold us back if we have enough will power. Science has its place, but there are many important things that can never possibly be understood by the human mind. Young people sometimes get rebellious ideas, but as they grow up they ought to get over them and settle down. What this country needs most, more than laws and political programs, is a few courageous, tireless, devoted leaders in whom the people can put their faith. It is essential for learning or effective work that our teachers or bosses outline in detail what is to be done and exactly how to go about it. Some leisure is necessary, but it is good, hard work that makes life interesting and worthwhile. When you come right down to it, it's human nature never to do anything without an eye to one's own profit. •Generally speaking, the less government we have the better off we will be. When a person has a problem or worry, it is best for him not to think about it, but to keep busy with more cheerful things. DEMOCRACY SCORE

This score is the sum of the scores of the fifteen items below. The response categories and the scoring are identical to those described for [325]

326

Appendix

A

the authoritarianism score. The sign of the response to the starred item (*) was reversed. Here the resulting sum ran from a theoretical low of 15 (most democratic) to a theoretical high of 105 (least democratic). Usually i t is not feasible f o r all members of a group t o take an equal interest a n d share in the activities of t h e group. Almost a n y job t h a t can be done by a committee can be done better by having one individual responsible f o r it. I n case of disagreement within a group the judgment of the leader should b e final. The best criterion f o r j u d g i n g any technique f o r dealing with other people is in terms of how efficiently it will get the job done. There are o f t e n occasions when an individual who is p a r t of a working group should do what he thinks is right regardless of what the group has decided t o do. I n most practical situations, t h e more experienced members of a group should assume responsibility f o r t h e group discussion. Sometimes one can be too open-minded about the possible solutions to a problem t h a t f a c e s a group. I n a group t h a t really wants to get something done, the leader should exercise f r i e n d l y b u t firm authority. Discipline should be the responsibility of the leader of a group. I t is sometimes necessary to use autocratic methods to obtain democratic objectives. I t is not always feasible t o t r y to be consistent with one's ideals in everyday behavior. F i g h t i n g to p u t one's ideals into practice is a luxury t h a t only a f e w can afford. Generally there comes a time when democratic group methods must be abandoned in order to solve practical problems. *In a democratic group, regardless of how one feels, he should n o t withdraw his supp o r t f r o m the group. Sometimes it is necessary to ignore the views of a f e w people in order to reach a decision in a group. POLITICAL-ECONOMIC CONSERVATISM SCORE

This score is the sum of the scores of the ten items below. The response categories and scoring are identical to those described for the authoritarianism score. The signs of responses to the starred item (*) were reversed. The resulting sum ran from a theoretical low of 10 (least conservative) to a theoretical high of 70 (most conservative). * I t is u p t o the government to make sure t h a t everyone has a secure job and a good s t a n d a r d of living. *The government should own and operate all public utilities (railroad, gas, electricity, etc.). Men like H e n r y F o r d or J. P . Morgan, who overcame all competition on the road to success, are models f o r all young men t o admire and imitate. *The only way to eliminate poverty is to make certain basic changes in our political and economic system.

Appendix

327

A

*In a new t a x program i t is essential not to rdeuce the income taxes on corporations and wealthy individuals. I n general, f u l l economic security is b a d ; most men wouldn't work if they didn't need money f o r eating and living. *There should be some upper limit, such as $25,000 per year, on how much a n y individual can earn. The t r u e American way of l i f e is disappearing so f a s t t h a t force m a y be necessary to preserve it. *Most of the present attempts to curb and limit unions would in the long run do more h a r m t h a n good. The businessman and the m a n u f a c t u r e r are probably more important to society t h a n the artist and t h e professor. STEREOTYPY SCORE

This score is the sum of the scores of the nine items below. The response categories and scoring are identical to those described for the authoritarianism score. The resulting sum ran from a theoretical low of 9 (least use of stereotype) to a theoretical high of 63 (greatest use of stereotype). Generally speaking, Negroes a r e born with more musical talent than white people. F a r m e r s a r e more self-sufficient t h a n city people. City people are less f r i e n d l y t h a n small-town people. City people are more radical t h a n country people. Most labor union officials are the sons of immigrants. Most of the P u e r t o Ricans in New York City came f r o m poor f a r m families on the island. T o u can recognize a New Yorker b y his aggressive manner. Southerners are generally lazy. Most Midwesterners are isolationists. CONSTITUTION SCORE

This score is the sum of weights assigned as indicated to responses to the seven items below. When large blocks of questions were left unanswered, the score was called Y (unascertainable). A score of 10 or higher was called X. The resulting sum ran from a theoretical low of 0 (pro-Constitution) to a theoretical high of 14 (anti-Constitution). The general public is not really qualified to vote on today's complex issues. Allowing just anyone—including uneducated people—to serve on a j u r y is a poor idea. Decisions to protect the government may have to be made even a t the expense of one person.

Agree

Disagree

f

N o answer

2

0

1

1

2

0

1

1

2

0

1

1

328

Appendix A

Unrestricted freedom of speech leads to mass hysteria. In passing laws about issues like price controls and taxation, Congress should pay more attention to what the experts say than to what the public says when they differ. Democracy depends fundamentally on the existence of free business enterprise. Whatever serves the interest of the government best is generally right. CIVIL RIGHTS SCORE

This score is the sum of weights assigned as indicated to responses to the five items below. When large blocks of questions were left unanswered, the score was called Y (unascertainable). A score of 10 was called X. The resulting score ran from a theoretical low of 0 (least discriminatory) to a theoretical high of 10 (most discriminatory). People of different races should not dance together. Swimming pools should admit people of all races and nationalities to swim in the same pool. There should be laws against marriage between persons of different races. Hotels are right to refuse to admit people of certain races or nationalities. Pupils of all races and nationalities should attend school together everywhere in this country.

Agree

Disagree

f

No answer

2

0

1

1

0

2

1

1

2

0

1

1

2

0

1

1

0

2

1

1

ACTION-APATHY SCORE

This score is the sum of weights assigned as indicated to responses to the items below. Only the first three were used in 1955. The resulting score ran from a theoretical low of 0 (least apathetic) to a theoretical high of 6 in 1955 or 10 in 1957-1958 (most apathetic). Sending letters to congressmen has little influence on legislation. If a person is uncertain how to vote, it is better if he does not vote. The average citizen is justified in remaining aloof from dirty politics that may exist in his community. A good many local elections aren't important enough to bother with. I t isn't so important to vote when you know your party doesn't have a chance to win.

Agree

Disagree

2

0

2

0

2

0

2

0

2

0

1

No answer

Appendix

329

A

ANOMIE SCORE

This score is the sum of weights assigned as indicated to responses to the eight items below. When large blocks of questions were left unanswered, the score was called Y (unascertainable). A score of 10 or higher was called X. The resulting score ran from a theoretical low of 0 (least anomie) to a theoretical high of 16 (most anomie). Most people cannot be trusted. Since 1890 people's ideas of morality have changed a lot, but there are still some absolute guides to conduct. Nobody cares whether you vote or not exexcept the politicians. Nobody cares whether you attend church or not except the clergy. People talk a lot about being decent to Negroes and other minority groups, but when it comes right down to it, most people don't really care how you treat these groups. Cheating on incomes taxes is nobody's business but the government's. What really made Dr. Salk work so hard on the polio vaccine was the thought of the money or fame he would get. Kinsey made his name with his report on sexual behavior, and the hope of publicity was what really led him to study sex in the first place.

Agree 2

CIVIL LIBERTIES

Disagree 0

? 1

No answer 1

1

1

1

1

1

1

A SCORE

This score is the sum of weights assigned as indicated to responses to the six items below. When large blocks of questions were left unanswered, the score was called Y. A score of 10 or higher was called X. The resulting score ran from a theoretical low of 0 (mos libertarian) to a theoretical high of 12 (least libertarian). Agree Only trained and competent people should be permitted to run for public office. People who talk politics without knowing what they are talking about should be kept quiet. Certain groups should not be allowed to hold public meetings even though they gather peaceably and only make speeches.

Disagree

No answer 1

330

Appendix A

In some criminal cases, a trial by jury is an unnecessary expense and shouldn't be given. Persons who refuse to testify against themselves (that is, give evidence that would show that they are guilty of criminal acts) should either be made to talk or be severely punished. Some of the petitions which have been circulated should not be allowed by the the government.

2

2 2

This score is the sum of weights assigned as indicated to responses to the five items below. When large blocks of questions were left unanswered, the score was called Y. A score of 10 was called X. The resulting score ran from a theoretical low of 0 (most libertarian) to a theoretical high of 10 (least libertarian). Agree

Disagree

1

No answer

In some cases the police should be allowed to search a person or his home even though they do not have a warrant. Newspapers and magazines should be allowed to print anything they want except military secrets. If a person is accused of a crime he should always have the right to know who is accusing him. In some criminal trial by jury is an unnecessary expense and shouldn't be given. The right of some working groups to call a strike should be abolished, as it is a threat to democracy and not in the general interest of society. ABSOLUTISM SCORE

This score is the sum of weights assigned as indicated to responses to the six items below. When large blocks of questions were left unanswered, the score was called Y. A score of 10 or higher was called X. The resulting score ran from a theoretical low of 0 (least absolutistic) to a theoretical high of 12 (most absolutistic).

Appendix Slum housing is the cause of most juvenile delinquency. Lack of education is the cause of most race prejudice. Biased news coverage is mainly due to the pressure of advertisers. Popular taste would be raised tremendously if the producers of radio and television had some decent programming. Class distinctions would disappear if economic opportunities were made more equal. A community-wide mass-media campaign could really sell the public on the goals of the UN.

331

A

Agree

Disagree

?

No answer

2

0

1

1

2

0

1

1

2

0

1

1

2

0

1

1

2

0

1

1

2

0

1

1

TOLERANCE SCORE

To obtain this score, weights were assigned as indicated to responses to the six items below. Their sums were transformed as follows: 15,14, 11, 10 13, 12, 9, 8 7, 6, 5, 3, 2 4, 1 0

1 2 3 4 5

If there were more than three "no" answers, responses were scored anyway and double-punched with X. If there was no answer to the entire scale, the score was called Y. A final score of 1 indicated greatest tolerance, and of 5, least tolerance. Agree A person who wants to make a speech in my community against churches and religion should be allowed to speak. A book favoring government ownership of all the railroads and big industries should be removed f r o m the public library if someone in the community suggests this. I would stop buying the soap advertised on a radio program that had an admitted Communist as the singer if someone suggested this to me. An admitted Communist should be put in jail. A book against churches and religion should be removed f r o m the public library if someone in the community suggests this.

1

Disagree

1

No answer

0

0

0

0

4

0

0

0

4

0

0

0

1

0

0

0

1

0

0

332

Appendix

A person who wants to make a speech in my community favoring government ownership of the railroads and big industies should be allowed to speak.

A

4

0

0

0

INDIVIDUAL POLITICAL POTENCY SCORE

This score is the sum of weights assigned to responses to the question below. The answer "A great deal" is given the weight of 2, "A moderate amount" is weighted as 1, and "Hardly anything" as 0. The resulting sum ran from a theoretical low of 0 (minimum potency) to a theoretical high of 10 (maximum potency). If the whole question was omitted, the score was not computed. Unanswered parts of the question were scored 0. In general, do you think that the individual citizen can do a great deal, only a moderate amount, or hardly anything at all about the following matters ? A great A moderate Hardly deal amount anything Prevention of war 2 1 0 Reduction of corruption in national government.... 2 1 0 Reduction of corruption in local government 2 1 0 Improvement of housing 2 1 0 Improvement of race relations 2 1 0 GROUP POLITICAL POTENCY SCORE

This score is the sum of weights assigned to responses to the question below. The answer "A great deal" is given the weight of 2, "A moderate amount" is weighted as 1, and "Hardly anything" as 0. The resulting sum ran from a theoretical low of 0 (minimum potency) to a theoretical high of 10 (maximum potency). If the whole question was omitted, the score was not computed. Unanswered parts of the question were scored 0. How about groups of people, or clubs? Can they do a great deal, only a moderate amount, or hardly anything at all about these matters? A great A moderate Hardly deal amount anything Prevention of war 2 1 0 Reduction of corruption in national government.... 2 1 0 Reduction of corruption in local government 2 1 0 Improvement of housing 2 1 0 Improvement of race relations 2 1 0

Appendix

333

A

ETHNOCENTRIC IMAGE OF DEMOCRACY SCORE

This score is the sum of the weights assigned to responses to the question below. The resulting sum ran from a theoretical low of 0 (least ethnocentric) to a theoretical high of 9 (most ethnocentric). If the whole question was omitted, the score was not computed. In some democracies there are certain customs or laws which you personally might or might not regard as democratic. Look over the following list and check whether each practice appears thoroughly democratic, somewhat undemocratic but tolerable in a democracy, or thoroughly undemocratic in your personal opinion: This practice seems to me Thoroughly Undemocratic Thoroughly democratic but tolerable undemocratic In Australia, every citizen is required by law to vote. Until recently, in Sweden, most babies were automatically registered as members of the Lutheran Church, the state religion. Until recently, in Sweden, the amount of alcohol that a person could buy each month was regulated by law. In France, you must pay for a license in order to have a radio set in your home. In the United States, all children must be vaccinated against smallpox In Great Britain, under the law, doctors no longer collect their fees from patients but bill the government. In Norway, any Jesuit is prohibited by law from entering the country. In Japan, all school children wear uniforms. In the United States, one may legally have only one husband or wife at a time.

0

0

0

0

0

,

0

0

0

0

0

0

0

0

0

0

0

0

0

CAMPUS ACTION-APATHY SCORE

This score is the sum of weights assigned as indicated to responses to the six items below. When large blocks of questions were left unanswered, the score was called Y. A score of 10 or higher was called X. The resulting score ran from a theoretical low of 0 (least apathetic) to a theoretical high of 12 (most apathetic).

334

Appendix A Agree

Disagree

1

No answer

Organized discussions of politics among students mainly allow those already convinced to air their beliefs. Even if their course work suffers somewhat, college students should take part in civic and political activities. Campus political activities should be regarded as vital to a well-functioning college. One should not bother to sign student petitions because nobody pays any attention to them anyway. Hearing well-known people at college on current events is as important as attending class. Because political activity as a student may jeopardize one's subsequent career, students ought not to take part in campus political organizations. COLLEGE TOLERANCE SCORE

This score is the sum of weights assigned as indicated to responses to the seven items in the question below. When large blocks of items were left unanswered, the score was called Y. A score of 10 or more was called X. The resulting score ran from a theoretical low of 0 (least tolerant) to a theoretical high of 14 (most tolerant). If the students of a college want to join them, which of the following organizations do you think ought to be allowed on their campus, which not allowed, and which are you not sure whether it should be allowed or not? Allowed Young Democratic Club 2 Young Republican Club 2 Young Socialist League 2 Young Communist League 2 Students for Peace 2 Students for America 2 Students for Democratic Action. 2

Not allowed 0 0 0 0 0 0 0

Not sure

Don't know 1 1 1 1 1 1 1

1 1 1 1 1 1 1

No answer 1 1 1 1 1 1 1

COLLEGE COMMITMENT SCORE

This score is the sum of weights assigned as indicated to responses to the items in the question below. When large blocks of items were left unanswered, the score was called YY. The resulting score ran from a theoretical low of 0 (most committed) to a theoretical high of 22 (least committed).

Appendix

A

335

Here are some reasons that students who left college gave for not finishing. Which do you personally feel are reasonable grounds for leaving college? Which do you feel are not reasonable grounds? Not It Reasonable reasonable depends No Parents object that college is doing student no good. 2 0 1 Is bored by studying but finds real satisfaction in working with his hands. 2 0 1 Wants a job helping people who have problems and cannot see how spending two or three more years in studying will be of much use. 2 0 1 Wants to get married and raise a family. 2 0 1 Is needed to help support his (her) family 2 0 1 Feels the wide world provides a better education than do colleges. 2 0 1 Considers his (her) grades too low to warrant continuing in college. 2 0 1 Is offered a well-paying position in a reputable company having a good promotion policy. 2 0 1 The expense of going to college is too great. 2 0 1 Prefers to work full time in order to be financially independent of parents. 2 0 1

answer 1 1

1 1 1 1 1 1 1 1

APPENDIX B

QUESTIONNAIRES

from the 1955 phase of the study are reproduced here. There are six of them:

QUESTIONNAIRES

I . Mailed to a sample of applicants in May, 1955. I I . Administered to f u l l Encampment a t the beginning of the Encampment in July, 1955. I I I . Administered to f u l l Encampment at the end, in August, 1955. I V . Mailed to campers in September, 1955. V. Mailed to campers in March, 1959. V I . Mailed to alumni of Encampments f r o m 1945 to 1954, in spring, 1955.

The questions are arranged here as they were on the questionnaires, except that answer boxes have been left off where the categories are contained in the wording of the question. In all but the Wave II schedule the particular scale items have been omitted in order to save space. The reader may consult Appendix A for the items that make up each scale and the Wave II schedule for the way items were arranged in other questionnaires. In 1957 and 1958 the questionnaires were, with minor exceptions, the same as those used in 1955, so far as the measurement of the main dependent variables is concerned. In those years, special attention was given to college-level campers, however, and the battery of questions used for this purpose is reproduced after the 1955 questionnaires. WAVE I — M A Y , 1 9 5 5 COLUMBIA UNIVERSITY BUREAU OF APPLIED SOCIAL RESEARCH I

First of all, we are interested

in how you feel about the forthcoming

Encampment.

1. Obviously you have applied f o r the Encampment with the expectations of doing some serious thinking and studying about democracy and citizenship. B u t no doubt there are some other things which you might expect or like to do while there. W h a t other aspects of the Encampment are attractive to you, t h a t you look forward tot 2. As you know, there will be important guest speakers and lecturers during the summer. Do you think you will have a chance t o get to know some of these people personally? (Yes, No, Don't know) 3. Do you think you'll have a great deal of freedom in regulating your daily routine, some freedom, or little or no freedom? 4. Do you think that most of the other Campers will be interested in social problems a great deal, somewhat, or only a little? 5. Do you expect t h a t most of the other Campers are going to be very liberal in their views on social problems, mildly liberal, mildly conservative, or very conservative? [ 336]

Appendix

337

B

6. How about the staff members, do you expect that most of them will be very liberal, mildly liberal, mildly conservative, or very conservative? 7. What percentage of this summer's Campers do you think will come f r o m each of the following family backgrounds ? A. F a r m families % Small town families % City families % TOTAL 100% B. Families that are relatively poor % Families of modest income % Families that are well-to-do % TOTAL 100% 8. I n general, have your parents been very favorable to your idea of going to the Encampment, moderately favorable, unfavorable, or don't they care? 9. I n general, do your good friends think that your going to the Encampment is a good idea, a bad idea, or don't they care? 10. At the Encampment this summer, different amounts of time will be devoted to a variety of daytime and evening activities. Thinking of the whole week as being about 100 hours, please look over the following list and indicate how much time you personally would like to have f o r each, writing your answer in Column A. Now go back over the list and indicate how much time you think actually will be available f o r each activity, writing your answer in Column B. ACTIVITY

COLUMN A I would like to have:

Listening to lectures and attending discussion groups. Personal "exploration" of New York City (not counting formal field trips). Sports and recreation. Routine chores around the Camp, e.g., cleaning up the living quarters. Informal conversations, "bull sessions." Studying and serious reading.

COLUMN B I think there will be available:

—

hrs. a week

hrs. a week

— —

hrs. a week hrs. a week

hrs. a week hrs. a week

— — —

hrs. a week hrs. a week hrs. a week

hrs. a week hrs. a week hrs. a week

At this point in the questionnaire, items for the following scales occurred: authoritarianism, political-economic conservatism, stereotypy, and democracy. For the items in each scale, see Appendix A. For the arrangement of the items in the questionnaire, see the next questionnaire (Wave II, administered on the opening day of the Encampment). II Here are a few other questions. Check the answer that comes closest to your own personal opinion. 1. I n peacetime, do you think that members of the Communist P a r t y in this country should be allowed to speak on the radio? (Yes, No, Don't know)

338

Appendix

B

2. In your opinion, which one of these two is more important? To find out all the Communists even if some innocent people should be hurt. To protect the rights of innocent people even if some Communists are not found out.

At this point in the schedule, items from the following scales occurred: constitutionalism, political action-apathy, civil rights (discrimination), anomie, civil liberties "A." For the items in each scale, see Appendix A. For the arrangement of the items in the schedule, see the next questionnaire (Wave II, administered on the opening day of the Encampment). Ill Sere are a few factual questions which will help us for statistical

purposes:

1. Please indicate below whether or not you have had as much as one week's stay or trip to any of the following places: YES

NO

New York City Another large city A farm A summer recreation camp A camp for citizenship or social service 2. Sex: Male Female 3. Date of birth: Month 4. Today's date 5. Signature (optional)

Year

THANK

YOU FOB

YOTJB

COOPERATION

WAVE I I — J U L Y , 1 9 5 5 COLUMBIA UNIVERSITY BUREAU OF APPLIED SOCIAL RESEARCH

I 1) I t has been several weeks now since you first applied to the Encampment for Citizenship. During these past few weeks, have your expectations about the Encampment changed in any way? I F YES: (A) In what way have they changed? (B) What things happened which brought about these changes? 2) Has anything happened during this period which has changed your views on political or social issues? I F YES: (A) In what way have they changed? (B) What things happened which brought about these changes? 3) When you first applied to the Encampment, no doubt you expected to do some serious thinking and studying about democracy and citizenship. As you look at it now, what other aspects of the Encampment seem attractive to you, that you look forward to?

339

Appendix B

4) As you consider things now, on the first day of Camp: (A) Do you think you'll have a great deal of freedom in regulating your daily routine, some freedom, or little or no freedom! ( B ) Do you think that most of the other Campers will be interested in social problems a great deal, somewhat, or only a little ? (C) Do you expect that most of the other Campers are going to be very liberal in their views on social problems, mildly liberal, mildly conservative, or very conservative ? 5) Are you very pleased, or very disturbed about any of these three situations, or don't they make any difference to you? Amount of freedom Campers' interest Campers' views on at Encampment in social problems social problems I am very pleased about: I am very disturbed about: I t makes no difference to me: I F V E R Y DISTURBED ABOUT ANY: Please explain. 6) At the Encampment this summer, there will be a variety of daytime and evening activities. Please check in Column A which two of the following activities you look forward to the most. Then go back over the list and cheek in Column B the two that you find the least appealing. COLUMN A COLUMN B I look forward I find these two ACTIVITY to these two: least appealing: Listening to lectures and attending discussion groups. Personal "exploration" of New York City (not counting formal field trips). Dates, dances, and parties. Sports and recreation. Routine chores around the Camp, e.g., cleaning up the living quarters. Informal conversations, "bull sessions." Studying and serious reading. Field trips. 7) In general, have your parents been very favorable to your idea of going to the Encampment, moderately favorable, or don't they care ? 8) In general, do your good friends think that your going to the Encampment is a good idea, a bad idea, or don't they care? 9) As you know, there have been nine previous Encampments and over a thousand alumni have gone back to their communities. In what ways do you think the conduct of the ideal Encampment graduate differs from that of most people in his community? (A) Do you think that most of the alumni are living up to this ideal? (Yes, No) ( B ) Do you think that you will live up to this ideal after you leave here? (Yes, No) II Sere are some questions on your fundamental

views about

life:

1) What are the three most worthy ambitions that an individual may have? 2) What would be the three most indispensable features of an ideal society?

340

Appendix

B

3) For what end would you be willing to make the greatest sacrifice of personal comfort, time, and money? 4) What two things or activities in your life do you expect to give you the most satisfaction? (Please mark the most important source of satisfaction with the number 1, the next most important with a 2.) Your career or occupation Family relationships Leisure-time, recreational activities Participation as a citizen in the affairs of your community Participation in activities directed toward national or international betterment Religious activities Other (specify) 5) In attacking most social problems, some people feel that the bulk of our efforts should go into long-range scientific study. Others feel that the emphasis should be on action which results in immediate improvements. While both approaches are valuable, which would you favor if you had to make a choice? 6) If you were to work for the betterment of the social life of mankind, which of the following would you prefer to direct your chief energies toward? (Check one) Local problems National problems International problems 7) What do you feel is the likelihood that mankind will be able to find a satisfactory solution for each of the following problems? How long do you think it will be before such a solution is achieved? (Express your opinion by checking the appropriate space for each problem and writing in the number of years you think it will be before a solution will be found.) Highly unlikely Doubtful

Fairly likely

Certain

How many years? years years years

Race prejudice War Cancer Economic depression and unemployment years 8) Which of the following statements comes closest to your own sentiments? (Please check one.) I often feel that I am pretty much alone in my way of thinking about social problems. Although not alone, I feel that only a small minority of people think the way I do. I feel that a sizable minority of Americans think the way I do. I feel that most Americans think the way I do. 9) In general, do you think that the individual citizen can do a great deal, only a moderate amount, or hardly anything at all about the following matters?

Appendix

B

341 A great A moderate Hardly deal amount anything

Prevention of war Beduction of corruption in national government Beduction of corruption in local government. Improvement of housing Improvement of race relations 10) How about groups of people or clubs? Can they do a great deal, only a moderate amount, or hardly anything at all about these matters? [Same answer boxes as in no. 9.] 11) The Republican platform in the 1956 presidential campaign will probably contain the following planks. Would you please indicate f o r each one your own personal attitude? I do not I'm I favor it favor it undecided Establishing universal military t r a i n i n g . . . . Setting a deadline f o r desegregation in the schools in the South Granting statehood f o r Hawaii Increased participation in joint CanadianAmerican control of uranium d e p o s i t s . . . . Governmental reorganization transferring certain former Congressional investigatory powers to the Department of Justice Strengthening the Executive Branch's immunity f r o m Congressional investigations 12) I n some democracies there are certain customs or laws which you personally might or might not regard as democratic. Look over the following list and check whether each practice appears thoroughly democratic, somewhat undemocratic but tolerable in a democracy, or thoroughly undemocratic in your personal opinion: T H I S PRACTICE SEEMS TO ME Thoroughly Undemocratic Thoroughly democratic but tolerable undemocratic In Australia, every citizen is required by law to vote Until recently, iu Sweden, most babies were automatically registered as members of the Lutheran church, the state religion Until recently, in Sweden, the amount of alcohol that a person could buy each month was regulated by law

342

Appendix B

In France you must pay for a license in order to have a radio set in your home In the United States all children must be vaccinated against smallpox In Great Britain, under the law, doctors no longer collect their fees from patients but bill the government In Norway, any Jesuit is prohibited by law from entering the country In Japan, all school children wear uniforms In the United States one may legally have only one husband or wife at a time 13A) Now here is a question on a number of occupations. For each of the following jobs, please indicate whether in your own personal opinion that job has an excellent standing, good standing, average standing, somewhat below average standing, or poor standing, by checking the appropriate eolumn. Try not to judge a job according to your own opinion of some one person you know that has such a job. STANDING I S Below Excellent Good Average average Poor Farm hand Public school teacher Playground director Diplomat in the U. S. Foreign Service A local official of a labor union Nuclear physicist Taxi driver Street sweeper Welfare worker for a city government Corporal in the regular Army 13B) When you say that certain jobs have "excellent standing," what do you think is the one main thing about such jobs that gives this standing? Ill This is a section on what people think about a number of social questions. The best answer to each statement below is your personal opinion. We have tried to cover many different points of view. You may find yourself agreeing strongly with some of the statements, disagreeing just as strongly with others, and perhaps uncertain, about

Appendix B

343

others. Whether you agree or disagree with any statement, you can be sure that many other people feel the same way you do. Mark each statement in the left margin according to how much you agree or disagree with it. Please mark every one. Write in +1, + 2, or + 3 ; or - 1 , - 2, - 3, depending on how you feel in each case. + 1 : 1 AGREE A L I T T L E + 2 : 1 AGREE P R E T T Y MUCH + 3 : 1 AGREE V E R Y MUCH

•

- 1 : 1 DISAGREE A L I T T L E - 2 : 1 DISAGREE P R E T T Y MUCH - 3 : 1 DISAGREE V E R Y MUCH

1. Human nature being what it is, there will always be war and conflict. 2. Obedience and respect for authority are the most important virtues children should learn. 3. Generally speaking, Negroes are born with more musical talent than white people. 4. I t is up to the government to make sure that everyone has a secure job and a good standard of living. 5. No weakness or difficulty can hold us back if we have enough will power. 6. Farmers are more self-sufficient than city people. 7. Science has its place, but there are many important things that can never possibly be understood by the human mind. 8. Young people sometimes get rebellious ideas, but as they grow up they ought to get over them and settle down. 9. City people are less friendly than small-town people. 10. The government should own and operate all public utilities (railroads, gas, electricity, etc.). 11. Usually it is not feasible for all members of a group to take an equal interest and share in the activities of the group. 12. City people are more radical than country people. 13. Almost any job that can be done by a committee can be done better by having one individual responsible for it. 14. Most labor union officials are the sons of immigrants. 15. Men like Henry Ford or J . P. Morgan, who overcame all competition on the road to success, are models for all young men to admire and imitate. 16. In case of disagreement within a group the judgment of the leader should be final. 17. Most of the Puerto Ricans in New York City came from poor farm families on the island. 18. The only way to eliminate poverty is to make certain basic changes in our political and economic system. 19. You can recognize a New Yorker by his aggressive manner. 20. The best criterion for judging any technique for dealing with other people is in terms of how efficiently it will get the job done. 21. What this country needs most, more than laws and political programs, is a few courageous, tireless, devoted leaders in whom the people can put their faith. 22. There are often occasions when an individual who is a part of a working group should do what he thinks is right regardless of what the group has decided to do.

344

Appendix

B

23. Southerners are generally lazy. 24. In a new tax program it is essential not to reduce the income taxes on corporations and wealthy individuals. 25. In most practical situations, the more experienced members of a group should assume responsibility for the group discussion. 26. In general, full economic security is bad; most men wouldn't work if they didn't need money for eating and living. 27. Most Midwesterners are isolationists. 28. Sometimes one can be too open-minded about the possible solutions to a problem that faces a group. 29. There should be some upper limit, such as $25,000 per year, on how much any individual can earn. 30. In a group that really wants to get something done, the leader should exercise friendly but firm authority. 31. The true American way of life is disappearing so f a s t that force may be necessary to preserve it. 32. Discipline should be the responsibility of the leader of a group. 33. I t is sometimes necessary to use autocratic methods to obtain democratic objectives. 34. Most of the present attempts to curb and limit unions would in the long run do more harm than good. 35. I t is essential for learning or effective work that our teachers or bosses outline in detail what is to be done and exactly how to go about it. 36. I t is not always feasible to try to be consistent with one's ideals in everyday behavior. 37. The businessman and manufacturer are probably more important to society than the artist and the professor. 38. Fighting to put one's ideals into practice is a luxury that only a few can afford. 39. Generally there comes a time when democratic group methods must be abandoned in order to solve practical problems. 40. Some leisure is necessary but it is good hard work that makes life interesting and worthwhile. 41. In a democratic group, regardless of how one feels, he should not withdraw his support from the group. 42. When you come right down to it, it's human nature never to do anything without an eye to one's own profit. 43. Sometimes it is necessary to ignore the views of a few people in order to reach a decision in a group. 44.1 would like to see a child of mine go into politics. 45. Generally speaking, the less government we have the better off we will be. 46. When a person has a problem or worry, it is best for him not to think about it, but to keep busy with more cheerful things. IV Here are a few other questions on social issues. Cheek the answer that comes closest to your own personal opinion. 1) I n peacetime, do you think that members of the Communist Party in this country should be allowed to speak on the radio? (Yes, No, Don't know)

345

Appendix B

2) In your opinion, which one of these two is more important? To find out all the Communists even if some innocent people should be hurt. To protect the rights of innocent people even i f some Communists are not found out. 3) How do you think the average American would answer these last two questions? (A) To Question # 1 , the average American would say: Yes No Don't know I don't know how he would answer ( B ) To Question # 2 , the average American would say: Find all Communists Protect the innocent I don't know how he would answer 4A) With respect to each of the following issues, do you feel that your views are more liberal or more conservative or about the same as those of the average American? (Please check) MY V I E W S ARE More About More conliberal the same servative Civil liberties Economic planning Urgency of social reforms Race relations International affairs 4 B ) On those issues where your views differ from the average American do you feel that this difference is very great, moderate, or very small? (Please check) Very great

Moderate

Very small

Don't know

Civil liberties Economic planning Urgency of social reforms Race relations International affairs 5) While you may feel that all of the following objectives are important to democracy, suppose that you had to choose only one of each pair, which would you choose? PAIR A : Economic security for all citizens or Guarantees of free elections PAIR B : Economic security for all citizens or Freedom of press, speech, and other civil liberties P A I R C: Guarantees of free elections or Freedom of press, speech, and other civil liberties 6) This question covers briefly many other topics. Indicate whether you agree or disagree with each statement. Circle A if you tend to agree with the statement; circle D if you tend to disagree; circle ? if you don't know or cannot make up your mind.

346

Appendix B Agree Disagree

1. The general public is not really qualified to vote on today's complex issues. 2. Only trained and competent people should be permitted to run for public office. 3. People of different races should not dance together. 4. Most people cannot be trusted. 5. Allowing just anyone—including uneducated people— to serve on a jury is a poor idea. 6. Swimming pools should admit people of all races and nationalities to swim in the same pool. 7. Since 1890 people's ideas of morality have changed a lot, but there are still some absolute guides to conduct. 8. Decisions to protect the government may have to be made even at the expense of one person. 9. People who talk politics without knowing what they are talking about should be kept quiet. 10. There should be laws against marriage between persons of different races. 11. Nobody cares whether you vote or not except the politicians. 12. Unrestricted freedom of speech leads to mass hysteria. 13. Pressure groups are useful and important features of representative government. 14. Hotels are right in refusing to admit people of certain races or nationalities. 15. Sending letters to Congressmen has little influence upon legislation. 16. Nobody cares whether you attend church or not except the clergy. 17. I f a person is uncertain how to vote, it is better if he does not vote. 18. Whether democratic or totalitarian, every nation has a right to its own kind of government. 19. Pupils of all races and nationalities should attend school together everywhere in this country. 20. People talk a lot about being decent to Negroes and other minority groups, but when it comes right down to it most people don't really care how you treat these groups. 21. In passing laws about issues like price controls and taxation, Congress should pay more attention to what the experts say than to what the public says, when they differ. 22. The average citizen is justified in remaining aloof from dirty politics that may exist in his community. 23. Democracy depends fundamentally on the existence of free business enterprise.

?

A

D

f

A A A

D D D

?

1 1

A

D

1

A

D

1

A

D

1

A

D

f

A

D

?

A

D

?

A A

D D

? ?

A

D

t

A

D

f

A

D

?

A

D

?

A

D

?

A

D

»

A

D

A

D

?

A

D

?

A

D

?

A

D

f

.

?

347

Appendix B 24. Cheating on income taxes is nobody's business but the government's. 25. Whatever serves the interest of the government best is generally right. 26. What really made Dr. Salk work so hard on the polio vaccine was the thought of the money or fame he would get. 27. Certain groups should not be allowed to hold public meetings even though they gather peaceably and only make speeches. 28. A person who wants to make a speech in my community against churches and religion should be allowed to speak. 29. Kinsey made his name with Ms report on sexual behavior, and the hope of publicity was what really led him to study sex in the first place. 30. In some criminal cases, a trial by jury is an unnecessary expense and shouldn't be given. 31. Persons who refuse to testify against themselves (that is, give evidence that would show that they are guilty of criminal acts) should either be made to talk or severely punished. 32. Some of the petitions which have been circulated should not be allowed by the government. 33. Slum housing is the cause of most juvenile delinquency. 34. In some cases, the police should be allowed to search a person or his home even though they do not have a warrant. 35. A book favoring government ownership of all the railroads and big industries should be removed from the public library i f someone in the community suggests this. 36. Lack of education is the cause of most race prejudice. 37. Newspapers and magazines should be allowed to print anything they want except military secrets. 38. Biased news coverage is mainly due to the pressures of advertisers. 39. I f a person is accused of a crime he should always have the right to know who is accusing him. 40.1 would stop buying the soap advertised on a radio program which had an admitted Communist as the singer, i f someone suggested this to me. 41. An admitted Communist should be put in jail. 42. Popular taste would be raised tremendously if the producers of radio and television had some decent programming. 43. In some criminal cases, a trial by jury is an unnecessary expense and shouldn't be given.

Agree Disagree

?

A

D

?

A

D

I

A

D

f

A

D

?

A

D

I

A

D

I

A

D

?

A

D

Ì

A A

D D

I ?

A

D

?

A A

D D

? ?

A

D

?

A

D

?

A

D

?

A A

D D

? ?

A

D

?

A

D

?

348

Appendix B Agree Disagree

44. A book against churches and religion should be removed from the public library if someone in the community suggests this. 45. Class distinctions would disappear if economic opportunities were made more equal. 46. A community-wide mass media campaign could really sell the public on the goals of the UN. 47. A person who wants to make a speech in my community favoring government ownership of the railroads and big industries should be allowed to speak. 48. The right of some working groups to call a strike should be abolished, as it is a threat to democracy and not in the general interest of society.

?

A

D

?

A

D

?

A

D

?

A

D

?

A

D

?

V Here are a few factual questions which will help us for statistical purposes: 1) Sex Male Female 2) Please indicate below whether or not you have had as much as one week's stay or trip to any of the following places: (Check any that apply) YES New York City Another large city A farm A summer recreation camp A camp for citizenship or social service 3) What is your major vocational goal? 4A) What is (was) your father's major occupation? 4 B ) What is (was) your mother's major occupation? 5) I f you were asked to use one of these four names for your social class, which would you say you belonged in—the middle class, lower class, working class, or upper class? Upper Middle Working Lower 6) What is your usual political party preference? Democrat Republican Other (specify) None 7) Date of birth: Month , Year 8) Signature (optional) THANK YOU FOE YOUR COOPERATION

Appendix B

349

WAVE III—AUGUST, 1955 COLUMBIA UNIVERSITY BUREAU OF APPLIED SOCIAL RESEARCH

I 1.. In general, would you say that the Encampment this summer has lived up to your expectations? (Yes, No) 2A. Are there any specific ways in which it worked out better than you expected? I F Y E S : How? 2B. Did anything happen that you feel had a very favorable influence on you? I F Y E S : What was it that happened? 3A. Are there any specific ways in which the Encampment did not meet your expectations? I F Y E S : How? 3B. Did anything happen here this summer that distributed you a great deal? I F Y E S : What was it that happened? As a result of that event, were your ideas and experiences during the rest of the Encampment changed in any way? I so, how? 4. As you consider things now: (A) Do you think that you have had enough freedom in regulating your daily routine here, too much freedom, or not enough freedom? ( B ) Do you think that most of the other Campers were interested in social problems a great deal, somewhat, or only a little? 5A. Do you feel that the amount of agreement between your views on the following issues and the views of most of the campers is very great, moderate, or very little? Very Very Great Moderate little Civil liberties Economic planning Urgency of social reforms Race relations Importance of polities 5B. Do you think that most of the other campers were too liberal in their views on the following issues, not liberal enough, or about right? Too liberal

Not liberal enough

About right

Civil liberties Economic planning Race relations 6. Please check in Column A which two of the following activities of the Encampment this summer you enjoyed the most. Then go back over the list and check in Column B the two that you found the least enjoyable.

350

Appendix

B

COLUMN A COLUMN B I enjoyed these I enjoyed these ACTIVITY two the most: two the least: Listening to lectures and attending discussion groups Personal "exploration" of New York City (not counting formal field trips) Dates, dances, and parties Sports and recreation Routine chores around the camp, e.g., cleaning up the living quarters Informal conversations, "bull sessions" Studying and serious reading Field trips 7. Did you find that the lectures and discussions were difficult to follow a lot of the time, only occasionally, or never? If they ever were difficult to follow: What was it that made it difficult for you? 8A. Do you think that what you learned in your workshop will be useful to you? (Yes, No) 8B. Which workshop was that? 9. How often, at the end of an average day of Encampment activities, did you have the feeling that it had been too much of a strain and a rush for you— almost always, frequently, occasionally, or never? 10. Naturally, there might have been a number of reasons which caused a camper to miss some event or activity during the Encampment. Thinking back over the entire summer, how many times would you say that you happened to miss each of the following Encampment activities? I missed about the following number of times: Lectures Discussion groups Workshops Field trips Kecreational afternoons Meals Town meetings Scheduled evening recreation Scheduled evening cultural or educational program 11. Which of the following Encampment facilities have you used this summer? (Please check as many as apply.) I have used : The Encampment library The music room The swimming pool The recreation room Newspaper subscription service Specify which paper 12. Of course, everyone has met some friendly people while here, but how many real friends do you think you have made, for example, people who will take the trouble to write and keep in touch with you after the Encampment ends?

Appendix B

351

13A. Who was your closest friend here at the Encampment 13B. Did you know each other before coming here? (Yes, No) 14. What do you think is the main reason that most of the staff members had for being at the Encampment this summer? 15. Would you say that, in general, the staff here this summer were greatly interested in the campers as individuals, somewhat interested, very little interested or not at all interested? 16. As you know, there have been nine previous Encampments and over a thousand alumni have gone back to their communities. In what ways do you think the conduct of the ideal Encampment graduate differs from that of most people in his community? (A) Do you think that most of the alumni are living up to this ideal? (Yes, No) ( B ) Do you think that you will live up to this ideal after you leave here? (Yes, No) II Here are some questions on your fundamental views about life: 1) What are the three most worthy ambitions that an individual may have? 2) What would be the three most indispensable features of an ideal society? 3) For what end would you be willing to make the greatest sacrifice of personal comfort, time, and money? 4) What two things or activities in your life do you expect to give you the most satisfaction? (Please mark the most important source of satisfaction with the number 1, the next most important with a 2.) Your career or occupation Family relationships Leisure-time, recreational activities Participation as a citizen in the affairs of your community Participation in activities directed toward national or international betterment Religious activities Other (specify) 5) In attacking most social problems, some people feel that the bulk of our efforts should go into long-range scientific study. Others feel that the emphasis should be on action which results in immediate improvements. While both approaches are valuable, which would you favor if you had to make a choice? 6) I f you were to work for the betterment of the social life of mankind, which of the following would you prefer to direct your chief energies toward? (Check one) Local problems National problems International problems 7) What do you feel is the likelihood that mankind will be able to find a satisfactory solution for each of the following problems? How long do you think it will be before such a solution is achieved? (Express your opinion by cheeking the appropriate space for each problem ajid writing m the number of years you think it will be before a solution will be found.)

352

Appendix

B

Highly unlikely Doubtful

Fairly likely

Certain

How many years? years years years

Race prejudice War Cancer Economic depression and unemployment years 8) Which of the following statements comes closest to your own sentiments? ( P L E A S E CHECK ONE) I often feel that I am pretty much alone in my way of thinking about social problems. Although not alone, I feel that only a small minority of people think the way I do. I feel that a sizable minority of Americans think the way I do. I feel that most Americans think the way I do. 9) I n general, do you think that the individual citizen can do a great deal, only a moderate amount, or hardly anything at all about the following matters? A great A moderate deal amount

Hardly anything

Prevention of war Reduction of corruption in national government Eeduction of corruption in local government. Improvement of housing Improvement of race relations 10) How about groups of people or clubs? Can they do a great deal, only a moderate amount, or hardly anything a t all about these matters? [Same answer boxes as in No. 9.] 11) The Democratic platform in the 1956 presidential campaign will probably contain the following planks. Would you please indicate f o r each one your own personal attitude. I do not I'm I favor it favor it undecided Establishing universal military t r a i n i n g . . . . Setting a deadline f o r desegregation in the schools in the South Granting statehood f o r Hawaii Increased participation in joint CanadianAmerican control of uranium d e p o s i t s . . . . Governmental reorganization transferring certain former Congressional investigatory powers to the Department of Justice Strengthening the Executive Branch's immunity f r o m Congressional investigations 12) In some democracies there are certain customs or laws which you personally might or might not regard as democratic. Look over the following list and check whether each practice appears thoroughly democratic, somewhat undemocratic but tolerable in a democracy, or thoroughly undemocratic in your personal opinion:

Appendix

B

353

THIS PRACTICE SEEMS TO ME Thoroughly Undemocratic Thoroughly democratic but tolerable undemocratic In Australia, every citizen is required by law to vote Until recently, in Sweden, most babies were automatically registered as members of the Lutheran church, the state religion Until recently, in Sweden, the amount of alcohol that a person could buy each month was regulated by law In France you must pay for a license in order to have a radio set in your home In the United States all children must be vaccinated against smallpox In Great Britain, under the law, doctors no longer collect their fees from patients but bill the government In Norway, any Jesuit is prohibited by law from entering the country In Japan, all school children wear uniforms In the United States, one may legally have only one husband or wife at a time 13A) Now here is a question on a number of occupations. For each of the following jobs, please indicate whether in your own personal opinion that job has an excellent standing, good standing, average standing, somewhat below average standing, or poor standing, by checking the apropriate column. Try not to judge a job according to your own opinion of some one person you know that has such a job. STANDING I S : Below Excellent Good Average average Poor Farm hand Public school teacher Playground director Diplomat in the U.S. Foreign Service A local official of a labor union Nuclear physicist

354

Appendix B STANDING I S : Below Good Average average —

Excellent Poor Taxi driver Street sweeper Welfare worker for a city government Corporal in the regular Army 13B) When you say that certain jobs have "excellent standing," what do you think is the one mam thing about such jobs that gives this standing?

At this point in the schedule items for the following seales occurred: authoritarianism, democracy, stereotypy, and political-economic conservatism. For the individual items for each scale, see Appendix A. For the arrangement of the items in the schedule, see the preceding questionnaire (Wave II, July, 1955). Ill Here are a few other questions on social issues. Check the answer that comes closest to your own personal opinion. 1) In peacetime, do you think that members of the Communist Party in this country should be allowed to speak on the radio? (Yes, No, Don't know) 2) In your opinion, which one of these two is more important? To find out all the Communists even if some innocent people should be hurt. To protect the rights of innocent people even if some Communists are not found out. 3) How do you think the average American would answer these last two questions? (A) To Question # 1 , the average American would say: Yes No Don't know I don't know how he would answer (B) To Question # 2 , the average American would say: Find all Communists Protect the innocent I don't know how he would answer 4A) With respect to each of the following issues, do you feel that your views are more liberal or more conservative or about the same as those of the average American? (Please check) MY VIEWS ARE More About More conliberal the same servative Civil liberties Economic planning Urgeney of social reforms Eace relations International affairs 4B) On those issues where your views differ from the average American do you feel that this difference is very great, moderate, or very small? (Please check)

355

Appendix B Very great

Moderate

Very small

Don't know

Civil liberties Economic planning Urgency of social reforms Race relations International affairs 5) While you may feel that all of the following objectives are important to democracy, suppose that you had to choose only one of each pair, which would you choose? PAIR A: Economic security for all citizens or Guarantees of free elections PAIR B : Economic security for all citizens or Freedom of press, speech, and other civil liberties PAIR C. Guarantees of free elections or Freedom of press, speech, and other civil liberties

At this point in the schedule, items for the following scales occurred: constitutionalism, civil rights (discrimination), political action/apathy, anomie, civil liberties "A," civil liberties "B," absolutism, tolerance. For the items in each scale, see Appendix A. For the arrangement of the items in the schedule, see the preceding questionnaire (Wave II, July, 1955). 6) In the course of these six weeks you have had a lot of opportunities for a variety of new experiences. Do you feel that these experiences have changed you personally a great deal, a moderate amount, or hardly at all? I F THEY HAVE CHANGED YOU AT ALL: In what ways do you feel you are a changed person? 7) Have your vocational or educational plans been changed or modified in any way as a result of the Encampment? I F YES: In what manner? 8) After you leave the Encampment and return to your home or school, do you have any specific plan for applying what you have gotten from the Encampment? I F YES: Please describe this plan, briefly. 9A) When you return to your home or school, do you think that there will be any changes in your relationships with your family? I F YES: In what ways? 9B) How about your relationships with old friends from your pre-Encampment days, do you think they will change? I F YES: In what ways? 9C) And your relationships with your neighbors and acquaintances, do you think they will change? I F YES: In what ways? 10) Upon returning to your home or school, do you think that any other of your activities will have changed as a result of the Encampment? I F YES: What? 11) Would you recommend the Encampment to a close friend? (Yes, No) 12) Do you think that six weeks is too long for this kind of Encampment, not long enough, or just about right? 13.) When you return to your home or school would you like to be in touch with other Campers and alumni frequently, once in a while, not at all, or doesn't it make much difference to you now?

356

Appendix B

IV Here are a few factual questions which will help us for statistical purposes: 1) Sex Male Female 2) Date of birth: Month , Year 3) This question concerns your acquaintances and friends before coming to the Encampment. Please check in Column A any groups in which you had acquaintances before coming to the Encampment. Then check in Column B any groups in which you had close friends before coming to the Encampment. (CHECK AS MANY AS APPLY.) A B Before this summer, I had Before this summer, I had some acquaintances who some close friends who were: were: Indian Negro White Catholic Jewish Protestant 4) What is your major vocational goal! 5) I f you were asked to use one of these four names for your social class, which would you say you belonged in—the middle class, lower class, working class, or upper class? 6) What is your usual political party preference? Democrat Republican Other (specify) None 7) Signature (optional) THANK YOU FOR YOUR COOPERATION WAVE IV—SEPTEMBER, 1 9 5 5 COLUMBIA UNIVERSITY BUREAU OF APPLIED SOCIAL RESEARCH

I 1. Now, as you think back to this summer's Encampment, what would you say was the most important experience that you had there? Why do you think it was important? 2. Looking back at the Encampment in the light of your present knowledge and experience, what suggestions would you make as to ways in which future Encampments might be improved ? 3. Since you left the Encampment, have you spent most of your time back in your regular home community, or have you been mainly away at other places? I F YOU HAVE B E E N AWAY FROM HOME: What sorts of things have you been doing?

Appendix

B

357

4. During the first few days a f t e r you l e f t the Encampment, what was the first thing that happened that made you think over some of the ideas you discussed a t the Encampment? (Please describe it.) 5. When you finally returned to your home community did it look any different to you, or just about the same as it always was? I F I T LOOKED D I F F E R E N T : I n what ways did it seem different? 6. Are there things about the community that bother you now, t h a t didn't used to bother you? I F Y E S : What things? Why do they bother you now? Have you been trying to do anything about these things since you returned? If yes: What? If No: Are you planning to do something about them in the future? 7. Now that you have had a chance to talk about your summer to your family, how do you think they feel about the Encampment? Do they think your going to the Encampment was a very good idea, a fairly good idea, a bad idea, or don't they care? I F T H E Y F E E L I T WAS A BAD I D E A : Why do they feel that way? 8. How about your good friends, do they think that your going was a very good idea, a fairly good idea, a bad idea, or don't they care? 9. Are there any changes in the following areas of your life which you believe were brought about by the Encampment experience: A. Relationships with old friends f r o m your pre-Encampment days? I F Y E S : What? B. Your feelings about the kind of person you want to date or to marry? I F Y E S : What? C. Relationships with your family? I F Y E S : What? D. Association with people of other religions or color? I F Y E S : What? 10A. As a result of going to the Encampment, would you say that you are now more interested in participating in the organizations in your community, or less interested, or about the same as you were before going to the Encampment? 10B. Since you returned would you say that you attend church more frequently than you used to, less frequently, or about the same? 11. Has there been any instance since you left the Encampment in which you felt that you wanted to apply something you had learned there? I F Y E S : Did you t r y to apply it? If you tried: Please describe this event. What specific principle were you trying to apply? What would have helped you to handle the situation better? If you did not try: What was the reason t h a t you didn't try to apply it? 12. Since you l e f t the Encampment, have you ever found yourself in an actual situation in which you witnessed some kind of undemocratic practice? I F Y E S : Would you describe the situation? What specific things did you do a t the time, if any? 13. I n general (thinking everything over) do you feel that you have "settled down" by now, or are you still somewhat unsettled? I F S T I L L U N S E T T L E D : W h a t is it that makes it hard to get settled down? 14. I n the course of your ordinary activities do you ever stop and think much about the Campers and staff that you knew a t the Encampment? (No, Occasionally, Frequently) 15. Since leaving the Encampment, have you had contacts with any of your fellow Campers in any of the following ways? (Please check all that apply.)

358

Appendix

B

Yes, I have: Met with them socially Spoken over the telephone Written to them Received mail f r o m them Worked with them on some project 16. Since you l e f t the Encampment, have you attended any meetings or reunions with former campers from other years? (Yes, No) 17. Whether or not you associate with them, how many former campers are there in the local area where you are now living? 18. Have you talked about public issues or politics with anyone recently? (Yes, No) 19. Thinking back to the time before the Encampment, would you say that you were asked your opinions about politics more than the average person in your group, less, or about as much? 20. Now that you have returned f r o m the Encampment, are you asked your opinions about politics more than the average person in your group, less, or about as much? 21. Do you regularly read any daily newspapers? (Yes, No) 22. Do you read any magazines regularly? I F Y E S : Which ones? II IA. As you know, there have been nine previous Encampments and over a thousand alumni have gone back to their communities. I n what ways do you think the conduct of the ideal Encampment graduate differs f r o m that of most people in his community; I B . Do you think that most of the alumni are living u p to this ideal? (Yes, No) 2. I n general, do you think that the individual citizen can do a great deal, only a moderate amount, or hardly anything a t all about the following matters? A great A moderate deal amount

Hardly anything

Prevention of war Reduction of corruption in national government Reduction of corruption in local government Improvement of housing Improvement of race relations 3. How about groups of people or clubs? Can they do a great deal, only a moderate amount, or hardly anything a t all about these matters? [Same answer boxes as in No. 2.] 4. What do you feel is the likelihood that men will be able to find a satisfactory solution f o r each of the following problems? How long do you think it will be before such a solution is found? (Express your opinion by checking the appropriate space f o r each problem and writing in the number of years you think it will be before a solution will be found.)

359

Appendix B Highly unlikely

Doubtful

Fairly likely

Certain

How many years?

Bace prejudice War Cancer Economic depression and unemployment 5. I n attacking most social problems, some people feel that the bulk of our efforts should go into long-range scientific study. Others feel that the emphasis should be on action which results in immediate improvements. While both approaches are valuable, which would you favor if you had to make a choice? 6. If you were to work f o r the betterment of the social life of mankind, which of the following would you prefer to direct your chief energies toward? (Check . one) local , national , or international problems 7. I n peacetime, do you think that members of the Communist P a r t y in this count r y should be allowed to speak on the radio? (Yes, No, Don't know) 8. How do you think the average American would answer Question 7? The average American would say: Yes No Don't know I don't know how he would answer 9A. With respect to each of the following issues, do you feel that your views are more liberal or more conservative or about the same as those of the average American? (Please check) MY V I E W S ARE More About More conliberal the same servative Civil liberties Economic planning Urgency of social reforms Race relations International affairs 9B. On those issues where your views differ from the average American do you feel that this difference is very great, moderate, or very small? (Please check) [Same items as in No. 9A.] 10. Think back to the way most of the Campers felt about the following issues during the summer. Do you feel that you are now in very great agreement with those views, in moderate agreement, or in very little agreement? [Same items as in No. 9A.]

At this point in the schedule, items for the following scales occurred: civil liberties "A," civil rights (discrimination), anomie, political action/apathy, tolerance. For the items in each scale, see Appendix A. For the arrangement of the items in the schedule, see the questionnaire for Wave II, July, 1955.

360

Appendix

B

III Here are a few factual questions which will help us for statistical

purposes:

1. Sex: 2. Date of birth: Month Year 3. What are your educational plans? 4. What is your major vocational goal! 5. Which of the following have you been doing within the last month or expect to be doing within the next month or so? (Please cheek as many as apply.) Going away to college or university Entering the Armed Forces Getting married Looking f o r a new job (not a part-time or temporary one) Starting a new job (not a part-time or temporary one) Moving 6. Today's date: 7. Signature (optional) : T H A N K T O U F O B YOUK COOPERATION WAVE V — M A R C H , 1 9 5 9 COLUMBIA UNIVERSITY BUREAU OF APPLIED SOCIAL RESEARCH

I 1. I n the course of your ordinary activities do you ever stop and think much about the campers and staff that you knew at the Encampment? (No, Occasionally, Frequently) 2. During the past year, have you had contacts with yowr group of campers (1955) in any of the following ways? (Please check in Column A all that apply.) Outside of this past year, have you ever had any contacts with your group of Campers in any of these ways? (Please check in Column B all that apply.) A During the past year:

B I n other past years:

Yes: I have Met with them socially Spoken with them on the telephone Written to them Received mail f r o m them Worked with them on some project N o : I have had no contracts 3. Since you l e f t the Encampment, have you attended any meetings or reunions with former campers f r o m other years? (Yes, No) 4. Whether or not you have associated with these people, how many former Campers are there in your community? 5. How often, since you l e f t the Encampment, have you received any official contact f r o m the Encampment staff in the following ways:

361

Appendix B

Indicate approximate number of times:

Visited by a member of the staff Received letter or literature from a member of the staff 6. How many people, i f any, from the 1955 Encampment are you really close friends with today? 7. Who was your closest friend while you were at that Encampment? 8. In the past year have you and this person ever: Approximate number of times Met together socially Talked together on the telephone Written letters to one another Worked together on a project 9. How many people, i f any, from other Encampments are you really close friends with today? 10. As you know, there have been thirteen Encampments and about fifteen hundred alumni have gone back to their communities. In what ways do you think the conduct of the ideal Encampment graduate differs from that of most people in his community. 11. Do you think that most of the alumni are living up to this ideal? (Yes, No) 12. I f you had the time, how interested would you be in any of the following Encampment activities: Very Somewhat Slightly Not at all interested interested interested interested Recruiting Campers Getting the Newsletter Raising funds for the Encampment 13. Have there been any major instances since leaving the Encampment in which you have tried to apply any of its ideas? I F Y E S : Which one stands out in your mind as most important? In what year did it occur? What was the instance like and what did you try to do? 14. Thinking back to recent elections, do you recall whether or not you happened to vote in any of the following: (Please check one emswer in each column.) A B 1956 1958 Presidential Congressional election election

C

D

Any state election

Any local election

Yes, voted No, but was eligible Was not eligible Can't remember — 15. Have you talked about public issues or politics with anyone recently? (Yes, No) 16. When you and your friends discuss political questions, what part do you usually take? (Please check one.) Even though I may have strong opinions, I usually just listen I listen a lot, but once in a while I express my opinion I take an equal share in the conversation I have definite ideas and try to convince the others

362

Appendix

B

17. Have you ever written a letter to any government or public official? (Yes, No) 18. Do you read any magazines regularly? If yes: Which ones? 19A. Would you please look down this list and cheek the two ways in which you usually spend most of your spare time? Check TWO Sports, outdoor activities, gardening Visiting with friends Dancing and party going Reading Painting, drawing, art work, handicrafts, playing a musical instrument Listening to the radio, watching television, playing records Going to the movies, theater, concerts Church and club work Civic and political activities Others (specify) 19B. If you were fortunate enough to have a lot more leisure time, at which one of the above activities would you like to spend this extra time ? 20. Of the following activities, which are the two from which you expect the greatest satisfaction in your life time? Check TWO Tour career Family relationships Leisure or recreational activities Participation as a citizen in the affairs of your community Participation in activities directed toward national and international betterment Religious activities 21. Here is a list of different kinds of clubs and organizations that people can belong to. Please look over the list. I n Column A, please put the number of each kind of group to which you now belong. If you used to belong to a group of a certain type but do not now belong to it, please write in Column B the years in which you were a member. Then, in Column C, please check those in which you ever held an office, either currently or in the past. Column A

I now belong: Business groups Church-connected groups Veterans' organizations Labor union Parent-teacher association Neighborhood clubs or community centers

Column B I used to belong: Year Year joined left 19 19 19 19 19 19 19 19 19

19

19

19

Column C

Held Office: Yes No

Appendix Column A

I now belong: Professional groups or societies Neighborhood improvement associations Youth councils or student organizations Other civic groups, e.g., League of Women Voters Charitable and welfare organizations Informal clubs or groups Political clubs or organizations Farmers' organizations Other (please list)

363

B Column B I used to belong: Year Year joined left 19

19

19

19

19

19

19

19

19 19

19 19

19 19

19 19

Column C

Held Office: Yes No

19 19 22. What do you feel is the likelihood that men will be able to find a satisfactory solution f o r each of the following problems? How long do you think it will be before such a solution is achieved? (Express your opinion by checking the appropriate space for each problem and writing in the number of years you think it will be before a solution will be found.) Highly unlikely

Doubtful

Fairly likely

Certain

How many years?

Race prejudice War Cancer Economic depression and unemployment 23. I n general, do you think that the individual citizen can do a great deal, only a moderate amount, or hardly anything a t all about the following matters? A great A moderate deal amount

Hardly anything

Prevention of war Reduction of corruption in national government Reduction of corruption in local government Improvement of housing Improvement of race relations •— 24. How about groups of people or clubs? Can they do a great deal, only a moderate amount, or hardly anything a t all about these matters? [Same answer boxes as in No. 23.]

364

Appendix B

25. In peacetime, do you think that members of the Communist party should be allowed to speak on the radio? (Yes, No, Don't know) 26. In your opinion, which one of these two is more important? To find out all the Communists even if some innocent people should be hurt, OR

To protect the rights of innocent people even if some Communists are not found out. 27. Now, how do you think other people would answer these two questions? Please check in Column A how you think most of the members of the 1955 Encampment group would answer the questions today. I n Column B, please check how you think most other people of your age and background in your community would answer them. And in Column C, please cheek how you think the average American would answer them. To Question 25, the answer would probably be: Yes, allow them to speak No, do not allow them Don't know I don't know what the answer would be

Most of 1955 Most people The Encampment of my age and average group today background American

To Question 26, the answer would probably be: Find out all Communists Protect the innocent I don't know what the answer would be

At this point in the schedule, items in the following scales occurred: civil liberties "A," tolerance, civil rights (discrimination), political action/apathy, and authoritarianism. For the items in each scale, see Appendix A. For the arrangement of the items in the schedule, see the questionnaire for "Wave II, July, 1955, above. 28A. With respect to each of the following issues, do you feel that your current views are more liberal or more conservative or about the same as those of the average American? (Please check one.) MY V I E W S ABE More About More conliberal the same servative Civil liberties Economic planning Urgency of social reforms Eace relations International affairs

Appendix

B

365

28B. On those issues where your views differ from the average American, do you feel that this difference is very great, moderate, or very small? (Please check.) [The items (civil liberties, etc.) are the same as in part A of this question.] 280. Think back to the way most of the Campers felt about the following issues during the summer of 1955. Compare the views you now hold with those views. Do you feel that you are now in very great agreement with the views most of the Campers had in 1955, in moderate agreement, or in very little agreement? [The items are the same as in part A of this question.] II Here are a few factual questions: 1. We are interested in learning something about the careers and experiences Campers have had since attending the Encampment in 1955. Would you please check or fill in the appropriate spaces for each item for each year? A. Family: In which years were you for most of the year: And None 1956 1957 1958 now? Single and lived with parents Single and lived away from home Married and lived with spouse Married but lived separately Divorced Widowed Eemarried In which years were children born: B. Education: During which academic years were you in: 1955195619571958None 1956 1957 1958 1959 High School College Graduate School Professional School (What kind) C. Military Service: Which years were spent mainly in service? And None 1956 1957 1958 now? D. Employment: In which past years were you employed full time for at least 6 months; are you employed full time now? And None 1956 1957 1958 now? E. Occupation: for the years in which you were employed full time for 6 months or longer, what kind of work were you doing? What work are you doing now? 1955: 1956:

F. 1iesidence: for each year, please write in the city and state in which you lived the longest during that year. Where are you living now? 1955: 1956:

366

Appendix

B

1957: 1957: 1958 : 1958 : Now: Now: 2. What led you into your present occupation ? 3. What are your major vocational goals a t present? 4. What is the highest grade in school or college you have completed to date? 5.Do you have any plans f o r further formal education? I F Y E S : please describe them. 6. If you were asked to use one of these f o u r names f o r your social class, which would you say you belonged in—the middle class, the lower class, the working class, or the upper class? 7. I s your church preference : Protestant Catholic Jewish Other (What? ) None 8. Sex: Male Female 9. Today's date: 10. Name (optional) T H A N K YOU V E R Y MUCH V I . A L U M N I QUESTIONNAIRE COLUMBIA U N I V E R S I T Y BUREAU OP APPLIED SOCIAL RESEARCH I

This first series of questions deals with your experiences dwring and after the Encampment. 1. During which summer were you at the Encampment? 19 2. I n the course of our ordinary activities do you ever stop and think much about the campers and staff that you knew a t the Encampment? (No, Occasionally, Frequently) 3. Thinking back to that summer, can you remember the names of some of the other Campers? If so, please list as many as you recall, up to sis. 4. During the past year, have you had contacts with your group of Campers in any of the following ways? (Please check in Column A all that apply.) Outside of this past year, have you ever had any contacts with your group of Campers in any of these ways? (Please check in Column B all that apply.) A Yes, during the past year I have : Met with them socially Spoke over the telephone Wrote to them Received mail f r o m them Worked with them on some project

B Yes, in other past years I have :

Appendix

B

367

5. Would you consider any of the people from your Encampment really close friends today? (Yes, No) 6A. Since you l e f t the Encampment, have you attended any meetings or reunions with former Campers f r o m other years? (Yes, No) 6B. Whether or not you have associations with these people, how many former Campers are there ill your community? 7. How often, since you l e f t the Encampment, have you received any official contact f r o m the Encampment staff in the following ways: Indicate approximate number of times Visited by a member of the staff Received letter or literature from a member of the staff 8A. Thinking back to the Encampment, what was the most important experience that you had there? 8B. Why do you think it was important? 9. Out of all you learned there, what do you think was most important to you ? 10. Some aspects of the Encampment must have made a more vivid or lasting impression than others. F o r each activity listed below, would you indicate how vividly you recall it, despite the passage of time. Remember Remember very well vaguely

Don't remember

First town meeting Trip to Mrs. Roosevelt's home Film series on Swedish cooperatives Farewell banquet Workshop on political action Field trip to the labor union Workshop on transportation Visit f r o m ambassador to the UN —• Staff-Camper sports contest 11. As you recall the group of Campers in your mind's eye, how would you describe the average Camper during your summer? 12A. I t has been some time since you went to the Encampment, but, as best you can recall, during that summer did you feel that the amount of agreement between your views on the following issues and the views held by most of the Campers was very great, moderate, or very little? Very great

Moderate

Very little

Don't recall

Civil liberties Economic planning — — Urgency of social reforms Race relations Importance of politics 12B. How about now, do you feel that the amount of agreement between your current views on these issues and the views most of the Campers held during that summer is very great, moderate, or Very little? [Same answer-boxes as in no. 12A.] 13. If you had the time, how interested would you be in any of the following Encampment activities:

368

Appendix B "Very Somewhat Slightly Not at all interested interested interested interested

Recruiting Campers Getting the Newsletter Raising funds for the Encampment A proposed citizen advisory system for youths (weekend seminars, consultation, ete.) 14A. Looking back at the Encampment in the light of your present knowledge and experience, what suggestions would you make as to ways in which future Encampments might be improved? 14B. What general feature of the Encampment did you like least? 15. Where did you go during the month immediately following your summer at the Encampment? 16A. Do you recall any instance during that first month in which you tried to apply what you had learned at the Encampment? (Yes, I tried; No, I didn't; Can't recall) I F T E S : B. Would you describe this event? C. What specific principle were you trying to apply? D. What would have helped you to handle the situation better? 17A. Have there been any other major instances since leaving the Encampment in which you have tried to apply any of its ideas? I F Y E S : B. Would you please describe one. C. What specific principle were you trying to apply? D. What would have helped you to handle the situation better? 18. Since leaving the Encampment, if you have participated in activities revolving around a public issue, problem or incident (e.g., housing, discrimination, etc.), describe briefly how you organized and what you did. 19. Thinking back to the year or so immediately following your summer at the Encampment, would you say there were any changes in the following areas of your life which you believe were brought about by the Encampment experience: A. Relationships with old friends from your pre-Encampment days? I F Y E S : What? B . Relationships with your family? I F Y E S : What? C. Relationships with your neighbors, acquaintances? I F Y E S : What? D. Association with people of other ethnic and religious groups? I F Y E S : What? 20A. At the time that you went to the Encampment, what were your major educacational plans? B. What were your major vocational plans then? C. What are your plans now? (1) Educational; (2) Vocational. D. I f they have changed or been modified in any way, do you believe that this change was brought about in any way by your summer at the Encampment ? I F Y E S : In what manner ? 21. When you think of the meaning of the entire Encampment experience, do you feel that it contributed to your personal growth in any way? In what way? II This series of questions deals with your current activities, interests, amci views. 1. Thinking back to recent elections, do you*recall whether or not you happened to vote in any of the following: (Please check one answer in each column.)

Appendix

369

B

B A 1952 1954 Presidential Congressional election election

C Any state election

D Any local election

Yes, voted No, but was eligible Was not eligible Can't remember 2A. Have you ever run for any political office? I F YES: What office? 2B. Have you ever held any political appointment? I F YES: What appointments? 3. Have you talked politics with anyone recently? (Yes, No, Don't know) I F YES: A. How did the discussion start? B. Did the other person ask your views? 4. When you and your friends discuss political questions, what part do you usually take? (Please check one.) Even though I may have strong opinions, I usually just listen. I listen a lot, but once in a while I express my opinion. I take an equal share in the conversation. I have definite ideas and try to convince the others. 5. Do you ever get as worked up about something that happens in politics or public affairs as you do about something that happens in your personal life? (Yes, No) 6. Have you ever written a letter to any government or public official? (Yes, No) 7. Since leaving the Encampment have you ever had occasion to address the public in any of the following ways? (Please check as many as apply.) Public speech Radio or television speech Writing in a newspaper or popular magazine Other (specify) 8. Do you regularly read any daily newspaper? (Yes, No) 1 9. Do you read any magazines r e g u l a r l y ? jj YES: Which ones? 10A. Would you please look down this list and check the two ways in which you usually spend most of your spare time: Check TWO Sports, outdoor activities, gardening Visiting with friends Dancing and party going Reading Painting, drawing, art work, handicrafts, playing a musical instrument Listening to radio, watching television, playing records Going to the movies, theater, concerts Church and club work Civic and political activities Other (specify) 10B. If you were fortunate enough to have a lot more leasure time, at which one of the above activities would you like to spend this extra time? 11. Of the following activities which are the two from which you expect the greatest satisfaction in your life time?

370

Appendix

B

Check TWO Your career Family relationships Leisure or recreational activities Participation as a citizen in the affairs of your community Participation in activities directed toward national and international betterment Religious activities 12. Here is a list of different kinds of clubs and organizations that people can belong to. Please look over the list and check in Column A any to which you now belong. Then indicate in Column B by writing in the approximate dates of your membership any to which you once belonged but do not any longer belong to. Then, in Column C, please list any offices which you ever held in any of these organizations, either currently or in the past. COLUMN A COLUMN B I now I used to belong : belong : Joined in L e f t in Business groups Church-connected groups Veterans' organizations Labor union Parent-teacher association . . . . Neighborhood clubs or community centers Professional groups or societies. Neighborhood improvement associations Youth councils or student organizations Other civic groups, e.g., League of Women Voters Charitable and welfare organizations Informal clubs or groups Political clubs or organizations. Farmers' organizations Other (please list) :

COLUMN C Offices held

—

13. What do you feel is the likelihood that men will be able to find a satisfactory solution f o r each of the following problems? How long do you think it will be before such a solution is achieved? (Express your opinion by checking the appropriate space for each problem, and writing in the number of years you think it will be before a solution will be found.)

371

Appendix B Highly unlikely

Doubtful

Fairly likely

Certain

How many years?

Race prejudice War Cancer Economic depression and unemployment 14. In peacetime, do you think that members of the Communist party in this country should be allowed to speak on the radio? (Yes, No, Don't know) 15. In your opinion, which one of these two is more important ? To find out all the Communists even if some innocent people should be hurt. To protect the rights of innocent people even if some Communists are not found out. 16A. How do you think most of the members of your Encampment group would answer these two questions? (Please check in Column A.) 16B. Thinking of most other people of your age and background in your community, in general how do you think they would answer the two questions? (Please check in Column B.) A B Most other My Encampment people of my group age and background To Question # 1 4 most would answer: Yes No Don't know I don't know how they will answer To Question # 1 5 most would answer: Find out all Communists Protect the innocent I don't know how they will answer

At this point in the schedule, items for the following scales occurred: tolerance, authoritarianism (modified version), civil liberties "A," constitutionalism. For the items in each scale, see Appendix A. For the arrangement of the items in the schedule, see the questionnaire for "Wave II, July, 1955. 17. Thinking of most other people of your age and background in your community, in general how do you think they would answer the following few questions? (Please circle as you think they would, or, if you cannot, check the last column.)

372

Appendix

B

I don't Most of them would: know how Be they would Agree Disagree uncertain answer 1) A book against churches and religion should be removed from the public library if someone in the community suggests this. 2) What this country needs most, more than laws and political programs, is a few courageous, tireless, devoted leaders in whom the people can put their faith. 3) Persons who refuse to testify against themselves (that is, give evidence that would show that they are guilty of criminal acts) should either be made to talk or severely punished. 4) I would stop buying the soap advertised on a radio program which had an admitted Communist as the singer, if someone suggested this to me. 5) So many other people vote in the national elections that it doesn't matter much to me whether I vote or not.

A

D

?

DK

A

D

?

DK

A

D

?

DK

A

D

?

DK

A

D

?

DK

Ill The factual questions below serve purely for statistical purposes in our analyzing and tabulating the replies. 1. Sex Male Female 2. Veteran's Status: Did you serve in any branch of the Armed Forces? I F YES: When were you inducted? When were you discharged? 3. If you were asked to use one of these four names for your social class, which would you say you belonged in—middle class, lower class, working class, or upper class? 4. family status: Married: (Specify year of marriage) (Specify number of children) Single: Living with parents? (Yes, No) Divorced or separated: (Specify number of children) Widowed: (Specify number of children)

Appendix

B

373

5. Present Place of Residence: A. City over 500,000 City 10,000 to 500,000 City 2,500 to 10,000 Rural B. State C. How long have you been living in your present community? D. How many different communities have you lived in during the last 10 years? 6. Present occupation: (Describe as specifically as possible) 7. What led you into that occupation? 8. Education: (Specify the highest grade in school or college completed) 9. Is your church preference Protestant Catholic Jewish Other (specify ) None 10. Please list any awards, fellowships, or honors which you have received since going to the Encampment. 11. Date: 12. Signature (Optional) V I I . COLLEGE BATTEKY QUESTIONS—ASKED IN 1957 AND 1958 ONLY

IA. About how much schooling do you think most young men need these days to get alone well in the world ? IB. What about young women—about how much schooling do you think most of them need these days to get along well in the world? 2. In thinking about a college education, some people consider it valuable mainly for one reason, others mainly for another. Which one of the following do you regard as the most valuable thing about a college education? Which one do you regard as the next most valuable? Please put a "1" after what you consider to be the most valuable thing about a college education, and a "2" a f t e r the next most valuable. College is valuable because i t : a. prepares a person for getting the kind of job he wants b. broadens a person's knowledge about the world he lives in 0. enriches one's appreciation of the traditions of thought lying behind present-day beliefs d. gives one time to find out what he wants to do in life e. equips one to be a better citizen f . provides the opportunity to meet new and influential people who will be of use to him later in life g. fosters the growth of a well-rounded personality h. develops a person's intellectual abilities 1. develops and strengthens a person's moral and character traits j. provides him with a few years of pleasurable respite before having to assume adult responsibilities fc. other (please specify)

374

Appendix

B

3. Since first starting at your present college, have you ever thought of going to another? (Quite seriously, never seriously, not at all) 4. Apart from military service, have you ever thought of taking at few years off from college? (Quite seriously, never seriously, not at all) 5 Have you ever thought of leaving college altogether and not completing your course of studies? (Quite seriously, never seriously, not at all) 6. Here are some reasons that students who left college gave for not finishing. Which do you personally feel are reasoanble grounds for leaving college? Which do you feel are not reasonable grounds? Not It Seasonable reasonable depends a. Parents object that college is doing student no good b. Is bored by studying, but finds real satisfaction in working with his hands c. Wants a job helping people who have problems and cannot see how spending two or three more years in studying will be of much use d. Wants to get married and raise a family — e. Is needed to help support his (her) family f . Feels the wide world provides a better education than do colleges g. Considers his (her) grades too low to warrant continuing in college h. Is offered a well-paying position in a reputable company having a good promotion policy i. The expense of going to college is too great j. Prefers to work full time in order to be financially independent of parents 7. Listed below are several areas of study. Would you first check in Column A those which interest you at all. Then, in Column B put the number "1" next to the area that interests you most and the number "2" next to the second most interesting area. A B Of some Two most interest interesting Business administration Education English and foreign languages, literature and d r a m a . . . Fine arts and music History — Philosophy and religion Physical and biological sciences and mathematics Social sciences and psychology

Appendix

B

375

8. Have you chosen a major or field of concentration yet? I F N O : I n which department or program do you think you will probably major? 9. Have you ever thought of doing any postgraduate work? Yes, I am already in graduate school Yes, I am planning to go to graduate or professional school Yes, but I have no definite plans Yes, but I've decided not to do it No, I've never really considered it 10. For college courses that deal with social problems or public affairs, which of the following ways of dealing with the subject matter do you think is right? а. Controversial matters should be discussed frequently because of the educational value of such discussion б. Controversial matters should be discussed honestly when they come up but the teacher should not seek out discussion o. Both students and teachers should in general try to avoid discussing controversial issues because such discussions have little educational value 11. Thinking of an ordinary week during the present college year, approximately how many hours do you spend during the week in each of the following activities? Attending classes Leisure-time reading Dating Studying Serious discussions with other students Taking part in organized extracurricular activities Other important activities (Please specify) 12. How much of the time t h a t you spend in class and on required assignments this school year is used in doing things that do not seem important to you? A lot of it Some of it Only a little of it None of it 13. Over this month, about how many classes would you say you have missed? 14. Do you think that, in general, you cut classes more or less often than most students you know? More often About as often Less often 15. Do most of the rules and regulations at your college seem useless to you? (Yes, No) 16. Do you ever feel that what you are doing at your college is a waste of time? Yes, often Sometimes, but not often Rarely Never

376

Appendix

B

17. I f you had an important pergonal problem you wanted to talk over with someone, whom would you be most likely to turn to ? а. One of my friends here a t college б. My brother or sister c. One of my friends f r o m home d. One of the college advisors to students e. My mother or f a t h e r f . One of my college instructors g. One of my friends at another college h. Other adults on the campus (specify) i. Other (specify) 18. Have you discussed political issues with anyone in the past week? (Yes, No) 19. When you and your friends here at college discuss political questions, what p a r t do you usually take? ( P L E A S E CHECK ONLY O N E ) I usually just listen I listen a lot, but once in a while I express my opinion I take an equal share in the conversation I usually express my ideas more frequently than others express theirs 20. Thinking back to last year in college, would you say t h a t then you were asked your opinions about politics more than the average person in your group, less, or about as much ? 21. What about now? Are you asked your opinions about politics more than the average person in your group, less, or about as much? 22A. Would you please look over the following list of kinds of student activities. First, would you check in Column A all those activities t h a t you are taking part in this year. A B Take p a r t in P r e f e r Campus religious groups Debating societies Departmental or preprofessional clubs (e.g., Biology Club) Intercollegiate organizations (except r e c r e a t i o n a l ) . . . . Language, literary, and cultural societies Musical and theatrical groups Publications (college newspaper, etc.) Political and public affairs groups Social clubs, dance committees, etc •——— Student government committees Team sports Other (Please specify) None 22B. Now, would you once more go over the list. If you could participate in only two kinds of activities, which two would you prefer the most? Please indicate in Column B your first choice with a "1" and your second choice with a "2". 23. Below are a number of speakers and topics. Would you please look down the list and cheek the three speakers whom you would be most interested in having speak, on the topic listed, a t your college?

Appendix

B

377

William Faulkner on techniques of novel writing William Jenner on the Communist threat Ralph Bunche on Arab-Israeli relations Leo Durocher on managing a major-league baseball team Eleanor Roosevelt on women in polities Earl Browder on a defense of communism Walter Reuther on forming a new labor party Bruno Walter on similarities in the music of Bach and Mozart William O. Eastland on maintaining segregrated schools Reinhold Niebuhr on Martin Luther's ideology George Shearing on jazz and American culture Emily Post on etiquette in dating relationships Dean Aeheson on recognizing Red China William Knowland on the UN and national sovereignty Cardinal Spellman on Catholic views of church-state relations.. Bob Hope on script writing for television William O. Douglas on desegregation and the Supreme Court.. 24. If the students of a college want to join them, which of the following organizations do you think ought to be allowed on their campus, which not allowed, and which are you not sure whether it should be allowed or not? Don't know Not Not enough about Allowed allowed sure organization Young Democratic Club Young Republican Club Young Socialist League Young Communist League Students for Peace Students for America Students for Democratic Action

APPENDIX C

T H E U S E OF AN INDEX OF EFFECTIVENESS TO CORRECT FOR CEILING EFFECTS

"effectiveness" has been indicated in this volume by net differences between campers' scores in the aggregate at two different times (e.g., just before and just after the Encampment). For example, in 1955, 74 per cent of the campers began the summer with the highest score possible on the Stouffer tolerance scale and 89 per cent ended with this score. The difference of 15 per cent is the net improvement brought about by the Encampment and is taken as the indicator of the Encampment's effectiveness in this area. As noted in chapter i, for a scale like this one, on which so many campers (three-fourths) begin the summer with the highest score possible on the scale, this measure gives a conservative and perhaps misleading impression of effectiveness. For, although the net gain is only 15 per cent, the largest net gain that could occur, given the initial scores, is only 26 per cent. More exactly, the scale could gauge improvements only among the 26 per cent who did not begin the Encampment with the highest possible score. The other 74 per cent may in fact have become more tolerant. But the scale would not show this. The problem becomes critical in comparisons between subgroups that start with different ceilings, but it affects descriptions of total effectiveness, too. The magnitude of change is necessarily understated if the scale can measure only the changes occurring in part of the population studied. One way to avoid understating the effect of the program in such instances is to measure change only among those whose improvement the scale in question could demonstrate. Hovland, Lumsdaine, and Sheffield, having analyzed the problem, suggest just such a measure: FOB THE MOST PAKT

P -P Effectiveness i n d e x = — ~ where P1 is the initial per cent and P 2 is the final per cent.1 Effectiveness is measured here, then, by net improvement divided by the percentage of persons whose improvement the scale is capable of showing. In the studies reported in this volume, the two scales most vulnerable to the problem of ceiling effects are the civil rights (discrimination) and the Stouffer tolerance scales, because from three-fifths to four-fifths of the campers each year began the summer with the highest possible 1 Carl I. Hovland, Arthur A. Lumsdaine, and Fred D. Sheffield, Experiments on Mass Communication (Princeton: Princeton University Press, 1949), pp. 284-289.

[ 379 ]

380

Appendix G

scores on these scales. Data are presented in table C - l in order to illustrate the reinterpretation of the Encampment's effectiveness that would follow if the Hovland-Lumsdaine-Sheffield index were used. When a rather high proportion (say, three-fifths) initially score in the highest scale category, then, one may balance the impression that only slight improvement has taken place by measuring improvement relative to those who by their initial scores could have shown improvement. Over all four Encampments, the simple-difference index shows net improvements ranging from 11 per cent to 26 per cent of the campers, in the areas of civil rights and tolerance. The alternative, more liberal index shows improvements ranging from 29 per cent to 64 per cent of the number of campers who could have improved in these areas. I n this, as in other respects, then, and as stated in chapter i, we employ a conservative procedure—one that tends to understate the effectiveness of the Encampments' program. TABLE

C-L

"EFFECTIVENESS" IN THE AREAS OF CIVIL RIGHTS (DISCBIMINATION) AND TOLERANCE, U S I N G THE HOVLAND-LUMSDAINE-SHEFFIELD I N D E X OF EFFECTIVENESS

Scale

Civil rights

Tolerance

Year

N

P e r cent with highest score WII

win

P2-P1 Diff.

Eff. Index = Pi - Pi 100 -

Pi

1955 1957 1958 a 1958"

100 104 81 36

62 58 59 72

81 70 85 86

19% 12 26 14

50% 29 63 50

All years

321

61

79

18%

46%

1955 1957 1958" 1958"

100 104 91 36

74 63 67 81

89 78 88 92

15% 15 21 11

58% 41 64 58

All years

321

69

85

16%

52%

• 1958, New Y o r k . •> 1958, California.

APPENDIX

D

TURNOVER TABLES AND TESTS OF SIGNIFICANCE

As STATED in chapter i, we consider net differences between campers' aggregate scores at two different times to be the appropriate measure of effectiveness. And we believe that ambiguity often attends the use of "tests of significance" of observed differences, as the conditions under which the tests give meaningful results are not usually met. Nevertheless, in order that others may assess the Encampment's program by alternative canons of effectiveness, we here summarize a large number of turnover tables; and, in order that they may judge whether the "significance" of the reported differences supports or qualifies the conclusions drawn, we give for each table summarized the probability (obtained by using a test described below) that the observed difference is due to chance. There are five of these tables. The first three, D - l through D-3, present evidence on the significance of the changes reported in chapter iv, "Immediate Changes Created by the Encampment." The last two, D-4 and D-5, present evidence on the significance of the stabilities reported in chapter vii, "Stability of Effects within the Home Setting ...," and in chapter ix, "Long-range Effects." All tables have the same format: in the first column is the name of the scale, next comes the particular Encampment, and then comes the summary of the turnover table. For example, the first line of table D - l reads: Summary of turnover table Scale

Civil liberties "A"

Year

1955

Improved

Same

Worsened

Total

Number

No answer

50%

36%

14%

100%

96

4

Thus, in 1955, on the Civil liberties "A" scale, 48 campers (48/96 = 50%) had lower scores (more libertarian) on the last day of the Encampment than on the first day, 35 remained the same, 13 had higher scores on the last day than on the first day, and 4 did not answer one or both sets of questions. The full set of data from which the above illustration was computed are presented in table D-6. The 35 campers on the diagonal stayed the same. The 13 to the right went from a lower to a higher score, that is, from a more to a less libertarian position, which, in the light of the [381]

382

Appendix D

Encampment's aims, is a worsening. The 48 to the left of the diagonal moved in the opposite direction, that is, improved. The two columns on the f a r right in tables D - l through D - 5 pertain to the test for the significance of the observed differences. The test used is based on the assumption that, if the Encampment's program had no effect, as many campers would change in one direction as in the other. 1 If the Encampment had not affected attitudes toward civil liberties, for example, half of the 61 campers in 1955 who changed at all on the civil liberties " A " scale should have improved and half should have worsened. As it was, 48 improved and 13 worsened. Is the difference between the observed ratio of those who improved to those who changed at all (48/61 = .79) and the chance ratio (.50) significant? That is, what is the probability that a difference so great (.79 - .50 = .29) could result from chance? The null hypothesis is that worsening is as frequent as improvement, and the test determines whether or not this is consistent with the fact that 48 of the 61 changes were improvements. An upper-tail probability is thus required. For the present data, if a = those who improved, and b = those who worsened, the formula provided by Wallis and Roberts for the standard normal variable becomes:2 K= (a-b-l)/(a In the example,

+ b)K

K = (48 - 1 3 - 1 ) / (48 + 1 3 ) 1 = 4.353.

This is significant at beyond the .001 level. This result is reported in the final column of table D - l , which is headed, "Probability that positive difference between ratio and .50 is owing to chance." I t might be noted that this heading is descriptive; to state the point more accurately, a figure in the column is the probability that a deviation from .50 in the expected direction as large as the one one observed would have occurred had there been no actual change in this direction. 1 The test is described in W. Allen Wallis and Harry V. Roberts, Statistics: A New Approach (Glencoe, 111.: Free Press, 1956), sec. 13.3.2.2. 2 Ibid., p. 427. Kenneth Lenihan worked out the simplification reported here and made the computations for the tables in this section of the Appendix.

i-I IO CCO to O 1-1 O M1-H Tt< O O

= = = =

= .74 69/93

.83 .69 .72 .75

V

< .001

8

20/24 25/36 18/25 6/8

1 ^ 1

1—1

87/119 = .73

co co

.76 .64 .82 .73

co

= = = =

i-H CM n

V

25/33 27/42 27/33 8/11

142/225 = .63

.68 .60 .66 .54 -=t
W o ftJîS >

p-o S D.a®H S .2 u

S >> " •í S •t 13 'S o PH ci&,

O •3 ^

o H

Tj 03

aO

3

a« MO •• d H 0) H a> OGG 2«

till ao

£o

mV fe M Sa Oü Qo

S o 05 fa S3 £ O S5 Di P H

5 fc *a

&
3 1 1 S'S teo So 3 s PÍ Ph Un J

o^ Ci aa> 9c3 So fi. u .2

a o O1-c 0oo Ph W

-eo fi.

o fi. o S »-•tí §•> B "2 O h -

â-S-r •ÏÏ oi C O o3 bü C ® "E .2 o * a j s oa» o i ì

•aS8-o2B " «.. « a h: «

392

Appendix D TABLE

D-6

COMPARISON OF EACH CAMPER'S ATTITUDE TOWARD CIVIL LIBERTIES AT BEGINNING AND END OP ENCAMPMENT Individual camper's civil liberties " A " score on last day of 1955 Encampment

Individual camper's civil liberties " A " score on first day of 1955 Encampment

8

10 5 3 2 5

0 1 2

3 4 5 6

—

2 1 2 1 2 13 — 2 4 4 1 —

9

10+

1 1 2 1 3

7

6

-

1

-

4

1

2

-

2

-

-

-

1

2

-

1

1

—

—

—

15

3

3

3

2

-

-

-

-

—

-

—

8 9

10+ Totals

27

11

23

9

-

-

INDEX OF NAMES

INDEX OF NAMES Adorno, T., 14 Allport, G., 102 American Friends Service Committee, 7, 11, 42, 81, 91, 115, 137, 139, 186, 215, 268, 310, 316 Anastasi, A., 34, 37 Antonitis, J . J., 44 Arrington, R. E., 77 Attia, S., 22 Barnitz, E., 43 Barton, Allen H., 68, 76, 302 Belson, W. A., 29 Bend, Emil, 69 Bendrx, Reinhard, 257 Bennett, J . W., 193 Berelson, B., 31 Bersh, P. J., 44 Black, Algernon D., v Black, Bertram, 24 Blenkner, M., 21 Bonilla, F., 33, 39,194 Borgatta, E. F., 7, 8, 24, 26, 27, 29, 41, 42, 231 f. Bureau of Applied Social Research, vi, 5, 154, 233, 316, 336, 338, 349, 356, 360, 366 Campbell, Angus, 280 Campbell, Donald T., 30, 239, 304 Canter, R. R., 32, 33 Carlson, R. O., 16 Chicago, University of, National Opinion Research Center, 18, 19, 34, 100, 113,153, 256, 281 Cochran, W. G., 30 Cohen, M. R., 6 Columbia Council f o r Research in the Social Sciences, vi Columbia University Bureau of Applied Social Research, vi, 5, 154, 233, 316, 336, 338, 349, 356, 360, 366 Cooper, E., 27, 28, 232 Cooper, Homer, 280 Crespi, L., 35 Davies, "V., 28 Dentler, Robert, 42, 52, 81, 91, 115, 137, 139, 142, 143, 146, 150, 186, 187, 268, 310, 311, 312, 316 Dexter, Carolyn, vi Dinerman, H., 28, 232 Dymond, R. F., 44

Fanshel, David, 16, 30 Festinger, L., 29 Ford Foundation, vi Foundation for Research on Human Behavior, 192 Freeman, H. E., 29 French, D. G., 20 Gaudet, H., 31 Gillespie, J., 102 Glock, C. Y., 33, 34, 37 Graybiel, A., 44 Gross, E., 28 Hassan, F . R., 22 Herzog, E., 21 Hovland, Carl, 10, 14 f., 28, 29, 32, 37, 65, 66, 68, 84,169, 263, 379 Huntington, Mary Jean, 302 Hutchinson, E. M., 7 Hyman, H., 77, 83, 257, 281, 316 Jacob, P. E., 76, 257 James, H. E. O., 43 Kader, M. A., 22 Katz, D., 77 Kelley, H., 29 Kish, L., 70 Klineberg, O., 4, 65 Lana, R. E., 33 Lazarsfeld, Paul F., 6, 14, 31, 64, 67, 68, 250 Lenihan, Kenneth, 382 Lesser, Saal D., v Levenson, Barbara, 153 Lewin, Kurt, v, 3 Lindzey, G., 263 Lippitt, R., 22, 76, 192, 215, 263 Lipset, Seymour Martin, 257 Lumsdaine, Arthur A., 10,169, 379 McKnight, R. K., 193 Mathur, J . C., 34 Menzel, Roslyn, vi Merrill, M. A., 35, 37 Merton, Robert K., 183,189, 302 Meyer, H. J., 7, 8, 24, 26, 27, 29, 41, 42, 231 f . Michael, John, vi Michael, Sanci, vi Michigan, University of, Survey Research Center, 34

[ 395]

396

Index

Miller, K. M., 7 Miller, Norman, 302, 306 Murphy, G., 14 Murphy, L., 14 Nagel, E., 6 National Opinion Research Center, 18, 19, 34, 100, 113, 153, 256, 281 Neurath, Paul, 34 Newcomb, T., 14, 183, 265, 266, 303, 305, 306, 315 Office of "War Information, 75 Olson, Willard, 20 Opler, Morris E., 15 Parten, M., 311 Passin, H., 193 Piven, Herman, 69 Purdue University Division of Educational Reference, 104, 108, 110 Research Center for Group Dynamics, 22 Riecken, Henry, vi, 3, 4, 7, 11,14,18, 25, 32, 40, 41, 42, 52, 76, 80, 82, 84, 85, 91, 115, 116, 137, 142, 143, 145, 151, 152, 153, 161, 179, 186, 212, 215, 216, 217, 224, 225, 265, 313 Roberts, Harry V., 382 Rogers, C., 43, 44 Rosenberg, M., 6, 64, 67, 68 Saenger, G., 25, 65,232 Schoenfeld, W. H., 44 Schwartz, S., 33

Schwarzhaupt Foundation, v f. Scott, F. D., 193,199 Selltiz, C., 43 Selvin, Hanan, 70 Shannon, William G., v Sheffield, Fred D., 10,169, 379 Short, J . , 28 Smith, G. H., 14 Smith, M. B., 9, 24,192 Soboi, M. G., 34 Solomon, R. L., 30, 32 Somers, Robert, 154 Stouffer, S. A., 19, 51, 55, 112, 113, 138, 153,161 Stover, R. E., 56 Tenen, C., 43 Terman, L. T., 35, 37 Thielens, Wagner, 250 Useem, John, 193,199, 203 Useem, Ruth, 193,199, 203 Wallis, W. Allen, 382 Wassif, I. M., 22 Watson, J . , 192, 263 Weeks, H. Ashley, 29, 51 Weir, J . M., 22, 44, 75 Wheeler, L., 44 White, P. D., 44 Williams, C., 44 Wilson, E. C., 33, 39,194 Winograd, B., 33 Wright, Charles R., 281, 316