The skills of argument 0521404517, 052142349X

Citation preview

The skills of argument Deanna Kuhn Columbia University Teachers College

Th,• rrghl

l{)fJ~l•,'im~

~·-~ •,;,•~·····

1@\?J .i. j\lj'l.c: \'½"1;;.. -~·

of rh,

Vmver. LACK OF EMPLOYMENT--> LACK OF INCOME--> monetary needs --, RC (88) Discrimination

against ex-convicts___________./ Scarcity of jobs

2. Regimentation of prison life--> DIFFICULTY IN READJUSTMENTTO SOCIETY--, stress--, RC (53) Lack of support from others_____________. /

Discriminationagainst ex-convicts/ 3. Poor childhood environment-->

PSYCHOLOGICALMALADJUSTMENT--> RC (25)

4. POORENVIRONMENT--> RETURN TO ENVIRONMENT--> criminal life becomes habitual --, RC (85) Inexperience with other ways of life/ 5. Genetic endowment--> 6. Unwillingness

[absence of other causal factors]-->

CRIMINAL NATURE--> RC (17)

to work for rewards --> REWARDGAINED FROMCRIME--, RC (28) Greed ___________.

7. THRILL, FEELING OF POWERGAINED FROMCRIME --> RC (9)

8. DIVERSION, HOBBY to occupy time--> RC (7)

9. Return to same peer environment-,. INFLUENCE OF PEERS-,.

RC (22)

10. Drug addiction ->NEED TO SUPPORTDRUG HABIT-> RC (13) 11. Negative past experiences --> bitterness --> REVENGEfor outstanding grievances --> RC (26) 12. PUNISHMENT INSUFFICIENTTO DETER--> willingness to risk punishment-->

13. Poor prison conditions-,. negative experience in prison-,. 14. Exposure to other prisoners-->

RC (15)

FAILURE OF PRISON TO REHABILITATE-,.

RC (50)

KNOWLEDGE, ACCEPTANCEOF CRIMINAL BEHAVIOR--> RC (21)

15. Unpleasantness of life outside prison--> DESIRETO RETURN TO PRISONLIFE--> RC (17)

16. Lack of moral training-,.

LACK OF MORAL OR RELIGIOUS VALUES PROHIBITING CRIME-,.

17. Positive image of crime, criminals in culture-,.

ADMIRATION

OF, IDENTIFICATION

RC (?)

WITH CRIMINALS-,.

RC (3)

18. POORDIET CONTAINING SUGARADDITIVES--> RC (I) I 9. LOW FEAR THRESHOLD--> RC (I)

Note: The numbers in parentheses at the end of each causal line indicate the prevalence of that causal line, i.e., the total

number of times that it appears in subjects' theories.

Table 2.2. Sufficient causes of school failure (SF) I. Physical problems (hearing, eyesight, nutrition)-. 2. Genes

----,>

LACK OF ABILITY, INTELLIGENCE---+

SF (28)

frustration---+ SF (33)

3. Lack of maturity, readiness---+ MISMATCH BETWEEN STUDENT AND CURRICULUM---+ frustration---+ SF (4) Individual learning needs, style_,..,....-,, 4. Other activities---+ LACK OF TIME DEVOTED TO SCHOOLWORK---+ SF (12)

5. LEARNINGDISORDER -. SF (9) 6. LACKOF ACADEMIC SKILLS(study, language, reading)-. 7. School regarded as useless-.

frustration-.

no desire to be in school-.

SF (9)

LACK_OF MOTIVATION-. lack of effort-.

// Lack of expectations by others / Lack of self-discipline 8. Work too hard-.

SF (52)

(laziness)

LACKOF INTEREST-. lack of attention (boredom) SF (27) /' _____________. Work too easy lack of effort

"

"

9. Distractions -. LACKOF ATTENTION -. SF ( 14) JO. Physical appearance-. low self-esteem-. Racial prejudice__......,....-,

LACKOF SELF-CONFIDENCE-.SF (15) .

11. NEGATIVEATTITUDE,AFFECTtoward school, teachers -. rebelliousness -. SF ( 11) 12. TEMPERAMENT, personality-.

poor adjustment to school-.

13. DRUGUSE--> poor mental functioning-. 14. Marital discord-.

SF (7)

SF (14)

problems at home-. EMOTIONAL DISTRESS-. lack of concentration--> SF (67) Prol!lems with peers__________.

15. Desire to please, be like peers-+ PEERPRESSURE, MODELS not to do well-+ SF (34) 16. Social class---+ educational background of parents---+ LACK OF PARENTAL SUPPORT---+ lack of motivation

Parents' employment --> parents unavailable to help/ 17.

lack'M positive attitude

PARENTAL PRESSURE TO ACHIEVE---+ stress---+ resentment---+

'--..,.SF (76)

SF (9)

18. Parental restrictions---+ resentment---+ REBELLION---+ SF (6)

I 9. Teacher's lack of skill -. POORTEACHING Teacher's lack of effo1/ -------,. lack of motivation--> SF (42) Poor curriculum

20. Teacher's lack of effort-.

LACKOF ENCOURAGEMENT FROMTEACHERS-. lack of motivation--,

21. NEGATIVE ATTITUDE OF TEACHER TOWARD STUDENT~

SF (4)

resentment---+ SF (5)

22. Large classes---+ LACK OF INDIVIDUAL ATTENTION FROM TEACHERS---+ frustration---+ SF (18) Teacher's lack of effort/

23. LACK OF DISCIPLINE IN SCHOOL ENVIRONMENT ---+ environment not conducive to learning ---+ SF (7) 24. POORCOMMUNICATION between teacher and parent-. /

SF (I)

lack of initiative -----------.

25. TELEVISION VIEWING'-.. _________. SF (3) "- lack of verbal skills 26. Poor PHYSICAL ENVIRONMENT of school-. 27. PUBERTY-. distraction from schoolwork-.

poor learning conditions-.

SF (2)

SF (I)

28. DESIRETO GET ATTENTION-->SF (3) 29. LACK OF EXTRACURRICULAR ACTIVITIES TO ATTRACT STUDENTS' INTEREST---+ SF (1)

30. INABILITYTO SEPARATE FROMPARENT-. SF (I)

Note:

The numbers in parentheses at the end of each causal line indicate the prevalence of that causal line, i.e., the

total number of times that it appears in subjects' theories.

28

The skills of argument

the defining elements that provide the basis for classifying a theory as reflecting that causal line. Other antecedent or intermediate elements that extend or elaborate the theory are optional and may or may not appear in a particular subject's use of this causal line. Optional elements that appear above or below one another are alternatives that generally would not both be included in an individual subject's theory. The causal lines are numbered consecutively for further reference. As will be illustrated shortly, a theory may involve a combination of more than one causal line. The total number of causal lines identified across all subjects' theories is 487 for the crime topic (an average of 3.04 per subject) and 515 for the school failure topic (an average of 3.22 per subject). The number in parentheses at the end of each causal line in tables 2.1 and 2.2 indicates its overall prevalence, that is, the number of times that causal line appears in total in subjects' theories. Differences in these frequencies as a function of the subject variables of age group, sex, and education level are of some interest. In contrast to what we will see to be the case for structural analyses, subject-rroup differences occur mostly as a function of age group and sex, but not education level. Detailed examination of these differences for individual causal lines would take us too far from the objectives of the present work, but frequencies by subject variable are presented in appendix 4 for the interested reader. As reflected in appendix 4, for both topics total number of causal lines increases slightly with increasing age and education level and is slightly higher for females than males. STRUCTURAL

ANALYSIS OF THEORIES

More important than the content of subjects' theories for our purposes are any structural features in terms of which these theories might be characterized, in particular, features likely to bear on the form that argumentation in support of the theories might take. A possible starting point for structural analysis is the number of different causal lines included in a subject's theory. What becomes evident, however, is that more important than the number of lines is the relationship that these lines bear to one another, and it is the latter feature on which structural analysis will focus. Another possible basis for structural analysis is the number of elements (and conceivably their configuration) within a causal line. Investigation of this feature, however, shows it to be only a surface feature of theories, dependent largely on the degree of verbal elaboration the subject provides. As the examples to which we now

Causal theories

29

turn will illustrate, some subjects' theories that are very detailed and incorporate many elements and lines can be regarded as less complex strUcturally than others that involve very few elements.

Single-causetheories Single causal lines

We begin with an example of a simple, unelaborated single-cause theory that includes only a single causal line. In the following examples, the symbols in parentheses at the beginning of the quotation indicate the subject's age group (T, 20, 40, or 60), noncollege or college education group (N or C), sex (m or f), and whether the topic is crime or school failure (c ors). These subject identification symbols will be used throughout the book, whenever subjects' responses are quoted. In all of the responses quoted in this chapter, unless otherwise indicated the question the subject is responding to is the introductory question indicated earlier, "What causes __ ?" The following is an example of a teen subject's theory regarding the cause of school failure: (TNfs) Emotional problems. Problems at home. (Anything else?) Yeah, that their parents are divorced. This theory is a simple expression of causal line 14. The interviewer's probe leads the subject only to further descriptive elaboration of this cause, but not to the introduction of any other causal lines. The following example for the crime topic is elaborated more fully but similar in reflecting a single causal line (2): (40Cfc) I believe that once a man or woman has spent time in prison ... I'm not sure but it's like having someone to tell you ... it's like a parental thing ... having someone to tell you when to wash, when to eat, when to sleep, you know,all workedout for you. You have no responsibilities.Your schedule is all worked out for you. And I believethat they cannotdeal with society,with the outside world. (Anything else?) Because I've known people that have, you know, spent time in prison. They come out, they stay out maybesix monthsand they are back in again. And I do believethis is because they cannot deal with society as a whole. The next example is likewise of a simple theory involving a single causal line, but it is instructive in illustrating the interrelations that . exist among a number of the causal lines in tables 2.1 and 2.2:

30

The skills of argument

(20Nms) Well, number one, what causes kids to fail in school I think has to deal with not being motivated. (Anything else?) And it stems from the way they are brought up at home. I think the parents have a lot to do with it. The subject's initial response by itself leads to a straightforward classification as causal line 7. The elaboration following the interviewer's probe, however, extends the locus of causality back to the parent. As the (implied) inferior parental treatment becomes the initial cause in the chain, the entire response is classified as reflecting causal line 16. Before proceeding to more complex theory structures, it is of interest to examine the remedies proposed by the three preceding subjects, in order to illustrate the consistency that most subjects exhibit in this respect. The teen subject whose causal theory is quoted initially, for example, refers to counseling at school that would help students with the emotional problems that the subject sees as the cause of school failure: (TNfs) (Is there any one important thing which, if it could be done, would lessen school failure?) Yes. You know, like we have, in this school, group guidance, and like one thing they could talk, you know, about how they feel about certain subjects. Maybe somebody could help them. The remedy proposed by the second subject quoted is less directly tied to the readjustment difficulty that she sees as the cause of return to crime, but the connection is nonetheless clear: (40Cfc) (ls there any one important thing which, if it could be done, would lessen prisoners' return to crime?) I think ... well, first of all, I believe better education. (How would that help?) Because I

believe that if you have a better education I believe that you will understand society better and by having a better education I believe that you can have better employment. Finally, the remedy that the third subject proposes reflects his sense that the parental factors he sees as causal might be difficult to alter, and he turns to a different factor. The consistency between causal theory and proposed remedy nonetheless remains clear: (20Nms) (ls there any one important thing which, ifit could be done, would lessen school failure') I think it is, to me, the teacher nowadays would have to be like a parent and motivate them.

Causal theories

31

Multiple causal lines

We turn now to examples of single-cause theories that include multiple causal lines. The following is a straightforward example: (40Nfs) I have plenty of ideas on that. I think that it's inadequate teaching and inadequate teachers. I think that the school system is not set up to handle, you know, to take care of the children's needs. I think it is too bureaucratic and the teachers are too interested in, you know, their pension and everything else to be bothered with kids. (Anything else?) And the individual schools too are also, you know, very geared toward getting high marks, you know, in this and that, and so forth, and they don't concentrate enough on the kids who are slow learners and, you know, who have problems ... individual problems.

This theory is classified as containing causal lines 19, 20, and 22. In contrast to the preceding examples, a number of different causal lines (as well as causal elements within lines) converge to produce the outcome. Nevertheless, this theory is classified in the larger category of single-cause theories because the subject's belief is clear that all of these causal elements operate together and contribute to the outcome. We will shortly compare single-cause theories with multiple-cause theories, in which more than one possibility is conceived of and different causal lines are considered as alternatives rather than contributory. The following is a similar example of a single-cause theory involving multiple causal lines for the crime topic: (40Nmc) I think some of our laws today are really not strict enough. (Anything else?) Maybe some of the rehabilitation supposedly that they are supposed to have did not do the job. And let's face it, today some of them even commit murder and they are out on the street the next day. Plus, they've got the overcrowding in the prisons and I don't think ... well, in some places they are letting them out before time. They just don't have the room to put them anymore.

This theory is classified as containing causal lines I 2 and 13. Like the preceding subject's theory, this theory incorporates a number of different elements. . It is possible to diagram subjects' causal theories to portray these Imes and elements and their interconnections. Such a diagram of the preceding theory is shown in figure 2.1 b, contrasted to a diagram of

32 (a)

The skills of argument DIVORCED PARENTS

PROBLEMS ATHOME

Causal theories EMOTIONAL

PROlll.EMS

LENIENT SENTENCll'll BYJLOGES,

the field but I can at least learn the fundamentals and then go on from there. So, I really think the education system is falling down.

SCfOJL FAILURE

This theory is classified as containing causal lines 16 and 19, as part of a single causal structure. The next example is similar in form, except that the subject is less explicit in integrating the two causal lines into a single structure:

REHABILITATION FAILS TO

OCCUR

RE1U1N TO

(b)

CRIME

OVERCROWDING

EARLY RELEASE

33

PRISON SENTENCE INSUFFICIENT TODETER FRCM NEWCRIME

Figure 2.1. Single-cause theories.

the simple single-line theory offered by the first subject quoted in this chapter (figure 2. la). As figure 2. lb reflects, the elements and lines all converge toward, and contribute to, the outcome. It is clear that for the subject, they are aspects of a single causal structure. In the examples we turn to now, the causal lines are more distinct, such that they could more readily reflect alternative causal theories. The subject makes it clear, however, that these lines are seen as aspects of a single causal complex that produces the outcome. The following subject, for example, explicitly says that this is the case: (40Nms) 1 really don't know if you can put your finger on one particular thing. I would like to say that it's all the teachers' fault, but I don't think that's true. I think it's teachers and family. I know a lot of children that their parents . . . the kids come home from school and they couldn't care Jess if they've got homework or don't. So it can't just be the teachers' fault. 1 think it has to be a whole bunch of things tied in there. But I would definitely put the education system right on top of the block. I think we are getting too far away from the basic things. I read an article in the paper the other day where some phenomenal percentage of children cannot even tell you where a certain country is, what continent it is on, because we don't teach

geography any more. When I was going to school that was a subject, geography. I don't think they are teaching our children how to study. I don't know a lot of things, but I can pick up a book and I can read and I can at least get the basics. That doesn't mean I'm an expert in

(60Cmc) The education they get in prison, and I don't think they do too much to rehabilitate them in prison. They have not got the facilities, they haven't got the time, they haven't got the equipment, they haven't got the manpower, and that's one of the causes. (Anything else?) Well, you have to remember that when a prisoner comes out of prison, he is almost to middle age. If they enter at say twenty-five or thirty [years old], and come out, and, as I said, they are not taught anything new, and I think it is very hard to adjust to a new way of life because life moves very fast and after a couple years being away from society and coming back to society, there is quite a change. It is very hard for a person to adapt themselves to changes. This theory is classified as containing causal lines 13 and 2, as part of a single causal structure. The fact that the second causal line is introduced in response to the interviewer's probe enhances the possibility that the subject is introducing an alternative cause. A revealing factor, however, is the fact that the subject refers again to the initial causal line in his discussion of the second line ("as I said, they are not taught anything new"), suggesting that the two lines are conceived of as sequential parts of a single causal scenario leading to the outcome. However, the definitive criterion used to differentiate single-cause from multiple-cause theories is the nature of the subject's response to the question regarding major cause. If the multiple causal lines are integrated into a single causal structure, the subject does not isolate a single causal line in response to this question. Instead, all of the initial causal lines are included in identification of the major cause, as is the case for the preceding subject. He begins with a clarification regarding causal line 13 but then proceeds to causal line 2: (Which of these would you say is the major cause of prisoners' returning to crime?) By education, I don't mean professionalism. I mean trades, incentives. A male needs a trade. Forget about professionalism. They are way past becoming doctors or lawyers. But as tradesmen, most of these people who go to prison have not got any trades and if they do learn a trade in prison, it is hard for them to

34

The skills of argument move into society because of unionism, and of course the fact that

they are elderlypeople now.Things are differentthan thingsthat they had known before and it is a new way of life which in itself is very hard to adapt for a ... I won't say middle-aged,but a person the age of thirty-fiveor forty. It is very, very hard. I can just try to picture myself, put myself in their place, coming out to what ... to what? This latter response clearly integrates the two causal lines into a single scenario leading to the outcome. Sometimes, the interviewer's question regarding major cause may even serve to elicit an explicit statement of the interrelation of causal lines, as in the following example: (60Nfc) I don't think they are rehabilitated.(Anything else?) I don't think there are jobs waiting for them. Probablytheir home lives are the same as when they left. (Which of these would you say is the major cause of prisoners' returning to crime?) I think it is a combination of everything. This response is classified as containing causal lines 13, 1, and 4, as part of a single causal structure.

Multiple-causetheories We turn now to the contrasting cases in which multiple causal lines are not integrated into a single causal structure. Multiple parallel causal lines

In the theories classified in this category, multiple causal lines are involved. In contrast to theories in the preceding, single-cause category, however, the subject presents the causal lines successively and gives no indication at any point in the interview that they are integrated into a single causal structure. In contrast to theories in the next category (multiple alternative causal lines), neither does the subject give any clear indication that the causal lines are regarded as alternatives. We therefore designate them simply as parallel. The following is an example: (TNmc) The way other prisoners talk to them about crime when they're in prison. (Anything else?) They can't find a job after they come out of prison.

Causal theories

35

This theory contains causal lines 14 and I. In contrast to the two final examples in the single-cause category (in which the content is similar), this subject gives no indication that the two causal lines are linked into a single causal structure, and it is entirely possible that the subject conceives of the causal line offered initially and the one offered following the interviewer's probe as distinct causes, either of which would be sufficient by itself to produce the outcome. The following is a similar example for the school topic: (TCfs) I'd say pressure from parents ... you know, expecting too much from their kids. From their friends, maybe ... pressure from their friends to fit in the crowd, maybe. (Anything else?) Lazy, maybe, themselves, not determined, not motivated. Then maybe teachers pressuring and picking on them, that sort of thing. This theory contains causal lines 17, 15, 7, and 21. Again, these causal lines are presented sequentially, with no indication of their being linked into a single causal structure, and any one by itself could be a sufficient cause. In all of the theories classified in this category, subjects' responses to the subsequent question regarding major cause ("Which of these would you say is the major cause of __ ?") are consistent with the interpretation of the theories as involving distinct causes. In contrast to subjects whose theories are classified in the single-cause category, the subject always identifies only one of the causal lines in response to this question, and then in the remainder of the interview is consistent in addressing only this cause, with no further reference made to other causal lines mentioned initially. Another pattern that occurs with some frequency resembles the preceding one in the initial sequential presentation of two or more causal lines, followed by identification of only one as the major cause. In the next segment of the interview, however, when the subject is asked for evidence in support of the theory (chapter 3), the cause referred to broadens to include the additional causal line(s) offered initially, making it clear that these causal lines are in fact integrated into a single causal structure that comprises the subject's theory. In these cases, definition of the major cause is broadened to include the additional causal line(s), and the subject's theory is classified in the single-cause category. (Reverse cases, in which the cause referred to in the generation of evidence narrows from that identified initially as major cause,

36

The skills of argument

do not occur.) It should be noted, then, that the final definition of major cause is the cause with respect to which the subject generates evidence. This definition is important, as what is identified as the major cause becomes the basis for the remainder of the interview. Finally, it should be noted that additional probing might have been introduced that would have enabled at least some of the theories in the multiple-parallel category to be classified as having the multiplealternative structure described next, in which the causal lines are clearly established as alternatives. Introduction of the subtle probing that would be required to ascertain this, however, would have disrupted the natural conversational flow that we thought important to establish early in the interview, particularly the progression from "What causes __ ?" to "How do you know that this is so?". The possible negative effect of such a disruption seemed to outweigh the value of the information that would be obtained, especially since later in the interview (chapter 4), subjects' ability to generate alternatives to their own theories is directly assessed. Structural analysis of initial causal theories is thus confined to the three-category system described here.

Multiple alternative causal lines We tum now to cases in which the subject indicates explicitly that alternative causal structures are involved. Subjects do so in a number of ways, the most straightforward of which is the simple or language illustrated in the following example: (TCmc) They probably don't havejobs. They can't make money in other ways, so they got money in the past by stealingit, so that's the only way they're used to getting money.Or else theyjust want to be cruel or somethinglike that. (Anythingelse?) A hobby, maybe. Or just money. The initial causal line identified here is an integration of I and, to a lesser extent, 4. The subject then introduces the or conjunction and presents a very different causal line, 5. In response to the interviewer's probe, he again uses the alternative or structure, contrasting a new causal line, 8, to the original 1/4. It is clear that the subject conceives of at least two distinct causal structures, either one of which is sufficient to produce the outcome. Consistent with this interpretation, in

Causal theories

37

response to the interviewer's probe, the subject clearly indicates only the initial cause presented (1/4) as the major cause. Another form in which the multiple-alternative structure is made explicit is in the use of some language to refer to different subgroups of people to whom different causal explanations apply. The following is an example: (40Cms)It's fairly complicated.It dependson circumstances.Obviously some children fail because of innate lack of skill. In other words, I think peoplecan fail becausethey don't have the intellectual capacity to pass. There is also a whole group probably who are not motivated.Therefore, they don't study and so forth. That's a different group who have the capacityto do it but they aren't motivatedto do it. And then I would say there is a group who haven't ... and how large that group is I don't know ... who haven't been reached by the system again. In other words, there is a group who because of their needs in terms of learning style or methodologyis not met by their particular school system, the circumstancesthey find themselves in.

Causal lines 2, 7, and 3 can be identified in this subject's theory. The structure is clearly an alternative one, with each causal line applying to a different subgroup. In response to the interviewer's probe ("Anything else?"), the subject elaborates causal line 3, and does not add any new causes. The subject's response to the interviewer's question regarding major cause is clear:

(Whichof thesewouldyou say is the majorcauseof children'sfailure in school?) I would say it's a combinationof the lack of specific needs being met by educationalmethodologyand lack of motivation. I think the two are kinds of ... they are slightly interrelated. Whether or not they are the same or not is another question. Thus, whereas lines 7 and 3 are integrated into a single structure, causal line 2 maintains its status as an alternative structure. The two resulting structures apply to different subgroups and are each sufficient causes to produce the outcome. Sometimes a subject makes it clear who the different subgroups are to whom different causal analyses apply. The following subject, for example, identifies two subgroups. (20Nms) Well, it depends. It depends on the background . . . whether they are from a rich family or a poor family. It is

38

The skills of argument

Causal theories

~

LACK OF JOB

PARENTALPAESSURE

I

PEER PRESSURE

H

SCHOOL FAIWRE

I

LACK OF MOTIVATIO~

]

·I

I

TEACHER'S BEHAVIOR

]

•

I

I

SCI-KXX.FAIWRE

SCHOOL FAIWRE

H

Causal line 13 (drugs) is present in both causal structures, but in the second structure, applicable to poor children, causal line 16 is also present. One final form of alternative-cause structure is different in that one of the structures is negated. In other words, the subject identifies a causal structure that he or she claims is not the case. Even though there may remain only a single simple causal structure that the subject endorses as true, responses of this form are classified in the alternativecause category as the subject has identified two (or more) alternative causal structures. The following is an example. (40Cms) I would say that most cases would be a lack of parental support. (Anything else?) No. I think, you know, there are probably other reasons, but that would be the main one to me. I don't think that the quality of education can really be questioned, you know, in all the schools. And I don't think that most ... maybe not all, but most ... kids are born without an opportunity, you know, you know, are born dumb. I think they can learn and I just think mainly parents are so involved in what they are doing that they are not involved in what the kids are doing, and they don't encourage them. Thus, the cause the subject endorses is a clear instance of causal line 16, but the subject also negates two different causal lines, 19 and 2. The various forms of causal structure that have been portrayed are summarized in figures 2.1 to 2.3, drawing on content from the examples in this chapter. As a comparison of them reflects, the difference in

RETURN TO CRIME

LACK OF MONEY

LACKOF ALTERNATIVE SOJRCESOF~

OR

(a)

I CRUEL~TIJRE

I •I

R~:erol

OR

EJ

Figure 2.2. Multiple-cause theory (parallel).

differentcauses. Cause I know a lot of kids that are doing pretty bad, and they are rich kids ... upper middle class, but they mess around with drugs too much. On the other hand, where they come from poor families, they are doing pretty good, but there are a lot of kids from poor families that do bad. There are probably drugs and a bad home.

39

0CM;

(b)

I ~7~~

•I

R~:roj

I

•[

SCHOO-FAIWRE

MISMATCH BETWEEN STUDENT

ANDCURRICULUM OCMe

SCHOO. FAIWRE LACKOF MOTIVATION

IL.ACKOF PARENTAl SUPPORT

(c)

NOT

NOT

POOR TEACHING

t

I SCl-01 I FAILURE

~

LACK OF

ABILITY

Figure 2.3. Multiple-cause theories (alternative).

structure is captured not by number of elements or causal lines but the relationship between causal lines, which can be either absent (if only a single causal line is present, as in fig. 2. la) or contributory (fig. 2. lb), in the case of single-cause theories, or either parallel (fig. 2.2) or alternative (fig. 2.3a-c) in the case of multiple-cause theories.

The expressive dimension To illustrate one other respect in which subjects' theories vary, we present one more example of a theory classified in the multiplealternative category. All of the examples presented thus far are ones in

40

Causal theories

The skills of argument

When it's not ... if it's not ... Even though I never think it's right, even if it's need, I think that if someone is doing it just to get more, instead of to live ... I have trouble saying this ... but if

which subjects are fairly direct and succinct in their answers to the interviewer's questions. In fact, subjects' responses cover a wide range in this respect. Toward one end of the continuum are responses like the ones quoted thus far. At the other end of the continuum are responses that are much longer and less articulate. Typically they contain unimportant detail and numerous digressions from the issue being addressed, and they reflect less fluidity and ease in expression of ideas. To make the reader's task easier, we shall in the remaining chapters of the book continue to draw largely on examples closer to the former than the latter end of the continuum. In order to give the reader a sense of the full range encountered, however, we present an example here at the opposing end of the continuum. A close reading makes it clear that the subject's theory falls into the multiple-alternative category, but the subject requires many false starts and ultimately many words to get his ideas across: (TCmc) What causes them? Alright, I've started reading a book on this, so ... It's called In Cold Blood, and it's about murdererswho were in jail. It's a documentary.I think ... I would say that most people who come out of jail, and I would say a majority of people that .... When you say "ex-cons"do you mean robbers?Youdon't mean murderersand real serious stuff like that? (Youcan define it any way you want.) Okay. I'd say that ... I'd say, in lots of examples, if · it's not murderers, who are mostly psychic ... they're really psychotic and crazy and they kill just for a liking to kill, unless, you know, they're hit men or something. I think most of the time it's because of money.That's what I was going to get to. Even when it's about ... you know, it's ... I always hear about those few people that just steal as a thrill of life, but I think realistically,for the most part, it's just because of money.People are poor. It can be need, but because it becomes need, it can become also a way of life, so then it becomes greed, because of the ... I don't think you can talk about murders and robberies in the same, you know ... because I think there is no reason to do murders. And, I think robberies ... I can see it. If someone is dying, even though it's not right ... I'm not saying that . . . If someone is dying of hunger and they have ten kids, they'd probably, out of need they'd rob a store because they're maybe out of food or just, you know ... 1 don't know but I'm sure there are a lot of people ... people for whom moneyis power, even though it's only of theirown ... If someonerobs a bank, they're not going to show off their money.It's for their own person, you know. It makes them feel good. That's when it gets to greed because ...

41

someone is just a middle-class, you know, if someone's just a

middle-classperson and they just do it becausethey want more, they could live the way they are and they could feed their children or whatever, you know. It's because I guess they think they deserve more or something, but that's ... It's still greed because ... Somehowthat's sort of like the failing topic because they should do something for themselves. They could ... it's not someone who is ...

You see, when it's a case of someone who is very poor and

who needs money, they don't have much that they could do for themselvesbecause they don't have the money to start a liquor store on the corner. I think that what happens when people could have another choice ... they have other alternatives ... then it's greed. I don't know. It's really a hard topic. (Anythingelse?) I guess there's always cases where there are people that do it for revengeand stuff. You know, they're just crazy. You see, there are lots of moral ... you know, you can think of reasons that, like money,that are much more realisticthan the Supermanmovies where you see people getting revenge and stuff. But ... there are cases ... I'm missing ... I'm sure there are other things that I'm just not thinking of.

In response to the interviewer's question regarding major cause, the subject replies "to get money," and the subsequent evidence generation is consistent with this major cause. The subject's causal theory is thus classified as containing causal lines I, 4, and 6 integrated into the major causal structure ("It can be need, but because it becomes need, it can become also a way of life, so then it becomes greed"), with causal lines 7 and 11 as alternative causal structures applying to subgroups of

.cases. Few subjects are consistently, throughout the interview, as verbose or inarticulate as the preceding one. Hence, as a general note, the fact that reasonably clear, concise responses will be used as examples in the remainder of the book does not imply that the reader will gain firsthand acquaintance ·,,ith the thinking of only a small proportion of particularly articulate pruticipants in the study. To the contrary, an effort was made to draw examples from as wide a representation of the participants as possible, and this goal was achieved. Of the total sample of Participants (excluding experts, who are treated separately in chapter 9), more than three-quarters are quoted at least once in the chapters covering the main part of the interview (chapters 2 through 6). More-

The skills of argument 42 over, the quoted excerpts are always verbatim and continuous (i.e., without deleted segments internal to the passage). Through these excerpts, then, we can truly hear the participants in this study speak for themselves. Quantitative results

The percentages of subjects whose causal theories are classified into the three major categories that have been described are listed in table ' 2.3. As reflected in the table, multiple-alternative theories are least common overall. Subjects are most likely to display a multiplealternative causal theory type for the school topic, while theories are ' most likely to remain in the single-cause category for the crime topic. , These topic differences reflect a pattern that will appear for most of the ! analyses in the book - the most complex form of reasoning is shown , most often (and/or the least complex shown least often) for the topic regarding which subjects are most likely to have personal knowledge (school failure), and the reverse pattern for the topic regarding which subjects are least likely to have personal knowledge (return to crime). These topic differences will be discussed further in chapter 7. Also reflecting a pattern that will emerge quite consistently throughout the book, no differences appear as a function of age group or sex, but subjects in the college education group are more likely to show more complex forms of thinking than those in the noncollege group. ' The percentages of subjects showing either of the multiple-cause theory types for the crime topic are 34% among the noncollege group and'' 67% among the college group. For the school topic, these percentage: are 65% among the noncollege group and 80% among the college group. For the unemployment topic, percentages are 76% among thei noncollege group and 82% among the college group. These differences reach statistical significance for the school and crime (but not the.. unemployment) topics. (See appendix 3 for a summary of statistical analyses.) As indicated earlier, subjects' proposed remedies tend to be consistent with their causal theories. The percentages of subjects' proposed remedies that do show such consistency are 79% for the crime topic, 82% for the unemployment topic, and 81% for the school topic. N significant differences appear in this respect as a function of age group, sex, or education group. In the remaining cases, (a) no remedy is. immediately apparent (e.g., in the case of causal line 5 for crime or:

Causal theories

43

Table 2.3. Percentages of subjects showing each causal theory type

Type

Crime

Unemployment

School

Single cause Multiple parallel Multiple alternative

49 38

21 54 26

28 46

12

27

causal line 2 for school failure), (b) the subject claims that no remedy is possible, or (c) no connection between the subject's causal theory and proposed remedy can be discerned. CONCLUSIONS

In this chapter, we have examined both the content and the structure of people's causal theories and identified three major forms that these theories take. These forms suggest important differences in the way people conceive of causal phenomena, a claim that we substantiate further in chapter 3. Subjects who hold multiple-cause theories do not necessarily incorporate a broader range of content in their theories (as reflected in number of causal lines), compared with subjects who hold single-cause theories, but their theories reflect a greater awareness of the causal complexity of the phenomena they are asked to reason about. In the next chapter, we investigate whether these differences have implications for how people justify their causal theories. Chapter 3 will bring us closer to the heart of our concern in this book, by examining the kinds of evidence people offer in support of • their theories. The present chapter lays important groundwork for this task by establishing that people do hold coherent causal theories regarding the phenomena they are asked about, theories characterizable in terms of a finite set of causal themes. Most significant, people claim to hold these theories with considerable conviction, a fact that makes it appropriate to proceed to the next phase of the study, in which they are asked to justify how they know that these theories are correct.

3 Evidence to support theories

To justify adequately an assertion that one has made is in some sense the heart of argumentive reasoning. An inability to answer the "How do you know?" question suggests that the assertion should not have been made. In subsequent chapters, a case will be made that there are other aspects of argumentive reasoning that warrant close attention as well, but first it is essential to take a very close look at the arguments people offer to support the theories they espouse. After subjects present their causal theories, the second segment of the interview for each of the three topics begins with the question, "How do you know that this is the cause __ ?" A number of probes follow, encouraging the subject to expand and become more specific, for example, "If you were trying to convince someone else that your view [that this is the cause] is right, what evidence [verbal emphasis] would you give to try to show this?" (See appendix I for full sequence of probes.) Probing ends with the question, "Is there anything someone could say or do to prove [verbal emphasis] that this is what causes __ ?" (If the answer is yes, the subject is asked to describe what it would be; if the answer is no, the subject is asked to explain why not.) Subjects' responses to the request for evidence cover a very wide range. Not all or even a majority of responses, it turns out, fall into the category that we shall define as genuine evidence. The most prevalent response is of a type we term pseudoevidence; a third, smaller category of responses we term nonevidence. In this chapter, we first describe and give examples of these three forms of evidence. Next, we examine frequencies of occurrence of each of the types of evidence, overall and in relation to the major subject variables of age, education level, and sex. We then evaluate consistency across topics in the forms of evidence subjects generate and, finally, examine relations between forms of evidence and the causal theory forms examined in chapter 2. We

Evidence to support theories

45

begin with genuine evidence, so as to provide a standard against which Jess successful responses can be compared. GENUINE EVIDENCE School failure, repeated crime, and unemployment are problems of such complexity that even experts are unable to offer definitive evidence regarding their causes. What kinds of evidence could we expect Jay people with no special knowledge or interest in the topics to generate? The evidence we categorize here as genuine is by no means conclusive, nor compelling, nor even necessarily high-quality evidence. The criteria adopted for genuine evidence, rather, are simply that it (a) be distinguishable from description of the causal sequence itself and (b) bear on its correctness. The importance of the first criterion will become more apparent when we examine responses in the pseudoevidence category, in which this differentiation is absent. The genuine evidence that subjects generate takes a wide range of forms. We begin with what is by far the most frequent form, covariation evidence, and then proceed to a description of less common forms. Genuine evidence is typically produced only in response to one or two of the interviewer's probes, usually those toward the end of the probe sequence, as the requests for evidence become more specific. In most of the following examples, for the sake of brevity, only that segment of a subject's response in which genuine evidence is generated is excerpted. As in chapter 2, examples are limited to the school and crime topics, and subjects are identified in parentheses by their age group (T, 20, 40, 60), education group (C, N), sex (m, f), and topic (s, c).

Covariationevidence The most common forms of genuine evidence show some reliance on the presence of covariation between alleged causal antecedent and outcome as support for the theory that the antecedent causes the outcome. Such evidence, in addition to being distinguishable from description of the causal sequence itself, as previously noted, is also characterized by some differentiation between antecedent and outcome, in contrast to the embedding of antecedent and outcome in a narrative scenario that we shall see is characteristic of pseudoevidence.

l

46

The skills of argument

Differentiation of antecedent and outcome makes it possible for variation in the antecedent to be related to variation in the outcome. Correspondence

We begin with the weakest form of covariation evidence - evidence that does no more than note a correspondence, or co-occurrence, of antecedent and outcome. The subject who provides the first example offers such evidence only in response to the final probe. He initially identifies emotional distress from problems at home (14) as the cause of school failure: (20Cms) (ls there anything someone could say or do to prove that this is the cause?) Just looking at their performanceand just looking at their problems. (How do you mean exactly?) I mean by ... well, if someone makes a study of cases of students where failures, dropouts ... students who drop out of school ... and sees where they have family problems, perhaps that would be solid evidence to prove what I believe. The next example is similar. The cause the subject identifies initially is lack of parental support (16): (40Cms) (ls there anything someone could say or do to prove that this is the cause?) Well, I think they can look at kids that are failing in school and see if the parent or parents are doing their job. (What do you think they would find out?) Well, I think they would find that the parents weren't there. They weren't behind the kids. The evidence offered in the two preceding examples is clearly minimal. In both cases, however, the antecedent is distinct from the outcome, and there is an attempt to establish that there is a correspondence between the two, even though the idea of this correspondence is not developed in explicit, quantitative terms. The following is an example for the crime topic. The cause identified is unemployment (]): (20Cmc) (How do you know that this is the cause?) Probably from the reading I've done, the associations I've made in my mind, in reading the Daily News accounts of people who are involved in crime, and it seems that they tend to be unemployed.They tend to be lower-incomepeople. A final example of a response assigned to the correspondence category of covariation evidence comes close to making the quantita-

Evide,,ce to support theories

47

tive comparisons that characterize the explicit covariat10n category described next. The cause is parental pressure to achieve (17): (TCfs) (How do you know that this is the cause?) Lots of times parents who force their kids to do well, you know, force them to study, Jots of times those kids will be the ones horsing around because they're angry, and it makes them want to go against their parents and not do it [do well in school]. Covariation

In this category, the idea of covariation becomes explicit. The two features that differentiate responses in this category from those in the correspondence category are some sense of (a) comparison and (b) quantification. Instances that represent one level of the antecedent are compared with those that represent another; the comparison is with respect to incidence of the outcome. In the first example, the subject begins with a response like those of the correspondence type but then, when asked about proof, adds the features of comparison and quantification. The cause the subject identifies is unemployment (I): (40Nmc) (ls there anything further you could say to help show that this is the cause?) I think just the fact that people that have criminal records as a whole - there are exceptionsto the rule - but as a whole don't seem to land any type of a decent job, any type of job whatsoever that has any kind of security.(ls there anything someone could say or do to prove that this is the cause?) I think if they just check past records, they could prove that, as a whole, anyone that has a criminal record ... should be able to prove that, as a whole, they don't make out as well as people without any kind of a record. The quantification in the preceding example remains implicit, deriving from the comparison of the two groups. In the following example, in contrast, the quantification is explicit. The cause is again unemployment (I): (TCmc) (If you were trying to convince someone else that your view is right, what evidence would you give to try to show this?) You could probably take a survey and find out the percentage of people who get jobs who have been convicts. I'm sure it's very low. The following is an example for the school topic. The quantification is only implicit, but the idea of comparison is clear. The cause is lack of parental support (16):

48

The skills of argument

Evidence to suppor, theories

(60Cfs) (ls there anything further you could say to help show that this is the cause?) Let's say the reading scores that are being published right now, and it somehow has a reflection on the [geographical] area they are in. The good ones are being in areas where parents take an interest, and the bad ones appear where there are many single-parent families, where perhaps they don't even have a home.

(ls there anything further you could say to help show that what you've said is correct?) Well, it's possible to read about some of the schools that are ... they are having a little bit more success with their school population, and with that kind of background there might be a decrease in crime and in kids planning for that kind of life.

Another subject offers a more sophisticated version of the same idea: (TCms) (ls there anything someone could say or do to prove that this is the cause?) Well, you mean in terms of an experiment or something? Well, I think that ... Let's see. I guess the best way would be simply to take a whole bunch of kids who're getting, well, a wide range of scores. I guess you'd want to include every type of kid. And do surveys of what their family life is like. And, I don't know, find out what similarities or differences there are between the kids who're getting As and the kids who're getting Bs and the kids who're getting Cs, or whatever.

i:

49

Here, one kind of early environment (in which there is "a little bit more success") is compared implicitly with another (baseline) set of circumstances, with the indication that they will lead to different incidences of the outcome. As the next two examples show, covariation evidence often originates in personal experience. The cause the first subject identifies is lack of home support (16): (40Nms) (If you were trying to convince someone else that your view

Making a meaningful comparison of the sort illustrated is not always straightforward, and a number of subjects make attempts that are unsuccessful. In the case of the following subject, he initially falters but eventually achieves success. The cause he identifies initially is maladjustment due to a poor childhood background (3): (60Cmc) (How do you know that this is the cause?) You read that they go back and they do a similar crime. You read it in the papers moreor less. I haven'tseen any official statisticson the situation,but you read that they very often go back. And you feel there is no reason why they, you know ... there's nothing to make them improve themselves really when they get out. (If you were trying to convince someone else that your view is right, what evidence would you give to try to show this?) I guess I would try to get the number of former inmates who go back to crime and ... uh ... compare them with . the population as a whole who weren't prisoners. I

This subject's initial response falls into the category we shall later • define as nonevidence - the "evidence" is the phenomenon itself, with the cause presumed. The interviewer's probe elicits an attempted comparison, but the subject again does not go beyond the phenomenon itself to link the comparison to the alleged cause, and the purpose of •' the comparison remains obscure. In response to the next probe, how- ., ever, this subject goes on to make a response that does fall into the covariation category, even though the comparison remains only implicit:

is right, what evidence would you give to try to show this?) Well, I could give some pretty good evidence. I could compare ... my kids have done fairly well ... to my brother's kids, who come from a broken home, they do, and they have been pushed around, and they have, none of them, done well in school. In fact, none of them have continued high school. So that would be one bit of evidence. (/s there anything further you could say to help show that what you've said is correct?) Well, maybe j~st from my own bringing up and going to school •.. that the kids who dropped out or the kids who dido't do well were usually the kids that came from the type of family who didn't seem to care much about those kids. Where the kids that came from better families - I mean, it's a hard word to put in there, but "better" families - seemed to do well.

The cause identified by the next subject is poor teaching (19): (40Nms) (How do you know that this is the cause?) Well, I have two kids myself. I have seen year after year what they have gone through, and one year, if the teacher really can teach, the kids made great strides forward. But, then, by the same token, there have been years that the student has gone down in his work and it has definitely shown. But yet when you talk with the teacher and everything you find that it is not just one student. It seems to be basically overall, all the students are having the same basic problem. (If you were trying to convince someone else that your view is right, what evidence would

you give to try to show this') Well, I think there again I could just pick out certain classes which students did good in and did not do good in and compare them, and speak with the other parents, and I would find basically the same basic thing, that the student did good

50

The skills of argument

Evidence to support theories

or did not do good in the particular class for one reason or another. (Is there anything further you could say to help show that what you've said is correct?) Basically that would be my best, would be just comparing notes with other parents and finding out how their kids did as opposed to mine and I think we would find out that overall, it would definitely be a make-or-break situation depending on the teacher. Both of the preceding subjects begin with the very restricted instances of their own families and then generalize, treating their own personal cases as part of a pattern of covariation over a broader set of instances. , We shall have more to say subsequently about the role of personal experience in subjects' thinking.

Both the preceding and following examples illustrate that, like covariation evidence, the correlated change form of evidence often originates in personal experience. In the preceding case, in fact, the evidence does not extend beyond the single instance involving the subject and his son that is described. The evidence offered by the following subject is based on a larger number of instances. The cause is problems at home (14): (60Nfs) (How do you know that this is the cause') My husband was a teacher, and he had a class of emotionally disturbed children. It was something new they were trying at the time, and all the problem children from all the different grades came into his class, and he had to cope with each one at each level. Nine out of ten children have problems at home. There were children who were shifted from one foster home to another. There were parents where there was illness

Correlated change Though it is usually present in the case of a causal relationship, covariation of course does not establish causality, because there may exist additional factors that also covary with antecedent and outcome and therefore have the status of potential causes. For this reason, a stronger form of evidence for a causal relationship is one in which , change in the antecedent co-occurs with change in the outcome. Co-: occurrence of change increases the likelihood that change in one factor: is responsible for the change in the other. For example, the following:'! subject identifies lack of motivation (7) as a cause and cites as evi-1 dence the following single-case instance: (60Cms) (I/you were trying to convince someone else that your view is right, what evidence would you give to try to show this?) Well, I' had a son who was very scienc~ involved, you know, involvedscien• tifically. He loved science, you know, and he didn't think ... he didn't think like things like ... what was that, social studies, or,, whatever it was, that he did poorly in. He just didn't like it, and I had! to have a long talk with him. After, I don't know ifhe failed or got bad mark or got a low mark or whatever it was, I had a very long t: with him and that, and by constantly getting after him. I had him ct, his homework in that subject in front of me. I had to do it with him. had to show ... l was trying to show him what's good about it,: cause I knew it was motivation. He just didn't care for it. And b:, trying to make it more interesting to him I think he must have pick, that up, because he started doing better. So to me that's proof enoug in that particular thing. Now if there are other cases, I simply don' remember, but I don't ...

51

it seems to me it's very obvious.

and the mother was paying too much attention to the father and not enough to the child. Drinking problems. There were problems. These were the problems. These were the problem children. (Is there anything further you could say to help show that what you've said is correct?) When their problems were resolved, many of the children

left that class environment and went back to their original classes and did very well. So, that in itself shows that it was the problems that were at home.

Though this subject needs prompting from the interviewer to complete her argument, the form the argument takes is clear. The next subject generates genuine evidence only as an afterthought. The cause identified initially is emotional stress from problems at home and school (14), to which she then adds drug use (13): (TCfs) (How do you know that this is the cause?) Well, I know this person ... she's just, like, my example ... I know this person from camp, and she has a lot of ... She goes to another school, and she has problems at home and problems with other people, and she used to have a drug problem. And she really does badly in school. She's just down on everything. (I/you were trying to convince someone else that your view is right, what evidence would you give to try to show that this is the cause?) I guess evidence from people that it's happened to, and I guess just try to give them an idea of how some of them feel. (Is there anything further you could say to help show that what you've said is correct?) I guess ... I could just ... I don't know. (ls there anything someone could say or do to prove that this is

the cause?) I think I could prove it because ... (How would you prove it?) Well, I don't know how to prove it ... Wait! I guess

52

Evidence to support theories

The skills of argument

people who have drugs and might not be doing so great in school, and so if they knew they took drugs they could see ... they could see what happened when they weren't on drugs and ... you know, ' prove it or whatever. The following subject progresses from covariation evidence to cor- , related change evidence. The cause is lack of motivation (7): (40Cms) (If you were trying to convince someone else that your view is right, what evidence would you give to try to show that this is the cause?) Examples of people who perhaps have not learned or have •

!

not passed courses or have not done well even in work situations ,.

because of the fact that they were perhaps not motivated. (Can you :,1 be very specific, and tell me some particular facts you could mention··'

to try to convince the person?) Perhaps a look for some statistical ii

data that would lead to that conclusion, although I don't have any.,! (What would it show') Well, if you could get a statistical study that I; showed that in a universe of failures in the school situation and eighty percent of them had not been motivated. Or perhaps the teacher's comments were that the child was not motivated. Something like that would perhaps convince somebody of the thesis. (ls there anything further you could say to help show that what you've said is correct?) Some examples from work situations where an-;

employee has not performed up to expectations and when motivated! changes his or her work habits to the point where they perform well.' In the preceding examples, subjects base their inferences on socalled natural experiments. In other words, the inference is based on' the examination of changes that occur naturally, that is, were not', induced specifically for the purpose of making such inferences. Be.l,j cause natural experiments do not allow all other potential causal factors to be ruled out, scientists characteristically wish to impose some, degree of experimental control on the evidence they examine. ThiS:j control is a matter of degree, rather than absolute, and takes severa1j forms: (a) assignment of treatments (interventions designed to induc, change) on a randomly assigned, rather than de facto, basis, (b) control of other sources of variation in the situation, and (c) comparison O' alternative treatments (including no treatment). Though subjects in tb present study rarely use the formal terminology of scientific method, some subjects do show some awareness of the desirability of thes, features. While remaining unsophisticated, these responses most ofte~ focus on the random-assignment, treatment-by-design concept, which accomplishes improved control of external sources of variation. These

53

responses, note also, are always presented in a hypothetical mode, in contrast to the "natural experiment" responses, which almost always refer to existing instances. Following are several examples. In the first, the cause is a combination of emotional problems (3) and return to the same environment (4): (40Cfc) (ls there anything someone could say or do to prove that this is the cause?) Try to treat them differently.(What would this prove?) It's hard, hard to prove. In the long run ... This takes time. In the long run, then, you can prove it, but right away ... (How would you prove it in the long run?) Make friends with a prisoner, and give them that sense that you trust them ... a feeling of trust. Then they would react differently. The next subject introduces the term "experiment," though his response is likewise unsophisticated. The cause is return to the same environment (4): (TCmc) (/s there anything further you could say to help show that this is the cause?) Maybe just, like, take a small ... you can do a very long-range experiment. I don't know how feasible that would be. It would be sort of a mean experiment, but get somebody and put them in a bad environment and see how long it would take them to get to stealing and see if they would repeat [crime]. That would take a while, and it would be kind of a really mean thing to do. The next subject, in contrast, appears to attempt the same form of argument but does not develop it successfully (and the response is therefore not included in the genuine evidence category). The cause identified is difficulty in readjusting to life outside prison (2): (TCfc) (ls there anything someone could say or do to prove that this is the cause?) Well, they could try to help some prisoners, maybe, like, readjust, but, you know, or just like help them. And they could maybe, if after they're helped, after they're finished their counseling or whatever, if they go back to stealingjust because they feel there's no hope or because they think that counseling is a waste of time, or something like that. If the experiment is to support the subject's theory that readjustment difficulty causes return to crime, the intervention ("counseling"), of course, should have an impact on the outcome, not leave it unchanged. !f the outcome remains unchanged, there is no basis for believing that •ts cause has been identified.

54

The skills of argument

The next subject introduces an explicit comparison between alternative treatments, but the implication is one of a natural experiment, with some control over extraneous factors achieved not by randomized assignment but by sampling a broad range of cases. The cause is lack of income (I): (20Cmc) (ls there anything someone could say or do to prove that this is the cause?) I suppose you could do a study of people's economic situations before and after, who were ... (What would that show?) Well, you'd be able to find out if people whose economic situations stayed the same or got worse were more likely to return to their previous life style [return to crime). The final example is a sophisticated one including the concepts of both randomized assignment to treatments and systematic comparison of alternative treatment conditions. The cause is return to the same '' environment (4): (40Cmc) (/s there anything someone could say or do to prove that , this is the cause?) Well, probably the only way to do it would be to try the classic situation of creating or describing certain populations of people who seem to be expressing the behavior that you are thinking about modifying, and then create three or four patterns of trying to modify that behavior, and then tracking what the experience would be. In other words, create, perhaps, take one group of this, · total group, and you give them training in job skills that are of real economic importance, not some of the job skills that are taught in prisons today, which have little or no real value to the people coming'i out. I mean it makes no sense to train a man who has a very high IQ,· to train him to be a welder. I mean, he's not going to be happy as a· welder, so why waste your time? So what I'm saying is you take one group and you give them job skills appropriate to their intellectual; capabilities, so that they can ... so that they are not easily frus . trated in whatever they do, and see if a percentage of them, a high,,

bJl

.Q

5 ....]

_____

i'= ia5

·-

0

"

~ ·.;

]

I

O

·= .S,."S< ~ ;.:= 0

..... ~_

Q.'3

Q.,t'IJ

8 o"'i:;

r':'! >,, ·>,, ~ fi -8---0

-~

197

>,, ~

"a~a ·,E C ·:

C:

& &id::&i &i

Subjects' responses regarding the certainty with which they hold their theories are grouped into the four categories shown in table 7.4. As seen from the quantitative summary presented in table 7.5, for each of the topics a majority of subjects report being sure or very sure that their theories are correct. Subjects report being least sure of their theories regarding the crime topic (with just over half reporting that they are sure or very sure), but for the other two topics only a quarter to a third of subjects do not claim to be sure or very sure that their theories are correct. These certainty ratings show no differences as a function of age group or sex. Furthermore, in contrast to previous findings, there are no differences as a function of education: Subjects with lesser education are as sure that their theories are correct as more educated subjects. Nor do certainty ratings differ as a function of the actual quality of evidence the subject has generated: Subjects generating nonevidence or pseudoevidence are as certain as those generating genuine evidence. The only dimension to which certainty ratings do relate is epistemological category. For each of the three topics, a statistically significant association appears between the three epistemological categories shown in table 7. I and the four levels of certainty indicated in tables 7.4 and 7 .5 (see appendix 3). Absolutists are most likely to be sure or very sure that their theories are correct - 63%, 76%, and 80% for the crime, unemployment, and school topics, respectively. In contrast, evaluative epistemologists are less likely to report being sure or very sure - 29% and 27% do so for the crime and unemployment topics, although 70% are sure or very sure for the school topic. (Multiplists are intermediate.) Parallel differences occur for the very sure category alone. Although a majority of subjects are certain that their theories are correct, it does not follow that they see themselves as highly knowledgeable about the topics. Responses to the knowledge question ("How much would you say that you know about this topic, compared to the average person?") are classified into three categories - more, same, and less. Subjects claim the most knowledge for the topic we ~1ticipated would be most familiar, the school topic. For the school topic, 49% of subjects rate their knowledge as above average, compared with 24% for crime and 29% for unemployment. Only a small minority of subjects rate their knowledge as below average for any of

~,,

: 1111

!

'I

198

The skills of argument

Epistemological theories

a comparable absolute number of subjects (between six and nine, depending on topic) who report above-average knowledge nevertheless report only low certainty.

Table 7 .4. Responses classified in categories of certainty Low

Medium

High

Very high

Not really Not that sure Not very Not at all Just my opinion

Half Maybe Medium A little Sort of Seventy-five percent

Fairly Almost Pretty Quite Ninety percent

Very Positive Clearly One hundred percent

Table 7.5. Percentages of subjects reporting each category of certainty Category of certainty

Crime

Unemployment

Low Medium High Very high

38 12 34

19 12 51 18

17

199

20 6 53 21

the topics - 16% for crime, 11% for unemployment, school. Knowledge ratings show no differences as a function of ago: group, sex, or education level. Nor do they relate to the quality evidence the subject has generated for the theory that is espoused. contrast to certainty ratings, neither do knowledge ratings relate epistemological categories. Nevertheless, knowledge and certainty ratings do show an ove relationship to one another, with subjects who claim greater knowled: also likely to report greater certainty. What is interesting, however, the deviations from this relationship, particularly among the minori of subjects who report below-average knowledge. Despite their lo~: perceived knowledge, these subjects very often report high or vert., high certainty. Percentages of low-knowledge subjects doing so 24% (six subjects), 44% (eight subjects), and 64% (seven subjects) ft the crime, unemployment, and school topics, respectively. Conversely,

Other epistemologicalassessment Strength of arguments (proof status) How strong do subjects perceive their own arguments to be? A measure of this perceived strength is whether a subject sees an argument as capable of proving (as opposed to merely supporting) the point in question. As the responses to questions about proof quoted in this chapter show, what is most informative about a subject's thinking are the reasons offered as to why an argument would or would not constitute proof. Nonetheless, we can examine the simple dichotomous yes (would prove) versus no (would not prove) response itself, in relation to the actual quality of the argument to which it refers. As already reported in chapter 3, in the case of evidence to support theories, subjects are not themselves good judges of the strength of their arguments: Subjects who offer only nonevidence or pseudoevidence are as likely to perceive this evidence as proof of the theory's correctness as are subjects who offer genuine evidence (roughly 80% in both cases). Results are similar for the other proof questions ("Could someone prove that you were wrong?" "Would you be able to prove this person wrong?"). In some cases, for some topics, those who offer valid counterarguments or rebuttals are slightly more likely to see them as constituting proof, compared with subjects whose counterarguments or rebuttals are not classified as successful, but these differences overall are small and statistically nonsignificant. Hence, overall, subjects appear not to be good judges of the quality or strength of their arguments. Origins of theories

In contrast, subjects on the whole are quite competent in reflecting on the origins of their theories. As shown in table 7.6, the large majority of ,subjects are able to offer some response to the questions about origins ("Can you remember when you began to hold this view?" "What led you to believe it?). Those who cannot answer indicate that

1111111 1

200

Epistemological theories

The skills of argument

the theory is either something that they "have always known" or, le, often, something that they have just constructed. The ability to reflect on origins shows no differences as a function q age group, sex, or education level. Nor do the types of origins re, ported. Type, however, does vary by topic, in a way consistent wi1•. the anticipated familiarity differences across topics. For the scb •• topic, with which subjects were expected to have the most perso experience, almost all subjects who are able to indicate origins me1 tion either their own experience or personal observation of othe~J Personal history or observation continues to be a prevalent repo source of theories for the other two topics, consistent with the con of the theories themselves, as noted in earlier chapters. For th topics, however, secondary sources (books, media, information s plied by other people) become frequent. As seen in table 7 .6, for unemployment topic subjects are more likely to report a second, source as origin than they are for the school topic, and for the cri topic a secondary source becomes the most common origin. Although, as reported in earlier chapters, argumentive skill lev, tend to be lower for the unemployment than school topics, and lo for the crime topic, these differences are rarely large, and a sizal number of subjects show success for all of the skills examined even the least familiar, crime topic. This fact, together with the res regarding origins just reported, suggest that reasoning is facilital . when subjects have first-hand experience with a topic, but it is • essential; subjects often show competent reasoning without it. Attitudes toward topics

In addition to rating their knowledge about each topic, subjects asked to rate how important they believe the topic is to society aa: whole and to them personally. To assist in these ratings, the subj--, provided a list of common "social problems" (e.g., housing sho: ••• drug abuse, government corruption) that includes the three inte 1 topics. The subject is then asked to indicate whether the topic0 question would fall into the upper third, the middle third, or the 10·• third. As seen in table 7. 7, ratings of social and personal impo: are similar. No differences appear as a function of age, sex, or ed1 tion group. In general, ratings are high, with a majority of sub:·· rating all three topics in the upper group in terms of both social personal importance. At least by their own report, then, subjects on

201

1'lillli

'li 1

Table 7 .6. Percentages of subjects reporting various origins of theories

i:li

Origin

Crime

Unemployment

School

Unknown Personalhistory Personalobservation Secondarysource

21 13 15 51

14 25 19 42

11

11111 1

64

Illli

16 11'1,:1

IO

1

Table 7. 7. Percentages of subjects reporting each category of social and personal importance Crime

Unemployment

School

1'1i.i,1 1

1 :I

I

111,1,,1

,I, !

Social importance

Lower Middle

7 37 56

22 71

3 34 63

21 35 44

27 21 52

13 24 63

8

111

Personal importance

Lower

Middle Upper

11r111"

I.

11

i,11 ,JI

wholebelieve the topics they are asked to reason about in this study are important ones.

!!i1 1

CONCLUSIONS: ARGUMENT AND KNOWLEDGE

:1111

11,1

In this chapter, we see that there is more to engaging in argument successfully than mastery of the argument skills that have been the subject of the previous chapters. Individuals must also hold the implicit epistemological theory that treats argument as worthwhile, as a fundamental path to knowing. In other words, people must see the llOintof argument, if they are to engage in it. If knowledge is entirely Objtjctive,certain, and simply accumulates, as the absolutist believes, or if knowledge is entirely subjective and subject only to the tastes and IYishesof the knower, as the multiplist believes, argument is superfluous. There is no need or place for the comparative weighing and

,1 1

1,1,11111

1 1r ,11'1'f• I

,I

ii

11 l1i'1 ~ 11 111111,

1 11

l1l]':1 i11 11

202

The skills of argument

evaluation of alternative claims that lie at the heart of skilled argu Only if knowledge is seen as the product of a continuing proce: examination, comparison, evaluation, and judgment of ditf, sometimes competing, explanations and perspectives does argu become the foundation upon which knowing rests. Knowledge is complete or finished, but rather remains open to further argumen1 one of the subjects quoted in this chapter put it, "At the moment I my opinion is justified, but if an expert were to prove me wrong way that I would find acceptable, then maybe my opinion co· swayed." If people possess this attitude toward argument, they are more to have acquired the skills needed for engaging in it. Those sul who espouse an evaluative epistemology, we found, are more r than others to exhibit the argumentive skills of counterargument, generation of alternative theories. In fact, in earlier chapters we glimpses of the epistemological stances examined here and ob ways in which they can work against the cultivation of argu skills. We noted in chapter 5, for example, how the belief viewpoints are equally valid works against the structuring of views into argument - counterargument relationships. And in c 6, we observed subjects who declined to counterargue, justify' refusal by claiming, in one subject's words, that "he's entitled opinion. He wants to believe it, that's fine." Such subjects, who see knowledge as absolute and certain and fore not subject to argument, or who see knowledge as subj, beliefs freely chosen in accordance with tastes and desires and fore similarly not subject to argument, have little reason to val1 engage in argument. They tend to feel firm and satisfied reg, what they know, as reflected in the consistently high certainty L, these subjects give their theories on all topics, often even when·! rate their knowledge as low. Coupled with this personal certainty; least an expressed tolerance for others' rights to hold the theori, choose. Paradoxically, this tolerance is expressed even among al ists, who claim that certain knowledge is possible, who are certain they themselves know, who may be unable to generate al1 live theories or counterarguments, and yet who proclaim that m1 views may be correct because "there are many ways of looking problem." A belief that is firmly held as correct, together with at 11 superficial tolerance for the right of others to hold their own,

1

1

Epistemological theories

203

111\ i"111 1I

firm beliefs, is exactly the combination least likely to foster reasoned dialogue and exchange of views. "I know I am correct; let others believe what they like" is the very prevalent attitude that the present investigation highlights. In this light, again it is not surprising that the theories people espouse are to a large degree unexamined, as we concluded at the end of chapter 6. In the final chapter, we explore further some of the implications of these conclusions. In the next chapter, we in a sense assess the preceding conclusions regarding subjects' openness to argument and exchange of views by asking them to examine evidence relevant to the theories they endorse. How do they interpret new information that may conflict with their own beliefs?

11ilii111I

11

l1l:i"111

l.11 \i1 1111I

1II.J11ll1 l 1

1\ll'li11 I 11111111 11 111i ii.11

I

11111111 11 il11i1lli IIJ1li,i1I

11111111 1 1

l/ ;1:: .il11·11 II

1

1

//111i1'11 1111i"11 I 1 11'11 11111 1

1

1 11 1 1111

l!i

11111111 ·111 11\ • •

1

111:/111111

111/11··

II

1i

11 i1 1II\""" 111I 1 1 1l11i

i""'

1 11111:11, /11111·1111

ilJ1liJI 1 1!11111 1 111jl'[\I 1111111 1l11J

Evaluation of evidence

8

UNDETERMINED

Evaluation of evidence

In this chapter, we examine a reasoning skill that plays a clear important role in everyday life. People come in constant contact information that bears on the beliefs they hold. How do they in this information? Although this processing of new information is usually connected to argumentive reasoning, in this chapter we see the two in fact bear a close and important relation to one another.: Subjects are asked to interpret two kinds of evidence related to of the three topics they have previously been questioned about - self failure and prisoners' return to crime. The main interview for these topics occurs during the first interview session, and evidence eva· tion takes place during the second session. Hence, subjects' disc, • of their theories is separated in time from their evaluation of evid, related to the theories. The two kinds of evidence, which we refer to as underdete and overdetermined evidence, stand in sharp contrast to one an," Underdetermined evidence is not really evidence at all. It simply: scribes the phenomenon (school failure or return to crime) in the • text of a single, specific instance of its occurrence, with at most ma! cues with respect to possible causes. In the case of overdete evidence, in contrast, three broad sets of antecedents are identifiej possible causes of the phenomenon. Each set of factors co-occurs • outcome (school failure or return to crime), and each is advoc causal by a different authority figure. Nothing is stated that favor one set of possible causes over the others. If all three ope: ,,. causes, the outcome can be regarded as overdetermined. Wllf! none, one, two, or all three are causal, however, is in fact indete nate from the information given. Subjects' interpretations of these two contrasting kinds of evi, can be examined in a number of respects. We begin with unde: mined evidence. (•'

i

205

EVIDENCE

The underdetermined evidence passages presented to subjects for the two topics are shown in table 8.1. The subject is shown the written passage and invited to follow along as the interviewer reads it; the written form then remains in front of the subject for reference while questions are asked. Underdetermined evidence is presented for both topics (in counterbalanced order across subjects) prior to presentation of overdetermined evidence. The major question to be asked about subjects' interpretation of the underdetermined evidence is whether they interpret it as evidence at all. Do they interpret the passage as establishing the causes of David's school failure or Pete's return to crime, or do they interpret it as the Table 8.1. Underdetermined evidence Crime topic Pete Johnson is someone who has spent a good portion of his adult life in prison. He was first convicted of a crime at age 14, when he took part in the theft of a newspaper stand. He began serving his first prison sentence at age 18, after being convicted on several charges of auto theft and robbery. He remained in a medium-security state prison until the age of 20. After he was released on parole, he returned to live with his mother in the same neighborhood where he had grown up and began to look for a job. After 3 months out of jail, he took part in the robbery of a grocery store. He was caught and convicted and returned to prison. Since then, Pete has served three more prison sentences for differentcrimes, with only brief periods out of prison between sentences.

School topic David Bell is a child who has shown a continuing failure to learn in school. David is 10 years old and repeating second grade. He also repeated first grade because of his poor work. David lives with his parents and younger sister in a medium-sized city. Since age 5, he has attended the elementary school in the family's neighborhood. David had great difficulty in learning to read and now reads only when he is required to and has trouble recognizing all but very simple words. David does no better in math than he does in reading. He dislikes schoolwork of any kind. Often he spends his time in the classroom daydreaming or talking to other children. David finds the work his teacher gives the children to do uninteresting. He says that he would rather spend his time doing other things.

: ·1r11ii

206

The skills of argument

nonevidence (of cause) that it in fact is? Hence, it is the certainty:! subjects profess regarding any causal interpretations, rather than their • particular content, that is of primary interest. A subject who recognizes that the information given is not sufficient to permit an unequivocal )' causal judgment will acknowledge that a number of explanations are possible and will indicate low certainty regarding which is correct. In contrast, subjects who make causal inferences with high certainty,., ignoring the possibility of alternatives, allow their own causal theories" to infuse, and dominate, interpretation of this instance of the phenomenon. The interviewer first elicits an initial interpretation of the under-i determined evidence, asking, "What do you think is the cause of David's failure in school [Pete's return to crime]?" The interviewer then assesses the subject's certainty regarding any cause(s) the subject_, has suggested, asking, "How sure are you that this is the cause of David's failure in school [Pete's return to crime]?" The interviewer; then probes the issue of certainty, asking, "Are there any other !)OS"., sibilities?" If the subject gives an affirmative answer, the interviewer} elicits these alternative possibilities and asks if the subject can choose among them. Finally, the interviewer reassesses certainty by means of:' the same question used previously. ,. Subjects' responses reflect the entire continuum of certainty, witli/ some subjects accurately noting that the passage provides little basil,\ for judgment and other subjects professing a high degree of certain1 that a particular causal factor accounts for the instance depicted in passage. We begin with responses that reflect uncertainty. Uncertainty Some subjects are quite explicit in recognizing that the information • the passage provides little basis for an inference regarding cause: (20Cfc) (What do you think is the cause of Pete's return to crime?), can't say. (Why not?) Cause all it says is that he is returning and hei returning. It doesn't give too many clues. I mean, if I was j • drawing on Pete as anyone, I would say that perhaps he hasn't fo1 anything else to be successful or lucrative as a way of life. (So, you say what the cause is?) No, because it doesn't really give reasons. It just gives examples of his returns. The following response for the school topic is similar:

Evaluation of evidence

1l11, 1

207

(20Cms) (What do you think is the cause of David's failure in school?) No idea. (Why not?) There's nothing in here that suggests a

1,

cause. There's nothing in here that describes what his home life is like. There's nothing in here that describes what the school or teacher is like other than the fact that other kids seem to be getting along okay. There's nothing here that really indicates any of the things that I would associate with learning problems.

1

111

1111;111

1:1

Other subjects initially suggest a cause but then make clear their awareness that insufficient information is available to permit certainty: (20Cmc) (What do you think is the cause of Pete's return to crime?) That he doesn't have any ... really from here I see that he doesn't have any other skills. (Why do you think this is the cause?) Because he has nothing else to do and it's an easy thing for him to do - just go to crime. (How sure are you that this is the cause of Pete's return to crime?) l don't really know. (Why not?) Because I don't ... this just gives me what he did. It doesn't give his motives. I'd have to, you know, ask him or ... [sentence not completed].

11'1:~

,111,

111;!1

111:i,,,

11

The following subject likewise suggests a cause but is aware that the passage does not provide the evidence needed to verify it, and he goes on to suggest the kinds of information that would be necessary to obtain in order to make a definitive inference: (60Cmc) (What do you think is the cause of Pete's return to crime?) This is something similar to what we have already had the first session. Well, first of all, it is difficult for a person who has a poor upbringing, and he certainly has if he committed a crime at age fourteen. It doesn't seem like a major crime, but if he did have a poor upbringing, one which did not emphasize caring or emphasize respect for authority, if he did poorly in school, things of that type, so that his formation itself ... the individual himself might be an important factor as to why he wasn't able to lead a life similar to many other people. (Why do you think this is the cause?) Well, I don't know. There isn't enough in here to say that. I am just making an assumption if that were the case it would be difficult for him to rehabilitate his own self to be a different kind of person. (So, how sure are you that this is the cause of Pete's return to crime?) Well, I don't know, becausethereis not enough informationhere to indicate what kind of childhood he had. I am making an assumption here and the assumptionmight be, and I am not sure that it is correct, but if we can compare his kind of upbringing with another kind of upbringing and see statistically at any rate whether such upbringing is a factor in

·1 11l1

1 11

Ir

1: 1,l:1

11

1 1

1111· ' ,,_, 11

1

I.I

1·

'I_I1'_11

)ii

l:

1111

!

~I

1

II

11:: ,

208

The skills of argument

Evaluation of evidence

crime. That might be an indication. But not everybody who had a . 1 poor upbringing, even the kind I have assumed, on an individual basis may go to crime.

know. (Why do you think so?) Well, it says he has trouble reading and recognizing words. I'd say almost definitely that is a problem, but then it becomes a question of is that the only problem. Because I don't think that would make him bored in class. So, the next thing would be his teacher, which we are assuming none of the other kids are getting bored in class, and therefore it's not the teacher's fault. The next relevant factor would be whether or not ... what subjects he's good in. ls he good at anything? If he's not good at anything, or he's gotten worse at certainthings, then you could assumeit's probably something at home going on.

In addition to suggesting an examination of covariation as a basis for ,, evaluating a potential causal factor ("compare his kind of upbringing , with another kind of upbringing"), this subject points to noncovariation 'i evidence as a basis for lack of certainty ("not everybody who had a l poor upbringing . . . may go to crime"). Whether or not subjects go on to point out why the cause of the phenomenon is uncertain or to suggest what further information would',, be necessary to achieve certainty, what is critical is that they recognize 1. the uncertainty that surrounds the minimal "evidence" that has beenl presented to them. What about subjects who do not recognize thial 1 uncertainty? The first possibility to consider is that the subject's cer. tainty is at least to some degree justified because the subject has.: identified something in the information given that provides a basis for) a causal inference. Though the passages were written with the intent of\' minimizing any clues as to the cause of the phenomenon, it is neces-.i' sary to examine whether this intent was achieved or whether subjects•,; nonetheless did identify something in the passage that served as a basis] for a causal inference and hence for the subject's certainty. "· To examine this question, responses to the underdetermined evl-,

'gss11 .!l (+,..i 8

8 "

'+-< ~

.E -~ ct::ct::·;;; (I')

§ ·- ·- s::

u" u r55c558

216

The skills of argument

Evaluation of evidence

to the content of subjects' initial causal theories. To do so, it is poss·· to make use of the same content coding scheme introduced in chap, for the purpose of analyzing the content of subjects' initial ca, theories. All causal statements appearing in subjects' evaluations oft underdetermined (as well as overdetermined) evidence were coded terms of the set of causal lines identified for each topic and presen1 in chapter 2. The most interesting question to be asked about these results is of consistency. Do subjects who make causal inferences in interpre"' the underdetermined evidence invoke the same causes in the s instance presented here as they do in describing their causal theori, the initial part of the interview? Unsurprisingly, the answer for majority of subjects is yes. A significant minority, however, s: inconsistency. The causal lines identified in the two contexts (initial causal th, ing and evidence evaluation) need not be identical in order to be jud 1 consistent. The most common pattern is one of overlap, but not ide: ty. In other words, at least some of the causal lines identified in the contexts are common to both, but the subject may include causal in the first context that do not occur again in the second, or in· second context the subject may introduce new causal lines that do • appear in the first. Even in terms of this fairly liberal definition consistency, there is a significant minority of subjects who do not sl it. In other words, these subjects interpret the underdetermined dence in terms of one or more causal lines that are completely cliff, from those that appear in their causal theories elicited in the main of the interview. The percentages of subjects judged inconsistent in terms of definition are 36% for the school topic and 25% for the crime Subjects inconsistent for one topic are not more likely to show i sistency for the other; only 13% of subjects show inconsistency: both topics. No sex differences appear with respect to consistency, only a slight age effect appears for the school topic, with subjects ilf oldest age group somewhat more likely to be consistent; no age e: appears for the crime topic. Effects of education group are in direction of greater consistency on the part of the college group: centages of inconsistent subjects in the noncollege group are 3 I% ' the crime topic and 40% for the school topic, compared with I 9%°' the crime topic and 30% for the school topic among college subj. These differences, however, do not reach statistical significance.··

r

217

Relations between certainty and consistency

A possible interpretation of the inconsistency shown by this minority of subjects is that they lack any strong conviction or certainty regarding their causal attributions. Hence, they are likely to invoke one cause or causes in one context and others in another context. If this interpretation is correct, subjects who are inconsistent should also express low certainty. The data already presented suggest that this may not be the case. The noncollege group, we found, are more likely to be inconsistent, but they also are more, not less, likely to be certain, compared with the college group. We can, however, examine this relationship directly by assessing the certainty levels of the groups of inconsistent versus consistent subjects. For the school topic, among inconsistent subjects, 44% conclude their evaluation of the underdetermined evidence with an assertion of high certainty (third and fourth categories in table 8.2). Among consistent subjects, the comparable percentage is 22%. Hence, subjects whose causal attributions are inconsistent with those displayed earlier in the interview are more, not less, certain that these attributions are correct. For the crime topic, however, this difference does not appear. Here the percentages are 50% for the inconsistent subjects, compared with 49% for the consistent subjects. In this case, the fact that subjects overall are more certain (when consistent as well as inconsistent) may overshadow any potential association with consistency. Relations between evidence interpretation and argument skills

A relationship appears between subjects' levels of certainty in interpreting evidence and their argumentive skills. These results will be reported in the next section for the overdetermined and underdetermined evidence together. OVERDETERMINED

EVIDENCE

We tum now to overdetermined evidence, before drawing more general conclusions. The overdetermined evidence passages presented to subjects for the two topics are shown in table 8.3. As in the presentation of underdetermined evidence, the subject is shown the written

218

The skills of argument

Table 8.3. Overdetermined evidence Crime topic A study was done of 25 prisoners who were about to be released from prison. All had served more than one prison sentence; some were in prisoll~J for the third or fourth time. All had been in prison for the past 3 years or longer, mostly for crimes of armed robbery.

A social worker investigated the prisoners' life histories. All had unhappy early lives with many personal and family problems. None had good school records. They tended to be uninterested in school, to do poorly, and to drop out without finishing. Almost all became involved in crime at an early age.

A government official did a study of their prison life. The prison was badly overcrowded; each prisoner shared a cell with two or three others. Because of crowded conditions, prisoners were able to have periods of exercise and outdoor recreation only infrequently. No prisoner received jo~i

training. .'~~ Another social worker followed their lives outside prison during the 6 months following their release. The majority had been unable to find jobs since they had been released. Some applied to training programs, but th, were long waiting lists with only a few openings. Many hadn't found suitable housing. School topic A study was done of 25 third-graders in a large city school system. All,~ of these children were reading at least two years below grade level. Many•• had difficulties in math and other subjects as well. All the children had always performed below their grade level, and many had repeated one or more grades. The school psychologist observed each of the children individually. She,, found that these children had many different individual problems. Some had vision or hearing problems. Others had difficulty paying attention. Some had trouble getting along with other people and were always getllll' into fights. The principal of the school did a study of the classrooms these childre4 came from. In both classrooms the teacher spent very little time with e, child individually. Often, what the teacher was teaching was too difficult for many of the children to understand. A social worker did a study of their home lives. She found that many came from single-parent homes where the mother worked and was not

the child during after-school hours. The social worker also found few books or other reading materials in most of the homes.

Evaluation of evidence

219

passage and invited to follow along as the interviewer reads it; the written form then remains in front of the subject for reference while questions are asked. For both passages shown in table 8.3, the order of the second, third, and fourth paragraphs is counterbalanced across subjects, with each paragraph occurring in either the second, third, or fourth positions for one-third of the subjects. This variation permits assessment of any effect of the order of mention of the different segments of evidence. After eliciting the subject's initial interpretation by means of the question "What does this study suggest about children failing in school [prisoners returning to crime]?" the interviewer assesses certainty regarding any possible causes the subject refers to by asking, "Does the study prove that this is the cause?" As in the case of the underdetermined evidence, the interviewer then raises the possibility of alternatives, asking, "Does the study suggest anything else about children failing in school [prisoners returning to crime]?" The proof question is then repeated. If all of the antecedent factors suggested in the passage are taken as causal, the outcome is "overdetermined" by these multiple causes. Because it is not established that any of these factors does play a causal role, however, the true causal relations are in fact indeterminate, and no causal inferences are warranted from the evidence presented. We begin with subjects who show some recognition of this indeterminacy and hence express uncertainty about the causal factors involved.

Uncertainty

A few subjects are very explicit in their recognition that the multiple factors that are identified preclude any causal inference, as the following response illustrates; (20Cms) (What does this study suggest about children failing in school?) I don't think this study suggests anything. Really, I think it's a classic case of studies being able to suggest anything they like. (How do you mean?) Well, three different people and three different studies, and they came up with three different sets of evidence, and ostensibly three different answers. Qther subjects are somewhat more willing to make inferences. The certainty they express, however, remains low. The following subject, for example, mentions lack of employment as a cause suggested by the passage;

111,11.:1111

!I!,

,1ililllll

220

221

The skills of argument

Evaluation of evidence

(40Cfc) (Does the study prove that this is the cause?) No, it doesn prove it because it was only done with twenty-fiveprisoners, and I' quite sure all over the United States there are more than twenty-fiv/'1

(60Cfc) (Does the study prove that this is the cause?) It indicates that this is so. I don't think ... proof in a scientific or controlled study sense is quite different. There would have to be a control group that got certain training and certain opportunitieswith probable previous life experiences, and the two groups would have to be traced over a

prisoners.

As the basis for their uncertainty, some subjects point specifically • the presence of multiple antecedents, recognizing that they may or ma• not be causaL The following subject, for example, initially mentio individual problems in vision, hearing, and attention as a cause: (20Cfs) (Does the study prove that this is the cause?) No, I guess j doesn't prove. I guess it just illustrateswhat is going on. It just • of like describes. It doesn't prove, it describes. (Why doesn't prove?) Because there are many factors that they haven't accoun1•

for. The subject recognizes, in other words, that the fact that the antecede is present does not establish that it plays a causal role. Another subj, puts it very simply, with respect to the factor of the teacher's role: (TCfs) (Does the study prove that this is the cause?) It's just a fl that the teacher didn't spend much time. It doesn't say that this~why. You know, it's not clear.

J

The following subject, after mentioning several possible cau~ refers explicitly to the distinction between covariation and causality: ::' I

(20Cmc) (Does the study prove that this is the cause?) It doesn'· prove it. It suggests it. It's just that it's describingitsituation, and it'1

describing another fact. And because A is true and B is true, • doesn't mean that A causes B. Several subjects refer to noncovariation as a justification for thi unwillingness to make a causal inference, as the following respo1 illustrates: (40Cms) (Does the study prove that this is the cause?) It indicates;! Jot of problems, but it doesn't prove any one. People can survive of these difficulties and still not fail, and so it doesn't to prove ... it provides indicatorsof why failure might occur, but proof that those are the reasons. Finally, the following subject, who initially notes Jack of job o tunities as a possible causal factor, articulates specific conditions would be necessary in order to allow a causal inference:

considerable period of time in order to come up with data that would

suggest proof, and even that would be challenged depending on the size of the data base.

Certainty Although the preceding examples reflect a thoughtful, critical evaluation of the overdetermined evidence, such responses are in the minority, as reported in detail shortly. More commonly, subjects are quite willing to make causal inferences based on the overdetermined evidence, inferences, moreover, that they claim can be made with high certainty - the evidence is in fact interpreted as "proof" of the correctness of the causal inference. What do subjects see as their basis for this certainty? In interpreting the underdetermined evidence, we saw, subjects who claim certainty commonly impose their own causal scripts on the meager information provided in the passage. In the present case of the overdetermined evidence, in contrast, the passage is full of material suggestive of a diversity of causes. In this case, then, if subjects are to impose their own causal scripts in interpreting the evidence, they must selectively focus on some of this material and ignore the remainder. In fact, this is precisely what happens, and the same imposition of preexisting causal scripts can be seen in subjects' interpretation of the overdetermined evidence. The following is a typical example: (40Cms) (What does this study suggest about children failing in school?) Well, it suggests to me that they don't have the right parental guidance. (Does the study prove that this is the cause?) Yes. (How does it prove it?) Because of basically the social worker's findings. They came from single-parenthomes. The mother was not around. They didn't have a great influence on the child and probably weren't guiding the child very well. Thus, the subject focuses on one of the sets of possible causes suggested in the passage, having to do with home life, and makes no mention of the others. Unsurprisingly, a check of this subject's causal theorizing in the main part of the interview confirms that his own

222

'I The skills of argument

theoretical beliefs regarding the cause of school failure center on 1 of parental guidance ( 16). In interpreting the overdetermined evidencei, not only does he ignore factors other than parental influence; he al, takes the presence of the factor on which his attention focuses sufficient to prove its causal role. Because the antecedent has documented as present, the evidence is interpreted as proving that factor plays a causal role in the outcome. The following example for the crime topic is similar: (TNmc) (What does this study suggest about prisoners returning crime?) It tells me that some of the prisoners are willing to go and work, but because the people of the area know that they are cons or such, they think that they are not good enough for the job, they try to put them down all the time. (Does the study prove that t, is the cause?) Yes, this study proves very much that this is so. (H, does it prove it?) Because the social worker went in and investig;

everything thoroughly for him- or herself and found out the causes why the person is acting in such a way.

This subject's respoif~e implies as a source of justification the authori1 of the professionals conducting the study. This appeal to authority ' more explicit in the following subject's response:

Evaluation of evidence

223

The question that arises next is the same one posed earlier in this chapter with respect to the underdetermined evidence, as well as with respect to the evidence subjects generate themselves, examined in chapter 3. Does the subject acknowledge the possibility of alternatives to the causal script on which his or her attention is focused? A failure to acknowledge alternatives is especially striking in the present case as alternatives are explicitly presented in the passage the subject is asked to evaluate. Nevertheless, it occurs commonly, as the following examples illustrate. One subject initially mentions inadequate parental interest as a cause: (40Nfs) (Does the study prove that this is the cause?) Sure the study proves it. (How does it prove it?) Because of the situation at home ... the home life. Everything comes from home first.

t,,

Note that the question is about what conclusions can be drawn from the study, whereas the subject's response concerns what she in general believes to be true - a significant confusion we return to shortly. The interviewer then goes on to ask about the possibility of alternatives: (Does the study suggest anything else about children failing in school?) No, not really. It's just, you know, it's just home life. It just

boils back down to the home life. Cause the problem lies there. (20Nms) (Does the study prove that this is the cause?) It proves (How does it prove it?) Because they studied. They know what are talking about. Or simply the conduct of the study itself may be sufficient. As ano1 subject puts it: (20Nmc) (Does the study prove that this is the cause?) Yes, I thi does. (How does it prove it?) Because it lists a whole bunch of fl As we observed in chapter 3 in examining the evidence subj, generate themselves, sometimes subjects appeal only to the outc, itself, with the cause they favor being simply assumed. A similar fi of response in interpretation of the overdetermined evidence is il!i !rated by the following response. The subject initially mentions factor of inability to find employment: (20Nfc) (Does the study prove that this is the cause?) l think ' (How does it prove it?) Well, that's cause they are going back prison. I mean, why else would they go back? If they did ii ag: they're going back.

'I

l1l

Ill

In contrast to the preceding subject's explicit rejection of alternatives, the following subject, when asked about alternatives, simply elaborates his initial causal attribution:

,11111

(TNms) (Whot does this study suggest about children failing in school?) It suggests that maybe it is because they come from singleparent homes. (Does the study prove that this is the cause?) In a way, yes. (How does it prove it?) Because it says that in the home there was mostly one parent, and the parent is not there during after-school hours. (Does the study suggest anything else about children failing in school?) That in the home, there's not ... there's only few books and not too many reading materials. (Does the study prove that this is the cause?) Yes, because it gives the facts about it, you know, the reasons why. liiril

When asked about alternatives, the following subject similarly simply puts a new twist on the same cause offered initially: , (60Nfc) (Whot does this study suggest about prisoners returning to crime?) They have nothing to return to. (Does the study prove thot this is the cause?) Yes. (How does it prove it?) There's no housing

,,,1111

,

;

11:llll

11

1 l

1l1

224

The skills of argument

Evaluation of evidence

for them. There are no jobs. All this is what I discussed with before. They even if they wanted to go into a training program, have a long waiting list and of course the impatience is there.

dealing with hard subjects and then not being able to spend any time with each child. (Does the study prove that this is the cause?) Yeah, that's part of it, yeah. (How does it prove it?) Because I still think that even though these children are having a hard time learning, it didn't say they couldn't learn.

The subject goes on to elaborate this causal script at some length. interviewer then asks about alternatives: (Does the study suggest anything else about prisoners returnini'. crime?) Well, they don't mention that they have nowhere to go. see, if they have a family to come back to ... if they had a mot"'• You see, some mothers will take back a child, no matter what. they don't have anyone to tum to, no one to care for them ... their immediate needs, for their mental stability. They have noth' • So what else are they supposed to do? (Does the study prove that is the cause?) Of course. (How does it prove it?) Why does it it? Because of the things I just said. Do you want me to repeat again?

Sometimes, in contrast, as in the following example, the • viewer's inquiry about alternative possibilities does lead the subject refer to an entirely new cause: (20Nfs) (What does this study suggest about children failinr ' school?) Well, this suggests that these children obviously in a 1, city may come from poor or very low income families. Maybe parent or parents, even though it says they mostly came from si _ parent homes. It suggests . , , I'm looking at this now from i social scales. When you think about social scales, that these may be very high educated parents either, since there is not much at home ... that these children ... their parents have no timo,. spend with them, and actually when you think about it, except for fact that there's not much home life together, children even in middle to upper middle classes, parents may not always spend with them. They may get home late after work. But it suggests the parent has no time very much at all to spend with the chi!, being a single parent. (Does the study prove that this is the ca, Yeah, I think that, you know, that this is a good study. (How prove it?) Because it seems they have done an in-depth ..• know, they haven't just looked at it from one angle. They've go1. three or four different areas of the child's life. (Does the study • gest anything else about children failing in school?) The itself. (How do you mean?) The class, the teachers ... The c obviously large and overcrowded maybe. It says here they

225

When prompted by the interviewer's question, the preceding subject acknowledges a second causal theme present in the passage. In contrast to the pattern frequently observed in subjects' interpretation of the underdetermined evidence, however, the acknowledgment of alternative possibilities does not reduce the subject's certainty regarding cause. Instead, she sees the evidence as proving both factors to be causes. In the following example, the subject similarly responds to the interviewer's question about alternatives by acknowledging a second causal theme suggested in the passage. This subject then assimilates this second cause to the initial causal script that she has elaborated at some length: (40Nfs) (What does this study suggest about children failing in school?) I think it starts from the home. Learning has to ... I once saw a show on TV that even showed while you're pregnant it's important to read and talk to the baby. I mean, I never did that. I saw it on TV. It was a whole family of geniuses. Did you see it? It was on Phil Donahue or something. They had a family that raised three or four children out in the West. All geniuses, and she said she always, while she was pregnant she used to read to them and colors and things like that. By the time they were one year old, they were talking ... or six months or nine months ... they were talking with full sentences. And by three years they were reading. But when you have a young child, you can't talk to them like a baby. You have to talk to them, so they learn. Watching "Sesame Street" even helps ... all these little things. Children who go to nursery school are more ahead. So you do have to give them ... Children who come from upper-class homes generally are better ... more well versed before they begin school than children from slum areas, as a rule, because they're not given the same attention. (Does the study prove that this is the cause?) I would think so. (How does it prove it?) I mean, they need the attention. Because it says here they did a study of their homes ... single-parent homes where the mother was not home with the child, few books in the house, no reading material. (Does the study suggest anything else about children failing in school?) Also they're not taken care of. Vision and hearing prob-

'11'!11 111(

1:-:

,!!:

~ !: ;I

,·11

11:,

I

1 : ii

I I

II.

226

The skills of argument

Evaluation of evidence

!ems. (Does the study prove that this is the cause?) That's right. So, mean, that all shows that their preschool care is not very good.

Modification

We conclude with an example in which the subject makes expli, reference to all three sets of causes but then arbitrarily claims one to of lesser causal status than another, in accordance with her own caus beliefs: (60Cfs) (What does this study suggest about children failing school?) Failure, as we have been saying, is caused by a variety things. It could be either with the child's physical makeup or with psychological makeup and based on what his home situation is marily. (Does the study prove that these are the causes of child, failing in school?) Yes. (How does it prove it?) Well, because study was done by a principal who looked in and got a basic feeli ·" of how effective the teacher ... certainly the teacher would bear least responsibility, although she would bear some, I suppose, providing too little time to each child individually.The social w, er's report is of utmost importance. A single-parenthome, the mot! er is not home with the child during the after-school hours, were few books or reading materials, so that the child has what have been saying - too little encouragementto be involved in schoor work. Epistemological

reasoning

In a final segment of their interpretation of the overdetermined e· dence, subjects are asked to reflect on the relation between the e· dence they have been asked to evaluate and their own beliefs about topic. The primary question that elicits this reflection is the follow· "Does anything from this study influence your own thinking al what causes children to fail in school [prisoners to return to crime]?" addition, although most subjects simply responded no, a second q1 lion on occasion also yields some epistemological reflection: "Do y,, have any doubts about what this study suggests?" In order to reflect on how the evidence influences their thi subjects must have a clear sense of what their thinking on the topic •· as well as an accurate representation of the evidence. Otherwise, it' impossible to contemplate the relationship between the two. Subj responses reflect a wide range of competence in this respect, catego1, able into the following types.

227

The following is an example of a subject who is very explicit in explaining how the overdetermined evidence has modified her thinking: (TCfs) (Does anything from this study influence your own thinking about what causes children to fail in school?) Yeah, it's another ... it's like opening another door to a different subject. (How do you mean?) Well, before, I wouldn't think about this. Like when you asked me the first time, I did not think about neglection [parental neglect] ... that it could have a lot to do with it. But now when I read this ... it could. More commonly, subjects are not as articulate in their epistemological reflection. In the following case, the subject is slightly less explicit than the preceding subject, but the meaning remains clear. (In this and the following examples, unless otherwise indicated, the question subjects are responding to is the one indicated in the preceding example, modified appropriately for the crime or school topic.) (TCfs) Yeah. (In what way?) Because it tells you about the schools and their problems that you might not have known before. (So how does that influence your thinking?) I don't know. It just makes me think that it's something outside that causes ... instead of the child himself or herself. In the following case, the subject is much less explicit in epistemological reflection. Yet the unemployment cause that she raises is a new one for her, not previously part of her causal theory, and the implication is that the passage has suggested it to her: (20Nfc) Yes. (In what way?) Probably the reason that they go back to crime is that they can't find jobs. A lot of them didn't have proper schooling. They didn't have any kind of job training. (So how does that influence your thinking?) it makes me think. I feel sorry for these kids that are dropping out now, because they are just heading for a lot of trouble. In the two following cases, the subjects state that the evidence has not influenced their thinking. The broader implication of their responses, however, is that the evidence has in fact suggested a new causal factor. Their responses are therefore included in this category:

i 1

ii/Ill

1 1 1111:lll: 1

228

229

The skills of argument

Evaluation of evidence

(20Cfs) It doesn't influence my own thinking, but it gives me more information about why kids fail. (How do you mean?) Like before I ' never thought of kids who fail. I never thought they were having vision or hearing problems.

probably drop out and not get a good job, and that could lead me into the same thing as these prisoners.

Or: (TCfs) I don't know. I never thought about this vision and hearing and that possibly people cannot do certain things since they can't hear. I don't know, maybe I have hearing problems! It doesn't affect my thinking. It just sort of opens my eyes a bit, because, you know, now that I think about it, it's very likely.

ll

Rejection

Occasionally, although much less frequently, subjects show a comparable competence in reflecting on the relation between the passage and their own thinking, but they reject suggestions about causality that they 1: see as present in the passage: (TCms) Well, it doesn't tell how many people were affected by each thing, so I don't really know. It doesn't influence me in either direc- • tion. Or: (TCms) No, because I think more [important] than visual and hearing problems is the thing about lack of interest. Absence of epistemological reasoning

In contrast to the preceding cases of competent epistemological reason.:•· ing are cases in which subjects show no ability to reflect on their o· thinking. They seem unaware of their thinking as itself a possibl~ object of contemplation. Typically, the question the interviewer about the influence of the passage on subjects' thinking is taken to be_i question about influence on the subjects themselves, rather than their beliefs: (TNmc) Can you repeat the question please? (Question repeated,1 Yes. (How does it influence your thinking?) Well, it does affect right, because if I don't really get a good start in life, and I don) really learn or do what is expected of me, before it's too late I

The following response is similar:

1

1

(TCms) No, because these are third-graders, and I'm a ninth-grader. And I know in ninth grade the reasons, though some of them are probably the same. Mostly the reasons are probably different. This subject shows the same epistemological confusion discussed in earlier chapters, one between a cause being true of a person and to a person. In this case, the subject is unable to reflect on his beliefs about the causal ascriptions suggested in the passage because his beliefs about them are not differentiated from their applicability to him. As discussed in chapter 7, the plane of beliefs is not well differentiated from the self in a more concrete, physical sense.

11

, ,II

"i,

,,

••

l'i

,,

I

1111 l•

j'I

1111

11111

Ii

Assimilation

We turn now to what is by far the most common response to the interviewer's request for epistemological reflection. Unlike the two preceding subjects, those whose responses fall into this category appear to address both the information in the passage and their own beliefs. The connection they make between the two, however, is to equate them. The responses in this category vary slightly in the terminology used, but they are essentially equivalent. The following are examples: (TCfs) I think it's about the same as my own opinion. (40Nfs) No, it's pretty much what I always felt. (TCfs) No. (Why not?) Because these are some of the reasons I always thought of as the problem.

I I 1

1

1111

1111II I; Ii 1

1

1'

1

I 'i11

"I '·11

1

''I

1:1

lfll1

11

(60Nmc) No, I think I've known some of this quite awhile. (20Cmc) (Do you have any doubts about what this study suggests?) No. (Why not?) Because it's what I've been thinking all along. (40Nms) There is nothing new in it for me. No change in attitude. (20Nmc) No. Before this study I knew what causes them. (So, it doesn't change your thinking in any way?) it goes with my thinking exactly.

I

I

1,

I

iljll:~ 1

111111

230

The skills of argument

Evaluation of evidence

(TNfs) (Do you have any doubts about what this study suggests?) No. (Why not?) Cause what it does I think is the way I believe it is.

(TNms) It says pretty much the same thing that I think. (Does it influence your thinking in any way?) it makes me believe that I said the right thing. It's actually the truth.

(Does anything from this study influence your own thinking about what causes children tofail in school?) No. (Why not?) it's the same.

Everything I believe is in the study. In interpreting this very common form of response, it needs to be kept in mind that what is in fact contained in the overdetermined evidence are explicit suggestions of three distinct families of causes and that no subjects include all of these causal lines in their own causal theories generated at the beginning of the interview. Hence, subjects' assertions that what is contained in the evidence and what they themselves believed before reading the passage are equivalent are in fact always incorrect. This inaccuracy can be explained in two ways. The subject either (a) has not processed that part of the evidence that pertains to causes outside the range of the subject's own theory, and hence is unaware that there exists anything in the evidence that falls outside of the subject's own causal analysis, or (b) has assimilated these external elements of the evidence to his or her own causal beliefs, evidently without awareness of having done so. In either case, subjects do not maintain firm boundaries between knowledge or beliefs that they themselves hold and information that derives from an external source.

231

(TNms) It'll just build up my theory, and it'll go more ways to showing what causes that problem. In these cases, the differentiation between theory and evidence appears to be clearer than it did in the case of responses in the preceding category. The subject has constructed a plausible relation between the two - that the evidence is congruent with the theory and hence reinforces it. What must be kept in mind in interpreting these responses, however, is the selectivity that the subject has in fact exercised in order to draw such a conclusion. It is not one but three distinct sets of causes that the passage in fact suggests, with each presented as equally viable and important in its potential causal role. Thus, although such subjects have reflected, probably accurately, on the relation between one theme contained in the evidence and their own beliefs, they have imposed their own beliefs on the evidence to such an extent that it has prevented them from accurate! y representing what the evidence in fact consists of. As we have already noted, accurate representation of the evidence, as well as of one's own beliefs, is a precondition for competent representation of the relation between the two. Exceeding the boundaries of evidence

Reinforcement

We turn finally to cases in which subjects do not just equate their beliefs with the external evidence but assert that this evidence reinforces their beliefs. The following are examples: (40Cfs) Well, I always thought this was the cause, but now I am quite sure. (TCms)No, most of it agrees with what I probablywould've thought happenedanyway.(So, it doesn't change your thinking in any way?) It convinces me more of my opinion.

(40Cmc) I think it just reinforces it. (TCmc) Yeah, the job thing, with all the jobs. (How do you mean?) Well, I was sort of for that before this, and it just sort of helps me have a strong feeling.

If the boundaries between the evidence presented in the passage and subjects' own beliefs are not firm, this should be evident in their interpretations of the evidence itself. Many of the examples presented earlier in this chapter are suggestive of these weak boundaries. In this final section, we examine a number of examples that clearly reveal a lack of distinction between the implications that can be drawn from the passage and what the subject knows or believes to be true in his or her own experience. In the responses cited here, the subjects clearly go beyond the boundaries of the presented evidence and base their evaluations of it on knowledge or beliefs that are external to what has been presented: (60Nfs) (Does the study prove that this is the cause?) I think so because the situationis prettymuch the same in so manyother areas. This is a study in one group, but I'm sure that happens in many, many others.

111111111

232

The skills of argument

The subject is queried about the evidence contained in the passage in i/ front of her, but the answer clearly is not confined to the information in,, the passage. Instead, she draws on knowledge or beliefs external to the ( passage as a means of justifying what she claims are the implications'. of this evidence. "' In the following example, the subject justifies his causal attributioni by referring to both the presented evidence and his external know!-.,, edge, in an additive way: (TNms) (Does the study prove that this is the cause?) Yes. (H~ does it prove it?) Cause, you know, it says right here, and I do find it-/ 1 around me. Most peopledo got hearingproblems.For myself, I can't1 hear very well outside this ear, so to me it does prove it. , We see in this response the confusion between what is true of tbJ1 subject and what is true to him. In claiming that "to me" it proves it,:: the subject implies that for someone else (without hearing problems)l 1 the evidence would not have the same implications or force. ,• In some cases, the external information that the subject imports inl evaluating the implications of the presented evidence is not just the! subject's own assertion of what is true but what we in chapter :i' categorized as high-quality, genuine evidence, evidence that, if it were,, established as true, would indeed support the causal claim. What is ati' issue here, then, is not the quality of this external information or i1 relevance to the topic, but its lack of relevance to the question at h - the question of what implications can be drawn from the prese: evidence that the subject has been asked to evaluate: /.;, (60Nms) (Does the studyprove that this is the cause?)I think so; think you couldjust look at our New Yorkschoolsystem. Look at -· various sections of the city where children coming from better vironments do better in school and the children coming from environments do worse in school.

In the following similar example, the interviewer tries to introd1 the distinction that the subject has not, between what she believes to , true and what can be inferred from the evidence she has been asked evaluate: (60Nfs) (What does this study suggest about childrenfailing school?)It might be that some childrenare late learners. They're bloomersand then they catch up. Because I know it happenedto own granddaughter.She was a late bloomer and we were very m1

Evaluation of evidence

•

233

concernedabout her. Today she is a beautiful therapist ... got her master's degree and she's beautiful. It took a little while for her to grasp it, and she is just marvelous. (Doesthe studyprove that this is the cause?)That proved it to me. (Butdoes this studyprove that this is the cause?) Yes, yes. (How does it prove it?) Because it can ... because if a child is a late bloomer like my granddaughter was and then caught on fast all of a sudden ... it all opened up for her and she ... and her studies and she became involved. The failure to maintain boundaries around the presented evidence, reflected in all of the preceding responses, is captured most succinctly in the following response: (20Nfs)(Doesthe studyprovethat this is the cause?)Yes. I feel that it is true. To someone who appreciates the distinction at stake here, the subject appears simply to have substituted a different question for the one that was asked, reflecting perhaps a lack of careful attention to the interviewer's question. To an individual who does not appreciate this distinction, however, there is only one question, not two. What is shown to be true by some circumscribed body of evidence is not distinguished from what is true in general. After examining quantitative results for evaluation of the overdetermined evidence, we return to the implications of this lack of boundary between theory and evidence.

Quantitative results Certainty Those subjects who claim that the evidence proves that the factors they mention are causal are regarded as exhibiting certainty in their evaluation of the overdetermined evidence. Certainty levels are higher for the overdetermined evidence than the levels reported earlier for the underdetermined evidence (table 8.2), and they rarely shift during questioning as we observed them to do in the case of the underdetermined evidence. For the crime topic, the percentage of subjects exhibiting certainty is 71 % , and for the school topic the percentage is 6 I%. As found for the underdetermined evidence, this percentage differs significantly only as a function of education level. Among the noncollege group, 82% are certain for the crime topic and 77% for the school topic. Among the college group, these percentages drop to 59%

''"Ill 111 234

The skills of argument

for the crime topic and 44% for the school topic. These differences / across education groups are statistically significant for both topics •• (appendix 3). Selectivity

A second issue that arises in the case of the overdetermined evidence is \• selectivity. Three distinct sets of potential causes are suggested for • each topic. As the examples quoted have illustrated, subjects are very , often selective in acknowledging some of these causes and not others. '' The percentages of subjects exhibiting selectivity are 69% for the\ crime topic and 62% for the school topic. Subjects not showing selectivity either decline to mention any causes in evaluating the passage, as :: several of the examples presented illustrate, or make some reference to i all three sets of potential causes. As is the case for certainty, these percentages differ only as a function of education level. For the school topic, 77% of the noncollege group and 47% of the college group show; selectivity, a statistically significant difference (appendix 3). For thei crime topic, the comparable percentages are 76% for the noncollege • group and 62% for the college group, a difference that falls just below:; statistical significance. • '· For the crime topic, overall frequencies of mention of the threej broad sets of causes are 19% (failure of prison to rehabilitate), 36%.i (life history), and 45% (readjustment to society). For the school topic,] they are 23% (individual problems), 35% (school factors), and 43% (home factors). For neither topic do these percentages vary signifi

l:

-~

..2

"' >

~