Using Dictionaries: Studies of Dictionary Use by Language Learners and Translators [Reprint 2014 ed.] 9783110929997, 9783484309883

Citation preview

Series Maior

LEXICOGRAPHICA Series Maior Supplementary Volumes to the International Annual for Lexicography Supplements ä la Revue Internationale de Lexicographie Supplementbände zum Internationalen Jahrbuch für Lexikographie

Edited by Sture Allen, Pierre Corbin, Reinhard R. K. Hartmann, Franz Josef Hausmann, Ulrich Heid, Oskar Reichmann, Ladislav Zgusta 88

Published in cooperation with the Dictionary Society of North America (DSNA) and the European Association for Lexicography (EURALEX)

Using Dictionaries Studies of Dictionary Use by Language Learners and Translators Edited by B.T. Sue Atkins

Max Niemeyer Verlag Tübingen 1998

Die Deutsche Bibliothek - CIP-Einheitsaufnahme [Lexicographica / Series maior] Lexicographica : supplementary volumes to the International annual for lexicography / publ. in cooperation with the Dictionary Society of North America (DSNA) and the European Association for Lexicography (EURALEX). Series maior. - Tübingen : Niemeyer. Früher Schriftenreihe Reihe Series maior zu: Lexicographica 88. Using dictionaries. - 1998 Using Dictionaries : studies of dictionary use by language learners and translators / ed. by B.T. Sue Atkins. - Tübingen : Niemeyer, 1998 (Lexicographica : Series maior ; 88) ISBN 3-484-30988-1

ISSN 0175-9264

© Max Niemeyer Verlag GmbH, Tübingen 1998 Das Werk einschließlich aller seiner Teile ist urheberrechtlich geschützt. Jede Verwertung außerhalb der engen Grenzen des Urheberrechtsgesetzes ist ohne Zustimmung des Verlages unzulässig und strafbar. Das gilt insbesondere für Vervielfältigungen, Übersetzungen, Mikroverfilmungen und die Einspeicherung und Verarbeitung in elektronischen Systemen. Printed in Germany. Gedruckt auf alterungsbeständigem Papier. Druck: Weihert-Druck GmbH, Darmstadt Einband: Industriebuchbinderei Nadele, Nehren

Table of Contents

Β. T. Sue Atkins Introduction

1

Jan H. Hulstijn & Β. T. Sue Atkins Empirical Research on Dictionary Use in Foreign-Language Learning: Survey and Discussion

7

Β. T. Sue Atkins & Krista Varantola Language Learners Using Dictionaries: The Final Report on the EURALEX / AILA Research Project on Dictionary Use

21

Β. T. Sue Atkins & Krista Varantola Monitoring Dictionary Use

83

Kristen Mackintosh An Empirical Study of Dictionary Use in L2-L1 Translation

123

Paul Bogaards What Type of Words do Language Learners Look Up?

151

Hilary Nesi Defining a Shoehorn: The Success of Learners' Dictionary Entries for Concrete Nouns

159

Krista Varantola Translators and their Use of Dictionaries

179

Margaret Rogers & Khurshid Ahmad The Translator and the Dictionary: Beyond Words?

193

Resumes

205

Zusammenfassungen

209

Notes on Contributors

213

Β. Τ. Sue Atkins

Introduction

Great strides have been made in dictionary making in recent years, thanks principally to the advent of computer typesetting, computer-assisted dictionary compiling, and the use by lexicographers of electronic corpora as a source of objective information about the language or languages they are describing. The contents of published dictionaries have suddenly been subjected to rigorous critical attention by computational linguists, intent on extracting from them by automatic means lexical information for the lexicons of language engineering systems. Scholars in linguistic disciplines are focusing as never before on lexical matters. More dictionaries than ever are being produced, distributed, sold, and (one assumes) used. A dictionary is a complex assembly of interwoven facts, often presented in semi-sentences, difficult to understand, and larded with abbreviations and references to concepts (transitive verb, colloquial, obsolete . . .) unfamiliar to the majority of its users. When the dictionary is bilingual, the situation is worse: almost 50% of its contents are guaranteed to be unfamiliar to any reader. Yet it is rare to find dictionary skills being taught at all, far less systematically, in schools and colleges anywhere. For their part, lexicographers try conscientiously to make their work accurate and accessible within the limits imposed on them by the print medium and budgetary constraints. However, research by dictionary makers is naturally enough oriented to market concerns: what kind of dictionary sells best? (The EURALEX- and AILAsponsored research described in this volume is a happy exception to this.) There is a general belief amongst those concerned with dictionaries that dictionary users do not get the best out of their dictionaries, and, conversely, that dictionaries themselves could be improved so as to serve their users better. The authors of the essays in this volume certainly agree with both these statements. The problem is, of course, what specific skills should be taught to students to enable them to use their dictionaries better? And how can the dictionary be made more informative and helpful for its users? The answers to these questions depend on knowing what actually happens when people use their dictionary. What are the users' assumptions? What kind of words do they look up? How do they interpret what they find? Do they read the whole entry or only the parts in roman type? Do they skip definitions, or translations, and subconsciously try to extract information from the examples? Do they skip all the words and abbreviations that they do not understand? Do they carry their view of language from their own language straight over with no concessions into the foreign language they are working in? How good are they at knowing when they should be turning to their dictionary? Does being taught 'dictionary skills' improve their chances of successful dictionary use? All these questions, and many others, are addressed in the papers in this volume, 1 which draws together studies of how dictionaries are used by language students and student transla-

1

With the exception of the two articles by Atkins and Vaiantola, these papers were first presented at the Symposium on Dictionary Use held at the 1996 AILA congress in Jyväskylä. Finland, and organized by Paul Bogaards, Batia Laufer and Krista Varantola. of the AILA Scientific Commission on Lexicology and Lexicography.

2

Β. Τ. Sue Atkins

tors, and in one case by language professionals, working with a foreign language. The dictionaries used in these projects range from large monolingual dictionaries designed for native speakers of the foreign language, through standard bilingual dictionaries of various sizes, to monolingual dictionaries in the language of the user, and specialised dictionaries of many types. The dictionary users who constitute the subjects of these projects come from many different language communities, and range in age from school students to senior professors. The tasks being carried out with the help of dictionaries for the various research projects include L2-L1 translation, L1-L2 translation, L2 comprehension, self-expression in L2, and various project-specific linguistic exercises. The volume opens with an overview, Empirical research on dictionary use in foreignlanguage learning, by Jan Hulstijn and Sue Atkins, which sets the scene for the subsequent accounts of recent empirical research amongst language learners. After a comprehensive survey of earlier studies, classified under seven headings, the authors list the key factors involved in the process of using a dictionary for help with a foreign-language task, and briefly discuss the type of research needed, suggesting that these variables be systematically treated if a reliable methodology is to be developed. They conclude with two theoretical examples of planned studies, to illustrate the points made in the discussion. This paper is accompanied by a full bibliography. The most ambitious and wide-ranging of the studies reported on in this volume is described in Language Learners Using Dictionaries by Sue Atkins and Krista Varantola, a detailed account of a large-scale research project sponsored by the European Association for Lexicography (EURALEX) and the Scientific Commission on Lexicology and Lexicography of the Association internationale des langues appliquees (AILA), and funded to a large extent by major dictionary publishers in several European countries. The research was carried out with the help of school and university teachers and over one thousand students in four European language areas (French, German, Italian and Spanish). Dictionary publishers in the UK design their English monolingual learners' dictionaries for use by learners of English worldwide. It is clearly impossible to define such a population in order to sample it in a statistically correct fashion: the subjects in this project spanned a wide range of native tongues, linguistic abilities, knowledge of the L2 (English), and dictionary skills, and were in that respect representative of the user group of the dictionaries involved in the project. The research investigated subjects' use of dictionaries in various tasks: translating to and from L2, comprehension of L2, and various lexical and grammatical exercises. The user profile information is highly detailed, including an evaluation of subjects' L2 (English) skills, and they recorded not only the specific dictionary they had on their desks, but for each question in the tests they noted whether or not that dictionary was used. The paper constitutes a complete record of the results of the investigations, and, equally important, a full and unvarnished account of the methodology used to elicit these results, for the benefit of others who may wish to employ similar investigative measures. In many cases the results confirm general beliefs about dictionary use, but in many others they give a tantalising glimpse into what really happens when people use their dictionaries. An evaluation of this broad-based study highlighted one type of information missing from its results, and indeed impossible to investigate in the multiple, varied and scattered locations in which the study was carried out: a detailed record of the dictionary look-up process. What made people reach for their dictionaries for help with a translation? What kind of information about L2 were they hoping to find? Did they find it? At what point in a search

Introduction

3

did they change from one dictionary to another? When did they call a halt to a search? Were they satisfied? If not, why not? Questions like these formed the focus of a complementary study, the EURALEX Oxford Workshop on Dictionary Use, which is reported on in Monitoring Dictionary Use, also by Atkins and Varantola. The methodology described in this paper was devised for, and tested at, the one-day workshop, which was attended by 71 participants (all language professionals: lexicographers, teachers, researchers, translators) from 15 different language communities, and later used in the analysis of dictionary use by a homogeneous group of 32 Finnish translation students. The paper gives an account, illustrated by various individual case studies, of the different choices and decisions, problems and solutions, that occur when dictionaries are used in translation tasks; it attempts to find answers to the questions already mentioned above, among others, and to provide facts about the outcome of individuals' searches, and the strategies employed when a look-up failed. The authors argue that studies such as this, in which every step of dictionary consultation is monitored and recorded, are required in order to provide the essential information needed by lexicographers (for compiling dictionaries) and by language teachers (for teaching dictionary skills). The methodology devised for tracking the steps in the dictionary consultation process at the EURALEX Workshop was also used in the first of the more narrowly targeted projects described in this volume : two related tests of dictionary use by student translators in the

University of Ottawa, and reported on in An empirical study of dictionary use in L2-L1 translation, by Kristen Mackintosh. Motivated by "the need to understand specific dictionary user groups" for whom may be created customized electronic dictionaries, with their virtually unlimited storage space, Mackintosh undertakes a study of the ways in which student translators use their dictionaries when translating from a foreign language into their own - in this instance, either from French to English or from English to French. The first of the two tests, performed by fifteen students, was designed to gather general information about how students use dictionaries in L2-L1 translation. The second was inspired by the specific findings of the first test, and devised to discover whether the language (LI or L2) in which definitions were couched had an effect on the students' dictionary skills and habits. The searches carried out by the two groups (anglophones and francophones), their outcome, and the problems experienced by the students, were recorded and analysed, and implications drawn for the teaching of dictionary skills, and for dictionary design; areas worth exploring in future research are also indicated. The findings appear to suggest that a "hybrid" dictionary format, containing both definitions and equivalents, is best suited to the needs of translators.

What type of words do language learners look up? by Paul Bogaards, focuses on one single aspect of dictionary consultation, perhaps the key to successful dictionary use: the type of L2 words in a translation passage that initiate a dictionary search and - possibly more important - the type of words that students tend not to consider worth looking up, for whatever reason. The kind of words to be studied in this project were classified under three headings, infrequent words, allosemic words (common words in unusual senses or idiomatic uses), and faux amis. Forty Dutch students of French took part in the research, using French test passages which had been selected as representative of different text types with regard to style and subject matter; details of their searches and the success or failure of these were noted. Among the conclusions drawn is the fact that students seem consistently to underestimate the difficulties of both allosemic words and faux amis, and that text type may have a bearing on whether or not a student will turn to a dictionary for help with such words.

4

Β. Τ. Sue Atkins

The paper by Hilary Nesi, entitled Defining a shoehorn: the success of learners' dictionary entries for concrete nouns, also focuses on one aspect of dictionary use, but in this case the 158 subjects are drawn from three different groups, all studying English as L2: undergraduates in Botswana, Japanese undergraduates in England, and a mixed group of adult learners in England. The object of the study was to evaluate the effectiveness of L2 definitions, in this instance the definitions of concrete nouns in English monolingual (mainly learners') dictionaries. The decision to focus on concrete rather than abstract nouns reflects the fact that the absence of valency information normally results in simpler definitions; moreover, their "picturability" was integral to the tests. The six nouns were selected because they denoted objects familiar to the subjects, for the most part, but were expected to lie outside the students' vocabulary range. Subjects recorded their initial familiarity or unfamiliarity with the words in question, evaluated the helpfulness of the definition offered for each, wrote sentences designed to show whether the definitions had got the meanings across, and finally picked out an object from a group to associate with each of the words. The detailed findings (such as the misleading ambiguity, in the definition of shoehorn, of heel which could be interpreted as part of a foot or part of a shoe) provide a lesson for lexicographers working on learners' dictionaries, and for those teaching dictionary skills, and lead to the conclusion that the wording of definitions plays a significant part in the success or failure of dictionary look-up. The last research project described in this volume used an adapted form of the methodology devised for the EURALEX Workshop, and also employed in the Canadian research, in which every step of the dictionary consultation process was recorded in full detail on a standardised form. However, in the project described by Krista Varantola in the paper entitled Translators and their use of dictionaries, the subjects did their own recording; the other difference from the Workshop experiment is that here the results of the searches were also recorded, and indeed the four subjects, all Finnish student translators, completed and handed in a full translation into English of the Finnish test passage. The small scale of the project, the homogeneous nature of the subject group, and the great depth of the investigation allowed Varantola not only to evaluate the translation equivalents selected in each problematic instance, but also, after studying the dictionaries used, to postulate reasons for dictionary consultation failure. The conclusion she draws is that no dictionary, or even a number of detailed and specialised dictionaries, can meet the translator's need for context-sensitive equivalents: the translator's workstation must be equipped with a far richer database of reference materials. This conclusion supports the initial hypothesis of Margaret Rogers and Khurshid Ahmad, who, in the final essay, entitled The translator and the dictionary: beyond words?, look at some of the ways in which technology may be harnessed to provide better reference sources for the translator. Building on Ahmad's concept of a "virtual corpus" (to be created interactively by a translator for the purposes of a specific translation or type of translation), they argue for a dynamic approach to solving the problems of specialist terminology translation, adducing evidence from dictionaries and corpora to support their case. The paper touches on the key aspects to be considered in the planning of a customizable reference source, and could well serve as a discussion document for the initial specification of such a resource. Indeed, "attention to detail" is perhaps the watchword for this volume, in which seven veiy different pieces of research are described and evaluated. The authors have tried to include

Introduction

5

enough detail to allow readers to replicate the tests, and adapt them to serve their own interests. Dictionary consultation is highly complex: many more such experiments are needed before lexicographers have enough information to allow them to make reasoned changes in dictionary design, and before those teaching dictionary skills know enough about their students' attitudes and habits to guide them through the decision-making steps o f the dictionary look-up.

The purpose of this volume is to provide a launch-pad for other

investigative experiments from which dictionary makers and dictionary users alike may benefit.

Jan Η. Hulstijn & Β. Τ. Sue Atkins

Empirical research on dictionary use in foreign-language learning: survey and discussion Abstract This paper begins with a brief survey, in the form of a classified bibliography of research into dictionary use. A discussion follows of the type of research required in order to increase one's insight into the cognitive processes involved in using a dictionary; the principal factois which affect the outcome are listed. Two samples of such research design are outlined. The objective of such studies is given as setting the scene for a new generation of electronic dictionaries, optimally customizable by users according to their individual look-up habits and linguistic needs.

1. Survey The subject of dictionary use is very much alive today, as is evidenced by the number of scholarly papers devoted to it. In order to bring up to date the survey by Bogaards (1988), a search of the literature was conducted, starting from the Linguistics and Language Behavior Abstracts, San Diego, California (LLBA) and thereafter tracing publications referred to in papers listed there. This brought to light some fifty published papers reporting on empirical investigations in which the dictionary was involved in one way or another. The studies thus identified were classified under seven headings, according to the aspects of dictionary use covered in each, and are listed here for reference without further comment. In the case of studies whose abstracts appear in the LLBA source, their LLBA entry numbers have been included in the bibliography at the end of this paper. It should be remembered that all the studies listed below are not wholly dedicated to the topic under which they are listed (although some are), but they all contain a substantial amount of material relating to that topic. 1.1 The attitudes, needs, habits and preferences of dictionary users This broad domain covers studies which gather information about dictionary users themselves, what they think about their dictionaries, what they expect from them, what they customarily use them for, and the types of dictionaries they choose for various tasks. The following articles relate to this topic: Atkins et al. (1987); Atkins & Knowles (1990); Atkins & Varantola (Language Learners Using Dictionaries in this volume); Baxter (1980); Bejoint (1981); Benbow, Carrington, Johannesen, Tompa & Weiner (1990); Bogaards (1985, 1988, 1992); Coviello (1987); Cumming, Cropp & Sussex (1994); De Andrada (1992); Diab (1990); Gallison, 1983; Hartmann (1982, 1983); Hatherall (1984); Hohne (1991); Houtman & Wouters (1994); Ibrahim & Zalessky (1989); Kernerman (1996); Kipfer (1987); Laufer & Melamed (1994); Nuccorini (1992); Taylor & Chan (1994); Tomaszczyk (1979); Tono (1991, 1992).

8

Hulstijn & Atkins

1.2 Text or word comprehension This type of research focuses on how readers arrive at an understanding of the text they are reading, having regard to the following factors: • the presence or absence of a dictionary (since that affects the time required for the reading, as well as the comprehension of the text); • the type of dictionary used: monolingual or bilingual; • the medium: electronic or print dictionary; • various factors relating to the text itself, e.g. the frequency or 'difficulty' of the vocabulary, its perceived importance in the understanding of the text, the inferability of the words looked up, etc.); • lexical aspects of the way in which the dictionary explains the meaning of words, i.e. the entry component or components involved: the style of the definitions, possible restrictions on the defining vocabulary, whether the examples are authentic or composed, etc.). The principal dependent variable being evaluated here is the reader's comprehension either of the text as a whole, or of selected lexical items in the text. The following articles relate to this topic: Atkins & Varantola (Monitoring Dictionary Use in this volume); Aust, Kelley & Roby (1993); Bensoussan, Sim & Weiss (1984); Bogaards (1994, and this volume); Brito (1992); Hulstijn (1993); Hulstijn, Hollander & Greidanus (1996); Laufer (1993); MacGregor & Thomas (1988); Roby (1993); Summers (1988); Tono (1987, 1988, 1989); Walberg & Ling (1985). 1.3 Text or word production The main research question in this type of research is how language production is influenced by the following factors: • the presence or absence of a dictionary (since that affects the quality of the text produced); • the type of reference work used: monolingual dictionary, monolingual thesaurus, or bilingual dictionary; • the medium: electronic or print; • lexical aspects of the explanation of word usage found by the reader in the work consulted. The variable being evaluated here is the proportion of errors in a written task resulting from misinterpretation of the reference information. The following articles relate to this topic: Ard (1982); Atkins et al. (1987); Atkins & Knowles (1990); Atkins & Varantola (Language Learners Using Dictionaries and Monitoring Dictionary Use, both in this volume); Hatherall (1984); Herbst, (1985); Huang (1985); Laufer (1992, 1993); Meara & English (1988); MacGregor & Thomas (1988); Müllich (1990); Nesi (1994); Nesi & Meara (1994); Nesi (this volume); Neubauer (1985); Nuccorini (1994). 1.4 Vocabulary learning This category contains studies on vocabulary learning, both incidental to other tasks and as a specific objective in itself, and covers: • incidental vocabulary learning through the use of a dictionary to facilitate reading or writing in a foreign language, taking account of:

Empirical Research on Dictionary

Use

9

- the type of dictionary (monolingual or bilingual); - the medium (electronic or print); - lexical aspects of the explanation of word usage found by the reader in dictionary consulted, etc. • specific language learning strategies, asking such questions as: - what strategies do L2 learners use to increase their vocabularies? - when do learners use a dictionary for that purpose? - to what extent do learners exhibit individual differences in dictionary use (e.g. their learning style)? The variable being evaluated here is the retention by the dictionary user of facts about new expressions: their meaning, their syntactic properties, etc. The following articles relate to this topic: Bialystok (1983); Bogaards (1991); Fischer (1994); Hulstijn, Hollander & Greidanus (1996); Luppescu & Day (1993); Summers (1988). 1.5 Dictionary-related performance in testing Studies in this area examine the relationship between L2 learners' performance in L2 reading or writing tasks and their use of dictionaries during the tasks, in an attempt to determine how far the use or non-use of the dictionary affects the reliability and validity of language tests. The following articles relate to this topic: Bensoussan (1983); Nesi & Meara (1991). 1.6 Teaching dictionary skills This type of research pertains to the effectiveness of pedagogical programmes, techniques and materials in the teaching of dictionary skills. The variable being evaluated here is proficiency in dictionary use. The following articles relate to this topic: Atkins et al. (1987); Atkins & Knowles (1990); Atkins & Varantola (Monitoring Dictionary Use in this volume); Barone (1979); Blok (1994). 1.7 Critical comparisons and reviews of dictionaries Strictly speaking, this category does not belong to the domain of empirical studies of dictionary use. However, some reviews which compare the performance of two or more dictionaries do so by evaluating samples of dictionary entries, and often include tables showing credits for features investigated. Since such investigation involves direct use of dictionaries by the experts reviewing the works, we include these studies here. The following articles relate to this topic: Bogaards (1995, 1996); Carter (1989); Hausmann & Gorbahn (1989); Herbst (1996); Fillmore (1989); Jansen et al. (1987); MacFarquhar & Richards (1983); Nesi (this volume).

2. Discussion In this section we outline some ideas for making research into dictionary use more coherent and systematic, by using a theory-based approach, in order to gain more insight into the cognitive process involved when a dictionary is used; the ultimate objective of this is some

10

Hulstijn & Atkins

day to create dictionaries, electronic of course, whose design and contents are rich and flexible, and which are open to extensive adaptation by users, according to their various needs and the different ways in which they use their dictionaries. This section owes much to Bogaards (1995) and Hartmann (1987, 1995), whose influence we gratefully acknowledge. Atkins (1996) gives some idea of one such dictionary might contain and how it might function. We do not concern ourselves with research into reading, writing or language learning per se, nor with market research, but with research which aims at bringing the dictionary to the user (how can the dictionary best serve its users' needs?) and bringing the user to the dictionary (how can people be made better dictionary users?). 2.1 Motivation for empirical research Looking at dictionary use from the perspective of language learners, Bogaards (1995) makes the following relevant points: • learners do not like using a dictionary; • learners do not know how to use the dictionary; • dictionaries are too difficult for learners; • dictionary use hinders reading comprehension. On the other hand lexicographers, looking at dictionary use from a different point of view, have been known to complain that users do not use the wealth of information that the dictionary offers. These contrasting perspectives illustrate the dilemma that faces us. The traditional print dictionary cannot be perfect; it can never be ideally suited to each individual user's needs, language skills, world knowledge, intelligence and preferred look-up methods. (One clear-cut example of an insoluble problem of this type is the situation of the bi-directional bilingual dictionary, created for users of two languages, in which each entry must meet the need both of the encoding source-language speaker and the decoding target-language speaker.) Recent years have seen the advent of bigger and more specialized dictionaries for various categories of users (for instance, the various advanced learners' dictionaries of English, Longman's Language Activator for language production, etc. ), and the production of electronic dictionaries and other electronic aids. These developments have contributed little to relieve the situation. How can users improve their dictionary skills? And how can dictionaries be made more useful for their users? If this is to come about, much more information is needed about the way in which people actually use their dictionaries. Technological development has, however, affected the dictionary scene: the dictionary is losing its independence, as it is more and more frequently integrated into larger information systems which can include spelling checkers, grammars, and encyclopedias, among other resources. Professional dictionary users can handle these new types of dictionary reasonably well, and relatively simple questionnaires elicit at least some information about how dictionaries can be improved for such users. These techniques are however suspect at the best of times (people notoriously answer questionnaires in a way which projects an acceptable image of the respondent), and in any case they are not feasible when dealing with non-professional dictionary users, that is, those who are not translators, professional writers, language teachers, academics, lexicographers, etc. More empirical research is needed in this area, to find out exactly the nature of the problems that beset the non-professionals.

Empirical Research on Dictionary Use

11

2.2 Some general questions Dictionary use is a subtle problem-solving activity involving many factors. For instance, one cannot easily determine whether, for a given task, the use of a monolingual dictionary is better or worse than the use of a bilingual dictionary. The effectiveness of either depends on, inter alia, the kind of lexical item involved, the context it is found in, and the user's knowledge of L2, as well as his or her metalinguistic proficiency and inferencing skills. Thus the results of empirical research which compares the use of monolingual and bilingual dictionaries across the board, without taking account of such variables, cannot be relied upon. The situation calls for an interactive, iterative approach, in order to determine eventually what types of language learners need what types (monolingual or bilingual) of information about what types of linguistic units for assistance with what types of linguistic tasks? We find ourselves today in an age of information technology in which it is easy to be overwhelmed by huge masses of information. There are two contradictory reactions to this situation,leading to two diametrically opposed solutions. The first course advocates that no information should be withheld from anyone: proponents of this approach argue that everyone should have access to all the information available, but should be taught how to handle this avalanche of complex information, how to develop strategies in order to exploit it to the full in their individual circumstances. The second course advocates that people should not be offered large quantities of information until they can make use of it, but should rather gain access to facts on a "need to know" basis. Routines should be set up after careful planning, in order to lead people quickly to the information they seek, and to avoid the distraction of large amounts of irrelevant detail. The relative efficiency of these approaches to information access is highly relevant to the use of dictionaries by non-professional users. Research must address the question of how complex dictionary information can be optimally made available to such users. So far, the best dictionary is undoubtedly a human dictionary, in the form of an encouraging and knowledgeable parent, teacher, or colleague. The reason for this is that a good human dictionary instinctively assesses the situation and tailors the information to the knowledge and needs of the enquirer. Fine-tuning comes from subsequent interaction of user and human dictionary. In contrast, there is not much interaction between the language user and the print dictionary. However careful the lexicographer, and skilled the user, by its very nature of having to be all things to all users, the print dictionary usually offers either too little or too much information. With the advent of computerized dictionaries, however, straightforward interaction is now possible between the language user and the dictionary. Such interaction can often be objectively monitored by the computer, and appears to be an ideal area for research. 2.3 Research design and planning For any piece of research to be valid, the operation studied must be analysed in enough detail, and systematically enough, for the various factors which affect its outcome to be isolated and identified; the investigation must then be structured in such a way as to ensure that the single factor to be studied is as far as possible the only variable in the experiment.

12

Hulstijn & Atkins

How can one investigate such a subtle and complex activity as dictionary use? We suggest that studies carried out in this technological age should take account of the following points: • Expert behaviour is a good starting point for devising research questions. When a language learner consults a teacher, the "experts" in this human dictionary consultation scenario are (i) the intelligent language user/learner (an expert human information seeker) and (ii) the good language teacher (an expert human information provider). • The interaction between seekers and providers of (lexical) information involves many interrelated variables (linguistic, psychological, pedagogical). These must be painstakingly teased out, and tasks devised in which only one factor is variable in any given operation. The emphasis must be on a systematic manipulation of these variables. Thus a one-shot study will not do the job; a series of investigations, planned as a whole, is required, with specific "switching" of the variables listed below. • Recent technological developments offer researchers in many fields the opportunity to revise and improve their methods: studies in dictionary use may certainly benefit from these new tools, both in ensuring a more objective approach, and in monitoring the activities studied. 2.4 Variables in the use of L2 dictionaries The following (of which some were adapted from Atkins et al. 1987) may serve as a start-up list of variables in the dictionary use scenario: 1. The user's "sophistication" (general educational level, reading, writing and inferencing skills). 2. The user's proficiency in the foreign language. 3. The user's understanding of dictionary metalanguage. 4. If actual dictionaries are involved, the user's familiarity with the dictionary or dictionaries being used in the task. 5. The user's knowledge of the subject matter of the text being understood, translated, read, written about etc. for the purposes of the investigation. 6. The format and type of the specific task (written or spoken; translating or comprehending, etc.), including any time limits imposed. 7. The level of difficulty of the task for the subjects carrying it out. 8. The language (LI or L2) in which the information is sought or provided. 9. The type of linguistic unit involved (e.g. content or function word), the linguistic information (semantic, grammatical, orthographic, encyclopedic etc.) sought about that unit, and the linguistic and cultural distance between LI and L2. (Linguistic and cultural distance tends to be reflected in the proportion of lexical items more or less easily "translatable" between the languages involved.) 10.The medium in which the dictionary information is presented (print or electronic). 1 l.The type of dictionary or dictionary entry used (monolingual, bilingual, "hybrid", etc.); 12.The way in which the information is presented in the dictionary or dictionaries being used: classification, layout, metalanguage, abbreviations, typeface conventions etc. 13 .The source of dictionary information available to individuals participating in the investigation: actual dictionaries or "cited" entries; moreover, the latter may either be extracted from a published dictionary or composed for the purposes of the test.

Empirical Research on Dictionary Use

13

14.The adequacy of coverage of a dictionary vis-a-vis the actual tasks involved (i.e. whether or not the information sought actually is to be found in the dictionary or dictionaries used). To summarize the above: variables 1 through 5 relate to the dictionary user; variable 6 to the task itself; variables 7 through 9 to both the user and the task; and variables 10 through 14 to the dictionary material used in the investigation. 2.5 Methodological recommendations It is suggested that any new investigation might benefit from using the above as a checklist of factors to be taken into account, if all but one are to be neutralized so that a single variable may be isolated and studied. The following recommendations may be helpful, but can only constitute a starting point in the detailed planning of dictionary use research: Subject profiling • Specify clearly the criteria for subject selection in terms of age, educational level, metalinguistic knowledge, inferencing skills etc., taking account of variables 1 through 5, and 7 through 9. Metalinguistic knowledge and inferencing skills can be assessed by means of tasks devised for this purpose, or estimated on participants' educational qualifications, professional experience, etc. • Specify clearly subjects' L2 proficiency, especially their knowledge of vocabulary, administering pre-tests to check this and selecting subjects accordingly. It is rarely if ever possible to have absolute control over the extent of the subjects' L2 vocabulary. If this aspect is crucial to the test, consider using pseudo-words (of which no subject can possibly have any preknowledge), as was done for instance in the study by Hulstijn (1993). For a more comprehensive discussion of using artificial linguistic materials in empirical research on language learning and language use, see Hulstijn (1997). Task description • Clearly specify the task to be performed by the subjects. Subjects must be explicitly informed about the goals of their reading/writing/translating etc. activity and about the conditions under which they have to perform their task (e.g. in terms of time and quality). • Give instructions on the carrying out of the tasks in the mother tongue of the subjects. Methodology and results • Try out every question informally on several test subjects before finalizing even the first draft of the questionnaire: it is hard for the deviser of the tasks to identify all possible misinterpretations by the subjects, inadvertent distractors in the questions, and subjective skewing of the results. • Once the tasks are fixed, a pilot study involving every aspect of the final investigation is always advisable. • When coding subjects' responses, conduct separate analyses for each linguistic category (i.e. for each type of linguistic unit consulted) and for each type of information (semantic, orthographic, etc.) sought.

14

Hulstijn ά Atkins

2.6 Elicitation techniques Electronic or print dictionaries may be involved in dictionary use research, and each type has its advantages and disadvantages. Electronic dictionaries are not familiar objects to many subjects, and their use may interfere with the results of the investigation, unless great care is taken to ensure that the subjects form a homogeneous group along the axis of "computer literacy". However, when it comes to monitoring dictionary use the computerized dictionary offers notable advantages. It can be designed to provide either unrestricted or restricted information (with stepwise access, using a HyperCard-type organization). In this type of program, the computer will unobtrusively keep track of everything the subjects do, including the time devoted to each operation, by making so-called log files, which can later be coded and analyzed; this technique is described, for example, in Hulstijn (1993) and was used in testing the COMPASS 1 dictionaries. Thus, the principal advantage of computer-controlled investigations is that they produce a complete record of subjects' dictionary use, and by unobtrusive on-line monitoring remove much of the danger of the observer affecting the experiment. The use of print dictionaries may also be recorded in detail, for later analysis. Questionnaires (cf. those shown in Atkins & Varantola's Monitoring Dictionary Use in this volume), and "think-aloud" protocols and other types of self-recording (cf. Varantola's Translators and their Use of Dictionaries in this volume) may be used in an effort to discover the skills and strategies deployed by the person using a print dictionary. If, however, the entry information is to be manipulated for the purpose of the experiment, paper dictionary entries must be created for the investigation, in order to provide only the entries needed for the task being assessed, and this is considerably more cumbersome than producing electronic pseudo-print entries. Examples of research using both types of dictionary, electronic and especially contrived paper dictionaries, are proposed in the next section. 2.7 M o d e l investigations In this section two investigations are described in order to illustrate the various points made in the preceding sections. It should be noted, however, that in order to cover as many aspects of the topic as possible, these samples have intentionally been made far too complex to be realistic. Initial investigations should have a much more modest design than those outlined here.

2.7.1 Sample investigation #1 In this example, variables 1, 2, 9 and 11 in the list above are being manipulated, viz. the educational level and L2 proficiency of the subjects, the linguistic units involved, and the type of dictionary used. All other potentially relevant variables must of course be held constant. The model investigation may be summarized as follows:

1

COMPASS: Adapting Bilingual Dictionaries for Online COMPrehension Assistance, an EC-funded project (LRE 62-080: DG-XIII).

Empirical Research on Dictionary Use

15

Text type, task goal, and task conditions "A friend of yours has recently moved to a new apartment. She has written a letter in (the subject 's LI) in which she gives an account of how she found the apartment, what repairs were necessary and the exciting events of the day of her move. She has asked you to translate this letter into (the subject 's L2) because she doesn't know that language but wants to send the letter to a friend w h o does not speak (the subject's LI). Translate the letter as precisely as you can. Use the dictionary always unless you feel absolutely sure about the translation of a word or phrase. You may take as much time as you want." Linguistic units of investigation 20 target words, consisting of 10 content words which can and 10 content words which cannot be directly translated from L I into L2. Dictionary information This model involves the use of two print dictionaries. Half of the subjects carry out their task with the aid of a large, unabridged dictionary containing extensive information (semantic, grammatical; examples of word use). The other half of the participants use an abridged dictionary giving little information but adequate vocabulary coverage for the task. Subjects There are four subject groups (age range 18-30) classified on the grounds of: •

educational level low versus high (to be specified);

•

L2 proficiency intermediate versus high (to be specified).

Half of each of the four subject groups perform the translation task with the unabridged dictionary, the other half with the abridged one. Principal dependent variables to be evaluated The number of correctly translated words and number of dictionary consultations for each of the four types of lexical items, for each of the two dictionary conditions, and for each of the four subject groups. Research questions The results when analysed should give information about the effect which the following axes of investigation have on the dictionary use process: •

translatability of linguistic units: high versus low;

• type of dictionary: extensive versus limited information; •

educational level of dictionary users: low versus high;

•

L2 proficiency of dictionary users: intermediate versus advanced.

16

Hulstijn & Atkins

2.7.2 Sample investigation #2 In this investigation, the electronic dictionary offers several alternative routes to the information sought, by means of a stepwise presentation o f nested information in response to a request. Research question Which access route is best for which user category? Task Subjects read a text containing target words calculated to be unfamiliar to them, and are told that they will subsequently be tested on their understanding of the text. Dictionary information The relevant dictionary entries are long, offering extensive information (orthographic, etymological, grammatical, semantic information; selection restrictions, authentic examples o f use, etc. 2 ). Three alternatives obtain with regard to access to the dictionary information: 1. The whole entry is simultaneously available (as it is in a normal paper dictionary). 2. The information in the entry is presented in various phases. At each step, users are given two or more options to choose from, and are thus led towards the information they will finally select (whether correct or incorrect), without seeing all the rest o f the information which the entry contains. 3. The computer offers preliminary customization by users of the type o f information offered, and users work using their own menus. Subjects Various subject groups can be envisaged, differing in L2 proficiency and experience in (conventional) dictionary use, ranging from novices to experts. Hypothesis to be tested Experts will profit more from the types o f access numbered (1) and (3) above, whereas the stepwise mode (2) is better suited to novices. 2.8 C o n c l u s i o n Until now, empirical research on dictionary use has been rather haphazard. What is needed is a systematic study of the way in which various variables interact when dictionary users consult a dictionary which contains complex information. The main variables whose interaction should be investigated are: (a) users' metalinguistic knowledge and inferencing skills;

2

This description fits only the as-yet-unpublishcd dictionary of the future! compiled for the purpose o f the investigation.

However, such entries may be

Empirical Research on Dictionary Use

17

(b)

users' L 2 proficiency;

(c)

the kind o f linguistic information needed from the dictionary; and

(d)

the language in w h i c h the information is sought and provided.

A systematic investigation o f the interplay o f these variables should help u s find an a n s w e r t o the question o f what information a dictionary should optimally provide, taking into account factors (a) through (d). T h e answer to this question will hopefully lead to s o m e reconciliation o f t w o o p p o s i n g v i e w s in current educational thinking: should b e w i t h h e l d

one, the belief that n o information

( p e o p l e must b e taught to deal w i t h the enormous quantity

and

c o m p l e x i t y o f that information, b y helping them to devise strategies w h i c h m e e t their o w n needs), and the other, the belief that information should b e withheld temporarily until the p e o p l e concerned are able to understand it and u s e it correctly (a "need to k n o w " basis, involving leading dictionary users quickly to what they need, w i t h n o distractions introduced by irrelevant information).

Juggling these t w o scenarios is the unenviable task o f the

lexicographer; understanding what's g o i n g on in their dictionaries and teaching dictionary users to understand this, and adapt t o it, falls to the language teacher.

Collaborative efforts in

dictionary u s e research are, w e believe, the w a y forward.

Bibliography Following many entries in the list below is the accession number (AN) in the Linguistics & Language Behavior Abstracts (LLBA), published by Sociological Abstracts Inc., San Diego, California. These have been included to allow readers with access to the LLBA (in electronic or print format) to consult the abstract there. Ard, J. (1982). The use of bilingual dictionaries by ESL students while writing. ITL, Review of Applied Linguistics, 58, 1-27. AN 8305457. Atkins, Β. T. S. (1996) Bilingual dictionaries: past, present and fiiture. EURALEX 1996 Proceedings, (eds.) M. Gellerstam et al. Göteborg: Department of Swedish, Göteborg University. H. Lewis, D. Summers & J. Whitcut (1987) A research project into the use of learners' dictionaries. In: The Dictionary and the Language Learner, (ed.) A. P. Cowie. Tübingen: Niemeyer, pp. 29-43. & F. Ε. Knowles (1990) Interim report on the EURALEX / AILA Research Project into Dictionary Use . In: T. M a g a y & J . Zigany (eds) Proceedings of BudaLex '88. Budapest: Akadömiai Kiadö. pp. 381-392 Aust, R., M-J. Kelley, & W. Roby (1993). The use of hyper-reference and conventional dictionaries. Educational Technology Research and Development, 41/4, 63-73. AN 9406084. Barone, R. (1979). On the use of the advanced learner's dictionary of current English. Rassegna Italiana di Linguistica Applicata, 11, 1-2. AN 8004840. Baxter, J. (1980). The dictionary and vocabulary behavior: A single word or a handful? TESOL Quarterly, 14/3, 325-336. AN 8205454. Bijoint, H. (1981). The foreign student's use of monolingual English dictionaries: A study of language needs and reference skills. Applied Linguistics 2/3, 207-222. Benbow, T., P. Carrington, G. Johannesen, F. Tompa, & E. Weiner (1990). Report on the New Oxfoid English Dictionary User Survey. International Journal of Lexicography, 3/1, 155-203. Bensoussan, M., D. Sim, & R. Weiss (1984). The effect of dictionary usage on EFL test performance compared with student and teacher attitudes and expectations. Reading in a Foreign Language, 2, 262-275. (1983). Dictionaries and tests of EFL comprehension. English Language Teaching Journal, 37/4, 341345. AN 8406030. Bialystok, E. (1983). Inferencing: Testing the "Hypothesis-Testing" hypothesis. In: H. W. Seliger & M.H. Long (eds.), Classroom oriented research in second language acquisition. Rowly, Mass: Newbury House. 104-123 Blok, H. (1994). Het woordenboek gebruiken: Een kunstje? De praktijk van het basisonderwijs. [Dictionary use: A piece of cake? The practice of instruction in elementary schools] Levende Talen, 490. 276-279. AN 9409758. Bogaaids, P. (1985). Een vergelijking van het gebruik van twee woordenboeken door leerlingen van het a.v.o. [A comparison in the use of two dictionaries by secondary-school students.] In G.E. Booij et al. (eds.), Coipusgebaseerde Woordanalvsc. Jaarboek 1985 Amsterdam: Vrije Universiteit. 17-19

18

Hulstijn & Atkins

(1988). A propos de l'usage du dictionnaire de langue dtrangire. [On the use of the foreign-language dictionary.] Cahiers de Lexicologie, 52/1, 131-152. AN 8903583. (1991). Dictionnaires pödagogiques et apprentissage du vocabulaire. [Pedagogical dictionaries anl vocabulary learning.] Cahiers de Lexicologie,59/2, 93-107. AN 9205416. (1992). French dictionary users and word frequency. In: EURALEX '92 Proceedings, (eds.) H. Tommola et at. Tampere: Department of Translation Studies, University of Tampere, . 51-59. (1994). Tuning the dictionary to the skills of intermediate learners. In G. Henrici & E. Zöfgen (eds.), Fremdsprachen Lehren und Lernen, vol. 23 (pp. 192-205. ). Tübingen: Gunter Narr. (1995). Dictionnaires et comprehension dcrite. [Dictionaries and reading comprehension], Cahiers de Lexicologie, 67, 37-53. (1996). Dictionaries for learners of English. International Journal of Lexicography, 9/4,277-320. Brito, E.V. (1992). Leitura de textos didaticos: Uma abordagem piagmatica [Reading didactic texts: A pragmatic approach], ESPecialist, 13/1, 93-102. AN 9308663. Carter. R. (1989) Review of LDOCE and COBUILD . International Journal of Lexicography, 2/1,30-43. Coviello, G. (1987). II dizionario oggi: Due gruppi di studenti messi a confronto su un "oggeto" molto discusso. [The dictionary today: two groups of students confront a much-debated issue.] Rassegna Italiana di Linguistica Applicata, 19/1, 109-129. AN 8706248. Cumming, G , S. Cropp, & R. Sussex (1994). On-line lexical resources for language learners: Assessment of some approaches to word definition. System, 22/3, 369-377. AN 9500387. De Andrada, M. A. F. (1992). Dicionarios: Mela, atalho ou mito? [Dictionaries: Goal, obstacle, or myth?] ESPecialist, 13/1, 65-74. AN 9308253. Diab, T. A. A. (1990). The role of dictionaries in ESP, with particular reference to student nurses at the University of Jordaa Dissertation University of Exeter. Dissertation abstracts International A: The Humanities and Social Sciences, 50/10, April, 3215-A. AN 9101695. Fillmore, Charles J. (1989) Two Dictionaries. International Journal of Lexicography, 2/1, 57-83. Fischer, U. (1994). Learning words from context and dictionaries: An experimental comparison. Applied Psycholinguistics, 15/4, 551-574. AN 9502373. Gallison, R. (1983). Images et usage du dictionnaire chez des etudiants (en langue) de niveau avance. Etudes de linguistique appliquee, 49, 5-88. Hartmann, R.R.K (1982) Das zweisprachige Wörterbuch im Fremdsprache nerwerb Studieren zur neuhochdeutschen Lexikographie II. (ed.) H.E. Wiegand. Germanistische Linguistik, 3-6/80. 73-86. Hildesheim: Georg Olms. (1983). The bilingual learner's dictionaiy and its uses. Multilingua, 2/4, 195-201. AN 8407034. (1987). Wozu Wörterbücher? Die Benutzungsforschung in der zweisprachigen Lexikographie. [Why dictionaries? Usage studies in bilingual lexicography.] Lebende Sprachen, 32/4, 154-156. AN 8804040. (1995). Pedagogical lexicography: some desiderata. In R. Dirven & J. Vanparys (eds ), Current approaches to the lexicon (pp. 405-411). Frankfurt am Main: Peter Lang. Hatherall, G. (1984). Studying dictionary use: some findings and proposals. In R.K.K. Hartmann (ed.), LEXeter83 Proceedings. Tübingen: Niemeyer. 183-189 Hausmann, F.J. & A. Gort>ahn( 1989) COBUILD and LDOCE II: A comparative review. International Journal of Lexicography, 2/1, pp. 44-56. Heibst,T. (1985). Von Fehlern, die vermeidbar waren. Ein weiteres Argument für mehr Wörterbücheraibeit im Englischunterricht. (Errors which could have been avoided: An additional argument for more instruction on dictionaiy woik in English) Bielefelder Beiträge zur Sprachlehrforschung 14, 1-2 & 236-248. AN 8707465. (1996) On the way to the perfect learners' dictionaiy: a first comparison of OALD5, LDOCE3, COBUILD2 and CIDE. International Journal of Lexicography, 9/4, 321-357. Hohne, S. (1991). Die Rolle des Wörterbuchs in der Sprachberatung. Eine Sekundaranalyse zur Wörteitmchbenutzungsforschung. [The role of the dictionary in linguistic consultation. A secondary analysis of dictionary use research.] Zeitschrift für Germanistische Linguistik, 19/3, 293-321. AN 9204187. Houtman, K., & G. Wouters (1994). "Een boek waar alles in Staat!" Gesprek met leerlingen over woordenboeken en een proef op de som. ("A book that's got everything in it!" A conversation with students about dictionaries and a test.) Levende Talen, 490, 306-310. AN 9410019. Huang, G. (1985). The productive use of EFL dictionaries. RELC Journal, 16, 54-71. AN 8700253. Hulstijn, J.H. (1993). When do foreign-language readers look up the meaning of unfamiliar words? The influence of task and learner variables. The Modern Language Journal, 77, 139-147. AN 9308491. (1997). Second-language acquisition research in the laboratory: possibilities and limitations. In: J.H. Hulstijn & R. DeKeyser (eds.), Testing SLA theory in the research lab. Studies in Second Language Acquisition, 19/2, 131-143. M. Hollander, & T. Greidanus (1996). Incidental vocabulary learning by advanced foreign-language students: The influence of marginal glosses, dictionaiy use. and reoccurrence of unfamiliar words. The Modern Language Journal, 80/3, 327-339. Ibrahim, A.H., & M. Zalessky (1989). Enquete: l'usage du dictionnaire. [Survey: The use of the dictionaiy.] Frangais dans le Monde, 29, Supplement, Aug-Sept, 24-30. AN 9001894.

Empirical Research on Dictionary Use

19

Jansen, J., J. P. Mergeai & J. Vanandroye (1987) Controlling LDOCE's controlled vocabulaiy. The Dictionaiy and the Language Learner, (ed.) A. P. Cowie. Tubingen: Niemeyer, 78-94. Kemerman, L. (1996). English learners' dictionaries: How much do we know about their use? In: EURALEX 1996 Proceedings, (eds.) M. Gellerstam, J. Jäiborg, S-G. Malmgren, Κ. ΝοΓέη, L. Rogström & C. Röjder Papmehl. Göteborg: Department of Swedish, Göteborg University. Kipfer, B. A. (1987). Dictionaries and the Intermediate Students : Communicative Needs and the Development of User Reference Skills. In: The Dictionary and the Language Learner (ed.)A. P. Cowie, Lexicographica Series Maior 17. Tübingen: Niemeyer, 44-54. Laufer, Β. (1992) Corpus-based as against lexicographer examples in comprehension and production of new words in EURALEX '92 Proceedings, (eds.) H. Tommola et al. Tampere: University of Tampere, 71-76. (1993) The effect of dictionaiy definitions and examples on the use and comprehension of new words. Cahiers de Lexicologie 63/2: 131-142. & L . Melamed (1994). Monolingual, bilingual and "bilingualized" dictionaries: Which are more effective, for what and for whom? EURALEX 1994 Proceedings, (eds.) W. Martin et al., Amsterdam: Vrije Universiteit, 565-576. Luppescu, S., & R. R. Day (1993). Reading, dictionaries, and vocabulary learning. Language Learning, 43/2, 263-287. AN 9308517. MacFarquhar, P. D. & J. C. Richards (1983). On dictionaries and definitions. RELC Journal, 14/1, 111-124. AN 8501547. MacGregor, S. K., & L. B. Thomas (1988). A computer-mediated text system to develop communication skills for hearing-impaired students. American Annals of the Deaf, 133/4,280-284. AN 8901902. Müllich, Η. (1990). "Die Definition ist blöd!" Herübersetzen mit dem einsprachigen Wörterbuch. Das fianzösische und das englische Lernwörteibuch in der Hand der deutschen Schüler ("That definition is weird!". Tübingen: Niemeyer. Meara, P. and F. English (1988) Lexical Errors and Learners' Dictionaries, ERIC ED 654 321, University College Swansea: Centre for Applied Language Studies Group. Nesi, H. (1994). The effect of language background and culture on productive dictionaiy use. EURALEX 1994 Proceedings, (eds.) W. Martin et al., Amsterdam: Vrije Universiteit 577-585. & P. Meara (1991). How using dictionaries affects performance in multiple-choice EFL tests. Reading in a Foreign Language, 8/1, 631-643. AN 9402515. & P. Meara (1994). Patterns of misinterpretation in the productive use of EFL dictionary definitions. System, 22/1, 1-15. AN 9407377. Neubauer, F. (1985). Auf der Spur des "unbekannten Wesens": der DaF Wörteibuchbenutzer. (On the track cf the "unknown creature": the German as a second language dictionary user.) Bielefelder Beiträge zur Spiachlehrforschung, 14/1-2,216-235. AN 8707473. NucCorini, S. (1992) Monitoring dictionaiy use. In EURALEX'92 Proceedings, (eds.) H. Tommola et al. Tampere: Univeisity of Tampere, 89-102. (1994). On dictionary misuse. EURALEX 1994 Proceedings, (eds.) W. Martin et al., Amsterdam: Vrije Universiteit, 586-597. Roby, W. B. (1993). Glosses and dictionaries in paper and computer formats as adjunct aids to the reading of Spanish texts by university students. Dissertation University of Kansas. Dissertation Abstracts International, A: The Humanities and Social Sciences, 53/9, March, 3182-A. AN 9400343. Summers, D. (1988). The role of dictionaries in language learning. Vocabulaiy and language learning, (eds.) R. Carter & M. McCarthy, London: Longman. 111-125. Taylor, Α., & Chan, A. (1994). Pocket electronic dictionaries and their use. In: EURALEX 1994 Proceedings, (eds.) W. Martin et al., Amsterdam: Vrije Universiteit. 598-606. Tomaszczyk, J. (1979). Dictionaries: Useis and uses. Glottodidactica, 12, 103-119. AN 8200476. Τοηο,Υ. (1987) Which Word Do You Look Up First? A study of Dictionaiy Reference Skills. Unpublished Μ Ed. thesis, Tokyo Gakugei University, [includes report on testing of EFL learners' dictionaiy tactics] (1988) Exploring the Cognitive Strategies of Dictionaiy Use: A Study of EFL Leamers'Idiom Look-Up Operations. Bulletin of the Kanto-Koshin-Etsu English Language Education Society, No. 2 January 1988. (1989) Can a dictionaiy help you read better? On the relationship between EFL learners' dictionaiy, reference skills and reading comprehension. Lexicographers and their Works (ed.) G. James, Exeter Linguistic Studies, 14, 192-200. (1991) A Good Dictionaiy User: What makes the difference? Recent Studies on English Language Teaching, (eds.) K. Ito, K. Kantatani & T. Nöda. Yumi Shobo. (1992) The Effect of Menus on EFL Learners' Look-up Processes LEXIKOS 2 Walberg, Η .J., & T. S. Ling (1985). Correlates of reading achievement and attitude: A national assessment study. Journal of Educational Research, 78/3, 159-167. AN 8802881.

Β. Τ. Sue Atkins & Krista

Varantola

Language Learners Using dictionaries: The Final Report on the EURALEX / AILA Research Project on Dictionary Use Abstract The project of which this is the final report was sponsored by EURALEX and the AILA Scientific Commission on Lexicology and Lexicography, and was carried out with the help of school and university teachers and over one thousand students in four European language areas. The interim report [Atkins & Knowles 1988] focussed principally on how the profile of the dictionaiy users (their L2 skills, their training or lack of it in dictionary use etc.) might be linked with their effective use of dictionaries: the principal points in that report have been incorporated into this paper. In addition, a closer examination has been made of the way the students used their dictionaries: what they found in the relevant entries, how they interpreted it, etc., in an attempt to discover the reason for some of the resultant errors. This ambitious project studied many aspects of dictionaiy use by many different types of language learners. Although the thousand-odd respondents were not a statistically representative group of the whole population of learners of English (how could such a population be defined?), they weie nonetheless typical of European students of English as L2. The objective of this paper is twofold: to make a complete record of the valid results of the investigations, and, equally important, to give a full and unvarnished account of the methodology used to elicit these results, for the benefit of others who may wish to use similar investigative measures.

1. The Project This is an account o f a detailed examination o f the w a y dictionaries were used by groups o f learners o f English as a foreign language, native speakers o f French, German, Italian and Spanish respectively. Since, as far as w e know, this w a s the first study on such a large scale, w e g i v e a rather full account 1 o f every aspect o f the project, w h o s e history is summarised in A p pendix 5, where w e also acknowledge the many colleagues without w h o s e help the work described here could not have been carried out.

1.1 Objectives The project set out to look at what foreign learners o f English actually d o w h e n they use a dictionary. W e hoped to find out something about aspects o f dictionary use o f interest both t o lexicographers and to language teachers wishing to instruct their students in dictionary skills including: •

h o w effective dictionaries are in helping students to carry out various (comprehension o f L2, translation into and out o f L2, self-expression in L2);

operations

•

whether bilingual and monolingual dictionaries are equally effective aids;

•

what attitude students have to these t w o types o f dictionary;

•

h o w much instruction is being given in the use o f dictionaries.

1

This paper is an attempt to set out in intelligible form as much as possible of the data collected, so that others who wish to investigate dictionary use may do so either by continuing the study of these data, or by selecting the effective parts of the methodology of the project and applying them in their own studies. Because the project was carried out on such a large scale, and consequently the data produced are so rich, we have restricted our objectives here to making as many of the facts as possible available to the reader. Some of the tables are extremely complex. We have tried to explain where we got the figures from, and how each table should be interpreted, and have kept our own commentary on the tables to a minimum.

22

Atkins & Varantola

We also intended this work to form a pilot for more detailed research into dictionary use by others who would build on our experience. This is perhaps the most important contribution that this project can make to the study of dictionary use, and we therefore try throughout this account to indicate honestly which parts of the methodology were inadequate to the task, as well as those which seemed to produce interesting results, in the hope that this information will be of use to other researchers in this field. 1.2 Scope It had originally been intended to study dictionary use over a number of different types of dictionaries, including those designed for native speakers of the various languages, but practical considerations restricted the research to the learning of English as a foreign language (EFL) by students of four language groups: French, German, Italian and Spanish. Within these limitations, however, we tried to cast our net as widely as possible, and the students who did the tests came from secondary schools, colleges, universities and adult education classes. 1600 sets of papers were distributed (400 in each language), and 1140 responses were eventually received. The respondents were each allocated a unique reference number, identifying the group to which they belonged, but the tests were performed anonymously. The groups came from educational establishments in Austria, Belgium, France, Germany, Italy, Spain and Switzerland. 1.3 Methodology No attempt was made to make the groups representative of the population of EFL learners world-wide, or even in Europe, since it was clearly impossible to define that population accurately enough to allow us to draw up criteria for a representative sample. In addition, the logistics of the project, even in its restricted form, were already extremely complex, without the added dimensions of representativity criteria, control groups, and so on. As a result, the general data may serve only as indications of possible trends in dictionary use, but we believe that these are valuable, and should offer some inspiration to subsequent researchers. Individual data, on the other hand, could be of immediate interest: these allow a study of the response by one individual to one specific question, knowing something about the user's L2 skills, whether he or she had been trained in dictionary use, whether or not a dictionary was used, and if one had been, often identifying the actual dictionary entry consulted. We include samples of this type of data in Section 5. The set of papers to be completed by each student consisted of three parts: a Dictionary User Profile Form (DUPF), a Placement Test (PT) which served to identify their L2 (English) skills, and a Dictionary Research Tests (DRT) consisting of questions to be answered with or without the use of a dictionary. In order to avoid distractions, it was decided that the metalanguage of the various questionnaires should be the respondent's native language: four sets of papers were therefore produced, in French, German, Italian, and Spanish respectively. Only the Placement Test was entirely in English. The original version, in English, of these papers is given in Appendices 1 and 2. A complete copy of the DUPF and DRT in French, German, Italian and/or Spanish may be obtained on application to EURALEX.2

2

Write, giving the full postal address to which the documentation should be sent and indicating the language of choice, to: EURALEX c/o Krista Varantola, Department of Translation Studies. University of Tampere, PO Box 607, 33101 Tampere. Finland or send email to [email protected].

Language Learners Using Dictionaries

23

1.3.1 Dictionary User Profile Form The DUPF (see Appendix 1) sought to establish for each respondent the current level and type of English studies, whether these took place in English, students' reasons for studying English, the amount of tuition received in dictionary skills, the dictionary or dictionaries they owned and the reasons for their purchase (the funding for the project came for the most part from dictionary publishers), and the respondent's experience of and attitude to bilingual and monolingual EFL dictionaries. 1.3 .2 Placement Test The Placement Test, devised by a British Council-approved Language School in London for the purpose of assigning new students to an appropriate class, consisted of 100 questions in English, mainly multiple-choice; it had to be completed in English, within one hour, supervised by the class teacher. The test was manually corrected and one of the following grades allocated to each student: • A: 81 -100% correct answers; • Β : 66-80% correct answers; • C : 51 -65% correct answers; • D : 0-50% correct answers. Cross comparisons with the results of the dictionary research tests appear to confirm that the placement test was quite a good assessment of students' knowledge of English. 1.3.3 Dictionary Research Tests With the exception of the translation passage, which was different for each language group, the questions were the same for all students. The questions were grouped according to the linguistic process or aspect of dictionary skills they were designed to test, as follows: 3 Q-l tested students' understanding of parts of speech and their English names Q-2 tested students' understanding of the grammatical metalanguage used in learners' dictionaries Q-3 tested students' ability to select from a list the correct item to fill slots in elementary-level contexts Q-4 discovered students' expectations of where in their dictionary multiword expressions (compound noun, verbal idiom and prepositional phrasal verb) are likely to be located; Q-5 tested students' ability to select from a list the correct prepositional complement of various English words Q-6 tested student's comprehension of a passage in English Q-7 tested translation from the student's language into English Q-8 tested student's ability to find a suitable item to fill slots in quite difficult contexts Q-9 tested students' ability to select from a list the correct item to fill slots in quite difficult contexts The aim in these tests was to replicate as far as possible the natural use of the learners' dictionary. For that reason, respondents were invited to consult when necessary their customary dictionary and to say exactly which dictionary that was. It is not always possible to tell from the replies precisely which version (standard, concise etc.) or which edition of a specific 3

Here, as throughout the text. Q - l (etc.) should be read as Question 1 (etc.).

Atkins & Varantola

24

dictionary was used, but in many cases this is identifiable; after each question, the student was asked to indicate whether or not the dictionary had been consulted in that instance.

1.4 Results All surveys strive for a high return rate, and this particular project can be reasonably well satisfied with its performance: 1140 returns were received out of the 1600 questionnaires distributed. However, the latter included 400 sets destined to form a control group of respondents doing the tests without any dictionary. Of the 1140 responses received, 297 came from that group. It became apparent, when we came to evaluating the tests and reporting on the research, that these responses were of little value, and in fact distorted the results: if they were included in the database it became necessary to formulate extremely complex queries in order to discover whether a respondent had made a conscious decision to use his or her dictionary, or whether that respondent belonged to the control group. The 297 control-group responses were therefore removed from the database, leaving 843 responses to be investigated. However, this number was further reduced. The operation was a tripartite one (Dictionary User Profile Form, Placement Test and Dictionary Research Tests), and from these 843 responses the number of complete triples1 returned was 723. These constituted our definitive dataset. All the figures in this final report are based on the dataset of 723 respondents, as are those in the previously distributed electronic version of the database. 4 Moreover, this database has been further slimmed down, some of the questions being omitted as misleading or badly thought out. This is discussed more fully later, when the questions are considered in more detail.

2. The Data In this section, the reader is introduced to the database. We give some idea of the types of information it contains, and show some samples of the facts that may be extracted, describing first the data collected by the Dictionary User Profile Form, then those given in response to the Dictionary Research Tests. The data discussed here were produced by students whose L2 skills were assessed according to their performance in the Placement Test (see Section 1.3.2). The distribution is shown in Table 1. No further direct reference will be made to the Placement Test itself, but the classification of the students into Grades A (most advanced) to D (least skilled) will be used throughout the analysis of the main data, collected by the DUPF and the DRT, and stored in the merged 723-dataset database. L2 skills level

Total number of students

Grade A

131

18%

Grade Β

216

30%

Grade C

195

27%

Grade D

181

25%

Total

723

100%

Table 1:

Distribution of grades across respondents

The remainder of this section is devoted to a survey of the kind of information collected by the DUPF (see Appendix 1) and the DRT (see Appendix 2). 4

Some of the figures here may therefore differ in some minor respects from the preliminary results reported in Atkins & Knowles (1988).

Language Learners Using Dictionaries

25

2.1 Dictionary User Profiles Information about the dictionary user respondents is drawn from their responses to the DUPF, their Placement Test results and their answers to the first two questions of the DRT. The following definite facts were known about respondents (the relevant question numbers are given in brackets in the list below): • their country of origin and native language (DUPF Q-l) • the number of years they had studied English (DUPF Q-2) • type of educational institution(s) where they had studied (DUPF Q-3) • the purpose of their current English studies (DUPF Q-4, Q-8) • whether or not their classes were held in English (DUPF Q-5) • textbooks used (DUPF Q-6) • whether they had received instruction in dictionary use (DUPF Q-7) • details of their dictionary ownership (DUPF Q-9-11) • their habits of dictionary use (DUPF Q-12-17) • their understanding of English parts of speech (DRT Q-l) • their understanding of the grammar metalanguage used in English learners' monolingual and bilingual dictionaries (DRT Q-2). 2.1.1 Sample facts Some of the known information is given in tabular form in this section; more is distributed where relevant in tables throughout the paper. The facts given in Tables 1-4 inclusive relate to the profiles of the respondents and are of no intrinsic interest in themselves. They are included here in order to give the reader some idea of the internal structure of the database.5 In later tables, of course, the native language of the respondents, their L2 skills and whether or not they were taught how to use a dictionary make positive contributions to the interpretation of the data. Tables 2 and 3 show the distribution of languages and L2 skills among the respondents. French speakers Grade A

10

German speakers

Italian speakers

Spanish speakers

Total

8%

9

8%

43

21%

69

26%

131

Grade Β

28

21%

64

55%

46

22%

78

29%

216

Grade C

54

41%

37

32%

50

24%

54

20%

195

Grade D

39

30%

7

6%

66

32%

69

26%

181

Total

131

100%

117

-100%

205

-100%

270

-100%

723

Table 2: Distribution of L2 skills within the various language groups Table 2 tells us how the various language groups fared in the Placement Test, showing (for instance) that the Italians and Spanish were fairly evenly divided among the groups, whereas most of the German speakers, coming from the younger classes in high school, tended to bunch together in Grades Β and C. From Table 2 also, we see the differences in size of the language groups, the Spanish (for instance) with 270 members, being more than twice the size of the German and the French groups, while the Italians lie somewhere in between. 5

Among those who funded the project were a number of dictionary publishers, to whom the answers to questions about the age and educational level of the respondents', and their choice and ownership of dictionaries were of particular interest.

Atkins & Varantola

26

From Table 3, we can see how the various L2 skills were distributed among the language groups. It shows, for instance, that most of the advanced respondents (those in Grades A and B) were Spanish or Italian speakers, while the Grade C group was fairly evenly divided among the four languages and - with the exception of the German respondents - a similar picture is drawn of the least skilled (Grade D) group. Grade A

Grade Β

Grade C

Grade D

Total

French

10

8%

28

13%

54

28%

39

22%

131

German

9

7%

64

30%

37

19%

7

4%

117

Italian

43

33%

46

21%

50

26%

66

36%

205

Spanish

69

53%

78

36%

54

28%

69

38%

270

Total

131

-100%

216

100%

195

-100%

181

100%

723

Table 3: Distribution of languages within various grades Table 4 summarises the information gathered about the number of years of English studies of the various language groups (see Appendix 1, DUPF Q-2). It explains some of the facts in Table 3, showing that the Spanish-speaking respondents, who constituted the largest proportion of the Grade A group, also constituted the bulk of those who had been studying English for the longest period (10 years or over). French speakers

Length of English Studies

German speakers

Italian speakers

Spanish speakers

0-4 years

8

6%

15

13%

51

25%

52

19%

5-9 years

115

88%

98

84%

120

58%

131

49%

104-years

8

6%

4

3%

34

17%

87

32%

Total

131

100%

117

100%

205

100%

270

100%

Table 4: Distribution of languages and years of English studies among respondents However, care must be taken in the analysis of the data in Table 4, where the responses to DUPF Q-2 have been grouped under three heads: "0-4 years", "5-9 years" and "10+ years". It would be easy to draw false conclusions from the fact that 84% of the German and 88% of the French speakers had studied English for "5-9 years" (see Table 4), yet while 63% of the German speakers (8% + 55%: see Table 2) were graded as A and B, only 29% (8%+21%) of the French speakers were so graded. This type of contrastive analysis of the material in this database is not valid. "5-9 years" is far too broad a band to be of interest to that type of conclusion. Much finer-grained information is required, not only about the exact number of years of English study of the members of these groups, but also about the kind of institution in which the study was carried out, the number of hours per week devoted to it, and so on. The database does contain all of this information, but because the groups themselves were not constituted on any representative basis, 6 it is not interesting to go further down this road.

6

We believed that the interest in this project would lie in the breadth of its scope and the detailed nature cf its findings: adding representativity of the respondents to this list proved impossible. Simply finding enough language teachers willing to cany out the quite onerous task of administering the tests in their classroom was the best we could do.

Language Learners Using Dictionaries

27

2.1.2 Instruction in the Use of Dictionaries One fact, however, is of general interest: the proportion of the respondents who had received some teaching in how to use their dictionaries. The database records that 60% of the student respondents (answering DUPF Q-7) claimed to have received no instruction whatsoever in the use of dictionaries, 25% said they had received some instruction, and only 14% reported that they had received "precise and systematic" instruction in how to use their dictionaries. It is difficult to reconcile this result with the fact that these were students whose teachers were so interested in dictionaries that they agreed to spend considerable time and energy on the administering of the questionnaires. Instruction in dictionary use

German speakers

French speakers

Italian speakers

Spanish speakers

Total

none

105

80%

58

50%

90

44%

184

68%

437

60%

some

19

14.5%

37

32%

60

29%

65

24%

181

25%

systematic

7

5%

21

18%

54

26%

18

7%

100

14%

unspecified

-

-

1

1%

1

-

3

1%

5

1%

Total

131

-100%

117

-100%

205

-100%

270

100%

723

100%

Table 5: Training in dictionary use according to language groups Table 5 shows how the various language groups fared in dictionary instruction and may usefully be combined with Table 4: 80% of the French speakers, for instance, had never been taught how to use a dictionary, although (as Table 4 shows) 88% of them had had between five and nine years of English teaching. A similar proportion of the German speakers (84%) had learnt English for 5-9 years, but of them 50% had been taught how to use a dictionary. The implications of Tables 4 and 5, taken together, are that dictionary skills figure on the timetable in more Italian and German language classrooms than in Spanish or French classrooms. 2.2 Dictionary Use Tasks The questions in the Dictionary Research Tests (DRT) are not homogeneous: this section concerns itself with those which asked the students to perform a particular linguistic task. Some questions do not match this criterion. Questions 1 and 2, as we have seen, belong rather with those in the DUPF whose purpose was to build a profile of each individual respondent. Question 4 was an attempt to find out where the users expect to find multiword expressions in their dictionaries and will be discussed in Section 3. Of the others, most yielded useful information, with the exception of Question 8, which was in fact not a valid set of multiple choices. Correction of Q-8 involved subjective decisions on the part of the person doing the correcting, and the whole of Question 8 was therefore excluded from consideration. The remaining questions in the DRT (3, 5, 6, 7 and 9) constituted the basis of our database. However, some parts of these questions, on closer inspection, appeared unreliable, and were discarded before the database was finalised. These were: QQ-3/12,3/13 and 3/14: because the relevant entries in most of the dictionaries did not contain the information necessary to answer them correctly; also because these

28

Atkins & Varantofo

questions were far too complex for the majority (perhaps all) of the respondents, demanding subtle dictionary strategies and a very sophisticated knowledge of English.7 Q-9/40: where sociable, social and socialist are all possible in the given context, although sociable, the most acceptable in standard English, was the one intended to be accepted as the correct answer. 2.2.1 Typology of the DRT questions All of the remaining 20 DRT questions (the "active questions"), the results of which, for the 723 respondents, constitute the database, involve tasks where the use of a dictionary can help the learner. However, they fall into two types: those where it is possible to decide which dictionary entry the student will have consulted ("Type A"), and those where it is impossible to say that any specific dictionary entry will be consulted by the student ("Type B"). 2.2.1.1 Type A questions (specific dictionary entry relevant) The Type A questions, for which it is possible to identify the relevant dictionary entries, are the following: Q-3/11 Supply the mi ssi ng verb i η: She decided to a photograph of the mountain. Q-5/19 Supply the missing preposition in: I am surprised you. Q-5/21 Supply the missing preposition in: I haven't muchfaith.... what he says. Q-5/22 Supply the missing preposition in. There is no danger.... being interrupted. Q-5.23 Supply the missing preposition in: It's not good ....you to eat so much. Q-9/43 Supply the missing compound noun in: In London the.... is very high. Section 5 is devoted to a detailed discussion of some of the results relating to these Type A questions, and Table 18 shows how the six most popular dictionaries (listed in "Dictionaries Cited" at the end of this paper) handle the item on which each question focuses. 2.2.1.2 Type Β questions (no specific dictionary entry relevant) The following constitute the Type Β questions, for which students did consult their dictionaries, often with favourable results, but for which it is impossible to pinpoint with confidence the entry which would have been consulted in each case: Q5/20 Supply the missing preposition in: The man... white hair was standing by the door. Q6/24 , Q6/25, Q6/26, Q6/27 and Q6/28 Comprehension questions relating to a given passage in English. Q7/29, Q7/30, Q7/31, Q7/32 and Q7/33 Multiple choice options on the correct L2 expression to fill gaps in a given translation of a given LI passage.

7

In fact they were calqued on the kind of learning practice question that is often included in "Leant to Use Your Dictionaiy" booklets, which are intended to be used with teacher guidance in the classroom (see Section 6).

Language Learners Using Dictionaries Q9/41, Q9/42, and Q9/44 Select the correct lexical item to fit the slot in a sentence: e.g. In the height of the

29

summer

Venice is... with tourists. (The choice from: full, inhabited, running, swarming). 3. Discussion of the data 1: Location of multiword expressions This relates to Question 4 of the D R T (see Appendix 2), in which the respondent is asked t o say where in the dictionary he or she would expect to find three different types of idiomatic multiword expressions (MWEs):

• compound noun • verbal idiom

(lame duck) (to split hairs).

• verb + particle (to do without) The fourth item in Question 4 (the plural noun greens, in the sense of "green vegetables") gave rise to too much confusion among the respondents, and was discarded from the database. We wished to see whether the student's expectation of where to find an M W E in the dictionary was affected by their native language, their L2 skills, or the fact that they had been trained in the use of dictionaries, and how their expectations compared with current dictionary practice. W e expected that this information might be of interest to dictionary editors, who, for every new dictionary, have to define the editorial policy regarding the location of M W E s . Every dictionary differs from almost every other in this respect: our sample dictionaries certainly differ from other dictionaries of similar type. N o dictionaries are wholly consistent in this regard (some of those w e surveyed randomly included some of the phrases twice). Experience teaches that it is indeed impossible for editors of a new dictionary to formulate criteria which will allow the lexicographers who compile it to identify every item which should be considered as a MWE. 8 Thus these items cannot be handled consistently throughout the dictionary text. The tables in this section (Tables 6 - 8 inclusive) all include an indication of where the six most popular dictionaries (those named in "Dictionaries Cited" at the end of this paper) locate the specific M W E , and are of interest only insofar as they might suggest trends: much more research (of the type described in Bogaards 1990 and 1992) is needed if dictionary editors are to receive any real guidance on this matter.

3.1 Expectations of the various language groups Table 6 shows where the student users, grouped according to their native language, would start looking for the M W E s . It should be noted here that the respondents knew that these were idiomatic expressions: this is of course not always the case when people come across an idiom in their reading and use the dictionary to find out the meaning of the phrase. The numbers which follow some of the dictionary-name abbreviations in Table 6 indicate that the item w a s included twice in that particular dictionary. Table 6 shows no strong tendencies which may be related to the languages of the respondents, despite the myth current among bilingual lexicographers that German speakers expect verb+noun phrases such as to split hairs to be located under the noun. What is striking and g Researchers working on machine-readable dictionaries have shown up the inconsistencies, but so far no theoretical linguist or theoretical lexicographer has come up with criteria which could improve the lexicographical practice. In an ideal world, the inconsistencies could be minimised by checking all the data tagged as MWEs in the completed dictionary, and ironing out anomalies before publication. In the real world, adding a year or two to a dictionary's budget is out of the question.

30

Atkins & Varantola

disturbing in Table 6 is that most of the respondents are unaware of how EFL dictionaries 9 handle verb+particle MWEs (as phrasal verbs which are often secondary headwords). The inevitable conclusion is that most dictionary users find information about phrasal verbs only after at least one abortive look-up. In the entiy for...

French speakers

German speakers

Italian speakers

Spanish speakers

What the dictionaries do

lame

58 : 44%

47 : 40%

97 : 47%

132 : 49%

ZNRD-1

duck

56 : 43%

54 : 46%

91 : 44%

113 : 42%

CSD HNSFD OALD ZNRD-2

lame duck

12 : 9%

12 : 10%

13 : 6%

12 : 4%

LDOCE

don't know

5 : 4%

4 : 3%

4 : 2%

13 : 5%

[not in LTE]

131 : 100%

117 : 100%

205 : 100%

270 : 100%

split

89 : 68%

73 : 62%

151 : 74%

226 : 84%

CSD OALD-1 LTE ZNRD-1

hair

31 : 24%

31 : 26%

37 : 18%

21 : 8%

HNSFD LDOCE OALD-2 ZNRD-2

split hairs

7 : 5%

11 : 9%

11 : 5%

11 : 4%

—

4 : 3%

2 : 1%

6 : 3%

12 : 4%

131 : 100%

117 : 100%

205 : 100%

270 : 100%

do

63 : 48%

65 : 56%

137 : 67%

197 : 73%

CSD LTE HNSFD-1 ZNRD-1

without

61 : 47%

45 : 38%

61 : 30%

61 : 23%

HNSFD-2 LDOCE-1 ZNRD-2

do without

2 : 1%

4 : 3%

5 : 2%

6 : 2%

LDOCE-2 OALD

don't know

5 : 4%

3 : 3%

2 : 1%

6 : 2%

131 : 100%

117 : 100%

205 : 100%

270 : 100%

don't know

Table 6: Expectations of students according to language groups compared with location of MWEs in dictionaries 3.2 Expectations of students in L2-skills groupings Table 7 sets out the students' expectations according to their L2 skills, and contrasts these with the actual location of the MWEs in the six dictionaries. The do without results shown in Table 7 imply that no students - not even those most advanced in English language skills - were aware of the way in which phrasal verb entries or subentries are handled by the leading EFL dictionaries. There is however a certain consensus among Grades A and Β on where they search for certain types of MWE (particularly for to split hairs and to do without), which may be ascribed to their lengthier experience of dictionary use, during which they have perhaps developed more effective strategies for finding items in dictionaries. They probably have a better instinct about varying lengths of entries, and are better able to select the shortest relevant entry. Alone

9

As well as OALD and LDOCE, the olher two major EFL dictionaries, the Cambridge International Dictionary of English and the Collins Cobuild English Dictionary make secondary headwords of phrasal veibs.

Language Learners Using Dictionaries

31

In the entry for...

Grade A students

Grade Β students

Grade C students

Grade D students

What the dictionaries do

lame

53 :40%

98 : 45%

101 : 52%

82 :45%

ZNRD-1

duck

73 : 56%

104 : 48%

68 : 35%

69 : 38%

CSD HNSFD OALD ZNRD-2

lame duck

5 : 4%

9 : 4%

15 : 8%

20 : 11%

LDOCE [not in LTE]

don't know

5 : 2%

11 : 6%

10 : 5%

131 : 100%

216 : 100%

195 : 100%

181 : 100%

split

106 : 81%

171 : 79%

142 : 73%

120 : 66%

CSD LTE OALD-1 ZNRD-1

hair

23 : 18%

30 : 14%

35 : 18%

32 : 18%

HNSFD LDOCE OALD-2 ZNRD-2

split hairs

1 : 1%

10 : 5%

11 : 6%

18 : 10%

don't know

-

1 : 1%

5 : 2%

7 : 4%

11 : 6%

131 : 100%

216 : 100%

195 : 100%

181 : 100%

do

109 : 83%

158 : 73%

119 : 61%

76 : 42%

CSD LTE HNSFD-1 ZNRD-1

without

20 : 15%

48 : 22%

71 : 36%

89 : 49%

HNSFD-2 LDOCE-1 ZNRD-2

do without

2 : 1%

5 : 2%

3 : 1%

7 : 4%

LDOCE-2 OALD

don't know

-

5 : 2%

2 : 1%

9 : 5%

131 : 100%

216 : 100%

195 : 100%

181 : 100%

Table 7: Expectations of students according to L2 skills compared with location of MWEs in dictionaries among the groups, most Grade As (56%) picked the location for lame duck favoured by most dictionaries. It is difficult to believe that this is caused by any factor other than sheer chance, for the fact which comes out most strongly from these listings is that the dictionaries themselves seem to have no clear policy over the inclusion and location of these MWEs. In Table 7, as in Table 6, we see that several dictionaries carry the items in two places (ZNRD in fact includes all three of our test MWEs in two places). The case of to split hairs points up the fact that, while the dictionaries do not agree on where to locate this MWE, there is quite considerable agreement amongst the students, who virtually regardless of L2 skills - expect to find this in the entry for the verb split. This raises the question of whether dictionary users might benefit from an attempt on the part of dictionary publishers to discover some systematicity in user expectations, and (if they find any) to make their treatment of MWEs conform as far as possible to user expectations. At the moment, every dictionary has its own approach to this problem; MWEs may be located, for instance, at the first immutable lexical item (as is the case in the Collins-Robert English-French Dictionary), or at the least frequent, and therefore probably least familiar to the user, lexical item (as is the case in the Oxford-Hachette English-French Dictionary): the list and strategies are endless. This is one aspect of lexicography which the authors believe will never be fully systematised: an interesting attempt to help the EFL user find idiomatic MWEs is made by the new International Dictionary of English , which, for those with extremely good eyesight, contains an index of the MWEs in the dictionary.

32

Atkins & Varantola

3.3 Expectations of students with and without dictionary use training It might be postulated that students w h o have been taught h o w to use dictionaries would be more successful in second-guessing their dictionaries on the location of M W E s . Table 8 shows what the position is, and here once again the fact that the dictionaries d o not themselves agree on the most appropriate location for the different types of M W E s , nor show any internal consistency in this regard, makes it impossible for a user to rely on a policy-based approach. In the entry for...

Of 281 with dictionaiy use training

Of 437 without dictionaiy use training

Of 5 with unspecified What the dictionaries dictionary use do training

lame

139 : 49%

194 : 44%

1 :20%

ZNRD-1

duck

116 : 41%

195 : 45%

3 :60%

CSD HNSFD OALD ZNRD-2

lame duck

16 : 6%

32 : 7%

1 : 20%

LDOCE

don't know

10 : 4%

16 : 4%

-

[not in LTE]

281 : 100%

437 : 100%

5 : 100%

split

200 : 71%

336 : 77%

3 : 60%

CSD LTE OALD-1 ZNRD-1

hair

45 : 16%

73 : 17%

2 : 40%

HNSFD LDOCE OALD-2 ZNRD-2

split hairs

21 : 7%

19 : 4%

don't know

-

15 : 5%

9 : 2%

281 : 100%

437 : 100%

5 : 100%

do

160 : 57%

298 : 68%

4 : 80%

CSD LTE HNSFD-1 ZNRD-1

without

104 : 37%

123 : 28%

1 : 20%

HNSFD-2 LDOCE1 ZNRD-2

do without

10 : 4%

7 : 2%

don't know

7 : 2%

9 : 2%

281 : 100%

437 : 100%

-

-

LDOCE-2 OALD

-

5 : 100%

Table 8: Expectations of students according to whether or not they have had instruction in dictionary use, compared with location of M W E s in dictionaries The expectations of those trained in dictionary use and of those untrained in dictionary use seem very much alike. (Figures for the very small group who did not specify their dictionary use training are included for the sake of completeness.)

4. Discussion of the data 2: Dictionary use in general In Section 2.2.1, we distinguished between Type A questions (those for which one identifiable dictionary entry is relevant) and Type Β questions (those for which w e cannot identify one specific dictionary entry which students would have consulted). Both Type A and T y p e Β questions are relevant to the discussion in Section 4: that is to say, for each question discussed in this section, we know that the dictionary is relevant, but we cannot always say which entry

33

Language Learners Using Dictionaries

is relevant. The scope here is therefore over the twenty DRT questions, for the usual 723 respondents in the database. Of the 723 respondents, 26% never once tried to look anything up in their dictionaries. It is of course possible that a more detailed analysis of the figures could be affected by a small group of students behaving atypically and looking up a dictionary for every possible question. This does not appear to be the case. The 74% who used the dictionary at least once, in answering the twenty DRT questions, constitute a database of 538 respondents who amongst them made 2278 look-ups.10 The average number of look-ups per student is thus 4.23, but of course the question arises whether a small group of atypical students looked up the dictionary often enough to distort results. This proved not to be the case. If those who made 11 or more look-ups are removed from the equation, the average number of look-ups per student is 4.16 (as against the average for the whole database of 4.23 look-ups per student). 4.1 Deciding to consult the dictionary The students' approach to dictionary use varied according to their command of L2, with those knowing less English having recourse more often to their dictionaries, as the figures show: 68% of those scoring Grade A or Grade Β in the Placement Test used the dictionary at least once, while 81% of Grade C and Grade D students did so ('A', it will be remembered, is the highest score, and 'D' the lowest). The statistics discussed in the preceding paragraph show no distortion of any group by atypical dictionary use: the fourteen respondents who made 11 or more look-ups each are fairly evenly distributed across the grade groups. In this section, therefore, we shall keep separate, for the most part, the figures relating to the use of their dictionaries by these four different groups of students. In most cases the grading corresponds roughly to the length of time the student had been studying English. 4.1.1 How often did they choose to consult the dictionary? Table 9 sets out, against each of the 20 "active" DRT questions, the number of students from each group who chose to use a dictionary when answering the question. The leftmost column of the first row, for instance, shows that of the 131 Grade A students, only one used his or her dictionary to answer Question 3/11 (and therefore 130 did not). The information in Table 9 (where asterisks indicate the Type A questions) should be interpreted as follows: • Out of 2620 questions answered by Grade A students: 11.5% (301) were answered with the help of the dictionary (and therefore 88.5% were answered without its help). • Out of 4320 questions answered by Grade Β students: 14% (600) were answered with the help of the dictionary (and therefore 86% were answered without its help). • Out of 3900 questions answered by Grade C students: 19% (743) were answered with the help of the dictionary (and therefore 81% were answered without its help). • Out of 3620 questions answered by Grade D students: 17.5% (634) were answered with the help of the dictionary (and therefore 82.5% were answered without its help). 10

Of those who used a dictionary at least once, the distribution of look-ups per respondent was as follows: Number of look-ups by number of students Total look-ups

1

2

3

4

5

6

7

8

9

10

11

92

84

80 65

65

53

26

23

21

15

8

92 168 240 260 325 318 182 184 189 150 88

12

13

14

15

16

1

2

2

0

1

12

26

28

0

16

34

Atkins & Varantola Question

of lookups by 131 students Grade A

of lookups by 216 students Grade Β

of lookups by 195 students Grade C

of lookups by 181 students Grade D

3/11*

1

7

15

25

5/19*

39

55

40

27

5/20

6

4

9

10

5/21*

23

57

75

66

5/22*

15

15

30

27

5/23*

2

5

11

9

6/24

7

40

69

50

6/25

3

31

41

34

6/26

4

16

35

46

6/27

10

28

32

32

.6/28

28

54

63

45

7/29

7

27

24

26

7/30

8

21

19

19

7/31

15

17

23

24

7/32

9

30

41

33

7/33

0

0

0

0

9/41

48

59

61

42

9/42

36

46

53

43

9/43*

22

33

31

26

9/44

18

55

71

50

Totals possible actual actual %

2620 301 11.5%

4320 600 14%

3900 743 19%

3620 634 17.5%

Table 9: Number of look-ups using dictionaries across the 20 dictionary-relevant questions according to L2 skills of students From the information given in Table 9, we see that, out of 14460 (2620 + 4320 + 3900 + 3620) possible points where a dictionary might have been consulted, a consultation took place in 2278 cases, or on 16% of the possible occasions. The number of times one turns to the dictionary obviously depends on h o w the question challenges one's L2 knowledge. Clearly, many of the respondents believed they knew the answer to the question without the help of their dictionaries (or did not care). That said, the hypothesis that the more advanced student will make fewer dictionary consultations than the intermediate or the beginner seems to be confirmed by Table 9, with the exception of Grades C and D, where the dictionary consultation rate was very similar. As may be seen from this table, there is considerable variation in dictionary consultation not only according to the question involved but also according to the level of L2 skills of the students. W e shall take a closer look at some of these disparities in Section 5.

Language Learners Using

35

Dictionaries

Appendix 3 {Listings of responses to six questions) exists in order to make this final report as complete as possible. It contains all relevant data from the database which has to do with dictionary consultation for the Type A questions and the students using one of the six popular dictionaries. From the Appendix 3 tables (26-31 inclusive), it is possible to discover the result of every consultation of any of these six dictionaries, and in each case to know the L2 level and native language of the student, the actual dictionary consulted, and the result of the consultation (or of the decision not to use the dictionary at that point). The listings in Appendix 3 show not only whether the answer given in each case was right or wrong, but exactly which of the possible wrong answers was selected. Appendix 4 contains the actual dictionary entries which the students will have consulted for each of the look-ups. 4.1.2 Did instruction in dictionary use make students more ready to use dictionaries? It might be hypothesised that a student who has been taught how to use a dictionary will more readily reach for the dictionary than a student who has had no training in dictionary skills. This is the question which Table 10 addresses. ALL STUDENTS 131 students Grade A

216 students Grade Β

195 students Grade C

181 students Grade D

723 students in all

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

301 11.5%

2620 100%

600 14%

4320 100%

743 19%

3900 100%

634 17.5%

3620 100%

2278 16%

14460 100%

THOSE TRAINED IN DICTIONARY USE 37 students Grade A

79 students Grade Β

79 students Grade C

86 students Grade D

281 students in all

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

86 12%

740 100%

232 15%

1580 100%

269 17%

1580 100%

293 17%

1720 100%

880 16%

5620 100%

THOSE NOT TRAINED IN DICTIONARY USE 94 students Grade A

137 students Grade Β

116 students Grade C

95 students Grade D

442 students in all

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

lookups using diet

out of total lookups

215 11%

1880 100%

368 13%

2740 100%

474 20%

2320 100%

341 18%

1900 100%

1398 16%

8840 100%

Table 10: Decisions on dictionary use or not according to L2 skills of student and instruction received in dictionary use

36

Atkins & Varantola

The "total look-ups" figures in the "all students" section in Table 10 repeat those in the bottom row of Table 9 and relate to each complete group of respondents. Table 10 should be read as follows (the example is taken working down the Grade A column): • Out of the 2620 questions answered by the 131 Grade A students: in 11.5 % of the cases, when a decision had to be made about whether or not to use the dictionary, they chose to consult their dictionary (and therefore in 88.5% of the cases they chose not to do so). • Out of the 740 questions answered by the 37 Grade A students who had been trained in dictionary use: in 12% of all cases they chose to consult their dictionary (and therefore in 88% of the cases they chose not to do so). • Out of the 1880 questions answered by the 94 Grade A students who had not been trained in dictionary use. in 11% of all cases, they chose to consult their dictionary (and therefore in 89% of the cases they chose not to do so). These proportions are virtually repeated in the other three columns (for Grades B, C and D), and Table 10 therefore shows that training in dictionary use apparently made little difference when it came to deciding whether or not to consult one's dictionary. The factor which does seem to affect the dictionary consultation rate is the L2 level of the student: as noted in Table 9 and as might be expected, the less advanced group (Grades C and D) show a higher rate of consultation than the more advanced (Grades A and B). 4.1.3 Did the type of task affect dictionary consultation? It would be reasonable to suppose that, as well as the level of the user's L2 knowledge, the various types of task might give rise to differences in dictionary consultation rates. Table 11, which relates only to cases where a dictionary was actually used by the respondent (the 20 "active" questions), shows the distribution of monolingual and bilingual dictionary consultation in conjunction with five of the principal tasks for which dictionaries are used, namely: • Support verb selection (Q-3) • Headword prepositional complementation (Q-5) • L2 reading comprehension (Q-6) • L1->L2 translation (Q-7) • Selecting the right word for the context slot (Q-9) Table 11 is complex, and may be explained as follows: the second composite row (for instance) relates to Question 5 of the DRT, which contained five individual questions (see Appendix 2), all amongst the "active" questions for which data are held. Of the 131 Grade A students, 53 were using L2 monolingual dictionaries, with the result that for the whole of Question 5, if they had all consulted their dictionaries on every occasion, the number of dictionary consultations for this group would have been 265 (5 χ 53: the "possible" figure). However, dictionaries were consulted in only 34 cases (the "actual" figure), i.e. in 13% (the "% actual" figure) of the maximum possible. The Q-5 composite row may therefore be read as follows: • For the whole of Question 5 (supplying the prepositional complementation of given words in given contexts), in 13% of possible cases Grade A students using L2 monolingual dictionaries actually consulted their dictionaries, and the same proportion (13%) of Grade A students using bilingual dictionaries consulted their dictionaries. • In 11% of possible cases, Grade Β students using L2 monolingual dictionaries consulted their dictionaries when answering the five questions of Question 5, while 13% of Grade Β students using bilingual dictionaries did so.

37

Language Learners Using Dictionaries

• In 12% of possible cases, Grade C students using their dictionaries when answering the five questions students using bilingual dictionaries did so. • In 16% of possible cases, Grade D students using their dictionaries when answering the five questions students using bilingual dictionaries did so. 131 students Grade A Lookups

53 with mono

78 with biling

216 students Grade Β 45 with mono

171 with biling

L2 monolingual dictionaries consulted of Question 5, while 18% of Grade C L2 monolingual dictionaries consulted of Question 5, while 15% of Grade D

195 students Grade C

181 students Grade D

37 with mono

158 with biling

22 with mono

159 with biling

37 2

(5%)

158 13 8%

22 5 22%

159 20 13%

Q3: Selecting support verb for context slot: over 1 question possible actual % actual

53 -

78 1 (1%)

45 1 (2%)

171 6 4%

Q5: Selecting preposition for context slot: over 5 questions possible actual % actual

265 34 13%

390 51 13%

225 24 11%

855 112 13%

185 22 12%

790 143 18%

110 18 16%

795 121 15%

225 14 6%

855 155 18%

185 46 25%

790 194 25%

110 22 20%

795 185 23%

225 16 7%

855 79 9%

185 19 10%

790 88 11%

110 15 14%

795 87 11%

148 38 26%

632 178 28%

88 25 28%

636 136 21%

740 125 17%

3160 603 19%

440 80 18%

3180 528 17%

Q6: L2 comprehension : over 5 questions possible actual % actual

265 20 8%

390 32 8%

Q7: L1>L2 translation : over 5 questions possible actual % actual

265 12 5%

390 27 7%

Q9: Selecting lexical item(s) for context slot: over 4 questions possible actual % actual

212 46 22%

312 78 25%

180 33 18%

684 160 23%

Total for Q3, Q5, Q6, Q7, and Q9 : over 20 questions possible actual %actual

1060 112 11%

1560 188 12%

900 87 10%

3420 506 15%

Table 11. Dictionary use decisions according to L2 skills, type of dictionary and type of task Table 11 reflects the fact that the students had either a bilingual or an L2 monolingual dictionary to use (none having either access to both, nor the choice of one or the other for a specific query). For that reason, the differences between bilinguals and monolinguals are small, but the groups are internally remarkably unanimous as to which question provokes them to consult a dictionary. Three types of comparison may be made from this table: 1. Horizontally, contrasting dictionary types (monolingual or bilingual) for the same grade and the same type of task: in this case the table should be read as follows:

38

Atkins & Varantola

When the question was (e.g.) comprehension of L2 (Q-6), in 8% of the cases, Grade A students with L2 monolingual dictionaries consulted their dictionary, and in 8% of the cases Grade A students with bilingual dictionaries consulted their dictionaries. 2. Horizontally, contrasting students' L2 level ("grades") for the same type of dictionary and task: in this case the table should be read as follows: When the question was (e.g.) comprehension of L2 (Q-6), in 8% of the cases, Grade A students with L2 monolingual dictionaries consulted their dictionary; in 6% of the cases Grade Β students with L2 monolingual dictionaries consulted their dictionary; in 25% of the cases Grade C students with L2 monolingual dictionaries consulted their dictionary; and in 20% of the cases Grade D students with L2 monolingual dictionaries consulted their dictionary. 3. Vertically, contrasting tasks for the same grade of L2 skills and the same type of dictionary: in this case the table should be read as follows (taking the "Grade B. 171 with biling" column as an example): 13% of Grade Β students, using a bilingual dictionary, consulted their dictionary for the prepositional complementation of a given word in a given context (Q-5), whereas 23% of the same students using the same dictionaries consulted their dictionary in the search for the correct lexical item to fit a slot in a given context (Q-9). Finally, Table 11 confirms the generally held belief that students reach for the bilingual dictionary before the monolingual L2 dictionary: it shows that, whatever the task and the L2 level of the student, the bilingual dictionary was (with one or two small exceptions) systematically consulted more frequently than the L2 monolingual. 4.2 Did using the dictionary help the students? It could be argued that this is an impossible question. How can one tell whether a student who used the dictionary to answer a question might not have produced the same answer without the dictionary (and vice versa)? In the case of individual students, and individual questions, it is of course impossible to tell. However, we believe that over a large group of respondents performing a number of different exercises, sometimes with and sometimes without their dictionaries, trends begin to emerge. In considering whether the dictionary actually helped the student, questions of both Type A (where the relevant dictionary entry can be pinpointed) and Type Β (where all that is known is that the dictionary is relevant to answering the question correctly) should be included in the study, and thus here again the scope is over the 20 "active" DRT questions and the 723 respondents. 4.2.1 Comparing individual responses with and without the dictionary Table 12 looks at whether the dictionary was used in cases where the various questions were answered correctly, and shows whether the use of the dictionary made a difference. In this table, some of the percentages are based on very small figures. We include them in the interests of completeness, bracketing the percentages in cases where the number corresponding to the percentage is fewer than 10. Table 9 shows how many of each group (Grades A, B, C and D) actually used a dictionary for each of the "active" questions. Table 12 shows how many of these look-ups were successful, and also how many of the look-ups performed without a dictionary were successful. The structure of Table 12 repeats that of Table 9. In each "+dict" box of Tablel2, therefore, the figure in the corresponding box of Table 9 represents the 100%: thus for Q-5/19,

Language Learners Using Dictionaries

39

39 Grade A students used their dictionary (see Table 9), and 27 of these got the right answer (as Table 12 shows): 27 is 69% of 39. Table 9 by implication shows that 92 of the 131 Grade A students did not choose to use the dictionary for Q-5/19; Table 12 shows explicitly that 50 Correct out of the lookups answer by 131 students to Question.. Grade A 3/11 * 5/19 * 5/20 5/21 * 5/22 * 5/23 * 6/24 6/25 6/26 6/27 6/28 7/29 7/30 7/31 7/32 7/33 9/41 9/42 9/43 * 9/44 Subtotals Totals

+dict 1 (100%) 27 69% 5 (83%) 22 96% 15 100%

-diet 127 98% 50 54% 119 95% 94 87%

2 (100%) 5 (71%) 3 (100%) 3 (75%) 8 (80%) 20 71% 6 (86%) 6 (75%) 7 (47%) 4 (44%) -

46 96% 31 86% 18 82% 10 56% 239 79% 2239 100%

111 96% 129 100% 115 93% 125 98% 119 94% 116 96% 86 83% 110 89% 112 91% 84 72% 100 82% 106 81% 54 65% 78 82% 86 79% 79 70% 2000 86%

out of the lookups by 216 students Grade Β

out of the lookups by 195 students Grade C

+dict 6 (86%) 34 62% 3 (75%) 47 82%

-diet 196 94% 47 29% 184 87% 95 60%

+dict 13 87% 19 47% 7 (78%) 49 65%

-diet 151 84% 25 16% 158 85% 55 46%

+dict 18 72% 12 44% 4 (40%) 38 58%

11 73% 4 (80%) 34 85% 23 74% 14 87% 17 61%

179 89% 203 96% 128 73% 151 82% 159 79% 157 83% 100 62% 123 65% 123 63% 102 51% 113 61% 131 61% 38 24% 101 59% 98 53% 73 45% 2501 67%

21 70% 8 (73%) 54 78% 30 73% 22 63% 20 62%

117 71% 167 91% 83 66% 96 62% 98 61% 91 56% 58 44%

9 (33%) 7 (78%) 29 58% 16 47% 27 59% 15 47% 16 36% 12 46% 9 (47%) 6 (25%) 13 39%

33 61% 14 52% 11 52% 5 (29%) 12 40% -

50 85% 38 83% 29 88% 12 22% 397 66% 2898 100%

32 51% 15 62% 9 (47%) 7 (30%) 18 44% -

47 77% 27 51% 24 77% 17 24% 439 59% 2156 100%

96 56% 94 53% 61 35% 79 51% 83 43% 22 16% 92 65% 53 32% 38 31% 1717 54%

out of the lookups by 181 students Grade D -diet 95 61% 5 3% 121 71% 37 32% 58 38% 147 85% 60 46% 58 39% 45 33% 51 34% 32 23% 50 32%

-

27 64% 29 67% 12 46% 11 20% 310 49%

58 36% 46 29% 70 47% 72 40% 20 14% 97 70% 24 15% 35 27% 1181 40%

1491 100%

Table 12: Successful answers to dictionary-relevant questions according to whether or not a dictionary was used

40

Atkins L2 706 responses

Using known L2 item 688 responses

L2 skills

mono

biling

both

mono

biling

both

mono

biling

both

Of 131 students Grade A

71 54%

52 40%

8 6%

9 7%

113 86%

9 7%

89 70%

32 25%

6 5%

Of 212 students Grade Β

84 40%

119 56%

9 4%

34 16%

172 82%

3 1%

29 62%

70 34%

9 4%

Of 188 students Grade C

44 23%

135 72%

9 5%

23 12%

162 86%

4 2%

86 47%

97 53%

1 (0%)

Of 177 students Grade D

41 23%

129 73%

7 4%

5 3%

170 96%

2 1%

68 40%

97 57%

4 2%

Total

240 34%

435 61%

33 5%

71 10%

617 87%

18 3%

372 54%

296 43%

20 3%

Table 15: Monolingual versus bilingual dictionary preferences according to type of tasks and users' L2 skills

11

As these are sample responses only, none of the figures should total 100%. The "38" in the top left-hand box of course represents 29% of 131 (the total number of students in the Grade A group), and so on.

44

Atkins & Varan tola

Here it was made clear that the monolingual dictionary in question was an English dictionary; for most of the students this was probably interpreted as an English EFL dictionary. The numbers indicated in the leftmost column for each of the four groups (131 Grade A, 212 Grade B, 188 Grade C and 177 Grade D) show how many students from each group answered the various parts of the question. Not all the respondents replied to this question, and the number of replies received are indicated in Table 15, namely 708 responses to the first point (i.e. as indicated in the bottom row, 240+435+33), 706 to the second, and 688 to the third. Table 15 should be read as follows (taking the Grade A section of the first row as an example): For the purpose of understanding an L2 item, 54% of the 131 Grade A students said they would choose an L2 monolingual dictionary, 40% said they would choose a bilingual dictionary and 6% said they would wish to consult both, thus indicating no preference between the two. Table 15 shows a distinct overall preference for bilingual dictionaries amongst all grades of student and for all types of task. Only in the following circumstances do the students say they would prefer to use an L2 monolingual dictionary: • Grade A students for understanding an L2 item in their reading, • Grade A students for using a known L2 item, and • Grade Β students for using a known L2 item. Not surprisingly, only a very few students claimed to prefer an L2 monolingual dictionary over a bilingual dictionary for the task of translating into L2 from their own language. Most wished for a bilingual dictionary for each of the three named tasks. 4.3.3 Dictionary users'choices We have seen in Sections 4.3 .1 and 4.3.2 the opinions of the students on the relative merits of L2 monolingual and bilingual dictionaries for the various tasks in which such dictionaries are normally used. How, in the event, were the two types of dictionaries actually distributed among the various grades (L2 skills level) of the students who used them during the tests? Table 16 answers this question: Of 131 students Grade A

Of 216 students Grade Β

Of 195 students Grade C

Of 181 students Grade D

Bilingual dictionaiy

78 60%

171 79%

158 81%

159 88%

Monolingual dictionary

53 40%

45 21%

37 19%

22 12%

Totals

131 100%

216 100%

195 100%

181 100%

Dictionary choice

Table 16: Choice of monolingual or bilingual dictionary for use in the tests by students according to L2 skills level From Table 16, it is clear that dictionary choice correlated with L2 skills. The greater the student's L2 knowledge, the more likely he or she was to be working with an L2 monolingual dictionary. However, it must be remembered that these tests were carried out in the classroom. Where some L2 monolingual and some bilingual dictionaries were available, it is possible that the teacher in charge made sure that the monolingual dictionaries were used by the better students. There is no means of knowing whether this conjecture is accurate. In

45

Language Learners Using Dictionaries

every group, the majority of the students used bilingual dictionaries. The proportion of those using an L2 monolingual dictionary decreases as the L2 skills level drops: 40% of Grade A students, 21% of Grade B, 19% of Grade C and only 12% of Grade D. 4.3.4 How bilingual and monolingual dictionaries fared Before leaving the subject of bilingual versus monolingual dictionaries, the question should be asked: how did each type of dictionary fare in helping its user to find the correct answer to the various types of questions. 131 students Grade A Lookups

53 with mono

78 with biling

216 students Grade Β 45 with mono

171 with biling

195 students Grade C

181 students Grade D

37 with mono

158 with biling

22 with mono

159 with biling

37 2 2 (100%)

158 13 11 85%

22 5 5 100%

159 20 13 65%

Q3: Selecting support verb for context slot: over 1 question possible actual successful % success

53 -

78 1 1 (100%)

45 1 1 (100%)

171 6 5 83%

Q5: Selecting preposition for context slot: over 5 questions possible actual successful % success

265 34 31 91%

390 51 40 78%

225 24 19 79%

855 112 80 66%

185 22 12 55%

790 143 92 64%

110 18 10 56%

795 121 60 50%

225 14 7 50%

855 155 114 74%

185 46 30 65%

790 194 128 66%

110 22 16 73%

795 185 87 47%

225 16 9 56%

855 79 33 42%

185 19 8 42%

790 88 41 47%

110 15 8 53%

795 87 32 37%

148 38 22 58%

632 178 93 52%

88 25 14 56%

636 136 65 48%

703 127 72 57%

3002 616 354 57%

418 85 48 56%

3021 549 244 44%

Q6: L2 comprehension : over 5 questions possible actual successful % success

265 20 15 75%

390 32 24 75%

Q7: L1>L2 translation : over 5 questions possible actual successful % success

265 12 8 67%

390 27 15 56%

Q9: Selecting lexical item(s) for context slot: over 4 questions possible actual successful % success

212 46 39 85%

312 78 66 85%

180 33 24 73%

684 160 105 66%

Total forQ3, Q5, Q6, Q7, and Q9 : over 20 questions possible actual successful % success

1007 211 93 83%

1406 189 145 77%

855 88 59 67%

3249 512 332 65%

Table 17: Success rate for students using monolingual versus those using bilingual dictionaries according to users' L2 skills and type of task The figures for the various L2 skills groups of successful answers with a dictionary compared with successful answers without one are to be found in Table 13. Each of the complex rows in Table 17 contains four types of data: the Grade A column "53 with mono" relating to Q-5

46

Atkins & Varan tola

is taken as an example in this explanation. Of the 131 Grade A students, 53 were using an L2 monolingual dictionary. Over the five questions in Q-5, the number of times a monolingual dictionary could have been consulted by a Grade A student is 265 (53 χ 5), and this is the figure shown against the "possible" heading. The "actual" figure in that box shows that the monolingual dictionary was actually consulted by members of this group 34 times. Of these 34 consultations, 31 (the "successful" figure) resulted in a correct answer. Against "% success" therefore is given 91% (since 31 is roughly 91% of 34). Table 17 should therefore be read as follows (pursuing the example of the two Grade A columns relating to Q-5): 53 of the 131 Grade A students worked with L2 monolingual dictionaries, and 78 worked with bilingual dictionaries. In 91% of the cases when Grade A students used L2 monolingual dictionaries to help them with the Q- 5 questions they got the right answers, compared with those using bilingual dictionaries who in 78% of cases got the right answers. (It must be remembered that an overwhelming majority of respondents were using a bilingual dictionary.) As was the case with Table 11, three types of comparison may be made from this table: 1. Horizontally, contrasting dictionary types (monolingual or bilingual) for the same grade and the same type of task; in this case the table should be read as follows: When the question involved (for instance) comprehension of an L2 expression (Q-6), 75% of the Grade A students who used an L2 monolingual dictionary answered the questions correctly, as did 75% of the Grade A students who used a bilingual dictionary. 2. Horizontally, contrasting students'L2 level ("grades") for the same type of dictionary and task; in this case the table should be read as follows: When the question involved (for instance) comprehension of an L2 expression (Q-6), 75% of Grade A students who used an L2 monolingual dictionary answered the questions correctly, as opposed to 50% of the Grade Β students using an L2 monolingual dictionary for this type of task. On the other hand, 65% of Grade C students in a similar situation found the correct answers, as did 73% of the Grade D students. 3. Vertically, contrasting tasks for the same grade of L2 skills and the same type of dictionary: in this case the table should be read as follows (taking the "Grade B: 171 with biling" column as an example): 66% of Grade Β students using a bilingual dictionary were successful in their search for the prepositional complementation of a given word in a given context (Q-5), whereas 42% of the same students using the same kind of dictionary were successful in their attempt to find the L2 translation of an LI item (Q-7). The figures in Table 17 may be contrasted with the corresponding columns headed "correct -diet" in Table 13, which show the success rates for the same grouping of questions in the cases where the respondents did not use a dictionary at all. Thus, in the second complex row in Table 13, that relating to Q5, we find that in 85 of the possible 655 cases students belonging to the Grade A group actually used a dictionary. The corresponding box in Table 17 shows that this figure of 85 breaks down into 34 look-ups in a monolingual and 51 look-ups in a bilingual dictionary. Bilingual look-ups had a 78% success rate, and monolingual look-ups a 91% success rate (Table 17), compared with the 88% success rate of answers without the help of any dictionary, shown in Table 13. It is interesting to note that Table 17 seems to indicate that look-ups in a monolingual dictionary had overall a better chance of success than those in a bilingual dictionary, whatever the L2 skills level of the dictionary user: the percentages in the bottom row show 83% monolingual over 77% for Grade A users, 67% over 65% in the case of Grade B, 57% and 57% for Grade C, and 56% and 44% for Grade D. In the cases of the individual questions, the same

Language Learners Using Dictionaries

47

pattern is discerned, with very few exceptions. The type of task seems to make little impact on these figures.

5. Discussion of the data 3: Individual performances of dictionaries and users Section 4 reported on the contents of the database in overall terms. In Section 5 w e look briefly at certain individual cases, as an introduction to further studies which readers may care to undertake on the basis of the information given in Appendix 3 (listings of individual responses, showing actual responses) and Appendix 4 (relevant dictionary entries), in conjunction with Appendix 2 (Dictionary Research Tests). 5.1 L i s t i n g s o f i n d i v i d u a l results Appendix 3 contains listings of data relating to all the students w h o answered certain questions with the help of specific dictionaries. The six questions are those designated above as "Type A", those where the contents of specific entries are known to be relevant: Q-3/11, Q 5/19, Q-5/21, Q-5/22, Q-5/23, and Q-9/43. The dictionaries, of which full details are to be found under Dictionaries Cited at the end of this paper, are the six most frequently used b y the respondents in the project: OALD, LDOCE, CSD, H N S F D , Z N R D and LTE. In the case of these six questions, as well as all the other D R T questions, there were many other students w h o answered using other dictionaries, but since well over eighty dictionaries were used in the course of the project it would not be feasible to attempt to include them all in the listings. Nor, as explained earlier, would it be possible to include all the relevant entries in Appendix 4, since in many cases, despite the detail asked for in the heading material of the D R T questionnaire (title, author, publisher, date of publication) it was often impossible to tell exactly what version (standard, concise, pocket, etc.) or what edition of a given dictionary was being used. The data given in Appendix 3 (results listings) and Appendix 4 (dictionary entries), together with Appendix 2 (the D R T questions) make it possible to study more than 700 instances of dictionary use, knowing approximately the user's level of English, his or her native language, whether he or she had ever been taught how to use a dictionary, the actual question asked, the actual answer given, and the actual contents of the dictionary entry consulted. Teachers from the four language areas involved, whose students use EFL monolingual dictionaries, or bilingual dictionaries, may wish to study these results in more detail, in order to have a clearer picture of how successful the average dictionary user is when consulting the dictionary. Such information could be useful in planning the teaching of dictionary skills. 5.2 H o w t h e six d i c t i o n a r i e s p e r f o r m e d Table 18 summarises the type of coverage given by the six dictionaries to the information required by D R T Questions 3/11, 5/19, 5/21, 5/22, 5/23 and 9/43, which seemed to us to be the questions for which it was possible to identify the actual dictionary entry that the students would consult. For instance, Question 3/11 asks the student to supply the support verb take in the phrase "take a photograph", forcing the student who consults a dictionary to look at the entry for photograph.. In Table 18, the focus of the dictionary search is in each case printed in the "Information" column in italics; the full set of dictionary entries cited in Table 18 is given in Appendix 4.

Atkins & Varantola

48 Q

Information

OALD

LDOCE

LTE

HNSFD

CSD

ZNRD

3/11

take a photograph

Y?

YG

YG

YG

YG

YG

5/19

surprised at NP

YE

Ν

Ν

YG

Ν

YG

5/21

faith in NP

Y+

YE

YG

YG

YG

YG

5/22

danger of V-ing

Y?

YE

Ν

Y?

Y?

Y?

5/23

good for NP to V

YE

Y+

Ν

Y?

YG

Y?

9/43

cost of living

YG

Y

YG

YG

YG

YG

Table 18 : Type of coverage in the relevant entries in the six dictionaries The codes used in Table 18 have the following meanings: Y yes, item is included as headword or subheadword entry Y+ yes, item is included as complementation of headword YG yes, item is included with gloss or translation YE yes, item is included, not highlighted, but in an example Y? yes, item is included but requires thought to find it Ν no, item is not in the dictionary entry Codes Y and Y+ indicate that the information needed is very visible in the entry, and certainly stated in a way which would lead one to believe that a reasonably skilled dictionary user could not miss it. Examples are: Y cost of living is a headword in L D O C E Y+ the O A L D entry for faith starts off as follows: faith /feie/ η 1 [U] ~ in sb/sth, trust; strong belief... Code Y G indicates that the user has to persevere further into the entry in order to find the information, but that it is still fairly prominent, e.g. YG the CSD entry for faith contains as its fifth translated example "to have faith in tener f e en". Code Y E means that the user will have had to employ quite subtle strategies to find the fact sought for, often involving abstracting grammatical information from an example sentence and its translation, as for instance: YE in the OALD entry for surprise, the phrase be surprised stands in boldface in its own meaning section, numbered as sense 2, followed by five examples, the first, third and fourth of

which are: We were ~d at the news ...We were ~d at finding the house empty. It's nothing to be ~d about/at. Code Y ? means that the item is in the entry somewhere, but finding it requires diligence, even perseverance, on the part of the user. Y? The H N S F D entry for good is very long, and contains numerous distractions before the reader hits pay dirt:

I. a. 1. (a) ...this is g. enough for me ... he's too g. for that job ... (b) ... (c) ... he, his credit, is g. for £25,000 ... this car ought to be g. for another five years ... he's g. for another ten years ... (d) g. for you! ... φ ... (g) this medicine is very g. for coughs ... beer is not g. for me ... to drink more than is g. for one ... For more details on the treatment of these facts by the various dictionaries, see Appendix 4, where all the cited entries are given. From Table 18, it may be seen that most of the items needed by the students in order t o answer the questions correctly are in fact in the dictionary entries somewhere, although not many of them are explicitly dealt with as an important aspect of the headword. Only LTE, a

49

Language Learners Using Dictionaries

much smaller dictionary than the other five, quite understandably does not perform as well as the others.12 The listings in Appendix 3 confirm that looking something up in the dictionary did not always result in a successful conclusion, even when the item sought was included in the dictionary entry. The summary of information in, and evaluation of, the dictionary entries (in Table 18) suggests that perhaps the failure rate of answers made with the help of the dictionary, some of which are set out in Tables 19-24, should be ascribed not to gaps in the dictionary but rather to the inability of the students in a number of cases to find and use the information appropriately. Tables 19-24 give some indication of the effectiveness of the dictionary consultations with regard to the six questions under study. All these tables contain information about students who answered the question with the help of the various dictionaries. (For each question in turn, those who did not use their dictionary have been excluded from the calculations, as have been those who used one of the eighty or so other dictionaries used during the experiment.) Dictionaries containing item sought Type of answers

LDOCE [YG]

LTE [YG]

HNSFD [YG]

CSD [YG]

ZNRD [YG]

OALD [Y?]

correct

2

1

4

1

3

6

wrong

-

-

-

1

-

don't know

-

-

-

-

-

-

2

1

4

1

4

6

Total

Table 19: Correctness of answers according to dictionary coverage in photograph entries for Q3/11 (take a photograph) Table 19 focuses on the search for the support verb take as a collocator to photograph. All the six dictionaries held this information in one form or another, and in Tables 19-24 inclusive we show the dictionaries from left to right in order of the transparency of the relevant information in the entries. From Table 19 it may be seen that variation in the form in which the information is offered seems to have had little effect on the success of the operation: 17 out of the 18 dictionary users found what they were looking for, and answered the question correctly. Dictionaries containing item sought Type of answers

HNSFD [YG]

correct wrong

Dictionaries not containing item sought

ZNRD [YG]

OALD [YE]

6

5

18

1

-

1

2

7

4

5

4

don't know

1

-

-

-

1

Total

8

7

5

5

6

25

CSD [Ν]

LDOCE [Ν]

LTE [Ν] 1

Table 20: Correctness of answers according to dictionary coverage in surprise for Q5/19 (I'm surprised at you)

12

entry

It must be remembered that many, probably all, of the dictionaries cited here are now out of date, having been replaced by a revised edition, in which many of the deficiencies described here have been corrected.

50

Atkins & Varantola

Table 20 looks at the search for the preposition at as a complement of surprised, and presents a more varied picture. Out of 40 look-ups in dictionaries which did contain the item, 29 were successful: the pattern seen in Table 19 of almost 100% success rate is not repeated here, but what these figures share with those in Table 19 is that neither set shows any bias towards any particular type of presentation of information. Naturally enough, the three dictionaries which do not hold the information proved to be of little help to their users. Dictionaries containing item sought Type of answers

OALD [Y+]

LTE [YG]

HNSFD [YG]

CSD [YG]

ZNRD [YG]

LDOCE [YE]

correct

15

13

10

5

6

7

wrong

2

-

-

1

-

2

don't know Total

-

-

-

-

-

17

13

10

6

6

1 10

Table 21: Correctness of answers according to dictionary coverage in faith entry for Q5/21 (have faith in you) Table 21 contains information relating to the search for the prepositional complement (in) of the noun faith, and shows very much the same patterning as that seen in Table 19: out of 62 look-ups for an item present in all six dictionaries, 56 were successful. Dictionary not containing item sought

Dictionaries containing item sought LDOCE [YE]

Type of answers

CSD [Y?]

OALD [Y?]

HNSFD [Y?]

ZNRD [Y?]

LTE [N]

coned

2

4

8

4

3

-

wrong

-

-

1

-

1

1

don't know

-

1

-

-

-

2

5

9

4

4

Total

-

1

Table 22: Correctness of answers according to dictionary coverage in danger entry for Q5/22 (danger of doing) Table 22 focuses on the search for of as the prepositional complement of the noun danger in the context danger of doing something. Here again, we find very similar results: five dictionaries contained the information, and were used by 24 students, of whom 21 found the facts and used them correctly. The sole user of the dictionary which did not contain the information failed to answer the question correctly. Dictionaries containing item sought Type of answers correct wrong don't know Total

-

-

-

-

-

-

-

-

-

-

-

1

2

4

-

1 1

-

HNSFD [Y7] 2

ZNRD [Y?] 3 1

OALD [YE] 1

LDOCE [Y+]

CSD [YG]

Dictionary not containing item sought LTE [N] -

Table 23: Correctness of answers according to dictionary coverage in good entry for Q5/23 (good for you to do)

Language Learners Using Dictionaries

51

Table 23 records the success rate of the search for to as the prepositional complement of good (in it's not good for you to eat so much). Here once again, when the dictionary held the information, the students in general (six out of eight) found it and used it correctly. Dictionaries containing item sought Type of answers

LDOCE Μ

OALD [YGJ

LTE [YG]

HNSFD [YG]

CSD [YG]

ZNRD [YG]

correct

4

8

3

10

5

3

wrong

1

2

1

-

4

1

-

-

-

-

-

-

5

10

4

10

9

4

don't know Total

Table 24: Correctness of answers according to dictionary coverage in cost entry for Q9/43 (cost of living) Table 24 is slightly puzzling. The phrase sought for, cost of living, was clearly present in all six dictionaries. The students were asked to supply the item to fill the slot in the following sentence: In London, the is very high. As may be seen from this table, not all students performed this task successfully. The other tables have shown us that these students are very capable of using their dictionaries effectively. It must be assumed, therefore, that the higher failure rate for this item arises either from students' inability to understand the context, and/or from the close semantic links between the components of the correct item, cost of living, and those of the distractors: cost of life, level of living, price of life. 5.3 More detail on one dictionary search We believe that in general the figures are reassuring both for language teachers and for lexicographers, showing as they do that when a dictionary contains the information sought, the students for the most part will discover the fact they want and use it correctly. However, occasionally this does not happen. The database contains instances of a dictionary offering information quite clearly and the student, for whatever reason, failing to find it. Here is one example. DRT Q-5 asked students to supply the correct prepositional complements of various types of headword. Here is what happened when students consulted one of the six dictionaries in an attempt to discover the correct preposition for the slot in this sentence: I'm surprised you. (Q- 5/19). Table 25 (where "DK" means that the respondent answered "Don't Know" to that question ) is a more detailed breakdown of the information summarised in Table 20. Table 25 contains details of the L2 skills of the students involved, and it should be remembered that the database shows that (predictably) the more advanced students had had in general more years of English teaching, and therefore presumably more experience in dictionary use. From Table 25, we can see the L2 level of the students who succeeded in finding and using the information, and of those who failed to do so. Of those who used the three dictionaries containing the facts required (HNFSD, ZNRD, and OALD), twelve of the thirteen Grade A students got the correct answer; nine out of thirteen Grade Β students got it right, as did five out of seven Grade Cs, and three out of seven Grade Ds. The numbers of course are very small, but there does seem to be some correlation between the L2 skills of dictionary users and their success in extracting information from dictionaries.

52

Atkins & Varantola Correct answers

Wrong answers

Dictionary

Coverage in that dictionary

Totals

Students' grades

Totals

Students' grades

HNSFD [YG]

included in three bold-face example sentences, and translated

6

2 A, 1 Β 2 C, 1 D

1 +1 DK

1 Β 1D

ZNRD [YG]

included in three bold-type example sentences, and translated

5

1 A, 1 Β 3 C

2

1 Β 1D

OALD [YE]

included in three italicized example sentences but not highlighted

18

9A.7B 2D

7

1 A, 2 Β

LDOCE [N]

not in the entiy at all

-

-

5

1 A, 1 Β 2 C, 1 D

LTE [N]

not in the entry at all

1

1Β

4 + 1DK

3 Β, 1 C 1D

CSD [Ν]

not in the entry at all

1

1C

4 + 1 DK

2 A, 2 Β 1 C

2C,2D

Table 25: Treatment of at complementation in the entries for surprise, showing the results of students' dictionary consultations

6. Conclusion Overall, we believe that the project has been worthwhile, producing some interesting results, and showing in some detail how students of various mother tongues and different English skills use their learner dictionaries. The fact that the results often reveal tendencies which confirm one's natural hypotheses about how students use L2 dictionaries is reassuring. Throughout this report, we have included some informal evaluation of the usefulness and feasibility of the Dictionary Research Tests (set out in full in Appendix 2). With hindsight, many of the questions in the DRT should not have been included in a dictionary use survey. We have tried to give an honest account of how effective the various questions were, indicating which of the DRT exercises proved useful and which were less so, for the purpose of this study. We believe that this has been a valuable learning experience, and we have included our "failure" details so that others may benefit from this. The principal problem with some of the test material was that it replicated the type of exercises designed to be used in the classroom, under the teacher's guidance, when dictionary skills are being taught.13 Experience shows that this kind of question, undoubtedly instructive in its proper context, did not always prove a useful format for extracting information about learners' actual dictionary use in a project such as this one. In all fairness, it must be added also that this project did not record what students do when they use their dictionaries in the normal course of their work. Our experience in this Dictionary Use Research Project led us to realise that there were other questions to be asked, and other facts to be discovered, if the steps performed in real-life dictionary look-ups were to be identified and recorded. This led to further studies within EURALEX, notably the 1991 Dictionary Use Workshop in Oxford, when the seventy or so participants used dictionaries in order to perform real translation tasks. Every step in the 13

See, for instance, Learning with LDOCE, by Janet Whitcut, Longman 1979, a booklet for use with the LDOCE dictionary, and specifically addressed to the teacher.

Language Learners Using Dictionaries

53

process was noted down on specially designed forms by observers, each of whom recorded the activities of a single dictionary-user. The findings of this experiment14 are reported in Atkins and Varantola {MonitoringDictionary Use, in this volume). Dictionaries Cited [CSD] [HNSFD] [LDOCE] [LTE] [OALD] [ZNRD]

Collins Spanish Dictionaiy (1971) (ed.) C. Smith. London & Glasgow: Collins. Harrap's New Shorter French Dictionaiy (1982) (eds.) P. Collin et al. London: Harrap. Longman Dictionaiy of Contemporaiy English (1978) (eds.) D. Summers et al. London: Longman. Langenscheidts Taschenwörterbuch Englisch (1983) (eds.) Ε. Klett et al. Berlin & Munich: Langenscheidt. Oxford Advanced Learner's Dictionaiy of Current English (1974) (ed.) A. S. Hornby. Oxford: Oxfoid University Press. II Nuovo Raggazzini, Dizionario Inglese-Italiano (1984) (ed) G. Ragazzini. Bologna: Zanichelli.

Bibliography Atkins, Β. T. S., H. Lewis, D. Summers & J. Whitcut (1987) "A Research Project into the Use of Learners' Dictionaries", in (ed.) A. P. Cowie The Dictionaiy and the Language Learner. Tübingen: Niemeyer: pp. 29-43. & F. Ε. Knowles (1988) "Interim Report on the EURALEX/AILA Research Project into Dictionary Use", in: (eds.) T. Magay and J. Zigäny BudaLEX '88 Proceedings, Budapest: Akadimiai Kiadö: pp. 381-392. Bogaards, P. (1990) Ό ύ cherche-t-on dans le dictionnaire?", in International Journal of Lexicography, 3:2: pp. 262-276. (1992) "A la recherche de collocations dans le dictionnaire de langue 0trang£re" in Aclas do XIX Congreso Internacional de linguistica e Filoloxia Romanica. Corufla, Fundaciön Pedro Barri de la Maza, Conde de Fenosa: pp. 175-185. Cambridge International Dictionary of English (1995), Cambridge: Cambridge University Press. Collins Cobuild English Dictionary (1995), London: HarperCollins pic. Collins-Robert English French Dictionary (1993), London: HarperCollins pic. Oxford-Hachette English-French Dictionary (1995) Oxford: Oxford University Press.

14

The experiment was later repeated with a homogeneous group of translation students at the University cf Tampere, and the data from that study were included in the report.

54

Atkins & Varantola

Appendix 1 The Dictionary User Profile Form was produced in French, German, Italian and Spanish, so that respondents could complete the form in their native language. The document here is essentially the original English version of the DUPF: in some of the multiple-choice questions (e.g. Question 3, relating to educational establishments) approximate cultural equivalents had to be selectedfor the various language versions. The "Student Number" at the top identified the language and country of residence of the student, his or her school or college, and the colleague (usually the English teacher) who collected this information.

DICTIONARY USER PROFILE FORM

]

Student Number

The aim of this project is to discover how dictionaries can be improved. You can help us by filling in this questionnaire (in block capitals, please). The questions ate about both monolingual dictionaries (those entirely in English) and bilingual dictionaries (in English and your own language). If you don't understand, please ask your teacher to help you.

1.

I) II)

2.

How many years have you been learning English for?

3.

In what kind of schools or colleges have you studied English? (If necessary, give an approximate answer.) I)

What country do you live in? What is your native language?

Primary School

How many years?

How many lessons per week?

(and the same two questions for the following) II)

4.

Middle School

III)

Senior School

IV)

Technical College

V)

University

VI)

Adult Education Class (evening class etc.)

Why are you learning English? Tick more than one box if you wish. I)

For a specific examination

a)

GCSEs ( Ό ' l e v e l s )

b)

Ά ' levels

c)

Cambridge First Certificate

d)

Cambridge Proficiency

Language Learners Using Dictionaries

5.

6.

55 e)

RS A (Royal Society of Arts)

0

TOEFL (English as Foreign Language)

g)

University Entrance

h)

Other (please specify).

II)

Because you are doing a degree or diploma in English

III)

Because you are studying another subject for which English is necessary

IV)

In order to improve your chances of employment

V)

In order to travel

VI)

Other (please specify)

Are you taught in English in your English lessons? Always Nearly always Half of the time

• •

•

I) Do you use a course book in your English class?

•

Never

Seldom

•

•

•

Yes

No

II) If you do, give the title of the course book and the author's name here.

7.

I) Have you been taught in class how to use a dictionary?

• •

Not recently

I

I

Yes

No

II) If yes, were you given systematic teaching and precise instructions on how to do this? Yes 8.

9.

I

No

I

•

Are you learning English because you need it for studying one or more of the following subjects? more than one box if you need to. I)

English language

II)

English or American literature

III)

Another language or other languages

IV)

Law

V)

Political science

VI)

Engineering

VII)

Science

VIII)

Medicine

IX)

Commercial subjects

X)

Teaching qualification

XI)

Other (please specify)

Tick

If you personally own a dictionary or dictionaries that you use for your English studies, say which ones: Title A

Published by

Date of publication

56 10.

Atkins & Varantola How long have you owned the dictionary or dictionaries that you listed above? Dictionary A Dictionary Β

11.

. years . years

Dictionary C Dictionary D

. years . years

What were your reasons for choosing thedictionaryor dictionaries? Dictionary A

I) Recommended by the teacher II) Recommended by the bookshop III) Recommended by parents IV) Recommended by a friend V) It didn't cost very much VI) It looked easy to use VII) . It had good illustrations Vin) It was a gift IX) Other (please specify) (The same choices were given for Dictionaries B, C and D.) 12.

13.

14.

Do you ever use a monolingual English dictionary? Never Seldom Often (every week or so)

Constantly (nearly every day)

Do you ever use a bilingual dictionary? Never Seldom Often (every week or so)

Constantly (nearly every day)

• •

• •

• •

Do you ever use a dictionary that you don't own yourself? Never I)

Seldom

Often

Constantly

In the school or college library

II) In class (provided by the school or college) III) At home IS.

If you do use other dictionaries, please give full details: Title

16.

Published by

Date of publication

What kind of dictionary would you normally use for each of the following purposes? English monolingual I)

To understand of an English word, e.g. during reading

II) III)

To find the English translation of a word in your language To check on how to use an English word that you already know, for instance, when you are writing an essay or a letter Other (please specify)

IV) 17.

Which of all the dictionaries you use seems the most useful?

Β

Bilingual

Β

Language Learners Using Dictionaries

57

Appendix 2 The Dictionary Research Test was produced in French, German, Italian and Spanish, to ensure that participants in the project would read the questions in their own language. The document here is essentially the original English version of the DRT, with the exception of the part which had no English original: Question 7, the own-language passage to be translated into English. Question 7 in the French version of the paper consisted of a passage in French, with its own set of questions; similarly the German DRT had a German Question 7, and so on. However, the four passages were selected and the questions devised in such a way as to ensure that they presented approximately the same level ofdifficulty.

DICTIONARY RESEARCH TEST Student Number

|

|

|

|

|

Thank you for helping us by doing these tests. The point is not to test you personally, but rather to assess the usefulness of the dictionaiy that you are using. (However, we will sometimes ask you not to use your dictionary for a particular question.) We are using the same tests for all students, whatever their level of English, and so you should not wony if you can't answer all the questions. Please use the dictionary that you normally use, whether it is a monolingual English dictionaiy, or a bilingual one (in English and your own language). It's important that you shouldn't change dictionaries in the course of doing the test.

Please give details of the dictionaiy you are going to use for the tests: Title

Published by

Is it a monolingual English dictionary or a bilingual dictionaiy?

1.

Date of publication

Monolingual I

I

Bilingual

I

ANSWER THIS QUESTION WITHOUT USING YOUR DICTIONARY. Do you know what part of speech the word is that you are looking up? For instance: in "I'm going to address the letters" the word address is a "veib" in "What's her address?" the word address is a "noun" So the correct answer to the question "What part of speech is address?" is marked like this: noun

adjective

preposition

I don't know

X

I'm going to address the letters What's her addressT

veib

X

Now answer the following questions by putting a cross in the appropriate box. In the next two sentences, what pait of speech is the word round? noun 1/1

We walked round the garden

1/2

The world is round, not flat.

adjective

veib

preposition

I don't know

I

58

Atkins &

Varantola

In the next two sentences, what pait of speech is the word past! noun 1/3

Her ancestors lived here in the past.

1/4

We've been here for the past week.

adjective

verb

preposition

I don't know

preposition

I don't know

In the next two sentences, what part of speech is the word just? noun 1/5

He lives just across the road.

1/6

She has a just claim to the money.

2.

adjective

advert)

ANSWER THIS QUESTION WITHOUT USING YOUR DICTIONARY. Dictionaries often contain items like: best super! of GOOD Here, superl is the abbreviation of the English word "superlative". The word best is the superlative form of the adjective good (good, better, best). So the following question should be read as "Mark the item which corresponds to the description "superl of GOOD", and the correct answer would look like this: better bad best goodly I don't know

X

superl of GOOD

Now answer the following questions by putting a cross in the appropriate box. 2/7

2/8

2/9

2/10

men manned manly mannish I don't know

pi of MAN

him they her his I don't know

poss of HE

could able cannot may I don't know

neg of CAN

I you me he I don't know

3rd pers

pronoun

Language Learners Using Dictionaries 3.

59

IF YOU WANT TO, USE YOUR DICTIONARY TO HELP YOU ANSWER THE NEXT SET OF QUESTIONS. Complete the sentences below by putting a cross against the word which fits the slot. For instance, if you think that brushed is the word which fits the gap in this example sentence, marie it like this: She

her hair

swept brushed dusted scrubbed I don't know

X

Now answer the following questions by putting a cross in the appropriate box. 3/11

She decided to

a photograph of the mountain

Did you use your dictionaiy to help you answer the question? 3/12

It's a bad

Yes

behaviour practice custom habit I don't know 1 No

Yes

grown evolved unfolded emerged I don't know I No

to bite your nails.

Did you use your dictionary to help you answer the question? 3/14

Yes

high long grand tall I don't know 1 No

My brother is small, but I am lm 80: I am quite

Did you use your dictionary to help you answer the question? 3/13

Yes

make do take put I don't know I No

A new problem has suddenly

Did you use your dictionaiy to help you answer the question?

ANSWER THIS QUESTION WITHOUT USING YOUR DICTIONARY. Put a cross against the word that you would look up first if you wanted to find the following items. For instance, electric light bulb [here the expression is translated into the language of the questionnaire]: if you think you would find electric light bulb under the word electric, mark the box as follows: at ELECTRIC at LIGHT at BULB after ELECTRICITY, as if it were a "word" in its own right I don't know

X

Now answer the following questions in the same way: say where you would really look for the expression - there's no "correct" answer to these.

60 4/15

Atkins & Varantola The expression lame duck means [explained here in student's language]: where would you look it up? at LAME at DUCK between LAME and LAMENT, as if it were a "word" in its own right I don't know

4/16

The expression to split hairs means [explained here]: where would you look it up? at SPLIT at HAIR between SPLIT and SPLITTING, as if it were a "word" in its own right I don't know

4/17

The expression to do without means [explained here]: where would you look it up?

4/18

at DO at WITHOUT between DOG and DOZEN, as if it were a "word" in its own right I don't know The word green is a colour, but greens is a plural noun meaning [...]: where would you look it up? at GREEN separately, at GREENS, as if it were a "word" in its own right I don't know

5.

IF YOU WANT TO. USE YOUR DICTIONARY TO HELP YOU ANSWER THE NEXT SET OF QUESTIONS. Complete the sentences below by putting a cross against the preposition which you think is the correct one to fill the slot. For instance, if you think that "It's cruel to pick an anima up by its tail" is correct, mark it like this: It's cruel to pick an animal up

with bv on at I don't know

its tail.

Now answer the following questions by putting a cross in the appropriate box. 5/19

I'm surprised

you.

Did you use your dictionary to help you answer the question? 5/20

A man

from

white hair was standing by the door.

Did you use your dictionary to help you answer the question? 5/21

Yes

I haven't much faith

tf

Yes

in with I don't know 1 No

Yes

with bv on in I don't know 1 No

what he says.

Did you use your dictionary to help you answer the question?

with bv for at I don't know 1 No

X

Language Learners Using Dictionaries 5/22

There's no danger

being interrupted.

Did you use your dictionary to help you answer the question? 5/23

It's not good

Yes

at with cf to I don't know I No

fa

you to eat so much.

in Jo

Did you use your dictionary to help you answer the question? 6.

Yes

I don't know I No

IF YOU WANT TO, USE YOUR DICTIONARY TO HELP YOU ANSWER THE NEXT SET OF QUESTIONS. Read the following English passage carefully, then answer the questions below. Keith Wadd's article in yesterday's paper gives a perceptive analysis as to why people drive fast on motorways. However, motorway accidents are caused by other types of bad driving too - for example, staying in the overtaking lane. We must concentrate our efforts on increasing the skill level of drivers. We must make the roads safe by making the drivers safer. Another dangerous nuisance is the use of rear fog lights when they are not necessary. Fog lights are extremely useful in heavy rain and bad light, but I see no excuse whatever for their use in a well-lit town centre (for example). They have a distracting effect on the driver of the car behind. I believe that most people who use these lights in this way are doing it as a form of one-upmanship - "Look," they seem to be saying, "I can afford a car with rear fog lights". Well, Mr. Moneybags, far from being impressed by this fact, I'm filled with a deep desire to ram your car hard from behind, breaking the offending lights beyond all repair. Put a cross against the answer you think is the right one; for instance, if you think that "written something in a newspaper" is the correct answer to the following question, mark it like this: What had Keith Wadd done? He had ... written a letter to a newspaper written something in a newspaper advertised his car for sale in the paper had his photograph in the paper I don't know Now answer the following questions by putting a cross in the appropriate box, and also maiking for each one whether or not you used your dictionary to help you answer it.

6/24

"We must concentrate our efforts on increasing the skill level of drivers" means ... we must think very hard about increasing drivers' skills we must do all we can to increase drivers' skills we must take more driving lessons we must force drivers to increase their skills I don't know Did you use your dictionary to help you answer the question? Yes | I

No

6/25

The writer thinks that most people who use rear fog lights badly are doing it... because they are men and think they are better than women because they think they will do it only once because one of the lights is higher up than the other because they want to make other drivers feel inferior I don't know ___ Did you use your dictionary to help you answer the question? Yes | |

No

62

Atkins & Varantola

6/26

He says that these people seem to be thinking "Look,... I know how to drive a car with rear fog lights" I have got time to put on the rear fog lights" I have enough money to buy a car with rear fog lights" I am clever enough to own a car with rear fog lights" I don't know Did you use your dictionary to help you answer the question?

Yes |

I

No

6/27

The name "Mr. Moneybags" means that someone ... has got bags of money in the car does not use credit cards is very rich indeed is carrying a lot of luggage I don't know Did you use your dictionary to help you answer the question?

Yes |

I

No

6/28

___

When the writer sees people like Mr. Moneybags he is filled with a deep desire to ... chase them away like sheep down a hill kick them veiy hard drive his own car into them behave like a male sheep I don't know Did you use your dictionary to help you answer the question? Yes | |

_ _

No

7.

The four passages for translation into English were different in the French, German, Italian and Spanish questionnaires, but the questions set on them (numbers 7/29, 7/30, 7/31, 7/32 and 7133) represented as far as possible similar levels of difficulty, and in each case (as, for instance, in questions like 6/28 above) the student was asked to note whether or not the dictionary had been used.

8.

IF YOU WANT TO. USE YOUR DICTIONARY TO HELP YOU ANSWER THE NEXT SET OF QUESTIONS. Read the following short English passage, and give one single word which will fill each of the numbered slots: in some cases, there are several correct alternatives. Accident Procedure for Leaders of Mountaineering Parties In the event of an accident - do not rush. It is essential that you (1) a grip on yourself and assert your control (2) the party at an early stage. Make (3) that other members of the group are busy: get them (4) something useful. Approach the casualty carefully and (5) a rapid examination, checking for life-threatening conditions: failure of breathing, no pulse, arterial bleeding. It is important to be thoroughly conversant (6) methods of resuscitation and life-saving first aid. Did you use your dictionaiy to help you to find these words? Answer below for each slot: 8/34 8/35 8/36

(1) (2) (3)

Yes Yes Yes

No No No

8/37 8/38 8/39

Yes Yes Yes

(4) (5) (6)

No No No

IF YOU WANT TO. USE YOUR DICTIONARY TO HELP YOU ANSWER THE NEXT SET OF QUESTIONS. Put across against the word which seems to you the best to fill the slots in the following sentences. The sentence below is given as an example: They spent many happy hours . . . . . . . their childhood reminiscing j j recalling j X j reminding j j thinking

j

j

I don't know J

J

Atkins & Varantola

53

Now answer the following questions

9/40

Our new neighbours are very sociable

I

I social

I

I socialist

I

I socialite

Did you use your dictionary to help you answer the question? 9/41

In the height of the summer Venice is full

I

I inhabited

I

I running

I

I swarming

I

I stimulant

|

I stimulator |

I simulator

Did you use your dictionary to help you answer the question? In London the

•

I

Yes |

I

I

I

I don't know I

I

Yes |

1

I

|

I don't know |

|

I

1

No I

I

No I

•

level of living

price of life

familiar

Yes |

•

No I

•

cost of living

Did you use your dictionary to help you answer the question? I am very

I

is very high.

•

cost of life

9/44

I don't know |

Coffee is quite a strong stimulus

9/43

I

with tourists.

Did you use your dictionary to help you answer the question? 9/42

I

I don't know

Yes I

I

No I

I

with this kind of problem. |

I accustomed

|

I used

I

I acquainted |

Did you use your dictionary to help you answer the question?

I

Yes |

I don't know |

I

I

I

No I

64

Atkins & Varantola

Appendix 3 Listings of responses to six questions for six dictionaries This document contains six tables, one for each of the six most popular dictionaries (CSD, HNSFD, LTE, ZNRD, LDOCE and OALD). The listings relate to cases where the use or non-use of a dictionary is certain, and the identity of that dictionary known. From these tables may be derived the following information for each individual look-up recorded: • the actual dictionary used • the respondent's native language • the respondent's L2 skills level (grade obtained in the Placement Test) • whether the respondent had received training in dictionary use • the respondent's specific answer to each of the six questions (this is marked with an asterisk if the dictionary were used to find it): from these data it may be seen not only whether the respondent gave an incorrect answer, but which of the incorrect options was selected in each case. The tables list the detailed responses to the six Type A questions (Q-3/11, Q-5/19, Q-5/21, Q-5/22, Q-5/23, and Q-9/43) for which we can identify one specific relevant dictionary entry and for which the respondent has indicated whether or not the (known) dictionary was used to help them answer the question. The tables contain enough detail to allow anyone interested in taking this research further to make other case studies of the type shown in Section 5 of the main report.

Explanation of codes used in the columns of the tables Column

Heading

Codes

1

Grade

A-D

2

Dictionary Use Training (question numbers)

Y=Yes Ν = No

3-8

R W1 W2 W3 DK

Indicate

Notes

level of L2 skills (A highest, D lowest) (to the question: "Has the respondent been taught how to use a dictionary?"' right answer to question wrong answer 1 wrong answer 2 wrong answer 3 Don't Know

from Placement Test result; from Dictionary User Profile Form Question 7 See below for the actual answer represented by thesi codes (R, W l , W2 etc.) for each specific question.

Column 3 Q.3/11 This shows the answer given to Question 3/11 of the DRT, which was: Complete the following: She decided to a photograph of the mountain. R = take

W1 = make

W2 =do

W3 = put

Language Learners Using Dictionaries

65

Column 4 Q.5/19 This shows the answer given to Question 5/19 of the DRT, which was: Complete the following: I'm surprised you. R = at

W1 = with

W2=by

W3 = for

Q.5/21 Column 5 This shows the answer given to Question 5/21 of the DRT, which was: Complete the following: I haven't much faith what he says. R = in

W2=by

W1 = with

W3=on

Column 6 Q.5/22 This shows the answer given to Question 5/22 of the DRT, which was: Complete the following: There's no danger being interrupted. R = of

W1 = at

W2 =with

W3 = t o

Column 7 Q.5/23 This shows the answer given to Question 5/23 of the DRT, which was: Complete the following: It's not good you to eat so much. R = for

W1 = in

W2=to

W3=by

Column 8 Q.9/43 This shows the answer given to Question 9/43 of the DRT, which was: Complete the following: In London the is very high. R = cost of living

W1 = cost of life

W2 = level of living

W3 = price of life

The tables of responses In the tables which follow, an asterisk in an entry in columns 3-8 inclusive indicates that for that specific question the dictionary in question was used (and this was noted in each case by the respondent). Each of the four bilingual dictionaries involved (CSD, HNSFD, LTE, ZNRD) was used by one language group, as indicated in the titles of Tables 26-29 inclusive. In the LDOCE and OALD tables (30 and 31), an additional column ("Language") is added as the leftmost column, and holds the respondents' native languages. Codes in that column are as follows: F

French

G

German

I

Italian

S

Spanish

66

Atkins & Varantola

Grade

Dictionary Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

D

Y

R

W1

•R

W3

R

R

D

Y

•R

W3

*R

W3

R

•Wl

C

Ν

R

*W2

W3

R

R

*W3

Β

Ν

R

*W1

W3

R

R

*W2

Β

Ν

R

W1

W3

R

R

*R

A

Ν

R

R

R

R

R

*R

C

Υ

R

*R

R

W3

R

R

A

Υ

R

Wl

*R

*R

R

R

A

Υ

R

*DK

R

*R

R

R

Β

Ν

R

Wl

W3

Wl

R

*R

D

Ν

W1

Wl

*W3

*DK

DK

Wl

D

Υ

R

W2

*R

R

R

•W2

C

Ν

R

Wl

R

*R

R

*R

A

Ν

R

*R

*R

*R

R

*R

Table 26: [ C S D ] Collins Spanish Dictionary ( 1 9 7 1 ) Respondents: all Spanish speakers

Language Learners Using Dictionaries

Grade

Dictionary Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

Β

Ν

R

R

*R

R

R

*R

C

Ν

•R

W2

*R

•R

*R

*R

C

Ν

R

*R

*R

*R

*R

*R

A

Ν

•R

W2

R

*R

R

*R

C

Ν

•R

R

W3

R

W2

R

D

Ν

R

*W2

*R

R

R

W2

D

Ν

*R

*R

R

R

R

W2

A

Ν

R

*R

R

R

R

*R

A

Ν

R

*R

R

R

R

R

Β

Υ

R

*DK

*R

R

R

*R

C

Ν

R

*R

*R

R

R

»R

Β

Υ

R

W1

*R

R

R

•R

Β

Υ

R

*R

R

R

R

R

D

Ν

R

W2

*R

*R

R

*R

C

Ν

R

W3

*R

R

R

W1

Β

Υ

R

W3

*R

W1

R

*R

Table 27: [HNSFD] Harrap's N e w Shorter French Dictionary (1982) Respondents: all French speakers

Atkins & Varan tola

68

Grade

Dictionaiy Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

Β

Y

R

*W2

*R

R

R

R

A

Y

R

W2

*R

R

R

R

C

Ν

W1

R

•R

R

R

R

Β

Ν

R

R

•R

R

R

R

Β

Ν

*R

R

*R

R

R

R

Β

Ν

W1

*R

R

R

R

R

Β

Ν

R

R

*R

R

R

R

C

Ν

W1

R

*R

R

R

R

C

Υ

W1

W1

R

R

R

*R

D

Υ

W2

*W3

R

W3

R

W2

Β

Υ

R

*W1

W2

R

R

W3

C

Υ

R

R

*R

R

R

W3

C

Ν

R

*W2

W3

W3

W2

R

C

Ν

R

W2

*R

R

R

W1

C

Ν

R

W2

*R

W3

*DK

R

Β

Ν

R

W2

*R

*W1

R

R

Β

Ν

R

•DK

*R

W3

R

*R

C

Ν

R

W2

*R

W3

R

*R

D

Ν

R

W3

W3

W3

R

*W2

Table 28: [LTE] Langenscheidts Taschenwörterbuch Englisch (1983) Respondents: all German speakers

Language Learners Using Dictionaries

Grade

Dictionary Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

C

Ν

R

•R

W3

R

R

R

Β

Y

R

W2

•R

W1

R

*W1

C

Y

R

•R

*R

R

R

R

C

Y

W1

W1

R

*R

R

W1

Β

Ν

•R

•W1

R

•R

*R

R

C

Ν

»R

W1

R

*R

*R

W1

C

Υ

R

*R

W3

W1

R

R

Β

Υ

R

*R

*R

R

R

W1

D

Υ

R

R

*R

R

R

W3

A

Ν

R

*R

R

R

R

*R

D

Υ

*R

*W3

R

R

W2

*R

D

Υ

*W2

W3

*R

»W3

*R

R

C

Υ

R

R

*R

R

W2

*R

Table 29: [ZNRD] Zanichelli: II Nuovo Raggazzini, Dizionario Inglese-Italiano (1984) Respondents: all Italian speakers

Atkins & Varantola

70

Language

Grade

Dictionaiy Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

G

Β

Y

R

*W2

•R

R

R

W2

I

A

Ν

R

W3

R

*R

R

R

I

A

Ν

R

*W2

R

R

R

W1

I

Β

Ν

R

W1

W3

R

R

•R

I

C

Υ

W1

W2

*R

R

W2

*R

I

C

Υ

R

W2

*R

R

R

*R

I

C

Υ

R

*W1

*R

R

R

*W2

I

D

Υ

R

*W2

*R

W3

R

W1

I

D

Υ

R

W3

*DK

W3

W3

R

I

C

Υ

R

W3

*R

R

W3

R

I

C

Υ

R

W3

*W3

R

R

W1

I

D

Υ

•R

W3

R

W3

R

W1

I

D

Υ

*R

W3

R

W3

R

W1

s

A

Ν

R

R

W3

R

R

*R

s

C

Ν

R

W1

*R

R

*DK

R

s

C

Υ

R

*W2

*W3

*R

R

W2

Table 30: [LDOCE] Longman Dictionary of Contemporary English (1978) Respondents' native languages are shown in column 1

Language Learners Using Dictionaries

Language

Grade

Dicty Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

F

D

Ν

R

*R

R

W3

R

W1

F

c

Ν

R

W2

*R

W3

W2

W3

G

Β

Υ

R

W2

R

R

R

*R

G

Β

Υ

R

•R

R

*DK

R

R

G

A

Υ

R

•R

R

R

R

W2

G

Β

Υ

R

*W2

*R

R

R

R

G

A

Υ

R

*R

•R

R

R

R

G

Β

Υ

R

*R

*R

R

R

W2

G

Β

Υ

W1

*R

R

R

R

W1

G

Β

Υ

R

*R

R

R

R

*R

G

Β

Υ

*R

*W2

*R

*R

R

*R

G

C

Υ

R

*W2

R

R

R

R

G

Β

Υ

R

*R

R

R

R

R

I

Α

Ν

R

R

R

*R

R

R

I

Α

Ν

R

R

R

*R

R

*R

I

Α

Ν

R

*R

R

R

R

R

I

Α

Ν

R

•R

R

*R

R

*W2

I

Α

Ν

R

R

R

R

R

*R

I

Β

Υ

R

•R

R

R

R

R

I

Α

Ν

R

*W2

R

R

R

R

I

C

Ν

R

R

R

*R

R

R

I

D

Υ

R

*W3

•R

W1

W2

W2

I

D

Υ

W2

W

W3

*W1

R

•R

continued on next page

72

Atkins & Varantola

continued from previous page ....

Language

Grade

Dicty Use Training

Q3/11

Q5/19

Q5/21

Q5/22

Q5/23

Q9/43

I

D

Y

R

•W2

•W1

R

R

»R

I

D

Y

W1

W3

•R

W3

R

W1

I

c

Y

•R

W1

*W3

R

W2

R

I

c

Y

W1

W3

*R

R

R

R

I

c

Y

W2

.W3

W2

*DK

R

•R

I

D

Y

•R

R

•DK

•R

•R

W1

I

D

Y

•R

W1

R

W1

R

*W1

I

C

Y

R

•W3

W3

R

R

W1

I

Β

Y

*R

W2

R

R

R

W1

I

D

Ν

*R

•R

*R

W3

R

W3

I

C

Y

R

W2

*R

*R

R

R

s

Β

Ν

W2

W2

*R

R

R

W2

s

Β

Ν

R

*R

W3

R

R

R

s

A

Ν

R

R

*R

R

R

R

s

A

Ν

R

*R

•R

R

R

R

s

A

Ν

R

•R

W3

R

R

R

s

A

Ν

R

R

*R

R

R

R

s

A

Ν

R

•R

R

R

R

W1

s

A

Ν

R

W2

*R

R

R

R

s

A

Υ

R

•R

W3

R

R

R

s

A

Ν

R

•R

R

•R

R

R

Table 31: [OALD] Oxford Advanced Learner's Dictionary of Current English (1974) Respondents' native Languages are shown in column 1

Language Learners Using Dictionaries

73

Appendix 4 The Dictionary Entries There f o l l o w the dictionary entries for photograph, living)

surprise,

faith,

danger,

good

and cost

(of

, reproduced w i t h the permission o f their respective publishers. T h e s e are the actual

dictionary entries w h i c h the students consulted for help w i t h the T y p e A Questions;

the

result o f t h e s e l o o k - u p s may b e found in the tables in A p p e n d i x 3. T h e source dictionaries are indicated as f o l l o w s (for full publication details s e e "Dictionaries Cited"): [CSD]

C o l l i n s Spanish Dictionary ( 1 9 7 1 )

[HNSFD]

Harrap's N e w Shorter French Dictionary ( 1 9 8 2 )

[LDOCE]

L o n g m a n Dictionary o f Contemporary English ( 1 9 7 8 )

[LTE]

Langenscheidts Taschenwörterbuch Englisch ( 1 9 8 3 )

[OALD]

Oxford A d v a n c e d Learner's Dictionary o f Current English ( 1 9 7 4 )

[ZNRD]

II N u o v o Raggazzini, Dizionario Inglese-Italiano ( 1 9 8 4 )

In the entries that f o l l o w , the ellipsis within parentheses (...) indicates that s o m e part o f the entry has b e e n omitted.

Entries for photograph p h o t o g r a p h ['fautsgrasf] I η fotografia / (foto)\ aerial - aerofoto/ colour - fotografia/ en colores; to take a - sacar una foto. 2 vt fotografiar, hacer una fotografia de, sacar una foto de; "-ed by X " "fotografia de X". 3 vi: to - well ser fotogenico, sacar buena foto. [CSD]

p h o to g r a p h [TautagrafH-grsf] 1. Photographie / Lichtbild η, A u f n a h m e / take a ~ e-e Aufnahme machen; 2. photographieren; pho tog rapher [fat'agrefs] Photograph(in); photographic [fsuts'grsfik] (r-ally) photographisch; ~ library Bildarchiv n\ Photothek / ; - print Lichtpause / ; pho tog ra phy [fat'ogrefi] Photographie/ [LTE]

p h o t o g r a p h 1 /'faut3gra:f || -graef/also (infinf) photo, picture - η l a picture obtained by using a camera and film sensitive to light: Have you seen John's photograph in the newspaper? (= a photograph of John) 2 take a photograph to use a camera to obtain this [LDOCE]

p h o t o g r a p h ' ['foutsgraef] n. photographic f\ to take s.o.'s p., prendre une Photographie de qn; he had his p. taken, il s'est fait photographier. [HNSFD]

p h o t o g r a p h /'fsutagraif US: - g r » f / / r [C] picture recorded by means of the chemical action e: a mass of ovule-bearing or pollen-bearing scales or bracts in trees of the pine family or in cycads that are arranged usu. on a somewhat elongated axis English equivalent or explanation: 7. Context: Lapalisse devient chere. palisse. Matiere souple provenant du poil de Pepiderme des ovides (et de quelques autres mammiferes) constituee par des fibres pouvant etre utilisees comme textile. English equivalent or explanation: 8. Context: Ού sont tes battinesl battine. the laterally projecting region of each side of the lower or posterior part of the mammalian trunk formed by the lateral parts of the pelvis and upper part of the femur together with the fleshy parts covering them English equivalent or explanation:

An Empirical Study of Dictionary Use in L2-L1 Translation

149

Test for Francophones Ce test vise ä evaluer votre comprehension des definitions de diverses notions. Ce sont des notions bien connues (comme chien, maison, etc.), mais dans les contextes ci-dessous, le signifiant normal de chaque notion a ete remplace par un mot invente, qui n ' a aucune ressemblance avec le signifiant normal. Chaque contexte est suivi d ' u n e definition, en franfais ou en anglais, pour le mot invente.

La täche a accomplir: •

Lisez le contexte et la definition fournie pour le mot invente.

•

En vous basant sur la definition, donnez un equivalent fran9ais pour le mot invente. Veuillez donner l'equivalent le plus precis possible.

•

Si l'equivalent vous echappe, mais vous croyez avoir compris la definition, donnez une explication en fran^ais de la notion. Soyez le plus precis possible.

Exemples (a)

Contexte: There are many different kinds of bewster. bewster. a widely cultivated ornamental climbing or prostrate or sometimes shrubby Eurasian vine (Hedera helix) of the ginseng family with evergreen leaves, small yellowish flowers, and black berries. Equivalent franfais ou explication en fran?ais: espece de vigne grimpant,

äfeuilles

vertes,

ä petites fleurs jannes et ä baies mires (b)

Contexte: He has good kalters. kalter. Chacun des deux visceres loges symetriquement dans la cage thoraxique. organes de la respiration oü se font les echanges gazeux. Equivalent frangais ou explication en franpais:

poumon

1. Contexte: H e doesn't like g/attle. glattle. a sour liquid obtained by acetic fermentation of dilute alcoholic liquids and used as a condiment or preservative Equivalent fransais ou explication en fran^ais:

2. Contexte: Brigcites are very popular in some countries. brigate. Instrument de musique a soufflet et ä anches metalliques.

Paul Bogaards

What type of words do language learners look up?

Abstract When reading a text in a foreign language, learners do not look up all words which cause problems. The central aim of this paper is to shed some light on the question why such learners decide to look up some (types of) words and not others. After a review of the relevant literature, the author reports on a piece of empirical research where texts containing three types of words were read or translated by advanced learners of French and where their dictionary use was monitored.

A foreign language learner who is confronted with a linguistic difficulty, be it in production or reception, may feel that it could be worthwhile consulting some type of dictionary. As is well known, however, not all words which cause problems are looked up, not even when a dictionary is ready to hand and when there are no real time constraints. One may wonder, therefore, whether foreign language learners go ahead in some systematic way or whether they decide more or less haphazardly on the words to look up. In other words, one would like to know if there are any criteria which could explain the search strategies of foreign language learners as far as the decision to look up or not to look up different words is concerned. Although this step in the process of dictionary use is mentioned in several descriptions or models (see for instance Bogaards 1993), up to now very little is known about the choices users make. I shall start with a brief discussion of the two or three papers that up to now have paid some attention to this topic. After that I will report on a piece of empirical research I have conducted.

1. Review of the literature Elaborating on the analogy between the learner not understanding a passage because of insufficient vocabulary knowledge and a mechanical breakdown of a machine because of a faulty part, Scholfield (1982 : 185) says: "We may think of learners as opening the dictionary and being provided with words to fit the context, but it is not so simple. Like a mechanic, they have to locate the faulty part first, extract it, and then seek a replacement in the stores, where it might or might not be in stock." In his description of the look-up process, Scholfield (1982 : 186) mentions as a first step : Locate the word(s) or phrase you don't understand. Whereas this does not seem to be difficult when unknown single words are involved, it is less easy, he claims, to locate the problem word(s) in sentences like 1. George was lucky to get off the charge. 2. The members moved a new proposal. All words look familiar, and yet the meaning of the whole may not be clear at all to learners. In this type of situation it may indeed be hard for them to know where the problem lies: have the verbs unusual senses, or the nouns, or both? Are there idioms involved? In many such cases they may feel at a loss, especially when high-frequency words with many different senses or uses are to be looked up.

152

Paul Bogaards

In his OMEGA-model, Müllich (1990) calls the first step Optionsphase or Ortungsphase ('option phase' or 'location phase'). Potential users have to decide whether they will consult a dictionary or not, and to ask themselves which word they would need to look up, at least insofar as they are aware of any difficulty. In his in-depth study of 86 intermediate students who had to translate French or English texts into their mother tongue (German) with the help of monolingual (learners') dictionaries, Müllich examined also the cases where subjects did not open the dictionary without being able to give a correct translation. He distinguished three possible sources of errors in this phase. The non-consulting of a dictionary could be due to. a. intralingual misjudgment cases of homonymy or polysemy (like French entendre used with the meaning of 'to intend') or grammatical changes (like la vase 'mud', where le vase 'vase' is far better known, or bonne used as a noun in the sense of'maid'); b. interlingual misjudgment these are the famous faux amis (like French action in the sense of 'share' and German

Aktion) c. "dislocative" misjudgment compounds and lexicalized phrases (like pouvoirs publics 'authorities' or un bout de 'a piece o f ) Although Müllich (1990 : 70) states that these categories presented themselves with a remarkable clarity in his material, one can have doubts about some examples given. Not only is bonne mentioned twice by Müllich himself, first as a case of homonymy/polysemy and second as a case of a word having a different grammatical status, but a word like action is not only a false friend but could be characterized as a case of intralingual misjudgment as well. Moreover, the German translation of French masculine vase being German feminine Vase, it is not sure what exactly causes the misjudgment. Finally, partly due to differences in grammatical and lexical structures, the three categories manifest themselves in quite different ways in the case of English texts (pp. 217 - 218). What the errors made in the option phase have in common, according to Müllich (1990 : 71), is that the words involved all have a somewhat familiar appearance. Totally unknown words did not appear to form much of a problem at this point of the look-up procedure. As was suggested by Scholfield, when students are allowed to use a dictionary, it is not the unknown words which form a problem - they are recognized as such and looked up in the dictionary - but those that seem to be familiar in some way or another. In a well-designed study, Hulstijn (1993) provided L2 learners with translation information about a number of words in a reading passage. This information became available when the subjects moved the cursor on a computer screen to the relevant word and pressed the Enter key. The first aim of the study was "to assess the influence of some task variables on FL readers' look-up behaviour, in particular the influence of reading goal, word relevance, and word inferability" (Hulstijn 1993 : 140). As to the choice of words to be translated, Hulstijn picked out 109 "difficult" words, considering that the rest (663 words) "were high frequency words which did not pose any problems to the subjects, who had had at least four years of EFL instruction". Hulstijn may have underestimated the importance of the "easy" words. No data are available as to the number of these words that subjects tried to look up. It is mentioned that in a pilot test, none of these words had been marked as being unfamiliar. It could be the case,

What Type of Words do Language Learners Look Up?

153

however, that, due to the effect described above, students systematically ignore all word forms that look familiar when asked to detect difficult words or to indicate which words they would like to look up in a dictionary for clarification. Nevertheless, several aspects of the study conducted by Hulstijn are interesting in connection with our research topic. The first thing is that Hulstijn (1993 : 141) found "enormous individual differences in the use of the translation facility. The number of words consulted ranged from one to 103, with a mean of forty-one" (with a standard deviation of 24). Subjects with greater vocabulary knowledge tended to look up fewer words than those with smaller vocabularies (p. 144). Second, subjects appear to be "capable of reading a FL text in a strategic manner, not looking up all unfamiliar words in an uncritical fashion, but deciding on the relevance (. ..) of unfamiliar words in relation to their reading goal before taking action or not taking action" (Hulstijn 1993 : 144). The relevance of words was manipulated by asking questions on some paragraphs of the text and not on others so that comprehension of some but not all unfamiliar words was necessary to get an answer right. It turned out that "relevant words were substantially and significantly more often consulted than irrelevant words" (p. 144). But not all relevant words were looked up, and not even the pseudo-words that were used in the second experiment and that could not be known nor be simply guessed from the context. The papers briefly discussed above support the hypothesis that three factors influence learners' decisions to look up something in a dictionary: •

differences in individual approaches

•

differences in the types of words

• differences in the relevance of words for the type of text comprehension involved. It should be noted that these differences have been discussed in the context of reading foreign languages only. Apart from Bland & al. (1990) who propose a qualitative interpretation of different types of searches made by L2 learners who are writing texts in French, nothing is known about the choice of words for productive purposes.

2. Empirical study In order to have a closer look at one of the factors mentioned - differences in the types of words - 1 conducted the following small-scale study. 45 first year students of French at the universities of Leiden and Amsterdam were presented with three French texts. The texts were presented in different orders. The students were asked to read the first two texts and to underline the words they would have looked up in a bilingual dictionary if they had had to translate the text into their mother tongue (Dutch for all subjects). As to the third text, they were asked to translate it and to use the bilingual dictionary they had brought with them; they had to underline the words looked up. A translation task was chosen because for translation all words are relevant. Moreover, subjects are accustomed to translation tasks, whereas it is rare that they have to answer comprehension questions on a reading passage. The whole procedure took about 45 minutes. The first two texts were read by 40 students; each of the last texts were translated by 13 to 18 students. Because text type might exert some influence on the look-up behaviour of the subjects, three texts of quite different characters were chosen. The first one was a text taken from a French youth magazine; it was on "mangas", Japanese comics, appearing on the French market. The second one was taken from a linguistics textbook and was about the influence of

154

Paul Bogaards

the German linguist Schlegel. The third text consisted of 11 isolated sentences. All texts were more or less manipulated so as to contain the same number of words (155). The words I was interested in belonged to one of three categories: • words which were unknown to (most of) the subjects (henceforth infrequent words) • well-known word forms which were used in a particular, often idiomatic sense, which was unknown to (most of) the subjects (henceforth allosemic words) • words with forms that look like words in the native tongue but with quite different meanings (henceforth faux amis). Each text contained 15 of these target words (see Appendix). The manga text contained 7 allosemic words and 8 infrequent words, the linguistics text contained 7 infrequent words and 8 faux amis, the sentences contained 7 faux amis and 8 allosemic words. So, there were 45 target words, 15 in each category. It was qifite impossible to guess the meanings of the target words from the context. All the other words in the texts were expected not to cause problems for the subjects. Whereas the non-target words were indeed only looked up incidentally, not all the target words turned out to be sufficiently unknown to the subjects. The words where one third or more of the translation subjects gave a correct translation without consulting the dictionary, were discarded. This left us with 29 valid cases, 11 infrequent words, 10 allosemic words and 8 faux amis.

Category

Percentage of target words looked up in translation

Percentage of target words underlined in reading

( N = 13 to 18)

(N = 40)

linguistics

67%

57%

manga

69%

55%

manga

88%

86%

sentences

63%

53%

linguistics

13%

3%

sentences

35%

28%

Text

(number of items) infrequent words

(5)

(6) allosemic words

(2)

(8) faux amis

(4) (4)

Table 1. Results The results are summarized in Table 1. As may be seen, there seems to be a big difference between infrequent words and faux amis: infrequent words are looked up in two thirds of the cases, faux amis in one third or less of the cases when subjects have to translate a text. When they are asked to just read the text, they underline fewer words: about half the cases of infrequent words and less or far less than one third of the faux amis. There seem to be no differences in the types of words which are underlined while reading and those that are looked up during translation. The case of the allosemic words is less clear. In the case of the manga text, only two allosemic words turned out to be sufficiently unknown to be retained. As to the allosemic words in the sentences, they seem to have been treated in about the same way as the infrequent words.

What Type of Words do Language Learners Look Up?

155 1

Chi-squares were computed for the scores on the two types of words for each text. The differences were highly significant (p < .000) between infrequent words and faux amis in the linguistics text. The allosemic words differed significantly (p < .01) from the faux amis in the sentences, but were not significantly different from the infrequent words in the manga text. As to text type, it is difficult to draw any general conclusions. Infrequent words, which appeared in two rather different types of texts, seem to be treated in about the same way: in both the manga text, which was taken from a youth magazine, and the rather abstract text on linguistics, about the same results are found. As for the faux amis, they seem less stable over texts: they are far less looked up in the linguistics text than in the sentences. This could be due to the poorer context provided by the isolated sentences: the subjects had to look up the meaning of these words if they were willing to write down any translation at all; without the translation of the faux amis almost nothing was left in the relevant sentences, whereas there remained always something to be written, even if it was utter nonsense, in the linguistics text. The sentences seemed to be different in still another way, though. Whereas in the manga text and in the linguistics text subjects looked up some 11 words, in the sentences, which contained the same total number of words, they looked up only about 8 words. It is not entirely clear whether it was text type or the combination of word types (allosemic words and faux amis) that was responsible for this low score. A look at the percentages of errors is enlightening (see Table 2). Unsuccessful look-ups seem to occur in about 30% of the cases for infrequent words, in some 40% of the cases for allosemic words and in 50% of the cases for faux amis. As to the errors found for words where the dictionary was ignored, they amount to half of the cases for infrequent words and to three quarters of the cases for allosemic words and faux amis. Maybe students are not prepared to open the dictionary if there is less than a sixty percent chance of success. Anyhow, they seem to underestimate almost systematically the difficulties of allosemic words and faux amis, running far too many risks. Diction ary used

Dictiona ry not used

% incorrect

No Look-ups

101

29%

47

51%

113

41%

57

77%

32

53%

96

96%

Category

Number of items

Look-ups

infrequent words

11

allosemic words

10

faux amis

8

% incorrect

Table 2. Numbers of correct and incorrect translations (N = 13 to 18) The linguistics text offered a peculiarity in that relatively many of the non-target words have been looked up: 25 different words have occasioned 92 look-ups. (Compare 18 non-target words with 34 look-ups in the manga text, 13 non-target words with 17 look-ups in the sentences). Two aspects seem to be notable. First, no fewer than 11 of the non-target words which were looked up were good cognates which could be translated literally. It may be the case that because of the scientific character of these words (typologie, grammairiens, genetique, etc), students tried to use the bilingual dictionary as a sort of encyclopedia. Words of this type accounted for 36 of the 92 look-ups of non-target words in this text. The others were well-known words like aborder, faiblesses, nuirs, or even questions. This may explain

I would like to thank Dr. Vincent van Heuven (Leiden University) for his help with the statistics.

156

Paul Bogaards

why the success score was somewhat higher than in the case of infrequent non-cognate words (83% against 71%). The success score for these familiar words could have been even higher if the five students who deemed necessary to look up a very simple word like memes ('conducted', said of a study), had been able to find an acceptable solution in the dictionary. Second, the linguistics text contained two grammatical items which were mistranslated by the vast majority of the subjects (23 out of 28): pas plus que 'no(t any) more than' and si 'while'. In total, these expressions were looked up four times, two times with success. This seems to mean that grammatical items are largely ignored, either because the students do not notice the difficulty or because they do not think the dictionary can be of any help. As a last point one could say something about individual differences. These were found also in the materials studied here. Where the average in the manga and the linguistics texts is at about 11 words per student, some look up only two words, whereas others consult the dictionary no less than 17 times. For the sentences, where the average was at eight words per student, the extremes were at 2 and 14 words. The same type of differences presented itself in the reading task.

3. Conclusion As the empirical study described above is the first one conducted on the topic of the choice of type of words that are looked up in a translation passage, its results have to be taken as first tentative answers to this question. Nevertheless some conclusions may be drawn, which are in line with what was suggested in earlier discussions of the matter. As a first point one can say that not all words are treated in the same way. There appears to be a considerable difference between infrequent words and faux amis. Whereas the former are looked up in about two thirds of the cases and, when not looked up, cause errors in about half of the cases, faux amis are at best looked up in one third of the cases and, when not looked up, are at the origin of errors in three quarters of the cases. Faux amis as well as allosentic words seem to be underestimated as sources of errors by most of the students. The differences in word type may be taken to be more important for the look-up behaviour of students than differences in type of text, although words presented in isolated sentences seem to provoke rather particular strategies. Although there does not seem to be any fundamental difference in the choice of words when students are asked to underline the words they think would be worthwhile looking up in a dictionary, or when they are asked to make a translation, it seems clear that students tend to underestimate the number of words in the first type of task in a rather systematic way. This point as well as others mentioned earlier on should be taken up in future research.

Bibliography Bland, S.K., J.S. Noblitt, S. Armington, G. Gay (1990), The Naive Lexical Hypothesis: Evidence from Computer-Assisted Language Learning, The Modern Language Journal 74,440-450. Bogaards, P. (1993), Models of dictionary use, Toegepaste Taalwetenschap in Artikelen 46/47, 17-28. Hulstijn, J.H. (1993), When do foreign-language readers look up the meaning of unfamiliar words? The influence of task and learner variables, The Modern Language Journal 77, 139-147. Müllich, Η. (1990), 'Die Definition is blöd!' Herübersetzen mit dem einsprachigen Wörterbuch. Das französische und englische Lernerwörterbuch in der Hand der deutschen Schüler, Tübingen: Niemeyer. Scholfield, P.J. (1982). Using the English dictionary for comprehension. TESOL Quarterly 16. 185 194.

What Type of Words do Language Learners Look Up?

157

Appendix The target words are followed by (i) in the case of infrequent words, by (a) in the case of allosemic words, and by (0 in the case o f f a u x amis. The target words that are treated in the analyses are underlined. a) Le "manga", la bande dessinee japonaise debarque (i) en France Dans le metro de Tokyo, les voyageurs, de 8 ä 70 ans, sont plonges (a) dans leurs bull es (a). Au Japon, le "manga" est imprime a des millions d'exemplaires par semaine et, apres usage, se jette dans des poubelles prevues (a) a cet effet. Ce premier media respecte les traditions nippones (i): il a son temple bouddhiste et ses ecoles specialisees. En France, ce phenomene a commence en 1978, avec l'arrivee de Goldorak, heros qui n'y va pas de main morte (a). Ont suivi Candy, Dragon Ball... En 1989, les editions Glenat publient, en franpais, Akira. Chaque volume de cette histoire epoustouflante (i) se vend ä 50 000 exemplaires. Chez les 8-15 ans, la mode s'installe (a). Avec Kameha, premier magazine de mangas en version frarnjaise, ils en ont pour leur argent (a): 200 ä 300 pages remplies de maint coup de force (a). Les innocents bambins (i) croulent (i) sous les produits derives : jouets, autocollants, tee-shirts, les librairies specialisees regorgent (i) de gadgets (i). Dur pour les tirelires (i)... b) Les etudes typologiques en linguistique L'echec de la tentative (i) de F. Schlegel de creer (f) une typologie generale des langues, pas plus que certaines faiblesses qui caracterisent l'examen (f) auquel il a soumis les langues indoeuropeennes, ne nous autorise (i) pas ä renoncer (i) ä aborder le probleme, mais exige au contraire qu'on y apporte une solution adequate. Si, depuis le debut du XIXe siecle, les etudes historiques ont ete menees avec beaucoup de möthode (f), les questions de typologie ont conserve (f) longtemps un caractere speculatif. Tandis que les volumes (0 de l'Histoire de la langue qui etaient consacres ä la classification genetique se font toujours respecter (f), les temps n'etaient pas encore mürs pour une classification typologique. La contribution de Schlegel ä la recherche ( 0 concernant les problemes genetiques laissa, au siecle dernier, une empreinte (i) particuliere sur les typologies, et les typologies susciterent (i) la mefiance des grammairiens. La doctrine (0 de Marr fut peut-etre le dernier rejeton (i) de cette lignee (i). c) Sentences 1. Pourriez-vous me dire quel est le capital social (0 de cette entreprise? 2. Tout comme sa mere, eile se fait remarquer par sa taille (f) 3. J'ai peut-ätre bon dos (a), mais il ne faut pas qu'on me pousse ä bout (a). 4. II a beau faire des pieds et des mains (a), un jour il devra casser le morceau (a). 5. Les parents de mon ami ont longtemps vecu dans la terreur (f). 6. J'espere que tu comprendras l'interet des legendes (f) qui se trouvent sous ces illustrations. 7. A une certaine epoque, beaucoup de jeunes gens ont du etre reformes (f) 8. Lorsque les gosses ont vu arriver les gendarmes, ils n'ont fait ni une ni deux (a) pour les mettre (a). 9. II est rare qu'on trouve ce type d'objets d'art dans leur etat primitif (f). 10. Ce n'est pas mon frere, mais un de ses collegues qui est charge de l'instruction (f) de cette affaire. 11. II a mange son pain blanc le premier (a); le reste pourrait bien lui faire froid dans le dos (a)

Hilary Nesi

Defining a Shoehorn: the Success of Learners' Dictionary Entries for Concrete Nouns Abstract The study reported in this paper investigates learners' perceptions and interpretations of definitions from the 1995 editions of four major learners' dictionaries (OALD-5, LDOCE-3, COBUILD-2 and CIDE-1), and the Concise Oxford Dictionary 1990 (COD-8). 158 non-native speaker subjects were required not only to rate definitions in terms of their perceived helpfulness, but also to identify visual representations of them in a multiple choice format. Subjects were found to judge the definitions from LDOCE-3 and CIDE-1 as most helpful, and to interpret CIDE-1 entries more successfully than the entries from OALD-5, LDOCE-3, COBUILD2 and COD-8. The results help to distinguish those defining features which aid receptive and productive performance, while also identifying features open to misinterpretation. The number of subjects who were able to progress from incorrect production to correct identification also seems to indicate the value of including illustrations in dictionary entries for concrete nouns.

Introduction Although publishing houses have invested enormously in the research and development of their learners' dictionaries there have been very few independent studies to compare the relative advantages of different types of learner dictionary entry. The most famous of the few is the often-cited study by MacFarquhar & Richards (1983), in which English language learners were asked to compare entries for the same words from the Oxford Advanced Learner's Dictionary, Longman Dictionary of Contemporary English and Webster's New World Dictionary, and judge which of the three entries was easiest to understand in each case. MacFarquhar & Richards' subjects demonstrated a marked preference for the Longman entries (51.5%), and liked the Webster entries least (20%), thus suggesting that they found learners' dictionaries more intelligible than those designed for native speakers, and preferred definitions to be written within a restricted defining vocabulary. The study was, however, solely concerned with learners' initial perceptions of intelligibility, and did not attempt to investigate the consequences of consulting various types of definition. Moreover the study took place several years before the publication of the learners' dictionary with the most idiosyncratic defining style, Collins COBUILD English Language Dictionary (1987). A more recent study by Cumming, Cropp & Sussex (1994) documented not only how helpful learners judged dictionary definitions to be, but also how useful the definitions were in comprehension and production tasks. The study compared Longman Dictionary of Contemporary English (LDOCE-2), a learners' dictionary using the more traditional phrasal defining style and a restricted vocabulary, with the Collins dictionary (COBUILD-1), noted for its full sentence definitions and slightly more complex language. Both types of definition were tested with and without usage examples. Although post-task evaluations favoured COBUILD's sentence defining style, neither initial helpfulness ratings nor production and comprehension scores were found to differ across the four conditions. The results of the study by Cumming, Cropp & Sussex find support in Nesi & Meara (1994), who found no significant difference in the accuracy of sentences produced after consulting entries from LDOCE-2, COBUILD-1, and the Oxford Advanced Learner's Dictionary (OALD-4) or in the amount of time taken to read these entries. Nesi & Meara

160

Hilary Nesi

(1994) and Nesi (1994, 1996) discuss the way learners misinterpret dictionary entries; they conclude that the learners' lack of success was partly due to their own poor reading strategies and inadequate grammatical knowledge, and partly because the entries themselves were inappropriately constructed. The small amount of prior research in this area thus suggests that, while English language learners profess a preference for one defining style over another (LDOCE-1 rather than 0ALD-3 in 1983, COBUILD-1 rather than LDOCE-2 in 1994) this preference has no effect on the success of dictionary consultation. Such a finding is particularly surprising given that publishers in recent years have made great efforts to develop very distinctive defining styles, some tending towards a more discursive entry with a wider vocabulary range, others favouring shorter phrasal definitions within a restricted vocabulary. The present study applies previous research techniques to an investigation of the most recent (1995) editions of four major learners' dictionaries: Longman Dictionary of Contemporary English (LDOCE-3), Collins COBUILD English Dictionary (COBUILD-2), Oxford Advanced Learner's Dictionary (0 ALD-5), and Cambridge International Dictionary of English (CEDE-1), together with a native speaker dictionary of similar size, the 1990 edition of the Concise Oxford Dictionary (COD-8). The study differs from prior research both in the range of dictionaries investigated, and the simplicity of the experimental task, which was designed to measure how effectively meaning, rather than usage information, was conveyed in the dictionary entries. Materials Five nouns denoting everyday household objects were used in the study: colander, insole, plunger, shoehorn and spout. These words were chosen because: • they are grammatically regular; • they are collocationally unrestricted; • the majority of subjects did not know them; • subjects were familiar with the objects they denote; • the objects they denote are easy to recognise in picture form. OALD, LDOCE, COBUILD and COD do not illustrate any of these words, but colander, insole and shoehorn appear in composite illustrations in CIDE, at some distance from their entries. Because the task involved the identification of everyday objects, rather than the use of more abstract words with more complex syntactic and collocational behaviour, any lack of grammatical knowledge and writing skill on the part of subjects was less likely to influence the experimental results. The study used entries from the five different dictionaries, including pronunciation and grammar information, any regional alternatives, illustrative examples and etymological information. In some cases only part of an entry was used, excluding idiomatic expressions that did not help identify the household item (such as up the spout) and less common alternative meanings. The entries varied in length, complexity of syntactic structure, defining vocabulary and use of illustrative examples.

Defining a Shoehorn: Learners' Dictionary Entries for Concrete Nouns

161

Three measures were used to judge the accessibility of the entries: the number of passive sentences they contained, their readability according to a standard readability formula, and their lexical density. The percentage of passive sentences was recorded because passive constructions are believed to be more difficult for the reader to process than active constructions (Miller 1962), and have been shown in studies of first language acquisition to be acquired long after their active counterparts (Brown 1973, 313). It is open to debate how greatly studies of first language processing may relate to the way non-native speakers access a specific written genre. Nevertheless there was some justification for a hypothesis that dictionary users would have greater problems interpreting dictionary entries containing a high number of passive constructions. Readability was measured according to the Flesch Reading Ease Index, one of the most widely used readability formulae for secondary and adult reading material, and one which is calculated automatically by the editing tool on Microsoft Word for Windows (cf. Microsoft 1993-94). The Index derives a score from the average number of syllables per word and the average number of words per sentence in the text. The ratings of reading ease are as follows (Bentley 1985 :22): score 90+ 80-90 70-80 60-70 50-60 30-50 0-30

very easy easy fairly easy standard fairly difficult difficult very difficult

Texts for adult native speakers are on average found to score between 60 and 70. As with other readabilty formulae, the Flesch Index works on the assumption that long words will be less well-known to the reader than short words, and long sentences will be more syntactically complex (and therefore more difficult to process). These assumptions are obviously open to debate, and in any case the formula does not directly measure many text features, such as style and content, that may also affect comprehension. Word and sentence length are, however, widely accepted as being symptomatic of the difficulty level of text, and readability scores derived from a measurement of these factors "continue to attract widespread use, as a reasonably convenient way of predicting (though not explaining) reading difficulty" (Crystal 1987 : 252). Lexical density was measured according to the formula devised by Stubbs (1986), which entails calculating the ratio of grammatical words to lexical words in the text (lexical density = 100 χ L/N, where L is the number of lexical words in the text, and Ν is the number of words in the text).The higher the proportion of lexical words in the text, the denser the information content. Density of information may affect text accessibility as much as the length of words and sentences does, because texts containing a high proportion of lexical words tend to require more processing on the part of the reader who must mentally "unpack" meaning and decide on word relationships not overtly marked by grammatical means. Lexical density is therefore a useful measure to use in conjunction with a standard readability formula. Generally speaking formal written language has a higher lexical density than spontaneous speech; Stubbs reports the following lexical densities for a sample of texts (Stubbs 1996 : 75):

162

Hilary Nesi

36% a conversation (Lund corpus) 41% a religious pamphlet (LOB corpus) 48% a political speech (Lund corpus) 58% a radio horse-racing commentary (Lund corpus) 58% a belles lettres text on art (LOB corpus). A comparison of the entries used for all five headwords in each dictionary (minus headwords, regional alternatives, pronunciation guides, grammar information and etymology) showed LDOCE to be the most accessible in terms of standard measures of readability. It was not possible, however, to predict the effect of COBUILD's grammatically complete and less lexically dense defining style, or the usefulness of the illustrative examples in OALD and CIDE. Moreover, although COD definitions are denser (and therefore shorter) than those in the learners' dictionaries, they have a higher readability score than OALD and CIDE entries; COD entries also contain etymological information of possible value to some groups of learners. A summary of the comparison of entries in OALD, LDOCE, COBUILD, CIDE and COD appears in Table 1. OALD

LDOCE

COBUILD

CIDE

COD

number of words

99

94

134

173

86

average sentence length

19

18.8

19.1

24.7

14.3

% passive sentences

40

0

14

42

16

Flesch reading ease

76.4

85.2

83.9

66.8

81.1

48

50

42

45

55

phrasal

phrasal

sentence

phrasal

phrasal

one

none

none

two

none

% lexical density defining style illustrative example

Table 1: A comparison of entries for the five headwords. Entries for the five headwords were used to create five different versions of a four-stage task. A small pilot study was conducted, in order to trial possible formats for the task: on the basis of results from this study, a format was devised that was economical, easy to read and easy to mark. The five versions of the task each consisted of one sheet of A3 paper, folded in half to form a four-sided leaflet. On the front page, subjects were asked to rate their familiarity with each of the headwords on a scale from 1 to 5. This page remained the same in each version of the task (see Appendix 2).

On page 2, each headword appeared with an accompanying dictionary entry and subjects were asked to rate the helpfulness of each entry on a scale from one to five. This page varied across the five versions of the task, because although the order of the headwords remained the same the dictionary entries used to define them were rotated. This ensured that comparisons of defining style were not affected by the varying difficulty of headwords. Table 2 shows the order of the dictionary entries for each headword in each version of the task. Page 2 of Version 1 of the task is shown in Appendix 3.

Defining a Shoehorn: Learners' Dictionary Entries for Concrete Nouns

163

Version 1

Version 2

Version 3

Version 4

Version 5

colander

OALD

COD

CIDE

COBUILD

LDOCE

insole

LDOCE

OALD

COD

CIDE

COBUILD

plunger

COBUILD

LDOCE

OALD

COD

CIDE

shoehorn

CIDE

COBUILD

LDOCE

OALD

COD

spout

COD

CIDE

COBUILD

LDOCE

OALD

Table 2: The rotation of dictionaries in the five versions of the task On page 3, subjects were asked to produce their own sentences with each of the five headwords. This page remained the same in each version of the task (see Appendix 4). On page 4, subjects were asked to identify the five objects denoted by the headwords from a choice of twelve illustrations. This page remained the same in each version of the task. The illustrations were taken from The Visual Dictionary (Corbeil & Manser 1988) and are reproduced with the kind permission of Facts on File Publications (see Appendix 5). Subjects were not required to compare defining styles, and attention was not drawn to the stylistic variation between entries. In this respect the study differed from that of MacFarquhar & Richards (1983) who exposed subjects to three entries for each headword, thus preventing any further comparison of the practical use made of different defining styles. The procedure also differed from that used by Cumming, Cropp & Sussex (1994) who asked subjects to state a preference for the COBUILD or LDOCE style, with and without usage examples. Subjects did not compare like with like, however, in the study by Cumming, Cropp & Sussex, because the entries pertained to different headwords which posed different kinds of defining problems. Some of the headwords used for the present study seemed to be much more problematic than others whatever the defining style employed. Therefore, as it seemed likely that subjects would compare entries according to the difficulty of the headword rather than the defining style, it was decided that the task should not call for direct comparison of entries.

Procedure 158 subjects took part in the experiment. 101 of these were undergraduates studying at an English medium university in Botswana (majority first language: Setswana), 19 were Japanese undergraduates studying at Warwick University on a Junior Year Abroad programme, and the remaining 38 were British-based adult English language students from a variety of language backgrounds. 82% of the subjects from Botswana were regular users of the Oxford Advanced Learner's Dictionary, the remainder used the Longman Dictionary of Contemporary English,

Collins COBUILD English Dictionary, the Concise Oxford Dictionary and other dictionaries intended for native speakers. Dictionary use in other groups was fairly evenly spread between OALD, LDOCE, COBUILD and bilingual dictionaries. None of the subjects reported using CIDE. The five versions of the task were distributed to subjects with approximately equal frequency; as can be seen from Table 3. As every versioij of the task contained entries from all five dictionaries, the five dictionaries were represented in exactly equal proportions.

164

Hilary Nesi

Version One

Version Two

Version Three

Version Four

Version Five

31 subjects

33 subjects

32 subjects

31 subjects

31 subjects

Table 3: The distribution of versions of the task No time limit was set, but subjects tended to take ten to fifteen minutes to complete the four stages of the task.

Analysis 790 cases (five cases for each subject) were analysed for familiarity, perceived helpfulness, sentence production and picture identification. Scores (1-5) were recorded for initial familiarity with each of the five headwords and the perceived helpfulness of each dictionary entry. As the data was ordinal, variation across dictionaries in perceived helpfulness was measured using the Friedman test for three or more conditions (in this case all five dictionary types), and the Wilcoxon text for two conditions (in this case the two dictionary types that had the highest and lowest scores for helpfulness). For the production task, one of three marks was assigned to each sentence: "right" (1), "wrong" (0), or "ambiguous". Missing values were recorded in those cases where subjects failed to produce a sentence. The ambiguous category was introduced as a safeguard, because the production of a vague sentence might be indicative of an incomplete understanding of the dictionary entry, but might equally well be indicative of laziness or an inability to write, or might simply be due to constraints of time and space. Most vague sentences were ambiguous because of the lack of surrounding context, and if the subjects had been required to write at greater length the intended use of the headword might well have become clearer. In fact, 85.8% of all ambiguous sentences were followed by correct answers in the identification task. However, as it was impossible in the case of ambiguous sentences to independently measure subjects' word knowledge, such sentences were placed in a separate category in the statistical analysis of results. Wherever there was any doubt as to the writer's intended meaning sentences were placed in the ambiguous category. In addition, 78 cases were marked as "missing values" for the productive task because no sentence at all had been written. Unambiguous sentences were marked "right" or "wrong" depending on whether they displayed understanding of the function of the household object denoted by the headword. Subjects were not expected to produce the equivalent of an illustrative example in a dictionary; some indication of the use of the object was sufficient to merit a correct mark. Errors of spelling, grammar and communicative appropriacy were ignored, and in any case rarely centred on the headwords. In the vast majority of cases scoring was quite straightforward, and the validity of the scoring system is apparent from the fact that 348 out of the 364 sentences judged correct (95.6%) were followed by a correct answer in the identification task. In this respect the presence of the objectively scored picture identification task served to triangulate the results of the sentence production task, marked on a more subjective basis. The following sentences from the data for the word plunger exemplify the way in which sentences were assigned to each of the three categories. All sentences marked "wrong", and a sample of sentences marked "ambiguous" and "right" are listed below.

Defining a Shoehorn: Learners' Dictionary Entries for Concrete Nouns

165

All sentences for plunger that were marked "wrong": 1. My grandfather smokes a pipe which always blocks so he uses a plunger to clean it. [OALD] 2. The plunger is o f f . [OALD] 3. In USA there is many car theft so when I go out my car, I must use plunger. [OALD] 4. The maintenance men used a plunger to block off the kitchen drainage system. [OALD] 5. If you want to make some sweet, it is betterfor you to use a plunger. [OALD] 6. Its a piece usedfor cleaning and desktop. [OALD] 7. His electric iron does not work. Its plunger way stolen. [LDOCE] 8. You use a plunger to keep the water in the sink. [COBUILD] 9. The doctor plungered the wax blocking his ears. [CIDE] 10. My friend is a plunger [CIDE] 11./ need a plunger to make my dirty tall glasses clean. [CEDE] 12. / really need a plunger to penetrate my blocked nose. [CIDE] A sample of sentences for plunger that were marked "ambiguous": 1. You must wash the plunger after using it. [OALD] 2. This plunger is old. [OALD] 3. The plunger has holes so it can't perform its function. [OALD] 4. The plumber was carrying a plunger together with other tools. [LDOCE] 5. The best plungers are made from tough rubber. [LDOCE] 6. The plunger is stuck in the bathroom pipes. [LDOCE] 7. What's the matter with a toilet, please take a plunger. [COBUILD] 8. The plunger which a plumber was using was not working very well. [COBUILD] 9. She took the plunger and hit the thief with it on the head [COBUILD] 10. The toilet plunger is broken. [CIDE] 11 .We cannot manage if we do not have a plunger. [CIDE] 12. / usedplunger to get rid of something in the kitchen. [CIDE] 13. The sewage system mechanic needed a plunger to finish hisjob. [COD] 14. My brother who is a plumber has a plunger as one of his tools. [COD] 15. When the toilet over flows, use a plunger! [COD] A sample of sentences for plunger that were marked "right": 1. My sister took my plunger last week and this morning my bath could not drain water I had to drive to her place to collect it and unblocked the bath with it. [OALD] 2. I used a plunger to unblock the kitchen zinc. [OALD] 3. After many unsuccessful attempts to unblock the pipe with a stick he was advised to use a plunger. [OALD] 4. I could not come in time because we had a problem with our bath. We didn 't have a plunger to unblock the pipes. [LDOCE] 5. It takes time to remove water from the bath and I think we need a plunger to work on the drain. [LDOCE] 6. I need a new plunger to unblock the sink. [LDOCE] 7. A plunger was used to unblock a sink which was blocked byfood leftovers. [COBUILD]

166

Hilary Nesi

8. I broke the plunger when I was unblocking the pipe of the sink. [COBUILD] 9. The drain way so clogged that the plunger was virtually no use. [COBUILD]. 10. Something is in the sink. We 7/ need a plunger to unblock it. [CIDE] 11. When your sink is stuffed you can remove the stuff with plunger. [CIDE] 12. Our drain was blocked and daddy used a plunger to unblock it. [CIDE] 13. A plunger uses pressure to suck materials which block pipes. [COD] 14. Use a plunger to clear blocked pipes otherwise the mess in the pipeline blocks the drainage system. [COD] 15. The plumber used a plunger to clear a pipe of the sink which was blocked. [COD] Correct picture identification was rated "right" and incorrect identification and no identification were rated "wrong". Missing values were assigned to data only in the case of four subjects who left the entire fourth page of their leaflet blank (having presumably forgotten that they had to turn the page), and in one case where the letter identification was unclear. The production task was marked correct in 19 out of the 21 cases where data for the identification task was missing, thus suggesting that the pages were left blank in genuine error, rather than because the subjects could not identify the items. The chi-square test was used to measure variation in the results from the sentence production and picture identification tasks. This was an appropriate test to use because the data was nominal ("right" or "wrong"). It had originally been intended to discount data in cases where the subject declared a high level of familiarity with the headword (for example at levels 4 and 5). On analysis it became clear, however, that declared familiarity was no guarantee of successful sentence production or picture identification, and indeed that some of the most extreme misinterpretations were made by subjects who declared high familiarity with the headword. It was therefore decided to include all data in the analysis of variance between dictionaries. Results The five headwords were found to be unknown or little known to the majority of subjects. Entries were judged to be quite helpful, and in about three quarters of all cases subjects were able to complete the identification task successfully. Results for subjects' initial familiarity with the five words and the perceived helpfulness of the dictionary entries are summarized in Table 4 . ( 1 = 1 don't know this word at all, and 5=1 know this word very well.) Five cases were recorded as missing values because subjects failed to indicate any number in the scale. 1

2

3

4

5

387

67

96

89

146

Table 4: the distribution of ratings for familiarity Table 5 summarizes the comparative perceived helpfulness of the five dictionaries according to their mean rank, measured by the Friedman test.1 1

The mean is calculated by adding together all the observed values and dividing by the number of cases. The Friedman test requires scores to be rank ordered in terms of their relative size .The data is ordinal, so an

Defining a Shoehorn: Learners' Dictionary Entries for Concrete Nouns

Helpfulness

167

OALD

LDOCE

COBUILD

CIDE

COD

2.92

3.13

3.11

3.13

2.17

Table 5: Helpfulness ratings for the five dictionaries The table shows that LDOCE and CIDE were judged most helpful, and COD was judged least helpful. The Wilcoxon test revealed a significant difference between dictionaries in the case of LDOCE and COD (p = .01) and CIDE and COD (ρ = 005).2 In the production task 364 (46%) cases were marked right, 107 (13.5%) were marked wrong, 240 (30.4%) were marked ambiguous and 79 (10%) were missing. In the identification task 598 (75.7%) cases were marked right, 171 (21.6%) were marked wrong and 21 (2.7%) were missing. Table 6 compares the frequency of right and wrong answers in the production task with their expected frequencies. Right

Wrong

Ambiguous

OALD

70

£=72.2

25

£=21.2

46

£ = 47.6

LDOCE

76

Ε =12.2

22

£=21.2

43

£=47.6

COBUILD

71

£=74.2

23

£=21.8

51

£=49.0

CIDE

83

Ε =73.2

13

£=21.5

47

£ = 47.6

COD

64

Ε =72.2

24

£=21.2

53

£=47.6

Table 6: Results for the production task Expected frequencies (£) are calculated by multiplying column totals with row totals and dividing by the total number of cases (711). Differences are not statistically significant ( d f = 8, χ 2 = 8.34),3 but the greater difference between the actual and expected frequencies of CIDE right and wrong answers should be noted. Ambiguous sentences are fairly evenly distributed across all five dictionaries at close to their expected frequencies. Table 7 compares the frequency of right and wrong answers in the identification task with their expected frequencies. Again, differences are not statistically significant (df = 4, χ 2 = 5.14), but the greater difference between the actual and expected frequencies of CIDE right and wrong answers should be noted

equal interval between 1, 2. 3, 4 and 5 is not assumed. For this reason means are calculated from ranks, rather than from the raw scores. 1

3

ρ is the probability that these differences are due to chance fluctuations caused by unknown variables. A probability of .01 means that there is a 1% likelihood that the difference occurred purely by chance. A probability of .005 means that there is a .5% likelihood. In both cases this means that it is highly likely that the different levels of perceived helpfulness were due to differences in the dictionary entries, rather than random variation. 2

χ = chi square. df= degrees of freedom, calculated for χ

2

by multiplying the number of rows in the table,

minus one, with the number of columns in the table, minus one. It is necessary to know the degrees cf freedom in order to look up the significance of χ 2 in a statistical table.

168

Hilary Nesi Right

Wrong

OALD

119

£ = 119.8

35

E = 34.2

LDOCE

120

£ = 121.3

36

£=34.7

COBUILD

116

£=119.0

37

£=34.0

CIDE

129

E = 119.0

24

£=34.0

COD

114

£ = 119.0

39

£ = 34.0

Table 7: Results for the identification task Statistical significance is recorded when the results of the production and identification tasks are combined and ambiguous sentences, which are not an indicator of comprehension, are discounted (p < .05, df= 4, χ 2 = 11.02). This is shown in Table 8. Right OALD

189

£ = 193.2

Wrong 60

£=55.8

LDOCE

196

£ = 197.1

58

£ = 57.0

COBUILD

187

£ = 191.6

60

£=55.4

CIDE

212

£ = 193.2

37

£=55.8

COD

178

E= 187.0

63

£=54.0

Table 8: Combined results for the production and identification tasks Greatest variation between actual and expected frequencies are recorded for CIDE, indicating that subjects interpreted CIDE entries more successfully than entries from the other four dictionaries. Little correlation was recorded between initial familiarity with the headword and sentence production (0.08), and only slightly more between initial familiarity and picture identification (0.09, ρ = < .01)) There was a positive correlation between initial familiarity and the perceived helpfulness of the dictionary entry (0.22 ρ = < 001), the perceived helpfulness of the dictionary entry and correct sentence production (0.24 ρ = < 001) and the perceived helpfulness of the dictionary entry and correct picture identification (0.21 ρ =