Introduction to the Taxometric Method: A Practical Guide 9780805847499, 0805847499

Citation preview

10

1%

Location 16 of 7927

1.5

INTRODUCTION TO THE TAXOMETRIC METHOD A Practical Guide

1%

Location 17 of 7927

INTRODUCTION TO THE TAXOMETRIC METHOD A Practical Guide

John Ruscio Elizabethto wrz College Nick Haslam Elizabethtown College Ayelet Meron Ruscio Elizabethto wn College

~ l Routledge I~

laylor &.trancis Group Ne.v Yori< loodoo

1%

Location 19 of 7927

~Ve

dedicate this book to the 1ne1r101y ofPaul Everett Meehl (1920-2003), pioneer o_fthe taxometric method, champion ofphilosophical clarity and scientific rigor, and a continuing insJJiration to each o_fus.

1%

Location 31 of 7927

Copyright© 2006 by La,vrence Erlbaurn Associates, Inc. All rights reserved. No part of this book nu\y be reprod uced in any forrn, by photostat, n1icroforrn, retrieval systen1 , or a n y oth er rneans, \vithout the prior \vritten perrnission of the publisher.

First published by La,vrence Erlbaun1 Associates , Inc., Publishers JO Industrial Avenue Mah\\·ah, Ne'v Jersey 07430

'fhis edition published 2012 by Routledge

Routledge

Routledge

'faylor & Francis Group

1'aylor & Francis Group

52 Vanderbilt Avenue

27 Church Road, Hove

Nevv York, NY 10017

East Sussex BN3 2FA

Cover design by Kathryn Houghtaling Lacey

Library of Congress Cataloging-in-Publication Data Ruscio,John.

Introduction to the taxornetric 111et hod: a practical g uide I John Ruscio, Nick Haslarn, Ayelet Meron Ruscio. p. ClTI. Includes b ib liographical referen ces (p.). ISBN 0-8058-4749-9 (alk. paper} ISBN 0-8058-5976-4 (pbk.: a lk. paper} I. Nun1erical taxonon1y. I. Hash1n1, Nick, 1963- II. Ruscio, Ayelet Nleron.

Ill. Title.

1%

Location 36 of 7927

QH83.R865 2006

578.01'2-dc22

2006040044 CIP

1%

Location 54 of 7927

Contents

Preface I

rNTRODUCTION AND BACKGROUND

l

Introduction Plan of the Book Conceptual Background Distinguishing Benveen Taxa and Dimensions

So1ne Possible Misunderstandings About ·raxa and Dimensions Conclusions 2 VVhy Latent Structure .Matters Classification Diagnosis Assessn1ent Research Causal Explanations Lay Conceptions of Important Constructs Conclusions 3 ·rhe Classification Problem

1%

Location SS of 7927

The C lassification Proble1n Bin1odalitv Finite M'L'Xture Modeling Cluster Analvsis Latent C lass Analvsis Dimcat Searching for Multiple Boundaries ·rhe 1'axometric Method Inferential Frame\.vorks for the 'faxo1netric Method Conclusions II TAXOJ\lfETRIC METl-100 4

Data Requirements for 1'axometrics Sampling Considerations Indicator Considerations Evaluating the Data by Generating En1pirical Sa1npling Distributions Conclusions

5 ·raxo1netric Procedures I: MAXSLOPE, MAMBAC, and L-Mode

l\1AXSLOPE l\tlANIBAC L-Mode (~onclusions

6

·raxon1etric Procedures II: MAXCOV and MAXEIG The General Covariance Mixture 1' heore1n MAXCOY MAXEIG Evaluating the Accuracy of the Base Rate Classification ·rechnique Blending Elements of the 'fraditional lVIAXCOY and JVIAXEIG Procedures Conclusions

2%

Page vii of 347 • Location 74 of 7927

7 Consistencv Tests Perfrir1ning ·raxometric Procedures Multiple 'I'iines in l\1ultiple Ways Examining Latent Parameters and Classified Cases Assessing Model Fit A Monte Carlo Studv of MAXCOV Consistencv 'I'ests Conclusions 8 lnterpretational Issues Graphing and Presentation of 'I'axometric Results 1' he Influence of Indicator Skew 'The Influences of Indicator Validity and VVithin-Group C:orrelations lnterpretational Safeguards Conclusions 9 A Taxometric Checklist Question I: Is a 'l'axo1netric Analysis Scientifically Justified? Qpestion 2: Are the Data Appropriate for Taxometric Analysis? Question 3: Has a Sufficient Variety of Procedures Been lmplen1ented Properly? Question 4: Have the H.esults Been Presented and Interpreted Appropriately? Question 5: Are hnplications of the Findmgs Clearly Articulated'? Conclusions Ill APPLICATIONS AND FUTURE DIRECTIONS

ll!. Applications of the 'faxometric Method 'The Latent Structure of Psychopathology Normal Personalitv Other Latent Variables Overvie\V of the Substantive Findings lmplen1entation of the ·raxon1etric Method Conclusions

2%

Page viii of 347 • Location 99 of 7927

11 'The Future ofTaxometrics Constructs in ·rraditional Domains Requiring Further Study Ne>v Domains for ·raxometric Investigation Ne>v Research Questions to Be Addressed Using Taxometrics Methodological Issues for Further Study Conclusions Appendi,x A: Simulating 'I'axonic and Dimensional Co1nparison Data Appendi,x B: Estimating Latent Para1neters and Classifying MAXCOV Appendi,x C: Estimating the ·raxon Base Rate Using MAXEIG References Author Index Subject Index

2%

Page ix of 347 • Location 126 of 7927

(~ases

Using

Preface

In 1962, Paul Meehl published the first in a series of technical reports that introduced a ne\v method for distinguishing categorical and continuous variables. These reports, printed v.rith yello\\' covers and circulated among many researchers, came to be kno>vn as the yellolv 111011s/ers. ln this innovative line of work, Meehl and his collaborators at the University of l\finnesota developed, evaluated, and refined a nu1nber of the data-analytic procedures that constitute Meehl 's taxometric method. As it evolved over the next fe>v decades, investigators began using taxo1netrics to study the latent structure of many constructs, especially in the areas of personality and psychopathology. Rather than follov\ring traditional disciplinary preferences or accepting authoritative pronouncements, researchers using the taxometric method performed empirical tests to detern1ine vvhether the latent variables giving rise to observed data >vere categorical or continuous. ln recent years, the volume of substantive and methodological taxo1netric research has been increasing at an accelerating pace. T'his book gathers together the current state of the art in taxometric methodology, drawing from classic and contemporary sources to provide a comprehensive and accessilble introduction to the method. O ur intended audience includes researchers andl students conducting taxometric studies, journal revievvers and editors evaluating such studies, and indhriduals vvho \\·i sh to make sense of these studies and incorporate taxometric results into their vvork. Interest in the taxometric method has spread to many countries and many disciplines as researchers have turned their attention to the importance of empirically evaluating latent structure a nd the data-analytic approaches for doing so. T he taxometric method vvas developed by psychologists \\'ith expertise in clinical and quantitative psychology, but it is \Veil suited to research in other social and behavioral sciences, physical sciences, ed ucation, biology, and beyond. At many universities, graduate-leve] courses involving psychological assessment or the

2%

Page xi of 347 • Location 138 of 7927

classification of nlental disorders have begun to incorporate instr uction in the taxo1netric method; in sorne cases, entire courses are being developed to train students in taxometric methodology. We cover a broad range of analytic techniques, describing in detail their logic and i1nplementation as v.rell as \Vhat is kno\vn about their perforn1ance from systen1atic study. We illustrate the application of taxometric analyses using a nu1nber of data sets and provide guidelines for the interpretation of results. Our overarching goals throughout the book are conceptu a l clarity, niathematical rigor, and accessibility to a \vide audience that includes researchers ne\v to the taxometric method as well as readers \Vho are already familiar \Vith so1ne of the senunal \vork in this area . In a fe\V p laces, tech1ucal 1naterial is p laced in an appendix to facilitate an understanding of the important concepts \Vithout getting lost or sidetracked in details. We recommend that readers \Vho initially bypass the appendixes revisit them once they firmly grasp the relevant issu es. 1'his book is organized into three parts. 'The three chapters in Part l introduce background 1naterial essential to understanding the research problems that the taxometric method \vas designed to address. In chapter 1, \Ve articulate the distinction benveen categor ical and continuous data structures and discuss many potential misunderstandings of this distinction. In chapter 2, v.re review so1ne of the reasons that it is iinportant to stu dy latent structure and explain hovv such studies can advance basic and applied science. In chapter 3, \Ve discuss several

methods that have been developed to distinguish categorical and continuous structure and describe k ey featu res that niake the taxometric niethod an especially attractive tool for making this distinction. 'I'he SL'< chapters in Part II cover taxometric 1nethodology. l n chapter 4, \Ve present the data requirements of taxometric analysis and introduce a technique for empirically evaluating the adequacy of data for p lanned analyses. In chapters 5 and 6, we focus on the nuts and bolts of the primary taxometric procedures. Vve discuss the logic of each procedure, revie\V key i1nplementation decisions, discuss the factors that can influence results, and ill ustrate how each procedure is performed through analyses of illustrative data sets. l n chapter 7, \Ve offer su ggestions for choosing a set oftaxometric procedures for a particular study and discuss strategies for obtaining additional evidence to examine the consistency of results. In chapter 8, \Ve consider factors that can lead to interpretational ambiguity or misleading impressions and highlight methodological safeguards

3%

Page xii of 347 • Location 154 of 7927

that can be used to prevent erroneous conclusions. Finally, in chapter 9, we vvork through a checklist of conceptual and methodological issues that \ ·V e believe should be considered carefully and addressed explicitly in any taxometric investigation. 'The hvo chapters in Part Ill of the book revievv applications of the taxometric method and promising directions for fi.1tu re taxo1netric research. In chapter IO, \Ve report the conclusions of published taxometric investigations and assess the \Vays in \Vhich the taxometric method has been in1ple1nented. We offer general observations about the findings yielded by taxometric studies and note changes in the iinple1nentation of the method over time. In chapter II, vve explore questions central to the conduct of taxo1netric research in the years ahead, including \vhich constructs and research domains are in particular need of taxometric investigation, ho\\' taxometric research might be nlost profitably conducted, and ho\v the method might be evaluated, refined, and strengthened. We outline \vhat \Ve believe to be especially profitable avenu es for future study, highlighting the primary challenges and promises that \Ve foresee in this exciting, rapidly gro\\ring research area. Although Meehl launched the taxometric method more than four decades ago, its popularity is a relatively recent pheno1nenon. In particular, the empirical evaluation of many iinportant 1nethodological issues is still in its infancy. Contributors to this literature vary vvidely in their \villingness or reluctance to

endorse specific approaches or to provide guidelines for ta."ometric research on the basis of vvhat is often extremely limited information. We have made every effort to revievv the available options as con1prehensively as possible and to describe the rationale for each a lternative. VVe are explicit about the sou rce of the recommendations that \Ve offer, vvhether they stem from systen1atic study, preliminary testing, or o ur experience in performing and reviewing taxometric studies. We believe that it vvould be pre1nature to devise a one-size-fits-a ll template for taxometric investigations. Instead, \Ve advocate a more flexible approach that balances the available empirical evidence with reasoned judgments. Our goal is to improve a reader's ability to make in.formed decisions when conducting, revie,ving, or reading taxometric studies. ·r,vo addition.al features of this book are \Vorthy of note. First, a unifying theme of our approach to taxometrics is the use of empii·ical sampling distributions. Specifically, to help determine vvhether data are acceptable for analysis as vvell as

3%

Page xii of 347 • Location 173 of 7927

to help interpret results, we reco1n1nend that investigators generate and analyze categorical and continuous co1nparison data sets. By doing this in a \.v ay that reproduces important aspects of a unique set of research data, one can ask and answer the question, Ho"'' "'' ould results differ if the data vvere categorical versus continuous? Although simulation studies can and should be perforn1ed to help address this question, the l\lfonte Carlo literature on the taxometric method is sparse. Moreover, simulation studies often involve idealized data that differ in critical vvays from research data, and virtually none of the choice points involved in implementing taxo1netric procedures has been studied systematically. ·ro supplement these gaps in the literature, \Ve recommend taking advantage of a "bootstrapping" approach that is increasingly popular for 1nany types of data analysis. rrhe basic idea is to tailor a small-scale simulation study to the conditions present in a particular investigation, including its unique configuration of data parameters and the particular \Vay in vvhich one or more analytic procedures vvill be perfor1ned. This approach combines rigor and feasibility in an informative and efficient manner. \/\/e explain hovv to use empirical sampling distributions in taxometric studies, emphasizing and illustrating the power of the approach at nlany points in the book. A second feature is that \.Ve provide a suite of taxon1etric programs \Vritten in R, a powerful and freely available data-analytic package. Our programs \\'ere used to perform a ll of the analyses presented in this book, and they can be used to generate empirical sampling distributions. 'The current version of R, our programs, and a detailed manual are provided on the accompanying CD-ROM vvhich can be found at \V\V\v. routledge.con1/ 9780805859768. Because these programs continue to evolve over time, updated versions are available on a companion Web site 1naintained by Lawrence Erlbaum Associates. We are grateful to many people for their contributions to this book. \/\/e vvould like to thank Erlbaum senior editor Debra Reigert for her guidance in shepherding this project tllrough the revievv process. Debra's keen sense for the strengths and vveaknesses of the initial proposal and successive drafts proved invaluable in successfi.tlly revising and improving this \\'ork. We also thank the revie\vers vvho cormnitted considerable time and energy to critique drafts of some or all chapters of this book: Scott Acton, Rochester Institute of Technology; 1' imothy Brovvn, Boston University; David H. Gleaves, University of Canterbury; Eric Knowles, University of Arkansas; Todd Little, University of Kansas; and

3%

Page xiii of 347 • Location 192 of 7927

David Marcus, University of Southern Mississippi. 'fheir detailed co1n1nents and constructive criticism led u s to rethink many issues and revvork 1nany sections of the book. Finally, vve are indebted to two colleagues vvho scrutinized a draft of this book for clarity of presentation: Michael Suvak, Departn1ent of Psychology, Boston University; and Eric Kuhn, National Center for P1' SD, Palo Alto Veterans Affairs l\tledical Center. Michael and Eric took this charge seriously and provided us \vith extremely helpful feedback. Of course, any fla\\•s that remain in the book despite the efforts of a ll these individuals to set us straight are our responsibility.

3% Pagexivof347 • Location211 of7927

PART ONE

INTRODUCTION AND BACKGROUND

3% Pagel of347 • Location216of 7927

CI-IAPTER ONE

Introduction

A graduate student once sought refuge fron1 his dissertation research by taking a vacation to India. Hoping to clear his mind of statistics and bigcity pressures, he \VOund up in a houseboat on a remote lake in Kashnur, a peaceful spot for solitary reflection. For several days, he \vas the only Westerner in the vicinity, and he felt distinctly isolated. 'fhen a young Mexican man arrived, another student, and took up residence on the neighboring boat. One evening, vvatching the sunset over the Hin1alayas, conversation turned to their \vork. Apologetically, our hero said he •vas conducting some obscure quantitative research on how to determine •vhether categories exist in psychological data sets. "Sounds like taxo-metrics," his ne\\' friend chimed in. It may be an exaggeration to say iliat taxometrics has reached every corner of tile globe or iliat it has become a corrunon topic of conversation, but it is undeniable that the popularity of this analytic approach has increased substantially in recent years (Hasla111 & Kim, 2002). The volume of psychological research e111ploying taxometric procedures is growing rapidly, and these procedures are beconung standard 111aterial in graduate-level statistics courses. The taxomeu·ic metl1od is being brought to bear on an increasing range of research questions and problems, and the method is undergoing rapid evolution, evaluation, and refinement. But vvhat is taxon1etrics? Our intention in vvriting iliis book is to ans•ver this question in a \vay that is conceptually clear, tl1eoretically compelling, and-most important-practically useful. On first exposure to taxometrics, many novice researchers find it some\.v hat forbidding: T'he ternlinology can see111 abstruse and specialized, the procedures quite different fro111 familiar analytic approaches, tlie interpretation of

3%

Page 3 of 347 • Location 218 of 7927

results for complex data sets hazy, and the implications of the findings difficult to infer. This volun1e aims to den1ystif)r ta.xometric research and make it 1nore accessible to a wider audience v.rithout sacrificing either precision or rigor. The book is not "'Taxo1netrics for Dummies," but a clear staten1ent of hov.r the taxometric method can be used appropriately and fruitfully to resolve important theoretical and applied questions in the behavioral sciences. Our goal is to leave readers not only vvith a solid understanding ofhovv good taxon1etric research may be conducted, but also vvitth a sense for the possibilities afforded by the method and a guide for putting these possibilities into practice.

PLAN OF THE BOOK 'flus volun1e is divided into three parts. Part 1, beginning with this chapter, lays out foundational issues in taxometrics, providing a rationale for the method and developing a conceptual context for later methodological material. We discuss the fundamental question that the taxometric 1nethod V.' as designed to ans>ver, the latent structures that are distinguished by the method, and the relevance of this structural distinction for theory, research, and practice. We then discuss the nature of the classification. proble1n in behavioral science, revie>v the challenges faced by classification researchers in behavioral disciplines, and introduce the

taxometric n1ethod as a pronusing rvay to meet these challenges. Part 11 includes an extended introduction, description, and demonstration of the taxometric method. 'fhese five chapters present an in-depth tutorial for conducting taxometric analyses, using an approach grounded in state-of-the-art e1npirical and simulation research. All chapters are written >vith an eye tov.'ard offering practical guidance on the real problems that behavioral researchers face, basing concrete suggestions on nlathe1natical and empirical grounds v.rhen these are available and on our observations and experience >vhen they are not. The chapters lay out the data require1nents for taxometric studies, present guidelines for conducting the five most vvidely used taxometric procedures, and demonstrate hovv the findings of multiple procedures can be integrated and tested for consistency- a hallmark of the taxometric method. Special attention is given to interpreting the output of taxometric analyses, focusing on the factors that can influence the accuracy of structural inferences. 1' he final chapter provides a

4% Page4of347 • Location 233 of7927

comprehensive, step-by-step checklist that can be consulted to ensure that a taxo1netric study is properly conducted and reported. 'fhroughout Part II, ~·e emphasize the extent to which rigorous research provides a foundation for making informed choices vvhen selecting or iinplementing taxometric procedures and consistency tests as ~·ell as \vhen interpreting their output. Part Ill con cludes the book by considering \vhat has been done, and ~·hat remains to be done, \vith the taxo1netric method. Although this volume is chiefly a guidebook for conducting ne\V taxometric investigations, ~·e believe that it is important for researchers to have a clear understanding of ho\v previous studies have iinple1nented the taxometric method and ~·hat these studies have found. Such understanding not only provides an intellectual context for future studies, but a lso suggests ho\v researchers can build on existing vvork in more methodologically rigorous ~·ays. T'o this end, ~re systematically revie•v the extant taxometric literature and identify pro1nising directions for future \Vork. Our revie•v highlights the range of constructs that have received taxometric scrutiny, summarizes the investigators' conclusions about the latent structure of these constructs, and examines ho~· taxometric practices and conventions have evolved over the past quarter century. \11/e then focus on future priorities for taxometric research, highlighting several unresolved methodological questions, suggesting scientific applications of the nletl1od that have yet to be fully exploited, and identifying promising psychological constructs and domains that have not yet

been explored in taxometric studies. We hope that this discussion gives nevv researchers, in particular, an inviting sense of the rich and largely untapped possibilities of taxometric investigation, motivating then1 to explore these possibilities for themselves and to add their contributions to the gro~·ing taxometrics literature.

CONCEPTUAL BACKGROUND Fundamentally, taxometrics is a ll about the nature of variation. It begins \vith the simple observation that not all differences are alike. 'fhe differences betvveen cats and dogs are not the same as the differences bet\\•een hot and cold objects. 'The differences betvveen gold and silver are distinct from the differences betvveen large rocks (e.g., boulders) and sn1all rocks (e.g., pebbles). Distinctions betvveen

4% Page4of347 • Location252of7927

branches of living organisms represent differences in quality or kind-at least \vhen the branches represent high-level groupings such as kingdcnns, phyla, classes, or orders, and sometimes less so \vhen the branches represent low-level groupings such as genera or species-as do distinctions betvveen chemical elements. By contrast, differences of ternperature or size represent differences of quantity or degree. Some things in the \Vorld see1n to fall into discrete categories. For exa1nple, an animal may be a fish or an insect, but it cannot be both a fish and an insect. Other things fall a long a sea1nless dimension, differing only in their magnitude. For example, a lthough a line can be drawn to distinguish very tall people frcnn all other individuals, no one \vould vie\v this line as anything but an arbitrary slice a long an unbroken continuum of human height. Paul Meehl (1992, 1995a), vvho created the ta.xometric method described in this book, is largely responsible for bringing this distinction betvveen differences in kind and differences of degree to the attention of psychologists. Ho\vever, even before this titne, such concepts had been floating around the discipline under several other guises. Depending on the context, this distinction has been fra1ned as one of cate