Exploring Data Analysis: The Computer Revolution in Statistics [Reprint 2020 ed.] 9780520338210

Citation preview

EXPLORING DATA ANALYSIS

Exploring Data Analysis The Computer Revolution In Statistics Edited by W. J. DIXON Department of Biomathematics University of California, Los Angeles

and W. L. NICHOLSON Battelle Pacific Northwest Laboratories and National Bureau of Standards

UNIVERSITY OF CALIFORNIA PRESS

Berkeley

Los Angeles

London

University of California Press Berkeley and Los Angeles, California University of California Press, Ltd. London, England ISBN: 0-520-02470-2 Library of Congress Catalog Card Number: 73-78549 Copyright © 1974 by The Regents of the University of California Printed in the United States of America

CONTENTS Prcfctcs

xviii

W. J. D i x o n Chapter 1

1

A D V A N C E D B R E A S T CANCER DATA J a m e s D i c k e y and Judy Walrath ( D i s c u s s i o n : R . M . E l a s h o f f , R. A. K r o n m a l and J. Tukey)

59

79

Chapter 2 BLOC VOTING IN THE UNITED NATIONS J. A. Hartigan ( D i s c u s s i o n : L. A. J a e c k e l and R. M o o r e ) Chapter 3

109 U3

CRAWFORD HILL R A I N F A L L D A T A L o u i s A. J a e c k e l and John D. Gabbe ( D i s c u s s i o n : R. M o o r e , J. D i c k e y and J. Tukey) Chapter 4

177 181

EEG F R E Q U E N C Y DISTRIBUTION DATA M.R. Mickey ( D i s c u s s i o n : D. M a r t i n and J. Tukey) Chapter 5

222 229

ROBUSTNESS STUDY ANALYSIS (A F u r t h e r A n a l y s i s of the F i r s t P h a s e of the P r i n c e t o n R o b u s t n e s s Study; E x a m p l e s of L e s s Standard T w o - W a y Table A n a l y s i s ) John W. Tukey ( D i s c u s s i o n : J. Hartigan and H. L. L u c a s )

v

294

Contents

Chapter 6

313

ANALYZING A SERIES OF SOIL F E R T I L I T Y E X P E R I MENTS FOR PREDICTION F . B . Cady, R. L. A n d e r s o n and D. M. A l l e n ( D i s c u s s i o n : R. M. E l a s h o f f and J. Tukey) Chapter 7

325 331

EVOKED O F F R E S P O N S E TO AN AUDIO TONE D. C. M a r t i n and H. L. L u c a s ( D i s c u s s i o n : M. R. M i c k e y and J. Tukey) Chapter 8

355 365

THE USE OF DENSITY ESTIMATES B A S E D ON ORTHOGONAL EXPANSIONS R i c h a r d A. K r o n m a l and M i c h a e l T a r t e r (Discussion:

D . M . A l l e n and J. Tukey)

Citation Index

396 ¿ni

vi

LIST OF TABLES Chapter 1 1. The weighted likelihood ratio f o r equal nonfailure rate f o r adrenalectomy and hypophysectomy, subsample of mastectomy-negative cases. 2. The weighted likelihood ratio f o r equal nonfailure rate for adrenalectomy and hypophysectomy, subsample of mastectomy-negative premenopausal-when-first-seen patients. 3. P e r f o r m a n c e of discriminant analysis procedures. (Discussion: R. M. Elashoff) 1. 2. 3. 4. 5.

The prediction of P. Classification of cases by one multiple logistic. Type of operative treatment. Classification of cases by discriminant function. Ball smoothing classification.

Chapter 2 1. Vote counts in the General Assembly, 1968. 2. Selected contingency tables, 1968. 3. Clustering countries and issues: votes of 19 selected nations on 12 selected issues, General Assembly, 1968. 4. Amalgamation distances. 5. Tree diagram of cluster based on weighted average algorithm (12 variables). 6. Clusters of issues with distance = 100 (1 - ABS (correlation)). Based on 19 countries, amalgamating clusters in order of average distance between them. 7. Direct clustering of U.N. data. F r o m initial marginal t r e e s in tables 1 and 3. 8. Direct clustering of data on agreement with USSR. 9. Clusters of countries based on weighted algorithms. 10. Tree diagram of cluster based on weighted averages algorithm (50 variables). 11. Direct clustering of data on agreement with USA. 12. Direct clustering of U.N. votes.

vii

Chapter 3 1. E s t i m a t e d d r i f t f o r r a i n s t o r m s in 1967. 2. E s t i m a t e d d r i f t s f o r the quadrants of the g r i d for two r a i n s t o r m s in 1967. 3. R e s u l t s of fitting attenuation to r a i n r a t e . 4. Gauge n u m b e r s , n o m i n a l grid c o o r d i n a t e s and r a i n r a t e s . 5. S e l e c t e d t i m e - s m o o t h e d data, f i t t e d v a l u e s and r e s i d u a l s . 6. P a r a m e t e r s , d e s c r i p t o r s and other s t a t i s t i c s f r o m the i n d i vidual f i t s t o e q u a l l y s p a c e d s c a n s . 7. Standard e r r o r s and c o r r e l a t i o n s f o r the c o e f f i c i e n t s of s c a n 126. 8. P a r a m e t e r s and other s t a t i s t i c s f o r the t i m e - d e p e n d e n t f i t . 9. S o m e d e s c r i p t o r s of the t i m e - d e p e n d e n t f i t . 10. Mean r e s i d u a l s o v e r all 49 s c a n s in t a b l e 5. Chapter 4 1. E x p e r i m e n t s in f i r s t data s e t . 2. D i s c r i m i n a n t a n a l y s i s c l a s s i f i c a t i o n of l a s t e x p e r i m e n t on b a s i s of f i r s t and f i r s t on b a s i s of l a s t . 3. D i s c r i m i n a n t a n a l y s i s c l a s s i f i c a t i o n of one c a t on b a s i s of data f r o m two other c a t s . 4. R e s u l t s of fitting r e s p o n s e c u r v e to high d o s e p h y s o s t i g m i n e , cat No. 9. Chapter 5 (Discussion:

J. Hartigan)

1. T w o - w a y c l u s t e r i n g of data u s i n g 65 e s t i m a t o r s , t i o n s and the 10% e r r o r e s t i m a t e .

18 s i t u a -

Chapter 6 1. The independent v a r i a b l e s and the e s t i m a t e d p a r t i a l r e g r e s s i o n c o e f f i c i e n t s f o r the f u l l and r e d u c e d m o d e l s .

viii

Chapter 7 1 and 2. E l e c t r i c a l a c t i v i t y of t r e a t m e n t f r o m s t r i p c h a r t r e c o r d e r showing (A) off r e s p o n s e and (B) on r e s p o n s e . 3. A v e r a g e r e s p o n s e c u r v e with a two s t a n d a r d d e v i a t i o n conf i d e n c e i n t e r v a l (1000 Hz sine w a v e s t i m u l u s ) . 4 , 5 and 6. A v e r a g e e v o k e d r e s p o n s e plots showing s t a n d a r d d e v i a t i o n , two s t a n d a r d d e v i a t i o n , and two s t a n d a r d d e v i a t i o n of the m e a n e s t i m a t e s of the c o n f i d e n c e i n t e r v a l . 7. A v e r a g e evoked r e s p o n s e and two s t a n d a r d d e v i a t i o n c o n f i d e n c e i n t e r v a l with an e l e v e n t e r m t r u n c a t e d F o u r i e r a p p r o x i mation. (Discussion:

M. R . M i c k e y )

1. E v o k e d off and on r e s p o n s e to an audio tone a r r a n g e d a c c o r d ing to g r a d a t i o n of r e s p o n s e . Chapter 8 1. E s t i m a t e of d e n s i t y f o r s a m p l e of s i z e 200 f r o m N(0, 1). 2. E s t i m a t e of c u m u l a t i v e d i s t r i b u t i o n f o r s a m p l e of s i z e 200 f r o m N(0, 1). 3. E s t i m a t e of c o n t o u r s of p r o b a b i l i t y d e n s i t y f o r N(0, 1 , 0 , 1 , . 8). 4. D e n s i t y e s t i m a t e f o r r e s t i n g h e a r t r a t e f o r the patient g r o u p . 5. D e n s i t y e s t i m a t e f o r m a x i m u m h e a r t r a t e f o r the patient g r o u p . 6. E s t i m a t e of b i v a r i a t e d e n s i t y of m a x i m u m and r e s t i n g h e a r t r a t e in the patient g r o u p . 7. C o n t o u r s of the e s t i m a t e d p r o b a b i l i t y d e n s i t y f o r m a x i m u m and r e s t i n g h e a r t r a t e in the n o r m a l g r o u p . 8. C o n t o u r s of the e s t i m a t e d p r o b a b i l i t y d e n s i t y f o r s y s t o l i c and d i a s t o l i c blood p r e s s u r e f o r the patient g r o u p . 9. E s t i m a t e of c o n t o u r s f o r s a m p l e f r o m b i v a r i a t e n o r m a l with m e a n s , v a r i a n c e s , and c o r r e l a t i o n e q u a l to the v a l u e s of t h e s e f o r the d a t a shown in f i g u r e 8. 10. E s t i m a t e d c o n t o u r s of the p r o b a b i l i t y d e n s i t y f o r s y s t o l i c and d i a s t o l i c blood p r e s s u r e in the n o r m a l g r o u p . 11. S c a t t e r g r a m of s y s t o l i c v e r s u s d i a s t o l i c blood p r e s s u r e f o r the n o r m a l g r o u p .

ix

LIST OF FIGURES Chapter 1 1. F i r s t Look At Graphs (FLAG) output. 2. S u b s a m p l e H i s t o g r a m s On P l o t s (SHOP) output. 3. S m o o t h e d d i s c r i m i n a t i o n p r o b a b i l i t y e s t i m a t i o n output. Chapter 2 1. H i s t o g r a m s of counts of voting c a t e g o r i e s , o v e r 50 q u e s t i o n s . Chapter 3 1. The r a i n gauge n e t w o r k . 2. I s o m e t r i c plots of r a i n f a l l s (a) 7 / 2 8 A , and (b) 7 / 1 1 . 3. E x a m p l e s of the p a i r w i s e c o r r e l a t i o n s a s f u n c t i o n s of l a g , for s t o r m 7/25. 4. R e s u l t s of "pseudojackknife" for the r a i n f a l l of 7 / 2 5 . 5. D r i f t v e c t o r s for e a c h of the 42 5 x 5 s q u a r e s , a r r a n g e d a c cording to p o s i t i o n of the s q u a r e ( s t o r m 7 / 2 8 A ) . 6 . P r e d i c t e d v a l u e s of attenuation f r o m fitting the r a i n r a t e s of g a u g e s 9, 17 and 25 for r a i n f a l l 7 / 2 8 A . 7. P r e d i c t e d v e r s u s o b s e r v e d attenuation f r o m fit of r a i n r a t e s of g a u g e s 9, 17 and 25 to the attenuation for r a i n f a l l 7 / 2 8 A . 8. R e s i d u a l s of the f i t of r a i n r a t e s of g a u g e s 9, 17 and 25 to the attenuation for r a i n f a l l 7 / 2 8 A . 9. N o r m a l plot of r e s i d u a l s f r o m the fit to 49 t i m e s m o o t h e d scans. Chapter 4 1. P l o t f r o m p r o g r a m BMD07M showing c l u s t e r i n g of EEG a m p l i tude p r o f i l e s . 2. R e s p o n s e to high d o s e p h y s o s t i g m i n e , cat No. 9. 3. C h a r a c t e r i s t i c p r o f i l e s of high d o s e p h y s o s t i g m i n e , cat No, 9. 4. A v e r a g e s of c a n o n i c a l v a r i a b l e s for d o s e r e s p o n s e , and c h a r a c t e r i s t i c p r o f i l e s c o r r e s p o n d i n g to the c a n o n i c a l a x e s . 5. C a n o n i c a l v a r i a b l e s for d o s e r e s p o n s e .

x

LIST OF EXHIBITS Chapter 4 1. Canonical c o r r e l a t i o n / m u l t i v a r i a t e r e g r e s s i o n analysis using p r o g r a m BMDX75. Chapter 5 1. The data as submitted - and later put in o r d e r . 2. Steps 1, 2 and 3 analyses. 3. Comparison of fit to 20 selected rows (20 selected e s t i m a t e s ) in the 8 x 65 and 8 x 20 of steps 1 and 2 for situation 2. 4. Comparison of " c o l " and " c o l * " for five situations. 5. Residuals f r o m r o w s - P L U S - c o l s fit to the 20 x 8 f r o m situation 16, as was and the signs after change. 6. P a r t s of the stages of analysis of the logs. 7. Comparison of the residuals after two fits of the f o r m common + row + col + r o w * • c o l * . 8. C l a s s i c a l r o w s - T I M E S - c o l u m n s fit to c o l * (eigen) effects. 9. Resistant r o w * * - T I M E S - c o l * * fit to " c o l " 8 - v e c t o r s for selected situations. 10. Intercept and residuals f r o m r e g r e s s i o n . 11. Step 7 analyses. 12. Latent values corresponding to r o w * - T I M E S - c o l * fits in steps 3 and 7. 13. The col* values found in step 7. 14. The row* values obtained in steps 3 and 7 for three f a r - f r o m Gaussian situations. 15. Step 8 analyses. 16. Step 9 analyses. 17. Two 8 - v e c t o r s plotted against (deviate)^. 18. Two m o r e 8 - v e c t o r s plotted against (deviate)^. 19. F i n a l calculation of estimate deficiencies. 20. Adjusted discrepancies (in . 0 0 1 ) for some selected e s t i m a t o r s for situation 22. 21. Adjusted discrepancies (in . 001) for selected e s t i m a t e s in situation 18.

xi

PREFACE

The g e n e s i s of this book w a s a c o n f e r e n c e on s t a t i s t i c a l c o m puting, o r g a n i z e d as a w o r k s h o p , to e x a m i n e the f r o n t i e r s of data a n a l y s i s b a s e d on c o m p u t e r u s e . It was held in the Health S c i e n c e s Computing F a c i l i t y (HSCF) at the U n i v e r s i t y of C a l i f o r nia at L o s A n g e l e s in S e p t e m b e r 1971. The o r i g i n a l i m p e t u s for s u c h a w o r k s h o p c a m e f r o m d i s c u s s i o n s with W e s l e y N i c h o l s o n during an international m e e t i n g in London s o m e y e a r s e a r l i e r . We w e r e d i s m a y e d at the c u r r e n t i v o r y tower t r e n d s in s t a t i s t i c s . M i m i c k i n g the m a t h e m a t i c i a n s , s t a t i s t i c i a n s w e r e i n c r e a s i n g l y avoiding the r e a l w o r l d of a p p l i c a t i o n , and w e r e p u r i f y i n g and s e p a r a t i n g the f i e l d f r o m o t h e r s c i e n c e s . The c o n f e r e n c e w a s planned as a c o u n t e r r e v o l u t i o n to that trend. The Health S c i e n c e s Computing F a c i l i t y p r o v i d e d an e x c e l l e n t p l a c e for the w o r k s h o p . The f a c i l i t y is d e d i c a t e d to s e r v i n g b i o m e d i c a l r e s e a r c h through r e s e a r c h in m a t h e m a t i c s , s t a t i s t i c s and c o m p u t e r s c i e n c e . It has an IBM 3 6 0 / 9 1 and n u m e r o u s t y p e w r i t e r , c h a r a c t e r s c o p e , and g r a p h i c s c o n s o l e s s e r v e d by a t i m e - s h a r i n g o p e r a t i n g s y s t e m . The s y s t e m s p e c i a l i z e s in i n t e r a c t i v e s t a t i s t i c a l t e c h n i q u e s and the p r o g r a m s to s e r v e t h e m . Of s p e c i a l i m p o r t a n c e to c o n f e r e n c e p a r t i c i p a n t s w a s the u s e of graphical statistical techniques. P a r t i c i p a n t s w e r e l i m i t e d to a s e l e c t group of p r a c t i c i n g data a n a l y s t s . The p a p e r s p r e s e n t e d r e a l p r o b l e m s and included a d i s c u s s i o n of the p h y s i c a l m e c h a n i s m s involved in g e n e r a t i n g data f o r the a n a l y s e s . With a r e a l p r o b l e m a s a f o c a l point, the a n a l y s e s p u r s u e d the needs of the p r o b l e m r a t h e r than s t r e s s i n g p a r t i c u l a r t e c h n i q u e s of s t a t i s t i c s . But any new t e c h n i q u e s u s e f u l

xiii

Preface

f o r the a n a l y s e s w e r e e m p h a s i z e d , and the d e g r e e to which the d e r i v a t i o n and u s e of the t e c h n i q u e s w a s dependent on the c o m puter was s t r e s s e d . E a c h p a p e r w a s a v a i l a b l e to s e v e r a l c r i t i c s in a d v a n c e of the m e e t i n g . T h e i r c o m m e n t s a r e included in this v o l u m e a s well a s a d d i t i o n a l c o m m e n t s b y the a u t h o r s and other c r i t i c s that d e v e l o p e d during the s e s s i o n s . The c o n f e r e n c e r e v e a l e d m a n y c h a r a c t e r i s t i c s of a d a t a a n a l y s t a t work. In c o n t r a s t to the b i o l o g i s t who e x a m i n e s his data with the cons t r u c t s of his own f i e l d in m i n d , the data a n a l y s t e x a m i n e s the data f o r i t s a p p a r e n t s i m i l a r i t y to a v a r i e t y of s t a t i s t i c a l m o d e l s he h a s in m i n d , letting the r e s u l t s of s u c c e s s i v e a n a l y t i c a l a t t e m p t s guide the d i r e c t i o n he p u r s u e s (and r e f i n e s ) a s he p r o c e e d s . The s t a t i s t i c i a n ' s a p p r o a c h m i g h t b e d e s c r i b e d a s one in which he s t a t e s : "if we a s s u m e n o r m a l i t y , i n d e p e n d e n c e , and p e r h a p s other f u n d a m e n t a l s , then the r e s u l t s i n d i c a t e the v a l i d i t y of c e r t a i n s t a t e d h y p o t h e s e s with a s s o c i a t e d p r o b a b i l i t i e s . " In c o n t r a s t , the data a n a l y s t m a y u s e m a n y of the s a m e t e c h n i q u e s , but he will e x p l o r e ( a l s o with s t a t i s t i c a l t e c h n i q u e s ) the d e g r e e to which t h e s e a s s u m p t i o n s might b e a f f e c t i n g h i s c o n c l u s i o n s , and the c o n s e q u e n c e s to the a p p l i c a t i o n a l f i e l d of d e v i a t i o n s f r o m r e a l i t y in the a n a l y t i c a l a s s u m p t i o n s . The data a n a l y s t s e e m s to b e m o r e involved in e x p l o r a t i o n than in r e f i n e m e n t . He i s s l o w to m a k e a s s u m p t i o n s b e f o r e he e x a m i n e s the d a t a . He i s quite s a t i s f i e d if any a d v a n c e i s m a d e in the p r o b l e m a r e a independent of the s o p h i s t i c a t i o n of the a n a l y s i s , the g o o d n e s s of a g r e e m e n t of his m o d e l , or the p r e s e n t a b i l i t y of the s t a t i s t i c a l a n a l y s i s i t s e l f . He i s quite p r e p a r e d to find that one m i g h t a r r i v e at the s a m e c o n c l u s i o n u s i n g quite d i f f e r e n t r o u t e s and quite d i f f e r e n t t e c h n i q u e s . The data a n a l y s t i s a l m o s t s u r e to have a d e e p i n v o l v e m e n t in c o m p u t e r s s i n c e he r e q u i r e s computing power f o r h i s f r e e d o m to u s e a wide v a r i e t y of t e c h n i q u e s . T e c h n i q u e s and a n a l y s t s a r e not independent. They i n t e r a c t . One obtains a m a x i m u m r e s u l t f r o m i n t e r a c t i o n s r a t h e r than f r o m m a i n e f f e c t s . A p a r t i c u l a r p e r s o n who u s e s c e r t a i n xiv

Preface

techniques m o r e powerfully than someone else m a y obtain b e t t e r a n s w e r s u s i n g t h o s e techniques than o t h e r s can. On the other h a n d , a n o t h e r p e r s o n m a y u s e h i s o w n t e c h n i q u e s a n d do e q u a l l y well, that i s , t h e r e is a n i n t e r a c t i o n in the p r o c e s s . Even when techniques a r e m a t h e m a t i c a l l y equivalent, different a n a l y s t s u s e t h e m i n d i f f e r e n t w a y s . O n e m a y t h i n k a n d do a n a l y s i s of v a r i a n c e , a n d a n o t h e r m a y t h i n k r e g r e s s i o n . They m a y b e doing the s a m e thing but their thought p r o c e s s e s and the w a y t h e y p r o c e e d t h r o u g h t h e a n a l y s i s of t h e p r o b l e m d i f f e r b e c a u s e of t h e w a y t h e y c o n c e p t u a l i z e a n a l y s i s of v a r i a n c e a n d r e g r e s s i o n ; although the l a n g u a g e m a y d i f f e r and e v e n c o m m u n i cation m a y b e difficult, the o v e r a l l a n a l y s e s m a y r e a l l y b e v e r y similar. B y t h e e n d of t h e c o n f e r e n c e i t w a s c l e a r t h a t t h e r e i s a h e a v y i n t e r a c t i o n b e t w e e n a n a l y s t s a n d s c i e n t i s t s i n o t h e r f i e l d s . In m o s t c a s e s , the a n a l y s t h a s b e c o m e v e r y involved with the s u b j e c t m a t t e r of t h e f i e l d ' s b a s i c t h e o r i e s a n d p r o b l e m s . The statistical r e s e a r c h for his data analysis is truly collaborative— i n m a n y c a s e s h e e n l i s t s t h e c o o p e r a t i o n of o t h e r s t a t i s t i c i a n s a s w e l l . T h e s t a t i s t i c a l a n a l y s i s is not s e p a r a t e d and p u r s u e d f o r its m a t h e m a t i c a l e l e g a n c e , r a t h e r it is o r i e n t e d t o w a r d the n e e d s of t h e p r o b l e m . P e r h a p s this " t e a m " w o r k and c o o p e r a t i o n is the m o s t i m p o r t a n t a n d f a r - r e a c h i n g r e v e l a t i o n of t h e c o n f e r e n c e . A s h o r t d e f i n i t i o n of d a t a a n a l y s i s w a s p r o p o s e d a t t h e c o n f e r e n c e : D a t a a n a l y s i s i s t h e a p p l i c a t i o n of o n e o r m o r e t e c h n i q u e s t o a s e t of d a t a s t e e r e d b y t h e p r o b l e m . C o m p u t e r f a c i l i t i e s a t H S C F w e r e a v a i l a b l e to p a r t i c i p a n t s b e f o r e and during the c o n f e r e n c e , and a UCLA "buddy" was a s s i g n e d to e a c h p a r t i c i p a n t to h e l p in a n y w a y n e c e s s a r y . Data p r e s e n t e d at the c o n f e r e n c e is a v a i l a b l e f r o m H S C F in m a c h i n e r e a d a b l e f o r m . A d a t a s e t d e s c r i p t i o n c o n t a i n i n g a t l e a s t a p a r t i a l l i s t i n g of t h e d a t a f r o m e a c h p a p e r is g i v e n i n t h i s b o o k . T h e c o m p u t a t i o n a l s u p p o r t w a s m a d e p o s s i b l e b y g r a n t R R - 3 of t h e B i o t e c h n o l o g y R e s o u r c e s B r a n c h , D i v i s i o n of R e s e a r c h a n d

xv

Preface

R e s o u r c e s of the National Institutes of Health. The c o n f e r e n c e i t s e l f w a s supported b y grant G J - 2 9 8 4 4 f r o m the National S c i e n c e Foundation. A c k n o w l e d g e m e n t s a r e due s e v e r a l m e m b e r s of m y staff for their h e l p with the c o n f e r e n c e and in p r e p a r i n g m a t e r i a l f o r this book. Ed Chen, D o l o r e s A d a m s and E l l e n S o m m e r s a s s i s t e d in p r e p a r a t i o n s f o r and during the c o n f e r e n c e . E l l e n S o m m e r s p r e p a r e d and edited the a s s o c i a t e d data s e t s . Lyda B o y e r e d i t e d , and B e t s y P o t t e r typed the m a n u s c r i p t s . Much of the work o r o r g a n i z a t i o n of the c o n f e r e n c e i t s e l f and w o r k i n g with the authors on the p r e p a r a t i o n of t h e i r m a n u s c r i p t s w a s done by W e s l e y N i c h o l s o n .

W. J. Dixon

xvi

CHAPTER

1

A D V A N C E D B R E A S T CANCER DATA

JAMES DICKEY S t a t i s t i c s D e p a r t m e n t , State U n i v e r s i t y of New York at B u f f a l o and JUDY WALRATH D e p a r t m e n t of E p i d e m i o l o g y and P u b l i c Health, Yale U n i v e r s i t y

The m a j o r i t y of m e d i c a l d a t a - a n a l y s i s p r o b l e m s a r i s e f r o m a p h y s i c i a n ' s hope that h i s r e c o r d s of p a s t c a s e s w i l l y i e l d u s e f u l i n f o r m a t i o n . The r e a l p r o b l e m s a r e m a t h e m a t i c a l l y v a g u e , but tangible: What l e s s o n s a r e to be l e a r n e d f r o m past e x p e r i e n c e f o r future c l i n i c a l p r a c t i c e ? What patient subpopulations have d i s t i n c t i v e b e h a v i o r p a t t e r n s ? What t r e a t m e n t s should be u s e d in what kinds of c a s e s ? In the language of John Tukey (1962, 1970), t h e s e a r e p r o b l e m s of " e x p l o r a t o r y data a n a l y s i s " — p r o b l e m s of how to Find I n t e r e s t i n g R e p o r t a b l e E f f e c t s (FIRE). F I R E p r o b l e m s , h o w e v e r , are not the subject of the bulk of s t a t i s t i c a l t h e o r y , w h i c h i s d e v i s e d f o r A f t e r The R e v e l a t i o n O r d e r l y P i c k l i n g of H Y p o t h e s e s (ATROPHY), and to Guard A g a i n s t S i l l y S e l e c t i o n E f f e c t s by D e f i n i t i o n (GASSED). R e s e a r c h for this study w a s supported by NIGMS-NIH Grant GM 16557.

1

Dickey and Walrath

L i n e a r d i s c r i m i n a t i o n p r o c e d u r e s have not b e e n v e r y p r o d u c t i v e i n r e a l m e d i c a l p r o b l e m s ( R a d h a k r i s h n a , 1964). E v e n t h e F I R E p r o b l e m - m o t i v a t e d s t e p w i s e l i n e a r p r o c e d u r e s ( r e g r e s s i o n and d i s c r i m i n a t i o n ) deliver l i n e a r functions that tend to be a l m o s t m e a n i n g l e s s a s f i n a l a n s w e r s t o p h y s i c i a n s and s t a t i s t i c i a n s a l i k e , e s p e c i a l l y l i n e a r f u n c t i o n s of t h r e e o r m o r e v a r i a b l e s . They m a y , h o w e v e r , be u s e f u l in pointing out the f e w i m p o r t a n t v a r i ables. In t h i s p a p e r we s t r i v e t o c o n c e n t r a t e on F I R E p r o b l e m s of c l i n i c a l - e x p e r i e n c e d a t a , w i t h t h e a i m of c o n t r i b u t i n g t o a g e n e r a l s y s t e m a t i c a p p r o a c h i n v o l v i n g t h e u s e of c o m p u t e r p r o g r a m s a s s t e p s i n a n a n a l y t i c s e q u e n c e . We d i s c u s s e x p l o r a t o r y d a t a a n a l y s i s f o r a n i m p o r t a n t c l a s s of p r o b l e m s — t h e p r e d i c t i o n of a dichotomized treatment-response variable. P r o f . W i l f r i d J . D i x o n ' s ( 1 9 6 9 , 1970) B M D b i o m e d i c a l c o m p u t e r p r o g r a m s are widely used for practical data analysis. Contribut i o n s to a s y s t e m a t i z e d a p p r o a c h , i n s p i r e d b y t h e B M D p r o g r a m s , a r e put f o r t h h e r e , t o g e t h e r w i t h a f e w r o u g h p r e d e c e s s o r F O R T R A N l a n g u a g e p r o g r a m s , and p r o g r a m s n o t y e t a v a i l a b l e . In the following s e c t i o n we i n t r o d u c e , as c o n c r e t e m o t i v a t i o n , t h e w e l l - s t u d i e d ( A r m i t a g e e t a l , 1969) a d v a n c e d b r e a s t c a n c e r d a t a a n a l y s i s , a n d t h e c l i n i c a l - d e c i s i o n p r o b l e m of B u l b r o o k e t a l ( I 9 6 0 ) , and A t k i n s e t a l (1968). E a c h of t h e r e m a i n i n g s e c t i o n s d e s c r i b e s a t y p e of c o m p u t e r p r o g r a m : •

F i r s t L o o k At G r a p h s

•

S u b s a m p l e H i s t o g r a m s O r P l o t s (SHOP);

•

S h o p In F u l l T o t a l i t y ( S I F T ) ; and

•

a d i s c u s s i o n of d i s c r i m i n a n t a n a l y s i s p e r s e , w i t h an e m p h a s i s on r e c e n t n o n p a r a m e t r i c p r o c e d u r e s .

2

(FLAG);

Advanced Breast Cancer

A DATA ANALYSIS PROBLEM The t y p i c a l m e d i c a l d a t a s e t f e a t u r e s a f e w (1 - 10) r e s p o n s e v a r i a b l e s and m a n y (10 - 100) m i x e d - t y p e ( d i c h o t o m o u s t o p r a c t i c a l l y c o n t i n u o u s ) p r e d i c t o r v a r i a b l e s , f o r a p r e c i o u s f e w (10 1000) o b s e r v e d c a s e s . M i s s i n g v a l u e s abound. The d e f i n i t i o n s of i n d i v i d u a l v a r i a b l e s a r e a m b i g u o u s and i l l - c o n c e i v e d . The data e m b o d y h i s t o r i e s of u n d i s c i p l i n e d c l e r k s ' m i s u n d e r s t a n d i n g s . In s h o r t , the s t a t i s t i c s t e a c h e r ' s n i g h t m a r e : i m p e r f e c t d a t a and vague problems. We c o n s i d e r h e r e a d e c i s i o n p r o b l e m in the m a n a g e m e n t of a d v a n c e d b r e a s t c a n c e r , and a r e l a t e d data s e t f r o m Guy's H o s p i t a l , L o n d o n (Atkins et a l , 1968), u n u s u a l f o r the p a i n s taking c a r e w i t h w h i c h it w a s c o l l e c t e d . T h i s c o n c r e t e d a t a a n a l y s i s p r o b l e m i s put f o r t h a s r e p r e s e n t a t i v e of m a n y in b e i n g s u i t e d to a g e n e r a l s y s t e m a t i c a p p r o a c h . T w o h u n d r e d and t e n a d v a n c e d b r e a s t c a n c e r p a t i e n t s w e r e i n c l u d e d in the s t u d y . A p p r o x i m a t e l y t w o - t h i r d s ( 1 3 9 / 2 1 0 ) of t h e m had u n d e r g o n e a t t e m p t e d c u r e by r a d i c a l ( 1 1 6 / 2 1 0 ) or s i m p l e ( 2 3 / 2 1 0 ) m a s t e c t o m y , and t h e n a y e a r o r s o l a t e r had a r e c u r r e n c e of t u m o r g r o w t h l o c a l l y o r at a d i s t a n t s i t e . The o t h e r o n e - t h i r d ( 7 1 / 2 1 0 ) had b e e n f i r s t d i a g n o s e d as a l r e a d y a d v a n c e d . Threef i f t h s ( 1 3 2 / 2 1 0 ) b e g a n the p a l l i a t i v e s t a g e of t h e i r t r e a t m e n t w i t h the a d m i n i s t r a t i o n of h o r m o n e s , w h i c h w e r e u s e f u l in s o m e c a s e s ( 1 7 / 1 3 2 ) f o r up t o one y e a r in c o n t r o l l i n g t u m o r g r o w t h . T h e n it w a s a q u e s t i o n of w h e t h e r or not s u r g e r y s h o u l d b e u s e d to a l t e r the h o r m o n a l e n v i r o n m e n t of the t u m o r s . If s o , w h i c h of two o p e r a t i o n s s h o u l d b e p e r f o r m e d : b i l a t e r a l a d r e n a l e c t o m y w i t h o o p h o r e c t o m y ( r e m o v a l of a l l a d r e n a l s and o v a r i e s ) , or h y p o p h y s e c t o m y ( r e m o v a l of p i t u i t a r y ) . E a c h p a t i e n t u n d e r w e n t a n o p e r a t i o n , about half e a c h kind ( 1 1 5 / 2 1 0 , 9 5 / 2 1 0 ) . F o r o n e - q u a r t e r of the p a t i e n t s ( 5 4 / 2 1 0 ) , the s u r g e r y w a s s u c c e s s f u l ( c o m p l e t e r e m i s s i o n of s y m p t o m s f o r o v e r s i x m o n t h s ) ; f o r another o n e - q u a r t e r (53/210), intermediate r e s u l t s (partial r e m i s s i o n ) ; and f o r the o t h e r half ( 1 0 3 / 2 1 0 ) , f a i l u r e (no i m p r o v e ment). 3

Dickey and Walrath

B o t h s u r g i c a l p r o c e d u r e s a r e r a d i c a l a t t e m p t s to p r o l o n g l i f e . H y p o p h y s e c t o m y i s a m o r e i n v o l v e d and d a n g e r o u s o p e r a t i o n , b u t i t s w h o l e - s a m p l e r e m i s s i o n p e r c e n t a g e s ( 2 8 / 9 5 and 2 4 / 9 5 ) w e r e e s s e n t i a l l y t h e s a m e a s t h o s e f o r a d r e n a l e c t o m y ( 2 6 / 1 1 5 and 29/115). N a t u r a l suggestions for v a r i a b l e s r e l a t e d to s u r g i c a l s u c c e s s include: 1.

m e a s u r e s of t u m o r g r o w t h r a t e a) age of p a t i e n t b) e x t e n t of d i s e a s e at m a s t e c t o m y c) t i m e f r o m m a s t e c t o m y t o r e c u r r e n c e ;

2.

tumor histology;

3.

menopausal status;

4.

h i s t o r y of m a s t e c t o m y ;

5.

s y s t e m i c (hence u r i n a r y ) h o r m o n e l e v e l s .

and

In I 9 6 0 , D r . R . D. B u l b r o o k and h i s c o i n v e s t i g a t o r s at G u y ' s H o s p i t a l d e v e l o p e d a l i n e a r d i s c r i m i n a n t f u n c t i o n of two 2 4 - h o u r u r i n a r y - s t e r o i d l e v e l s , a e t i o c h o l a n o l o n e (E) and 1 7 - h y d r o x y c o r t i c o s t e r o i d (17 OHCS), 80 - 80(17 OHCS) + E ,

(1)

p o s i t i v e v a l u e s of w h i c h t e n d to p r e d i c t f a v o r a b l e r e s p o n s e t o s u r g e r y . A f t e r f u r t h e r p r o s p e c t i v e s t u d i e s , A t k i n s e t a l (1968) r e p o r t e d " t h e d i s c r i m i n a n t f u n c t i o n b y i t s e l f p r o v i d e s an e f f i c i e n t g u i d e to r e s p o n s e t o h y p o p h y s e c t o m y b u t d o e s n o t do s o f o r a d r e n a l e c t o m y in t h i s s e r i e s . " T h e y a l s o f o u n d s m a l l e f f e c t s f o r t h e f a c t o r s 1. c), 3. , and 4. a b o v e . A r m i t a g e e t a l (1969) c a r r i e d out e x t e n s i v e F I R E - l i k e a n a l y s e s of t h e s e s a m e d a t a . F i r s t , e a c h of t h r e e r e s p o n s e v a r i a b l e s w a s

4

Advanced Breast Cancer

d i c h o t o m i z e d and fit by H i l l s ' (1967) s t e p w i s e s ample - s plitting d i s c r i m i n a t i o n p r o c e d u r e for d i c h o t o m i z e d p r e d i c t o r v a r i a b l e s . Then they p e r f o r m e d s p e c i a l a n a l y s e s , e a c h suited to e a c h o r i g i nal r e s p o n s e v a r i a b l e . The r e s p o n s e , a c l i n i c a l a s s e s s m e n t of s u c c e s s (as s u c c e s s , i n t e r m e d i a t e , and f a i l u r e , defined above), w a s d i c h o t o m i z e d into nonfailure and f a i l u r e , and then r e l a t e d to v a r i o u s s e t s of p r e dictor v a r i a b l e s . Our d i s c u s s i o n i s r e s t r i c t e d to this c h o i c e of a d i c h o t o m o u s r e s p o n s e v a r i a b l e and to d i c h o t o m i z e d r e s p o n s e s in g e n e r a l , thus n e g l e c t i n g other important d e v e l o p m e n t s of m e t h o dology, for e x a m p l e , s u r v i v a l - t i m e data. At the s u g g e s t i o n , and through the k i n d n e s s , of P r o f . M a r v i n Z e l e n , a c a r d copy of the A r m i t a g e et al (1969) data w a s obtained f r o m John Copas, and a s l i g h t l y updated v e r s i o n of the o r i g i n a l patient r e c o r d s (including 16 new c a s e s ) f r o m Dr. R. D. Bulbrook. The updated r e c o r d s of all 210 c a s e s are on f i l e at HSCF under the t i t l e "Advanced B r e a s t Cancer Data (J. Dickey). " A c o m p l e t e l i s t i n g of the c a n c e r data in card i m a g e f o r m i s g i v e n in the Data Set D e s c r i p t i o n at the end of this c h a p t e r . This i n c l u d e s a d e s c r i p t i o n of the 50 v a r i a b l e s a s s o c i a t e d with e a c h patient, and, p a r e n t h e t i c a l l y , single word a c r o n y m s which identify v a r i a b l e s . FIRST LOOK AT GRAPHS (FLAG) N e w l y punched data w i l l , with high probability, contain m i s t a k e n v a l u e s appearing as 1.

o v e r p u n c h e s and i l l e g a l c h a r a c t e r s ;

2.

data-to-format mismatches;

3.

n o n s e n s e v a l u e s of a v a r i a b l e a) o f f - r a n g e n u m e r i c v a l u e s b) m e a n i n g l e s s m u l t i p l e - c h o i c e v a l u e s ;

4.

n o n s e n s e c o m b i n a t i o n s of v a r i a b l e v a l u e s , autopsy date p r e c e d i n g date of death; 5

e.g.,

Dickey and Walrath

5.

multivariate outliers;

and

6.

undetectable-per-se mistaken values.

C o m p u t e r p r o g r a m - p r o c e s s i n g s y s t e m s t e n d to a b o r t r u n s when d a t a input c o n t a i n s m i s t a k e s of t y p e s 1 and data-analysis programs abort or deliver unacceptable f r o m input m i s t a k e s of type 3, and l e s s c o m m o n l y , of

program 2. Many output type 4 .

One of the f u n c t i o n s of o u r c o m p u t e r p r o g r a m , F L A G ( G o l d m a n e t a l , 1 9 7 1 ) i s to d e t e c t , and i d e n t i f y b y f l a g g e d output, m i s t a k e n d a t a v a l u e s of t y p e s 1 - 4 without a b o r t i n g o r o t h e r w i s e d i s r u p t i n g p r o g r a m f u n c t i o n s . T h e p r o g r a m m i n g of m i s t a k e s 1 and 2 i s c o n c e i v e d a s i s o l a t e d in s y s t e m - s p e c i f i c s u b p r o g r a m s . T h e p r o g r a m h a s an option f o r a o n e - l i n e - p e r - c a r d p r i n t e r l i s t i n g of the d a t a , a s s u m i n g input i s in s t a n d a r d c a s e w i s e f o r m with a c o n s t a n t n u m b e r of c a r d s p e r c a s e . O p t i o n a l l y , s p e c i a l l i n e s on e a c h page should be u s e d to i n d i c a t e the a c t u a l c a r d c o l u m n s of each data v a r i a b l e . T h e p r i n c i p a l f u n c t i o n of F L A G i s to d e l i v e r c o a r s e p a r a l l e l p l o t s of t h e v a r i a b l e s a g a i n s t s e q u e n t i a l c a s e n u m b e r s with f l a g s f o r m i s s i n g v a l u e s and m i s t a k e s 1 - 4 . F i g u r e 1 d e p i c t s s u c h output for our f i r s t nine p r e d i c t o r v a r i a b l e s . E a c h print column c o r r e sponds to a v a l u e o r g r o u p i n g i n t e r v a l f o r a v a r i a b l e . T h e p r i n t c o l u m n s should be h e a d e d with the a c t u a l d a t a v a l u e s t o o , r a t h e r than as p r e s e n t l y k e y e d . And s p e c i a l p r i n t c o l u m n s should b e u s e d f o r u s e r - p r o g r a m m e d n o n s e n s e c h e c k s ( m i s t a k e of type 4 ) . P r o f . C h e s t e r I . B l i s s s t a t e d in c o n v e r s a t i o n with one of the a u t h o r s t h a t d a t a a n a l y s i s b y d e s k c a l c u l a t o r h a s the a d v a n t a g e of f o r c i n g the a n a l y s t into i n t i m a t e f a m i l i a r i t y with the a c t u a l d a t a v a l u e s . We hope F L A G o f f e r s s o m e of the s a m e a d v a n t a g e s . R u n s , t r e n d s , and m a n y o t h e r u n i v a r i a t e p a t t e r n s a r e e a s y to r e c o g n i z e . T o a l e s s e r e x t e n t , m u l t i v a r i a t e p a t t e r n s c a n be recognized, for example, simultaneously missing values.

6

Advanced Breast Cancer

o z z Lü ÜJ

O ü

03

a (Ö >H Ol

«X 1 L> f J- -4" J-J-

lt\ CM

fc-h (< /Ir H H _J -J H »-z *r a: or UJ w w V) IH M M M M M M HHH z z z z z z z Ntrr

z z r

CM

w w w w w M MW H H z z z z z

T5 15 o ( 5 z z T z M IH IH l-l W W w w W W w w

M CO in

W W CO M M M M Mz z z z C

15 15 15 15 15 Z Z Z Z Z M M IH IH M W W w w w

CM

M IH w M Z Z I Z

15 Z M W W W M CM Z CM

CM L E O O O

CM O

•H

O

F)

CM O

Z

W W MW H Z Z

W W

15

15 7 T-L

Z

H CO

w w M CM NtN

CM

O

W W T/> W H H Z Z

w K-l

W W CO W CO w w w w w w w w w w w w w w w w w w w ro w M I-» M M M K> IH M W W M IH Z Z X Z X z z X Z Z Z Z

O

O

O

O

15

19

15 15

15 15 15

H

L-L

M

H H H

Z

w w

L-L M Z CM Z CM Z CM

O

w w M

CM «H

O

NIVJCO O

O

^

^

IF\ F

O

O

IR\ IF\ IR, ¿t

J-

N M

IA

IN

J-

M J-

^

9

z

W

M

W W

IH IH

Z CM

CM Z

W w M Z

WW w w M M X Z

M H M z Z Z

15 15

Z T 7

z

M

WW

CM

z z

(VI

M

15

o T M

z z

CM

z

IH

w W W w CO R/I H M l-l z z r

IH IH

O

Z z

M M

w w w w MM z x

15 15 I T M

w w w w IH M z Z

W w IH T"

CM

W W

W W

O

M

N

M

M

F V J W N M W O

J-J-

J

W

O

z

W

Z CM £

W W

O

15 15 15 15 15 15 15 15 10 z z z z z z Z Z z H H H H H IH IH IH IH w W w w W w

15 15 15 15 15 CO z Z z z z z l-l H H H H H

15 Z M

w w

W

CM O

M

O Z

O

W W W

W M M

M M

W W

W

CM

( 5 15 15 Z Z Z

15 15 Z Z

M

(M F\J E

IFI

O

O

IR* J - J- J- J - J-

O

IR< ^

CM O

O

E

N N -H

U\ IF> IF> J-

IR\ J - -T

W O

W O

U> J-

M M

M J-

N O

IN

Dickey and Walrath

(VJ m * in O »H (VJ n CM (VI (SI CVI (VJ CVJ (VJ (VI n ro n n n n n ^ T4 H H «H r* vH

N n « H v t H rtrt H

(9 Z M VI in

M

z

19 19 z z M M (/I l/l l/l H HI W tvj r Z

19 T HI VI VI HI

19 Z HI VI VI HI

z z

-t

m

CJ (M

*

«H

*

*

IS z HI in 10 M n Z (VI

*

N

19 19 7 Z IH HI VI VI l/l VI HI M z z

z

19 Z M l/l VI HI z

19 z IH l/l VI HI Z

CM o

19 19 19 19 19 19 19 19 z z Z z z z z z H HI HI M H H H H V) CO 'CO VI VI CO t/1 t/1 1/1 t/1 t/1 t/1 t/1 t/1 t/1 (/) HI HI HI HI HI W HI HI z z z z z z Z Z

19 19 19 ~r Z Z HI M HI t/1 l/l VI l/l VI t/1 HI M HI z z (VJ Z

fsj (\J o

19 19 Z Z HI HI VI VI VI VI Ht HI (VI z (VI Z

to

to to to

OJ lf> If»

r>-

19 Z Ht 00 «0 HI z

^

to

N

o

u\ ifN * *

19 7 HI VI VI HI

19 Z HI V) t/1 M *H Z (VJ Z O z M V)

*

to ro

e

*

z

19 «9 Z y M HI l/l 1/1 1/1 VI HI HI Z T

If»

J"

19 19 Z z HI HI t/1 VI V) CO HI HI z z

19 Z HI VI t/1 HI z

19 z HI V) t/1 HI Z N

19 19 Z Z HI HI VI CO VI V) HI M i l

19 Z HI VI VI M z

19 Z HI VI VI HI z

19 19 z z HI HI VIV) CO VI HI HI z z

19 z HI to CO HI z

19 z HI (0 CO HI z

19 19 z z HI HI to to VI to HI HI z (VI (VJ Z

19 2 H to to HI z

19 19 Z 7 H H to to to to HI HI z z

to

to

19 Z HI l/l t/1 M Z

19 Z HI CO l/l HI Z

19 Z HI t/1 VI HI z

t9 7 HI VI VI HI Z

19 z HI VI VI HI z

to

IT> UN

f\J Od o

-J J"

O

o

o

CT* O

if« tr»

W

(VJ

(SJ o

ir>

J"

lf\ if\ If» UN If» If» lf\

if» rr;

(M

M

19 z HI VI t/1 HI Z

19 19 z Z HI HI to VI CO to M M z tvj Z

o

fo

if»

iC

(O

19 19 z Z HI HI VI VI VI VI M HI Z z

19 Z H4 VI VI HI z

19 z HI VI VI

19 z HI l/l V) HI HI z Z

to

19 19 Z z HI HI to IO to to HI HI tvj Z (VJ z

19 Z HI t/1 VI HI z

19 z HI VI VI HI z

19 z HI VI VI M z

19 19 Z HI HI to to IO to M W z (VI Z (VJ

«H

J"

CM «5-

M

irt pj

t» K 3 -> H «* ll. z o z X tX

>

X o o UJ -J

«

s

at o

«

o • PJ in•

O • •H

CM CM K • •wl

• Ul •H a. Z> •H PJ a • IL TH o X • t•H H P-

in • »-I

(VI • •H

(M •«•I O rH O -H

fw •

•

in

in (VI •

o • I •

• • • • • • • • • • • • • a « * in .i m tu o 0* ' i N

>•

r o tPg o 111 _l C3 4 Z • K O 4 •

•

u< 4 1 n M n

18

Advanced Breast Cancer

p ( D | H) = E ~

,

w

p(D|7T),

TT | -H-

(5)

~ ~

(6)

p(D ~ | H ) = E 7TI, -rlp ( D~| 7 T ~), (D not r a n d o m , n o r c o n d i t i o n e d on, in the e x p e c t a t i o n s ) , w h e r e £ r e p r e s e n t s the p a r a m e t e r s of the s t a t i s t i c a l m o d e l s P(D|TT). The w e i g h t e d l i k e l i h o o d r a t i o i s i n t e r p r e t e d a s the r a t i o of the p o s t e r i o r o d d s f o r H to the p r i o r o d d s , no m a t t e r what the (finite n o n z e r o ) p r i o r o d d s , P ( H | D ) / P ( H | D) = L

(H, H) •

P(H)/P(H).

(7)

H e n c e the w e i g h t e d l i k e l i h o o d r a t i o r e p o r t s the i n f e r e n t i a l content of the d a t a D f o r H. The w e i g h t e d l i k e l i h o o d r a t i o i s a f u n c t i o n of the p r i o r d i s t r i b u t i o n , and s o s h o u l d be r e p o r t e d f o r s e v e r a l c h o i c e s of p r i o r d i s t r i b u t i o n u n d e r H. The p r i o r d i s t r i b u t i o n s u n d e r the s i n g u l a r s u b s e t H a r e a s s u m e d to b e i n d u c e d a s the l i m i t of c o n d i t i o n a l d i s t r i b u t i o n s u n d e r H ( S a v a g e ' s d e n s i t y p r i n c i p l e , D i c k e y and L i e n t z , 1970). The w e i g h t e d l i k e l i h o o d r a t i o s a r e g i v e n h e r e f o r t h r e e c o n t e x t s : 1.

2 x 2 4 - c e l l m u l t i n o m i a l m o d e l , h y p o t h e s i s of i n d e p e n d e n c e (Gunel and D i c k e y , 1972). D = (n

,n

,n

), n . . > 0, i, j = 1 , 2 , n

= N fixed.

D IT ~ M u l t i n o m i a l (TT, N). P(tt| H) = [ n . . ^ . - 1 ] r ( 4 a ) / [ r ( a ) l 4 , ~ li n i j iJ flattening constant

a. 19

a > 0, D i r i c h l e t with

Dickey and Walrath

[n.r(n. + 2 a - l ) ] [ n . r ( n L

(H,H)

1

=

[r(a)]4[r(4a-2)]2 [r(2a-l)]

2.

4

r

+ 4a)

—

[nijr(nij+a)][r(n

.

.+ 2a-l)]r(n

i

+4a-2)]

.

(

(4a)

T w o - b i n o m i a l m o d e l , h y p o t h e s i s of e q u a l r a t e s ( D i c k e y a n d Lientz, 1970). For i = 1,2, D. = n . , , n . f i x e d , I ll 1»

£= ( W -

V W

D. TT ~ B i n o m i a l (TT.,n. ), i n d e p e n d e n t f o r i = 1 , 2 . l ~ l i * P(7T I H) = TT^"1 (1 - T T . ) a " 1 / B ( a , a ) , a > 0, b e t a w i t h p a r a m e t e r s (a, a ) , i n d e p e n d e n t f o r i = 1 , 2 . H:

7T1 = TT2.

D =

(DlfD2). [n.r(n

L

d

( H , H ) =

~

.+ 2 a - l ) ] [ n i r ( n .

+4a-2)

[n..r(n..+a)]r(n

i]

. —

ij

[r(a)]4r(4a-2) 2

[r(2a-l)] [r(2a)]

+2a)3

' 2

(

Advanced Breast Cancer

3.

O n e - b i n o m i a l m o d e l , h y p o t h e s i s of a f i x e d r a t e ( J e f f r e y s , 1961, p. 256). D = s , f i x e d n. D 17T ~ B i n o m i a l (tt , n). P ( t t | h ) = 7T a_1 (1 - 7 T ) a _ : l / B ( a , a), a >

0, b e t a w i t h

p a r a m e t e r s (a, a ) . H : TT = 7T ri r

/«

tt

H

fixed,

„

r(n+2a)

[r(a)]

2 Q())

S e e a l s o Good (1950, 1955) f o r s i m i l a r B a y e s f a c t o r s . T a b l e 1 r e p o r t s the e v i d e n c e f o r an o p e r a t i o n e f f e c t in the m a s t e c t o m y - n e g a t i v e subs a m p l e . Note that p r i o r odds for n o e f f e c t a r e d i m i n i s h e d b y t h e d a t a t o p o s t e r i o r o d d s of a b o u t 1% of t h e p r i o r o d d s and t h a t t h e s t r o n g e r e v i d e n c e f o r t h i s c h a n g e c o m e s f r o m the a d r e n a l e c t o m y c a s e s . Table 2 r e p o r t s the e v i d e n c e i n t h e m a s t e c t o m y - n e g a t i v e and p r e m e n o p a u s a l - w h e n - f i r s t s e e n s u b s u b s a m p l e ; h e r e t h e d i m i n u t i o n of o d d s i s t o a b o u t 3%, which is w e a k e r evidence f o r the analogous i n f e r e n c e . The w e a k e n i n g a p p e a r s t o f o l l o w f r o m t h e r e d u c t i o n in s a m p l e s i z e . T h e r e p o r t i n g of t h e s e s u b s a m p l e w e i g h t e d l i k e l i h o o d r a t i o s i s l e s s v u l n e r a b l e to s p u r i o u s s e l e c t i o n e f f e c t s f r o m the m u l t i p l i c i t y of c a n d i d a t e s u b s a m p l e s t h a n i s t h e u s e of t h e u s u a l s u b s a m p l e P v a l u e s . E a c h p r i o r (and h e n c e p o s t e r i o r ) o d d s r a t i o f o r n o - r e a l e f f e c t in a s u b p o p u l a t i o n t e n d s t o b e g r e a t e r t h e m o r e t h e n u m b e r of s u c h s u b p o p u l a t i o n s u n d e r c o n s i d e r a t i o n ; h e n c e t h e m o r e e x t r e m e m u s t b e e a c h w e i g h t e d l i k e l i h o o d r a t i o to a t t r a c t a t t e n tion.

21

Dickey and Walrath

IK

X

o in

00 i—* o

i—i o

r-H O

a o •H fl D o o

co QO o o

CO |-H o

in !—I o

m r-

in in o o

i—i o

ii «j o CO

Q? nt Q

o CO

CO

W + +


>

to

b

•

«

+ X i S u w

•

oo 00 r• • i—i i—i

fM CO i-H

•

rO

ro ID i—( co •

1—1 r-H

ID O • ro

ro CT00 • i—i i-H

i-H

co

vO SÛ o • • IT)

ro •

•

ro

•

IT) 00 ro

CT^

•

co o

CO sO t-H ro

rO

00

ro

ao

5 O -u -(-> O

s X >s © ì b b Q- <

* T3 S3 o n) rd co fl «J CO o •i rg •-< ^r-rg>c rn rsj 0vj 0M 0oVo->r-oo-o oin •t 0•O C Mn00-s 0 >tocow —'oCT*•0 r- 0rg 0®C < rci 0o p l n -* --< •o m— m ei fio OSNNH r- in -« rin • • • • » • A • • t •• • • o o o

2 t

CCNTINUEC

SITUATION

2

BEST

20

2027 2033 2107 2243 2330 2254 2 332 2453 2358 2417 2520 2457 2349 2447 2319 2358 2574 2603 2608 2502

2304 2323 2461 2598 2684 2752 2812 2653 2846 2864 2683 2876 2886 2943 2926 2953 3008 3046 3066 3045

ruATIQN

2

DATA

STEP 2

FOR

STEP

2057 2068 2153 228 1 2372 2310 2377 2499 2416 2470 2556 2509 2418 2525 2396 2432 2624 2646 2661 2570

2

21 5C 2165 2276 24 02 2504 2473 2525 2645 2571 2631 2684 2654 26 C 3 27C9 2599 263C 2779 2786 2816 2770

ROWS'- P L U S - C O L U M N S

ANALYSIS

2923

-50

-498

-450

-283

-586 -562 -413 -282 -191 -127 -88 -18 -11 -4 5 11 27 63 79 108 136 176 199 195

19 14 3 9 4 8 29 0 -13 -3 6 -6 -12 8 -24 -26 0 -2 -4 -22

190 172 97 102 98 -41 -2 48 -53 -1 91 22 -101 -39 -183 -173 14 2 -14 -116

172 159 95 92 92 -33 -5 46 -43 2 79 26 -80 -9 -154 -147 16 - 1 -9 -96

98 89 51 46 57 -37 -24 25 - 55 -3 40 4 -62 7 -118 -116 4 - 29 -21 -64

rUATION

2

STEP

3

2369 23 S 7 2557 2683 2779 2838 2865 2956 2976 2981 2973 2988 3C14 3055 3078 31C9 3110 3140 3178 32C3

ROWS - T I M E S - C O L U M N S A N A L Y S I S

OF

31C4 3196 3364 36 15 35 75 4359 4555 39 37 4 159 40 19 3931 3950 4336 42 30 4528 4507 4230 4445 4332 4332

BEST 20

332

571

1190

- 18 -13 -2 -8 -3 -7 - 19 0 14 11 -5 3 13 17 25 27 0 -10 5 33

-119 -97 -55 -53 -70 17 3 -37 79 16 -53 -2 76 -6 158 150 -4 15 13 87

- 2 06 -175 -108 -89 -134 63 61 -68 114 10 -91 -14 131 -25 256 233 -4 38 5 107

-422 -453 -335 -215 -346 373 531 -157 57 -89 -186 -174 195 53 3 36 286 -19 155 20 23

-195

- 3 50

- 8 16

-10 11 19 1 9 -52 -87 0 51 29 -5 29 20 -15 58 61 0 -13 7 59

-11 19 26 8 8 -61 -100 -1 65 33 -4 42 33 -41 78 75 5 -13 -5 57

30 0 -21 11 -15 82 154 -2 -55 -35 16 -42 -3 3 16 -77 -8 2 3 34

OF

20

281

250

169

-21

385 386 266 193 281 -246 -319 132 -95 46 173 112 -194 -31 -350 -312 19 -101 -20 -98

8 3 -3 3 -3 15 38 -3 -10 -4 1 -9 -6 9 -14 -17 0 0 -3 -19

34 16 -9 24 -15 59 127 -4 -14 -20 21 -22 -22 -26 -40 -46 7 44 -5 -76

33 20 0 22 -8 56 110 - 1 -8 -13 17 -13 -9 2 -27 -34 9 34 - 1 -61

4 -4 - 13 -1 - 11 22 53 -7 -32 -14 -1 -22 -15 14 - 32 - 40 0 -4

-6 -1 5 -2 5 - 15 -29 4 11 12 0 6 7 16 14 17 0 -13 4 30

240

2701 2755 7971 31 22 3168 3429 3466 34C7 3596 3500 3407 3490 3652 3531 3829 3835 3625 3709 3698 3797

51

1441

- 16 -40

2549 2594 27e5 291 9 2993 3144 31 6 9 3199 3322 3267 32C6 32 64 3358 3312 3492 3513 3386 3447 3467 3538

BEST

20

-4 -93

Robustness Study Analysis

EXHIBIT

2,

C ONT I NU EC

SITUATION

16

STEP

1

ROWS-PLUS-COLUMNS

2 549

17

-417

2 8 0 4 5 53 25222 4489

7192863 7263 1240

-2775004

17

-652

-631

-21 -32

172 232

157

950 -161 -7 -164 366

-20659 -3340

-16

193

8 -4 485 8430996 -17

-219 117 -966 -1543335 149

5 183 49

273 -4004 -2090 -1119 -245

108 -202

69 5 -7 -30

857406 1966

-70 1448 1566421

-390 -1B42943 - 19984 -3185

218 170 -226 100

ANALYSIS

- 15

350

7 02

2063

1516319 - 7262 -see

1516320 24813 889

7192179

-16814 -25S2

59031 4241

7190816 1337 27 1R699

-5C6

- 17

632

1447

3976

22 33

-138 -2C5

-1499

17

- 1 59

-268 -441 -2 99

-288 -174C2C7

1 16 169 126 - 1 91 69

-

-7 5 -445

-953 -875681 132 240

-846 -e01847 97

-657647 17

172

-5

-2877 -1615

-182 -48

0 206

-3777 -2013 -1078 -243 -10 190

-49453 172

- 8 5 59 22 -925

-840161 -890

1010

109

-511

241 -51

57 0 -81

-105 78 -452 -67

-125 -486 6288 3314 1841 518

1411 497 508

1110 249

2403 629

7620 1445

0 30

21 -172

149 -364

188 -1306

49453 -172

1779537 944

2333160 2991

2981885

-714

-503

-421

-1C9

-113 66 -430

-113 35 -285

-56 -20

516 65 -104

1600 394

-67 -73

163

301 24 154

- 13 - 16 222 40 125

- 16 146 -638191

e2 132 135

-43 -35

-335 -520 -609

-38 -28 -25

123 213 276

117 200 256

98 154 195

-456 -302 -417

-2 3 -33 -27

197 104 277

184 102 257

141

38 28 25 24

85 196

34 28

-534 8566 -475

-21 3260 -20

343 -7670 376

-7473 351

234 -6760 261

22 -3021

827

50

-39 5

-39 5

-335

-461

316 71 418 418 42

286 48 391 391

219

-315 -2548 -2548 -307

-23 0 -16 -16 0

-374

0

-507 1533

7 364

188 206 -590

-70

-33 -29 -18

-73 334 19

314

1 2 89

- 2 63 - 3 93 -491 - 3 42

- 1 1 15 -1323 -1553 -1137 -929 -1604

- 2 84

-215 -493 -581

3022 -316

12333 -644

541

-1777 51908 - 1 9 39

1360

2514 -1691

-210 -218

- 5 64 - 2 67 -701 -701 - 2 57 -404

- 2 75 -249 -349

1306 -2062 -2062 271

-408 2892 0

1 72 199

30 18

-210 -251 266

-442 - 5 02 863

1931

-51

-83

- 5 51

-63 15

142 77

2252 192

15 -53

-258

5 00 231 - 5 09

113 1C49

5 03 2815

-186

32

0

-23

81

68

51

275 -89

64 -14

-257 -42

-253 -46

-215 587

-15 54

315 -184

280 -189

-216 - 39 2C5

1589 21489

272 16431

70345

26022 8724433

-3460 -373 368 8 39

-143 -1617 -8 81 -1229

-228 4 78 0

0

-61416 -878451 -3313 -364 334

-172 -99 -235

-205

323 -543 -114 -331

-6 -363 29

222 266 -214

-89 1 -18610

23 -1C6

-18 -151 -114 - 1 88 -239

1658 0 592

-104 782 300

138 -563 3

0

-900 -19150 -64205 - 1 2 5 1956

20 -49

230 -272

7 3 14 4292

130

289 0

33 173 19 1 -619

17 62 35

443 219

28355 12357

16 16 -60 0

237 287 -227

-22 44

16432 8428949 - 7 26 - 6 94

- 68 -4

-863 -194

-73 -419

1366 53

-211

-923 1527 - 3 19

-465 6229 3793

-356

1272981 4783 662 -670 1839

446 657648 -119

768 -124 3772 6262058

-256 2725 1725

14 17 -17

-294 -502 574

3C2 -97

-966

-162 -791 -15478 -53291 - 8 362 2 5

- 31 24

-271 -3492 -26021 -60e768

-2718 -3C9

-978 - 53

2 52 454

23 -44

731

241

-349

3493 97181 608769 978 3 68 -3C6 -522

12661 236016 5934445 5049 980 - 6 28 -943

-748 -651 9741 - 3 51 -1474 -1474

-1475 2185 13079 120556 778030 8722386 21154 3754 - 1 9 11 - 2 4 32

Tukey EXHIBIT

F)

2,

CONTINUED

SITUATION

16

BEST

1828 1797 19 39 1823 1829 1916 1918 2032 1986 1871 1874 1990 1944 2109 1919 1886 1865 19 32 2073 2231

1872 1930 2009 2016 2065 2045 2085 2070 2081 2085 2110 2120 2190 2192 2192 2249 2258 2229 2241 2335 Gt

H»

SITUATION

16

OAT A

20

STEP

2

FÜR

STEP

2

1841 1845 1959 1893 189C 1957 1946 2046 2018 1944 1965 2038 2015 2126 2023 1945 1952 2042 2137 2250

1821 1804 1937 1837 1841 1922 1912 2034 1983 1885 1892 1997 1956 2103 1940 1890 1883 1957 2085 2223

ROWS-PLUS-COLUMNS

ANALYSIS

2111

-5

-164

-161

-118

-231 -171 -95 -82 -86 -63 -41 -36 -22 -17 18 21 55 82 103 45 85 136 140 223

0 -2 0 -5 47 4 22 2 0 -1 -12 -5 30 5 -15 99 68 -11 -3 7

114 23 89 -39 -30 34 14 123 63 -56 -89 23 -56 81 -129 -104 -166 -149 -13 62

104 27 84 -28 -21 37 5 12 2 57 -45 -74 27 -47 72 -111 -103 -151 -127 -4 51

81 25 63 - 15 - 15 29 - 3 91 49 -29 -44 25 - 31 52 -71 -91 -125 - 85 4 35

SITUATION

16

STEP

3

1885 1948 2020 204C 2C18 2C49 2041 2078 2095 21 CO 2148 2143 2158 2194 2236 21 10 2144 22 64 22 68 2333

ROWS-T I M E S - C O L U M N S

ANALYSIS

OF

1 921 2C49 2C79 2189 2162 2145 2156 21 C7 2162 2269 2327 2245 23C5 2270 2449 2333 2380 2496 2393 2425 BEST

1951 2149 2134 2 3 36 23 34 2246 2299 2131 2225 2451 2499 2339 2471 2351 2652 2667 2685 2732 2513 2526

?029 ?i-I • CQ r*

i

1 i + +

+ + + + + + + + + + + + + 1 + + + + + + +

a o ÍH

• •M 1-1 Mi i— O u

(M m r^ co — iH i 1

uy

1 1 1 + +

+ 1 + + + + + + + + + + + 1 +

+ + 4+ + + + + + 1 + + + + + + + + + + + + + + + + + + + + O + +

CO vO r- i-H NÛ r- CTN ^ i vO in i

+

1 + + + + +

+ + + + + + + + + + + + 1 i + + + + + + + + + +

1

+ + + + + + + i + + +

+ + + + + + + + + + + + i i + + + + + + + + + + + + + + + + + +

1 i + +

1

it ON CM — CO o 00 vO 00 CO rONv£> o m i-H oo 00 i—i vO r^ CM CM i-H CM vO CM CM r- t^ ^ i i 1 1 1 i H CM

m CM vO m 00 00 CO ON r- CM m CO O vO CO i-H i-H m 00 xO vO O i-H CM CO m 00 ONvO i 1 1—1 i-H — i—i i1 i i i-H CM CM CM i i

i 1 CM i-H ^ 00 ON CM 00 ^ vO ^ (M m o m CM r- 00 CM i-H — rH i-H 00 i—i NO CO I-H CM CO 0 s in in O m in CM i-H CO i-H 1 1 i 1 i 1 1 1 i 1 i-H

t) PH ai ¡> o>

i ( CM CO vO CO vO i—i r^ CM i-H O CO o vO 1—1 CO CO i-H — 1—1 1 CO 1 1—1 1 i-H m m i—i i-H i-H 1 1 1 i i

u

1

d co S ^ h T3 3 £ •SP o h i

Off Response to Audio Tone

ACKNOWLEDGEMENTS T h i s study w a s supported in p a r t b y NIH G r a n t C o m p u t e r u s e in the H e a l t h S c i e n c e s , 5 P 0 7 P R 0 0 0 1 1 - 0 9 . M r . A. Angelone d e s i g n e d and built the s q u a r e wave tone s t i m u l u s g e n e r a t o r , the switching l o g i c so that the wave w a s cut off at the end of a full c y c l e and the i n t e r t r i a l i n t e r v a l t i m e c o n t r o l unit. He then loaned us h i s H i F i s t e r e o h e a d p h o n e s . G r e a t e r love of knowledge i s indeed r a r e ! M r . J . C. C h r i s t o p h e r w r o t e the p r o g r a m to digitize the data and the s o f t w a r e to g e n e r a t e the I B M 360 c o m p a t i b l e data t a p e . He a l s o d i g i t i z e d the d a t a .

349

Martin and Lucas DATA SET DESCRIPTION

DATA - EVOKED OFF RESPONSE TO AN AUDIO TONE (D. C. MARTIN AND H. L. LUCAS) CASES AND VARIABLES THERE ARE SEVEN SUBJECTS WITH THREE RUNS EACH. EACH RUN CONSISTS OF 75 TRIALS WITH 256 SAMPLES PER TRIAL. THE DATA ARE AVAILABLE ON A STANDARD LABELED 9—TRACK TAPE, BRIEF DESCRIPTION VAR

NAME

***

****

1 2 3 * 5 6 7 8 9 10 11 12-267

SUBJECT NUMBER AND RUN NUMBER NUMBER OF POINTS IN TRIAL STAT PACKAGE HEADER WORD STAT PACKAGE HEADER WORD STAT PACKAGE HEADER WORD PAD PAD TRIAL NUMBER NORMAL (TONE-ON) OR INVERTED (TONE-OFF) NUMBER OF INTERRUPTS SAMPLE RATE DATA POINTS

PURPOSE OF STUDY THE STUDY EXAMINED EVOKED ELECTRICAL POTENTIALS OF THE BRAIN TO AN AUDIO STIMULUS. ELECTRICAL POTENTIALS WERE AMPLIFIED BY A STANDARD GRASS MODEL 7 POLYGRAPH WITH 7T5 PREAMPLIFIERS. THE DATA WERE COLLECTED ON AN FM AMPEX ANALOGUE FR 1300 PORTABLE TAPE RECORDER AND A CONVENTIONAL INK STRIP CHART RECORDER. THREE CHANNELS OF DATA WERE COLLECTED. ONE CHANNEL WAS A MARKER CHANNEL TO INDICATE WHEN TO SAMPLE THE EEG. ONE CHANNEL WAS A TONE GENERATED BY THE STIMULUS GENERATOR. THE LAST WAS AN EEG CHANNEL. THE DATA RUNS ARE APPROXIMATELY SIX MINUTES LONG. THERE ARE SEVEN SUBJECTS WITH THREE RUNS EACH. EACH RUN CONSISTS OF 75 TRIALS WITH 256 SAMPLES PER TRIAL. REFERENCE MARTINt 0. C. AND H. L. LUCAS. FIRST TIME. DETAILED VAR

***

PUBLISHED

DESCRIPTION COL ***

FORMAT ******

DESCRIPTION ***********

350

IN THIS VOLUME FOR THE

Off Response to Audio Tone

2 3 4 5 6 7 8 9

10 11

2-10

xxxxxxxxx.

11- 19 2 0 - 28 2 9 - 37 38- 4 6 4 7 - 55 56- 64 6 5 - 73 7 4 - 82

XXXXXXXXX. XXXXXXXXX. XXXXXXXXX. XXXXXXXXX. XXXXXXXXX. XXXXXXXXX. XXXXXXXXX. XXXXXXXXX.

83-91 92-100

XXXXXXXXX. XXXXXXXXX.

SUBJECT NUMBER AND RUN NUMBER F I R S T D I G I T = SUBJECT NUMBER ( 1 - 7 ) SECONO D I G I T = RUN NUMBER ( 1 - 3 ) NUMBER OF POINTS IN T R I A L ( 2 5 6 ) STAT PACKAGE HEADER WORD ( 3 2 7 7 0 2 ) STAT PACKAGE HEADER WORD ( 5 0 ) STAT PACKAGE HEADER WORD ( 4 1 9 4 3 2 0 ) PAD ( 0 ) PAD ( 0 ) T R I A L NUMBER ( 1 - 7 5 ) NORMAL (TONE--ON) OR INVERTED ( T O N E - O F F ) 1 = NORMAL (TONE-ON) 2 = INVERTED ( T O N E - O F F ) NUMBER OF INTERRUPTS (1 M S ) / 6 SAMPLE RATE 1000 HZ

12

2-7

XXXXXX.

FIRST

30

110-115

XXXXXX.

N I N E T E E N T H DATA

DATA

POINT POINT

THE NEXT 13 RECORDS ARE IDENTICAL IN FORMAT TO THE PRECEDING ONE EXCEPT THAT THE LAST RECORD CONTAINS ONLY N I N E DATA P O I N T S . FORMAT

IS

LOCATION OF THE DATA DSL IB 1 .

(IX,UF9.0/13(1X,19F6.0/)

DATA ARE A V A I L A B L E

DATA **»* SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT SUBJECT

tlX,9F6.0)

1. 1. It 2, 2* 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6» 6, 6, 7, 7, 7,

L I S T I N G OF DATA

ON A STANDARD L A B E L E D 9—TRACK

DSNAME ****** RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN RUN

FOR

1 2 3 1 2 3 1 2 3 I 2 3 1 2 3 1 2 3 1 2 3

LABEL *****

SUB1RUN1 SUB1RUN2 SUB1RUN3 SUB2RUN1 SUB2RUN2 SUB2RUN3 S UB3RUN1 SUB3RUN2 SUB3RUN3 SUB4RUN1 SUB4RUN2 SUB4RUN3 SUB5RUN1 SUB5RUN2 SUB5RUN3 SUB6RUN1 SUB6RUN2 SUB6RUN3 SUB7RUN1 SUB7RUN2 SUB7RUN3

SUBJECT

l

f

TRIAL

351

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

1

TAPE

LABELED

Martin and Lucas (THE

FIRST

FIVE

LINES

ARE

COLUMN

GUIDES)

S T U D I E S WITH A T R E M E N D O U S AMOUNT OF D A T A t S U C H AS T H I S O N E . MUST U T I L I Z E T A P E OR D I S K . THE S A M P L E OF DATA L I S T E D BELOW C O N S I S T S OF TWO T R I A L S FROM RUN 1 FOR S U B J E C T 1 . THE E N T I R E F I L E FOR T H I S S T U D Y I S A L M O S T 8 0 0 T I M E S AS L A R G E AS T H I S S A M P L E . 0 1 2 3 4 5 6 1234567890123456789012345678901234567890123456 78901234567890123456 1 1 7 8 9 0 1 7890123456789012345678901234567890123456789012345 256

11 -1425

-463

831

-837

-1633

-545

-3329

-3681

-3813

-561

-1415

-2089

-6159 -6947 -5633 -4407 -465 7 -4439 -2369 -2733

-2959

-2689 -1867

-469

511

343

1487

4991

4209

3315

1493

1881

2047

3199

3 543

2673

4095

4451

3427

2 869

1515

1913

327702 1 1971 1303 -257 319 -287 -1331 -2373 -1639 -4007 -3495 -2097 -2273 -2247 -2431 -3137 -4017 -5421 -5857 -5645 -5361 - 3 7 8 3 - 3 8 89 -2401 -2593 -2817 -2913 -1711 -599 -185 -1217 -701 -1137 2927 4591 4479 4847 3441 2921 1613 1407 2105 2047 3967 5231 3519 3761 1919 1727 3823 3389 1969 3627 2559 1663

50 2 1129 -571 -219 -1089 -2967 -2243 -2641 -2855 -6321 -4401 -3885 -2801 -2539 -199 -1619 -809 4847 5183 3001 2175 1407 3839 3199 2227 4479 4863 313

4194320 0 5089 1000 2679 2659 1489 -449 -335 -1285 -129 -2717 -3143 -1769 -2689 -3339 -2353 -2129 -1861 -2467 -2113 -1247 -2195 -2183 -2539 -2651 -2319 -1249 -4743 -4783 -4961 -5001 -5311 -5559 -3985 -3377 -3265 -2305 -2033 -2177 -2205 -2115 -2087 -971 -1153 -1373 -823 -385 -413 - 5 79 -719 -341 5311 4031 3953 4479 4415 4095 2623 2225 2337 2047 1623 1745 1249 1585 3071 3041 2047 1319 2047 1883 2407 3199 4095 3763 5567 5887 5103 4735 4991 5487 383 511 255

4194320 327702 50 0 2 5551 1000 2 3199 -979 2047 2639 4607 7599 6271 -577 1521 -913 -965 -151 369 1023 -481 1347 1727 1981 2175 745 1203 1991 1651 2297 2687 2641 3455 3967 3525 2473 2175 2175 1887 1407 1641 1383 1151 639 1241 -845 -1231 -1541 -1715 -1725 -691 -2735 -3131 -3613 -3521 -3985 -3765 -3985 -4865 -4953 -5123 -4509 -4331 -4043 -3873 -3905 -3377 -2999 -1927 -1667 -2889 -3045 -2775 -2707 -2457 -2545 -3161 -3521 -4145 -4503 -4395 -3483 -2753 -2997 -3137 -3313 -3527 -3249 -3201 -3249 -3343 -2017 -1409 -1689 -2619 -2757 747 3275 2337 -455 2639 22 73 3135 -1889 -1551 3429 2815 2793 2943 2303 2673 11

0 -163 -667 -2177 -3775 -1921 -1537 -1567 -2357 -5557 -4893 -3073 -2001 -1921 -1039 -1483 -257 4095 4463 2111 1779 2559 1131 1913 3 519 3481 4991

0

2 56

352

51 -77 -1817 -2993 -2389 -915 -2049 -4353 - 5 6 39 -5105 -2621 -1873 -2129 -1587 -1121 -283 4351 5439 2401 1469 2389 2429 1599 4095 1919 4479

1023 767 2291 2675 191 -2183 -4681 -4289 -2239 -4487 -3025 -1921 3017 2685

1279 975 2375 1787 -157 -2583 -4815 -4033 -2063 -4273 -3593 -2101 1983 2765

Off Response to Audio Tone 3839

3839

4575

3281

3581

2647

2785

2943

1879

1897

2343

1735

-385

-485

-641

3261

2929

1645

497

115

319

5359 4095 2417 26 73 1239 895 1149 -17 -305 895 431 1649 207

5183 3957 1851 2397 1535 1279 637 437 481 733 639 995 823

3125 3647 2559 2485 1373 1791 -129 369 1407 1535 383 255 735

353

2815 3665 3041 1625 1535 2453 63 -1 2289 1971 337 -705 613

3041 3709 3391 751 15 85 1407 -65 615 3433 3007 79 -371 959

3583 4351 2877 1279 1747 1225 -205 625 3327 2813 -1 73 1215

4479 4847 3071 1247 1791 1619 213 633 2267 1621 1717 767

4543 3929 2547 1633 1023 1279 187 -519 1279 2673 2257 1087

Martin and Lucas

REFERENCES B a r t l e t t , F . and E . R . John ( 1 9 7 0 ) . ments. Science 169, 3 0 4 - 3 0 5 .

R e p l y to S c h w a r t z ' s

com-

B i n g h a m , C . , M. D. G o d f r e y and J . W. T u k e y ( 1 9 6 7 ) . Modern t e c h n i q u e s of power s p e c t r u m e s t i m a t i o n . I. E . E . E . T r a n s a c t i o n s on Audio and E l e c t r o a c o u s t i c s , A U 1 5 , No. 2, 5 6 - 6 6 . Donchin, E . { 1 9 6 9 ) . D a t a a n a l y s i s t e c h n i q u e s in a v e r a g e evoked potential r e s e a r c h . In A v e r a g e E v o k e d P o t e n t i a l s , E . Donchin and D. B . L i n d s l e y , E d s . , W a s h i n g t o n , D. C . Govt. P r i n t i n g O f f i c e s , N A S A S P - 1 9 1 , 1 9 9 - 2 1 7 . F r e e m a n , W . J . (1964). prepyriform cortex.

A l i n e a r d i s t r i b u t e d f e e d b a c k m o d e l for E x p . Neurol. 10, 5 2 5 - 5 4 7 .

K i a n g , N . Y . S . and T . T . Sandel ( 1 9 6 1 ) . O f f - r e s p o n s e s f r o m the a u d i t o r y c o r t e x of u n a n e s t h e t i z e d c a t s . A r c h . Ital. B i o l . 99, 121-134. M o r s e , A. P . and F . E . Grubbs ( 1 9 4 7 ) . The e s t i m a t i o n of d i s p e r s i o n f r o m d i f f e r e n c e s . Ann. M a t h . S t a t i s t . 18, 1 9 4 - 2 1 4 . Sandel, T . T . and N. Y . S. Kiang ( 1 9 6 1 ) . a u d i t o r y c o r t e x of a n e s t h e t i z e d c a t s . 105-120.

O f f - r e s p o n s e s f r o m the Arch. Ital. Biol. 99,

S c h w a r t z , M . ( 1 9 7 0 ) . M e a n s and v a r i a n c e s of a v e r a g e - r e s p o n s e wave f o r m s . S c i e n c e 1 6 9 , 3 0 3 - 3 0 4 .

354

DISCUSSION O F T H E L U C A S AND M A R T I N C H A P T E R

R.

Mickey

P r o f e s s o r s M a r t i n and L u c a s have p r e s e n t e d us with a l a r g e d a t a s e t f r o m a c a r e f u l l y planned e x p e r i m e n t . It even a p p e a r s that t h e y applied to t h e m s e l v e s the advice t h e y would give to o t h e r s , p a r t i c u l a r l y with r e s p e c t to advance planning of the a n a l y s i s . U n f o r t u n a t e l y — in one s e n s e at l e a s t — t i m e did not p e r m i t the plans to c o m e to f r u i t i o n . On the o t h e r hand, as i l l u s t r a t i v e d a t a f o r a n a l y s i s this i n c o m p l e t e n e s s l e a v e s us with f r e e s c o p e to s p e c u l a t e on what m i g h t be i n t e r e s t i n g to t r y , without being i n h i b i t e d b y knowing what w o r k e d out w e l l . I a g r e e with M a r t i n and L u c a s that the p r o b l e m i s r e p r e s e n t a t i v e of a s u b s t a n t i a l c l a s s ; it p e r h a p s h a s the advantage of being m o r e " n e a t " than m a n y s i m i l a r p r o b l e m s in that it m a y not p r e s e n t as m a n y side i s s u e s to d i v e r t our a t t e n t i o n . One q u e s t i o n that a r i s e s f o r m e i s w h e t h e r or not t h e r e i s in f a c t a r e s p o n s e . I f e e l a bit u n c o m f o r t a b l e in a s k i n g the q u e s t i o n , s i n c e t h o s e in the evoked r e s p o n s e b u s i n e s s d o u b t l e s s have m e t h o d s of d e t e r m i n i n g t h i s that a r e a c c e p t a b l e to the c o m m u n i t y . Or i s t h i s p o s s i b i l i t y one of the q u e s t i o n s f o r us to t r e a t in the d a t a a n a l y s i s ? T h e q u e s t i o n of r e s p o n s e m a y not be s e n s i b l e to a s k of a given t r i a l o r r u n , although i t would s e e m n a t u r a l to s a y of a given run that the s u b j e c t showed a r e s p o n s e , p a r t i c u l a r l y if the r e s p o n s e was l a r g e in s o m e s e n s e . One of the r e a s o n s f o r r a i s i n g the q u e s t i o n at all c o m e s f r o m wondering if one d o e s n ' t n e e d p r e - s t i m u l u s data to give an a n s w e r . P e r h a p s the q u e s t i o n i s a side i s s u e . What is a v a i l a b l e to look at in the way of r e s u l t s i s the s e t of t r a c i n g s of p o s t - s t i m u l u s a v e r a g e p o t e n t i a l for e a c h of the t h r e e r u n s for e a c h of the s e v e n t e s t s u b j e c t s . B y way of s u b j e c t i v e a n a l y s i s , I r e m o v e d the t r e a t m e n t and run n u m b e r i d e n t i f i c a t i o n and p r e s e n t e d the t r a c i n g s to a couple of s u b j e c t s , t e l l i n g t h e m that f o r e a c h t e s t two of the t r a c i n g s followed one s t i m u l u s and the t h i r d t r a c i n g followed a s e c o n d s t i m u l u s ; the s u b j e c t s w e r e 355

Martin and Lucas, discussion

a s k e d to pick the one t h a t w a s d i f f e r e n t . The t a s k a p p e a r e d to p r e s e n t no d i f f i c u l t i e s and b o t h s u b j e c t s c o r r e c t l y s o r t e d the t r a c i n g s f o r e a c h of t h e s e v e n t e s t s . T h e r e p l i c a t e t r a c i n g s s e e m to be quite s i m i l a r and a r e s u f f i c i e n t l y d i f f e r e n t f r o m t h e t h i r d that t h e r e w a s no point in continuing with additional s u b j e c t s . The r e p l i c a t e t r a c i n g s a r e a l w a y s the s e c o n d and t h i r d r u n s , so it is p o s s i b l e t h a t the d i s t i n c t i o n is an o r d e r r a t h e r t h a n a t r e a t m e n t e f f e c t . The s u b j e c t s w e r e then a s k e d to s o r t the t r a c i n g s into two g r o u p s to c o r r e s p o n d to the two t r e a t m e n t s . One s u b j e c t s o r t e d the t r a c i n g s in c o m p l e t e c o r r e s p o n d e n c e with t r e a t m e n t and t h e o t h e r h a d a f e w e r r o r s . The s e c o n d t a s k h a s s u b s t a n t i a l l y m o r e u n c e r t a i n t y t h a n t h e f i r s t b e c a u s e of t h e v a r i a b i l i t y among t e s t s u b j e c t s , but n e v e r t h e l e s s does not s e e m to p r e s e n t a g u e s s i n g s i t u a t i o n . M y c o n c l u s i o n i s t h a t a t l e a s t o n e of t h e s t i m u l i e l i c i t s a r e s p o n s e a n d , a l t h o u g h t h e r e i s q u i t e a b i t of v a r i a t i o n a m o n g r e s p o n d e r s , t h e r e s p o n s e p a t t e r n s a m o n g t e s t s u b j e c t s a r e s i m i l a r enough to g e n e r a l l y d i s t i n g u i s h the two stimuli. I also t r i e d o r d e r i n g the t e s t s u b j e c t s by r e s p o n s e p a t t e r n s . This w a s d o n e s e p a r a t e l y f o r t h e o n r e s p o n s e a n d t h e off r e s p o n s e . My object was to group together p a t t e r n s that s e e m e d quite s i m i l a r a n d t o s e e if t h e r e w a s a g r a d a t i o n a m o n g t h e p a t t e r n s . I d i d t h i s s e p a r a t e l y f o r t h e o n r e s p o n s e a n d t h e off r e s p o n s e . T h e r e w a s n o r a n k c o r r e l a t i o n b e t w e e n t h e t w o o r d e r i n g s of t h e s u b j e c t s . While t h e r e w e r e s o m e t r a c i n g s that s e e m e d r e l a t i v e l y quite s i m i l a r , (for e x a m p l e , the on r e s p o n s e for s u b j e c t s 1 a n d 4) i n g e n e r a l I d o u b t t h a t m y o r d e r i n g s w o u l d b e v e r y r e p r o d u c i b l e w e r e I to r e p e a t the p r o c e s s after a suitable t i m e lapse. T h e r e s u l t s of o n e s u c h o r d e r i n g a r e s h o w n i n f i g u r e 1. I d o n ' t t h i n k t h e r e i s any p a r t i c u l a r s i g n i f i c a n c e to the o r d e r i n g , o r e v e n t h a t a t t e m p t i n g t o o r d e r i s p a r t i c u l a r l y i n s t r u c t i v e . I do t h i n k t h e f i g u r e b r i n g s o u t s i m i l a r i t i e s a m o n g p a t t e r n s and m a y be u s e f u l for thinking about n u m e r i c a l analysis. F o r e x a m p l e , t h e r e a p p e a r s to be an i m p o r t a n t a m p l i t u d e v a r i a t i o n a m o n g s u b j e c t s . A l s o t h e r e is s o m e s u g g e s t i o n t h a t s u b j e c t s v a r y in

356

Off Response to Audio Tone, discussion

OFF

Subject

4

RESPONSE x Run

ON

3

RESPONSE

Subject

/

Run I

Subject

4

Run l

Run 2

Subject 2

Run 1

Subject

Run 1

6

Subject 2

,Run 3 Run 2

Subject

/

Subject 6

Run 2

Run 2

Run 3

Subject 3

Run 1

Subject 5

Run 1

F i g u r e 1. E v o k e d off and on r e s p o n s e s to a n a u d i o tone a r r a n g e d a c c o r d i n g to g r a d a t i o n of r e s p o n s e . T r a c i n g s a r e f r o m t h o s e s u p p l i e d by L u c a s and M a r t i n . T h e r e is no d i f f i c u l t y in s o r t i n g out the r e s p o n s e s of a g i v e n s u b j e c t to the r e p e a t e d s t i m u l u s , and l i t t l e to m o d e r a t e d i f f i c u l t y in d i s t i n g u i s h i n g b e t w e e n on and off r e s p o n s e in an u n l a b e l e d p r e s e n t a t i o n of the response curves.

357

Martin and Lucas, discussion

t h e i r time scale. R e p r e s e n t a t i o n s that explicitly allow for these v a r i a t i o n s m i g h t be m o r e u s e f u l t h a n r e p r e s e n t a t i o n s t h a t do n o t . I t a k e t h e r e s u l t s a s shown in f i g u r e 1 to i n d i c a t e t h a t t h e r e p r o b a b l y a r e s o m e n i c e l y i n f o r m a t i v e w a y s of a n a l y z i n g t h e d a t a , and I l o o k f o r w a r d to the c o m p l e t e d a n a l y s i s b y M a r t i n and L u c a s . O n e of t h e d i f f i c u l t i e s w i t h t h i s t y p e of p r o b l e m s e e m s t o b e t h a t d e s p i t e a l l of t h e n u m b e r s t h a t e n t e r t h e d a t a b a s e , t h e r e a r e n o t m a n y d e g r e e s of f r e e d o m w h e n i t g e t s d o w n t o d e v e l o p i n g t h e f i n a l c o n c l u s i o n s . T h i s a s p e c t of t h e a n a l y s i s s e e m s m o r e s e v e r e if one w e r e to take into account other a t t r i b u t e s (such as h e a r i n g s e n s i t i v i t y , p i t c h d i s c r i m i n a t i o n a b i l i t y , and p e r h a p s i n d i c a t o r s of n e r v o u s s e n s i t i v i t y ) t h a t m i g h t p l a u s i b l y r e l a t e t o t h e r e s p o n s e . S i n c e c o v a r i a t e t y p e i n f o r m a t i o n i s n o t a v a i l a b l e w e do n o t n e e d t o c o n s i d e r h o w it m i g h t a f f e c t the a n a l y s i s . Noting t h e p o s s i b i l i t y d o e s r a i s e t h e q u e s t i o n of t h e e x t e n t t o w h i c h t h e d a t a a n a l y s i s n e e d s t o b e b a s e d o n i d e a s of p h y s i o l o g i c a l i n t e r e s t . I w i l l h a z a r d t h e o p i n i o n t h a t d a t a a n a l y s t s c a n d e v e l o p f i n d i n g s of p h y s i o l o g i c a l i n t e r e s t w i t h a m i n i m u m of g u i d a n c e f r o m p h y s i o l o g y a n d t h a t i t i s q u i t e a p p r o p r i a t e t o p r o c e e d o n o u r o w n . In o r d e r to c o n t r i b u t e e f f e c t i v e l y to t h e s c i e n t i f i c s u b s t a n c e , h o w e v e r , I think that a s e c o n d , t h i r d , e t c . p a s s at the d a t a w i l l o r d i n a r i l y be n e e d e d , and that t h e s e will r e q u i r e a p p r o p r i a t e c o l l a b o r a t i o n .

Jukey (preconference

comments)

The m a i n i s s u e s h e r e s e e m to be the e x t e n t to which b a c k g r o u n d i n t e r f e r e n c e (mainly f r o m alpha r h y t h m s ) can be allowed f o r , and w h a t o u g h t to b e d o n e t o l o o k m o r e c l e a r l y i n t o t h e d a t a , e s p e c i a l l y i n v i e w of t h e p o s s i b i l i t y of e n t r a i n m e n t of a l p h a r h y t h m s . L e t u s s u p p o s e t h a t w e h a v e b e e n a b l e t o d i g i t i z e a l o n g p i e c e of r e c o r d for m a n y c a s e s . F o r definiteness I suppose details that m a y not be e s s e n t i a l , s p e c i f i c a l l y that we have d a t a for a d j a c e n t t i m e intervals as follows:

358

Off Response to Audio Tone, discussion

I n t e r v a l A = 256 p o i n t s at 8 m i l l i s e c o n d s I n t e r v a l B = 256 p o i n t s at 1 m i l l i s e c o n d (stimulus change) I n t e r v a l D = 256 p o i n t s a t 1 m i l l i s e c o n d T h e f i r s t s t e p i s to a n a l y z e i n t e r v a l s A a n d B t o g e t h e r , s e e k i n g f o r a good p r e d i c t o r a t e a c h p o i n t of i n t e r v a l B b a s e d o n t h e v a l u e s i n i n t e r v a l A. A s a p r a c t i c a l m a t t e r i t m a y w e l l s u f f i c e t o f i n d r e g r e s s i o n s f o r p o i n t s of i n t e r v a l B s p a c e d e v e r y 8 t o 16 m i l l i s e c o n d s and i n t e r p o l a t e b e t w e e n . F i n d i n g a s m o o t h e d m e a n s p e c t r u m f o r i n t e r v a l A will guide us to f a i r l y good l i n e a r p r e d i c t o r s , a n d s t e p w i s e a d j u s t m e n t ( u s i n g b o t h r a w and l o c a l l y smoothed values as possible c a r r i e r s ) can polish these p r e d i c t o r s up easily. L e t n o w i n t e r v a l C c o n s i s t of 256 p o i n t s at 8 m i l l i s e c o n d s p a c i n g r u n n i n g u p t o j u s t b e f o r e t h e s t i m u l u s c h a n g e . It w i l l c o n t a i n t h e l a s t 256 - 32 = 2 2 4 p o i n t s of i n t e r v a l A e v e r y 8 t h p o i n t , 32 i n a l l , of i n t e r v a l B . We c a n n o w a p p l y t h e s a m e p r e d i c t o r s t o i n t e r v a l C t h a t w e f o u n d d i d w e l l w h e n a p p l i e d t o i n t e r v a l A. T h e y m a y o r m a y n o t do w e l l i n s u p p r e s s i n g b a c k g r o u n d o v e r i n t e r v a l D. (If w e h a d n o t changed the s t i m u l u s , they would have done well. ) Since we a r e in doubt a s to how t h e s e p r e d i c t o r s w i l l w o r k , it is n a t u r a l t o t r e a t t h e m a s c o v a r i a t e s , and a s k w i t h w h a t c o e f f i c i e n t s t h e y w i l l do t h e i r b e s t . T h e s e c o e f f i c i e n t s m i g h t w e l l c h a n g e a s w e m o v e a c r o s s i n t e r v a l D. If t h e p a t t e r n of t h e i r change is c l e a r enough, t h i s p a t t e r n ought to t e l l us m o r e about w h a t is r e a l l y going on. Plausibility a r g u m e n t s can be m a d e f o r trying both " p r e d i c t o r v a l u e " a n d " s m o o t h e d r a t e of c h a n g e of p r e d i c t o r v a l u e w i t h t i m e p r e d i c t e d f o r " a s c o v a r i a t e s . (An e m p i r i c i s t m i g h t a l s o b e w i l l i n g to d u m p i n t h e s q u a r e of t h e l a t t e r . ) 359

Martin and Lucas, discussion

In a n y e v e n t , c o v a r i a n c e , a c r o s s t r i a l s b u t w i t h i n i n d i v i d u a l s , o u g h t t o b e t r i e d a s a w a y to g e t b e t t e r d a t a f o r t h e f i n a l a n a l y s i s . Another, m o r e complex h y p o t h e s i s , would be that the stimulus c h a n g e e n t r a i n s t h e a l p h a r h y t h m s . A w a y t o t r y to u s e t h i s would be the following: •

m a k e predictions through i n t e r v a l D as b e f o r e ;

•

find t h e i r F o u r i e r t r a n s f o r m s ;

•

r o t a t e e a c h ( c o s i n e c o e f f , s i n e c o e f f ) 2 - v e c t o r i n two w a y s , so t h e r e s u l t i n g c o v a r i a t e (a) h a s a + m a x i m u m at t h e s t i m u l u s c h a n g e , (b) h a s a + m a x i m u m s l o p e there;

•

t r a n s f o r m e a c h of (a) and (b) b a c k ; and

•

u s e t h r e e c o v a r i a t e s , t h e o r i g i n a l p r e d i c t o r and t h e r e t r a n s f o r m s of (a) and (b).

If t h i s d o e s b e t t e r , i t i s t h e r e s i d u a l s f r o m t h i s t h a t o u g h t t o go to f u r t h e r analysis. What f u r t h e r a n a l y s i s ? It s e e m s t o m e t h a t a D F T ( d i s c r e t e F o u r i e r t r a n s f o r m ) s h o u l d c o m e f i r s t . We k n o w t h a t d i f f e r e n t f r e q u e n c y b a n d s b e h a v e q u i t e d i f f e r e n t l y . We o u g h t t o h a v e a l o o k at w h a t w e c a n do a b o u t t h i s f i r s t . W h i l e i t w o u l d b e p o s s i b l e t o D F T t h e 256 v a l u e s (which m a y h a v e b e e n a d j u s t e d by c o v a r i a n c e ) d i r e c t l y , which c o r r e s p o n d s to u s i n g a r e c t a n g u l a r d a t a w i n d o w , and w h i l e it m a y be w o r t h w h i l e t o do t h i s f o r c o m p a r i s o n , p a s t e x p e r i e n c e s u g g e s t s s t r o n g l y t h a t t h i s w i l l n o t t u r n o u t t o b e t h e b e t t e r t h i n g to d o . D F T u s i n g a d a t a w i n d o w of t h e f o r m flat at 1 p i e c e s of c o s i n e data here 360

Off Response to Audio Tone, discussion

with the ends about 2 0 m i l l i s e c o n d s long is m u c h m o r e likely to give c l e a n r e s u l t s . What do we do with o u r sine and c o s i n e c o e f f i c i e n t s ? Having obtained t h e m f o r m a n y r u n s f o r a single individual, we c a n think of plotting the s i n e - a n d - c o s i n e - c o e f f i c i e n t v e c t o r s f o r e a c h f r e q u e n c y . We will get s o m e kind of point cloud whose c h a r a c t e r i s t i c s will change f r o m f r e q u e n c y to f r e q u e n c y . One s i m p l e a p p r o a c h t h a t could be t a k e n would be to a s k what shrinking of the m e a n v e c t o r of e a c h point cloud l e a d s to m i n i m u m m e a n s q u a r e e r r o r . A f t e r finding the shrunken v a l u e s , t h e y c a n be r e t r a n s f o r m e d to a t i m e function that m i g h t be a b e t t e r t y p i c a l p a t t e r n f o r an individual than the a c t u a l t i m e m e a n s ( t h e m s e l v e s a r e t r a n s f o r m a t i o n of the u n s h r u n k e n m e a n v e c t o r s ) . Another a p p r o a c h would be to r e l a t e the D F T c o m p o n e n t s f o r i n t e r v a l D with t h o s e f o r i n t e r v a l B . H e r e r e l a t i o n s h i p could mean: •

o r d i n a r y ( v e c t o r ) r e g r e s s i o n , which would take c a r e of u n u s u a l d e p e n d e n c e s of p h a s e shift on f r e q u e n c y ;

•

r e g r e s s i o n on (or a l s o on) the amplitude of the e a r l i e r v e c t o r ; and

•

s o r t i n g out, s a y into 3 o r 4 e q u i n u m e r o u s p a r t s , in a c c o r d with the v a l u e s of the e a r l i e r a m p l i t u d e , followed by looking at s p r e a d s to s e e if weighting by s o m e function of the e a r l i e r amplitude should be helpful.

W h e t h e r o r not s o m e t i m e - s i d e function e x p a n s i o n i s to be u s e d , it s e e m s l i k e l y t h a t an i n i t i a l e x c u r s i o n to the f r e q u e n c y side will be w e l l w o r t h w h i l e . A n o t h e r p o s s i b i l i t y f o r the p r e d i c t i o n g a m e , which would be m o r e c o m p a c t , and m i g h t be m o r e e f f e c t i v e , would be to p r e d i c t s h o r t p e r i o d ( i n t e r v a l B o r D) F o u r i e r c o e f f i c i e n t s f r o m long p e r i o d

361

Martin and Lucas, discussion

( i n t e r v a l A o r C) F o u r i e r c o e f f i c i e n t s . H e r e only about 8 o r 10 (or m a y b e 16 o r 20) 2 - v e c t o r s would n a t u r a l l y a p p e a r a s c a r r i e r s f o r a g i v e n 2 - v e c t o r , and the s a m p l e s i z e c o u l d o f t e n be d o u b l e d b y 90° r o t a t i o n s . D i f f e r e n c e s in r e g r e s s i o n c o e f f i c i e n t s f o r B on A and D on C c o u l d be a s s i g n e d to " e n t r a i n m e n t " and m i g h t t h r o w a d d i t i o n a l l i g h t on the p r o b l e m . ( l a t e r c o m m e n t s , a f t e r r e a d i n g f i n a l v e r s i o n of p a p e r ) 1. We h a v e not yet s e e n e i t h e r m i d d l e o r l a t e r s t a g e s of a n a l y s i s for this p r o b l e m . 2. I w o u l d l i k e , w e r e I a n a l y z i n g the d a t a , to s e e p l o t s of " v a r i a n c e about m e a n " a g a i n s t " t i m e a f t e r s t i m u l u s " and a g a i n s t " s l o p e of m e a n . " (And the l a t t e r f o r a l l but the f i r s t p a r t of the r e s p o n s e . ) H a r d l o o k s at t h e s e m i g h t l e a d u s to s o m e i n s i g h t s a s to p l a u s i b l e m o d e l s (e. g . , a t i m e s t r e t c h - s h r i n k c o m p o n e n t in the r e s p o n s e ought to show u p a s i n c r e a s e d v a r i a n c e n e a r h i g h slopes). 3. A l l of u s — u n w i s e s t a t i s t i c i a n s t h a t we a r e — would l i k e to l o o k at the f u l l p r i n c i p a l c o m p o n e n t a n a l y s i s of d i m e n s i o n 2 5 6 . ( E x c e p t that one w o u l d n e e d about 400 r u n s , one c o u l d think of d o i n g t h i s . I h a v e done e i g e n v a l u e s in 104 d i m e n s i o n s . ) Why not p l a n to do s o m e t h i n g l e s s than t h i s , but m o r e f l e x i b l e than f i t t i n g c h o s e n s e t s of f u n c t i o n s ? One a p p r o a c h would be to t a k e a s t e p w i s e a p p r o a c h . Consider t i m e s l 6 h + 1, l 6 h + 2 , . . . , l 6 h + 16 f o r h = 0, 1, 2 , . . . , 15. F o r e a c h h s e p a r a t e l y , l o o k at the d e v i a t i o n s of the 75 r u n s f r o m t h e i r m e a n (or the t h r e e s e t s of 2 5 , e a c h f r o m i t s own m e a n s , e t c . ) and do a f u l l p r i n c i p a l c o m p o n e n t s a n a l y s i s . H o p e f u l l y 0 to 3 c o m p o n e n t s w i l l be o u t s t a n d i n g at e a c h h. If 0, we w i l l i n c l u d e the l a r g e s t a v a i l a b l e , g i v i n g u s 1 to 3. T h e s e c o n d s t e p i s to c o m b i n e the 16 to 40 r e s u l t i n g c o m p o n e n t s in a n o t h e r p r i n c i p a l c o m p o n e n t a n a l y s i s . The t r u l y o u t s t a n d i n g c o m p o n e n t s h e r e , if a n y , c a n b e t a k e n a s r e a s o n a b l e l e a d s .

362

Off Response to Audio Tone, discussion

G i v e n one such, we c a n s e p a r a t e the t r i a l s into 3 to 5 z o n e s b a s e d o n t h e v a l u e of t h i s c o m p o n e n t . T h e m e a n s of t h e z o n e s — m o r e p r e c i s e l y the d i f f e r e n c e s of, or a p p r o p r i a t e c o n t r a s t s among, t h e s e m e a n s — c a n g i v e u s t h e n a n e v e n b e t t e r p i c t u r e of w h i c h c o m p o n e n t s s e e m to show l a r g e v a r i a n c e (within s u b j e c t ) . An a l t e r n a t i v e t o 16 b l o c k s of 16 l a i d e n d t o e n d i s 15 b l o c k s of 32 l a i d " 1 6 t o w e a t h e r " (in s h i n g l e t e r m i n o l o g y ) , t h a t i s I 6 h + 1 , l 6 h + 2, . . . , I 6 h + 32 f o r h = 0, 1, . . . , 1 5 . In e i t h e r c a s e , t h e s t e p w i s e c h a r a c t e r of t h e p r i n c i p a l c o m p o n e n t s w o u l d k e e p u s f r o m looking at two w e i r d c o m p o n e n t s . (The l i m i t e d r e s u l t s of t h e s t e p w i s e e i g e n i n g m a y w e l l p r o v e s a f e r t h a n a f u l l 2 5 6 - d i m e n s i o n a l a n a l y s i s f o r f e w e r t h a n , s a y , 2000 t r i a l s . Who k n o w s ? ) 4 . T h e t r a c i n g of 6 0 H z s y n c h r o n i s m t o t h e s t i m u l u s c o n t r o l offers a simple design opportunity for later experiments. A r e v e r s i n g s w i t c h in the p o w e r l e a d f o r t h e c o n t r o l only should a l l o w l o c k i n g - i n t h e 60 H z i n r e v e r s e d p h a s e o n h a l f t h e t r i a l s , so that its e f f e c t will c a n c e l out in the m e a n . 5. D r . J o h n H a r t i g a n ' s c o m m e n t s f r o m "the p a t t e r n r e c o g n i t i o n p o i n t of v i e w " t a c i t l y a s s u m e t h a t t h e f e a t u r e s of i m p o r t a n c e w i l l show m o s t c l e a r l y in a t i m e h i s t o r y . I see no r e a s o n to a s s u m e t h i s — a n d , I g u e s s , n o r e a s o n t o a s s u m e t h e c o n t r a r y . It i s a n e v e r - p r e s e n t d a n g e r of d a t a a n a l y s i s t o a s s u m e t h a t , if t h e d a t a would like to show u s s o m e t h i n g , we can see it in the f o r m f i r s t p r e s e n t e d t o u s . ( T h e g r e a t u s e f u l n e s s of p o l y n o m i a l f i t s , a n d r o w - P L U S - c o l u m n a n a l y s e s of 2 - w a y t a b l e s of r e s p o n s e s a r e p r o m i n e n t d e n i a l s of a n y s u c h a s s u m p t i o n . )

363

CHAPTER 8 T H E USE O F D E N S I T Y E S T I M A T E S B A S E D ON O R T H O G O N A L EXPANSIONS

R I C H A R D A.

KRONMAL

D e p a r t m e n t of B i o s t a t i s t i e s , U n i v e r s i t y of W a s h i n g t o n ,

Seattle

and MICHAEL TARTER D i v i s i o n of B i o s t a t i s t i c s ,

U n i v e r s i t y of C a l i f o r n i a ,

Berkeley

In t h i s c h a p t e r w e d i s c u s s t h e a p p l i c a t i o n of o r t h o g o n a l p o l y n o m i a l e s t i m a t e s of t h e u n i v a r i a t e a n d b i v a r i a t e d e n s i t i e s a s w e l l a s t h e c u m u l a t i v e f o r a s e t of d a t a c o n t r a s t i n g c o r o n a r y h e a r t d i s e a s e p a t i e n t s w i t h a g e m a t c h e d " n o r m a l s . " We a r e c o n c e r n e d p r i m a r i l y w i t h t h e u s e of t h e s e t e c h n i q u e s f o r d e s c r i p t i o n of t h e d a t a , p a r t i c u l a r l y t h e i r u s e as p o s s i b l e r e p l a c e m e n t s f o r the h i s t o g r a m , two v a r i a b l e plot and s t e p function. A d e s c r i p t i o n of t h e s t u d y i s g i v e n b e l o w , a n d a d a t a s e t d e s c r i p t i o n t a k e n f r o m a p u b l i s h e d p a p e r d e s c r i b i n g t h e r e s u l t s of t h e s t u d y , i s g i v e n a t t h e e n d of t h e c h a p t e r . T h e r e s u l t s of t h e u s e of t h e s e m e t h o d s o n t h e d a t a a r e d e s c r i b e d , w i t h p a r t i c u l a r a t t e n t i o n g i v e n t o t h e e s t i m a t i o n of t h e c u m u l a t i v e a n d of t h e b i v a r i a t e d i s t r i b u t i o n s of s e v e r a l of t h e v a r i a b l e s . The general theory for orthogonal polynomial density estimation f o r t h e o r t h o n o r m a l s y s t e m b a s e d on t h e c o m p l e x t r i g o n o m e t r i c functions is also given.

365

Kronmal and Tarter

DESCRIPTION O F T H E STUDY This analysis is b a s e d on a study conducted by K a s s e r and B r u c e (1969) c o m p a r i n g a group of c o r o n a r y h e a r t d i s e a s e patients with a group of age m a t c h e d n o r m a l s . The d e s c r i p t i o n given below is taken f r o m the published paper with s e c t i o n s omitted for b r e v i t y . C o r o n a r y h e a r t d i s e a s e exhibits m a r k e d v a r i a b i l i t y in its c l i n i c a l m a n i f e s t a t i o n s . While s o m e patients a r e disabled by angina p e c t o r i s , o t h e r s surviving m y o c a r d i a l i n f a r c t i o n pursue t h e i r usual a c t i v i t i e s with little or no s y m p t o m s . N e v e r t h e l e s s , h e m o d y n a m i c studies indicate that m o s t c o r o n a r y patients have evidence of i m p a i r e d m y o c a r d i a l c o n t r a c t i l i t y . When s t r e s s e d by e x e r c i s e or i s o p r o t e r e n o l i n fusion, they show s u b n o r m a l i n c r e m e n t s in c a r d i a c output, stroke volume, and s y s t o l i c e j e c t i o n r a t e . M o r e advanced d i s e a s e m a y m a n i f e s t o v e r t left v e n t r i c u l a r f a i l u r e with elevated r e s t i n g left v e n t r i c u l a r e n d - d i a s t o l i c and p u l m o n a r y capillary pressures. In a p a r a l l e l m a n n e r , advancing age a l s o a d v e r s e l y a f f e c t s c a r d i o v a s c u l a r function. W h e r e a s older individuals show only slight a b n o r m a l i t i e s at s u b m a x i m a l work l o a d s , t h e i r m a x i m a l e x e r c i s e p e r f o r m a n c e is significantly l i m i t e d . Oxygen consumption, h e a r t r a t e , s t r o k e v o l u m e , and c a r d i a c output a r e all r e d u c e d at high w o r k l o a d s . Consequently, m o s t m i d d l e - a g e d c o r o n a r y patients suffer f r o m the added limitations of both d i s e a s e and i n c r e a s i n g age. Many c l i n i c i a n s l i m i t e x e r c i s e testing of c o r o n a r y patients to simply detecting m y o c a r d i a l i s c h e m i a f r o m p o s t e x e r t i o n a l S - T d e p r e s s i o n . Y e t with slightly m o r e e f f o r t , an objective a s s e s s m e n t of e a c h patient's functional c a p a c i t y can be obtained. In this r e g a r d , m a x i m a l e x e r c i s e on a m u l t i s t a g e t r e a d m i l l has proved to be a useful t e s t . In l e s s than 12 m i n . , a s y m p t o m a t i c as well as a c l a s s IV c a r d i a c patients (New Y o r k H e a r t A s s o c i a t i o n c l a s s i f i c a t i o n ) can be e x e r c i s e d usirtg the s a m e t e s t p r o c e d u r e , and t h e i r p e r f o r m a n c e c o m p a r e d quantitatively to that of n o r m a l s u b j e c t s .

366

Density Estimates

This study w a s u n d e r t a k e n to define the m a x i m a l e x e r c i s e c a p a c i t y of n o r m a l m e n a n d c o r o n a r y p a t i e n t s a n d a s s e s s t h e r e l a t i v e c o n t r i b u t i o n s of a g i n g a n d d i s e a s e t o t h e o v e r a l l f u n c t i o n a l l i m i t a t i o n s of a n i n d i v i d u a l p a t i e n t . . . . O n e h u n d r e d s e v e n t e e n (117) m e n , w h o s e a g e s r a n g e d f r o m 34 t o 7 3 y e a r s w i t h a m e a n a g e of 52 y e a r s w e r e s e l e c t e d b e c a u s e of a c l i n i c a l d i a g n o s i s of d e f i n i t e c o r o n a r y h e a r t d i s e a s e . T h e d i a g n o s i s w a s b a s e d o n (1) r e c u r r e n t e x e r t i o n a l c h e s t p a i n t y p i c a l of a n g i n a p e c t o r i s , (2) a Q w a v e of at l e a s t 0. 0 4 - s e c d u r a t i o n on a r e s t i n g e l e c t r o c a r d i o g r a m , o r (3) a c l i n i c a l e p i s o d e c o m p a t i b l e w i t h a c u t e m y o c a r d i a l infarction substantiated by elevated serum enzymes or myoc a r d i a l i n j u r y r e c o r d e d on t h e e l e c t r o c a r d i o g r a m , o r b o t h . All m e n w e r e in s i n u s r h y t h m , and none w e r e r e c e i v i n g d i g i t a l i s . No p a t i e n t w a s e x e r c i s e d e a r l i e r t h a n 2 m o n t h s following a m y o c a r d i a l infarction. F o r c o m p a r i s o n , 117 a g e - m a t c h e d n o r m a l m e n , w h o s e a g e s r a n g e d f r o m 33 t o 7 2 , w i t h a m e a n a g e of 51 y e a r s , w e r e c h o s e n f r o m a g r o u p of h e a l t h y f a c u l t y a n d Y M C A v o l u n t e e r s . In a d d i t i o n , a t h i r d g r o u p of 6 2 n o r m a l y o u n g m e n , w h o s e a g e s r a n g e d f r o m 15 t o 35 y e a r s , w i t h a m e a n of 25 y e a r s , w e r e e v a l u a t e d in the o u t p a t i e n t c l i n i c . B o t h t h e s e g r o u p s of m e n w e r e j u d g e d t o b e n o r m a l b y h i s t o r y , p h y s i c a l e x a m i n a t i o n , c h e s t x - r a y s , and 1 2 - l e a d e l e c t r o c a r d i o g r a m . All n o r m a l s u b j e c t s w e r e n o r m o t e n s i v e a c c o r d i n g to World H e a l t h O r g a n i z a t i o n c r i t e r i a , with a r e s t i n g blood p r e s s u r e of l e s s t h a n 1 6 0 / 9 5 m m H g . D a t a w e r e a l s o a v a i l a b l e on 79 of t h e o l d e r n o r m a l m e n w h o h a d b e e n t e s t e d p r e v i o u s l y . The m u l t i s t a g e e x e r c i s e t e s t involves an u n i n t e r r u p t e d s e r i e s of w o r k l o a d s on a m o t o r d r i v e n t r e a d m i l l ; t h e i n i t i a l s u b m a x i m a l l o a d ( s t a g e 1) r e q u i r e s w a l k i n g s l o w l y o n a 10% g r a d e . S i n c e t h e r e i s no i n c r e a s e in o x y g e n u p t a k e a f t e r 3 m i n . of s u b m a x i m a l e x e r t i o n , t h e s p e e d a n d g r a d e a r e i n c r e a s e d e v e r y 3 m i n . Each p e r s o n continues e x e r c i s e until a s e l f - d e t e r m i n e d e n d p o i n t of e x h a u s t i n g f a t i g u e , m a r k e d

367

Kronmal and Tarter

d y s p n e a , aching o r w e a k n e s s in the l e g s , d i z z i n e s s , pain, o r v a r i o u s c o m b i n a t i o n s of t h e s e . . . .

chest

E a c h s u b j e c t w a s e x a m i n e d b e f o r e and a f t e r the t e s t by a p h y s i c i a n who r e m a i n e d in a t t e n d a n c e during the e n t i r e p r o cedure. . . . . H e a r t r a t e s w e r e r e c o r d e d at r e s t and during e a c h m i n u t e of e x e r c i s e and r e c o v e r y , while blood p r e s s u r e s w e r e obtained at r e s t and on i n i t i a l r e c o v e r y as w e l l a s 3 and 6 m i n . a f t e r e x e r t i o n . E x e r c i s e p e r f o r m a n c e w a s e v a l u a t e d b y the following p a r a m e t e r s of c a r d i a c function: (1) t o t a l d u r a t i o n of e x e r t i o n , (2) m a x i m a l h e a r t r a t e , and (3) m a x i m a l s y s t o l i c blood p r e s s u r e . . . . R e s t i n g h e a r t r a t e and blood p r e s s u r e w e r e obtained while e a c h s u b j e c t w a s sitting j u s t b e f o r e the s t a r t of the e x e r c i s e . The a i m of the study w a s p r i m a r i l y d e s c r i p t i v e and e x p l o r a t o r y . Thus the a u t h o r s w e r e i n t e r e s t e d in t e c h n i q u e s that would d e s c r i b e t h e i r d a t a w e l l and allow t h e m to d e v e l o p h y p o t h e s e s f o r f u t u r e s t u d i e s . The u s e of o r t h o g o n a l p o l y n o m i a l n o n p a r a m e t r i c d e n s i t y e s t i m a t i o n p r o v i d e d t h e m with a u s e f u l t o o l f o r t h e s e purposes. SOME E X A M P L E S O F T H E USE O F ORTHOGONAL P O L Y NOMIAL D E N S I T Y E S T I M A T E S Although t h e r e a r e m a n y u s e s of d e n s i t y e s t i m a t e s , p a r t i c u l a r l y in the a r e a of m u l t i v a r i a t e a n a l y s i s , the application d e s c r i b e d h e r e is r e s t r i c t e d t o the d e s c r i p t i o n of d a t a . The f o c u s i s on e s t i m a t i o n of the c u m u l a t i v e , the u n i v a r i a t e d e n s i t y , and b i v a r i a t e d e n s i t y ( T a r t e r and K r o n m a l , 1 9 7 0 ; K r o n m a l and T a r t e r , 1 9 6 8 ; and C e n c o v , 1 9 6 2 ) . We i l l u s t r a t e this a p p l i c a t i o n by e x a m p l e s f r o m the d a t a d e s c r i b e d e a r l i e r a s w e l l as with s o m e Monte C a r l o s i m u l a t i o n d a t a .

368

Density Estimates

The u s e of a n e w technique f o r the d e s c r i p t i o n of data m u s t be j u s t i f i e d on at l e a s t t h r e e grounds: i t s m a t h e m a t i c a l s t a t i s t i c a l p r o p e r t i e s , i t s a c c e p t a b i l i t y to applied s t a t i s t i c i a n s , and m o s t i m p o r t a n t l y i t s u s e f u l n e s s to the r e s e a r c h e r . Although t h e r e a r e a n u m b e r of p a p e r s on the m a t h e m a t i c a l a s p e c t s of o r t h o g o n a l p o l y n o m i a l e s t i m a t i o n , t h e r e i s l i t t l e to s u g g e s t the u s e of t h e s e m e t h o d s f o r d e s c r i p t i v e p u r p o s e s . H o w e v e r , this i s to be e x p e c t e d for a topic as new and as m a t h e m a t i c a l l y d i f f i c u l t as t h i s one b e c a u s e of its c o m p l e x i t y and the lack of a c l e a r d e f i n i t i o n as to what c o n s t i t u t e s a good d e s c r i p t i o n of the data. The l a r g e Monte Carlo s t u d i e s d e s c r i b e d b y A n d e r s o n (1969) s e e m to support the v i e w that for s a m p l e s i z e s as s m a l l a s 50, e x c e l l e n t e s t i m a t e s of the underlying p r o b a b i l i t y d e n s i t y a r e obtained through the u s e of the o r t h o g o n a l p o l y n o m i a l e s t i m a t e s . F i g u r e s 1 and 2 show the e s t i m a t e s of a d e n s i t y and c u m u l a t i v e r e s p e c t i v e l y f r o m a s a m p l e of s i z e 200 f r o m the n o r m a l d i s t r i bution with m e a n 0 and v a r i a n c e 1. F i g u r e 3 s h o w s a graph of the a p p r o x i m a t e p r o b a b i l i t y c o n t o u r s for a b i v a r i a t e n o r m a l with m e a n s e q u a l to z e r o and v a r i a n c e s equal to 1 and c o r r e l a t i o n equal to 0. 69 b a s e d on a s a m p l e s i z e of 100. The p r o g r a m g e n e r a t i n g the c o n t o u r s i s s t i l l in the d e v e l o p m e n t a l s t a g e , so t h e s e plots appear on a line printer and thus a r e s o m e w h a t d i s t o r t e d . Although t h i s p r o g r a m w a s u n a v a i l a b l e at the t i m e the K a s s e r - B r u c e study w a s a n a l y z e d , we u s e it to i l l u s t r a t e the potential v a l u e of g r a p h i c a l d i s p l a y s of b i v a r i a t e c o n t o u r s . One c a n s e e f r o m f i g u r e s 1, 2 and 3 that the e s t i m a t e s g e n e r a t e d by t h e s e m e t h o d s s e e m to be quite adequate for d e s c r i b i n g the populations f r o m w h i c h the data w e r e g e n e r a t e d . To our k n o w l e d g e , l i t t l e u s e h a s b e e n m a d e of any of the m e t h o d s f o r n o n p a r a m e t r i c d e n s i t y e s t i m a t i o n (with the p o s s i b l e e x c e p t i o n of i t s u s e in c l a s s i f i c a t i o n ) . In a s t a t i s t i c a l s y s t e m of c o m p u t e r p r o g r a m s (CCSS) for the m a i n t e n a n c e and d e s c r i p t i o n of c o m p l i c a t e d data f i l e s , a p r o g r a m i s included that p r o d u c e s e s t i m a t e s of the u n i v a r i a t e d e n s i t y and c u m u l a t i v e (Kronmal et al, 1970). Other than t h i s package of p r o g r a m s and its u s e by

369

Kronmal and Tarter

~4

-3

-2

F i g u r e 1. Estimate of density for sample of size 200 f r o m N(0, 1).

370

Density Estimates

F i g u r e 2 . E s t i m a t e of cumulative distribution for sample s i z e 2 0 0 f r o m N(0, 1).

371

Kronmal and Tarter

• • o o I* I/) < O UJ > -111 I O! Zl^-oi h- 00 oc u. id o o a u. • > u. >- I- X O K O —• V) w ^ V> Z Ui Uj Z UJ X ID UJ C) X O X3 uj 3 z >- X X t- KX • _J U. < t"OIO CO _J < t- ai a. CO X X QOhUi OC !-> X a. uj u. hXo u. z o x >- o Z> -J x uj r\j 0C > x S< H«-< t- x u O} Z uj < OUJH 4UJZ u. x a: o a»- x xMM UJ o in t£t < t C MC MC M C M C MC C MC M M •i MC M C MC M •J' C MC C MIM o »0 >c •4C MC M 0 >0 C MC M •O >C C M si" 00 CO00 CO 00 O C M IM C M ^ >0 to C O00 CD CajO w a>

376

Density Estimates

IM IM CM

UJ

OC

CM IM

CM CM CM CM CM IM IM IM IM z z 3O ZM -I tX < < > X et UJ ^ CO O CO o » o Z Of

111 3 K < Z M OC • < I/) > oc «•> 3 CO O I0. z o o o UJ K U. t -4# * M CM CM •4" * 1- -4- •4" * IM IM IM •4" -t -4- •4"•4 CM * •4- >0 •c «o 0 «O CMCM «t >0 «o * « « O > 0 S 3 Z Z Z • O C M I M •4« «O »O vO «o

CO

IM

•o

CD CO CO

rg CM

-o

eo

CM

•c

3 O K Z O U

oo

rg fM

X i

« Z a> o < «« CO t o < oe z o. < _i a a. UJ x

«t rg

>0

00

-o

oo 0

00 00

* *

* o * o # » * tri • CO

rg

oo

X

*

rg

0

CM rg rg -i-

*

* *

rg

-o

CO 00

00 CD

*

•

IM -i-

00 00

o >0

rg rg

co oo oo oo

rg rg rg o-c- op CO oo00 00 00 CO 00 00 00 LL Z a.

„o „ HH o o oo o

•

*

CT"

381

Kronmal and Tarter o + o• i in o f\J

•m

•

m H » oo M iz I/) tM JO a a. m x u. >- o i/i oe UJ UJ Xm t- x 3

*

a ert

m • 00 II o 00 i

+

< —

+

X at i/i O UJ z t< UJ z X M KQ OC (to o o u. u

o (Pi • 01 ^

•

+

o o • m >0

>

•o o o

t—
UJ UJ ac x a. tO >• O -J O uj _i Ha> < X o « i- X -i a o or i- a. < I00 z u. a

• a

+

+

•

o m •

00

a

o

00 I X

m CM -Io

II 0c

*

(M

*

m

»

•o -i

i . e . , w e u s e e x p r e s s i o n (16) f o r o u r i n c l u s i o n r u l e . A

It i s w o r t h n o t i n g t h a t f ^ w i l l b e a r e a l v a l u e d f u n c t i o n if t h e t e r m s e l e c t i o n r u l e g i v e n in f o r m u l a (16) i s u s e d s i n c e if B^. i s selected then w i l l a l s o be s e l e c t e d and t h i s h a s t h e e f f e c t of c a n c e l l i n g o u t t h e i m a g i n a r y p o r t i o n of f j ^ . CONCLUSION We w o u l d l i k e to e m p h a s i z e t h a t w e p r o f e s s no s p e c i a l i n s i g h t i n t o t h e u s e f u l n e s s o r v a l u e of t h e s e m e t h o d s . We do f e e l , h o w e v e r , t h a t s u c h m e t h o d s m a y f i n d a p l a c e in t h e t o o l s a v a i l a b l e t o t h e s t a t i s t i c i a n and i n v e s t i g a t o r f o r t h e d e s c r i p t i o n of d a t a . ACKNOWLEDGEMENT T h a n k s t o M r . S. R a m a n , a c a n d i d a t e f o r a P h . D. in B i o s t a t i s t i c s a t B e r k e l e y , f o r t h e u s e of h i s p r o g r a m f o r t h e c o n t o u r s e s t i m a t i o n . We w o u l d a l s o l i k e t o e x p r e s s o u r a p p r e c i a t i o n t o D r s . B r u c e and K a s s e r f o r p e r m i s s i o n to r e p r i n t p o r t i o n s of t h e i r p a p e r and f o r the u s e of t h e i r d a t a . This r e s e a r c h w a s s u p p o r t e d in p a r t by PHS R e s e a r c h C a r e e r Award Grant 1 -K03-GM38645-01.

387

Kronmal and Tarter DATA SET DESCRIPTION

DATA - KASSER AND BRUCE CORONARY HEART DISEASE STUDY (R. KRONMAL) CASES AND VARIABLES 234 CASES (SUBJECTS) 34 VARIABLES - 16 MEASUREMENT, 16 CATEGORICAL, 2 IDENTIFICATION BRIEF DESCRIPTION VAR

NAME

***

****

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

SUBJECT NUMBER CARD NUMBER SOURCE SEX AGE FUNCTIONAL CLASS ACTIVE DIGITALIS TREATMENT 12-LEAD ECG RESULTS RHYTHM SYSTOLIC BLOOD PRESSURE RESTING HEART RATE MAXIMUM HEART RATE DURATION ON TREADMILL SYSTOLIC BLOOD PRESSURE AT PHYSICAL EXAM DIASTOLIC BLOOD PRESSURE AT PHYSICAL EXAM MAXIMUM SYSTOLIC BLOOD PRESSURE ON TREADMILL MAXIMUM DIASTOLIC BLOOD PRESSURE ON TREADMILL HISTORY OF MYOCARDIAL INFARCTION HISTORY OF ANGINA PECTORIS ECG XRAY HISTORY OF HIGH BLOOD PRESSURE SURVIVAL STATUS MONTHS NUMBER OF PAST MYOCARDIAL INFARCTIONS ANGINA PECTORIS CONGESTIVE HEART HEART RATE AT STAGE ONE ON TREADMILL CHANGE IN HEART RATE SYSTOLIC BLOOD PRESSURE TIMES HEART RATE VOLUME OF OXYGEN PREDICTED FOR HEALTHY MAN WITH SIMILAR CHARACTERISTICS ACTUAL ESTIMATED VOLUME OF OXYGEN FOR THIS MAN FUNCTIONAL AEROBIC IMPAIRMENT

PURPOSE OF STUDY THE PURPOSE OF THE STUDY IS TO EXPLORE THE RELATIVE CONTRIBUTIONS OF AGE AND CORONARY HEART DISEASE TO THE IMPAIRMENT OF FUNCTIONAL CAPACITY. THE SAMPLE CONSISTS OF 117 MALE CORONARY PATIENTS AND 117 AGE-MATCHED HEALTHY MIDDLE-AGED MEN. REFERENCE KASSER, IRWIN S. AND ROBERT A. BRUCE. COMPARATIVE EFFECTS OF AGING AND CORONARY HEAiRT DISEASE AND SUBMAXIMAL AND MAXIMAL EXERCISE. CIRCULATION 39, 1969, 759-774.

388

Density Estimates DETAILED DESCRIPTION VAR

COL

FORMAT

DESCRIPTION

***

***

******

***********

1

1-3

XXX.

2

4

X.

3

5

X.

4

6

X.

5 6

7-8 9

XX. X.

7

10

X.

8

11

X.

9

12

X.

10

13

X.

11

14-16

XXX.

12 13 14

17-19 20-22 23-25

XXX. XXX. XXX.

15 16

26-28 29-31

XXX. XXX.

17

32-34

XXX.

18

35-37

XXX.

19

38

X.

20

39

X.

21

40

X.

SUBJECT NUMBER 001 - 364 = CONTROL GROUP 401 - 517 = CORONARY GROUP CARD NUMBER 1 = CARD 1 SOURCE 1 = CLINIC 5 = FACULTY 7 = YMCA SEX 1 = MALE AGE (YEARS) FUNCTIONAL CLASS 0 = NONE 1 = MINIMAL 2 = MODERATE 3 = MODERATE TO SEVERE 4 = SEVERE ACTIVE 0 = UNKNOWN 1 = VERY 2 = NORMAL 3 = LIMITED DIGITALIS TREATMENT 0 = NO 1 = YES 12-LEAD ECG RESULTS 0 = UNKNOWN 1 => NORMAL 3 = ABNORMAL RHYTHM 0 = UNKNOWN 1 = NORMAL 2 = ATRIAL FIBRILLATION 3 = OTHER SYSTOLIC BLOOD PRESSURE 0 = UNKNOWN RESTING HEART RATE MAXIMUM HEART RATE DURATION ON TREADMILL (SECONDS) 999 = UNKNOWN SYSTOLIC BLOOD PRESSURE AT PHYSICAL EXAM DIASTOLIC BLOOD PRESSURE AT PHYSICAL EXAM 999 = UNKNOWN MAXIMUM SYSTOLIC BLOOO PRESSURE ON TREADMILL 999 = UNKNOWN MAXIMUM DIASTOLIC BLOOD PRESSURE ON TREADMILI 999 = UNKNOWN HISTORY OF MYOCARDIAL INFARCTION 0 = NONE 1 = POSSIBLE 2 = DEFINITE 9 = CONTROL GROUP HISTORY OF ANGINA PECTORIS 0 = NONE 1 = POSSIBLE 2 = DEFINITE 9 = CONTROL GROUP ECG

389

Kronmal and Tarter

22

41

X.

23

42

X.

24

43

X.

25

44--45

XX.

26

46

X.

27

47

X.

28

48

X.

29

49--51

XXX.

30

52--54

XXX.

31

55--57

XXX.

32

58--60

XXX.

33

61--63

XXX.

34

64--66

XXX.

FORMAT IS

0 = NORMAL 1 = POSSIBLE MYOCARDIAL INFARCTION 2 = PROBABLE MYOCARDIAL INFARCTION 3 = OTHER 9 = UNKNOWN OR CONTROL GROUP XRAY 0 = NORMAL 1 = ENLARGED 9 = UNKNOWN OR CONTROL GROUP HISTORY OF HIGH BLOOD PRESSURE 0 = NORMAL 1 = HYPERTENSIVE 9 = CONTROL GROUP SURVIVAL STATUS I = ALIVE 2 = DEAD 8 = CONTROL GROUP 9 = UNKNOWN MONTHS 99 = UNKNOWN OR CONTROL GROUP NUMBER OF PAST MYOCARDIAL INFARCTIONS 1 . ] 1 — I 2 = 2 3 = NONE 8 = CONTROL GROUP 9 = UNKNOWN ANGINA PECTORIS 1 = SLIGHT 2 = SIGNIFICANT 3 = NONE 8 = CONTROL GROUP 9 = UNKNOWN CONGESTIVE HEART I = YES 3 = NONE 8 * CONTROL GROUP 9 = UNKNOWN HEART RATE AT STAGE ONE ON TREADMILL 999 = UNKNOWN CHANGE IN HEART RATE (MAXIMUM HEART RATE RESTING HEART RATE) SYSTOLIC BLOOD PRESSURE TIMES HEART RATE 999 = UNKNOWN VOLUME OF OXYGEN PREDICTED FOR HEALTHY MAN WITH SIMILAR CHARACTERISTICS ACTUAL ESTIMATED VOLUME OF OXYGEN FOR THIS MAN 999 = UNKNOWN FUNCTIONAL AEROBIC IMPAIRMENT 999 = UNKNOWN

(F3.0t3F1.0»F2.0,5F1.0,8F3.0,6F1.0,F2.0,3F1.0,6F3.0)

N = 23* LOCATION OF DATA CARD IMAGE - FS.C073.CKASS1 LISTING OF DATA (THE FIRST TWO LINES ARE COLUMN GUIDES» 0 1 2 3 4 5 6 12345 6789 0123456789 012345678 901234567 89012 345678 901 234 567 890 123 456 117 1480 1031 80 76 180577120 217 1490 1011 80 68 176785120

90200 80 99999 899888 114 104 359 388 359 7 80200 80 99999 899888 95 108 351 385 461 - 1 9

390

Density Estimates 317 417 617 1417 1617 2117 2517 2917 3017 3117 3317 3417 3617 3817 4417 4517 4817 4917 5117 5317 5517 5617 5717 5917 6017 6217 6617 6717 6817 7017 7117 7217 7317 7417 7917 8217 8417 9017 9217 9417 10317 10517 10817 10917 11117 11717 11917 12017 12217 12517 13117 13217 13417 13617 13717 14117 14917 15217 15317 15517 15717 16317 16817 17417 17617 17917 18017

1410 1520 1520 1480 1450 1510 1590 1440 1590 1440 1620 1420 1520 1460 1510 1460 1660 1390 1520 1450 1540 1540 1490 1560 1450 1600 1480 1500 1600 1410 1600 1520 1480 1550 1590 1390 1720 1550 1390 1540 1520 1560 1460 1390 1380 1610 1460 1490 1530 1380 1600 1490 1500 1510 1570 1390 1520 1480 1530 1530 1480 1420 1520 1610 1^30 1630 1470

1011 4 8 70 1031 46 57 1011 40 66 1011 10 6 0 2011 38 84 2031 84 68 2011 70 64 1 0 1 1 1 8 0 72 1011 9 4 75 1011 0 57 1011 78 76 1011 36 63 1011 76 74 2011 80 65 1011 6 8 54 1011 40 62 1011 60 70 1 0 1 1 1 4 0 64 1011 50 45 1011 70 66 1011 0 68 1011160 67 1011 70 60 1011110 68 1011 6 4 55 1011 56 84 1011 26 66 1031 90 58 1011 30 77 1011 80 63 1011 46 65 1011110 60 1011 54 80 1031 56 78 1031 38 68 2011 50 65 1 0 3 1 3 0 SO 2031 76 88 1031 20 78 1011 88 64 1011 30 56 1011 2 4 64 1011 84 80 1011 40 66 1011 20 68 1011 62 88 2011 50 80 2011 2 2 54 2011 62106 2011 58 56 2011 48 75 2033 82 64 2011 76 80 2 0 1 1 2 0 72 2011 6 0 88 2031 6 0 86 2011 20 62 1011 62 76 2011 60 76 2 0 1 1 1 0 2 80 2011 3 4 75 1 0 1 1 5 0 56 2011 4 0 80 1011 10 68 2011 0 84 1011 52 58 1 0 1 1 2 6 72

174775132 170624144 170567110 174720130 190570112 182517 96 148558120 194607110 180583116 184765190 152523112 190688146 152496104 174445120 150635122 170630130 156476150 194585140 158705140 170675150 184695150 190740120 152523150 176675130 160555 96 184555130 174676114 180750130 174602110 180708110 180590144 166624130 166681156 170647120 184578112 180575110 130218170 182573136 180610130 184660138 156640130 168580130 176560126 180742140 170623120 150262118 190580130 182585118 190596148 184575122 160425112 168550108 190575164 190610130 200455140 200610120 184618110 176540118 184615140 184624118 184583106 176 790130 170443100 148487130 194575130 156465134 180551124

8 2 1 8 0 60 9 9 9 9 9 8 9 9 8 8 8 92190 60 99999 899888 68150 60 99999 899888 9 0 1 2 0 60 9 9 9 9 9 8 9 9 8 8 8 82150 60 999 99 899888 62180 30 99999 8 9 9 8 8 8 8 0 1 9 0 50 9 9 9 9 9 8 9 9 8 8 8 70290 60 99999 899888 8 8 2 1 0 80 99999 899888 90190 60 99999 899888 66190 80 9 9 9 9 9 899888 96110 60 99999 899888 84180 80 99999 899888 80200 70 99999 899888 7 0 1 9 0 70 9 9 9 9 9 8 9 9 8 8 8 8 0 1 7 0 70 9 9 9 9 9 8 9 9 8 8 8 76210 60 99999 899888 80280 60 99999 899888 9 0 1 9 0 70 9 9 9 9 9 8 9 9 8 8 8 70220 80 99999 899888 9 0 2 2 0 80 9 9 9 9 9 8 9 9 8 8 8 80280999 99999 899888 80 2 2 0 80 9 9 9 9 9 8 9 9 8 8 8 8 0 2 4 0 60 9 9 9 9 9 8 9 9 8 8 8 80160 70 9 9 9 9 9 8 9 9 8 8 8 8 0 1 8 6 70 9 9 9 9 9 8 9 9 8 8 8 8 0 1 4 0 80 9 9 9 9 9 8 9 9 8 8 8 100220100 99999 899888 8 0 1 4 0 50 9 9 9 9 9 899888 9 0 1 9 0 70 9 9 9 9 9 8 9 9 8 8 8 100190 80 99999 899888 9 0 2 4 0 50 9 9 9 9 9 8 9 9 8 8 8 100210 90 99999 899888 78176 60 9 9 9 9 9 899888 78150 60 99999 899888 7 6 1 6 0 50 9 9 9 9 9 8 9 9 8 8 8 90200 90 99999 899888 8 2 2 1 2 80 9 9 9 9 9 8 9 9 8 8 8 8 0 1 5 0 70 9 9 9 9 9 8 9 9 8 8 8 1 0 0 2 3 6 90 9 9 9 9 9 899888 8 0 1 6 0 50 9 9 9 9 9 8 9 9 8 8 8 9 0 1 5 4 80 9 9 9 9 9 8 9 9 8 8 8 8 0 2 1 0 40 9 9 9 9 9 8 9 9 8 8 8 90180999 99999 899888 88140 70 99999 899888 82180 80 99999 899888 80180 60 99999 899888 82140 70 99999 899888 94210 96 99999 899888 9 0 1 8 0 80 9 9 9 9 9 899888 8 0 1 6 0 60 9 9 9 9 9 899888 74190 60 99999 899888 9 0 2 4 0 80 9 9 9 9 9 899888 100150 90 99999 899888 8 6 2 0 0 80 9 9 9 9 9 899888 90180 92 99999 899888 8 0 1 3 0 70 9 9 9 9 9 8 9 9 8 8 8 90180 60 99999 899888 80200 70 99999 899888 76220999 99999 899888 90140 60 99999 899888 84180 80 999 99 899888 8 0 1 4 0 80 9 9 9 9 9 8 9 9 8 8 8 9 0 1 2 0 70 9 9 9 9 9 8 9 9 8 8 8 8 0 1 3 0 70 9 9 9 9 9 899888 100186 60 9 9 9 9 9 899888 80150 40 99999 899888

391

91 98 96 96 126 106 88 106 106 106 126 102 100 106 86 95 116 110 70 96 108 108 90 94 104 118 108 85 106 90 100 90 116 100 112 106 120 114 118 100 86 92 108 95 100 138 112 92 130 108 114 106 116 102 132 136 102 128 9 99 120 100 96 122 999 124 106 104

104 113 104 114 106 114 84 122 105 127 76 127 78 109 96 108 86 130 113 104 116 123 92 108 105 100 108 12 2 97 117 115 106 86 92 116 115 50 94 102 120 100 104 96 114 102 62 H O 128 84 128 85 104 110 118 112 114 122 100 108 104 109 120 90 80 110 98 108

313 322 254 208 284 327 281 562 377 349 288 208 273 34 7 284 288 32 7 543 300 373 404 531 334 422 255 342 243 395 243 341 341 398 348 299 275 287 259 385 269 434 249 258 369 323 237 269 341 254 398 331 255 319 455 284 399 359 239 316 367 404 257 316 237 177 252 290 269

407 456 377 382 377 354 388 429 365 355 349 329 328 349 399 373 359 362 399 451 351 332 404 413 377 319 362 294 380 387 393 385 340 309 412 363 377 421 396 40 7 372 417 372 439 385 332 367 407 396 348 356 348 388 407 383 443 356 371 4 0 7 42 3 356 365 377 382 388 410 369 393 359 359 381 358 324 183 338 357 412 375 372 399 377 390 367 360 393 350 412 440 415 381 353 204 362 360 354 363 344 368 384 358 325 284 354 345 352 358 349 375 333 299 381 375 346 379 388 341 344 377 344 382 357 362 4 0 4 46 3 346 293 353 315 344 358 348 304 391 346

-12 -1 6 -10 2 5 -6 6 0 -13 5 -2 15 18 -1 2 8 12 -11 -2 -11 -17 13 -10 12 2 -4 -15 -4 -3 -2 -1 -5 -6 0 6 43 -5 9 -7 -3 1 10 -6 8 42 0 -2 -7 6 12 2 -1 -7 10 1 -9 12 -9 -11 -1 -14 15 10 -4 12 11

Kronmal and Tarter 18417

1480

2011

30

78

180545120

72150

50

99999

899888

100

102

269

357

343

18817

1520

1011

20

56

184605140

88160

60

9 9 9 9 9

899888

103

128

294

377

372

1

19017

1400

2011

80

64

164450118

84198

60

99999

899888

100

100

324

378

296

21

3

19117

1480

1011

52

62

176650118

80170

60

99999

899888

104

114

299

388

394

- 1

19317

1540

1011

76

62

184

90210

80

99999

899888

102

122

386

372

429

- 1 5

19517

1390

2011

32

62

184630110

70142

50

99999

899888

94

122

261

381

385

0

19817

1450

1011

54

64

190710136

66190

70

99999

899888

110

126

360

396

424

- 7

720134

19917

1570

2011

18

80

130245124

80142

70

9 9 9 9 9

899888

999

50

184

333

196

41

20017

1480

1011

40

60

164605160

80200

60

99999

899888

94

104

327

388

372

4

20117

1420

1011

36

74

186595100

80136

60

99999

899888

114

112

252

404

368

9

20317

1390

2011

28

75

146450112

80140

50

9 9 9 9 9

899888

999

71

204

381

296

22

20617

1490

1011

80

72

1 7 4 4 7 8 1 5 0

70230

50

99999

899888

102

102

400

385

310

19

20717

1720

1031

82

65

1 6 0 3 6 0 1 2 4

82206

90

9 9 9 9 9

899888

156

95

329

324

252

22

20917

1390

1011110

74

1 9 4 7 6 3 1 1 0

90220

60

9 9 9 9 9

899888

98

120

426

412

450

- 9

21017

1440

2011

3 0

72

1 8 0 5 5 0 1 1 0

80140

40

99999

899888

100

108

251

368

345

6

21717

1391

1011

9 4

72

176598156

100250

60

99999

899888

104

104

439

412

369

10 4

21817

1430

1011

6 0

84

178628130

90190

60

99999

899888

114

94

338

4 0 1

384

22117

1540

1011

90

80

1 7 4 5 9 3 1 7 0

110260

60

99999

899888

96

94

452

372

36 7

2 2 4 1 7

1550

1011

60

58

1 8 0 6 6 0 1 2 0

80180

80

9 9 9 9 9

899888

94

122

323

369

399

- 8 2

1

22517

1420

2011

20

68

184590110

84130

70

9 9 9 9 9

899888

110

116

239

373

365

30115

1610

2031

10

68

1 6 0 4 6 5 1 1 4

90105

80

99999

899888

108

92

167

322

304

5

30715

1510

2011

66

86

164414130

88196

60

9 9 9 9 9

899888

114

78

321

349

279

20 - 7

31315

1630

1011

46

55

180610130

80176

70

9 9 9 9 9

899888

96

125

316

348

375

31715

1610

2011

30

72

158401130

80160

90

99999

899888

106

86

252

322

272

15

31915

1520

2001

38

66

172564122

9 0 1 6 0

82

9 9 9 9 9

899888

102

106

275

346

352

- 1 - 4

32015

1550

2011

90

68

164568120

84210

70

9 9 9 9 9

8 9 9 8 8 8

74

96

344

338

354

32315

1710

2011

24

61

160360120

84144

60

9 9 9 9 9

899888

122

99

230

29 5

252

14

32415

1380

2011

3 0

88

2 0 0 6 3 7 1 5 0

84140

70

99999

899888

150

112

279

384

388

- 1

32615

1630

2031

20

74

170405130

94150

80

99999

899888

116

96

254

317

274

13

32715

1640

2001

48

68

1 4 0 3 5 7 1 2 0

80168

72

9 9 9 9 9

899888

105

72

235

314

251

20

32815

1500

2011

80

68

2 0 0 6 0 0 1 1 0

8 4 1 9 0 1 0 0

9 9 9 9 9

899888

114

132

379

352

370

- 5

3 2 9 1 5

1580

2031

46

74

166530140

90186

80

9 9 9 9 9

899888

124

92

308

330

336

- 1 18

33315

1580

1011

62

66

1 7 2 4 4 8 1 1 0

70172

70

9 9 9 9 9

899888

112

106

295

361

2 9 6

33415

1380

0001

88

65

174477112

68200

70

9 9 9 9 9

899888

999

109

347

384

310

19

33715

1390

2032

66

86

190646140

9 9 9 2 0 6

80

9 9 9 9 9

899888

126

104

391

381

393

- 2

33815

1570

2011

40

58

1 6 8 5 9 6 1 2 0

80160

80

9 9 9 9 9

899888

104

110

268

333

368

- 1 0

33915

1500

1011

54

76

1 8 0 5 0 0 1 5 0

110210

90

9 9 9 9 9

899888

130

104

377

383

321

16

34815

1610

2011

38

84

188508142

100180

80

99999

899888

134

104

338

322

325

35215

1680

2033

40

56

1 7 5 5 7 7 1 4 0

80180

80

99999

899888

100

119

314

303

359

35315

1410

1011

4 4

74

174605

70140

60

9 9 9 9 9

899888

102

100

243

407

372

8

35415

1550

1011

50

76

168497130

80180

90

99999

899888

104

92

302

369

320

13

35515

1500

2011

12

76

180512118

84130

40

9 9 9 9 9

899888

999

104

233

352

32 7

35615

1540

2011

44

82

1 8 0 5 5 0 1 2 6

86170

70

99999

899888

114

98

305

341

345

96

0 - 1 8

7 - 1

3 5 7 1 5

1480

1011

66

63

1 8 0 5 9 0 1 2 4

80190

60

9 9 9 9 9

899888

110

117

341

388

365

5

36015

1 5 5 0

2031

40

70

152438120

76160

70

9 9 9 9 9

899888

106

82

243

338

291

14

36115

1620

2011

74

72

154510146

76250

78

99999

899888

104

82

384

320

326

- 1

36315

1710

2001

30

60

1 6 0 4 5 0 1 6 0

90190

70

99999

899888

114

100

303

295

296

0

3 6 4 1 5

1480

2031

50

85

180637110

80160

90

9 9 9 9 9

899888

112

95

287

357

388

- 8

40111

1422

2031

56

88

166545104

70160

80

12200

141123

122

78

265

373

343

8

40211

1662

2001

56

90

156345150

9 0 1 9 0

90

12200

9 9 9 9 9 9

122

66

296

309

245

20 64

40311

1562

0000

9

90

153

80130

80

22000

212199

153

63

198

336

120

40411

1552

2001

20

82

1 3 0 4 1 0 1 0 0

80128

80

2 2 9 0 0

9 9 9 9 9 9

110

48

166

338

277

18

40511

1412

2001

30

84

160385170

120210

90

12211

145999

135

76

335

376

265

29

9 0 1 0 4

40611

1620

0000

50

82

160240150

90200

80

20201

148333

152

78

319

320

194

39

40711

1462

2001

40

75

130415150

100190

98

12001

154313

130

55

246

362

279

22

40811

1442

1001

0

83

1 3 5 7 1 0 1 5 0

90190

85

02001

2

112

52

256

399

424

- 6

40911

1501

2031

40

55

1 6 0 5 6 5 1 4 0

9 0 1 8 0 : LOO

02201

223131

78

105

287

352

353

0

41011

1733

3031

10

92

140135120

75130

75

02210

221199

135

48

181

290

142

50 33

9123

41111

1482

2031

10

65

145330130

85140

85

22200

124929

110

80

202

357

238

41211

1532

2001

10

56

120275130

80140

80

22210

2 2 9 3 1 3

102

64

167

344

211

38

41311

1513

1031

18

70

1 1 8 3 1 5 1 0 4

80140

85

12301

2 2 0 1 2 3

100

48

165

380

230

39

125

72

346

328

267

18

97

24

239

341

207

39

41411

1590

0000

60

82

154390165

1 0 0 2 2 5 1 2 5

02311

146331

41511

1543

3031

45

76

1 0 0 2 6 8 1 9 5

105240120

12301

2

8129

41611

1412

2031

58

61

164565130

9 0 1 8 8 1 0 5

21201

122313

108

103

308

376

353

6

41711

1562

2031

0

76

1 3 0 1 0 0 1 3 0

100999999

20211

2

130

54

999

336

.125

62

41811

1380

2001

50

70

162710150

90200110

02001

148323

97

92

323

384

424

- 1 0

41911

1403

3031

20

85

1 6 0 2 5 0 1 1 0

70130

22200

1

138

75

207

378

198

4 7

90

392

4199 8313

Density Estimates 42011 42111 42211 42311 42411 42511 42611 42711 42811 42911 43011 43111 43211 43311 43411 43511 43 6 1 1 43711 43811 43911 44011 44111 44211 44311 44411 44511 44611 44711 44811 44911 45011 45111 45211 45311 45411 45511 45611 45711 45811 45911 46011 46111 46211 46311 46411 46511 46611 46711 46811 46911 47011 47111 47211 47311 47411 47511 47611 47711 47811 47911 48011 48111 48211 48311 48411 48511 48611

1421 151 1 1521 13 70 1481 1350 1351 1483 1522 1462 1513 1500 1723 1563 1563 1632 1531 1530 1573 1571 1622 1732 1442 1633 1591 1511 1523 1640 1532 1581 1530 1582 1451 1423 1602 1341 1642 1351 1422 1532 1581 1381 1352 1342 1683 1493 1552 1580 1432 1392 1663 1502 1453 1530 1564 1492 1490 1562 1380 1390 1622 1703 1532 1682 1502 1462 1583

2 0 3 1 74 82 2 0 1 1 6 0 75 0001 0 73 1 0 1 1 6 0 56 2131 10102 0 0 0 0 3 0 75 1 0 3 1 1 1 0 98 3 0 3 1 10 84 2 0 1 1 4 8 74 3011 0 70 0001 12107 0 0 0 0 10 58 3011 35106 3031 35 65 3 0 3 1 2 5 56 2 0 1 1 4 5 77 2011 45104 1 0 0 0 5 8 76 2031 75 65 2 0 3 1 2 0 56 2 0 3 1 3 4 85 2 0 3 1 1 5 2 60 0 0 0 0 14 68 2 0 0 1 2 0 75 1001 20 68 0 0 0 0 44 62 0001 20 72 0 0 0 0 4 0 98 2031 40 70 2 0 1 1 2 4 96 2 0 3 1 4 0 75 2 0 1 1 3 8 52 2 0 3 1 7 0 82 2 0 1 1 4 5 78 2 0 3 1 15 72 0001 50 80 2000 30 58 2 0 3 1 4 0 88 2 0 1 1 35 76 2 0 3 1 6 4 87 2 0 0 0 5 0 97 2031 30 84 2031 14 90 3011 4 0 84 2 0 0 1 2 0 72 2 0 1 1 4 0 72 2031 40 70 2 0 0 0 27 70 2 0 1 1 12 6 3 1011 25 66 3031 0 61 2031 24 66 3 0 0 1 2 7 80 0000 0 78 3 0 3 1 2 5 78 2 0 3 1 1 5 75 0 0 0 0 30 93 2001 0 82 0 0 0 0 26 77 0 0 0 0 1 0 88 2 0 0 0 3 0 56 2 0 0 1 2 0 95 0000 30112 2 0 1 1 25 53 0 0 3 1 20 80 2031 45 70 2 0 3 1 2 8 82

170400110 175510140 173999100 190675130 172305130 130430130 180628140 160360105 166470122 150130150 141135168 125405150 126 3 0 1 7 0 148420125 114140105 162390130 146235105 132390100 120260125 132265190 154270180 104204152 128280116 134180140 110270140 150410116 122 85140 156475154 160421150 162590116 156420160 105135140 150412150 120110105 138213115 200420130 94210110 174445150 128140140 154 540136 152300160 134510110 140235128 152395120 100110130 164385120 120285150 120360118 112150140 118305115 122175200 134222120 175310140 128540118 124280205 154360165 168575140 165290160 120250134 120160130 115 7 5 1 9 0 98210120 152220130 110285135 138208160 136240115 130240100

80184 90 8 0 2 0 0 90 90999999 8 0 1 6 0 80 8 8 1 4 0 70 7 0 1 6 0 90 95250100 70130 84 85170 95 88190 90 98178100 8 0 1 7 0 80 8 5 2 0 5 90 7 8 1 6 0 90 6 0 1 3 0 80 9 0 1 7 5 90 65150 60 7 0 1 5 8 86 80200 80 7 8 2 1 0 85 9 0 2 1 4 96 88186104 72130 70 8 5 1 6 0 80 8 0 1 6 0 80 7 8 1 6 0 70 90160110 8 0 1 9 4 80 9 0 1 9 0 80 7 4 1 4 0 84 1 1 0 2 0 0 94 8 0 1 7 8 85 90220110 6 0 1 5 0 90 70160110 80180100 7 5 1 2 0 80 1 0 0 1 9 0 80 9 0 1 7 5 80 78200 40 8 8 2 1 0 85 8 0 1 4 0 80 90142 92 90160 90 6 0 1 5 0 80 70160 70 9 5 1 9 0 95 8 2 1 4 5 80 8 2 1 5 2 75 8 5 1 4 0 90 100220100 7 5 1 5 0 80 9 0 1 4 0 90 75170 70 110230120 122195152 9 0 1 7 0 90 100200100 8 6 1 6 0 80 90140100 1 0 0 1 6 0 90 8 6 1 4 0 70 8 0 1 6 0 90 7 5 1 6 0 80 96180100 80160 95 5 2 1 2 8 50

393

12200 02000 21210 02000 20210 12210 20200 02001 02001 02001 02300 02990 01201 22200 22300 12200 22010 20200 02210 02001 11200 02390 02000 22200 02000 20210 22200 10200 20200 02000 22201 02100 22001 02000 02211 10201 22000 20201 02100 20210 20211 01210 22200 22000 12300 02000 02301 22310 22000 02300 02211 22200 01200 02000 22201 22211 20200 12000 12010 22000 12100 22211 12301 22300 00211 0 2 2 00 22010

111323 111313 171313 1 5313 1 5333 141323 999999 1 8323 119323 1 6999 172313 156313 112333 139323 114323 120323 999999 157333 118121 120333 142123 999999 999999 999999 153313 161333 166323 151333 999999 999999 123313 122313 999999 130999 210199 999999 146333 123333 124323 122333 138333 999999 119323 124333 143313 117323 126323 242333 123313 129323 122323 131323 116323 143313 1 8323 128311 153333 168333 999999 999999 230121 252113 230199 122323 126333 126999 136323

144 1 16 113 97 150 108 134 144 100 140 141 125 116 98 114 116 134 100 114 104 999 100 118 128 98 88 122 124 115 116 118 103 107 120 136 138 90 120 128 128 140 98 122 115 100 128 120 104 112 114 116 116 170 120 116 114 117 120 105 120 100 98 142 90 128 128 112

88 100 100 134 70 55 82 76 92 80 34 67 20 83 58 85 42 56 55 76 69 44 60 59 42 88 50 58 90 66 81 53 68 42 66 120 36 86 52 67 55 50 50 68 28 92 50 50 49 52 61 68 95 50 46 79 75 83 43 32 59 3 40 57 58 66 48

312 349 999 303 240 207 449 207 282 284 250 212 258 236 148 283 218 208 239 277 329 193 166 214 175 239 195 302 303 226 311 186 329 179 220 359 112 330 223 307 319 187 198 243 149 262 227 173 170 165 268 200 244 217 285 300 285 329 191 167 183 137 243 175 248 217 166

373 349 346 418 357 392 423 357 346 362 349 352 293 336 336 317 344 375 333 333 320 290 368 317 359 349 346 314 344 330 344 330 365 373 325 395 314 392 373 344 330 384 392 395 303 354 338 330 370 412 309 352 365 344 336 354 354 336 384 381 320 298 344 303 352 362 330

27 272 326 6 999 999 407 2 36 225 287 26 384 9 29 252 306 11 140 61 59 142 274 21 91 68 16 282 145 56 267 15 191 44 28 267 38 203 206 38 34 208 176 39 41 213 48 164 41 208 277 20 118 65 309 1 17 282 365 - 1 0 282 17 56 142 23 278 65 130 180 44 28 282 179 42 294 24 61 145 341 0 223 32 326 15 191 51 270 31 130 57 265 25 216 36 23 252 149 59 225 45 162 47 185 47 37 228 341 0 213 36 252 28 358 0 218 34 198 48 154 59 113 64 179 39 184 46 216 28 178 49 194 46 41 194

Kronmal and Tarter 48711 48811 48911 49011 49111 49211 49311 49411 49511 49611 49711 49811 49911 50011 50111 50211 50311 50411 50511 50611 50711 50811 50911 51011 51111 51211 51311 51411 51511 51611 51711

1572 1553 1520 1612 1452 1512 1553 1511 1461 1691 1513 1491 1583 1383 1501 1381 1581 1690 1660 1492 1620 1440 1583 1452 1583 1542 1552 1682 1682 1471 1550

2031 3031 0001 2101 2031 0101 3031 0001 0001 2011 3031 1001 3011 3131 1031 3031 0001 0001 0000 2011 0000 0001 0101 2031 3031 2001 2001 2001 2031 2031 0000

50 62 30 80 40 75 52 68 50 86 20 85 40 70 10 66 20 90 55 60 20 88 70 77 18 76 30 75 90 76 15 98 30 87 20 75 10 70 52 64 4100 20100 40 78 55 77 5 77 20 88 40 88 20 64 45 60 26 75 40 84

95 154558120 92 261 76170 68 00200 110313 40 173 120120115 75145 65 20200 110333 118 50 212 125150130 85170 90 12300 231299 110 173610118 70170 70 00200 212311 125 105 294 56 283 142243150 100200100 02000 127323 126 69 200 154355110 70130 90 22210 2 8111 140 140540140 94180 90 21201 144213 103 70 251 168295150 90170 90 02200 999999 120 102 285 90160 80 22200 999999 118 80 271 170360140 56 220 116240135 70190 70 02000 128999 114 68 249 156240140 100160100 22211 124123 136 96 363 173640140 90210 94 01201 125333 128 64 173 60124 70 12000 140913 115 140240102 160570120 88150 95 222 00 124323 116 85 239 174580140 100230 90 20000 121333 124 98 400 62 239 160 65135 75150 80 22200 129323 160 160360140 88999999 20200 172333 125 73 999 110 80170 35 208 90190110 02110 165333 102 44 170 114240160 90150 90 02000 212999 110 66170 60 02010 999999 98 43 181 107180118 50 164 150195116 90110 96 22000 218999 142 72 309 172215160 95180100 02001 159333 142 64 255 142165140 80180100 22210 229111 124 150390115 80170 80 12200 133313 9 99 73 254 134100165 90170 90 22211 128323 126 57 227 175 60130 90150100 12200 157323 110 87 262 150260126 90166 90 21211 999999 116 62 248 51 172 115120130 80150 90 12000 999999 110 60 221 120315140 85185 70 22210 129999 100 180745104 68130 70 22201 134333 110 105 233 56 237 140115130 70170 70 02000 145323 136

394

333 338 346 322 365 349 338 349 362 301 349 385 330 384 383 384 330 301 309 354 320 368 330 365 330 341 338 303 303 360 338

349 - 4 135 60 149 56 375 - 1 6 46 195 250 28 341 0 221 36 30 252 194 35 44 194 390 -1 41 194 355 7 5 360 108 71 252 23 115 61 194 37 164 53 46 172 181 50 157 52 267 26 125 62 105 68 203 39 135 55 24 230 441 - 2 2 60 132

Density Estimates

REFERENCES A n d e r s o n , G. D. (1969). A c o m p a r i s o n of m e t h o d s f o r e s t i m a t i n g a probability density function. Unpublished doctoral d i s s e r t a t i o n , U n i v e r s i t y of W a s h i n g t o n , S e a t t l e . C e n c o v , N . N . (1962). E v a l u a t i o n of an unknown d i s t r i b u t i o n d e n s i t y f r o m o b s e r v a t i o n s . S o v i e t Math. 3, 1 5 5 9 - 1 5 6 2 . K a s s e r , I. S. and R . A. B r u c e (1969). C o m p a r a t i v e e f f e c t s of a g i n g and c o r o n a r y h e a r t d i s e a s e on s u b m a x i m a l and m a x i m a l e x e r c i s e . C i r c u l a t i o n 39, 7 5 9 - 7 7 4 . K r o n m a l , R. A. , L. B e n d e r and J. M o r t e n s e n (1970). A c o n v e r s a t i o n a l s t a t i s t i c a l s y s t e m f o r m e d i c a l r e c o r d s . J. R o y a l S t a t i s t . S o c . 19, 8 2 - 9 2 . K r o n m a l , R. A. and M. T a r t e r (1968). The e s t i m a t i o n of p r o b a b i l i t y d e n s i t i e s and c u m u l a t i v e s by F o u r i e r s e r i e s m e t h o d s . J. A m e r . S t a t i s t . A s s o c . 6 3 , 9 2 5 - 9 5 2 . T a r t e r , M. E . , R . L . H o l c o m b and R . A. K r o n m a l (1967). A d e s c r i p t i o n of n e w c o m p u t e r m e t h o d s f o r e s t i m a t i n g the p o p u l a t i o n d e n s i t y . In P r o c e e d i n g s , A s s o c i a t i o n f o r C o m p u t i n g M a c h i n e r y 2 2 , 5 1 1 - 5 1 9 . W a s h i n g t o n , D. C . , T h o m p s o n B o o k Company. T a r t e r , M . E . and R . A . K r o n m a l (1970). On m u l t i v a r i a t e d e n s i t y e s t i m a t e s b a s e d o n o r t h o g o n a l e x p a n s i o n s . Ann. M a t h . S t a t i s t . 4 1 , N o . 2, 7 1 8 - 7 2 2 . Z y g m u n d , A. (1959). T r i g o n o m e t r i c S e r i e s , V o l . II. Cambridge P r e s s .

395

London,

Kronmal and Tarter, discussion

DISCUSSION O F T H E K R O N M A L AND T A R T E R C H A P T E R

P.M.

Allen

I a g r e e w i t h t h e o r i g i n a l p r e m i s e t h a t if d a t a a r e g e n e r a t e d b y o b s e r v i n g continuous r a n d o m v a r i a b l e s we ought to s u m m a r i z e t h e d a t a u s i n g a c o n t i n u o u s e s t i m a t e d d e n s i t y f u n c t i o n and a c o n t i n u o u s c u m u l a t i v e d i s t r i b u t i o n f u n c t i o n . H o w e v e r , I t e n d to t h i n k t h a t an a r t i s t w o u l d do a good j o b of d r a w i n g a d i s t r i b u t i o n f u n c t i o n . If t h e a r t i s t d r a w s h i s c u r v e t h r o u g h t h e c e n t e r of the " j u m p s " of t h e e m p i r i c a l d i s t r i b u t i o n f u n c t i o n , t h e c u r v e w i l l be a good c o n t i n u o u s a p p r o x i m a t i o n of t h e e m p i r i c a l d i s t r i b u t i o n function. Density functions p r e s e n t a m o r e difficult p r o b l e m . Dr. Edwin Chen m a d e s o m e h i s t o g r a m s using the d a t a p r e s e n t e d in the K r o n m a l - T a r t e r p a p e r . S o m e of t h e s e h i s t o g r a m s w e r e m u l t i m o d a l while the corresponding polynomial approximations were u n i m o d a l . T h e l e n g t h and l o c a t i o n of t h e i n t e r v a l s h a s l o n g b e e n a p r o b l e m w h e n m a k i n g h i s t o g r a m s . It a p p e a r s t h a t t h e c h o i c e of t h e d e g r e e of t h e a p p r o x i m a t i n g p o l y n o m i a l is a l s o a d i f f i c u l t p r o b l e m . I h a v e b e e n l i s t e n i n g to D r . T u k e y ' s c o m m e n t s at the e n d of e a c h p a p e r . M y i n t e r p r e t a t i o n of e a c h of h i s c r i t i q u e s h a s b e e n " h a n g l o o s e and t r y s o m e t h i n g e l s e . " T h e s o m e t h i n g e l s e I s u g g e s t i n t h i s c a s e i s to p r e s e n t a s u c c e s s i o n of e s t i m a t e d d e n s i t i e s . S t a r t with s o m e t h i n g that is as flexible as p o s s i b l e , then a p p l y s u c c e s s i v e d e g r e e s of s m o o t h i n g . One w o u l d s t i l l h a v e to c h o o s e one of t h e a p p r o x i m a t i o n s and I h e s i t a t e to s u g g e s t a c r i t e r i o n f o r t h i s p u r p o s e . H o w e v e r , I t h i n k one could be g u i d e d b y t h e e x t e n t of t h e d i f f e r e n c e b e t w e e n s u c c e s s i v e d e n s i t i e s . I f e e l t h a t the t e c h n i q u e of p o l y n o m i a l a p p r o x i m a t i o n of d e n s i t i e s h a s p o t e n t i a l and t h a t a d d i t i o n a l e f f o r t in t h i s a r e a i s w o r t h w h i l e . One of t h e f o f m u l a s in t h e p a p e r a p p e a r e d t o b e s i m i l a r to t h e c h a r a c t e r i s t i c f u n c t i o n of t h e e m p i r i c a l d i s t r i b u t i o n f u n c t i o n . I w o u l d b e i n t e r e s t e d to k n o w if t h e r e i s a r e l a t i o n s h i p b e t w e e n them.

396

Density Estimates, discussion

An e m p i r i c a l B a y e s i a n m i g h t f i n d t h e s e t e c h n i q u e s u s e f u l s i n c e a n a n a l y t i c r e p r e s e n t a t i o n of t h e d e n s i t y i s g i v e n w i t h m i n i m a l assumptions.

J.

Tukey

T h e i s s u e s h e r e s e e m to m e to be i s s u e s of how w e a s s e s s b e l i e v a b i l i t y , h o w m u c h w i l l t h e g l o b a l o v e r r i d e t h e l o c a l , and h o w we d e a l w i t h t h e n e g a t i v e d e n s i t i e s w h i c h w i l l i n e v i t a b l y o c c u r f o r s o m e s e t s of d a t a . T h e f i r s t and t h e t h i r d l e a d u s to w a n t to l o o k at \[£ r a t h e r t h a n f. B o t h h a v e t e n d e n c i e s to b r i n g i n c o m p a r i s o n d i s t r i b u t i o n s . T h e s e c o n d l e a d s u s t o look at t h e actual smoothing kernel. I f e e l t h a t t h e s i m p l e s t and m o s t e f f e c t i v e a p p r o a c h to t h e f i r s t issue is that associated with hanging or suspended r o o t o g r a m s . T h e b a s i c i d e a i s s i m p l e . F i t s o m e s o r t of c o m p a r i s o n d i s t r i b u tion. Divide the line or plane up into c e l l s . F o r each c e l l find t h e v a l u e of 2 ( \/observed - \fitted ) o r , if w e f e e l p e r s n i c k e t y , t h e v a l u e of + 4 • observed -

1 + 4 • fitted

and d i s p l a y t h e s e v a l u e s , w h i c h , e x c e p t f o r a s m a l l a l l o w a n c e f o r t h e f i t t i n g of c o n s t a n t s t h a t w e a r e u s u a l l y w i s e t o n e g l e c t , a r e m o d e r a t e l y G a u s s i a n (0, 1). T h e a p p e a r a n c e of t r e n d s t h a t do n o t c o r r e s p o n d to m i s f i t t i n g c a n t h e n b e t a k e n a s s p e a k i n g f o r t h e e x i s t e n c e of c o r r e s p o n d i n g d e v i a t i o n s f r o m f i t . T h e c o m b i n a t i o n of s u c h a p l o t , u s i n g a G a u s s i a n r e f e r e n c e , and f i g u r e 12 w o u l d to m e o f f e r a v e r y m u c h c l e a r e r p i c t u r e of h o w s e r i o u s l y I w a s to t a k e b i m o d a l i t y (and t h e h u m p n e a r 198. 91) t h a n d o e s f i g u r e 12 a l o n e .

397

Kronmal and Tarter, discussion

T h i s h u m p ( n e a r 198. 91) c a n f o c u s our a t t e n t i o n on the p o s s i b i l i t y of the g l o b a l o v e r r i d i n g the l o c a l . While it i s p r e s u m a b l y at l e a s t quite d i f f i c u l t f o r s u c h t h i n g s to h a p p e n , what do w e know a b o u t the p o s s i b i l i t y that j u s t w h e r e the o t h e r d a t a v a l u e s f a l l h a s produced this h u m p ? T h e c o n t r o l l i n g entity i s the f u n c t i o n -2Ttik'x 2TT i k ' y (y, x ) = 2 e e = ( y - x ) k w h i c h r e l a t e s the a p p e a r a n c e of an o b s e r v a t i o n at x to n t i m e s the e s t i m a t e d d e n s i t y at y. With a l i m i t a t i o n on the s u m m a t i o n , s u c h f u n c t i o n s do h a v e r i p p l e s n e a r t h e i r c u t o f f , which c a n o f t e n be r e d u c e d b y r e p l a c i n g the " z e r o - o r - o n e " n a t u r e of the s u m m a t i o n , that i s , b y i n s e r t i n g i n t e r m e d i a t e w e i g h t s n e a r the c u t o f f . T h e f a c t t h a t the f i t t i n g p r o c e s s i s e x p r e s s i b l e a s

f(y) = ~

n

s

thus e l i m i n a t i n g any e x c r u t i a t i n g l y s m a l l v a l u e s f o r l/\fi

.

W h e t h e r the l a t t e r p r o c e s s is helpful enough f o r i = 1 o r beyond, helpful enough to c o u n t e r b a l a n c e its o v e r r i d i n g of v e r y s m a l l N/L w h e r e t h e s e a r e d e s e r v e d , i s u n c l e a r .

400

'

Citation Index

CITATION INDEX A l k e r , H . R . ( 1 9 6 4 ) . Dimensions of conflict in the G e n e r a l Assembly. A m e r . Pol. Sci. Rev. 58, 6 4 2 - 6 5 7 . Allen, D. M. ( 1 9 7 1 ) . The p r e d i c t i o n s u m of s q u a r e s as a c r i t e r i o n for selecting p r e d i c t o r v a r i a b l e s . T e c h n i c a l R e p o r t No. 2 3 , D e p a r t m e n t of S t a t i s t i c s , U n i v e r s i t y of Kentucky. A m o r o c h o , J . and A. B r a n d s t e t t e r ( 1 9 6 7 ) . The r e p r e s e n t a t i o n of s t o r m p r e c i p i t a t i o n fields n e a r ground l e v e l . J . Geophys. R e s . 72, 1 1 4 5 . A n d e r s o n , G . D . ( 1 9 6 9 ) . A c o m p a r i s o n of m e t h o d s for e s t i m a t i n g a probability density function. Unpublished d o c t o r a l d i s s e r t a tion, U n i v e r s i t y of Washington, S e a t t l e . A n d e r s o n , R . L . , D. M. Allen and F . B . Cady (1972). Selection of p r e d i c t o r v a r i a b l e s in l i n e a r multiple r e g r e s s i o n . In S t a t i s t i c a l P a p e r s in Honor of G e o r g e W. S n e d e c o r , T . A . B a n c r o f t , E d . , Iowa State U n i v e r s i t y P r e s s . A n d r e w s , D . F . , P . J . B i c k e l , F . R . H a m p e l , P . J . Huber, W . H . R o g e r s and J . W . Tukey ( 1 9 7 2 ) . Robust E s t i m a t e s of L o c a t i o n : S u r v e y and A d v a n c e s , P r i n c e t o n , New J e r s e y , P r i n c e t o n University P r e s s . A r m i t a g e , P . , C . K . M c P h e r s o n and J . B . Copas ( 1 9 6 9 ) . t i c a l studies of p r o g n o s i s in advanced b r e a s t c a n c e r . J . Chron. P i s . 2 2 , 3 4 3 - 3 6 0 .

Statis-

Atkins, H. , R . D . Bulbrook, M. A. F a l c o n e r , J . L . H a y w a r d , K . S . M a c l e a n and P . H . S c h u r r ( 1 9 6 4 ) . U r i n a r y s t e r o i d e s t i m a t i o n s in the p r e d i c t i o n of r e s p o n s e to a d r e n a l e c t o m y o r h y p o p h y s e c t o m y . L a n c e t , Nov. 2 8 , 1 1 3 3 - 1 1 3 6 .

401

Citation Index

A t k i n s , H. , R . D . B u l b r o o k , M . A. F a l c o n e r , J. L . H a y w a r d , K . S . M a c l e a n and P . H . S c h u r r ( 1 9 6 8 ) . T e n y e a r s ' e x p e r i e n c e of s t e r o i d a s s a y s i n the m a n a g e m e n t of b r e a s t c a n c e r , a r e v i e w . L a n c e t , D e c . 14, 1 2 5 5 - 1 2 6 0 . A t k i n s , H. , M . A. F a l c o n e r , J. L. H a y w a r d , e t al ( 1 9 6 6 ) . The t i m i n g of a d r e n a l e c t o m y and of h y p o p h y s e c t o m y in the t r e a t m e n t of a d v a n c e d b r e a s t c a n c e r . L a n c e t , A p r i l 16, 8 2 7 - 8 3 0 . A t k i n s , H. J. , M . A . F a l c o n e r , J. L. H a y w a r d , K . S . M a c l e a n , P . H . S c h u r r and P . A r m i t a g e ( I 9 6 0 ) . A d r e n a l e c t o m y and h y p o p h y s e c t o m y f o r a d v a n c e d c a n c e r of the b r e a s t . Lancet 1, 1 1 4 8 - 1 1 5 3 . B a h a d u r , R . R . ( I 9 6 0 ) . On the a s y m p t o t i c e f f i c i e n c y of t e s t s and e s t i m a t e s . Sankhya 22, 2 2 9 - 2 5 2 . B a l l , M . M . (1951). B l o c voting in the G e n e r a l A s s e m b l y . I n t e r n a t . O r g a n . 5, 3 - 3 1 . B a r t l e t t , F . and E . R . John ( 1 9 7 0 ) . m e n t s . S c i e n c e 169, 3 0 4 - 3 0 5 .

Reply to Schwartz's c o m -

B i n g h a m , C. , M . D . G o d f r e y and J . W . T u k e y ( 1 9 6 7 ) . M o d e r n t e c h n i q u e s of p o w e r s p e c t r u m e s t i m a t i o n . I. E . E . E. T r a n s a c t i o n s o n Audio and E l e c t r o a c o u s t i c s , A U 1 5 , N o . 2, 5 6 - 6 6 . B r i e r , G. W. and R . A. A l l e n ( 1 9 5 1 ) . V e r i f i c a t i o n of w e a t h e r f o r e c a s t s . In C o m p e n d i u m of M e t e o r o l o g y , T . F . M a l o n e , Ed. , B o s t o n , A m e r . M e t e o r o l . Soc. , 8 4 1 - 8 4 8 . B u l b r o o k , R . D . , F . C. G r e e n w o o d and J. L . H a y w a r d ( I 9 6 0 ) . S e l e c t i o n of b r e a s t c a n c e r p a t i e n t s f o r a d r e n a l e c t o m y o r h y p o p h y s e c t o m y b y d e t e r m i n a t i o n of u r i n a r y 1 7 - h y d r o x y c o r t i c o s t e r o i d and a e t i o c h o l a n o l o n e . L a n c e t 1, 1 1 5 4 . C a c o u l l o s , T. ( 1 9 6 6 ) . E s t i m a t i o n of a m u l t i v a r i a t e d e n s i t y . Ann. I n s t . S t a t i s t . M a t h . T o k y o 18, 1 7 9 - 1 8 9 .

402

Citation Index

C a d y , F . B . and D . M . A l l e n (1972). C o m b i n i n g e x p e r i m e n t s to p r e d i c t f u t u r e y i e l d data. A g r o n . J. 6 4 , 2 1 1 - 2 1 4 . C e n c o v , N . N . (1962). E v a l u a t i o n of an unknown d i s t r i b u t i o n d e n s i t y f r o m o b s e r v a t i o n s . S o v i e t M a t h . 3, 1 5 5 9 - 1 5 6 2 . C o v e r , T . M . (1968). E s t i m a t i o n b y the n e a r e s t - n e i g h b o r r u l e . I. E . E . E . T r a n s . I n f o r . T h e o r y I T - 1 4 , 5 0 - 5 5 . C o x , D. R. (1966). S o m e p r o c e d u r e s c o n n e c t e d w i t h the l o g i s t i c q u a l i t a t i v e r e s p o n s e c u r v e . In R e s e a r c h P a p e r s in S t a t i s t i c s : E s s a y s in Honour of J. N e y m a n ' s 70th B i r t h d a y , F . N . D a v i d , Ed. , London, W i l e y . C u t l e r , S . J . , M. M. B l a c k , G . H . F r i d e l l , et al (1966). Progn o s t i c f a c t o r s in c a n c e r of the f e m a l e b r e a s t . II. R e p r o d u c i b i l i t y of h i s t o p a t h o l o g i c c l a s s i f i c a t i o n . C a n c e r 19, 7 5 - 8 2 . D a n i e l , C. a n d F . S . Wood (1970). New York, Wiley.

F i t t i n g E q u a t i o n s to Data.

D i c k e y , J . M . (1968a). S m o o t h e d e s t i m a t e s f o r m u l t i n o m i a l c e l l p r o b a b i l i t i e s . Ann. Math. S t a t i s t . 39, 5 6 1 - 5 6 6 . D i c k e y , J . M . (1968b). E s t i m a t i o n of d i s e a s e p r o b a b i l i t i e s c o n d i t i o n e d on s y m p t o m v a r i a b l e s . M a t h . B i o s c i . 3, 2 4 9 - 2 6 5 . D i c k e y , J . M . (1969). S m o o t h i n g b y c h e a t i n g . Statist. 40, 1477-1482.

Ann. M a t h .

D i c k e y , J . M . and B . P . L i e n t z (1970). The w e i g h t e d l i k e l i h o o d r a t i o , s h a r p h y p o t h e s e s on c h a n c e s , the o r d e r of a M a r k o v c h a i n . Ann. Math. S t a t i s t . 41, 2 1 4 - 2 2 6 . D i x o n , W . J . (1969). B M P B i o m e d i c a l C o m p u t e r P r o g r a m s , X - S e r i e s S u p p l e m e n t . B e r k e l e y and L o s A n g e l e s , U n i v e r s i t y of C a l i f o r n i a P r e s s .

403

Citation Index

D i x o n , W. J . ( 1 9 7 0 ) . B M P B i o m e d i c a l C o m p u t e r P r o g r a m s . 2nd ed. , 3 r d p r i n t i n g , r e v i s e d . B e r k e l e y and L o s A n g e l e s , U n i v e r s i t y of C a l i f o r n i a P r e s s . Dixon, W . J . ( 1 9 7 1 ) . B M P B i o m e d i c a l C o m p u t e r P r o g r a m s . B e r k e l e y , L o s A n g e l e s , L o n d o n , U n i v e r s i t y of C a l i f o r n i a Press. P o n c h i n , E . ( 1 9 6 9 ) . P a t a a n a l y s i s t e c h n i q u e s in a v e r a g e evoked p o t e n t i a l r e s e a r c h . In A v e r a g e E v o k e d P o t e n t i a l s , E . P o n c h i n and P . B . L i n d s l e y , E d s . , W a s h i n g t o n , P . C . Govt. P r i n t i n g O f f i c e s , N A S A S P - 1 9 1 , 1 9 9 - 2 1 7 . P r a p e r , N. R . and H. Smith ( 1 9 6 6 ) . s i s . New Y o r k , Wiley.

Applied R e g r e s s i o n A n a l y -

F i s h e r , R . A. ( 1 9 3 6 ) . The u s e of m u l t i p l e m e a s u r e m e n t s in t a x o n o m i c p r o b l e m s . Ann. E u g e n . 7 , 1 7 9 - 1 8 8 . F i x , E . and J . L . H o d g e s , J r . ( 1 9 5 1 ) . P i s c r i m i n a t o r y a n a l y s i s , n o n p a r a m e t r i c d i s c r i m i n a t i o n . U S A F S c h o o l of Aviation Med. , Randolph F i e l d , T e x a s , P r o j e c t 2 1 - 4 9 - 0 0 4 , R e p o r t 4 , Contract A F 4 1 ( 1 2 8 ) - 3 1 , F e b r u a r y , 1951. F r e e m a n , W. J . ( 1 9 6 4 ) . prepyriform cortex.

A linear distributed feedback model for Exp. Neurol. 10, 5 2 5 - 5 4 7 .

F r e e n y , A. E . ( M r s . ) ( 1 9 6 9 ) . S t a t i s t i c a l t r e a t m e n t of r a i n gauge calibration data. B e l l Sys. Tech. J . 48, 1757. F r e e n y , A . E . ( M r s . ) and J . P . Gabbe ( 1 9 6 9 ) . A s t a t i s t i c a l d e s c r i p t i o n of i n t e n s e r a i n f a l l . B e l l S y s . T e c h . J . 4 8 , 1 7 8 9 . G i l b e r t , E . S. ( 1 9 6 8 ) . On d i s c r i m i n a t i o n using q u a l i t a t i v e v a r i ables. J. A m e r . Statist. Assoc. 63, 1399-1412. G o l d m a n , R . , J . W a l r a t h , E . J a c o b son and J . P i c k e y ( 1 9 7 1 ) . F i r s t L o o k at G r a p h s . R e s e a r c h R e p o r t 4 1 , r e v i s e d . S t a t i s t i c s P e p a r t m e n t , State Univ. of New Y o r k at B u f f a l o .

404

Citation Index

G o o d , I . J . (1950). P r o b a b i l i t y and t h e W e i g h t i n g of E v i d e n c e . New York, H a f n e r . Good, I . J . M . I . T.

(1965). Press.

T h e E s t i m a t i o n of P r o b a b i l i t i e s .

Cambridge,

G u n e l , E . and J . D i c k e y (1972). B a y e s f a c t o r s f o r i n d e p e n d e n c e i n c o n t i n g e n c y t a b l e s . (In p r e p a r a t i o n ) H a r t i g a n , J . A. (1972). D i r e c t c l u s t e r i n g of a d a t a m a t r i x . J . A m e r . S t a t i s t . A s s o c . 67, 123-129. H a y w a r d , J . L . (1966). A s s e s s m e n t of r e s p o n s e to t r e a t m e n t at G u y ' s H o s p i t a l B r e a s t C l i n i c . In C l i n i c a l E v a l u a t i o n in B r e a s t C a n c e r , New York, A c a d e m i c P r e s s . H i l l s , M . (1967). D i s c r i m i n a t i o n and a l l o c a t i o n w i t h d i s c r e t e d a t a . A p p l . S t a t i s t . 16, 237. H o v e t , T . ( I 9 6 0 ) . B l o c P o l i t i c s in t h e U n i t e d N a t i o n s . Cambridge, Harvard University P r e s s . J a c o b s o n , E . (1971). S H O P : a c o m p u t e r p r o g r a m f o r p r i n t e r p l o t s and h i s t o g r a m s of s u b - c l a s s e s . R e s e a r c h R e p o r t 52, S t a t i s t i c s D e p a r t m e n t , S t a t e U n i v e r s i t y of N e w Y o r k at B u f f a l o . J a c q u e z , J . A . (1964). T h e d i a g n o s t i c p r o c e s s . In C o m p u t e r D i a g n o s i s and D i a g r a m m a t i c M e t h o d s , J . A. J a c q u e z , E d . , Springfield, Illinois, Charles C Thomas. J e f f r e y s , H . (1961). T h e o r y of P r o b a b i l i t y , 3 r d e d . , O x f o r d , Clarendon P r e s s . J e n d e n , D. J . , M . D . F a i r c h i l d , M . R . M i c k e y , R . W. S i l v e r m a n and C. Y a l e (1972). A m u l t i v a r i a t e a p p r o a c h t o the a n a l y s i s of d r u g e f f e c t s on t h e e l e c t r o e n c e p h a l o g r a m . B i o m e t r i c s 28, 73-80.

405

Citation Index

J o i n e r , B . L . , J . R . R o s e n b l a t t a n d J . W . D e a n (1970). OMNITAB - and an e x a m p l e in d a t a a n a l y s i s . (Preliminary d r a f t for d i s c u s s i o n - not for publication), Revised 9 / 2 1 . K a s s e r , I . S. a n d R . A. B r u c e ( 1 9 6 9 ) . C o m p a r a t i v e e f f e c t s of aging and c o r o n a r y h e a r t d i s e a s e on s u b m a x i m a l and m a x i m a l e x e r c i s e . C i r c u l a t i o n 39, 7 5 9 - 7 7 4 . K i a n g , N . Y . S . and T . T . S a n d e l (1961). O f f - r e s p o n s e s f r o m the a u d i t o r y c o r t e x of u n a n e s t h e t i z e d c a t s . A r c h . I t a l . B i o l . 99, 121-134. K r o n m a l , R . A . a n d M . T a r t e r ( 1 9 6 8 ) . T h e e s t i m a t i o n of p r o b a bility d e n s i t i e s and c u m u l a t i v e s by F o u r i e r s e r i e s m e t h o d s . J. A m e r . Statist. A s s o c . 63, 925-952. K r o n m a l , R . A . , L . B e n d e r and J . M o r t e n s e n ( 1 9 7 0 ) . A c o n v e r sational statistical s y s t e m for m e d i c a l r e c o r d s . J. Royal S t a t i s t . S o c . 19, 8 2 - 9 2 . L a i r d , R . J . a n d F . B . C a d y ( 1 9 6 9 ) . C o m b i n e d a n a l y s i s of y i e l d d a t a f r o m f e r t i l i z e r e x p e r i m e n t s . A g r o n . J. 61, 829-834. L e d l e y , R . S . and L . B . L u s t e d (1959). R e a s o n i n g f o u n d a t i o n s of m e d i c a l d i a g n o s i s . S c i e n c e 130 ( 3 3 6 6 ) , 9 - 2 1 . L i j p h a r t , A . ( 1 9 6 3 ) . T h e a n a l y s i s of v o t i n g i n t h e G e n e r a l A s s e m b l y . A m e r . P o l . S c i . R e v . 57, 9 0 2 - 9 1 7 . L u s t e d , L . B . (1968). I n t r o d u c t i o n to M e d i c a l D e c i s i o n M a k i n g . Springfield, Illinois, Charles C Thomas. M a r q u a r d t , D . W. ( 1 9 6 3 ) . A n a l g o r i t h m f o r l e a s t s q u a r e s e s t i m a t i o n of n o n l i n e a r p a r a m e t e r s . J . S o c . I n d u s t . A p p l . M a t h . 2, 431. M e d h u r s t , R . G . ( 1 9 6 5 ) . R a i n f a l l a t t e n u a t i o n of c e n t i m e t e r w a v e s : c o m p a r i s o n of t h e o r y a n d m e a s u r e m e n t . I. E . E . E . T r a n s , on A n t e n n a s a n d P r o p a g a t i o n 5 5 0 .

406

Citation Index

M o r s e , A . P . and F . E . G r u b b s ( 1 9 4 7 ) . T h e e s t i m a t i o n of d i s p e r s i o n f r o m d i f f e r e n c e s . Ann. M a t h . S t a t i s t . 1 8 , 1 9 4 - 2 1 4 . P a r z e n , E . ( 1 9 6 2 ) . On e s t i m a t i o n of a p r o b a b i l i t y d e n s i t y f u n c t i o n and m o d e . Ann. M a t h . S t a t i s t . 3 3 , 1 0 6 5 - 1 0 7 6 . R a d h a k r i s h n a , S . ( 1 9 6 4 ) . D i s c r i m i n a t i o n a n a l y s i s in m e d i c i n e . S t a t i s t i c i a n 14, 1 4 7 - 1 6 7 . R i g g s , R . E . ( 1 9 5 8 ) . P o l i t i c s in the U n i t e d N a t i o n s . U n i v e r s i t y of I l l i n o i s P r e s s .

Champaign,

R u s s e t t , B . M . ( 1 9 6 6 ) . D i s c o v e r i n g v o t i n g g r o u p s in the U n i t e d Nations. A m e r . Pol. Sci. Rev. 66, 327-339. S a n d e l , T . T . and N. Y . S . K i a n g ( 1 9 6 1 ) . auditory c o r t e x of anesthetized c a t s . 105-120.

O f f - r e s p o n s e s f r o m the Arch. Ital. B i o l . 99,

S a n d e r s , F . ( 1 9 5 8 ) . T h e e v a l u a t i o n of s u b j e c t i v e p r o b a b i l i t y f o r e c a s t s . S c i e n t i f i c R e p o r t No. 5, C o n t r a c t A F C R C - T N - 5 8 4 6 5 . C a m b r i d g e , M a s s . I n s t , of T e c h . S a r f a t y , G . and M . T a l l i s ( 1 9 7 0 ) . P r o b a b i l i t y of a w o m a n with a d v a n c e d b r e a s t c a n c e r r e s p o n d i n g to a d r e n a l e c t o m y o r h y p o physectomy. L a n c e t , Oct. 3, 6 8 5 - 6 8 7 . S c h w a r t z , M . ( 1 9 7 0 ) . M e a n s and v a r i a n c e s of a v e r a g e - r e s p o n s e wave f o r m s . S c i e n c e 169, 3 0 3 - 3 0 4 . S e m p l a k , R . A . ( 1 9 6 6 ) . Gauge f o r c o n t i n u o u s l y m e a s u r i n g r a t e of r a i n f a l l . R e v . S c i . I n s t r . 3 7 , 1 5 5 4 . S e m p l a k , R . A . and H. E . K e l l e r ( 1 9 6 9 ) . A d e n s e n e t w o r k f o r r a p i d m e a s u r e m e n t of r a i n f a l l r a t e . B e l l S y s . T e c h . J . 4 8 , 1745.

407

Citation Index

S e m p l a k , R . A . and R . H . T u r r i n (196 9). S o m e m e a s u r e m e n t s of attenuation by r a i n f a l l at 18. 5 GHz. B e l l S y s . T e c h . J . 48, 1767. S p e c h t , D. F . ( 1 9 7 1 ) . S e r i e s e s t i m a t i o n of a p r o b a b i l i t y d e n s i t y f u n c t i o n . T e c h n o m e t r i c s 13, 4 0 9 - 4 2 4 . T a r t e r , M. E . , R . L . H o l c o m b and R . A. K r o n m a l ( 1 9 6 7 ) . A d e s c r i p t i o n of new c o m p u t e r m e t h o d s f o r e s t i m a t i n g the population d e n s i t y . In P r o c e e d i n g s , A s s o c i a t i o n f o r Computing M a c h i n e r y 2 2 , 5 1 1 - 5 1 9 , Washington, D . C . , T h o m p s o n B o o k Company. T a r t e r , M . E . and R . A . K r o n m a l ( 1 9 7 0 ) . On m u l t i v a r i a t e d e n s i t y e s t i m a t e s b a s e d on o r t h o g o n a l e x p a n s i o n s . Ann. M a t h . S t a t i s t . 4 1 , No. 2, 7 1 8 - 7 2 2 . T r u e t t , J . , J . C o r n f i e l d and W. K a n n e l ( 1 9 6 7 ) . A m u l t i v a r i a t e a n a l y s i s of the r i s k of c o r o n a r y h e a r t d i s e a s e in F r a m i n g h a m . J . Chron. P i s . 20, 5 1 1 - 5 2 4 . Tukey, J . W . (1962). Statist. 33, 1 - 6 7 .

The f u t u r e of data a n a l y s i s .

Ann. M a t h .

T u k e y , J . W . ( 1 9 4 9 ) . One d e g r e e of f r e e d o m for n o n - a d d i t i v i t y . B i o m e t r i c s , 5, No. 3. Tukey, J . W . (1970). Exploratory Data Analysis (Limited P r e l i m i n a r y E d i t i o n ) , V o l s . I, II, III. R e a d i n g , M a s s . , Addison-Wesley. Van Ryzin, J . (1965). N o n - p a r a m e t r i c B a y e s i a n decision p r o c e d u r e s f o r (pattern) c l a s s i f i c a t i o n with s t o c h a s t i c l e a r n i n g . T r a n s . F o u r t h P r a g u e Conf. I n f o r m a t i o n T h e o r y , S t a t i s t i c a l D e c i s i o n F u n c t i o n s and R a n d o m P r o c e s s e s . V a n R y z i n , J . ( 1 9 6 6 ) . B a y e s r i s k c o n s i s t e n c y of c l a s s i f i c a t i o n p r o c e d u r e s u s i n g d e n s i t y e s t i m a t i o n . Sankhya A 2 8 , 2 6 1 - 2 7 0 .

408

Citation Index

W a l k e r , S . H . and D . B . D u n c a n ( 1 9 6 7 ) . E s t i m a t i o n of t h e p r o b a b i l i t y of a n e v e n t a s a f u n c t i o n of s e v e r a l i n d e p e n d e n t v a r i a b l e s . B i o m e t r i k a 54, 1 6 7 - 1 7 9 . W a l r a t h , J . , R . G o l d m a n and J . D i c k e y (1971). Nonparametric d i s c r i m i n a t o r s : c o m p u t e r p r o g r a m s f o r r e s e a r c h in c o m puter a s s i s t e d m e d i c a l d i a g n o s i s . R e s e a r c h R e p o r t 48, r e v i s e d . S t a t i s t i c s D e p a r t m e n t , S t a t e U n i v . of N e w Y o r k at Buffalo. W a r n e r , H . R . , A . F . T o r o n t o , L . G. V e a s e y a n d R . S t e p h e n s o n (1961). A m a t h e m a t i c a l a p p r o a c h to m e d i c a l d i a g n o s i s : a p p l i c a t i o n to c o n g e n i t a l h e a r t d i s e a s e . J. A m e r . M e d . A s s o c . 177, 1 7 7 - 1 8 3 . W h i t t l e , P . ( 1 9 5 8 ) . On t h e s m o o t h i n g of p r o b a b i l i t y d e n s i t y f u n c t i o n s . J. R o y . S t a t i s t . Soc. , B 20, 3 3 4 - 3 4 3 . Z i p p i n , C . a n d N . P e t r a k i s ( 1 9 7 1 ) . I d e n t i f i c a t i o n of h i g h r i s k g r o u p s in b r e a s t c a n c e r . C a n c e r 23, No. 6. Z y g m u n d , A. (1959). Cambridge Press.

T r i g o n o m e t r i c S e r i e s , Vol. II,

409

London,