Applied Regression Analysis [3 ed.] 0471170828, 9780471170822

Table of contents :
Cover ......Page 1
Title Page ......Page 5
Copyright ......Page 6
Contents ......Page 7
Preface to the Third Edition ......Page 15
About the Software ......Page 19
Gamma Function ......Page 23
t-distribution ......Page 24
F-distribution ......Page 25
0.2. Confidence Intervals (or Bands) and T-tests ......Page 26
Matrix, Vector, Scalar ......Page 28
Transpose ......Page 29
Multiplication ......Page 30
Special Matrices and Vectors ......Page 31
Obtaining an Inverse ......Page 32
Determinants ......Page 33
Common Factors ......Page 35
1.0. Introduction: the Need for Statistical Analysis ......Page 37
1.1. Straight Line Relationship Between Two Variables ......Page 40
1.2. Linear Regression: Fitting a Straight Line by Least Squares ......Page 42
Meaning of Linear Model ......Page 43
Least Squares Estimation ......Page 44
Pocket-calculator Form ......Page 46
Calculations for the Steam Data ......Page 47
Centering the Data ......Page 49
1.3. The Analysis of Variance ......Page 50
Sums of Squares ......Page 51
Analysis of Variance Table ......Page 52
Steam Data Calculations ......Page 54
R2 Statistic ......Page 55
1.4. Confidence Intervals and Tests for ß0 and ß1 ......Page 56
Standard Deviation of the Slope B1; Confidence Interval for ß1 ......Page 57
Test for Ho: ß1 = ß10 Versus H1: ß1 ≠ ß10......Page 58
Confidence Interval Represents a Set of Tests ......Page 59
1.5. F-test for Significance of Regression ......Page 60
F = T2 ......Page 61
1.6. the Correlation Between X and Y ......Page 62
Rxy and R Connections ......Page 64
Testing a Single Correlation ......Page 65
Pocket-calculator Computations ......Page 66
1.8. Historical Remarks ......Page 67
Exercises ......Page 68
General Discussion of Variance and Bias ......Page 69
Genuine Repeats Are Needed ......Page 70
Calculation of Pure Error and Lack of Fit Mean Squares ......Page 71
Split of the Residual ss ......Page 72
Looking at the Data and Fitted Model ......Page 75
Approximate Repeats ......Page 76
Generic Pure Error Situations Illustrated Via Straight Line Fits ......Page 77
Bartlett's Test ......Page 78
Levene's Test Using Means ......Page 79
Some Cautionary Remarks ......Page 80
2.3. Examining Residuals: the Basic Plots ......Page 81
How Should the Residuals Behave? ......Page 82
Normal Plot of Residuals ......Page 83
2.5. Checks for Time Effects, Nonconstant Variance, Need for Transformation, and Curvature ......Page 84
Three Questions and Answers ......Page 85
2.6. Other Residuals Plots ......Page 89
Dependencies Between Residuals ......Page 90
2.7. Durbin-watson Test ......Page 91
Appendix 2a. Normal Plots ......Page 92
Normal Scores ......Page 94
Outliers ......Page 97
Appendix 2b. Minitab Instructions ......Page 98
Exercises ......Page 99
Covariance of Two Linear Functions ......Page 101
3.1. Standard Error of Y ......Page 102
Intervals for Individual Observations and Means of q Observations......Page 103
3.2. Inverse Regression (straight Line Case) ......Page 105
Experimental Strategy Decisions ......Page 108
An Example ......Page 109
3.4. Straight Line Regression When Both Variables Are Subject to Error1 ......Page 111
Practical Advice ......Page 113
Geometric Mean Functional Relationship ......Page 114
Exercises for Chapters 1–3 ......Page 118
4.1. Fitting a Straight Line in Matrix Terms ......Page 137
Manipulating Matrices ......Page 138
Setup for a Quadratic Model ......Page 139
Transpose ......Page 140
Inverses of Small Matrices ......Page 141
Diagonal Matrices ......Page 142
Inverting Partitioned Matrices with Blocks of Zeros ......Page 143
Back to the Straight Line Case ......Page 144
A Small Sermon on Rounding Errors ......Page 145
Section Summary ......Page 146
4.2. Singularity: What Happens in Regression to Make X'x Singular? an Example ......Page 147
Singularity in the General Linear Regression Context ......Page 148
4.3. The Analysis of Variance in Matrix Terms ......Page 149
4.4. The Variances and Covariance of B0 and B1 from the Matrix Calculation ......Page 150
Correlation Between B0 and B1 ......Page 151
4.6. Summary of Matrix Approach to Fitting a Straight Line (nonsingular Case) ......Page 152
4.7. The General Regression Situation ......Page 153
Exercises for Chapter 4 ......Page 154
5.1. General Linear Regression ......Page 157
A Justification for Using Least Squares ......Page 158
5.2. Least Squares Properties ......Page 159
The R2 Statistic ......Page 160
Adjusted R2 Statistic ......Page 161
5.3. Least Squares Properties When E ~ N(0, 1σ2) ......Page 162
Properties, Continued ......Page 163
5.4. Confidence Intervals Versus Regions ......Page 164
5.5. More on Confidence Intervals Versus Regions ......Page 165
When F-test and T-tests Conflict ......Page 168
Appendix 5a. Selected Useful Matrix Results ......Page 169
Exercises ......Page 170
6.1. The "extra Sum of Squares" Principle ......Page 171
Other Points ......Page 172
Sequential Sums of Squares ......Page 173
Special Problems with Polynomial Models ......Page 174
When T = F1/2 ......Page 175
6.2. Two Predictor Variables: Example ......Page 176
How Useful Is the Fitted Equation? ......Page 178
What Has Been Accomplished by the Addition of a Second Predictor Variable (namely, X6)? ......Page 179
The Standard Error S ......Page 180
Extra Ss F-test Criterion ......Page 181
Standard Error of bi ......Page 182
Confidence Limits for the True Mean Value of Y, Given a Specific Set of Xs ......Page 183
6.3. Sum of Squares of a Set of Linear Functions of Y's ......Page 184
Appendix 6a. Orthogonal Columns in the X Matrix ......Page 187
Appendix 68. Two Predictors: Sequential Sums of Squares ......Page 189
Exercises for Chapters 5 and 6 ......Page 191
7.1. Serial Correlation in Residuals ......Page 201
7.2. The Durbin–watson Test for a Certain Type of Serial Correlation......Page 203
Primary Test, Tables of Dl and Du ......Page 205
A Simplified Test ......Page 207
Width of the Primary Test Inconclusive Region ......Page 212
Mean Square Successive Difference ......Page 213
7.3. Examining Runs in the Time Sequence Plot of Residuals: Runs Test ......Page 214
Runs ......Page 215
Larger n1 and n2 Values ......Page 217
Exercises for Chapter 7 ......Page 220
8.1. The Hat Matrix H and the Various Types of Residuals ......Page 227
Other Facts About H ......Page 228
Extra Sum of Squares Attributable to ej ......Page 229
Externally Studentized Residuals2 ......Page 230
Partial Residuals ......Page 231
8.3. Detection of Influential Observations: Cook's Statistics ......Page 232
Higher-order Cook's Statistics ......Page 234
8.5. Reference Books for Analysis of Residuals ......Page 236
Exercises for Chapter 8 ......Page 237
9.1. Testing a General Linear Hypothesis ......Page 239
Testing a General Linear Hypothesis Cß = 0 ......Page 241
9.2. Generalized Least Squares and Weighted Least Squares ......Page 243
General Comments ......Page 245
9.3. an Example of Weighted Least Squares ......Page 246
9.4 a Numerical Example of Weighted Least Squares ......Page 248
9.6. Inverse Regression (multiple Predictor Case) ......Page 251
Basic Method ......Page 253
Is the Solution a Maximum or Minimum? ......Page 254
Exercises for Chapter 9 ......Page 255
10.1. Bias in Regression Estimates ......Page 257
10.2. The Effect of Bias on the Least Squares Analysis of Variance ......Page 260
10.3. Finding the Expected Values of Mean Squares ......Page 261
10.4. Expected Value of Extra Sum of Squares ......Page 262
Exercises for Chapter 10 ......Page 263
11.1. Is My Regression a Useful One? ......Page 265
An Alternative and Simpler Check ......Page 266
11.2. a Conversation About R2 ......Page 267
References ......Page 268
The γm Criterion......Page 269
Exercises for Chapter 11 ......Page 272
12.1. More Complicated Model Functions ......Page 273
Third-order Models ......Page 274
Models Involving Transformations Other Than Integer Powers ......Page 275
12.2. Worked Examples of Second-order Surface Fitting for K = 3 and K = 2 Predictor Variables ......Page 276
Do We Need X2? ......Page 282
Treatment of Pure Error When Factors Are Dropped ......Page 286
Treatment of Pure Error When a Design Is Blocked ......Page 287
Example 1. Quadratic Equation in X ......Page 288
Example 2. Second-order Polynomial in Two X's ......Page 289
Example 3. Third-order Polynomial in Three Factors ......Page 290
Example 5. Second-order Polynomial in Two X's ......Page 291
Application of Rules 1 and 2 Together ......Page 292
References ......Page 293
Exercises for Chapter 12 ......Page 294
Thinking About the Error Structure ......Page 299
Predictions in Y-space ......Page 300
Points to Keep in Mind ......Page 301
13.2 Power Family of Transformations on the Response: Box-cox Method ......Page 302
Some Conversations on How to Proceed ......Page 303
The Confidence Statement Has Several Forms ......Page 304
13.3. a Second Method for Estimating a ......Page 308
Advantages of the Likelihood Method ......Page 310
13.4. Response Transformations: Other Interesting and Sometimes Useful Plots ......Page 311
A Modulus Family of Response Transformations ......Page 312
13.6. Response Transformations Chosen to Stabilize Variance ......Page 313
Transformations for Responses That Are Proportions ......Page 314
Exercises for Chapter 13 ......Page 316
14.1. Dummy Variables to Separate Blocks of Data with Different Intercepts, Same Model ......Page 321
Other Possibilities ......Page 322
Three Categories, Three Dummies ......Page 323
R Categories, R Dummies ......Page 324
An Alternative Analysis of Variance Sequence ......Page 326
Will My Selected Dummy Setup Work? ......Page 327
Other Verification Methods ......Page 328
Two Sets of Data, Straight Line Models ......Page 329
Hierarchical Models ......Page 330
Three Sets of Data, Straight Line Models ......Page 331
Two Sets of Data: Quadratic Model ......Page 332
14.3. Dummy Variables for Segmented Models ......Page 333
One Segment ......Page 334
Case 1: When It Is Known Which Points Lie on Which Segments ......Page 335
Straight Line and Quadratic Curve ......Page 337
Exercises for Chapter 14 ......Page 339
15.0. Introduction ......Page 349
Comments ......Page 350
15.1. All Possible Regressions and “best Subset” Regression ......Page 351
Use of the Residual Mean Square, S2 ......Page 352
Use of the Mallows Cp Statistic ......Page 354
Example of Use of the Cp Statistic ......Page 355
“best Subset” Regression ......Page 356
15.2. Stepwise Regression ......Page 357
Stepwise Regression on the Hald Data ......Page 358
Minitab Version of Stepwise Regression ......Page 360
15.3. Backward Elimination ......Page 361
Remarks ......Page 363
A Drawback to Understand but Not Be Overly Concerned About ......Page 364
Variations on the Previous Methods ......Page 365
Summary ......Page 366
15.6. Selection Procedures Applied to the Steam Data ......Page 367
Remarks ......Page 368
Appendix 15a. Halo Data, Correlation Matrix, and All 15 Possible Regressions ......Page 370
Exercises for Chapter 15 ......Page 377
A Simple Example ......Page 391
Demonstrating Dependence in X Via Regression ......Page 392
Centering ......Page 393
16.3. Centering and Scaling Regression Data ......Page 395
Centering and Scaling and Singularity ......Page 396
Recommendations on Suggestions 1–6 ......Page 397
16.5. Belsley's Suggestion for Detecting Multicollinearity ......Page 398
Comments ......Page 403
Appendix 16a. Transforming X Matrices to Obtain Orthogonal Columns ......Page 404
Example ......Page 405
Exercises for Chapter 16 ......Page 407
17.2. Basic Form of Ridge Regression ......Page 409
17.3. Ridge Regression of the Hald Data ......Page 411
Automatic Choice of θ*......Page 412
17.4. In What Circumstances Is Ridge Regression Absolutely the Correct Way to Proceed? ......Page 413
Comments ......Page 415
17.5. The Phoney Data Viewpoint ......Page 416
Opinion ......Page 417
Appendix 17b. Mean Square Error Argument ......Page 418
Appendix 17c. Canonical Form of Ridge Regression ......Page 419
Some Alternative Formulas ......Page 421
Exercises for Chapter 17 ......Page 422
Acronym ......Page 423
Some Members of the Exponential Family ......Page 424
Expected Value and Variance of a(u)......Page 425
Example: Binomial Distributions, Indices Ni, Parameters Pi ......Page 426
Deviance ......Page 427
18.4. Performing the Calculations: an Example ......Page 428
Exercise for Chapter 18 ......Page 430
Three Ingredients ......Page 431
Four Ingredients ......Page 432
Five or More Ingredients ......Page 433
Three Ingredients, First-order Model ......Page 434
Example: the Hald Data ......Page 435
Three Ingredients, Second-order Model ......Page 436
Example: the Hald Data ......Page 437
19.3. Mixture Experiments in Restricted Regions ......Page 438
19.4. Example 1 ......Page 440
19.5. Example 2 ......Page 441
Appendix 19a. Transforming Q Mixture Variables to q - 1 Working Variables......Page 444
General q......Page 445
Exercises for Chapter 19 ......Page 447
20.1. The Basic Geometry ......Page 449
Further Split-up of a Regression Sum of Squares ......Page 451
Orthogonalizing the Vectors of X in General ......Page 452
20.3. Analysis of Variance and F-test for Overall Regression ......Page 454
20.4. The Singular X'x Case: an Example ......Page 455
Example ......Page 456
20.5 Orthogonalizing in the General Regression Case ......Page 457
Projection Matrices ......Page 459
20.7. the Algebra and Geometry of Pure Error ......Page 461
Appendix 20a. Generalized Inverses M- ......Page 463
A Method for Getting M- ......Page 464
Example ......Page 465
Exercises for Chapter 20 ......Page 466
21.1. The Geometry of a Null Hypothesis: a Simple Example ......Page 469
21.2. General Case H0: Aß = C: the Projection Algebra ......Page 470
21.3 Geometric Illustrations ......Page 471
21.4. The F-test for H0, Geometrically ......Page 472
21.6. Change in R2 for Models Nested Via Aß = 0, Not Involving ß0 ......Page 474
21.7. Multiple Regression with Two Predictor Variables as a Sequence of Straight Line Regressions ......Page 476
Geometrical Interpretation ......Page 479
Exercises for Chapter 21 ......Page 481
22.2. Orthogonal Polynomials ......Page 483
Orthogonal Polynomials for N = 3, ... , 12 ......Page 487
22.3. Regression Analysis of Summary Data ......Page 489
Exercises for Chapter 22 ......Page 491
Fixed Effects, Variable Effects ......Page 495
23.2. the One-way Classification: Standard Analysis and an Example ......Page 496
23.3. Regression Treatment of the One-way Classification Example ......Page 499
A Caution ......Page 502
23.4. Regression Treatment of the One-way Classification Using the Original Model ......Page 503
23.5. Regression Treatment of the One-way Classification: Independent Normal Equations ......Page 508
23.6. the Two-way Classification with Equal Numbers of Observations in the Cells: an Example ......Page 510
23.7. Regression Treatment of the Two-way Classification Example ......Page 511
23.8. the Two-way Classification with Equal Numbers of Observations in the Cells ......Page 515
23.9. Regression Treatment of the Two-way Classification with Equal Numbers of Observations in the Cells ......Page 516
An Alternative Method ......Page 519
23.10. Example: the Two-way Classification ......Page 520
23.11. Recapitulation and Comments ......Page 521
Exercises for Chapter 23 ......Page 522
Nonlinear Models ......Page 527
Least Squares in the Nonlinear Case ......Page 528
24.2. Estimating the Parameters of a Nonlinear System ......Page 530
A Geometrical Interpretation of Linearization ......Page 533
Steepest Descent ......Page 535
Marquardt's Compromise ......Page 537
Grids and Plots ......Page 538
Getting Initial Estimates .0 ......Page 539
24.3. an Example ......Page 540
A Solution Through the Normal Equations ......Page 541
A Solution Through the Linearization Technique ......Page 544
Further Analysis ......Page 546
Confidence Regions ......Page 548
Some Typical Nonlinear Program Output Features ......Page 549
Curvature Measures ......Page 550
24.4. A Note on Reparameterization of the Model ......Page 551
24.5. The Geometry of Linear Least Squares ......Page 552
The Sample Space ......Page 554
The Sample Space When N = 3, P = 2 ......Page 555
The Sample Space Geometry When the Model Is Wrong ......Page 557
Geometrical Interpretation of Pure Error ......Page 558
The Parameter Space ......Page 559
The Parameter Space When P = 2 ......Page 560
The Sample Space ......Page 561
24.7. Nonlinear Growth Models ......Page 565
An Example of a Mechanistic Growth Model ......Page 566
The Logistic Model ......Page 567
How Do We Get the Initial Parameter Estimates? ......Page 569
The Gompertz Model ......Page 570
Von Bertalanffy's Model ......Page 571
Design of Experiments in the Nonlinear Case ......Page 572
A Useful Model-building Technique ......Page 573
Multiple Responses ......Page 574
Exercises for Chapter 24 ......Page 575
25.1. Least Absolute Deviations Regression (l1 Regression) ......Page 589
25.2. M-estimators ......Page 590
The M-estimation Procedure ......Page 591
25.3. Steel Employment Example ......Page 595
Adjusting the First Observation ......Page 596
25.4. Trees Example ......Page 597
25.6. Robust Regression with Ranked Residuals (rreg) ......Page 599
Other Weights ......Page 601
25.8. Comments and Opinions ......Page 602
Articles ......Page 603
Exercises for Chapter 25 ......Page 606
26.1. Resampling Procedures for Regression Models ......Page 607
26.2. Example: Straight Line Fit ......Page 608
Using the Original Data ......Page 609
26.4 Reference Books ......Page 610
Appendix 26a. Sample Minitab Programs to Bootstrap Residuals for a Specific Example ......Page 611
Appendix 26b. Sample Minitab Programs to Bootstrap Pairs for a Specific Example ......Page 612
Exercises for Chapter 26 ......Page 613
Bibliography ......Page 615
True/false Questions ......Page 627
Answers to Exercises ......Page 631
Normal Distribution ......Page 706
Percentage Points of the T-distribution ......Page 708
Percentage Points of the X2-distribution ......Page 709
Percentage Points of the F-distribution ......Page 710
Index of Authors Associated with Exercises ......Page 717
Index ......Page 719