Advanced Graphics Programming Using OpenGL 9781558606593, 1558606599

Citation preview

Advanced Graphics Programming Using OpenGL TOM McREYNOLDS DAVID BLYTHE

AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO MORGAN KAUFMANN PUBLISHERS IS AN IMPRINT OF ELSEVIER

TEAM LinG

Advanced Graphics Programming Using OpenGL

TEAM LinG

The Morgan Kaufmann Series in Computer Graphics and Geometric Modeling Series Editor: Brian A. Barsky University of California, Berkeley This series publishes the finest works for the accomplished and aspiring graphics professional. The series includes intermediate and advanced textbooks, graphics programming books, surveys of important new areas and methods, and reference works. Advanced Graphics Programming Using OpenGL Tom McReynolds and David Blythe

The Computer Animator’s Technical Handbook Lynn Pocock and Judson Rosebush

Digital Geometry Geometric Methods for Digital Picture Analysis Rienhard Klette and Azriel Rosenfeld

Advanced RenderMan: Creating CGI for Motion Pictures Anthony A. Apodaca and Larry Gritz

Digital Video and HDTV Algorithms and Interfaces Charles Poynton

Curves and Surfaces in Geometric Modeling: Theory and Algorithms Jean Gallier

Real-Time Shader Programming Ron Fosner Complete Maya Programming: An Extensive Guide to MEL and the C++ API David Gould MEL Scripting for Maya Animators Mark R. Wilkins and Chris Kazmier Digital Video and HDTV Algorithms and Interfaces Charles Poynton Texturing & Modeling: A Procedural Approach, Third Edition David S. Ebert, F. Kenton Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley Geometric Tools for Computer Graphics Philip Schneider and David Eberly Understanding Virtual Reality: Interface, Application, and Design William Sherman and Alan Craig Jim Blinn’s Corner: Notation, Notation, Notation Jim Blinn Level of Detail for 3D Graphics: David Luebke, Martin Reddy, Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner Pyramid Algorithms: A Dynamic Programming Approach to Curves and Surfaces for Geometric Modeling Ron Goldman Non-Photorealistic Computer Graphics: Modeling, Rendering, and Animation Thomas Strothotte and Stefan Schlechtweg Curves and Surfaces for CAGD: A Practical Guide, Fifth Edition Gerald Farin Subdivision Methods for Geometric Design: A Constructive Approach Joe Warren and Henrik Weimer Computer Animation: Algorithms and Techniques Rick Parent

Andrew Glassner’s Notebook: Recreational Computer Graphics Andrew S. Glassner Warping and Morphing of Graphical Objects Jonas Gomes, Lucia Darsa, Bruno Costa, and Luiz Velho Jim Blinn’s Corner: Dirty Pixels Jim Blinn Rendering with Radiance: The Art and Science of Lighting Visualization Greg Ward Larson and Rob Shakespeare Introduction to Implicit Surfaces Edited by Jules Bloomenthal Jim Blinn’s Corner: A Trip Down the Graphics Pipeline Jim Blinn Interactive Curves and Surfaces: A Multimedia Tutorial on CAGD Alyn Rockwood and Peter Chambers Wavelets for Computer Graphics: Theory and Applications Eric J. Stollnitz, Tony D. DeRose, and David H. Salesin Principles of Digital Image Synthesis Andrew S. Glassner Radiosity & Global Illumination François X. Sillion and Claude Puech Knotty: A B-Spline Visualization Program Jonathan Yen User Interface Management Systems: Models and Algorithms Dan R. Olsen, Jr. Making Them Move: Mechanics, Control, and Animation of Articulated Figures Edited by Norman I. Badler, Brian A. Barsky, and David Zeltzer Geometric and Solid Modeling: An Introduction Christoph M. Hoffmann An Introduction to Splines for Use in Computer Graphics and Geometric Modeling TEAM LinG Richard H. Bartels, John C. Beatty, and Brian A. Barsky

Publishing Director: Diane Cerra Publishing Services Manager: Simon Crump Project Manager: Brandy Lilly Editorial Coordinator: Mona Buehler Cover Design: Dutton & Sherman Design Text Design: Julio Esperas Composition: Cepha Imaging Pvt. Ltd. Illustrations: Dartmouth Publishing, Inc. Copyeditor: Daril Bentley; Graphic World Proofreader: Graphic World Indexer: Graphic World Interior printer: China Translation & Printing Services, Ltd. Cover printer: China Tranalation & Printing Services, Ltd. Morgan Kaufmann Publishers is an imprint of Elsevier. 500 Sansome Street, Suite 400, San Francisco, CA 94111 This book is printed on acid-free paper. © 2005 by Elsevier Inc. All rights reserved.

Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopying, scanning, or otherwise—without prior written permission of the publisher. Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail: [email protected]. You may also complete your request on-line via the Elsevier homepage (http://elsevier.com) by selecting “Customer Support” and then “Obtaining Permissions.” Library of Congress Cataloging-in-Publication Data Application Submitted ISBN: 1-55860-659-9 For information on all Morgan Kaufmann publications, visit our Web site at www.mkp.com or www.books.elsevier.com

Printed in China 10 9 8 7 6 5 4 3 2 1

TEAM LinG

To my friends and colleagues from Silicon Graphics; it was a fabulous time and place to learn about 3D graphics. – DB To Ede Forney and Harry McGinnis; you were there when it counted. – TM

TEAM LinG

Contents

Preface

xxiii

Acknowledgments Biographies

xxvii

xxviii

PART I

Concepts

1

CHAPTER 1

Geometry Representation and Modeling 1.1

Polygonal Representation

1.2

Decomposition and Tessellation

1.3

Shading Normals 1.3.1 1.3.2

1.4

12 Greedy Tri-stripping

Vertices and Vertex Arrays 1.5.1 1.5.2

1.6

3 4

11

Triangle Stripping 1.4.1

1.5

8 Smooth Shading 9 Vertex Winding Order

3

Vertex Buffer Objects Triangle Lists 16

13 14 15

Modeling vs. Rendering Revisited

17

TEAM LinG vi

Contents

vii

CHAPTER 2

3D Transformations

19

2.1

Data Representation

2.2

Overview of the Transformation Pipeline 2.2.1 2.2.2 2.2.3 2.2.4 2.2.5

19

20 Object Space and the Modelview Transform 20 Eye Space and Projection Transform 21 Clip Space and Perspective Divide 22 NDC Space and the Viewport Transform 22 Window Space 23

2.3

Normal Transformation

2.4

Texture Coordinate Generation and Transformation 2.4.1 2.4.2

23

Texture Matrix 25 Texture Coordinate Generation

25

2.5

Modeling Transforms

2.6

Visualizing Transform Sequences

2.7

Projection Transform

2.8

The Z Coordinate and Perspective Projection 2.8.1

27

Vertex Programs

2.10

Summary

28

30

Z Coordinates and Fog

2.9

25

30

32

32

34

CHAPTER 3

Color, Shading, and Lighting 3.1

35

Representing Color 3.1.1 3.1.2 3.1.3 3.1.4

3.2

Shading

3.3

Lighting 3.3.1 3.3.2

35 Resolution and Dynamic Range Gamma 37 Alpha 39 Color Index 39

36

40 43 Intensities, Colors, and Materials Light Source Properties 47

46

TEAM LinG

viii

Contents

3.3.3 3.3.4

3.4

50

Fixed-Point and Floating-Point Arithmetic 3.4.1

3.5

Material Properties 49 Vertex and Fragment Lighting Biased Arithmetic

Summary

53

54

56

CHAPTER 4

Digital Images and Image Manipulation 4.1

Image Representation

4.2

Digital Filtering

4.3

Convolution

4.4

Images in OpenGL

4.5

Positioning Images

4.6

Pixel Store Operations

4.7

57

60

62 63 65 65

Pixel Transfer Operations 4.7.1 4.7.2

4.8

57

67 Scale and Bias 67 Pixel Mapping Operations

67

ARB Imaging Subset 4.8.1 4.8.2 4.8.3 4.8.4 4.8.5 4.8.6

68 Convolution 68 Color Matrix Transform 68 Histogram 69 MinMax 70 Color Tables 70 Blend Equation and Constant Color Blending

4.9

Off-Screen Processing

4.10

Summary

71

72

72

CHAPTER 5

Texture Mapping 5.1

73

Loading Texture Images 5.1.1 5.1.2

73 Texture Borders 74 Internal Texture Formats

75

TEAM LinG

Contents

5.1.3 5.1.4

5.2

Compressed Textures Proxy Textures 77

76

Texture Coordinates 5.2.1

77 Texture Coordinate Generation and Transformation

5.3

Loading Texture Images from the Frame Buffer

5.4

Environment Mapping 5.4.1 5.4.2 5.4.3 5.4.4

5.5

79

80 Generating Environment Map Texture Coordinates Texture Maps Used in Environment Mapping 82 Cube Mapping 83 Sphere Mapping 85

88 Using 3D Textures to Render Solid Materials

5.6

Filtering

5.7

Additional Control of Texture Level of Detail

5.8

Texture Objects

5.9

Multitexture 5.9.1 5.9.2

81

89

90 91

93

95 Multitexture Model 96 Multitexture Texture Environments

97

Texture Environment 5.10.1 5.10.2

5.11

79

3D Texture 5.5.1

5.10

ix

Summary

98 Advanced Texture Environment Functionality Fragment Programs 100

99

102

CHAPTER 6

Rasterization and Fragment Processing 6.1

Rasterization 6.1.1 6.1.2 6.1.3 6.1.4 6.1.5

6.2

103

104 Rasterization Consistency 105 Z-Fighting 105 Bitmaps and Pixel Rectangles 107 Texture, Color, and Depth Interpolation w Buffering 109

108

Fragment Operations 6.2.1 6.2.2 6.2.3

110 Multisample Operations Alpha Test 111 Stencil Test 111

111

TEAM LinG

x

Contents

6.2.4 6.2.5

6.3

112 114

Framebuffer Operations 6.3.1

6.4

Blending Logic Op

Summary

Accumulation Buffer

115 116

117

CHAPTER 7

Window System and Platform Integration 7.1

Renderer and Window State

7.2

Address Space and Threads

7.3

Anatomy of a Window 7.3.1 7.3.2

7.4

7.4.1 7.4.2

120 121

122 Overlay and Underlay Windows Multiple Displays 123

Off-Screen Rendering GLX Pbuffers WGL Pbuffers

119

122

124 125 126

7.5

Rendering to Texture Maps

7.6

Direct and Indirect Rendering

126 127

CHAPTER 8

OpenGL Implementations

129

8.1

OpenGL Versions

8.2

OpenGL Extensions

8.3

OpenGL ES for Embedded Systems 8.3.1 8.3.2 8.3.3 8.3.4

129 131

131 Embedded Profiles 132 Common and Common-Lite Profiles Safety Critical Profile 136 OpenGL ES Revisions 136

8.4

OpenGL Pipeline Evolution

8.5

Hardware Implementations of the Pipeline 8.5.1 8.5.2 8.5.3 8.5.4

133

137

Rasterization Acceleration 138 Primitive Setup Acceleration 141 Transform and Lighting Acceleration Pipeline Balance 142

138

141

TEAM LinG

Contents

8.5.5 8.5.6 8.5.7

8.6

Parallelism Opportunities 142 Reordering the Pipeline 149 Mixed Software and Hardware Implementations

The Future

xi

150

151

P A R T II

Basic Techniques

153

CHAPTER 9

Multiple Rendering Passes 9.1

Invariance

9.2

Multipass Overview

9.3

The Multipass Toolbox 9.3.1 9.3.2 9.3.3 9.3.4 9.3.5

9.4 9.5

155 156

159 Arithmetic Operations 159 Arbitrary Functions 160 Conditionals 161 Variables 162 Parameters 163

Multipass Limitations

165

Multipass vs. Micropass 9.5.1

155

Multitexture

9.6

Deferred Shading

9.7

Summary

165

166 167

167

CHAPTER 10

Antialiasing

169

10.1

Full-Scene Antialiasing

10.2

Supersampling 10.2.1 10.2.2 10.2.3 10.2.4

170

171 Supersampling by Overdrawing 172 Supersampling with the Accumulation Buffer Multisample Antialiasing 175 Drawbacks 176

173

TEAM LinG

xii

Contents

10.3

Area Sampling

10.4

Line and Point Antialiasing

10.5

Antialiasing with Textures

10.6

Polygon Antialiasing

181

Temporal Antialiasing

182 183

10.7

10.7.1

10.8

177

Motion Blur

Summary

178 180

184

CHAPTER 11

Compositing, Blending, and Transparency 11.1

Combining Two Images 11.1.1 11.1.2 11.1.3

185 Compositing 186 Compositing Multiple Images Alpha Division 190

11.2

Other Compositing Operators

11.3

Keying and Matting

11.4

190

192

192 Arithmetic Errors 192 Blending with the Accumulation Buffer Approximation Errors 193 Gamma Correction Errors 193

11.5

Compositing Images with Depth

11.6

Other Blending Operations

11.7

Dissolves

11.8

Transparency

194

195

196 199

200 Dynamic Object Transparency Transparency Mapping 203 Transparency Sorting 204 Depth Peeling 205

202

Screen-Door Transparency 11.10.1

11.11

193

Alpha-Blended Transparency 11.9.1 11.9.2 11.9.3 11.9.4

11.10

187

Blending Artifacts 11.4.1 11.4.2 11.4.3 11.4.4

11.9

185

Summary

205 Multisample Transparency 208

207

TEAM LinG

Contents

xiii

CHAPTER 12

Image Processing Techniques 12.1

OpenGL Imaging Support

12.2

Image Storage

12.3

Point Operations 12.3.1 12.3.2 12.3.3 12.3.4 12.3.5 12.3.6 12.3.7 12.3.8

12.4

12.6

212

213 Color Adjustment 213 Interpolation and Extrapolation Scale and Bias 215 Thresholding 215 Conversion to Luminance 216 Manipulating Saturation 216 Rotating Hue 218 Color Space Conversion 219

225

Convolution

227 Separable Filters 227 Convolutions Using the Accumulation Buffer Convolution Using Extensions 230 Useful Convolution Filters 230 Correlation and Feature Detection 233

228

Geometric Operations 12.7.1 12.7.2 12.7.3 12.7.4

235 Pixel Zoom 235 Scaling Using Texture Mapping 236 Rotation Using Texture Mapping 237 Distortion Correction 237

12.8

Image-Based Depth of Field

12.9

High Dynamic Range Imaging 12.9.1 12.9.2 12.9.3

12.10

213

223 Contrast Stretching 224 Histogram Equalization 224

Reduction Operations 12.6.1 12.6.2 12.6.3 12.6.4 12.6.5

12.7

211

Region-based Operations 12.4.1 12.4.2

12.5

211

Summary

238 241

Dynamic Range 241 Tone Mapping 242 Modeling Adaptation 245 245

TEAM LinG

xiv

Contents

CHAPTER 13

Basic Transform Techniques

247

13.1

Computing Inverse Transforms Efficiently

13.2

Stereo Viewing

13.3

Depth of Field

13.4

Image Tiling

13.5

Billboarding Geometry

13.6

Texture Coordinate vs. Geometric Transformations 13.6.1 13.6.2 13.6.3

13.7

249 252 254 257

261 Direct Vertex to Texture Coordinate Mapping 263 Overlaying an Entire Scene with a Texture 263 Overlaying a Scene with an Independent Texture Projection 264

Interpolating Vertex Components through a Perspective Transformation 265 13.7.1 13.7.2 13.7.3

13.8

247

Transforming Vertices in the Application Interpolating Vertex Components 266 Computing LOD 267

Summary

265

268

CHAPTER 14

Texture Mapping Techniques

269

14.1

Loading Texture Images into a Framebuffer

270

14.2

Optimizing Texture Coordinate Assignment

270

14.3

3D Textures

14.4

Texture Mosaics

14.5

Texture Tiling

14.6

Texture Paging 14.6.1 14.6.2 14.6.3

14.7

271 274 277

279 Texture Subimage Loading 282 Paging Images in System Memory 285 Hardware Support for Texture Paging 286

Prefiltered Textures 14.7.1

287 Computing Texel Aspect Ratios

288

TEAM LinG

Contents

14.8

xv

Dual-Paraboloid Environment Mapping 14.8.1 14.8.2 14.8.3

291 The Mathematics of Dual-Paraboloid Maps Using Dual-Paraboloid Maps 294 OpenGL Dual-Paraboloid Support 296

14.9

Texture Projection

14.10

Texture Color Coding and Contouring

14.11

2D Image Warping

300

14.12

Texture Animation

302

14.13

Detail Textures 14.13.1 14.13.2

291

296

306 Signed Intensity Detail Textures Creating Detail Textures 311

14.14

Texture Sharpening

312

14.15

Mipmap Generation

313

14.16

Texture Map Limits

315

14.17

Summary

298

309

316

CHAPTER 15

Lighting Techniques 15.1

317

Limitations in Vertex Lighting 15.1.1 15.1.2

317 Static and Adaptive Tessellation 319 Local Light and Spotlight Attenuation 320

15.2

Fragment Lighting Using Texture Mapping

15.3

Spotlight Effects Using Projective Textures

322

15.4

Specular Lighting Using Environment Maps

325

15.4.1

15.5

Multitexture

321

326

Light Maps 15.5.1 15.5.2

327 2D Texture Light Maps 3D Texture Light Maps

15.6

BRDF-based Lighting

15.7

Reflectance Maps 15.7.1 15.7.2

327 330

332

332 Gloss Maps 332 Emission Maps 334

TEAM LinG

xvi

Contents

15.8

Per-fragment Lighting Computations

15.9

Other Lighting Models 15.9.1 15.9.2 15.9.3 15.9.4 15.9.5

15.10

15.13

345

352 353

High Dynamic Range Lighting Bloom and Glare Effects

354 354

Global Illumination 15.14.1 15.14.2 15.14.3

15.15

352 Vector Normalization

Bump-mapped Reflections 15.13.1

15.14

343 Approximating Bump Mapping Using Texture Tangent Space 346 Forward Differencing 347 Limitations 351

Normal Maps 15.11.1

15.12

335 Fresnel Reflection 335 Gaussian Reflection 336 Anisotropic Lighting 337 Oren-Nayar Model 340 Cook-Torrance Model 342

Bump Mapping with Textures 15.10.1 15.10.2 15.10.3 15.10.4

15.11

334

Summary

355 Virtual Light Technique 355 Combining OpenGL Lighting with Radiosity Ambient Occlusion 357

356

359

P A R T III

Advanced Techniques

361

CHAPTER 16

CAD and Modeling Techniques 16.1

363

Picking and Highlighting 16.1.1 16.1.2 16.1.3 16.1.4 16.1.5 16.1.6

363 OpenGL Selection 364 Object Tagging in the Color Buffer 365 Proxy Geometry 366 Mapping from Window to Object Coordinates Other Picking Methods 367 Highlighting 367

367

TEAM LinG

Contents

16.1.7 16.1.8

XOR Highlighting 368 Foreground Object Manipulation

16.2

Culling Techniques

16.3

Occlusion Culling 16.3.1 16.3.2 16.3.3 16.3.4 16.3.5

16.4

369

369

370 Choosing Occluders 371 Building the Occlusion Map 371 Building the Depth Estimation Buffer 372 Occlusion Testing 372 Other Occlusion Testing Methods 373

Geometric Level of Detail 16.4.1 16.4.2

373 Changing Detail 374 Transition Techniques 375

16.5

Visualizing Surface Orientation

16.6

Visualizing Surface Curvature

16.7

xvii

377 379

Line Rendering Techniques 16.7.1 16.7.2 16.7.3 16.7.4 16.7.5 16.7.6 16.7.7 16.7.8

380 Wireframe Models 381 Hidden Lines 382 Polygon Offset 384 Depth Range 384 Haloed Lines 385 Silhouette Edges 386 Preventing Antialiasing Artifacts End Caps on Wide Lines 390

16.8

Coplanar Polygons and Decaling

16.9

Capping Clipped Solids

16.10

Constructive Solid Geometry

389

390

392 393

CHAPTER 17

Scene Realism 17.1

403

Reflections 17.1.1 17.1.2 17.1.3 17.1.4 17.1.5

404 Object vs. Image Techniques Planar Reflectors 407 Curved Reflectors 411 Interreflections 419 Imperfect Reflectors 422

404

TEAM LinG

xviii

17.2

Contents

Refraction 17.2.1 17.2.2 17.2.3 17.2.4 17.2.5 17.2.6

17.3

Shadows 17.4.1 17.4.2 17.4.3 17.4.4

17.5

430

Creating Environment Maps 17.3.1 17.3.2 17.3.3 17.3.4 17.3.5 17.3.6

17.4

424 Refraction Equation 424 Planar Refraction 426 Texture Mapped Refraction 428 Environment Mapped Refraction 429 Modeling Multiple Refraction Boundaries Clipping Refracted Objects 431

432 Creating Environment Maps with Ray Casting 433 Creating Environment Maps with Texture Warping 434 Cube Map Textures 437 Sphere Map Textures 440 Dual-paraboloid Maps 443 Updating Environment Maps Dynamically 448 449 Projective Shadows 450 Shadow Volumes 452 Shadow Maps 459 Creating Soft Shadows 463

Summary

465

CHAPTER 18

Natural Detail 18.1

467

Particle Systems 18.1.1 18.1.2 18.1.3 18.1.4 18.1.5

467 Representing Particles 469 Number of Particles 473 Modeling Particle Interactions 473 Updating and Rendering Particles 475 Applications 478

18.2

Dynamic Meshes

18.3

Procedural Texture Generation 18.3.1 18.3.2 18.3.3 18.3.4 18.3.5 18.3.6

484

487 Filtered Noise Functions 487 Generating Noise Functions 489 Filtering Using Texture Convolution Optimizing the Convolution Process Spectral Synthesis 495 Turbulence 496

490 492

TEAM LinG

Contents

18.3.7 18.3.8

18.4

xix

Random Image Warping 498 Generating 3D Noise 498

Summary

500

CHAPTER 19

Illustration and Artistic Techniques 19.1

Projections for Illustration 19.1.1 19.1.2

19.2

501 Axonometric Projection 502 Oblique Projection 503

Nonphotorealistic Lighting Models 19.2.1 19.2.2

Edge Lines

19.4

Cutaway Views 19.4.1

507

508 Surface Texture

19.5

Depth Cuing

19.6

Patterns and Hatching 19.6.1 19.6.2

512 Cross Hatching and 3D Halftones Halftoning 515

513

516 Accuracy in 2D Drawing Line Joins 517 2D Trim Curves 518

516

520 Image-based Text 520 Geometry-based Text 523

Drawing and Painting 19.9.1 19.9.2 19.9.3

19.10

511

Text Rendering 19.8.1 19.8.2

19.9

510

2D Drawing Techniques 19.7.1 19.7.2 19.7.3

19.8

505

Matte Surfaces 505 Metallic Surfaces 506

19.3

19.7

501

Summary

525 Undo and Resolution Independence Painting in 3D 527 Painting on Images 529

527

530

CHAPTER 20

Scientific Visualization 20.1

531

Mapping Numbers to Pictures

531

TEAM LinG

xx

Contents

20.2

Visual Cues and Perception

20.3

Data Characterization

20.4

560 Icons 561 Particle Tracing 561 Stream Lines 563 Illuminated Stream Lines 563 Line Integral Convolution 564

Tensor Field Visualization 20.7.1

20.8

538 Line Graphs 538 Contour Lines 539 Annotating Metrics 539 Image Display 540 Surface Display 543 Isosurfaces 545 Volume Slicing 546 Volume Rendering 547 Texture Slicing 549 Splatting 556 Creating Volume Data 559

Vector Field Visualization 20.6.1 20.6.2 20.6.3 20.6.4 20.6.5

20.7

534 Scatter Plots 534 Iconographic Display 535 Andrews Plots 536 Histograms and Charts 537

Scalar Field Visualization 20.5.1 20.5.2 20.5.3 20.5.4 20.5.5 20.5.6 20.5.7 20.5.8 20.5.9 20.5.10 20.5.11

20.6

532

Point Data Visualization 20.4.1 20.4.2 20.4.3 20.4.4

20.5

531

Summary

Hyperstreamlines

568 569

570

CHAPTER 21

Structuring Applications for Performance 21.1

Structuring Graphics Processing 21.1.1 21.1.2 21.1.3

21.2

571

Scene Graphs 572 Vertex Updates 575 Texture Updates 576

Managing Frame Time 21.2.1

571

Input Phase

577 579

TEAM LinG

Contents

21.2.2 21.2.3 21.2.4

21.3

Rendering Phase 579 Computation Phase 580 The Safety Margin 581

Application Performance Tuning

581 Locating Bottlenecks 581 Finding Application Bottlenecks 583 Measuring Performance 587 Measuring Depth Complexity 589 Pipeline Interleaving 591

21.3.1 21.3.2 21.3.3 21.3.4 21.3.5

21.4

xxi

Summary

592

APPENDIX A

Using OpenGL Extensions

593

A.1

How OpenGL Extensions are Documented

A.2

Finding OpenGL Extension Specifications

A.3

How to Read an OpenGL Extension Specification A.3.1

ARB Extensions

593 594 594

598

A.4

Portable Use of OpenGL Extensions

599

A.5

Using Extension Function Pointers

602

APPENDIX B

Equations B.1

3D Vectors B.1.1 B.1.2 B.1.3

B.2

605 605 Spherical Coordinates 606 Linear Interpolation of 3D Vectors Barycentric Coordinates 607

606

Projection Matrices B.2.1 B.2.2 B.2.3 B.2.4

607 Orthographic Projection 607 Perspective Projection 607 Perspective z-Coordinate Transformations Alternative Perspective Projection 608

B.3

Viewing Transforms

B.4

Modeling Transforms B.4.1

Scaling

608

608 609

609

TEAM LinG

xxii

Contents

B.4.2 B.4.3

Translation 609 Rotation 609

B.5

Parallel and Perpendicular Vectors

B.6

Reflection Vector

B.7

Lighting Equations

B.8

Function Approximations B.8.1 B.8.2 B.8.3

610

610 610

612 Taylor Series Expansion 612 Newton-Raphson Method 612 Hypotenuse 613

Bibliography Subject Index

615 629

TEAM LinG

Preface

Overview Computer graphics has come a long way from the early days of line drawings and light pens. Today anyone can run interactive and realistic graphics applications on the hardware available on an affordable personal computer. While hardware progress has been impressive, widespread gains in software expertise has been more elusive. There are many computer graphics professionals and enthusiasts out there, but a comprehensive understanding of the accelerated graphics pipeline and how to exploit it is less widespread. This book attempts to bring the computer graphics enthusiast, whether professional or amateur, beyond the basics covered in computer graphics texts, and introduce them to a mix of more intense practical and theoretical discussion that is hard to obtain outside of a professional computer graphics environment. We emphasize the algorithmic side of computer graphics, with a practical application focus. We try to strike a balance between useful examples and approachable theory. We present usable techniques for real world problems, but support them with enough theory and background so the reader can extend and modify the ideas presented here. This book is about graphics techniques, techniques that don’t require esoteric hardware or custom graphics libraries, that are written in a comprehensible style, and do useful things. This book will teach you some graphics, especially areas that are sometimes underrepresented in graphics texts. But it also goes further, showing you how to apply those techniques in real world applications, filling real world needs. Since there are already a number of excellent books that provide an introduction to computer graphics (Foley, 1994; Watt, 1989; Rogers, 1997; Angel, 1997; Newman, 1973) and to OpenGL programming (Neider, 1997; Angel, 1997) we have been necessarily brief in these areas. We assume that the reader is comfortable with these fundamentals; however, we have included extra introductory material where we thought it would improve understanding of later sections. We also note that the computer graphics field has a lot of competing notation and vocabulary. We have tried to be consistent with terminology and notation used in the OpenGL specification and the “standard” OpenGL books while at the same time providing some mention of alternative terminology when it is relevent.

TEAM LinG xxiii

xxiv

Preface

OpenGL We chose OpenGL as our base graphics language for a number of reasons. It is designed to be full featured, to run efficiently on a wide range of graphics architectures, and is clean and straightforward to use. It also dictates very little policy. It is easy to mix and match graphics features in OpenGL to get new effects that may not have even been considered when the language was designed. Its clear specification gives the application programmer confidence that applications written in OpenGL will act predictably on many different graphics hardware and driver implementations. OpenGL is also widely available. It can be obtained for free on all the important architectures today: Apple Machintosh, all flavors of Microsoft Windows, nearly all Unix variants including Linux, and OS/2. Most commercial system and graphics hardware vendors support OpenGL as well, and support for hardware accelerated implementations has grown rapidly, especially in the personal computer space. OpenGL runs on a wide range of graphics hardware; from “big iron” compute clusters, to OpenGL ES, which is designed to provide 3D graphics on embedded devices as small as a cell phone. Given the broad applicability, scalability, and wide availability, OpenGL is an easy choice as the basis for describing graphics algorithms. However, even if you don’t use OpenGL, the graphics APIs in common use are conceptually similar enough that you will still find this book useful. OpenGL remains an evolving specification. Throughout the book we make references to various revisions of the specification (versions 1.0–1.5) and discuss both OpenGL architecture review board (ARB) extensions and various vendor-specific extensions when we believe they enhance the discussion of a particular subject. Rather than focus on the feature set of the most advanced versions of OpenGL, we have included a broad range of algorithms with varying requirements. For many techniques we describe algorithm variations that cover a range of earlier and more advanced versions of OpenGL. We have followed this path since a wide range of OpenGL versions are deployed across various environments including the burgeoning embedded space.

Book Organization This book is divided into three parts. We start with a conceptual overview of computer graphics, emphasizing areas important to the techniques in this book, with extra attention in some overlooked areas. Hand in hand with our introduction to computer graphics, we’ll describe the OpenGL pipeline, with extra detail on the parts of the pipeline most techniques rely on heavily: lighting, texture mapping, rasterization, and depth buffering. We also use this opportunity to describe OpenGL system deployment, including the platform embedding layer and an overview of common hardware acceleration techniques for the pipeline.

TEAM LinG

Preface

xxv

With this foundation in place, Part II introduces a set of important basic techniques. Useful by themselves, they help re-enforce the insights gleaned from the overview. These sequences are also used as building blocks for more complex and sophisticated techniques. To help tie them more tightly to the graphics concepts described in the previous part, these techniques are presented and organized with respect to the OpenGL architecture. The third and final part covers more sophisticated and complex techniques. These techniques are categorized into application areas to help organize the material. The start of each application section has an introduction to the problems and issues important for that area, making these sections more interesting and useful to readers not well versed in that particular field. The book is heavily cross-referenced. Complex techniques reference the simple ones they build upon, and both levels of technique reference the conceptual overview. This allows the reader to use the book as a self-guided tutorial, learning more about techniques and concepts of interest in greater depth.

Example Code To avoid cluttering the text with large fragments of example code, code fragments are used sparingly. Algorithms are usually described as a sequence of steps. However, since details are often the difference between a working program and a black screen, we have tried to include full blown example programs for most algorithms. This example code is available for internet download from www.mkp.com/opengl.

Conventions We use conventions and terminology similar to that found in the OpenGL specification and in the “red-blue-green-white” series of OpenGL books. In addition, we use the following conventions: •

Equations involving matrices, vectors, and points use single uppercase letters for most variables. Vectors are emboldened (V), whereas points and matrices are not (M, P). In rare occasions vectors or points are in lower case.

•

Occasionally symbols are context specific, but generally the following meanings hold: –

N - normal vector

–

L - light vector

TEAM LinG

Preface

xxvi

–

R - reflection vector

–

T - tangent vector

–

B - binormal vector

–

s, t, r, q - texture coordinates

–

x, y, z, w - vertex coordinates

–

θ , ϕ - spherical coordinate angles

–

RGBA - red, green, blue, and alpha components

–

I - intensity

–

C - color (usually RGB or RGBA)

–


V
- length of vector V

–

[n, m] a number between n and m including the end points

–

A · B - inner product of vectors A and B

–

A ⊙ B - max{0, A · B} – the clamped inner product

–

A × B - cross product of vectors A and B

TEAM LinG

Acknowledgments This book reflects a significant part of our collective experience in working with OpenGL applications for the past 13 years. While the book couldn’t possibly cover everything known about using OpenGL, we are pleased to provide this useful subset. Of course, we didn’t figure it out all on our own; we are indebted to the many people that helped us one way or the other: either by providing ideas, correcting misconceptions, prototyping early algorithms, teasing us about taking so long to complete the book, or providing the occasional encouraging word. Many people contributed to this effort: any omissions from the list below are inadvertent. The following people contributed directly to the original 1996–2000 SIGGRAPH course notes that were the genesis for this book: Celeste Fowler, Brad Grantham, Mark Kilgard, Scott Nelson, Simon Hui, and Paula Womack. We had invaluable production help with the course notes over the years from Dany Galgani (illustrations), Linda Rae Sande (production editing), Bob Brown, and Chris Everett. Bowen ‘Cheetah’ Goletz helped with the logistics of sharing the source material over the internet. We are also indebted to those who have made tools such as TeX/LaTeX, GhostScript/Ghostview, and cvs freely available on the three different computing platforms that we used for preparing the original course notes. The original notes benefited from the patient attention of an army of reviewers. They include Dave Shreiner, Paul Strauss, David Yu, Hansong Zhang, Sharon Clay, Robert Grzeszczuk, Phil Lacroute, Mark Peercy, Lena Petrovic, Allan Schaffer, Mark Stadler, John Airey, Allen Akin, Brian Cabral, Tom Davis, Bob Drebin, Ben Garlick, Michael Gold, Paul Haeberli, Michael Jones, Phil Keslin, Erik Lindholm, Mark Young, and Mark Segal. This book would not exist without the wealth of experience, cool ideas, tricks, hacks, and wisdom generously provided to us. It is hard to acknowledge everyone properly. Here is our attempt to do so: Kurt Akeley, Brian Cabral, Amy Gooch, Wolfgang Heidrich, Detlev Stalling, Hansong Zhang, Luis Barcena, Angus Dorbie, Bob Drebin, Mark Peercy, Nacho Sanz-Pastor Revorio, Chris Tanner, David Yu, John Airey, Remi Arnaud, Greg Ward, Phil Lacroute, and Peter-Pike Sloan. We would also like to acknowledge Atul Narkhede, Rob Wheeler, Nate Robbins, and Chris McCue for coding prototype algorithms. A number of people helped refine the raw material from the course notes into this manuscript and have earned our gratitude: Ben Luna, Jeff Somers, Brandy Lilly, and Jessica Meehan in particular. We are also greatly indebted to our reviewers: Ian Ashdown, Dave Shreiner, Noah Gibbs, Brian Paul, and Clay Budin.

TEAM LinG xxvii

Biographies

David Blythe David Blythe has worked in the 3D graphics field professionally for the last 14 years, including serving as Chief Engineer at Silicon Graphics, a representative on the OpenGL Architecture Review Board, editor for the OpenGL ES 1.0 specification, and a frequent SIGGRAPH course presenter. While at Silicon Graphics, David contributed to the development of the RealityEngine and InfiniteReality graphics systems. He has worked extensively on implementations of the OpenGL graphics library, OpenGL extension specifications, and high-level toolkits built on top of OpenGL. David’s other industry experience includes embedded and system-on-a-chip design, mobile devices, and wireless networking. David is currently a graphics architect in the Windows Graphics and Gaming Technologies division at Microsoft working on DirectX and OpenGL graphics technologies. Tom McReynolds Tom McReynolds has worked on 3D graphics at Sun Microsystems, Silicon Graphics, Gigapixel, 3Dfx, and NVIDIA. He has worked in software organizations, writing graphics libraries and drivers, and on the hardware side, writing simulators and verification software for 3D hardware. He presented 3D graphics courses at a number of SIGGRAPH conferences, as well as at a number of Silicon Graphics Developer conferences, an X technical conference, and at Linux World. Tom is currently managing a development team that writes 3D graphics drivers for embedded GPUs at NVIDIA, and contributing to the evolution of OpenGL ES by participating in the Khronos working group.

TEAM LinG xxviii

I PART

■ ■ ■ ■ ■ ■ ■ ■ ■ ■

Concepts

TEAM LinG

TEAM LinG

1 CHAPTER

■ ■ ■ ■ ■ ■ ■ ■ ■ ■

Geometry Representation and Modeling

Two principal tasks are required to create an image of a three-dimensional scene: modeling and rendering. The modeling task generates a model, which is the description of an object that is going to be used by the graphics system. Models must be created for every object in a scene; they should accurately capture the geometric shape and appearance of the object. Some or all of this task commonly occurs when the application is being developed, by creating and storing model descriptions as part of the application’s data. The second task, rendering, takes models as input and generates pixel values for the final image. OpenGL is principally concerned with object rendering; it does not provide explicit support for creating object models. The model input data is left to the application to provide. The OpenGL architecture is focused primarily on rendering polygonal models; it doesn’t directly support more complex object descriptions, such as implicit surfaces. Because polygonal models are the central manner in which to define an object with OpenGL, it is useful to review the basic ideas behind polygonal modeling and how they relate to it.

1.1 Polygonal Representation OpenGL supports a handful of primitive types for modeling two-dimensional (2D) and three-dimensional (3D) objects: points, lines, triangles, quadrilaterals, and

TEAM LinG 3

4

C H A P T E R 1 Geometry Representation and Modeling

(convex) polygons. In addition, OpenGL includes support for rendering higher-order surface patches using evaluators. A simple object, such as a box, can be represented using a polygon for each face in the object. Part of the modeling task consists of determining the 3D coordinates of each vertex in each polygon that makes up a model. To provide accurate rendering of a model’s appearance or surface shading, the modeler may also have to determine color values, shading normals, and texture coordinates for the model’s vertices and faces. Complex objects with curved surfaces can also be modeled using polygons. A curved surface is represented by a gridwork or mesh of polygons in which each polygon vertex is placed on a location on the surface. Even if its vertices closely follow the shape of the curved surface, the interior of the polygon won’t necessarily lie on the surface. If a larger number of smaller polygons are used, the disparity between the true surface and the polygonal representation will be reduced. As the number of polygons increases, the approximation improves, leading to a trade-off between model accuracy and rendering overhead. When an object is modeled using polygons, adjacent polygons may share edges. To ensure that shared edges are rendered without creating gaps between them, polygons that share an edge should use identical coordinate values at the edge’s endpoints. The limited precision arithmetic used during rendering means edges will not necessarily stay aligned when their vertex coordinates are transformed unless their initial values are identical. Many data structures used in modeling ensure this (and save space) by using the same data structure to represent the coincident vertices defining the shared edges.

1.2 Decomposition and Tessellation Tessellation refers to the process of decomposing a complex surface, such as a sphere, into simpler primitives such as triangles or quadrilaterals. Most OpenGL implementations are tuned to process triangles (strips, fans, and independents) efficiently. Triangles are desirable because they are planar and easy to rasterize unambiguously. When an OpenGL implementation is optimized for processing triangles, more complex primitives such as quad strips, quads, and polygons are decomposed into triangles early in the pipeline. If the underlying implementation is performing this decomposition, there is a performance benefit in performing it a priori, either when the database is created or at application initialization time, rather than each time the primitive is issued. Another advantage of having the application decompose primitives is that it can be done consistently and independently of the OpenGL implementation. OpenGL does not specify a decomposition algorithm, so different implementations may decompose a given quadrilateral or polygon differently. This can result in an image that is shaded differently and has different silhouette edges when drawn on two different OpenGL implementations. Most OpenGL implementations have simple decomposition algorithms. Polygons are trivially converted to triangle fans using the same vertex order and quadrilaterals are divided into two triangles; one triangle using the first three vertices and the second using the first plus the last two vertices.

TEAM LinG

S E C T I O N 1 . 2 Decomposition and Tessellation

5

These simple decomposition algorithms are chosen to minimize computation overhead. An alternative is to choose decompositions that improve the rendering quality. Since shading computations assume that a primitive is flat, choosing a decomposition that creates triangles with the best match of the surface curvature may result in better shading. Decomposing a quad to two triangles requires introducing a new edge along one of the two diagonals. A method to find the diagonal that results in more faithful curvature is to compare the angles formed between the surface normals at diagonally opposing vertices. The angle measures the change in surface normal from one corner to its opposite. The pair of opposites forming the smallest angle between them (closest to flat) is the best candidate diagonal; it will produce the flattest possible edge between the resulting triangles, as shown in Figure 1.1. This algorithm may be implemented by computing the dot product between normal pairs, then choosing the pair with the largest dot product (smallest angle). If surface normals are not available, then normals for a vertex may be computed by taking the cross products of the two vectors with origins at that vertex. Surface curvature isn’t the only quality metric to use when decomposing quads. Another one splits the quadrilateral into triangles that are closest to equal in size. Tessellation of simple surfaces such as spheres and cylinders is not difficult. Most implementations of the OpenGL Utility (GLU) library use a straightforward latitude-longitude tessellation for a sphere. While this algorithm is easy to implement, it has the disadvantage that the quads produced from the tessellation have widely

U=A×B

B A

U

θ

D

C V=C×D

V

F i g u r e 1.1 Quadrilateral decomposition.

TEAM LinG

6

C H A P T E R 1 Geometry Representation and Modeling

F i g u r e 1.2 Latitude-longitude tessellation of a sphere.

F i g u r e 1.3 Triangle subdivision: starting octahedron.

varying sizes, as shown in Figure 1.2. The different sized quads can cause noticeable artifacts, particularly if the object is lighted and rotating. A better algorithm generates triangles with sizes that are more consistent. Octahedral and icosahedral tessellations work well and are not very difficult to implement. An octahedral tessellation starts by approximating a sphere with a single octahedron whose vertices are all on the unit sphere, as shown in Figure 1.3. Since each face of the octahedron is a triangle, they can each be easily split into four new triangles.

TEAM LinG

S E C T I O N 1 . 2 Decomposition and Tessellation

Starting octahedron

Find midpoints of each edge

Connect points to form new triangles

7

Normalize points to coincide with surface of unit sphere

F i g u r e 1.4 Octahedron with each triangle being subdivided into four.

F i g u r e 1.5 Triangle subdivision: starting icosahedron.

Each triangle is split by creating a new vertex in the middle of each of the triangle’s existing edges, then connecting them, forming three new edges. The result is that four new triangles are created from the original one; the process is shown in Figure 1.4. The coordinates of each new vertex are divided by the vertex’s distance from the origin, normalizing them. This process scales the new vertex so that it lies on the surface of the unit sphere. These two steps can be repeated as desired, recursively dividing all of the triangles generated in each iteration. The same algorithm can be used with an icosahedron as the base object, as shown in Figure 1.5, by recursively dividing all 20 sides. With either algorithm, it may not be optimal to split the triangle edges in half when tesselating. Splitting the triangle by other

TEAM LinG

8

C H A P T E R 1 Geometry Representation and Modeling

amounts, such as by thirds, or even an arbitrary number, may be necessary to produce a uniform triangle size when the tessellation is complete. Both the icosahedral and octahedral algorithms can be coded so that triangle strips are generated instead of independent triangles, maximizing rendering performance. Alternatively, indexed independent triangle lists can be generated instead. This type of primitive may be processed more efficiently on some graphics hardware.

1.3 Shading Normals OpenGL computes surface shading by evaluating lighting equations at polygon vertices. The most general form of the lighting equa