313 100 4MB
English Pages 672 Year 2005
Advanced Graphics Programming Using OpenGL TOM McREYNOLDS DAVID BLYTHE
AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO MORGAN KAUFMANN PUBLISHERS IS AN IMPRINT OF ELSEVIER
TEAM LinG
Advanced Graphics Programming Using OpenGL
TEAM LinG
The Morgan Kaufmann Series in Computer Graphics and Geometric Modeling Series Editor: Brian A. Barsky University of California, Berkeley This series publishes the finest works for the accomplished and aspiring graphics professional. The series includes intermediate and advanced textbooks, graphics programming books, surveys of important new areas and methods, and reference works. Advanced Graphics Programming Using OpenGL Tom McReynolds and David Blythe
The Computer Animator’s Technical Handbook Lynn Pocock and Judson Rosebush
Digital Geometry Geometric Methods for Digital Picture Analysis Rienhard Klette and Azriel Rosenfeld
Advanced RenderMan: Creating CGI for Motion Pictures Anthony A. Apodaca and Larry Gritz
Digital Video and HDTV Algorithms and Interfaces Charles Poynton
Curves and Surfaces in Geometric Modeling: Theory and Algorithms Jean Gallier
Real-Time Shader Programming Ron Fosner Complete Maya Programming: An Extensive Guide to MEL and the C++ API David Gould MEL Scripting for Maya Animators Mark R. Wilkins and Chris Kazmier Digital Video and HDTV Algorithms and Interfaces Charles Poynton Texturing & Modeling: A Procedural Approach, Third Edition David S. Ebert, F. Kenton Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley Geometric Tools for Computer Graphics Philip Schneider and David Eberly Understanding Virtual Reality: Interface, Application, and Design William Sherman and Alan Craig Jim Blinn’s Corner: Notation, Notation, Notation Jim Blinn Level of Detail for 3D Graphics: David Luebke, Martin Reddy, Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner Pyramid Algorithms: A Dynamic Programming Approach to Curves and Surfaces for Geometric Modeling Ron Goldman Non-Photorealistic Computer Graphics: Modeling, Rendering, and Animation Thomas Strothotte and Stefan Schlechtweg Curves and Surfaces for CAGD: A Practical Guide, Fifth Edition Gerald Farin Subdivision Methods for Geometric Design: A Constructive Approach Joe Warren and Henrik Weimer Computer Animation: Algorithms and Techniques Rick Parent
Andrew Glassner’s Notebook: Recreational Computer Graphics Andrew S. Glassner Warping and Morphing of Graphical Objects Jonas Gomes, Lucia Darsa, Bruno Costa, and Luiz Velho Jim Blinn’s Corner: Dirty Pixels Jim Blinn Rendering with Radiance: The Art and Science of Lighting Visualization Greg Ward Larson and Rob Shakespeare Introduction to Implicit Surfaces Edited by Jules Bloomenthal Jim Blinn’s Corner: A Trip Down the Graphics Pipeline Jim Blinn Interactive Curves and Surfaces: A Multimedia Tutorial on CAGD Alyn Rockwood and Peter Chambers Wavelets for Computer Graphics: Theory and Applications Eric J. Stollnitz, Tony D. DeRose, and David H. Salesin Principles of Digital Image Synthesis Andrew S. Glassner Radiosity & Global Illumination François X. Sillion and Claude Puech Knotty: A B-Spline Visualization Program Jonathan Yen User Interface Management Systems: Models and Algorithms Dan R. Olsen, Jr. Making Them Move: Mechanics, Control, and Animation of Articulated Figures Edited by Norman I. Badler, Brian A. Barsky, and David Zeltzer Geometric and Solid Modeling: An Introduction Christoph M. Hoffmann An Introduction to Splines for Use in Computer Graphics and Geometric Modeling TEAM LinG Richard H. Bartels, John C. Beatty, and Brian A. Barsky
Publishing Director: Diane Cerra Publishing Services Manager: Simon Crump Project Manager: Brandy Lilly Editorial Coordinator: Mona Buehler Cover Design: Dutton & Sherman Design Text Design: Julio Esperas Composition: Cepha Imaging Pvt. Ltd. Illustrations: Dartmouth Publishing, Inc. Copyeditor: Daril Bentley; Graphic World Proofreader: Graphic World Indexer: Graphic World Interior printer: China Translation & Printing Services, Ltd. Cover printer: China Tranalation & Printing Services, Ltd. Morgan Kaufmann Publishers is an imprint of Elsevier. 500 Sansome Street, Suite 400, San Francisco, CA 94111 This book is printed on acid-free paper. © 2005 by Elsevier Inc. All rights reserved.
Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopying, scanning, or otherwise—without prior written permission of the publisher. Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail: [email protected]. You may also complete your request on-line via the Elsevier homepage (http://elsevier.com) by selecting “Customer Support” and then “Obtaining Permissions.” Library of Congress Cataloging-in-Publication Data Application Submitted ISBN: 1-55860-659-9 For information on all Morgan Kaufmann publications, visit our Web site at www.mkp.com or www.books.elsevier.com
Printed in China 10 9 8 7 6 5 4 3 2 1
TEAM LinG
To my friends and colleagues from Silicon Graphics; it was a fabulous time and place to learn about 3D graphics. – DB To Ede Forney and Harry McGinnis; you were there when it counted. – TM
TEAM LinG
Contents
Preface
xxiii
Acknowledgments Biographies
xxvii
xxviii
PART I
Concepts
1
CHAPTER 1
Geometry Representation and Modeling 1.1
Polygonal Representation
1.2
Decomposition and Tessellation
1.3
Shading Normals 1.3.1 1.3.2
1.4
12 Greedy Tri-stripping
Vertices and Vertex Arrays 1.5.1 1.5.2
1.6
3 4
11
Triangle Stripping 1.4.1
1.5
8 Smooth Shading 9 Vertex Winding Order
3
Vertex Buffer Objects Triangle Lists 16
13 14 15
Modeling vs. Rendering Revisited
17
TEAM LinG vi
Contents
vii
CHAPTER 2
3D Transformations
19
2.1
Data Representation
2.2
Overview of the Transformation Pipeline 2.2.1 2.2.2 2.2.3 2.2.4 2.2.5
19
20 Object Space and the Modelview Transform 20 Eye Space and Projection Transform 21 Clip Space and Perspective Divide 22 NDC Space and the Viewport Transform 22 Window Space 23
2.3
Normal Transformation
2.4
Texture Coordinate Generation and Transformation 2.4.1 2.4.2
23
Texture Matrix 25 Texture Coordinate Generation
25
2.5
Modeling Transforms
2.6
Visualizing Transform Sequences
2.7
Projection Transform
2.8
The Z Coordinate and Perspective Projection 2.8.1
27
Vertex Programs
2.10
Summary
28
30
Z Coordinates and Fog
2.9
25
30
32
32
34
CHAPTER 3
Color, Shading, and Lighting 3.1
35
Representing Color 3.1.1 3.1.2 3.1.3 3.1.4
3.2
Shading
3.3
Lighting 3.3.1 3.3.2
35 Resolution and Dynamic Range Gamma 37 Alpha 39 Color Index 39
36
40 43 Intensities, Colors, and Materials Light Source Properties 47
46
TEAM LinG
viii
Contents
3.3.3 3.3.4
3.4
50
Fixed-Point and Floating-Point Arithmetic 3.4.1
3.5
Material Properties 49 Vertex and Fragment Lighting Biased Arithmetic
Summary
53
54
56
CHAPTER 4
Digital Images and Image Manipulation 4.1
Image Representation
4.2
Digital Filtering
4.3
Convolution
4.4
Images in OpenGL
4.5
Positioning Images
4.6
Pixel Store Operations
4.7
57
60
62 63 65 65
Pixel Transfer Operations 4.7.1 4.7.2
4.8
57
67 Scale and Bias 67 Pixel Mapping Operations
67
ARB Imaging Subset 4.8.1 4.8.2 4.8.3 4.8.4 4.8.5 4.8.6
68 Convolution 68 Color Matrix Transform 68 Histogram 69 MinMax 70 Color Tables 70 Blend Equation and Constant Color Blending
4.9
Off-Screen Processing
4.10
Summary
71
72
72
CHAPTER 5
Texture Mapping 5.1
73
Loading Texture Images 5.1.1 5.1.2
73 Texture Borders 74 Internal Texture Formats
75
TEAM LinG
Contents
5.1.3 5.1.4
5.2
Compressed Textures Proxy Textures 77
76
Texture Coordinates 5.2.1
77 Texture Coordinate Generation and Transformation
5.3
Loading Texture Images from the Frame Buffer
5.4
Environment Mapping 5.4.1 5.4.2 5.4.3 5.4.4
5.5
79
80 Generating Environment Map Texture Coordinates Texture Maps Used in Environment Mapping 82 Cube Mapping 83 Sphere Mapping 85
88 Using 3D Textures to Render Solid Materials
5.6
Filtering
5.7
Additional Control of Texture Level of Detail
5.8
Texture Objects
5.9
Multitexture 5.9.1 5.9.2
81
89
90 91
93
95 Multitexture Model 96 Multitexture Texture Environments
97
Texture Environment 5.10.1 5.10.2
5.11
79
3D Texture 5.5.1
5.10
ix
Summary
98 Advanced Texture Environment Functionality Fragment Programs 100
99
102
CHAPTER 6
Rasterization and Fragment Processing 6.1
Rasterization 6.1.1 6.1.2 6.1.3 6.1.4 6.1.5
6.2
103
104 Rasterization Consistency 105 Z-Fighting 105 Bitmaps and Pixel Rectangles 107 Texture, Color, and Depth Interpolation w Buffering 109
108
Fragment Operations 6.2.1 6.2.2 6.2.3
110 Multisample Operations Alpha Test 111 Stencil Test 111
111
TEAM LinG
x
Contents
6.2.4 6.2.5
6.3
112 114
Framebuffer Operations 6.3.1
6.4
Blending Logic Op
Summary
Accumulation Buffer
115 116
117
CHAPTER 7
Window System and Platform Integration 7.1
Renderer and Window State
7.2
Address Space and Threads
7.3
Anatomy of a Window 7.3.1 7.3.2
7.4
7.4.1 7.4.2
120 121
122 Overlay and Underlay Windows Multiple Displays 123
Off-Screen Rendering GLX Pbuffers WGL Pbuffers
119
122
124 125 126
7.5
Rendering to Texture Maps
7.6
Direct and Indirect Rendering
126 127
CHAPTER 8
OpenGL Implementations
129
8.1
OpenGL Versions
8.2
OpenGL Extensions
8.3
OpenGL ES for Embedded Systems 8.3.1 8.3.2 8.3.3 8.3.4
129 131
131 Embedded Profiles 132 Common and Common-Lite Profiles Safety Critical Profile 136 OpenGL ES Revisions 136
8.4
OpenGL Pipeline Evolution
8.5
Hardware Implementations of the Pipeline 8.5.1 8.5.2 8.5.3 8.5.4
133
137
Rasterization Acceleration 138 Primitive Setup Acceleration 141 Transform and Lighting Acceleration Pipeline Balance 142
138
141
TEAM LinG
Contents
8.5.5 8.5.6 8.5.7
8.6
Parallelism Opportunities 142 Reordering the Pipeline 149 Mixed Software and Hardware Implementations
The Future
xi
150
151
P A R T II
Basic Techniques
153
CHAPTER 9
Multiple Rendering Passes 9.1
Invariance
9.2
Multipass Overview
9.3
The Multipass Toolbox 9.3.1 9.3.2 9.3.3 9.3.4 9.3.5
9.4 9.5
155 156
159 Arithmetic Operations 159 Arbitrary Functions 160 Conditionals 161 Variables 162 Parameters 163
Multipass Limitations
165
Multipass vs. Micropass 9.5.1
155
Multitexture
9.6
Deferred Shading
9.7
Summary
165
166 167
167
CHAPTER 10
Antialiasing
169
10.1
Full-Scene Antialiasing
10.2
Supersampling 10.2.1 10.2.2 10.2.3 10.2.4
170
171 Supersampling by Overdrawing 172 Supersampling with the Accumulation Buffer Multisample Antialiasing 175 Drawbacks 176
173
TEAM LinG
xii
Contents
10.3
Area Sampling
10.4
Line and Point Antialiasing
10.5
Antialiasing with Textures
10.6
Polygon Antialiasing
181
Temporal Antialiasing
182 183
10.7
10.7.1
10.8
177
Motion Blur
Summary
178 180
184
CHAPTER 11
Compositing, Blending, and Transparency 11.1
Combining Two Images 11.1.1 11.1.2 11.1.3
185 Compositing 186 Compositing Multiple Images Alpha Division 190
11.2
Other Compositing Operators
11.3
Keying and Matting
11.4
190
192
192 Arithmetic Errors 192 Blending with the Accumulation Buffer Approximation Errors 193 Gamma Correction Errors 193
11.5
Compositing Images with Depth
11.6
Other Blending Operations
11.7
Dissolves
11.8
Transparency
194
195
196 199
200 Dynamic Object Transparency Transparency Mapping 203 Transparency Sorting 204 Depth Peeling 205
202
Screen-Door Transparency 11.10.1
11.11
193
Alpha-Blended Transparency 11.9.1 11.9.2 11.9.3 11.9.4
11.10
187
Blending Artifacts 11.4.1 11.4.2 11.4.3 11.4.4
11.9
185
Summary
205 Multisample Transparency 208
207
TEAM LinG
Contents
xiii
CHAPTER 12
Image Processing Techniques 12.1
OpenGL Imaging Support
12.2
Image Storage
12.3
Point Operations 12.3.1 12.3.2 12.3.3 12.3.4 12.3.5 12.3.6 12.3.7 12.3.8
12.4
12.6
212
213 Color Adjustment 213 Interpolation and Extrapolation Scale and Bias 215 Thresholding 215 Conversion to Luminance 216 Manipulating Saturation 216 Rotating Hue 218 Color Space Conversion 219
225
Convolution
227 Separable Filters 227 Convolutions Using the Accumulation Buffer Convolution Using Extensions 230 Useful Convolution Filters 230 Correlation and Feature Detection 233
228
Geometric Operations 12.7.1 12.7.2 12.7.3 12.7.4
235 Pixel Zoom 235 Scaling Using Texture Mapping 236 Rotation Using Texture Mapping 237 Distortion Correction 237
12.8
Image-Based Depth of Field
12.9
High Dynamic Range Imaging 12.9.1 12.9.2 12.9.3
12.10
213
223 Contrast Stretching 224 Histogram Equalization 224
Reduction Operations 12.6.1 12.6.2 12.6.3 12.6.4 12.6.5
12.7
211
Region-based Operations 12.4.1 12.4.2
12.5
211
Summary
238 241
Dynamic Range 241 Tone Mapping 242 Modeling Adaptation 245 245
TEAM LinG
xiv
Contents
CHAPTER 13
Basic Transform Techniques
247
13.1
Computing Inverse Transforms Efficiently
13.2
Stereo Viewing
13.3
Depth of Field
13.4
Image Tiling
13.5
Billboarding Geometry
13.6
Texture Coordinate vs. Geometric Transformations 13.6.1 13.6.2 13.6.3
13.7
249 252 254 257
261 Direct Vertex to Texture Coordinate Mapping 263 Overlaying an Entire Scene with a Texture 263 Overlaying a Scene with an Independent Texture Projection 264
Interpolating Vertex Components through a Perspective Transformation 265 13.7.1 13.7.2 13.7.3
13.8
247
Transforming Vertices in the Application Interpolating Vertex Components 266 Computing LOD 267
Summary
265
268
CHAPTER 14
Texture Mapping Techniques
269
14.1
Loading Texture Images into a Framebuffer
270
14.2
Optimizing Texture Coordinate Assignment
270
14.3
3D Textures
14.4
Texture Mosaics
14.5
Texture Tiling
14.6
Texture Paging 14.6.1 14.6.2 14.6.3
14.7
271 274 277
279 Texture Subimage Loading 282 Paging Images in System Memory 285 Hardware Support for Texture Paging 286
Prefiltered Textures 14.7.1
287 Computing Texel Aspect Ratios
288
TEAM LinG
Contents
14.8
xv
Dual-Paraboloid Environment Mapping 14.8.1 14.8.2 14.8.3
291 The Mathematics of Dual-Paraboloid Maps Using Dual-Paraboloid Maps 294 OpenGL Dual-Paraboloid Support 296
14.9
Texture Projection
14.10
Texture Color Coding and Contouring
14.11
2D Image Warping
300
14.12
Texture Animation
302
14.13
Detail Textures 14.13.1 14.13.2
291
296
306 Signed Intensity Detail Textures Creating Detail Textures 311
14.14
Texture Sharpening
312
14.15
Mipmap Generation
313
14.16
Texture Map Limits
315
14.17
Summary
298
309
316
CHAPTER 15
Lighting Techniques 15.1
317
Limitations in Vertex Lighting 15.1.1 15.1.2
317 Static and Adaptive Tessellation 319 Local Light and Spotlight Attenuation 320
15.2
Fragment Lighting Using Texture Mapping
15.3
Spotlight Effects Using Projective Textures
322
15.4
Specular Lighting Using Environment Maps
325
15.4.1
15.5
Multitexture
321
326
Light Maps 15.5.1 15.5.2
327 2D Texture Light Maps 3D Texture Light Maps
15.6
BRDF-based Lighting
15.7
Reflectance Maps 15.7.1 15.7.2
327 330
332
332 Gloss Maps 332 Emission Maps 334
TEAM LinG
xvi
Contents
15.8
Per-fragment Lighting Computations
15.9
Other Lighting Models 15.9.1 15.9.2 15.9.3 15.9.4 15.9.5
15.10
15.13
345
352 353
High Dynamic Range Lighting Bloom and Glare Effects
354 354
Global Illumination 15.14.1 15.14.2 15.14.3
15.15
352 Vector Normalization
Bump-mapped Reflections 15.13.1
15.14
343 Approximating Bump Mapping Using Texture Tangent Space 346 Forward Differencing 347 Limitations 351
Normal Maps 15.11.1
15.12
335 Fresnel Reflection 335 Gaussian Reflection 336 Anisotropic Lighting 337 Oren-Nayar Model 340 Cook-Torrance Model 342
Bump Mapping with Textures 15.10.1 15.10.2 15.10.3 15.10.4
15.11
334
Summary
355 Virtual Light Technique 355 Combining OpenGL Lighting with Radiosity Ambient Occlusion 357
356
359
P A R T III
Advanced Techniques
361
CHAPTER 16
CAD and Modeling Techniques 16.1
363
Picking and Highlighting 16.1.1 16.1.2 16.1.3 16.1.4 16.1.5 16.1.6
363 OpenGL Selection 364 Object Tagging in the Color Buffer 365 Proxy Geometry 366 Mapping from Window to Object Coordinates Other Picking Methods 367 Highlighting 367
367
TEAM LinG
Contents
16.1.7 16.1.8
XOR Highlighting 368 Foreground Object Manipulation
16.2
Culling Techniques
16.3
Occlusion Culling 16.3.1 16.3.2 16.3.3 16.3.4 16.3.5
16.4
369
369
370 Choosing Occluders 371 Building the Occlusion Map 371 Building the Depth Estimation Buffer 372 Occlusion Testing 372 Other Occlusion Testing Methods 373
Geometric Level of Detail 16.4.1 16.4.2
373 Changing Detail 374 Transition Techniques 375
16.5
Visualizing Surface Orientation
16.6
Visualizing Surface Curvature
16.7
xvii
377 379
Line Rendering Techniques 16.7.1 16.7.2 16.7.3 16.7.4 16.7.5 16.7.6 16.7.7 16.7.8
380 Wireframe Models 381 Hidden Lines 382 Polygon Offset 384 Depth Range 384 Haloed Lines 385 Silhouette Edges 386 Preventing Antialiasing Artifacts End Caps on Wide Lines 390
16.8
Coplanar Polygons and Decaling
16.9
Capping Clipped Solids
16.10
Constructive Solid Geometry
389
390
392 393
CHAPTER 17
Scene Realism 17.1
403
Reflections 17.1.1 17.1.2 17.1.3 17.1.4 17.1.5
404 Object vs. Image Techniques Planar Reflectors 407 Curved Reflectors 411 Interreflections 419 Imperfect Reflectors 422
404
TEAM LinG
xviii
17.2
Contents
Refraction 17.2.1 17.2.2 17.2.3 17.2.4 17.2.5 17.2.6
17.3
Shadows 17.4.1 17.4.2 17.4.3 17.4.4
17.5
430
Creating Environment Maps 17.3.1 17.3.2 17.3.3 17.3.4 17.3.5 17.3.6
17.4
424 Refraction Equation 424 Planar Refraction 426 Texture Mapped Refraction 428 Environment Mapped Refraction 429 Modeling Multiple Refraction Boundaries Clipping Refracted Objects 431
432 Creating Environment Maps with Ray Casting 433 Creating Environment Maps with Texture Warping 434 Cube Map Textures 437 Sphere Map Textures 440 Dual-paraboloid Maps 443 Updating Environment Maps Dynamically 448 449 Projective Shadows 450 Shadow Volumes 452 Shadow Maps 459 Creating Soft Shadows 463
Summary
465
CHAPTER 18
Natural Detail 18.1
467
Particle Systems 18.1.1 18.1.2 18.1.3 18.1.4 18.1.5
467 Representing Particles 469 Number of Particles 473 Modeling Particle Interactions 473 Updating and Rendering Particles 475 Applications 478
18.2
Dynamic Meshes
18.3
Procedural Texture Generation 18.3.1 18.3.2 18.3.3 18.3.4 18.3.5 18.3.6
484
487 Filtered Noise Functions 487 Generating Noise Functions 489 Filtering Using Texture Convolution Optimizing the Convolution Process Spectral Synthesis 495 Turbulence 496
490 492
TEAM LinG
Contents
18.3.7 18.3.8
18.4
xix
Random Image Warping 498 Generating 3D Noise 498
Summary
500
CHAPTER 19
Illustration and Artistic Techniques 19.1
Projections for Illustration 19.1.1 19.1.2
19.2
501 Axonometric Projection 502 Oblique Projection 503
Nonphotorealistic Lighting Models 19.2.1 19.2.2
Edge Lines
19.4
Cutaway Views 19.4.1
507
508 Surface Texture
19.5
Depth Cuing
19.6
Patterns and Hatching 19.6.1 19.6.2
512 Cross Hatching and 3D Halftones Halftoning 515
513
516 Accuracy in 2D Drawing Line Joins 517 2D Trim Curves 518
516
520 Image-based Text 520 Geometry-based Text 523
Drawing and Painting 19.9.1 19.9.2 19.9.3
19.10
511
Text Rendering 19.8.1 19.8.2
19.9
510
2D Drawing Techniques 19.7.1 19.7.2 19.7.3
19.8
505
Matte Surfaces 505 Metallic Surfaces 506
19.3
19.7
501
Summary
525 Undo and Resolution Independence Painting in 3D 527 Painting on Images 529
527
530
CHAPTER 20
Scientific Visualization 20.1
531
Mapping Numbers to Pictures
531
TEAM LinG
xx
Contents
20.2
Visual Cues and Perception
20.3
Data Characterization
20.4
560 Icons 561 Particle Tracing 561 Stream Lines 563 Illuminated Stream Lines 563 Line Integral Convolution 564
Tensor Field Visualization 20.7.1
20.8
538 Line Graphs 538 Contour Lines 539 Annotating Metrics 539 Image Display 540 Surface Display 543 Isosurfaces 545 Volume Slicing 546 Volume Rendering 547 Texture Slicing 549 Splatting 556 Creating Volume Data 559
Vector Field Visualization 20.6.1 20.6.2 20.6.3 20.6.4 20.6.5
20.7
534 Scatter Plots 534 Iconographic Display 535 Andrews Plots 536 Histograms and Charts 537
Scalar Field Visualization 20.5.1 20.5.2 20.5.3 20.5.4 20.5.5 20.5.6 20.5.7 20.5.8 20.5.9 20.5.10 20.5.11
20.6
532
Point Data Visualization 20.4.1 20.4.2 20.4.3 20.4.4
20.5
531
Summary
Hyperstreamlines
568 569
570
CHAPTER 21
Structuring Applications for Performance 21.1
Structuring Graphics Processing 21.1.1 21.1.2 21.1.3
21.2
571
Scene Graphs 572 Vertex Updates 575 Texture Updates 576
Managing Frame Time 21.2.1
571
Input Phase
577 579
TEAM LinG
Contents
21.2.2 21.2.3 21.2.4
21.3
Rendering Phase 579 Computation Phase 580 The Safety Margin 581
Application Performance Tuning
581 Locating Bottlenecks 581 Finding Application Bottlenecks 583 Measuring Performance 587 Measuring Depth Complexity 589 Pipeline Interleaving 591
21.3.1 21.3.2 21.3.3 21.3.4 21.3.5
21.4
xxi
Summary
592
APPENDIX A
Using OpenGL Extensions
593
A.1
How OpenGL Extensions are Documented
A.2
Finding OpenGL Extension Specifications
A.3
How to Read an OpenGL Extension Specification A.3.1
ARB Extensions
593 594 594
598
A.4
Portable Use of OpenGL Extensions
599
A.5
Using Extension Function Pointers
602
APPENDIX B
Equations B.1
3D Vectors B.1.1 B.1.2 B.1.3
B.2
605 605 Spherical Coordinates 606 Linear Interpolation of 3D Vectors Barycentric Coordinates 607
606
Projection Matrices B.2.1 B.2.2 B.2.3 B.2.4
607 Orthographic Projection 607 Perspective Projection 607 Perspective z-Coordinate Transformations Alternative Perspective Projection 608
B.3
Viewing Transforms
B.4
Modeling Transforms B.4.1
Scaling
608
608 609
609
TEAM LinG
xxii
Contents
B.4.2 B.4.3
Translation 609 Rotation 609
B.5
Parallel and Perpendicular Vectors
B.6
Reflection Vector
B.7
Lighting Equations
B.8
Function Approximations B.8.1 B.8.2 B.8.3
610
610 610
612 Taylor Series Expansion 612 Newton-Raphson Method 612 Hypotenuse 613
Bibliography Subject Index
615 629
TEAM LinG
Preface
Overview Computer graphics has come a long way from the early days of line drawings and light pens. Today anyone can run interactive and realistic graphics applications on the hardware available on an affordable personal computer. While hardware progress has been impressive, widespread gains in software expertise has been more elusive. There are many computer graphics professionals and enthusiasts out there, but a comprehensive understanding of the accelerated graphics pipeline and how to exploit it is less widespread. This book attempts to bring the computer graphics enthusiast, whether professional or amateur, beyond the basics covered in computer graphics texts, and introduce them to a mix of more intense practical and theoretical discussion that is hard to obtain outside of a professional computer graphics environment. We emphasize the algorithmic side of computer graphics, with a practical application focus. We try to strike a balance between useful examples and approachable theory. We present usable techniques for real world problems, but support them with enough theory and background so the reader can extend and modify the ideas presented here. This book is about graphics techniques, techniques that don’t require esoteric hardware or custom graphics libraries, that are written in a comprehensible style, and do useful things. This book will teach you some graphics, especially areas that are sometimes underrepresented in graphics texts. But it also goes further, showing you how to apply those techniques in real world applications, filling real world needs. Since there are already a number of excellent books that provide an introduction to computer graphics (Foley, 1994; Watt, 1989; Rogers, 1997; Angel, 1997; Newman, 1973) and to OpenGL programming (Neider, 1997; Angel, 1997) we have been necessarily brief in these areas. We assume that the reader is comfortable with these fundamentals; however, we have included extra introductory material where we thought it would improve understanding of later sections. We also note that the computer graphics field has a lot of competing notation and vocabulary. We have tried to be consistent with terminology and notation used in the OpenGL specification and the “standard” OpenGL books while at the same time providing some mention of alternative terminology when it is relevent.
TEAM LinG xxiii
xxiv
Preface
OpenGL We chose OpenGL as our base graphics language for a number of reasons. It is designed to be full featured, to run efficiently on a wide range of graphics architectures, and is clean and straightforward to use. It also dictates very little policy. It is easy to mix and match graphics features in OpenGL to get new effects that may not have even been considered when the language was designed. Its clear specification gives the application programmer confidence that applications written in OpenGL will act predictably on many different graphics hardware and driver implementations. OpenGL is also widely available. It can be obtained for free on all the important architectures today: Apple Machintosh, all flavors of Microsoft Windows, nearly all Unix variants including Linux, and OS/2. Most commercial system and graphics hardware vendors support OpenGL as well, and support for hardware accelerated implementations has grown rapidly, especially in the personal computer space. OpenGL runs on a wide range of graphics hardware; from “big iron” compute clusters, to OpenGL ES, which is designed to provide 3D graphics on embedded devices as small as a cell phone. Given the broad applicability, scalability, and wide availability, OpenGL is an easy choice as the basis for describing graphics algorithms. However, even if you don’t use OpenGL, the graphics APIs in common use are conceptually similar enough that you will still find this book useful. OpenGL remains an evolving specification. Throughout the book we make references to various revisions of the specification (versions 1.0–1.5) and discuss both OpenGL architecture review board (ARB) extensions and various vendor-specific extensions when we believe they enhance the discussion of a particular subject. Rather than focus on the feature set of the most advanced versions of OpenGL, we have included a broad range of algorithms with varying requirements. For many techniques we describe algorithm variations that cover a range of earlier and more advanced versions of OpenGL. We have followed this path since a wide range of OpenGL versions are deployed across various environments including the burgeoning embedded space.
Book Organization This book is divided into three parts. We start with a conceptual overview of computer graphics, emphasizing areas important to the techniques in this book, with extra attention in some overlooked areas. Hand in hand with our introduction to computer graphics, we’ll describe the OpenGL pipeline, with extra detail on the parts of the pipeline most techniques rely on heavily: lighting, texture mapping, rasterization, and depth buffering. We also use this opportunity to describe OpenGL system deployment, including the platform embedding layer and an overview of common hardware acceleration techniques for the pipeline.
TEAM LinG
Preface
xxv
With this foundation in place, Part II introduces a set of important basic techniques. Useful by themselves, they help re-enforce the insights gleaned from the overview. These sequences are also used as building blocks for more complex and sophisticated techniques. To help tie them more tightly to the graphics concepts described in the previous part, these techniques are presented and organized with respect to the OpenGL architecture. The third and final part covers more sophisticated and complex techniques. These techniques are categorized into application areas to help organize the material. The start of each application section has an introduction to the problems and issues important for that area, making these sections more interesting and useful to readers not well versed in that particular field. The book is heavily cross-referenced. Complex techniques reference the simple ones they build upon, and both levels of technique reference the conceptual overview. This allows the reader to use the book as a self-guided tutorial, learning more about techniques and concepts of interest in greater depth.
Example Code To avoid cluttering the text with large fragments of example code, code fragments are used sparingly. Algorithms are usually described as a sequence of steps. However, since details are often the difference between a working program and a black screen, we have tried to include full blown example programs for most algorithms. This example code is available for internet download from www.mkp.com/opengl.
Conventions We use conventions and terminology similar to that found in the OpenGL specification and in the “red-blue-green-white” series of OpenGL books. In addition, we use the following conventions: •
Equations involving matrices, vectors, and points use single uppercase letters for most variables. Vectors are emboldened (V), whereas points and matrices are not (M, P). In rare occasions vectors or points are in lower case.
•
Occasionally symbols are context specific, but generally the following meanings hold: –
N - normal vector
–
L - light vector
TEAM LinG
Preface
xxvi
–
R - reflection vector
–
T - tangent vector
–
B - binormal vector
–
s, t, r, q - texture coordinates
–
x, y, z, w - vertex coordinates
–
θ , ϕ - spherical coordinate angles
–
RGBA - red, green, blue, and alpha components
–
I - intensity
–
C - color (usually RGB or RGBA)
–
V - length of vector V
–
[n, m] a number between n and m including the end points
–
A · B - inner product of vectors A and B
–
A ⊙ B - max{0, A · B} – the clamped inner product
–
A × B - cross product of vectors A and B
TEAM LinG
Acknowledgments This book reflects a significant part of our collective experience in working with OpenGL applications for the past 13 years. While the book couldn’t possibly cover everything known about using OpenGL, we are pleased to provide this useful subset. Of course, we didn’t figure it out all on our own; we are indebted to the many people that helped us one way or the other: either by providing ideas, correcting misconceptions, prototyping early algorithms, teasing us about taking so long to complete the book, or providing the occasional encouraging word. Many people contributed to this effort: any omissions from the list below are inadvertent. The following people contributed directly to the original 1996–2000 SIGGRAPH course notes that were the genesis for this book: Celeste Fowler, Brad Grantham, Mark Kilgard, Scott Nelson, Simon Hui, and Paula Womack. We had invaluable production help with the course notes over the years from Dany Galgani (illustrations), Linda Rae Sande (production editing), Bob Brown, and Chris Everett. Bowen ‘Cheetah’ Goletz helped with the logistics of sharing the source material over the internet. We are also indebted to those who have made tools such as TeX/LaTeX, GhostScript/Ghostview, and cvs freely available on the three different computing platforms that we used for preparing the original course notes. The original notes benefited from the patient attention of an army of reviewers. They include Dave Shreiner, Paul Strauss, David Yu, Hansong Zhang, Sharon Clay, Robert Grzeszczuk, Phil Lacroute, Mark Peercy, Lena Petrovic, Allan Schaffer, Mark Stadler, John Airey, Allen Akin, Brian Cabral, Tom Davis, Bob Drebin, Ben Garlick, Michael Gold, Paul Haeberli, Michael Jones, Phil Keslin, Erik Lindholm, Mark Young, and Mark Segal. This book would not exist without the wealth of experience, cool ideas, tricks, hacks, and wisdom generously provided to us. It is hard to acknowledge everyone properly. Here is our attempt to do so: Kurt Akeley, Brian Cabral, Amy Gooch, Wolfgang Heidrich, Detlev Stalling, Hansong Zhang, Luis Barcena, Angus Dorbie, Bob Drebin, Mark Peercy, Nacho Sanz-Pastor Revorio, Chris Tanner, David Yu, John Airey, Remi Arnaud, Greg Ward, Phil Lacroute, and Peter-Pike Sloan. We would also like to acknowledge Atul Narkhede, Rob Wheeler, Nate Robbins, and Chris McCue for coding prototype algorithms. A number of people helped refine the raw material from the course notes into this manuscript and have earned our gratitude: Ben Luna, Jeff Somers, Brandy Lilly, and Jessica Meehan in particular. We are also greatly indebted to our reviewers: Ian Ashdown, Dave Shreiner, Noah Gibbs, Brian Paul, and Clay Budin.
TEAM LinG xxvii
Biographies
David Blythe David Blythe has worked in the 3D graphics field professionally for the last 14 years, including serving as Chief Engineer at Silicon Graphics, a representative on the OpenGL Architecture Review Board, editor for the OpenGL ES 1.0 specification, and a frequent SIGGRAPH course presenter. While at Silicon Graphics, David contributed to the development of the RealityEngine and InfiniteReality graphics systems. He has worked extensively on implementations of the OpenGL graphics library, OpenGL extension specifications, and high-level toolkits built on top of OpenGL. David’s other industry experience includes embedded and system-on-a-chip design, mobile devices, and wireless networking. David is currently a graphics architect in the Windows Graphics and Gaming Technologies division at Microsoft working on DirectX and OpenGL graphics technologies. Tom McReynolds Tom McReynolds has worked on 3D graphics at Sun Microsystems, Silicon Graphics, Gigapixel, 3Dfx, and NVIDIA. He has worked in software organizations, writing graphics libraries and drivers, and on the hardware side, writing simulators and verification software for 3D hardware. He presented 3D graphics courses at a number of SIGGRAPH conferences, as well as at a number of Silicon Graphics Developer conferences, an X technical conference, and at Linux World. Tom is currently managing a development team that writes 3D graphics drivers for embedded GPUs at NVIDIA, and contributing to the evolution of OpenGL ES by participating in the Khronos working group.
TEAM LinG xxviii
I PART
■ ■ ■ ■ ■ ■ ■ ■ ■ ■
Concepts
TEAM LinG
TEAM LinG
1 CHAPTER
■ ■ ■ ■ ■ ■ ■ ■ ■ ■
Geometry Representation and Modeling
Two principal tasks are required to create an image of a three-dimensional scene: modeling and rendering. The modeling task generates a model, which is the description of an object that is going to be used by the graphics system. Models must be created for every object in a scene; they should accurately capture the geometric shape and appearance of the object. Some or all of this task commonly occurs when the application is being developed, by creating and storing model descriptions as part of the application’s data. The second task, rendering, takes models as input and generates pixel values for the final image. OpenGL is principally concerned with object rendering; it does not provide explicit support for creating object models. The model input data is left to the application to provide. The OpenGL architecture is focused primarily on rendering polygonal models; it doesn’t directly support more complex object descriptions, such as implicit surfaces. Because polygonal models are the central manner in which to define an object with OpenGL, it is useful to review the basic ideas behind polygonal modeling and how they relate to it.
1.1 Polygonal Representation OpenGL supports a handful of primitive types for modeling two-dimensional (2D) and three-dimensional (3D) objects: points, lines, triangles, quadrilaterals, and
TEAM LinG 3
4
C H A P T E R 1 Geometry Representation and Modeling
(convex) polygons. In addition, OpenGL includes support for rendering higher-order surface patches using evaluators. A simple object, such as a box, can be represented using a polygon for each face in the object. Part of the modeling task consists of determining the 3D coordinates of each vertex in each polygon that makes up a model. To provide accurate rendering of a model’s appearance or surface shading, the modeler may also have to determine color values, shading normals, and texture coordinates for the model’s vertices and faces. Complex objects with curved surfaces can also be modeled using polygons. A curved surface is represented by a gridwork or mesh of polygons in which each polygon vertex is placed on a location on the surface. Even if its vertices closely follow the shape of the curved surface, the interior of the polygon won’t necessarily lie on the surface. If a larger number of smaller polygons are used, the disparity between the true surface and the polygonal representation will be reduced. As the number of polygons increases, the approximation improves, leading to a trade-off between model accuracy and rendering overhead. When an object is modeled using polygons, adjacent polygons may share edges. To ensure that shared edges are rendered without creating gaps between them, polygons that share an edge should use identical coordinate values at the edge’s endpoints. The limited precision arithmetic used during rendering means edges will not necessarily stay aligned when their vertex coordinates are transformed unless their initial values are identical. Many data structures used in modeling ensure this (and save space) by using the same data structure to represent the coincident vertices defining the shared edges.
1.2 Decomposition and Tessellation Tessellation refers to the process of decomposing a complex surface, such as a sphere, into simpler primitives such as triangles or quadrilaterals. Most OpenGL implementations are tuned to process triangles (strips, fans, and independents) efficiently. Triangles are desirable because they are planar and easy to rasterize unambiguously. When an OpenGL implementation is optimized for processing triangles, more complex primitives such as quad strips, quads, and polygons are decomposed into triangles early in the pipeline. If the underlying implementation is performing this decomposition, there is a performance benefit in performing it a priori, either when the database is created or at application initialization time, rather than each time the primitive is issued. Another advantage of having the application decompose primitives is that it can be done consistently and independently of the OpenGL implementation. OpenGL does not specify a decomposition algorithm, so different implementations may decompose a given quadrilateral or polygon differently. This can result in an image that is shaded differently and has different silhouette edges when drawn on two different OpenGL implementations. Most OpenGL implementations have simple decomposition algorithms. Polygons are trivially converted to triangle fans using the same vertex order and quadrilaterals are divided into two triangles; one triangle using the first three vertices and the second using the first plus the last two vertices.
TEAM LinG
S E C T I O N 1 . 2 Decomposition and Tessellation
5
These simple decomposition algorithms are chosen to minimize computation overhead. An alternative is to choose decompositions that improve the rendering quality. Since shading computations assume that a primitive is flat, choosing a decomposition that creates triangles with the best match of the surface curvature may result in better shading. Decomposing a quad to two triangles requires introducing a new edge along one of the two diagonals. A method to find the diagonal that results in more faithful curvature is to compare the angles formed between the surface normals at diagonally opposing vertices. The angle measures the change in surface normal from one corner to its opposite. The pair of opposites forming the smallest angle between them (closest to flat) is the best candidate diagonal; it will produce the flattest possible edge between the resulting triangles, as shown in Figure 1.1. This algorithm may be implemented by computing the dot product between normal pairs, then choosing the pair with the largest dot product (smallest angle). If surface normals are not available, then normals for a vertex may be computed by taking the cross products of the two vectors with origins at that vertex. Surface curvature isn’t the only quality metric to use when decomposing quads. Another one splits the quadrilateral into triangles that are closest to equal in size. Tessellation of simple surfaces such as spheres and cylinders is not difficult. Most implementations of the OpenGL Utility (GLU) library use a straightforward latitude-longitude tessellation for a sphere. While this algorithm is easy to implement, it has the disadvantage that the quads produced from the tessellation have widely
U=A×B
B A
U
θ
D
C V=C×D
V
F i g u r e 1.1 Quadrilateral decomposition.
TEAM LinG
6
C H A P T E R 1 Geometry Representation and Modeling
F i g u r e 1.2 Latitude-longitude tessellation of a sphere.
F i g u r e 1.3 Triangle subdivision: starting octahedron.
varying sizes, as shown in Figure 1.2. The different sized quads can cause noticeable artifacts, particularly if the object is lighted and rotating. A better algorithm generates triangles with sizes that are more consistent. Octahedral and icosahedral tessellations work well and are not very difficult to implement. An octahedral tessellation starts by approximating a sphere with a single octahedron whose vertices are all on the unit sphere, as shown in Figure 1.3. Since each face of the octahedron is a triangle, they can each be easily split into four new triangles.
TEAM LinG
S E C T I O N 1 . 2 Decomposition and Tessellation
Starting octahedron
Find midpoints of each edge
Connect points to form new triangles
7
Normalize points to coincide with surface of unit sphere
F i g u r e 1.4 Octahedron with each triangle being subdivided into four.
F i g u r e 1.5 Triangle subdivision: starting icosahedron.
Each triangle is split by creating a new vertex in the middle of each of the triangle’s existing edges, then connecting them, forming three new edges. The result is that four new triangles are created from the original one; the process is shown in Figure 1.4. The coordinates of each new vertex are divided by the vertex’s distance from the origin, normalizing them. This process scales the new vertex so that it lies on the surface of the unit sphere. These two steps can be repeated as desired, recursively dividing all of the triangles generated in each iteration. The same algorithm can be used with an icosahedron as the base object, as shown in Figure 1.5, by recursively dividing all 20 sides. With either algorithm, it may not be optimal to split the triangle edges in half when tesselating. Splitting the triangle by other
TEAM LinG
8
C H A P T E R 1 Geometry Representation and Modeling
amounts, such as by thirds, or even an arbitrary number, may be necessary to produce a uniform triangle size when the tessellation is complete. Both the icosahedral and octahedral algorithms can be coded so that triangle strips are generated instead of independent triangles, maximizing rendering performance. Alternatively, indexed independent triangle lists can be generated instead. This type of primitive may be processed more efficiently on some graphics hardware.
1.3 Shading Normals OpenGL computes surface shading by evaluating lighting equations at polygon vertices. The most general form of the lighting equa