Programming Microsoft SQL Server 2012 9780735658226, 0735658226

Citation preview

Programming Microsoft SQL Server 2012 ®

Leonard Lobel Andrew Brust

®

Pub shed w th the author zat on of M crosoft Corporat on by O’Re y Med a, Inc 1005 Gravenste n H ghway North Sebastopo , Ca forn a 95472 Copyr ght © 2012 by S eek Techno og es Inc , and B ue Badge Ins ghts, Inc A r ghts reserved No part of the contents of th s book may be reproduced or transm tted n any form or by any means w thout the wr tten perm ss on of the pub sher ISBN 978-0-7356-5822-6 123456789 M 765432 Pr nted and bound n the Un ted States of Amer ca M crosoft Press books are ava ab e through bookse ers and d str butors wor dw de If you need support re ated to th s book, ema M crosoft Press Book Support at msp nput@m crosoft com P ease te us what you th nk of th s book at http //www m crosoft com/ earn ng/booksurvey M crosoft and the trademarks sted at http //www m crosoft com/about/ ega /en/us/Inte ectua Property/ Trademarks/EN-US aspx are trademarks of the M crosoft group of compan es A other marks are property of the r respect ve owners The examp e compan es, organ zat ons, products, doma n names, ema addresses, ogos, peop e, p aces, and events dep cted here n are fict t ous No assoc at on w th any rea company, organ zat on, product, doma n name, ema address, ogo, person, p ace, or event s ntended or shou d be nferred Th s book expresses the author’s v ews and op n ons The nformat on conta ned n th s book s prov ded w thout any express, statutory, or mp ed warrant es Ne ther the authors, O’Re y Med a, Inc , M crosoft Corporat on, nor ts rese ers, or d str butors w be he d ab e for any damages caused or a eged to be caused e ther d rect y or nd rect y by th s book Acquisitions Editor: Russe Jones Developmental Editor: Russe Jones Production Editor: Me an e Yarbrough Editorial Production: Chr st an Ho dener, S4Car s e Pub sh ng Serv ces Technical Reviewer: John Pau Meu er Copyeditor: Andrew Jones Indexer: WordCo Index ng Serv ces Cover Design: Tw st Creat ve • Seatt e Cover Composition: ContentWorks, Inc Illustrator: Rebecca Demarest

To my partner, Mark, and our children, Adam, Jacqueline, Joshua, and Sonny. With all my love, I thank you guys, for all of yours. —  Leonard Lobel

For my three boys: Miles, Sean, and Aidan. And for my sister, Valerie Hope. — Andrew Brust

Contents at a Glance Introduction xxi Part I

Core SQL Server Development

Chapter 1

Introducing SQL Server Data Tools

Chapter 2

T-SQL Enhancements

Chapter 3

Exploring SQL CLR

125

Chapter 4

Working with Transactions

169

Chapter 5

SQL Server Security

207

Part II

Going Beyond Relational

Chapter 6

XML and the Relational Database

255

Chapter 7

Hierarchical Data and the Relational Database

299

Chapter 8

Native File Streaming

323

Chapter 9

Geospatial Support

367

Part III

Applied SQL

Chapter 10

The Microsoft Data Access Juggernaut

427

Chapter 11

WCF Data Access Technologies

509

Chapter 12

Moving to the Cloud with SQL Azure

579

Chapter 13



SQL Azure Data Sync and Windows Phone Development

619

Chapter 14

Pervasive Insight

675

Chapter 15

xVelocity In-Memory Technologies

701

3 45

Index 737

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxvii

Part I

Core SQL Server Development

Chapter 1 Introducing SQL Server Data Tools Introduc ng SSDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 4

Database Too ng Des gned for Deve opers. . . . . . . . . . . . . . . . . . . . .

4

Dec arat ve, Mode -Based Deve opment. . . . . . . . . . . . . . . . . . . . . . . .

5

Connected Deve opment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

D sconnected Deve opment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

Vers on ng and Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

Target ng D fferent P atforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

Work ng w th SSDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

Connect ng w th SQL Server Object Exp orer. . . . . . . . . . . . . . . . . . .

10

Gather ng New Requ rements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

16

Us ng the Tab e Des gner (Connected). . . . . . . . . . . . . . . . . . . . . . . . .

17

Work ng Offl ne w th a SQL Server Database Project. . . . . . . . . . . .

22

Tak ng a Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25

Us ng the Tab e Des gner (Offl ne Database Project). . . . . . . . . . . . .

25

Introduc ng Loca DB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

Refactor ng the Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31

Test ng and Debugg ng

33

Compar ng Schemas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

Pub sh ng to SQL Azure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39

Adopt ng SSDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

42

Summary

43

What do you think of this book? We want to hear from you! M crosoft s nterested n hear ng your feedback so we can cont nua y mprove our books and earn ng resources for you. To part c pate n a br ef on ne survey, p ease v s t:

microsoft.com/learning/booksurvey

vii

Chapter 2 T-SQL Enhancements Tab e-Va ued Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 46

More Than Just Another Temporary Tab e So ut on . . . . . . . . . . . . .

46

Subm tt ng Orders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

Us ng TVPs for Bu k Inserts and Updates. . . . . . . . . . . . . . . . . . . . . . .

49

Pass ng TVPs Us ng ADO NET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

Pass ng Co ect ons to TVPs Us ng Custom Iterators. . . . . . . . . . . . .

54

TVP L m tat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57

Date and T me Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

Separat on of Dates and T mes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

More Portab e Dates and T mes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

T me Zone Awareness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

59

Date and T me Accuracy, Storage, and Format. . . . . . . . . . . . . . . . . .

60

Date and T me Funct ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

62

The MERGE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65

Defin ng the Merge Source and Target. . . . . . . . . . . . . . . . . . . . . . . .

67

The WHEN MATCHED C ause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

68

The WHEN NOT MATCHED BY TARGET C ause . . . . . . . . . . . . . . . . .

69

Us ng MERGE for Tab e Rep cat on . . . . . . . . . . . . . . . . . . . . . . . . . . .

70

The WHEN NOT MATCHED BY SOURCE C ause. . . . . . . . . . . . . . . . .

71

MERGE Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

Choos ng a Jo n Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

MERGE DML Behav or. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75

The INSERT OVER DML Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A F terab e A ternat ve to OUTPUT…INTO. . . . . . . . . . . . . . . . . . . . .

76 77

Consum ng CHANGES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

viii Contents

The GROUPING SETS Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83

Ro ng Up by Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

Ro ng Up A Leve Comb nat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . .

86

Return ng Just the Top Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

88

M x ng and Match ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89

Hand ng NULL Va ues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

90

W ndow ng (OVER C ause) Enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . S d ng Aggregat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93 96

Us ng RANGE versus ROWS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 New T-SQL Funct ons n SQL Server 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . . New Ana yt c Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

97 98

New Convers on Funct ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

103

New Date and T me Funct ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

104

New Log ca Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

106

New Str ng Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

107

Changed Mathemat ca Funct on. . . . . . . . . . . . . . . . . . . . . . . . . . . .

109

The THROW Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

109

Re-Throw ng Except ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

110

Compar ng THROW and RAISERROR. . . . . . . . . . . . . . . . . . . . . . . . .

111

Server-S de Pag ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

113

Us ng ROW NUMBER. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

113

Us ng OFFSET/FETCH NEXT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

114

The SEQUENCE Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sequence L m tat ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 117

Metadata D scovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

118

Summary

122

Chapter 3 Exploring SQL CLR

125

Gett ng Started Enab ng CLR Integrat on. . . . . . . . . . . . . . . . . . . . . . . . . . V sua Stud o/SQL Server Integrat on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

126 126

SQL Server Database Projects n V sua Stud o. . . . . . . . . . . . . . . . .

127

Automated Dep oyment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

129

SQL CLR Code Attr butes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

129

Your F rst SQL CLR Stored Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

130

CLR Stored Procedures and Server-S de Data Access. . . . . . . . . . . . . . . . .

132

P p ng Data w th SqlDataRecord and SqlMetaData. . . . . . . . . . . . . 134 Dep oyment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

136

Contents ix

Gett ng Ready. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137

Dep oy ng Your Assemb y. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

138

Dep oy ng Your Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . .

141

Test ng Your Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

142

CLR Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

143

CLR Tr ggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

148

CLR Aggregates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

151

SQL CLR Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

156

Secur ty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

161

Exam n ng and Manag ng CLR Types n a Database. . . . . . . . . . . . . . . . . .

162

Best Pract ces for SQL CLR Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

168

Summary

168

Chapter 4 Working with Transactions What Is a Transact on?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understand ng the ACID Propert es. . . . . . . . . . . . . . . . . . . . . . . . . . Loca Transact on Support n SQL Server. . . . . . . . . . . . . . . . . . . . . . . . . . .

170 170 172

Autocomm t Transact on Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

173

Exp c t Transact on Mode

173

Imp c t Transact on Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

176

Batch-Scoped Transact on Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . .

176

Iso at on Leve s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

179

Read Uncomm tted Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . .

179

Read Comm tted Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

181

Repeatab e Read Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

182

Ser a zab e Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

182

Snapshot Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

182

Read Comm tted Snapshot Iso at on Leve . . . . . . . . . . . . . . . . . . . .

183

Iso at on Leve s n ADO NET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

184

D str buted Transact ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D str buted Transact on Term no ogy. . . . . . . . . . . . . . . . . . . . . . . . .

x Contents

169

186 186

Ru es and Methods of En stment. . . . . . . . . . . . . . . . . . . . . . . . . . . .

187

D str buted Transact ons n SQL Server . . . . . . . . . . . . . . . . . . . . . . .

189

D str buted Transact ons n the NET Framework. . . . . . . . . . . . . . .

190

Us ng a Resource Manager n a Successfu Transact on . . . . . . . . .

198

Transact ons n SQL CLR (CLR Integrat on). . . . . . . . . . . . . . . . . . . . . . . . . .

201

Putt ng It A Together. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

204

Summary

206

Chapter 5 SQL Server Security

207

Four Themes of the Secur ty Framework . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure by Des gn. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure by Defau t. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure by Dep oyment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure Commun cat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

SQL Server Secur ty Overv ew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

209

SQL Server Log ns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

210

Database Users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

211

The guest User Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

212

Authent cat on and Author zat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

213

How C ents Estab sh a Connect on. . . . . . . . . . . . . . . . . . . . . . . . . .

213

Password Po c es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

215

User-Schema Separat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

216

Execut on Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

218

Encrypt on Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

222

Encrypt ng Data on the Move. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

223

Encrypt ng Data at Rest

224

Transparent Data Encrypt on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

229

SQL Server Aud t. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creat ng an Aud t Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

234 235

Aud t ng Opt ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

236

Record ng Aud ts to the F e System. . . . . . . . . . . . . . . . . . . . . . . . . .

238

Record ng Aud ts to the W ndows Event Log. . . . . . . . . . . . . . . . . .

239

Aud t ng Server Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

239

Aud t ng Database Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

240

V ew ng Aud ted Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

242

Query ng Aud t Cata og V ews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

244

Contents xi

Part a y Conta ned Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

245

Creat ng a Conta ned User. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

245

Other Part a y Conta ned Database Features. . . . . . . . . . . . . . . . . .

246

How Hackers Attack SQL Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

D rect Connect on to the Internet. . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

Weak System Adm n strator Account Passwords. . . . . . . . . . . . . . .

249

SQL Server Browser Serv ce. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

SQL Inject on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

250

Inte gent Observat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

250

Summary

Part II

244

Creat ng a Part a y Conta ned Database. . . . . . . . . . . . . . . . . . . . . .

251

Going Beyond Relational

Chapter 6 XML and the Relational Database

255

Character Data as XML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

256

The xml Data Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

257

Work ng w th the xml Data Type as a Var ab e. . . . . . . . . . . . . . . . .

257

Work ng w th XML n Tab es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

258

XML Schema Defin t ons (XSDs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

259

XML Indexes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

266

FOR XML Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

268

FOR XML RAW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 FOR XML AUTO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 FOR XML EXPLICIT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Add t ona FOR XML Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

276

The TYPE Opt on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

276

FOR XML PATH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Em tt ng a ROOT E ement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Produc ng an In ne XSD Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . .

281

Produc ng E ement-Based XML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

282

Shredd ng XML Us ng OPENXML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

284

Query ng XML Data Us ng XQuery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

285

Understand ng XQuery Express ons and XPath. . . . . . . . . . . . . . . . xii Contents

280

285

SQL Server XQuery n Act on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

288

XML DML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

296

Summary

Chapter 7 Hierarchical Data and the Relational Database

298

299

The hierarchyid Data Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

300

Creat ng a H erarch ca Tab e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

301

The GetLevel Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Popu at ng the H erarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

302 303

The GetRoot Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

303

The GetDescendant Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

304

The ToString Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

305

The GetAncestor Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

310

H erarch ca Tab e Index ng Strateg es . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

313

Depth-F rst Index ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

314

Breadth-F rst Index ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

314

Query ng H erarch ca Tab es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

315

The IsDescendantOf Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

315

Reorder ng Nodes w th n the H erarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . .

317

The GetReparentedValue Method. . . . . . . . . . . . . . . . . . . . . . . . . . . .

318

Transp ant ng Subtrees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

319

More hierarchyid Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

321

Summary

322

Chapter 8 Native File Streaming Trad t ona BLOB Strateg es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

323 323

BLOBs n the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

324

BLOBs n the F e System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

324

Introduc ng FILESTREAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

325

Enab ng FILESTREAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

326

Enab ng FILESTREAM for the Mach ne . . . . . . . . . . . . . . . . . . . . . . .

326

Enab ng FILESTREAM for the Server Instance . . . . . . . . . . . . . . . . .

328

Contents xiii

Creat ng a FILESTREAM-Enab ed Database. . . . . . . . . . . . . . . . . . . . . . . . . Creat ng a Tab e w th FILESTREAM Co umns. . . . . . . . . . . . . . . . . . .

329 330

Stor ng and Retr ev ng FILESTREAM Data

331

De et ng FILESTREAM Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

334

D rect Stream ng n NET w th SqlFileStream. . . . . . . . . . . . . . . . . . . . . . . .

335

Understand ng SqlFileStream. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Bu d ng the W ndows Forms C ent. . . . . . . . . . . . . . . . . . . . . . . . . .

337

Programm ng SqlFileStream Data Access

338

Creat ng a Stream ng HTTP Serv ce . . . . . . . . . . . . . . . . . . . . . . . . . .

348

Bu d ng a WPF C ent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

352

FILESTREAM L m tat ons and Cons derat ons. . . . . . . . . . . . . . . . . . . . . . . . Introduc ng F eTab e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

357

Creat ng a F eTab e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

360

Man pu at ng a F eTab e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

362

Search ng Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

365

Summary

366

Chapter 9 Geospatial Support

367

SQL Server Spaces Out. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

367

Spat a Mode s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

368

P anar (F at-Earth) Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

368

Geodet c (E pso da Sphere) Mode. . . . . . . . . . . . . . . . . . . . . . . . . .

368

Spat a Data Standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

370

Import ng We -Known Text (WKT) . . . . . . . . . . . . . . . . . . . . . . . . . .

370

Import ng WKB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

373

Import ng Geography Markup Language (GML). . . . . . . . . . . . . . .

374

Spat a Data Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

374

Work ng w th geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

375

Work ng w th geography

388

Spat a Enhancements n SQL Server 2012. . . . . . . . . . . . . . . . . . . . . . . . . .

xiv Contents

355

400

New Spat a Data C asses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

401

New Spat a Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

405

Other Enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

411

Part III

Integrat ng w th M crosoft B ng Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

413

Summary

423

Applied SQL

Chapter 10 The Microsoft Data Access Juggernaut

427

NET Data Access Evo ut on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

427

Prepar ng the Samp e Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

430

Mon tor ng Database Act v ty w th SQL Server Profi er. . . . . . . . . . . . . . .

435

Convent ona ADO NET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

436

Us ng the Raw Data Access Objects. . . . . . . . . . . . . . . . . . . . . . . . . .

436

Work ng w th DataSets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Language-Integrated Query (LINQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LINQ to DataSet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Object Re at ona Mode ng (ORM) Comes to NET. . . . . . . . . . . . . . . . . . Mu t p e ORM Offer ngs from Redmond. . . . . . . . . . . . . . . . . . . . . .

472 473 477 479

LINQ to SQL Then and Now. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

479

Ent ty Framework Now and n the Future. . . . . . . . . . . . . . . . . . . . .

482

Summary

Chapter 11 WCF Data Access Technologies

508

509

Defin ng Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

509

WCF Data Access Opt ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

510

WCF Data Serv ces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

511

Bu d ng a WCF Data Serv ce. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

512

Creat ng the Ent ty Data Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

513

Test ng WCF Data Serv ces w th Internet Exp orer. . . . . . . . . . . . . .

515

Bu d ng C ent App cat ons for WCF Data Serv ces . . . . . . . . . . . .

518

Extend ng WCF Data Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

544

WCF RIA Serv ces

548

Estab sh ng a WCF RIA Serv ces L nk. . . . . . . . . . . . . . . . . . . . . . . . .

549

Creat ng the Ent ty Data Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

551

Bu d ng the Doma n Serv ce and Metadata C asses. . . . . . . . . . . .

552

Bu d ng the S ver ght C ent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

561

Contents xv

Inspect ng the NET Fram ng Protoco w th F dd er. . . . . . . . . . . . .

569

Test ng the Comp ete WCF RIA Serv ces So ut on . . . . . . . . . . . . . .

569

Mak ng the R ght WCF Data Access Cho ce. . . . . . . . . . . . . . . . . . . . . . . . .

577

Summary

578

Chapter 12 Moving to the Cloud with SQL Azure H story. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

581

But What Is SQL Azure?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

581

Why the L m tat ons?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

582

Pr c ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

583

The F rst One’s Free. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

583

Gett ng Set Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

584

Beyond the Prerequ s tes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

585

Prov s on ng Your Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

586

Prov s on ng Your Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

589

Manag ng Your Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

589

Creat ng Tab es and Enter ng Data. . . . . . . . . . . . . . . . . . . . . . . . . . .

590

Query ng n the Browser. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

592

Index Des gn. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

592

Management and V sua zat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

593

Connect ng from Down Be ow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

596

M grat ng and Sync ng Between Earth and C oud. . . . . . . . . . . . . . . . . . .

xvi Contents

579

599

DACPACs to the Rescue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

600

Extract, Dep oy, Export, and Import DAC fi es. . . . . . . . . . . . . . . . .

600

Scenar os. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

602

SQL Azure Federat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

607

A SQL Azure Federat ons Lex con. . . . . . . . . . . . . . . . . . . . . . . . . . . .

607

Creat ng a Federat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

608

Federated Tab es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

609

Us ng a Federat on Member . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

610

Sp tt ng and Dropp ng Federat on Members. . . . . . . . . . . . . . . . . .

610

Centra Tab es and Reference Tab es . . . . . . . . . . . . . . . . . . . . . . . . .

610

Fan-Out Quer es and Mu t -Tenancy . . . . . . . . . . . . . . . . . . . . . . . . .

611

Federat ons Support n SSMS and SSDT. . . . . . . . . . . . . . . . . . . . . . .

611

Federat ons Make Sense n the C oud . . . . . . . . . . . . . . . . . . . . . . . .

612

SQL Azure Report ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

612

Prov s on ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

613

Report Author ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

614

Dep oy ng Reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

615

Gett ng Your Bear ngs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

617

Summary

Chapter 13 SQL Azure Data Sync and Windows Phone 7 Development Character st cs of an Occas ona y Connected System. . . . . . . . . . . . . . . . Data Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gett ng to Know SQL Azure Data Sync. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

617

619 620 620 621

Capab t es and Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

621

Data Sync Term no ogy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

622

Sync Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

623

The C ent Sync Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

624

SQL Azure Data Sync Cons derat ons. . . . . . . . . . . . . . . . . . . . . . . . .

625

Creat ng an Occas ona y Connected System. . . . . . . . . . . . . . . . . . . . . . . .

626

Prerequ s tes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

629

Configur ng SQL Azure Data Sync. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

630

Prov s on ng the SQL Azure Data Sync Server . . . . . . . . . . . . . . . . .

630

Creat ng the Sync Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

631

Host ng WCF Data Serv ces n W ndows Azure. . . . . . . . . . . . . . . . . . . . . . About W ndows Azure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

641 641

Creat ng the F xPo So ut on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

642

Add ng the F xPo Data Serv ce. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

643

Add ng the Ent ty Data Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

644

Creat ng the F xPo C ent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

647

Consum ng OData on W ndows Phone . . . . . . . . . . . . . . . . . . . . . . . . . . . .

662

SQL Server on the Phone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

666

Dep oy ng to W ndows Azure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

672

Summary

674

Contents xvii

Chapter 14 Pervasive Insight The M crosoft BI Stack What’s It A About?. . . . . . . . . . . . . . . . . . . . . . . .

676

Master Data Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

677

Data Qua ty Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

680

Integrat on Serv ces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

681

SQL Server RDBMS, Fast Track DW, and SQL Server PDW. . . . . . . . . . . . .

683

Data Marts and Data Warehouses . . . . . . . . . . . . . . . . . . . . . . . . . . .

683

The Star Schema. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

684

SQL Server Data Warehouse App ances. . . . . . . . . . . . . . . . . . . . . .

684

Ana ys s Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

686

The Mu t d mens ona Eng ne . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

686

PowerP vot and SSAS Tabu ar Mode. . . . . . . . . . . . . . . . . . . . . . . . . .

687

Data M n ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

690

Power V ew. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

691

Report ng Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

692

Report Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

693

A ert ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

693

Dashboard Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

694

Exce and Exce Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Us ng Exce Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

694 694

PerformancePo nt Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

696

StreamIns ght. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

697

SQL Server Ed t ons and SharePo nt Vers on Requ rements. . . . . . . . . . .

697

Summary

699

Chapter 15 xVelocity In-Memory Technologies Co umn Store Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Co umn Store Tech n the BI Industry. . . . . . . . . . . . . . . . . . . . . . . . . xVe oc ty n the RDBMS Co umnstore Indexes. . . . . . . . . . . . . . . . . . . . . .

xviii Contents

675

701 702 703 704

Bu d ng a Co umnstore Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

704

What You Can’t Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

704

How Co umnstore Indexes Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

706

xVe oc ty for Ana ys s PowerP vot and SSAS Tabu ar Mode s. . . . . . . . . .

709

C ear ng Up the Ana ys s Serv ces Vocabu ary . . . . . . . . . . . . . . . . .

710

The Lowdown on BISM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

711

Fr ends, Countrymen, Br ng Me Your Data. . . . . . . . . . . . . . . . . . . .

711

Bu d ng the BISM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

712

D a M for Mode ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

715

Mode ng, Part Deux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

718

Query ng n Exce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

724

PowerP vot for SharePo nt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

726

Mov ng to SSAS Tabu ar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

727

Power V ew Here We Come. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

732

We come Back to Vert Paq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

734

Summary

735

Index 737 About the Authors

773

What do you think of this book? We want to hear from you! M crosoft s nterested n hear ng your feedback so we can cont nua y mprove our books and earn ng resources for you. To part c pate n a br ef on ne survey, p ease v s t:

microsoft.com/learning/booksurvey

Contents xix

Introduction —Leonard Lobel

W

e come! Th s s a book about M crosoft SQL Server 2012 wr tten just for you, the ­deve oper Whether you are programm ng aga nst SQL Server d rect y at the ­database eve or further up the stack us ng M crosoft NET, th s book shows you the way The atest re ease of M crosoft’s flagsh p database product de vers an ­unprecedented, h gh y sca ab e data p atform capab e of hand ng the most demand ng tasks and ­work oads As w th every re ease, SQL Server 2012 adds many new features and ­enhancements for deve opers, adm n strators, and ( ncreas ng y) end users a ke Co ect ve y, these ­product enhancements re nforce—and advance—SQL Server’s pos t on as a ­prom nent contender n the ndustry As the product cont nues to evo ve, ts stack of offer ngs ­cont nues to expand And as the comp ete SQL Server stack s too arge for any one book to cover effect ve y, our emphas s n this book s on programmability Spec fica y, we exp ore the p ethora of ways n wh ch SQL Server (and ts c oud cous n, M crosoft SQL Azure) can be programmed for bu d ng custom app cat ons and serv ces

How Significant Is the SQL Server 2012 Release? SQL Server, part cu ar y ts re at ona database eng ne, matured qu te some t me ago So the “s gn ficance” of every new re ease over recent years can be v ewed— n some ways—as re at ve y nom na The ast watershed re ease of the product was actua y SQL Server 2005, wh ch was when the re at ona eng ne (that, for years, defined SQL Server) stopped occupy ng “center stage,” and nstead took ts pos t on a ongs de a set of ­serv ces that today, co ect ve y, define the product These nc ude the ­Bus ness ­Inte gence (BI) components Report ng Serv ces, Ana ys s Serv ces, and Integrat on Serv ces—features that began appear ng as ear y as 1999 but, pr or to SQL Server 2005, were ntegrated sporad ca y as a patchwork of oose y coup ed add-ons, ­w zards, and management conso es SQL Server 2005 changed a that w th a comp ete ­overhau For the first t me, the overa SQL Server product de vered a broader, r cher, and more conso dated set of features and serv ces wh ch are bu t nto—rather than bo ted onto—the p atform None of the product vers ons that have been re eased s nce that t me—SQL Server 2008, 2008 R2, and now 2012—have changed under y ng ­arch tecture th s rad ca y That sa d, each SQL Server re ease cont nues to advance tse f n v ta y s gn ficant ways SQL Server 2008 (re eased August 6, 2008) added a host of new features to the

xxi

re at ona eng ne—T-SQL enhancements, Change Data Capture (CDC), Transparent Data Encrypt on (TDE), SQL Aud t, FILESTREAM—p us powerfu BI capab t es w th Exce P votTab es, charts, and CUBE formu as SQL Server 2008 R2 (re eased Apr 21, 2010), nterna y dubbed the “BI Refresh” wh e n deve opment, added a revamped vers on of Report ng Serv ces as we as PowerP vot for Exce and SharePo nt, Master Data Serv ces, and StreamIns ght, but offered tt e more than m nor tweaks and fixes to the re at ona eng ne The newest re ease—SQL Server 2012—offic a y aunched on March 7, 2012 L ke ­every new re ease, th s vers on mproves on a of the key “ab t es” (ava ab ty, ­sca ab ty, manageab ty, programmab ty, and so on) Among the ch ef re ab ty mprovements s the new H gh Ava ab ty D saster Recovery (HADR) a ternat ve to database m rror ng HADR (a so common y known as “A ways On”) ut zes mu t p e secondary servers n an “ava ab ty group” for sca e-out read-on y operat ons (rather than forc ng them to s t d e, just wa t ng for a fa over to occur) Mu t subnet fa over c uster ng s another notab e new manageab ty feature SQL Server 2012 adds many new features to the re at ona eng ne, most of wh ch are covered n th s book There are powerfu T-SQL extens ons, most notab y the ­w ndow ng enhancements, p us 22 new T-SQL funct ons, mproved error hand ng, server-s de pag ng, sequence generators, r ch metadata d scovery techn ques, and conta ned databases There are a so remarkab e mprovements for unstructured data, such as the ­F eTab e abstract on over FILESTREAM and the W ndows fi e system API, fu -text p ­ roperty search ng, and Stat st ca Semant c Search Spat a support gets a b g boost as we , w th support for c rcu ar data, fu -g obe support, ncreased performance, and greater par ty between the geometry and geography data types And new “­co umnstore” techno ogy drast ca y ncreases performance of extreme y arge cubes (xVe oc ty for PowerP vot and Ana ys s Serv ces) and data warehouses (us ng an xVe oc ty- ke mp ementat on n the re at ona eng ne) The aforement oned re at ona eng ne features are mpress ve, but st amount to tt e more than “­ add t ves” over an a ready estab shed database p atform A new re ease needs more than just extra c ng on the cake for customers to perce ve an upgrade as ­compe ng To that end, M crosoft has nvested heav y n BI w th SQL Server 2012, and the ­effort shows The BI port on of the stack has been expanded great y, de ver ng key ­advances n “pervas ve ns ght ” Th s nc udes major updates to the product’s ­ana yt cs, data ­v sua zat on (such as se f-serv ce report ng w th Power V ew), and master data ­management capab t es, as we Data Qua ty Serv ces (DQS), a brand new data qua ty eng ne There s a so a new Bus ness Inte gence ed t on of the product that nc udes a of these capab t es w thout requ r ng a fu Enterpr se ed t on cense F na y, SQL Server Data Too s (SSDT) br ngs brand new database too ng ns de V sua Stud o SSDT xxii  Introduction

prov des a dec arat ve, mode -based des gn-t me exper ence for deve op ng databases wh e connected, offl ne, on-prem se, or n the c oud

Who Should Read This Book Th s book s ntended for deve opers who have a bas c know edge of re at ona ­database terms and pr nc p es

Assumptions In ta or ng the content of th s book, there are a few assumpt ons that we make about you F rst, we expect that you are a deve oper who s a ready know edgeab e about ­re at ona database concepts—whether that exper ence s w th SQL Server or ­non-M crosoft p atforms As such, you a ready know about tab es, v ews, pr mary and fore gn keys (re at onsh ps), stored procedures, user-defined funct ons, and tr ggers These essent a s are assumed know edge and are not covered n th s book S m ar y, we don’t exp a n proper re at ona des gn, ru es of data norma zat on, strateg c ­ndex ng pract ces, how to express bas c quer es, and other re at ona fundamenta s We a so ­assume that you have at east bas c fam ar ty w th SQL statement syntax—aga n, e ther T-SQL n SQL Server or SQL d a ects n other p atforms—and have a bas c work ng know edge of NET programm ng n C# on the c ent Hav ng sa d a that, we have a fa r y bera po cy regard ng these prerequ s tes For examp e, f you’ve on y dabb ed w th T-SQL or you’re more comfortab e w th M crosoft V sua Bas c NET than C#, that’s okay, as ong as you’re w ng to try and p ck up on th ngs as you read a ong Most of our code samp es are not that comp ex However, our exp anat ons assume some bas c know edge on your part, and you m ght need to do a tt e research f you ack the exper ence

Note  For the sake of consistency, all the .NET code in this book is written in C#. However, this book is in no way C#-oriented, and there is certainly ­nothing C#-specific in the .NET code provided. As we just stated, the code samples are not very complex, and if you are more experienced with Visual Basic .NET than you are with C#, you should have no trouble translating the C# code to Visual Basic .NET on the fly as you read it. W th that base ne estab shed, our approach has been to add va ue to the SQL Server documentat on by prov d ng a deve oper-or ented nvest gat on of ts features,



Introduction  xxiii

espec a y the new and mproved features n SQL Server 2012 We start w th the brand new database too ng, and the many r ch extens ons made to T-SQL and the ­re at ona database eng ne Then we move on to w der spaces, such as nat ve fi e stream ng, ­geospat a data, and other types of unstructured data We a so have chapters on ­secur ty, transact ons, c ent data access, secur ty, mob e/c oud deve opment, and more W th n these chapters, you w find deta ed coverage of the atest and most ­mportant SQL Server programm ng features You w atta n pract ca know edge and techn ca understand ng across the product’s numerous programmab ty po nts, empower ng you to deve op the most soph st cated database so ut ons for your end users Converse y, th s s not ntended as a resource for system adm n strators, database adm n strators, project managers, or end users Our genera ru e of thumb s that we don’t d scuss features that are not part cu ar y programmab e

Who Should Not Read This Book Th s book s not ntended for SQL Server adm n strators; t s a med square y at deve opers—and on y deve opers who have mastery of bas c database concepts

Organization of This Book The chapters of th s book are organ zed n three sect ons ■

Core SQL Server features

■

Beyond re at ona features

■

App ed SQL for bu d ng app cat ons and serv ces

By no means does th s book need to be read n any part cu ar order Read t from start to fin sh f you want, or jump r ght n to just those chapters that su t your needs or p que your nterests E ther way, you’ find the pract ca gu dance you need to get your job done The fo ow ng overv ew prov des a summary of these sect ons and the r chapters After the overv ew, you w find nformat on about the book’s compan on webs te, from wh ch you can down oad code samp es and work hands-on w th a the examp es n the book

xxiv  Introduction

Core SQL Server Development In Part I, we focus on core SQL Server features These nc ude brand new too ng (SSDT), enhancements to T-SQL, extended programmab ty w th SQL CLR code n NET ­anguages such as M crosoft V sua Bas c NET and C#, transact ons, and secur ty ■

Chapter 1  Introduc ng SQL Server Data Too s Our open ng chapter s a about SQL Server Data Too s (SSDT) W th the re ease of SQL Server 2012, SSDT now serves as your pr mary deve opment env ronment for bu d ng SQL Server app cat ons Wh e SQL Server Management Stud o (SSMS) cont nues to serve as the pr mary too for database adm n strators, SSDT represents a brand new deve oper exper ence SSDT p ugs n to ­M crosoft V sua Stud o for connected deve opment of on-prem se databases or SQL Azure ­databases runn ng n the c oud, as we as a new database project type for ­offl ne deve opment and dep oyment Us ng pract ca , rea -wor d scenar os, you w a so earn how to everage SSDT features such as code nav gat on, ­Inte Sense, refactor ng, schema compare, and more

■

Chapter 2  T-SQL Enhancements In Chapter 2, we exp ore the s gn ficant enhancements made to Transact-SQL ­( T-SQL)—wh ch st rema ns the best programm ng too for custom SQL Server deve opment We cover severa powerfu extens ons to T-SQL added n SQL Server 2008, beg nn ng w th tab e-va ued parameters (TVPs) You earn how to pass ent re sets of rows around between stored procedures and funct ons on the server, as we as between c ent and server us ng M crosoft ADO NET Date and t me features are exp ored next, nc ud ng separate date and t me data types, t me zone awareness, and mprovements n date and t me range, storage, and prec s on We then show many ways to use MERGE, a flex b e data ­man pu at on anguage (DML) statement that encapsu ates a the nd v dua ­operat ons typ ca y nvo ved n any merge scenar o From there, you earn about INSERT OVER DML for enhanced change data capture from the OUTPUT c ause of any DML statement We a so exam ne GROUPING SETS, an extens on to the trad t ona GROUP BY c ause that ncreases your opt ons for s c ng and d c ng data n aggregat on quer es We then d ve n to the new T-SQL enhancements ntroduced n SQL Server 2012, start ng w th windowing features The first w ndow ng funct ons to a ­ ppear n T-SQL date back to SQL Server 2005, w th the ntroduct on of ­severa ­rank ng funct ons W ndow ng capab t es have been qu te m ted ever s nce, but SQL Server 2012 fina y de vers some ­major ­mprovements to change a that F rst



Introduction  xxv

you w grasp ­w ndow ng c­ oncepts and the pr nc p es beh nd the OVER c ause, and then everage that ­know edge to ­ca cu ate runn ng and s d ng aggregates and perform other ana yt c ­ca cu at ons You w earn about every one of the 22 new T-SQL funct ons, ­nc ud ng 8 ana yt c w ndow ng funct ons, 3 convers on funct ons, 7 date and t me re ated funct ons, 2 og ca funct ons, and 2 str ng funct ons We a so ­exam ne mproved error hand ng w th THROW, server-s de pag ng w th ­OFFSET/FETCH NEXT, sequence generators, and r ch metadata d scovery ­techn ques We exp a n a of these new funct ons and features, and prov de c ear code samp es demonstrat ng the r use ■

Chapter 3  Exp or ng SQL CLR Chapter 3 prov des thorough coverage of SQL CLR programm ng—wh ch ets you run comp ed NET code on SQL Server—as we as gu dance on when and where you shou d put t to use We go beyond mere stored procedures, ­tr ggers, and funct ons to exp a n and demonstrate the creat on of CLR types and ­aggregates—ent t es that cannot be created at all n T-SQL We a so cover the d fferent methods of creat ng SQL CLR objects n SQL Server Database Projects n V sua Stud o and how to manage the r dep oyment, both from SSDT/V sua Stud o and from T-SQL scr pts n SQL Server Management Stud o and e sewhere

■

Chapter 4  Work ng w th Transact ons No matter how you wr te and package your code, you must keep your data cons stent to ensure ts ntegr ty The key to cons stency s transact ons, wh ch we cover n Chapter 4 Transact ons can be managed from a var ety of p aces, ke many SQL Server programmab ty features If you are wr t ng T-SQL code or c ent code us ng the ADO NET SqlClient prov der or System.Transactions, you need to be aware of the var ous transact on so at on eve s supported by SQL Server, the appropr ate scope of your transact ons, and best pract ces for wr t ng transact ona code Th s chapter gets you there

■

Chapter 5  SQL Server Secur ty Chapter 5 d scusses SQL Server secur ty at ength and exam nes your cho ces for keep ng data safe and secure from pry ng eyes and ma c ous ntent We beg n w th the bas c secur ty concepts concern ng og ns, users, ro es, ­authent cat on, and author zat on You then go on to earn about key-based encrypt on ­support, wh ch protects your data both wh e n trans t and at rest We then ­exam ne other powerfu secur ty features, nc ud ng Transparent Data ­Encrypt on (TDE) and SQL Server Aud t TDE a ows you to encrypt ent re databases n the background w thout spec a cod ng requ rements W th SQL

xxvi  Introduction

Server Aud t, v rtua y any act on taken by any user can be recorded for aud t ng n e ther the fi e system or the W ndows event og We a so show how to create contained databases, a new feature n SQL Server 2012 that e m nates host nstance dependenc es by stor ng og n credent a s d rect y n the database The chapter conc udes by prov d ng cruc a gu dance for adher ng to best pract ces and avo d ng common secur ty p tfa s

Going Beyond Relational W th the re ease of SQL Server 2012, M crosoft broadens support for sem -­structured and unstructured data n the re at ona database In Part II, we show how to ­everage the “beyond re at ona ” capab t es n SQL Server 2012—features that are ­becom ng ncreas ng y cr t ca n today’s wor d of b nary pro ferat on, and the ­emergence of h ghperformance so-ca ed “No SQL” database p atforms ■

Chapter 6  XML and the Re at ona Database SQL Server 2005 ntroduced the xml data type, and a ot of r ch XML support to go a ong w th t That nnovat on was an mmeasurab e mprovement over the use of p a n varchar or text co umns to ho d str ngs of XML (wh ch was common n ear er vers ons of SQL Server), and thus revo ut on zed the storage of XML n the re at ona database It empowers the deve opment of database ­app cat ons that work w th h erarch ca data natively—w th n the env ronment of the re at ona database system—someth ng not poss b e us ng ord nary str ng co umns In Chapter 6, we take a deep d ve nto the xml data type, XQuery extens ons to T-SQL, server-s de XML Schema Defin t on (XSD) co ect ons, XML co umn ndexng, and many more XML features

■

Chapter 7  H erarch ca Data and the Re at ona Database But XML s not your on y opt on for work ng w th h erarch ca data n the ­database In Chapter 7, we exp ore the hierarchyid data type that enab es you to cast a h erarch ca structure over any re at ona tab e Th s data type s ­mp emented as a “system CLR” type, wh ch s noth ng more rea y than a SQL CLR user-defined type (UDT), just ke the ones we show how to create on your own n Chapter 3 The va ue stored n a hierarchyid data type encodes the ­comp ete path of any g ven node n the tree structure, from the root down to the spec fic ord na pos t on among other s b ng nodes shar ng the same ­parent Us ng methods prov ded by th s new type, you can now effic ent y bu d, query, and man pu ate tree-structured data n your re at ona tab es Th s data type a so p ays an mportant ro e n SQL Server’s new F eTab e feature, as we exp a n n the next chapter on nat ve fi e stream ng



Introduction  xxvii

■

Chapter 8  Nat ve F e Stream ng In Chapter 8, you earn a about the FILESTREAM, an nnovat ve feature that ntegrates the re at ona database eng ne w th the NTFS fi e system to prov de h gh y effic ent storage and management of arge b nary objects (BLOBs)—­mages, v deos, documents, you name t Before FILESTREAM, you had to choose between stor ng BLOB data n the database us ng varbinary(max) (or the ­now-deprecated image) co umns, or outs de the database as unstructured b nary streams (typ ca y, as fi es n the fi e system) FILESTREAM prov des a powerfu abstract on ayer that ets you treat BLOB data og ca y as an ntegra part of the database, wh e SQL Server stores the BLOB data phys ca y separate from the database n the NTFS fi e system beh nd the scenes You w earn everyth ng you need to program aga nst FILESTREAM, us ng both T-SQL and the h gh-performance SqlFileStream NET c ass The wa kthroughs n th s chapter bu d W ndows, web, and W ndows Presentat on ­Foundat on (WPF) app cat ons that use FILESTREAM for BLOB data storage You w a so see how F eTab e, a new feature n SQL Server 2012, bu ds on FILESTREAM F eTab e comb nes FILESTREAM w th the hierarchyid (covered n Chapter 7) and the W ndows fi e system API, tak ng database BLOB ­management to new eve s As mp ed by the two words jo ned together n ts name, one F eTab e funct ons as two d st nct th ngs s mu taneous y a tab e and a fi e system—and you w earn how to exp o t th s new capab ty from both ang es

■

Chapter 9  Geospat a Support Chapter 9 exp ores the wor d of geospat a concepts and the r ch spat a s­ upport prov ded by the geometry and geography data types W th these system CLR types, t s very easy to ntegrate ocat on-awareness nto your ­app cat ons— at the database eve Respect ve y, geometry and geography enab e spat a ­deve opment aga nst the two bas c geospat a surface mode s p anar (flat) and geodet c ­(round-earth) W th spat a data (represented by geometr c or ­geograph c ­coord nates) stored n these data types, you can determ ne ­ntersect ons and ­ca cu ate ength, area, and d stance measurements aga nst that data The chapter first qu ck y covers the bas cs and then prov des wa kthroughs n wh ch you bu d severa geospat a database app cat ons, nc ud ng one that ntegrates mapp ng w th M crosoft B ng Maps We a so exam ne the s gn ficant spat a enhancements added n SQL Server 2012 A though ent re books have been wr tten on th s vast and ever-expand ng top c, our chapter de ves nto ­suffic ent depth so you can get busy work ng w th geospat a data r ght away

xxviii  Introduction

Applied SQL After we’ve covered so much nformat on about what you can do on the server and n the database, we move to Part III of the book, where we exp ore techno og es and ­demonstrate techn ques for bu d ng c ent/server, n-t er, and c oud so ut ons on top of your databases Whatever your scenar o, these chapters show you the most effect ve ways to extend your data’s reach We then conc ude w th coverage of SQL Azure, the BI stack, and the new co umnstore techno ogy known as xVe oc ty ■

Chapter 10  The M crosoft Data Access Juggernaut Chapter 10 covers every c ent/server data access strategy ava ab e n the NET Framework today We beg n w th ear est M crosoft ADO NET techn ques us ng raw data access objects and the DataSet abstract on, and d scuss the ­ongo ng re evance of these NET 1 0 techno og es We then exam ne ater data ­access techno og es, nc ud ng the concepts and syntax of anguage- ntegrated query (LINQ) We ook at LINQ to DataSet and LINQ to SQL, and then turn our focus heav y on the ADO NET Ent ty Framework (EF), M crosoft’s current ­recommended data access so ut on for NET You w earn Object Re at ona Mapp ng (ORM) concepts, and d scover how EF’s Ent ty Data Mode (EDM) prov des a powerfu abstract on ayer to dramat ca y stream ne the app cat on deve opment process

■

Chapter 11  WCF Data Access Techno og es After you have mastered the c ent/server techn ques taught n Chapter 10, you are ready to expose your data as serv ces to the wor d Chapter 11 prov des you w th deta ed exp anat ons and code samp es to get the job done us ng two techno og es based on W ndows Commun cat ons Foundat on (WCF) The first part of Chapter 11 covers WCF Data Serv ces, wh ch everages ­Representat ona State Transfer Protoco (REST) and Open Data Protoco (OData) to mp ement serv ces over your data source After exp a n ng these key ­concepts, you w see them put to pract ca use w th concrete examp es As you mon tor background network and database act v ty, we zone n and ock down on the cr t ca nterna s that make t a work The second part of the chapter demonstrates data access us ng WCF RIA Serv ces, a ater techno ogy that t­ argets S ver ght c ents n part cu ar (a though t can support other c ents as we ) We art cu ate the s m ar t es and d fferences between these two ­WCF-based techno og es, and arm you w th the know edge of how and when to use each one



Introduction  xxix

■

Chapter 12  Mov ng to the C oud w th SQL Azure In Chapter 12, we ook at the wor d of c oud database comput ng w th SQL Azure We exp a n what SQL Azure s a about, how t s s m ar to SQL Server and how t d ffers We ook at how SQL Azure s pr ced, how to s gn up for t, and how to prov s on SQL Azure servers and databases We exam ne the SQL Azure too ng and how to work w th SQL Azure from SSMS and SSDT We ­exp a n the many ways that Data-T er App cat ons (DACs) can be used to m grate databases between SQL Server and SQL Azure, us ng SSMS, SSDT, and the nat ve too ng of SQL Azure as we We fin sh up the chapter by exam n ng a spec a part t on ng feature ca ed SQL Azure Federat ons and we ook at SQL Azure Report ng, too

■

Chapter 13  SQL Azure Data Sync and W ndows Phone Deve opment Chapter 13 covers the broad top c of so-ca ed occas ona y connected ­systems by bu d ng out a comp ete so ut on that ncorporates SQL Azure Data Sync, W ndows Azure, and the W ndows Phone 7 deve opment p atform On the back end, an on-prem se SQL Server database s kept synchron zed w th a pub c-fac ng SQL Azure database n the c oud us ng SQL Azure Data Sync The c oud database s exposed us ng WCF Data Serv ces (a so hosted n the c oud by dep oy ng to W ndows Azure), and consumed v a OData by a mob e c ent app cat on runn ng on a W ndows Phone 7 dev ce The end-to-end so ut on deta ed n th s chapter demonstrates how these techno og es work to keep data n sync across on-prem se SQL Server, SQL Azure databases n the c oud, and oca storage on W ndows Phone 7 dev ces

■

Chapter 14  Pervas ve Ins ght In Chapter 14, we prov de an overv ew of the ent re SQL Server BI stack, ­nc ud ng SQL Server Fast Track Data Warehouse app ances, SQL Server ­Para e Data Warehouse ed t on, SQL Server Integrat on Serv ces, Ana ys s Serv ces, Master Data Serv ces, Data Qua ty Serv ces, Report ng Serv ces, Power V ew, PowerP vot, and StreamIns ght In the nterest of comp eteness, we a so prov de br ef overv ews of Exce Serv ces and PerformancePo nt Serv ces n SharePo nt and how they comp ement SQL Server We exp a n what each BI component does, and how they work together Perhaps most mportant, we show you how these techno og es from the BI arena are re evant to your work w th re at ona data, and how, n that ght, they can be qu te approachab e These techno og es shou dn’t be thought of as segregated or tangent a They are ntegra parts of SQL Server, and we seek to make them part of what you do w th the product

xxx  Introduction

■

Chapter 15  xVe oc ty In-Memory Techno og es In Chapter 15, we ook at M crosoft’s xVe oc ty co umn store techno ogy, and how to use t from the SQL Server re at ona database, as we as PowerP vot and Ana ys s Serv ces We exp a n how co umn-or ented databases work, we exam ne the new co umnstore ndexes n SQL Server 2012, and d scuss ts batch process ng mode, too We ook at how easy t s for re at ona database experts to work w th PowerP vot and SSAS Tabu ar mode, and we show how to br ng a these techno og es together w th the SQL Server Power V ew data ana ys s, d scovery, and v sua zat on too

Conventions and Features in This Book Th s book presents nformat on us ng convent ons des gned to make the nformat on readab e and easy to fo ow ■

Boxed e ements w th abe s such as “Note” prov de add t ona nformat on or a ternat ve methods for comp et ng a step successfu y

■

Text that you type (apart from code b ocks) appears n bo d

■

Code e ements n text (apart from code b ocks) appear n ta c

■

■

A p us s gn (+) between two key names means that you must press those keys at the same t me For examp e, “Press A t+Tab” means that you ho d down the A t key wh e you press the Tab key A vert ca bar between two or more menu tems (for examp e, F e C ose), means that you shou d se ect the first menu or menu tem, then the next, and so on

System Requirements To fo ow a ong w th the book’s text and run ts code samp es successfu y, we ­recommend that you nsta the Deve oper ed t on of SQL Server 2012, wh ch s ­ava ab e to a great number of deve opers through M crosoft’s MSDN Prem um subscr pt on, on your PC You w a so need V sua Stud o 2010; we recommend that you use the ­Profess ona ed t on or one of the Team ed t on re eases, each of wh ch s a so ava ab e w th the correspond ng ed t on of the MSDN Prem um subscr pt on product A the code samp es w a so work w th the upcom ng V sua Stud o 11, n beta at the t me of th s wr t ng



Introduction  xxxi

Important  To cover the widest range of features, this book is based on the Developer edition of SQL Server 2012. The Developer edition possesses the same feature set as the Enterprise edition of the product, although Developer edition licensing terms preclude production use. Both editions are high-end platforms that offer a superset of the features available in other editions (Standard, Workgroup, Web, and Express). We believe that it is in the best ­interest of developers for us to cover the full range of developer features in SQL Server 2012, including those available only in the Enterprise and Developer editions. To run these ed t ons of SQL Server and V sua Stud o, and thus the samp es n th s book, you’ need the fo ow ng 32-b t hardware and software (The 64-b t hardware and software requ rements are not sted here but are very s m ar ) ■

1 GHz or faster (2 GHz recommended) processor

■

Operat ng system, any of the fo ow ng

• M crosoft W ndows Server 2008 R2 SP1 • W ndows 7 SP1 (32- or 64-b t) • W ndows Server 2008 SP2 • W ndows V sta SP2 ■

■

■

■

xxxii  Introduction

For SQL Server 2012, 4 GB or more RAM recommended for a ed t ons (except the Express ed t on, wh ch requ res on y 1 GB) For SQL Server 2012, approx mate y 1460 MB of ava ab e hard d sk space for the recommended nsta at on Approx mate y 375 MB of add t ona ava ab e hard d sk space for SQL Server Books On ne, SQL Server Mob e Everywhere Books On ne, and samp e databases For V sua Stud o 2010, max mum of 20 GB ava ab e space requ red on ­nsta at on dr ve Note that th s figure nc udes space for nsta ng the fu set of MSDN documentat on A work ng Internet connect on (requ red to down oad the code samp es from the compan on webs te) A few of the code samp es a so requ re an Internet ­connect on to run

■

Super VGA (1024 × 768) or h gher reso ut on v deo adapter and mon tor ­recommended

■

M crosoft Mouse or compat b e po nt ng dev ce recommended

■

M crosoft Internet Exp orer 9 0 or ater recommended

Installing SQL Server Data Tools SSDT does not get nsta ed w th e ther V sua Stud o or SQL Server Instead, SSDT sh ps separate y v a the Web P atform Insta er (WebPI) Th s enab es M crosoft to d str bute t me y SSDT updates out-of-band w th (that s, w thout wa t ng for major re eases of) V sua Stud o or SQL Server Before you fo ow a ong w th the procedures n Chapter 1, down oad and nsta SSDT from http://msdn.microsoft.com/en-us/data/hh297027

Using the Book’s Companion Website V s t the book’s compan on webs te at the fo ow ng address http://go.microsoft.com/FWLink/?LinkId=252994

Code Samples A the code samp es d scussed n th s book can be down oaded from the book’s ­compan on webs te W th n the compan on mater a s parent fo der on the s te s a ch d fo der for each chapter Each ch d fo der, n turn, conta ns the samp e code for the chapter Because most of the code s exp a ned n the text, you m ght prefer to create t from scratch rather than open the fin shed vers on supp ed n the compan on samp e code ­However, the fin shed vers on w st prove usefu f you make a sma error a ong the way or f you want to run the code qu ck y before read ng through the narrat ve that descr bes t

Sample AdventureWorks Databases As of SQL Server 2005, and updated through SQL Server 2012, M crosoft prov des the popu ar AdventureWorks fam y of samp e databases Severa chapters n th s book reference the AdventureWorks2012 on ne transact on process ng (OLTP) database, and Chapter 15 references the AdventureWorksDW2012 data warehous ng database



Introduction  xxxiii

To fo ow a ong w th the procedures n these chapters, you can down oad these databases d rect y from the book’s compan on webs te The databases posted there are the exact vers ons that th s book was wr tten aga nst, or g na y obta ned from ­CodeP ex, wh ch s M crosoft’s open source webs te ( n fact, a of M crosoft’s offic a product code samp es are hosted on CodeP ex) To ensure you rece ve the same resu ts as you fo ow a ong w th certa n chapters n th s book, we recommend down oad ng the AdventureWorks2012 OLTP and AdventureWorksDW2012 data warehous ng databases from the book’s compan on webs te rather than from CodeP ex (where updated ­vers ons may cause d fferent resu ts than the or g na vers ons) You can find the d rect ons to attach (use) the samp e databases on the samp e ­database down oad page

Previous Edition Chapters In add t on to a the code samp es, the book’s compan on webs te a so conta ns severa chapters from the 2008 and 2005 ed t ons of th s book that were not updated for th s ed t on n order to accommodate coverage of new SQL Server 2012 features You can down oad SQL Server 2005 chapters that cover Serv ce Broker, nat ve XML Web Serv ces, SQL Server Management Stud o, SQL Server Express ed t on, Integrat on Serv ces, and debugg ng, as we as SQL Server 2008 chapters on data warehous ng, on ne ana yt ca process ng (OLAP), data m n ng, and Report ng Serv ces

Errata & Book Support We’ve made every effort to ensure the accuracy of th s book and ts compan on ­content Any errors that have been reported s nce th s book was pub shed are sted on our M crosoft Press s te at oreilly.com http://go.microsoft.com/FWLink/?LinkId=252995 If you find an error that s not a ready sted, you can report t to us through the same page If you need add t ona support, ema M crosoft Press Book Support at [email protected] P ease note that product support for M crosoft software s not offered through the addresses above

xxxiv  Introduction

We Want to Hear from You At M crosoft Press, your sat sfact on s our top pr or ty, and your feedback our most va uab e asset P ease te us what you th nk of th s book at http://www.microsoft.com/learning/booksurvey The survey s short, and we read every one of your comments and deas Thanks n advance for your nput!

Stay in Touch Let’s keep the conversat on go ng! We’re on Tw tter http://twitter.com/MicrosoftPress



Introduction  xxxv

Acknowledgements

I

t’s hard to be eve I first began research for th s book at an ear y Software Des gn Rev ew for SQL Server “Dena ” n Redmond back n October 2010 Th s s my second ed t on as ead author of th s book, and a though I enjoyed the work even more th s t me around, t was certa n y no eas er than the 2008 ed t on My goa —upfront—was to produce the most comprehens ve (yet approachab e) SQL Server 2012 deve oper resource that I cou d, one that best answers, “How many ways can I program SQL Server?” I cou d not have even contemp ated pursu ng that goa w thout the a d of numerous other ta ented and car ng nd v dua s—fo ks who deserve spec a recogn t on The r generous support was ent out n many d fferent yet equa y essent a ways So the order of names ment oned be ow s by no means an nd cat on of degree or proport on; s mp y put, I cou dn’t have wr tten th s book w thout everyone’s he p

W th so many peop e to thank, Cra g Brann ng (CEO of Ta an, Inc ) s at the top of my most wanted st Back n m d-2010, Cra g was qu ck to approach me about ­tak ng on th s project Next th ng I knew, I was on board and we were scarfing down unch (smooth work!) Thank you (and a the other wonderfu fo ks at Ta an) for ­gett ng th s book off the ground n the first p ace, and prov d ng a cont nuous source of ­support throughout ts product on I’m a so extreme y fortunate to have teamed up w th my co eague and fr end, ­co-author Andrew Brust (M crosoft MVP/RD) Th s s actua y Andrew’s th rd t me around contr but ng h s know edge and expert se to th s resource; he not on y ­co-authored the 2008 ed t on, but was ead author of the first ed t on for SQL Server 2005 So I thank h m once aga n for wr t ng four ste ar chapters n th s new 2012 ­ed t on And Pau De cog ano (who a so contr buted to the 2008 ed t on) d d a superb job confront ng the top c of end-to-end c oud deve opment w th SQL Azure Data Sync, W ndows Azure, and W ndows Phone 7—a n a s ng e outstand ng chapter Pau , your amb t on s adm rab e, and I thank you for your t re ess work and the great job done! Ken Jones, my pa at O’Re y Med a, gets spec a ment on for h s expert gu dance, p us h s steady pat ence through a the adm n strat ve shenan gans Thank you Ken, and to your ove y w fe, Andrea, as we , for her ns ghtfu he p w th the geospat a ­content I was a so very ucky to have worked c ose y w th Russe Jones,Me an e Yarbrough, John Mue er, and ­Chr st an Ho dener, whose superb ed tor a contr but ons, techn ca rev ew, and overa gu dance were v ta to the successfu product on of th s book The ass stance prov ded by a number of peop e from var ous M crosoft product teams he ped tack e the cha enge of wr t ng about new software as t evo ved through

xxxvii

severa beta re eases Thank you to Roger Doherty, for nv t ng me out to Redmond for the SDR n 2010, as we as for connect ng me w th the r ght peop e I needed to get my job done Gert Drapers and Adam Mahood were part cu ar y he pfu for the ­ns de scoop on SSDT as t changed from one CTP to the next Adam’s a ways d rect and ­a ways ava ab e nes of commun cat on turned an ent re chapter’s hard work nto fun work Doug Laudensch ager a so prov ded va uab e ns ght, wh ch enhanced new ­coverage of unstructured FILESTREAM data And natura y, a great b g thank you to the ent re product team for creat ng the best re ease of SQL Server ever! I’m a so part cu ar y proud of a the brand new NET data access coverage n th s book, and wou d ke to g ve spec a thanks to my pa Marce de Vr es, M crosoft MVP and RD n the Nether ands Marce s a master of d str buted arch tectures, and h s ­nva uab e ass stance great y he ped shape coverage n the WCF data access chapter Ik ben heel dankbaar voor jouw inbreng! Th s book cou d not have been wr tten, of course, w thout the ove and support of my fam y I have been consumed by th s project for much of the past e ghteen months—wh ch has at t mes transformed me nto an absentee I owe an enormous debt of grat tude to my wonderfu partner Mark, and our awesome k ds Adam, ­Jacque ne, Josh, and Sonny, for be ng so pat ent and to erant w th me And greatest thanks of a go out to my dear Mom, b ess her sou , for a ways encourag ng me to wr te w th ­“express on ” —Leonard Lobel

When you’re not a fu -t me author, there’s rea y never a “good” t me to wr te a book It’s a ways an added extra, and t typ ca y takes s gn ficant y more t me than ant c pated That creates burdens for many peop e, nc ud ng the author’s fam y, the book’s ed tors, and co-authors as we When one of the authors s start ng a new bus ness, burdens doub e, a around I’d ke to thank my fam y, the M crosoft Press team and espec a y th s book’s ead author, Lenn Lobe , for endur ng these burdens, w th flex b ty and uncommon y nfin te pat ence —Andrew Brust

xxxviii  Acknowledgements

Par t I

Core SQL Server Development chapter 1

Introduc ng SQL Server Data Too s

chapter 2

T-SQL Enhancements

chapter 3

Exp or ng SQL CLR

125

chapter 4

Work ng w th Transact ons

169

chapter 5

SQL Server Secur ty

207

3 45

1

C hapter 1

Introducing SQL Server Data Tools —Leonard Lobel

W

th the re ease of SQL Server 2012, M crosoft de vers a powerfu , new ntegrated ­deve opment env ronment (IDE) for des gn ng, test ng, and dep oy ng SQL Server databases— oca y, ­offl ne, or n the c oud—a from r ght ns de M crosoft V sua Stud o (both the 2010 vers on and the ­upcom ng V sua Stud o 11, current y n beta at the t me of th s wr t ng) Th s IDE s ca ed SQL Server Data Too s (SSDT), and t represents a major step forward from prev ous y ava ab e too ng—notab y, SQL Server Management Stud o (SSMS) and V sua Stud o Database Profess ona ed t on (DbPro) SSDT s not ntended to be a rep acement for SSMS, but nstead can be v ewed much more as a great y evo ved mp ementat on of DbPro Indeed, SSMS s a ve and we n SQL Server 2012, and t cont nues to serve as the pr mary management too for database administrators who need to ­configure and ma nta n hea thy SQL Server nsta at ons And a though DbPro was a good first step towards offl ne database deve opment, ts re at ve y m ted des gn-t me exper ence has prec uded ts w despread adopt on So for years, programmers have been pr mar y us ng SSMS to conduct ­deve opment tasks (and before SQL Server 2005, they typ ca y used two database ­adm n strator [DBA] too s—SQL Enterpr se Manager and Query Ana yzer) It’s a ways been necessary for ­programmers to use management-or ented too s (such as SSMS) rather than deve oper-focused too s (such as V sua Stud o) as a pr mary database deve opment env ronment when bu d ng database ­app cat ons—unt now The re ease of SSDT prov des a s ng e env ronment hosted n V sua Stud o, w th database ­too ng that spec fica y targets the deve opment process Thus, you can now des gn and bu d your ­databases w thout constant y togg ng between V sua Stud o and other too s In th s chapter, you’ earn how to use SSDT ns de V sua Stud o to s gn ficant y enhance your product v ty as a SQL Server ­deve oper We beg n w th an overv ew of key SSDT concepts and features, and then wa k you through ­demonstrat ons of var ous connected and d sconnected scenar os

3

Introducing SSDT Database Tooling Designed for Developers The nconven ent truth s database deve opment s hard Gett ng everyth ng done correct y s a huge cha enge—proper schema and re at ona des gn, the ntr cac es of Transact-SQL (T-SQL) as a anguage, performance tun ng, and more, are a d fficu t tasks n and of themse ves However, w th respect to the deve opment process—the way n wh ch you create and change a database—there are some part cu ar scenar os that the r ght too ng can mprove great y SSDT de vers that too ng

The SSDT Umbrella of Services and Tools SSDT encompasses more than just the new database too ng covered n th s chapter; t s actua y a packag ng of what was former y the V sua Stud o 2008–based Bus ness Inte gence Deve oper Stud o (BIDS) too SSDT supports the trad t ona BIDS project types for SQL Server Ana ys s Serv ces (SSAS), Report ng Serv ces (SSRS), and Integrat on Serv ces (SSIS), n add t on to the new database too ng So w th SSDT, M crosoft has now brought together a of the SQL Server database deve opment exper ences ns de a s ng e vers on of V sua Stud o Desp te ts broader defin t on, th s chapter uses the term SSDT spec fica y as t perta ns to the new database deve opment too s that SSDT adds to the V sua Stud o IDE

Here are some of the cha enges that deve opers face when des gn ng databases ■

■

■

■

Dependencies  By ts very nature, the database s fu of dependenc es between d fferent k nds of schema objects Th s comp cates deve opment, as even the s mp est changes can very qu ck y become very comp ex when dependenc es are nvo ved Late Error Detection  You can spend a ot of t me bu d ng comp ex scr pts, on y to find out that there are prob ems when you try to dep oy them to the database Or, your scr pt may dep oy w thout errors, but you have an ssue somewhere that doesn’t man fest tse f unt the user encounters a run-t me error “Drift“ Detection  The database s a constant y mov ng target After dep oyment, t’s fa r y common for a DBA to come a ong and tweak or patch someth ng n the product on d ­ atabase; for examp e, add ng ndexes to mprove query performance aga nst part cu ar tab es When the env ronments fa out of sync, the database s n a d fferent state than you and your ­app cat on expect t to be—and those d fferences need to be dent fied and reconc ed Versioning  Deve opers have grown so accustomed to work ng w th “change scr pts“ that t makes you wonder, where s the definition of the database? Of course you can re y on t be ng in the database, but where s t from the standpo nt of preserv ng and protect ng t? How do you ma nta n the defin t on across d fferent vers ons of your app cat on? It’s very d fficu t to revert to a po nt n t me and reca an ear er vers on of the database that matches up w th an

4  Part I  Core SQL Server Development

ear er vers on of an app cat on So you can’t eas y synchron ze vers ons and vers on h story between your database and app cat on ■

Deployment  Then there are the cha enges of target ng d fferent vers ons, nc ud ng most recent y, SQL Azure You may need to dep oy the same database out to d fferent ocat ons, and must account for vary ng compat b ty eve s when d fferent ocat ons are runn ng ­d fferent vers ons of SQL Server (such as SQL Server 2005, 2008, 2008 R2, 2012, and SQL Azure)

Many of these pa n po nts are rooted n the not on that the database s “statefu “ Every t me you bu d and run a NET app cat on n V sua Stud o, t s a ways n t a zed to the same “new“ state but as soon as the app cat on goes off to access the database, t’s the same “o d“ database w th the same schema and data n t Thus, you are forced to th nk not on y about the des gn of the database, but a so about how you mp ement that des gn—how you actua y get that des gn moved nto the ­database g ven the database’s current state If the root of these prob ems es n the database be ng statefu , then the heart of the so ut on es n work ng dec arat ve y rather than mperat ve y So rather than just work ng w th change scr pts, SSDT ets you work w th a declaration of what you believe (or want) the database to be Th s a ows you to focus on the des gn, wh e the too takes care of wr t ng the appropr ate change scr pts that w safe y app y your des gn to an actua target database SSDT takes a dec arat ve, mode -based ­approach to database des gn—and as you advance through th s chapter, you’ see how th s approach remed es the aforement oned pa n po nts

Declarative, Model-Based Development We’ve started exp a n ng that SSDT uses a dec arat ve, mode -based approach What th s means s that there s a ways an n-memory representat on of what a database ooks ke—an SSDT database ­model—and a the SSDT too s (des gners, va dat ons, Inte Sense, schema compare, and so on) operate on that mode Th s mode can be popu ated by a ve connected database (on-prem se or SQL Azure), an offl ne database project under source contro , or a po nt- n-t me snapshot taken of an offl ne database project (you w work w th snapshots n the upcom ng exerc ses) But to re terate, the too s are agnost c to the mode ’s back ng; they work exc us ve y aga nst the mode tse f Thus, you enjoy a r ch, cons stent exper ence n any scenar o—regard ess of whether you’re work ng w th ­on-prem se or c oud databases, offl ne projects, or vers oned snapshots The T-SQL representat on of any object n an SSDT mode s a ways expressed n the form of a ­CREATE statement An ALTER statement makes no sense n a dec arat ve context—a ­CREATE ­statement dec ares what an object shou d ook ke, and that’s the on y th ng that you (as a ­deve oper) need to be concerned w th Depend ng on the state of the target database, of course, a change scr pt conta n ng e ther a CREATE statement ( f the object doesn’t ex st yet) or an appropr ate ­ALTER ­statement ( f t does) w be needed to proper y dep oy the object’s defin t on Furthermore, f ­dependenc es are nvo ved (wh ch they very often are), other objects need to be dropped and ­re-created n the dep oyment process Fortunate y, you can now re y on SSDT to dent fy any changes (the “d fference“) between your mode defin t on and the actua database n order to compose the

Chapter 1  ntroduc ng SQL Server Data Too s   5

necessary change scr pt Th s keeps you focused on just the defin t on F gure 1-1 dep cts the SSDT mode -based approach to database deve opment

SQL Server Data Tools (SSDT)

Database Model SQL Server Database Project

SQL Server 2005, 2008, 2008 R2, 2012

LocalDB

SQL Azure

Database Snapshot File (.dacpac)

Figure 1-1  SSDT works w th a mode backed by connected databases (on prem se or n the c oud), offl ne ­database projects, or database snapshot fi es.

Connected Development A though SSDT p aces great emphas s on the dec arat ve mode , t n no way prevents you from ­work ng mperat ve y aga nst ve databases when you want or need to You can open query w ndows to compose and execute T-SQL statements d rect y aga nst a connected database, w th the ass stance of a debugger f des red, just as you can n SSMS The connected SSDT exper ence s dr ven off the new SQL Server Object Exp orer n V sua ­Stud o You can use th s new dockab e too w ndow to accomp sh common database deve opment tasks that former y requ red SSMS Us ng the new SQL Server Object Exp orer s str k ng y s m ar to ­work ng aga nst a connected database n SSMS’s Object Exp orer—but remember that (and we’ r sk ­overstat ng th s) the SSDT too s operate on y on a database model So when work ng n connected mode, SSDT actua y creates a mode from the rea database—on the fly—and then ets you ed t that mode Th s “buffered“ approach s a subt e, yet key, d st nct on from the way that SSMS operates When you save a schema change made w th the new tab e des gner, SSDT works out the n ­ ecessary scr pt needed to update the rea database so t reflects the change(s) made to the mode Of course, the end resu t s the same as the connected SSMS exper ence, so t sn’t str ct y necessary to ­understand th s buffered behav or that’s occurr ng beh nd the scenes But after you do grasp t, the too ’s offl ne project deve opment and snapshot vers on ng capab t es w mmed ate y seem natura and ntu t ve to you Th s s because offl ne projects and snapshots are s mp y d fferent back ngs of 6  Part I  Core SQL Server Development

the very same SSDT mode When you’re work ng w th the SQL Server Object Exp orer, the mode ’s ­back ng just happens to be a ve connected database There’s an add t ona nuance to SSDT’s buffered-wh e-connected approach to database ­deve opment that bears ment on ng There are n fact two mode s nvo ved n the process of app y ng schema changes to a database Just before SSDT attempts to app y your schema changes, t a ­ ctua y creates a new mode of the current y connected database SSDT uses th s mode as the target for a mode compar son w th the mode you’ve been ed t ng Th s dua -mode approach prov des the “dr ft detect on“ mechan sm you need to ensure that the schema compare operat on (upon wh ch SSDT w be bas ng ts change scr pt) accounts for any schema changes that may have been made by another user s nce you began ed t ng your vers on of the mode Va dat on checks w then catch any prob ems caused by the other user’s changes (wh ch wou d not have been present when you began mak ng your changes)

Disconnected Development The new SQL Server Object Exp orer ets you connect to and nteract w th any database r ght from ns de V sua Stud o But SSDT offers a great dea more than a mere rep acement for the connected SSMS exper ence It a so de vers a r ch offl ne exper ence w th the new SQL Server Database Project type and oca database runt me (Loca DB) The T-SQL scr pt fi es n a SQL Server Database Project are a dec arat ve n nature (on y CREATE statements; no ALTER statements) Th s s a rad ca y d fferent approach than you’re accustomed to when “deve op ng“ databases n SSMS (where you execute far more ALTER statements than CREATE statements) Aga n, you get to focus on defin ng “th s s how the database shou d ook,“ and et the too determ ne the appropr ate T-SQL change scr pt needed to actua y update the ve database to match your defin t on If you are fam ar w th the Database Profess ona (DbPro) ed t on of V sua Stud o, you w nstant y recogn ze the many s m ar t es between DbPro’s Database Projects and SSDT’s SQL Server Database Projects Desp te major over ap however, SSDT project types are d fferent than DbPro project types, and appear as a d st nct project temp ate n V sua Stud o’s Add New Project d a og The new tab e des gner, buffered connect on mechan sm, and other SSDT features covered n th s chapter work on y w th SSDT SQL Server Database Projects However, and as you may have guessed, t’s easy to upgrade ex st ng DbPro projects to SSDT projects Just r ght-c ck the project n So ut on Exp orer and choose Convert To SQL Server Database Project Note that th s s a one-way upgrade, and that DbPro ­art facts that are not yet supported by SSDT (such as data generat on p ans, see the fo ow ng Note) w not convert

Note  There are several important features still available only in DbPro, most notably data generation, data compare, schema view, and database unit testing. Eventually, SSDT plans on providing key elements of the DbPro feature set and will completely replace the Database Professional edition of Visual Studio. For now though, you must continue to rely on DbPro for what’s still missing in SSDT.

Chapter 1  ntroduc ng SQL Server Data Too s   7

The new SQL Server Database Project type enjoys many of the same capab t es and features as other V sua Stud o project types Th s nc udes not on y source contro for each nd v dua ­database object defin t on, but many of the common code nav gat on and refactor ng parad gms that ­deve opers have grown to expect of a modern IDE (such as Rename, Goto Defin t on, and F nd A ­References) The SSDT database mode ’s r ch metadata a so prov des for far better Inte Sense than what SSMS has been offer ng s nce SQL Server 2008, g v ng you much more of that “strong y-typed“ confidence factor as you code You can a so set breakpo nts, s ng e step through T-SQL code, and work w th the Loca s w ndow much ke you can when debugg ng NET project types W th SSDT, app cat on and database deve opment too ng has now fina y been un fied under one roof V sua Stud o A major advantage of the mode -based approach s that mode s can be generated from many d fferent sources When connected d rect y v a SQL Server Object Exp orer, SSDT generates a mode from the connected database, as we exp a ned a ready When you create a SQL Server Database Project (wh ch can be mported from any ex st ng database, scr pt, or snapshot), you are creat ng an offl ne, source-contro ed project ns de V sua Stud o that fu y descr bes a rea database But t’s ­actua y a project—not a rea database Now, SSDT generates a mode that’s backed by your SQL Server Database Project So the exper ence offl ne s just the same as when connected—the des gners, Inte Sense, va dat on checks, and a the other too s work exact y the same way As you conduct your database deve opment w th n the project, you get the same “background comp at on“ exper ence that you’re used to exper enc ng w th typ ca NET deve opment us ng C# or V sua Bas c (VB) NET For examp e, mak ng a change n the project that can’t be subm tted to the database because of dependency ssues w resu t n des gn-t me warn ngs and errors n the Error L st pane You can then c ck on the warn ngs and errors to nav gate d rect y to the var ous dependenc es so they can be dea t w th Once a the bu d errors d sappear, you’ be ab e to subm t the changes to update the database

Versioning and Snapshots A database project g ves you an offl ne defin t on of a database As w th a V sua Stud o projects, each database object (tab e, v ew, stored procedure, and every other d st nct object) ex sts as a text fi e that can be p aced under source code contro The project system comb ned w th source ­contro enab es you to secure the defin t on of a database n V sua Stud o, rather than re y ng on the ­defin t on be ng stored n the database tse f At any po nt n t me, and as often as you’d ke, you can create a database snapshot A snapshot s noth ng more than a fi e ( n the Data-t er App cat on Component Package, [dacpac] format) that ho ds the ser a zed state of a database mode , based on the current project at the t me the ­snapshot s taken It s essent a y a s ng e-fi e representat on of your ent re database schema Snapshots can ­ater be deser a zed and used w th any SSDT too (schema compare, for examp e) So you can deve op, dep oy, and synchron ze database structures across oca /c oud databases and d fferent y vers oned offl ne database projects, a w th cons stent too ng

8  Part I  Core SQL Server Development

Pause for a moment to th nk about the powerfu capab t es that snapshots prov de A snaphot encapsu ates an ent re database structure nto a s ng e .dacpac fi e that can be nstant y d ­ eser a zed back nto a mode at any t me Thus, they can serve as e ther the source or target of a schema ­compare operat on aga nst a ve database (on-prem se or SQL Azure), an offl ne SQL Server Database Project, or some other snapshot taken at any other po nt n t me Snapshots can a so be he pfu when you don’t have access to the target database, but are expected nstead to hand a change scr pt off to the DBA for execut on In add t on to the change scr pt, you can a so send the DBA a snapshot of the database project taken just before any of your offl ne work was performed That snapshot s your change scr pt’s assumpt on of what the ve database ooks ke So the DBA, n turn, can perform a schema compare between your snapshot and the ve database (th s can be done from SSDT’s command- ne too w thout V sua Stud o) The resu ts of that schema ­compare w nstant y et the DBA know f t’s safe to run your change scr pt If the resu ts revea d screpanc es between the ve database and the database snapshot upon wh ch your change scr pt s based, the DBA can reject your change scr pt and a ert you to the prob em

Targeting Different Platforms SQL Server Database Projects have a target p atform sw tch that ets you spec fy the spec fic SQL Server vers on that you ntend to dep oy the project to A the va dat on checks aga nst the ­project-backed mode are based on th s sett ng, mak ng t tr v a for you to test and dep oy your database to any part cu ar vers on of of SQL Server (2005 and ater), nc ud ng SQL Azure It’s s mp y a matter of choos ng SQL Azure as the target to ensure that your database can be dep oyed to the c oud w thout any prob ems If your database project defines someth ng that s not supported n SQL Azure (a tab e w th no c ustered ndex, for examp e), t w get flagged as an error automat ca y

Working with SSDT Our ntroduct on to the SSDT too set s comp ete, and t’s now t me to see t n act on The rest of th s chapter presents a samp e scenar o that demonstrates, step-by-step, how to use many of the SSDT features that you’ve just earned about Pract c ng a ong w th th s examp e w so d fy your know edge and understand ng of the too , and prepare you for us ng SSDT n the rea wor d w th rea database projects

Important  SSDT does not get installed with either Visual Studio or SQL Server. Instead, SSDT ships separately via the Web Platform Installer (WebPI). This enables Microsoft to ­distribute timely SSDT updates out-of-band with (that is, without waiting for major releases of) Visual Studio or SQL Server. Before proceeding, download and install SSDT from http://msdn.microsoft.com/en-us/data/hh297027.



Chapter 1  ntroduc ng SQL Server Data Too s   9

Connecting with SQL Server Object Explorer The journey w start by creat ng a database You w first use SSDT to execute a prepared scr pt that creates the database n a query w ndow connected to a SQL Server nstance Then you w start w ­ ork ng w th SSDT d rect y aga nst the connected database Th s exper ence s s m ar to us ng ­prev ous too s, so t’s the perfect p ace to start Later on, you’ sw tch to work ng d sconnected us ng an offl ne database project Launch V sua Stud o 2010, c ck the V ew menu, and choose SQL Server Object Exp orer Th s d sp ays the new SQL Server Object Exp orer n a V sua Stud o pane (docked to the eft, by defau t) Th s new too w ndow s the ma n act v ty hub for the connected deve opment exper ence n SSDT From the SQL Server Object Exp orer, you can eas y connect to any server nstance for wh ch you have credent a s In our scenar o, the localhost nstance runn ng on the deve opment mach ne s a fu SQL Server 2012 Deve oper ed t on nsta at on Th s nstance s assum ng the ro e of a ve product on database server that you can mag ne s runn ng n an on-prem se datacenter You’re now go ng to connect to that “product on“ database server R ght-c ck the SQL Server node at the top of the SQL Server Object Exp orer, choose Add SQL Server, and type n your mach ne name as the server to connect to A though you can certa n y, a ternat ve y, type localhost nstead (or even s mp y the s ng e-dot shorthand syntax for localhost), we’re d rect ng you to use the mach ne name nstead Th s s because you’ soon earn about the new oca database runt me (Loca DB) that SSDT prov des for offl ne test ng and debugg ng The Loca DB nstance always runs oca y, whereas the product on database on the other hand just ­happens to be runn ng oca y Because t can be potent a y confus ng to see both localhost and (localdb) n the SQL Server Object Exp orer, us ng the mach ne name nstead of localhost makes t c ear that one ­represents the product on database wh e the other refers to the database used for oca ­(offl ne) deve opment and test ng w th SSDT The screen snapshots for the figures n th s chapter were taken on a W ndows Server 2008 R2 mach ne named SQL2012DEV, so we’ be us ng that mach ne name throughout the chapter when referr ng to the product on database runn ng on localhost Of course, you’ need to rep ace the assumed SQL2012DEV mach ne name w th your own mach ne name ­wherever you see t ment oned If you have nsta ed SQL Server to use m xed-mode authent cat on and you are not us ng ­ ndows authent cat on, then you’ a so need to choose SQL Server authent cat on and supp y your W ­credent a s at th s po nt, before you can connect Once connected, SQL Server Object Exp orer shows the ­product on server and ets you dr down to show a the databases runn ng on t, as shown n F gure 1-2

10  Part I  Core SQL Server Development

Figure 1-2  The new SQL Server Object Exp orer n V sua Stud o expanded to show severa connected databases.

Once connected, r ght-c ck the server nstance node SQL2012DEV and choose New Query V sua Stud o opens a new T-SQL code ed tor w ndow, ke the one shown n F gure 1-3

Figure 1-3  A connected query w ndow.

Th s env ronment shou d seem very fam ar to anyone exper enced w th SSMS or Query ­Ana yzer Not ce the status bar at the bottom nd cat ng that the query w ndow s current y connected to the SQL2012DEV nstance (aga n, t w actua y read your mach ne name) The too bar at the top ­nc udes a drop-down st nd cat ng the current defau t database for the nstance you’re connected to As w th prev ous too s, th s w be the master database (as shown at the top of F gure 1-3) You must st take care to change th s sett ng (or ssue an appropr ate USE statement) so that you don’t ­nadvertent y ­access the master database when you rea y mean to access your app cat on’s database In th s ­exerc se, you’re creat ng a brand new database, so t’s fine that the current database s set to master at th s t me



Chapter 1  ntroduc ng SQL Server Data Too s   11

Tip This concern stems from the fact that, by default, the master database is established as every login’s default database. A great way to protect yourself from accidentally t­ rampling over the master database when working with SSDT (or any other tool) is to change your login’s default database to be your application’s database, which will then become the ­default database (rather than master) for every new query window that you open. This will greatly reduce the risk of unintentional data corruption in master, which can have ­disasterous consequences. When you navigate to your login from the Security node in SQL Server Object Explorer, you’ll be able to see its default database set to master in the Properties window, but you won’t be able to change it. This is a management task that is not supported in the SSDT tooling, although you can still use SSDT to execute the appropriate ALTER LOGIN statement in a query window. Alternatively, you can easily make the change in SSMS as follows. Start SSMS, connect to your server instance, and drill down to your login beneath the Security and Logins nodes in the SSMS Object Explorer. Then right-click the login and choose Properties. Click the default database drop-down list, change its value from master to your application’s database, and click OK. From then on, your database (not master) will be set as the default for every new SSDT query window that you open. Type the code shown n L st ng 1-1 nto the query w ndow (or open the scr pt fi e ava ab e n the down oadab e code on th s book’s compan on webs te; see the sect on “Code Samp es“ n the ­“Introduct on“ for deta s on how to down oad the samp e code) You m ght next be nc ned to press F5 to execute the scr pt, but that won’t work W th SSDT n V sua Stud o, press ng F5 builds and deploys a SQL Server Database Project to a debug nstance (you’ be creat ng such a project ater on, but you don’t have one yet) Th s s very d fferent to the way F5 s used n SSMS or Query Ana yzer to mmed ate y execute the current scr pt (or current y se ected scr pt) SSDT uses a d fferent keyboard shortcut for th s purpose In fact, there are two keyboard shortcuts (w th correspond ng too bar buttons and r ght-c ck context menu tems); one to execute w thout a debugger (Ctr +Sh ft+E) and one to execute us ng an attached debugger (A t+F5) You’ pract ce ­debugg ng ater on, so for now just press Ctrl+Shift+E to mmed ate y execute the scr pt and create the database (you can a so c ck the Execute Query button n the too bar, or r ght-c ck anywhere w th n the code w ndow and choose Execute from the context menu) Listing 1-1  T SQL scr pt for creat ng the SampleDb database

CREATE DATABASE SampleDb GO USE SampleDb GO -- Create the customer and order tables CREATE TABLE Customer(

12  Part I  Core SQL Server Development

CustomerId bigint NOT NULL PRIMARY KEY, FirstName varchar(50) NOT NULL, LastName varchar(50) NOT NULL, CustomerRanking varchar(50) NULL) CREATE TABLE OrderHeader( OrderHeaderId bigint NOT NULL, CustomerId bigint NOT NULL, OrderTotal money NOT NULL) -- Create the relationship ALTER TABLE OrderHeader ADD CONSTRAINT FK_OrderHeader_Customer FOREIGN KEY(CustomerId) REFERENCES Customer(CustomerId) -- Add a few customers INSERT INTO Customer (CustomerId, FirstName, LastName, CustomerRanking) VALUES (1, 'Lukas', 'Keller', NULL), (2, 'Jeff', 'Hay', 'Good'), (3, 'Keith', 'Harris', 'so-so'), (4, 'Simon', 'Pearson', 'A+'), (5, 'Matt', 'Hink', 'Stellar'), (6, 'April', 'Reagan', '') -- Add a few orders INSERT INTO OrderHeader(OrderHeaderId, CustomerId, OrderTotal) VALUES (1, 2, 28.50), (2, 2, 169.00), -- Jeff's orders (3, 3, 12.99), -- Keith's orders (4, 4, 785.75), (5, 4, 250.00), -- Simon's orders (6, 5, 6100.00), (7, 5, 4250.00), -- Matt's orders (8, 6, 18.50), (9, 6, 10.00), (10, 6, 18.00) -- April's orders GO -- Create a handy view summarizing customer orders CREATE VIEW vwCustomerOrderSummary WITH SCHEMABINDING AS SELECT c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerID = oh.CustomerID GROUP BY c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking GO

Th s s a very s mp e scr pt that we’ d scuss n a moment But first, not ce what just happened SSDT executed the scr pt d rect y aga nst a connected SQL Server nstance, and then sp t the code w ndow hor zonta y to d sp ay the resu t ng server messages n the bottom pane The green con ­abe ed Query Executed Successfu y n the ower- eft corner offers assurance that a went we w th the scr pt execut on Because of the two mu t -row INSERT statements used to create samp e ­customers and order data, you can see the two “rows affected“ messages n the bottom Message pane, as shown n F gure 1-4 Overa , the exper ence thus far s very s m ar to prev ous too s, ensur ng a smooth ­trans t on to SSDT for deve opers a ready fam ar w th the o der too ng

Chapter 1  ntroduc ng SQL Server Data Too s   13

Figure 1-4  The query w ndow after successfu y execut ng a T SQL scr pt.

Th s s mp e scr pt created a database named SampleDb, w th the two tab es Customer and ­ rderHeader It a so defined a fore gn key on the CustomerId co umn n both tab es, wh ch estab shes O the parent-ch d (one-to-many) re at onsh p between them It then added a few customer and re ated order rows nto the r respect ve tab es F na y, t created a v ew summar z ng each customer’s orders by aggregat ng a the r order tota s Now run two quer es to v ew some data At the bottom of the code w ndow, type the fo ow ng two SELECT statements SELECT * FROM Customer SELECT * FROM vwCustomerOrderSummary

Not ce the Inte Sense as you type After you fin sh typ ng, hover the cursor over Customer, and then aga n over vwCustomerOrderSummary V sua Stud o d sp ays too t ps descr b ng those ­objects respect ve y as a tab e and a v ew Now hover the cursor over the star symbo n each ­SELECT ­statement V sua Stud o d sp ays a too t p st ng a the fie ds represented by the star symbo n each query Th s funct ona ty s prov ded by the SSDT T-SQL anguage serv ces runn ng n the background that ­cont nuous y query the database mode backed by the connected SampleDb database Now se ect just the two SELECT statements ( eave the ent re scr pt above them unse ected) and press Ctrl+Shift+E The resu t s s m ar to press ng F5 n SSMS or Query Ana yzer on y the se ected text s executed (wh ch s what you’d expect) SSDT runs both quer es and d sp ays the r resu ts, as shown n F gure 1-5 You don’t need the query w ndow any onger, so go ahead and c ose t now (you a so don’t need to save the scr pt) R ght-c ck the Databases node n SQL Server Object Exp orer and choose Refresh You’ see the new SampleDb database node appear Expand t to dr down nto the database As ­F gure 1-6 shows, the env ronment s s m ar to the Object Exp orer n SSMS, and ets you carry out most (but not a ) of the deve oper-or ented tasks that SSMS ets you perform 14  Part I  Core SQL Server Development

Figure 1-5  V ew ng the resu ts of se ected statements executed n the query w ndow.

Figure 1-6 The SampleDb database n SQL Server Object Exp orer expanded to show severa of ts objects.



Chapter 1  ntroduc ng SQL Server Data Too s   15

The database s now up and runn ng on SQL2012DEV Everyth ng s perfect—unt that ema from market ng arr ves The r team has just put together some new requ rements for you, and now there’s more work to be done

Gathering New Requirements The new requ rements perta n to the way customers are ranked n the Customer tab e Or g na y, the market ng team had requested add ng the CustomerRanking co umn as a ghtwe ght mechan sm for data entry operators to rank customer performance Th s ad-hoc rat ng was supposed to be oose y based on the customer’s tota purchases across a orders, but you can see from the CustomerRanking va ues n F gure 1-5 that users fo owed no cons stency whatsoever as they entered data (no surpr se there) They’ve typed th ngs ke A+, so-so, and Good And some customers have no mean ngfu data at a , such as empty str ngs, wh tespace, or NULL va ues To mprove the s tuat on, the market ng team wou d ke to ret re the ad-hoc data entry co umn and rep ace t w th a forma zed customer rank ng system that s more a gned w th the r or g na ntent In the r change request ema (wh ch s natura y flagged Urgent), they have attached the spreadsheet shown n F gure 1-7 conta n ng new reference data for var ous pre-defined rank ng eve s They’ve scr bb ed someth ng about dep oy ng to SQL Azure as we , and then they s gn off w th “P S , We need t by Fr day“ (and no surpr se there, e ther)

Figure 1-7  Reference data for the new customer rank ng system.

After g v ng the matter some thought, you organ ze a h gh- eve task st Your st tem zes the deve opment steps you p an on tak ng to fu fi the market ng department’s requ rements 1. Remove the CustomerRanking co umn from the Customer tab e 2. Create a new CustomerRanking tab e based on the spreadsheet n F gure 1-7, w th a pr mary

key co umn CustomerRankingId stor ng the va ues 1 through 5 3. Add a new co umn CustomerRankingId to the Customer tab e 4. Create a fore gn key on the CustomerRankingId co umn to estab sh a re at onsh p between

the Customer tab e and the new CustomerRanking tab e 16  Part I  Core SQL Server Development

5. Update the vwCustomerOrderSummary v ew to jo n on the new CustomerRanking tab e 6. Create a new uspRankCustomers stored procedure to update the Customer tab e’s new fore gn

key co umn, based on each customer’s tota order va ue 7. Va date for SQL Azure, then dep oy to the c oud

The rest of th s chapter wa ks through th s procedure n deta , step by step A ong the way, you’ earn to everage many mportant SSDT features and w ga n ns ght nto the way the new too ng works It’s t me to get started w th the first step remov ng a co umn from a tab e

Note  The scenario we’ve presented here is admittedly somewhat artificial. We are not necessarily advocating these particular steps as the best way to solve a given problem, and certainly hope you are working with better designs than this in your own database. But for the purpose of this exercise—namely, learning how to use SSDT—we ask that you go along with it. The premise may be contrived, but the steps we’ve outlined for the solution are in fact quite representative of typical recurring activities in the everyday life of an average SQL Server developer.

Using the Table Designer (Connected) In SQL Server Object Exp orer, r ght-c ck the Customer tab e and choose V ew Des gner to open the SSDT tab e des gner, as shown n F gure 1-8

Figure 1-8  The new SSDT tab e des gner.

The top- eft pane of th s des gner w ndow sts the defined co umns n a gr d just as n the SSMS tab e d ­ es gner, but the s m ar ty ends there A very d fferent mechan sm s at p ay w th the new SSDT ­des gner, one that shou d be easy to understand after a the d scuss on we’ve had around ­dec arat ve, mode -based des gn The CREATE TABLE statement n the bottom T-SQL pane g ves t away Knowng that the tab e a ready ex sts n the database, why s th s a CREATE statement? We , that’s because

Chapter 1  ntroduc ng SQL Server Data Too s   17

th s sn’t actua y T-SQL code that you ntend to execute aga nst the database as- s (wh ch wou d fa of course, because the tab e ex sts) Rather, t’s a T-SQL declaration of “how th s tab e shou d ook,“ whether t ex sts or not—and ndeed, whether t ex sts w th a different schema or not— n the target database Here’s what’s actua y happen ng The des gner s operat ng over a memory-res dent database mode ns de ts work ng env ronment Because you are connected at the moment, that mode s backed by the ve SampleDb database But when you sw tch to work ng offl ne w th a SQL Server ­Database Project (as you w n the next sect on of th s chapter), you’ nteract w th the very same tab e des gner over a mode backed by a project nstead of a rea database A mode can a so be backed by a database snapshot Because the tab e des gner just operates over a mode , the same too works cons stent y n any of these scenar os You want to remove the CustomerRanking co umn, and that can be done e ther by de et ng t from the gr d n the top pane or ed t ng t out of the dec arat ve T-SQL code n the bottom pane Both panes are mere y v ews nto the same tab e, so any changes appear b d rect ona y Throughout th s exerc se, you’ exper ment w th d fferent ed t ng techn ques n the tab e des gner, start ng w th the qu ckest method Just r ght-c ck CustomerRanking n the top gr d and choose De ete The co umn s removed from the gr d and, as you’d expect, the T-SQL code s updated to reflect the change That was a pretty easy change App y ng that change to the database shou d be easy, too Go ahead and c ck the Update button on the too bar Unfortunate y, nstead of just work ng as you’d ke, you rece ve the fo ow ng error message Update cannot proceed due to validation errors. Please correct the following errors and try again. SQL71501 :: View: [dbo].[vwCustomerOrderSummary] contains an unresolved reference to an object. Either the object does not exist or the reference is ambiguous because it could refer to any of the following objects: [dbo].[Customer].[c]::[CustomerRanking], [dbo].[Customer]. [CustomerRanking] or [dbo].[OrderHeader].[c]::[CustomerRanking]. SQL71501 :: View: [dbo].[vwCustomerOrderSummary] contains an unresolved reference to an object. Either the object does not exist or the reference is ambiguous because it could refer to any of the following objects: [dbo].[Customer].[c]::[CustomerRanking], [dbo].[Customer]. [CustomerRanking] or [dbo].[OrderHeader].[c]::[CustomerRanking]. SQL71558 :: The object reference [dbo].[Customer].[CustomerID] differs only by case from the object definition [dbo].[Customer].[CustomerId]. SQL71558 :: The object reference [dbo].[OrderHeader].[CustomerID] differs only by case from the object definition [dbo].[OrderHeader].[CustomerId].

What went wrong? Referr ng back to L st ng 1-1, not ce that the v ew defin t on for vwCustomerOrderSummary spec fies the WITH SCHEMABINDING c ause Th s means that the co umns of the v ew are bound to the under y ng tab es exposed by the v ew, wh ch protects you from “break ng“ the v ew w th schema changes—as you’ve done just now The prob em, as reported by the first two errors, s that the schema-bound v ew’s CustomerRanking co umn has sudden y been removed from the Customer tab e that under es the v ew The second two errors are actua y on y case-sens t v ty warn ngs that, on the r own, wou d not prevent the update from succeed ng We w exp a n these ­case-sens t v ty warn ngs a b t ater; for now, rema n focused on the dependency ssue that’s b ock ng the update 18  Part I  Core SQL Server Development

The nterest ng th ng worth not ng at th s po nt s that SSDT caught the cond t on before even attempting to app y your changes to the ve database (wh ch wou d certa n y have thrown an ­error) In fact, you cou d have been aware of th s ssue even before c ck ng Update f you had ­prev ous y opened the Error L st pane, because SSDT constant y va dates changes to the mode n the ­background wh e you ed t t n the des gner C ck the Cance button to d sm ss the error w ndow Then c ck the V ew menu and choose ­Error L st to open the pane Not ce how the errors and warn ngs appear, just ke comp at on errors ­appear for C# and VB NET projects And just ke those project types, you can doub e-c ck tems n the Error L st and nstant y nav gate to the offend ng code to dea w th the errors In th s case, both ­dependency errors are n vwCustomerOrderSummary, so doub e-c ck e ther one now to open a code ed tor nto the v ew, as shown n F gure 1-9

Figure 1-9  Detect ng and nav gat ng va dat on errors.

You want to rev se th s v ew to jo n aga nst the new CustomerRanking tab e, but that’s not ­com ng up unt step 4 n your task st So for now, just perform the m n mum ed ts necessary to c ear the ­va dat on errors (wh ch are dent fied by red squ gg es ke you’ve seen n other V sua Stud o code w ndows) so you can update the database and move on De ete (by comment ng out) the two ­references to c.CustomerRanking co umn from the v ew (one s n the co umn st, the other n the GROUP BY c ause) Not ce how the errors d sappear from the Error L st pane as you correct the code You’re now beg nn ng to exper ence the effects of mode -based deve opment w th SSDT n V sua Stud o W th a c ear Error L st, you know that a your changes are va d You have a tered a tab e and a v ew, but those changes have been made on y to the memory-res dent mode The changed objects are current y open n separate V sua Stud o w ndows, and both w ndows have an Update button

Chapter 1  ntroduc ng SQL Server Data Too s   19

Yet t makes no d fference wh ch of the two buttons you c ck— n e ther case, Update means that all changes that have been buffered get sent to the database So wh chever Update button you c ck, your ed ts to both the tab e and the v ew are go ng to get app ed to the database at once How s that go ng to happen? The ed ts were s mp e enough, but the T-SQL change scr pt needed to app y those ed ts s actua y a b t more comp ex And there n ay the beauty of th s new too ng—a of that scr pt ng comp ex ty s hand ed for you by SSDT The too compares the ed ted mode w th a brand-new mode based on the ve database, and then works out the change scr pt automat ca y Creat ng a fresh mode from the database at th s t me makes sure you’re work ng w th ts atest state, n case t dr fted because t was mode ed for the current ed t ng sess on Then t runs an nterna schema compare operat on between the ed ted mode (the source) and the atest mode based on the ve ­database (the target) to dent fy a the r d fferences F na y, SSDT generates a change scr pt that can be executed on the ve database to app y the changes C ck Update now to generate the change scr pt Before runn ng the change scr pt, SSDT d sp ays an nformat ve report of a the act ons that the change scr pt s go ng to take C ck Update now to d sp ay the Prev ew Database Updates w ndow, as shown n F gure 1-10

Figure 1-10  The Prev ew Database Updates w ndow.

You shou d a ways scrut n ze th s prev ew to make sure t’s cons stent w th the act ons and resu ts you wou d expect of the ed ts you’ve made In th s case, you’re be ng warned about data oss n the Customer tab e by dropp ng the CustomerRanking co umn You’re a so be ng to d that the scr pt w drop and then re-create the schema b nd ng of the vwCustomerOrderSummary v ew, before and after the tab e s a tered A of th s s expected Now you can c ck Update Database to mmed ate y execute the change scr pt, or you can c ck Generate Scr pt to oad the change scr pt nto a code ed tor so you can v ew, poss b y mod fy, and choose to e ther execute t or not In most cases, you’ fee comfortab e just c ck ng Update Database, part cu ar y f you’ve rev ewed the warn ngs and act ons reported by the database update prev ew Do ng so w mmed ate y 20  Part I  Core SQL Server Development

­execute the change scr pt to update the ve database But be ng that th s s your very first update, c ck Generate Scr pt nstead so you can exam ne the scr pt before you run t The scr pt s shown n L st ng 1-2 (to conserve space, error-check ng code has been commented out) Listing 1-2  The change scr pt for the a tered tab e and v ew automat ca y generated by SSDT.

/* Deployment script for SampleDb */ // ... :setvar DatabaseName "SampleDb" GO // ... USE [$(DatabaseName)]; GO // ... BEGIN TRANSACTION GO PRINT N'Removing schema binding from [dbo].[vwCustomerOrderSummary]...'; GO ALTER VIEW [dbo].[vwCustomerOrderSummary] AS SELECT c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerID = oh.CustomerID GROUP BY c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking; // ... GO PRINT N'Altering [dbo].[Customer]...'; GO ALTER TABLE [dbo].[Customer] DROP COLUMN [CustomerRanking]; GO // ... PRINT N'Adding schema binding to [dbo].[vwCustomerOrderSummary]...'; GO -- Create a handy view summarizing customer orders ALTER VIEW vwCustomerOrderSummary WITH SCHEMABINDING AS SELECT c.CustomerID, c.FirstName, c.LastName, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerID = oh.CustomerID GROUP BY c.CustomerID, c.FirstName, c.LastName



Chapter 1  ntroduc ng SQL Server Data Too s   21

GO // ... IF @@TRANCOUNT>0 BEGIN PRINT N'The transacted portion of the database update succeeded.' COMMIT TRANSACTION END ELSE PRINT N'The transacted portion of the database update failed.' GO DROP TABLE #tmpErrors GO PRINT N'Update complete.' GO

It’s great that you d dn’t have to write the change scr pt, but t’s st the change scr pt Let’s ook t over qu ck y now

mportant that you understand

Us ng var ab e subst tut on, the scr pt first ssues a USE statement that sets SampleDb as the ­current database and then t beg ns a transact on The transact on w get comm tted on y f the ent re change scr pt comp etes successfu y Then the scr pt ssues an ALTER VIEW statement that removes the schema b nd ng from vwCustomerOrderSummary w thout yet chang ng ts defin t on So t st conta ns those two references to the CustomerRanking co umn that’s about to get dropped from the Customer tab e, but that w not present a prob em because WITH SCHEMABINDING has been removed from the v ew Next, the scr pt ssues the ALTER TABLE statement that actua y drops the co umn from the tab e After the co umn s dropped, another ALTER VIEW statement s ssued on vwCustomerOrderSummary w th the new vers on that no onger references the dropped co umn and s once aga n schemabound F na y, the transact on s comm tted and the update s comp ete Press Ctrl+Shift+E The scr pt s executed and output descr b ng act ons taken by the scr pt are d sp ayed n the Messages pane Removing schema binding from [dbo].[vwCustomerOrderSummary]... Altering [dbo].[Customer]... Adding schema binding to [dbo].[vwCustomerOrderSummary]... The transacted portion of the database update succeeded. Update complete.

You can c ose a open w ndows now V sua Stud o w prompt to save changes, but t’s not ­ ecessary to do so because the database has just been updated R ght-c ck on the database and n choose Refresh, and then dr down nto SampleDb n SQL Server Object Exp orer to confirm that the tab e and v ew changes have been app ed You w see that the CustomerRanking co umn has been ­removed from the database, and that comp etes step 1

Working Offline with a SQL Server Database Project W th SQL Server Database Projects, you can deve op databases w th no connect on whatsoever to a SQL Server nstance A SQL Server Database Project s a project that conta ns nd v dua , ­dec arat ve, T-SQL source code fi es These source fi es co ect ve y define the comp ete structure of a database 22  Part I  Core SQL Server Development

Because the database defin t on s ma nta ned th s way ns de a V sua Stud o project, t can be preserved and protected w th source code contro (SCC) just ke the art facts n a your other V sua Stud o project types SSDT generates a mode from the project structure beh nd the scenes, just ke t generates a mode from the ve database when work ng connected Th s ets you use the same SSDT too s whether work ng offl ne or connected You carr ed out your first task on ne wh e connected d rect y to a ve database Now you’ create a SQL Server Database Project for the database so that you can cont nue your work offl ne A though (as you’ve seen) t’s easy to use SSDT for connected deve opment, you shou d dea y deve op your databases offl ne w th SQL Server Database Projects, and pub sh to ve servers whenever you’re ready to dep oy your changes By do ng so, you w der ve the benefits of source contro , snapshot vers on ng, and ntegrat on w th the rest of your app cat on’s code through the V sua Stud o so ut on and project system There are severa ways to create a SQL Server Database Project You can start w th an empty project, des gn a database structure from the ground up, and then pub sh the ent re structure to a new database on a SQL Server nstance oca y or n the c oud on SQL Azure Or, as n th s scenar o, you have an ex st ng database on a oca SQL Server nstance from wh ch you want to generate a SQL Server Database Project And you want th s project popu ated w th a the dec arat ve T-SQL source fi es that comp ete y define the ex st ng database structure It’s easy to do th s w th the Import Database d a og R ght-c ck the SampleDb database under the SQL2012DEV nstance n SQL Server Object Exp orer and choose Create New Project to d sp ay the Import Database d a og, as shown n F gure 1-11

Figure 1-11  Creat ng a SQL Server Database Project from an ex st ng database.



Chapter 1  ntroduc ng SQL Server Data Too s   23

The source database connect on confirms that your new project w be generated from the S­ ampleDb database on SQL2012DEV Change the target project name from Database1 to SampleDb (and set the ocat on, too, f you w sh) Check the Create New So ut on checkbox, and f you have an SCC prov der for V sua Stud o, check Add To Source Contro as we Then c ck Start If you checked Add To Source Contro , you w be prompted at th s po nt to supp y credent a s and server nformat on (th s w depend on your defau t SCC prov der n V sua Stud o) It takes a few ­moments for V sua Stud o to scour the database, d scover ts schema, and generate the dec arat ve T-SQL source fi es for the new project When done, V sua Stud o d sp ays a w ndow w th ­nformat on descr b ng a the act ons t took to create the project C ck F n sh to c ose th s w ndow The new ­project s then opened n the So ut on Exp orer automat ca y (docked to the r ght, by defau t) The dbo fo der represents the dbo schema n the database Expand t, and then expand the Tab es and V ews fo ders, as shown n F gure 1-12

Figure 1-12  The source contro ed SQL Server Database Project after mport ng a database.

SSDT set up your project th s way because the Fo der Structure sett ng n the Import Database d a og (F gure 1-11) was set to Schema\Object Type Th s te s SSDT to create a fo der for each ­schema, and then a fo der for each object type (tab e, v ew, and so on) conta ned n that schema You are free to create add t ona fo ders as you extend your project Un ess you have a very spec fic or un que convent on, t s best pract ce to ma nta n a cons stent project structure based on the schema ­organ zat on n the database ke we’ve shown here

More Information  Schemas in SQL Server bear similarity to namespaces in .NET. Our simple example database has only a single namespace (dbo, short for database owner), but more complex databases typically consolidate database objects into multiple schemas. Just as namespaces are used to organize many classes in large .NET applications, schemas help manage many objects defined in large databases. Like classes and namespaces, two database objects can be assigned the same name if they are contained in two different schemas. For example, both Sales.Person and Billing. Person refer to two completely different Person tables (one in the Sales schema and one in the Billing schema). SQL Server schemas can define objects at just a single level however, whereas .NET namespaces can be nested in as many levels as desired to define an elaborate hierarchy of classes. 24  Part I  Core SQL Server Development

Taking a Snapshot Before you make any offl ne database changes, take a snapshot Th s w create a s ng e-fi e mage of the current database schema that you can refer to or revert to at any t me n the future You’ take another snapshot when you’ve comp eted a your database changes, and thereby preserve the two po nts n t me— just before, and just after, the changes are made And because they are ma nta ned as part of the project, snapshot fi es are a so protected under source contro R ght-c ck the SampleDb project n So ut on Exp orer and choose Snapshot Project After v­ a dat ng the project, V sua Stud o creates a new Snapshots fo der and, ns de the Snapshots fo der, t creates a new .dacpac fi e w th a fi ename based on the current date and t me You’ usua y want to g ve the snapshot a better name, so rename the fi e to Version1Baseline.dacpac

Using the Table Designer (Offline Database Project) W th your “base ne“ snapshot saved, you’re ready to create the new CustomerRanking tab e R ­ eca that th s s the new reference tab e based on the spreadsheet n F gure 1-7 In So ut on Exp orer, ­r ght-c ck the project’s Tab es fo der (under the dbo schema fo der) and choose Add Tab e Name the tab e CustomerRanking and c ck Add A new offl ne tab e des gner w ndow opens You’ find that t ooks and fee s exact y the same as the one n F gure 1-8 that you used when work ng on ne That’s because t is the same too , on y th s t me t’s the des gner over a mode backed by a source-contro ed project fi e (CustomerRanking.sql) rather than a mode backed by a ve tab e Because there’s no connected database, the tab e des gner has no Update button— nstead, when work ng offl ne, schema changes are saved to the project scr pt fi e for that tab e Th s n turn updates the mode , and then the same va dat on checks and ­Inte Sense you exper enced when work ng wh e connected are run aga nst the project So you can find out r ght away f and when you make a break ng change, before dep oy ng to a rea database Ear er, when you removed the CustomerRanking co umn from the Customer tab e, we ment oned that you can des gn the tab e us ng e ther the gr d n the top pane or the T-SQL code w ndow n the ­bottom pane You can a so v ew and change parts of the schema defin t on from the Propert es gr d We’ ­demonstrate a of these techn ques now as you ay out the schema of the new CustomerRanking tab e SSDT starts you off w th a new tab e that has one co umn n t an int pr mary key named Id To rename th s co umn to CustomerRankingId, se ect the co umn name Id n the top pane’s gr d, rep ace t w th CustomerRankingId, and press Enter Beneath t, add the RankName co umn, set ts data type to varchar(20), and uncheck A ow Nu s You can see that SSDT updates the T-SQL code n the bottom pane w th a CREATE TABLE statement that reflects the changes made n the top pane Add the th rd co umn by ed t ng the T-SQL code n the bottom pane Append a comma after the second co umn and type [Description] VARCHAR(200) NOT NULL As expected, the gr d n the top pane s updated to show the new Description co umn you just added n code F na y, tweak the data type us ng the Propert es gr d C ck the Description co umn n the top pane and scro to the Length property n the Propert es gr d (to d sp ay the Propert es gr d f t’s not ­current y v s b e, c ck V ew and choose Propert es W ndow) C ck the drop-down st and se ect MAX

Chapter 1  ntroduc ng SQL Server Data Too s   25

to change the data type from varchar(200) to varchar(max) When you’re done, the tab e des gner shou d ook s m ar to F gure 1-13

Figure 1-13  The tab e des gner for the new CustomerRanking tab e n an offl ne SQL Server Database Project.

Save CustomerRanking.sql and c ose the tab e des gner Th s comp etes step 2 You are now ready to add the fore gn key co umn to the Customer tab e (step 3) that jo ns to th s new C ­ ustomerRanking tab e Doub e-c ck Customer.sql n So ut on Exp orer to open the tab e des gner for the Customer tab e Use any techn que you’d ke to add a new nu ab e int co umn named CustomerRankingId ( t must be nu ab e at th s po nt, because t doesn’t have any data yet) Now you can estab sh the fore gn key re at onsh p to the CustomerRanking tab e (step 4) In the upper-r ghthand corner of the Tab e Des gner s a Context V ew area that summar zes other ­pert nent objects re ated to the tab e In the Context V ew, r ght-c ck Fore gn Keys and choose Add New ­Fore gn Key Name the new fore gn key FK Customer CustomerRanking ( t s best pract ce to ass gn fore gn key names that nd cate wh ch tab es part c pate n the re at onsh p) Then ed t the ­FOREIGN KEY temp ate code added to the T-SQL code w ndow n the bottom pane to be FOREIGN KEY (CustomerRankingID) REFERENCES CustomerRanking(CustomerRankingId) The tab e des gner w ndow shou d now ook s m ar to F gure 1-14 After rev ew ng the tab e schema, save the Customer.sql fi e and c ose the des gner

26  Part I  Core SQL Server Development

Figure 1-14  The tab e des gner for the Customer tab e after creat ng the fore gn key on CustomerRankingId.

Introducing LocalDB Your next tasks nvo ve a ter ng a v ew (step 5) and creat ng a stored procedure (step 6) It w be very he pfu to have a test SQL Server env ronment ava ab e as you mp ement these steps You don’t want to use SQL2012DEV, because that’s the “ ve“ server You need another SQL Server that can be used just for test ng offl ne Loca DB g ves you that test env ronment Th s s a new, ghtwe ght, s ng e-user nstance of SQL Server that sp ns up on demand when you bu d your project Th s s extreme y handy when ­work ng offl ne and there s no other deve opment server ava ab e to you The offic a name for th s new ­var ant of SQL Server s “SQL Express Loca DB,“ wh ch can be m s ead ng because t s d st nct from the Express ed t on of SQL Server To avo d confus on, we refer to t s mp y as “Loca DB “

Note  The new LocalDB does not support every SQL Server feature (for example, it can’t be used with FILESTREAM). However, it does support most functionality required for typical database development. Press F5 to bu d the project Th s first va dates the ent re database structure defined by the project and then dep oys t to Loca DB Note, however, that th s s just the defau t behav or; you can change the project propert es to target another ava ab e server for test ng f you requ re features not supported by Loca DB (for examp e, FILESTREAM, wh ch we cover n Chapter 8) The dep oyment s carr ed out by perform ng a schema compare between the project and L­ oca DB on the target server More prec se y, and as a ready exp a ned, mode s of the source ­project and ­target database are generated, and the schema compare actua y works on the two mode s ­Be ng your very first bu d, the database does not ex st yet on the target server, so the schema ­compare generates a scr pt that creates the who e database from scratch As you mod fy the project,

Chapter 1  ntroduc ng SQL Server Data Too s   27

s­ ubsequent bu ds w generate ncrementa change scr pts that spec fy just the act ons needed to br ng the target database back n sync w th the project Look back over n SQL Server Object Exp orer and you’ see that SSDT has started a new Loca DB nstance The host name s (localdb)\SampleDb, and t s a comp ete y separate nstance than the SQL2012DEV nstance (wh ch has not yet been updated w th the new CustomerRanking tab e and the CustomerRankingId fore gn key n the Customer tab e) F gure 1-15 shows SampleDb dep oyed to Loca DB, expanded to revea ts tab es Not ce that t does nc ude the new CustomerRanking tab e

Figure 1-15  The oca database runt me (Loca DB) after bu d ng and dep oy ng the SQL Server Database Project.

Now you have a test database to p ay around w th, but of course there’s no data n t You w add some now so that you can test the v ew you’re about to a ter and the stored procedure you’re about to create Us ng s mp e copy/paste, SSDT ets you mport sma sets of rows from any “tab e“ source ( nc ud ng M crosoft Word and Exce ) nto a database tab e that has compat b e co umns F rst, br ng n the reference data from the rank ng defin t ons prov ded by the spreadsheet n F gure 1-7 You can eas y grab the data stra ght out of the spreadsheet and dump t r ght nto the new CustomerRanking tab e Open the spreadsheet n Exce , se ect the five rows of data (comp ete 28  Part I  Core SQL Server Development

rows, not ce s or co umns), then r ght-c ck the se ect on and choose Copy Back n SQL Server Object Exp orer, r ght-c ck the CustomerRanking tab e and choose V ew Data The Ed tab e Data Gr d n SSDT opens w th a temp ate for nsert ng a new row R ght-c ck the row se ector for the new row temp ate and choose Paste (be sure to r ght-c ck the row se ector n the eft gray marg n area, and not a ce , before past ng) As shown n F gure 1-16, SSDT fa thfu y cop es the data from Exce nto the ­CustomerRanking tab e

Figure 1-16  Reference data mported nto a database tab e from Exce v a the c pboard.

You a so need a few customers to work w th Us ng the same copy/paste techn que, you w ­transfer rows from the Customer tab e n the product on database to the test database on Loca DB (for th s ­exerc se, you won’t transfer re ated order data n the OrderHeader tab e) There are on y a handfu of customers, so you’ just copy them a over Typ ca y though, you’d extract just the subset of data that prov des a representat ve samp ng good enough for test ng purposes Expand the product on server (SQL2012DEV) node n SQL Server Object Exp orer and dr down to the Customer tab e R ght-c ck the tab e and choose V ew Data Se ect a the customer rows, then r ght-c ck the se ect on and choose Copy Next, r ght-c ck the Customer tab e n the test database on (localdb)\SampleDb and choose V ew Data R ght-c ck the new row se ector and choose Paste to copy n the s x customer rows You are now at step 5, wh ch s to update the vwCustomerOrderSummary v ew Reca that th s s the same v ew you ed ted back n step 1 (wh e connected), when you removed the schema-bound reference to the o d CustomerRanking co umn that was be ng dropped from the Customer tab e W th the new reference tab e and fore gn key re at onsh p now n p ace, you w rev se the v ew once aga n (offl ne, th s t me) to jo n w th the CustomerRanking tab e on CustomerRankingId, so that t can expose the d sp ay name n the reference tab e’s RankName co umn In So ut on Exp orer, doub e-c ck vwCustomerOrderSummary.sql n the project’s V ews fo der (­under the dbo schema fo der) The v ew opens up n a new code w ndow, and your attent on may first be drawn to severa squ gg es that V sua Stud o pa nts n the v ew’s code They’re not red, ­because there s rea y noth ng s gn ficant y wrong w th the v ew, and so these are just warn ngs ­Hover the cursor over one of them to v ew the warn ng text n a too t p (you can a so see a of them sted as warn ng tems n the Error L st pane) The warn ngs nd cate that the v ew uses CustomerID (end ng n a cap ta D) to reference a co umn that s actua y defined as CustomerId (end ng n a ­owercase d) These are the same case-sens t v ty warn ngs you saw ear er when attempt ng to update the ­database w th dependency ssues Object names n SQL Server are norma y not case-sens t ve

Chapter 1  ntroduc ng SQL Server Data Too s   29

( ke VB NET), but non-defau t co at on sett ngs can change that behav or so that they are case-sens t ve ( ke C#) Th s wou d cause a prob em f you dep oyed the v ew to a SQL Server nstance configured for a ­case-sens t ve co at on of object names Add another LEFT OUTER JOIN to the v ew to add n the CustomerRanking tab e jo ned on the ­CustomerRankingId of the Customer tab e, and add RankName to the SELECT and GROUP BY ­co umn sts You want your code to be squeaky-c ean, so now s a so a good t me to reso ve those ­case-sens t v ty warn ngs Rep ace CustomerID w th CustomerId n the four p aces that t occurs (once n the SELECT co umn st, tw ce n the first JOIN, and once more n the GROUP BY co umn st) L st ng 1-3 shows the v ew defin t on after mak ng the changes Listing 1-3  The updated vwCustomerOrderSummary v ew defin t on jo n ng on the new CustomerRanking tab e.

-- Create a handy view summarizing customer orders CREATE VIEW vwCustomerOrderSummary WITH SCHEMABINDING AS SELECT c.CustomerId, c.FirstName, c.LastName, r.RankName, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerId = oh.CustomerId LEFT OUTER JOIN dbo.CustomerRanking AS r ON c.CustomerRankingId = r.CustomerRankingId GROUP BY c.CustomerId, c.FirstName, c.LastName, r.RankName

Save the vwCustomerOrderSummary.sql fi e to update the offl ne project You know that p ­ ress ng F5 now w dep oy the changed v ew to the test database on Loca DB But what f you attempt to ­execute the scr pt d rect y by press ng Ctrl+Shift+E, r ght here n the code w ndow? Go ahead and try You’ rece ve th s error message n response Msg 2714, Level 16, State 3, Procedure vwCustomerOrderSummary, Line 2 There is already an object named 'vwCustomerOrderSummary' in the database.

Here’s what happened F rst, SSDT connected the query w ndow to the (localdb)\SampleDb ­nstance Then t attempted to execute the scr pt mperat ve y aga nst the connected database, just as you’ve a ready seen w th Ctr +Sh ft+E But be ng part of an offl ne project, th s scr pt s dec arat ve and so t’s expressed as a CREATE VIEW statement The v ew a ready ex sts n the database, so the error message makes perfect sense Aga n, the proper way to update the database s to dep oy t v a an ncrementa dep oyment scr pt by debugg ng w th F5 However, you are ndeed connected to the test database on Loca DB, even though you’re ­work ng ns de the query w ndow of an offl ne project that hasn’t yet been dep oyed to Loca DB Th s means that you can actua y test the v ew before you dep oy t Se ect a the text from SELECT unt the end of the scr pt (that s, eave on y the CREATE VIEW port on of the w ndow unse ected) and press Ctrl+Shift+E aga n Th s t me, you get a much better resu t On y the chosen SELECT statement executes, wh ch s perfect y va d T-SQL for the connected database The query resu ts show you what 30  Part I  Core SQL Server Development

the v ew s go ng to return, and you got that nformat on w thout hav ng to dep oy first In th s mode, you are actua y work ng connected and offl ne s mu taneous y! You can se ect any T-SQL to nstant y execute t, test and debug stored procedures, and even get execut on p ans, a wh e “offl ne “

Refactoring the Database The v ew s ready to be dep oyed, but now you dec de to change some names first Customers are the on y th ng be ng ranked, so shorten ng the tab e name CustomerRanking to Ranking and ­co umn names CustomerRankingId to RankingId s go ng to make your T-SQL more readab e (wh ch s ­mportant!) W thout the proper too ng, t can be very ted ous to rename objects n the database But the refactor ng capab t es prov ded by SSDT make th s remarkab y easy In the new LEFT OUTER JOIN you just added, r ght-c ck on the CustomerRanking tab e reference, and then choose Refactor, Rename Type Ranking for the new tab e name and c ck OK You are presented w th a prev ew w ndow (F gure 1-17) that w appear very fam ar f you’ve ever used the refactor ng features n V sua Stud o w th ord nary NET projects

Figure 1-17  Prev ew ng changes before refactor ng s app ed to the database.

Th s d a og shows a the references to the CustomerRanking tab e that w be changed to ­Ranking when you c ck App y (not ce that checkboxes are prov ded so that you can a so choose wh ch ­references shou d get updated and wh ch shou d not) Scro through the change st to prev ew each one, and then c ck App y to mmed ate y nvoke the rename operat on Every affected project fi e s updated accord ng y, but V sua Stud o won’t actua y rename project fi es themse ves The project scr pt fi e defin ng the new y renamed Ranking tab e s st named CustomerRanking.sql R ght-c ck the fi e n So ut on Exp orer, choose Rename, and change the fi ename to Ranking.sql Now rename the pr mary key co umn n the Ranking tab e a ong w th ts correspond ng fore gn key co umn n the Customer tab e, both of wh ch are current y named CustomerRankingId The two key co umns are referenced on the same LEFT OUTER JOIN ne, so th s w be easy R ght-c ck the r.CustomerRankingId key co umn n the jo n and choose Refactor, Rename Type RankingId for the new name, c ck OK, prev ew the changes, and c ck App y to update the pr mary key co umn name

Chapter 1  ntroduc ng SQL Server Data Too s   31

n the Ranking tab e Then repeat for the c.CustomerRankingId key co umn to update the fore gn key co umn name n the Customer tab e (the actua order n wh ch you refactor the co umn names n these tab es s mmater a ) There’s one more th ng to rename, and that’s the fore gn key defin t on n the Customer tab e Th s sn’t str ct y necessary of course, but the (se f- mposed) convent on to name fore gn keys ­defin t ons after the tab es they jo n d ctates that FK Customer CustomerRanking shou d be renamed to ­FK ­Customer Ranking The Customer tab e s spec fied first n the v ew’s FROM c ause, so r ght-c ck on t now and choose Go to Defin t on Th s nav gates d rect y to the Customer tab e defin t on n a new query w ndow In the CONSTRAINT c ause, r ght-c ck FK Customer CustomerRanking and choose Refactor, Rename Type FK Customer Ranking for the new name, c ck OK, prev ew the changes (just one, n th s case), and c ck App y You’re a set to dep oy the changes w th another bu d, so press F5 once aga n After the bu d comp etes, c ck Refresh n the SQL Server Object Exp orer too bar and ook at the test database runn ng under (localdb)\SampleDb to confirm that the CustomerRanking tab e has been renamed to Ranking R ght-c ck the Ranking tab e and choose V ew Data to confirm that a the data n the renamed tab e s ntact When you rename objects n a SQL Server Database Project, SSDT generates a change scr pt w th correspond ng EXECUTE sp rename statements n t, as opposed to dropp ng one object and creat ng another (wh ch, for tab es, wou d resu t n rrevocab e data oss) So the too does the r ght th ng, re y ng u t mate y on the SQL Server sp rename stored procedure to proper y change the object’s name nterna y w th n the database It’s t me to create the stored procedure that ranks the customers F rst, create a Stored ­Procedures fo der beneath the dbo fo der n So ut on Exp orer (to do th s, r ght-c ck the dbo fo der, and choose Add New Fo der) Th s fo der wou d have a ready been created when you mported the database nto the project, had there been any stored procedures n the database at the t me Then r ght-c ck the new Stored Procedures fo der and choose Add Stored Procedure Name the stored procedure uspRankCustomers and c ck Add SSDT creates a new fi e named uspRankCustomers. sql and opens t n a new T-SQL ed tor w ndow Rep ace the temp ate code w th the scr pt shown n ­L st ng 1-4 and save t, but keep the w ndow open Now press F5 to perform another bu d and push the new stored procedure out to the test database on Loca DB Listing 1-4  The stored procedure to rank customers based on the r tota order amount.

CREATE PROCEDURE uspRankCustomers AS DECLARE @CustomerId int DECLARE @OrderTotal money DECLARE @RankingId int

DECLARE curCustomer CURSOR FOR SELECT CustomerId, OrderTotal FROM vwCustomerOrderSummary



OPEN curCustomer FETCH NEXT FROM curCustomer INTO @CustomerId, @OrderTotal

32  Part I  Core SQL Server Development

WHILE @@FETCH_STATUS = 0 BEGIN IF @OrderTotal = 0 SET @RankingId = 1 ELSE IF @OrderTotal < 100 SET @RankingId = 2 ELSE IF @OrderTotal < 1000 SET @RankingId = 3 ELSE IF @OrderTotal < 10000 SET @RankingId = 4 ELSE SET @RankingId = 5 UPDATE Customer SET RankingId = @RankingId WHERE CustomerId = @CustomerId END

FETCH NEXT FROM curCustomer INTO @CustomerId, @OrderTotal

CLOSE curCustomer DEALLOCATE curCustomer

Th s stored procedure “ranks“ the customers, exam n ng them nd v dua y and ass gn ng each a va ue based on the r order tota It does th s by open ng a cursor aga nst the order summary v ew, wh ch returns one row per customer w th the r nd v dua orders aggregated nto a s ng e order tota Based on the do ar va ue of the tota , t then updates the customer w th a rank ng va ue between one and five Then t advances to the next customer unt t reaches the end of the cursor As ment oned at the outset, th s so ut on may be a b t contr ved (and we’re sure you can th nk of a better approach), but t su ts our demonstrat on purposes here just fine

Testing and Debugging Are you n the hab t of runn ng new or untested code on ve databases? We certa n y hope not Though you cou d, you shou d not s mp y push a of the changes you’ve made n the project (steps 2 through 6) back to the ve database on SQL2012DEV, and then run the stored procedure there for the very first t me It’s much safer to test the stored procedure offl ne first w th Loca DB You w now earn how to do that us ng the ntegrated debugger n V sua Stud o Then you can confident y dep oy everyth ng back to SQL2012DEV, and fina y (step 7), to the c oud! The uspRankCustomers stored procedure s st open n the code ed tor C ck ns de the eft marg n on the OPEN curCustomer ne to set a breakpo nt just before the cursor s opened The breakpo nt appears as a red bu et n the marg n where you c cked Th s s exact y how breakpo nts are set n C# or VB NET code, and SSDT now de vers a s m ar debugg ng exper ence for T-SQL code as we In SQL Server Object Exp orer, expand the Stored Procedures node ( ocated beneath Programmab ty, just as n SSMS) for SampleDb beneath the Loca DB nstance R ght-c ck the Stored Procedures node, choose Refresh, and you w see the uspRankCustomers stored procedure you just dep oyed R ghtc ck on the stored procedure and choose Debug Procedure SSDT generates an EXEC statement to nvoke uspRankCustomers and opens t n a new query w ndow The debugger s a ready started, and s paused on the USE [SampleDb] statement above the EXEC statement



Chapter 1  ntroduc ng SQL Server Data Too s   33

More Info  The debugging session began instantly in this case because the ­uspRankCustomers stored procedure being debugged has no parameters. When stored procedure parameters are expected, SSDT will first display a dialog to solicit the parameter values, and then plug those values into the EXEC statement before starting the debugger. Press F5 to cont nue execut on The debugger reaches the EXEC statement, enters the stored procedure, and then breaks on the OPEN curCustomer statement where you prev ous y set the breakpo nt Now start s ng e stepp ng through the stored procedure’s execut on w th the debugger’s F10 keystroke Press F10 three t mes to step over the next three statements Th s opens the cursor, fetches the first customer from t, and you are now paused on the first IF statement that tests the first customer’s order tota for zero do ars Ear er, you cop ed s x customer rows from the SQL2012DEV database to Loca DB, but we s­ pec fica y nstructed you not to copy any order data So th s oop w terate each customer, and (based on an order tota of zero do ars) ass gn a rank ng va ue of 1 for every customer Rather than nterrupt ng your debugg ng sess on now to mport some samp e order data and start over, you w use the debugger’s Loca s w ndow to s mu ate non-zero order tota s for the first two customers C ck the Debug menu, and then choose W ndows Loca s In the Loca s w ndow, you can see that @CustomerId s 1 (th s s the first customer) and ­@­OrderTotal s 0 (expected, because there’s no samp e order data) @RankingId s not yet set, but f you a ow execut on to cont nue as- s, the customer w be ranked w th a 1 Doub e-c ck the 0 0000 va ue for @OrderTotal n the Loca s w ndow, type 5000 and press Enter Now the stored procedure th nks that the customer actua y has $5,000 n tota orders Press F10 to s ng e step Because @OrderTotal no onger equa s zero, execut on advances to the next IF cond t on that tests the order tota for ­be ng under $100 Press F10 aga n and execut on advances to the next IF cond t on that tests for under $1,000 Press F10 once more to reach the IF cond t on test ng for under $10,000 Th s ­cond t on y e ds true (there are $5,000 n tota orders), so press ng F10 to s ng e step once more advances to the SET statement that ass gns a rank ng va ue of 4 Th s s the correct va ue for orders n the range of $1,000 to $10,000 F gure 1-18 shows the debugg ng sess on paused at th s po nt Cont nue press ng F10 to s ng e step through the rema n ng SET, UPDATE, and FETCH NEXT s­ tatements, and then back up aga n to the first IF statement test ng the second customer’s order tota va ue for zero do ars Use the Loca s w ndow to fake another amount; th s t me change @OrderTotal to 150 S ng e step a few more t mes to make sure that th s resu ts n the stored procedure ass gn ng a rank ng va ue of 3 th s t me, wh ch s the correct va ue for orders n the range of $100 to $1,000 Now press F5 to et the stored procedure fin sh process ng the rest of the customers w th no more ­ntervent on on your part When the stored procedure comp etes execut on, r ght-c ck the Customer tab e n SQL Server Object Exp orer (be sure to p ck the Loca DB nstance and not SQL2012DEV) and choose V ew Data The tab e data confirms that the first customer’s rank ng was set to 4, the second customer’s rank ng was set to 3, and a the other customer rank ngs were set to 1 ( f you a ready have a Customer tab e

34  Part I  Core SQL Server Development

w ndow open from before, the prev ous va ues w button n the too bar to update the d sp ay)

st

be d sp ayed; you need to c ck the Refresh

Figure 1-18  T SQL debugg ng sess on of a stored procedure n V sua Stud o.

Th s was by no means exhaust ve test ng, but t w suffice for demonstrat on purposes The key po nt s that SSDT prov des an env ronment you can use for debugg ng and test ng as you deve op your database offl ne, unt you’re ready to dep oy to a ve env ronment (as you are now)

Comparing Schemas You are ready to dep oy to the database back to the ve server on SQL2012DEV As you may have correct y surm sed by now, the process s fundamenta y the same as work ng offl ne w th Loca DB each t me F5 s pressed SSDT runs a schema compare to generate a change scr pt The project ­propert es (by defau t) spec fy a connect on str ng that po nts to Loca DB So bu d ng w th F5 uses the test database as the target for the schema compare w th the project as the source, and then executes the generated change scr pt aga nst the test database on Loca DB Th s a happens as a comp ete y unattended set of operat ons every t me you press F5 Now you w carry out those very same steps once aga n, on y th s t me you’ get more nvo ved n the process In part cu ar, you w spec fy the ve SQL2012DEV nstance as the target for the schema compare, rather than Loca DB You w a so rev ew the resu ts of the schema compare, and have the chance to choose to dep oy or not dep oy spec fic detected changes F na y, you’ get the ­opportun ty to v ew, ed t, save, and execute the change scr pt after t s generated, rather than hav ng

Chapter 1  ntroduc ng SQL Server Data Too s   35

t execute automat ca y So there’s a b t more ntervent on nvo ved n the process now, but you want t that way The schema compare process tse f s the same as the F5 bu d—you just get to exerc se more contro over t to support d fferent dep oyment scenar os R ght-c ck the SampleDb project n So ut on Exp orer and choose Schema Compare to open a new schema compare w ndow You need to spec fy a source and target for any schema compare, natura y Because you aunched the w ndow from the SQL Server Database Project context menu n So ut on Exp orer, V sua Stud o sets the source to the project automat ca y, eav ng you to set just the target To set the target, c ck ts drop-down st and choose Se ect Target to d sp ay the Se ect Target Schema d a og, shown n F gure 1-19

Figure 1-19  SSDT ets you choose between projects, databases, and snapshots for schema compare operat ons.

Not ce how you can choose between three schemas for the target—a project, a database, or a data-t er app cat on fi e (snapshot) The same cho ces are a so supported for the source, a though the SQL Server Database Project was assumed as the source automat ca y n th s case Any comb nat on of source and target source schemas s supported; SSDT s mp y creates source and target mode s from your cho ce of back ngs Then, work ng off the mode s, t shows you the d fferences and generates a change scr pt for you Th s flex b ty s a major benefit of mode -based database deve opment w th SSDT The Se ect Target Schema d a og has correct y assumed that you want to use a database as the target A you need to do s choose the ve SampleDb database runn ng on SQL2012DEV C ck New Connect on to open a Connect on Propert es d a og, type your actua mach ne name for the server name (wh ch s SQL2012DEV n the current examp e), choose SampleDb from the database name drop-down st, and c ck OK (V sua Stud o w remember th s connect on for the future, and make t ava ab e for reca n the database dropdown the next t me you run a schema compare ) C ck OK once more, and then c ck the Compare button n the too bar to start the schema compare It takes just a few moments for the operat on to comp ete When t fin shes, the schema compare d sp ays a the changes you’ve made offl ne s nce creat ng the project (steps 2 through 6) The report ets you see each added, changed, and dropped object, and t can be organ zed by type (tab e, v ew, and so on), schema, or act on (change, add, or de ete) Se ect ng any object n the top pane presents ts T-SQL dec arat on n the bottom pane, s de-by-s de (source on the eft, target on the r ght), w th synchron zed scro bars The T-SQL s co or-coded to h gh ght every one of the object’s d fferences 36  Part I  Core SQL Server Development

If des red, you can exc ude spec fic objects from the change scr pt (wh ch hasn’t been generated yet) by c ear ng the r nd v dua checkboxes back up n the top pane Se ect the vwCustomerOrderSummary v ew n the top pane to see the source and target ­vers ons of the v ew n the bottom pane As shown n F gure 1-20, the r ch v sua d sp ay rendered by the schema compare too makes t easy to dent fy a the changes made to the v ew

Figure 1-20  V ew ng the schema d fferences between a SQL Server Database Project and a ve database.

As w th the tab e des gner, you can choose to update the ve database mmed ate y by ­generat ng and runn ng the change scr pt w thout prev ew ng t Or you can choose to be more caut ous, and just generate the change scr pt Then you can v ew, ed t, and u t mate y dec de whether or not to execute t Your confidence eve shou d be very h gh by now, so just c ck the Update button n the too bar (and then c ck Yes to confirm) to et t run SSDT updates the target database and d sp ays a ­comp et on message when t’s done C ck OK to d sm ss the message The d fferences from before the update are st d sp ayed n the w ndow, now d mmed n gray (you can c ck Compare aga n to confirm that there are no onger any d fferences between the project and the ve database on SQL2012DEV) In SQL Server Object Exp orer, dr down on SampleDb under SQL2012DEV (or refresh a ready dr ed down nodes) to ver fy that t reflects a the work performed n the project for steps 2 through 6 on your task st You are a most ready to run the new uspRankCustomers stored procedure and update the ve ­ ustomer tab e, but there’s one more th ng to do before that A though the dep oyment created the C schema of the Ranking tab e, t d dn’t copy ts data You need to mport the reference data from the spreadsheet n F gure 1-7 aga n, th s t me nto the ve database on SQL2012DEV You can certa n y use the same copy/paste tr ck we showed ear er when you mported the spreadsheet nto the test database on Loca DB, but we’ take th s opportun ty now to show you how to scr pt tab e data w th SSDT

Chapter 1  ntroduc ng SQL Server Data Too s   37

Note The Ranking table is a typical example of reference data. Databases often rely on r­ eference data, which are usually small sets of read-only entries, to help define their structure. Although technically not schema, it would certainly be convenient to mark the contents of the Ranking table as such, so that the five entries it contains become part of the SSDT database model and travel with the schema definition wherever you deploy it to. Unfortunately, this feature could not make it in time for the final product release, but Microsoft is evaluating plans to add this capability to a future version of SSDT. Under the (localdb)\SampleDb node (the Loca DB nstance) n SQL Server Object Exp orer, r­ ght-c ck the Ranking tab e and choose V ew Data to open a w ndow show ng the five rows n the tab e Next, c ck the Scr pt button on the too bar (the next to the ast button) SSDT generates INSERT statements for the five rows of data n the Ranking tab e, and d sp ays them n a new query w ndow You want to execute these INSERT statements n a query w ndow connected to the ve database on SQL2012DEV, so se ect a the INSERT statements and press Ctrl+C to copy them to the c pboard Then under the SQL2012DEV node n SQL Server Object Exp orer, r ght-c ck the SampleDb database and choose New Query Press Ctrl+V to paste the INSERT statements nto the new query w ndow and then press Ctrl+Shift+E to execute them The reference data has now been mported nto the ve database and you’re ready to update the customers In the same query w ndow, type EXEC uspRankCustomers, se ect the text of the statement, and press Ctrl+Shift+E The stored procedure executes and updates the customers (You can gnore the nu va ue warn ng; t refers to the SUM aggregate funct on n the v ew, wh ch does not affect the resu t ) To see the fina resu t, type SELECT * FROM vwCustomerOrderSummary, se ect t, and press Ctrl+Shift+E once aga n As shown n F gure 1-21, each customer’s rank ng s correct y ass gned based on the r tota order amount

Figure 1-21  V ew ng the fina resu ts of offl ne deve opment n the ve database.

38  Part I  Core SQL Server Development

Publishing to SQL Azure The market ng team’s ast request was that you dep oy a copy of the database to SQL Azure To ensure that the database s c oud-ready, you just need to te SSDT that you are target ng the SQL Azure p atform by chang ng a property of the project Then, f any SQL Azure compat b ty ssues are dent fied, they can be reso ved before you dep oy As you m ght expect by now, you w use the very same techn ques you’ve earned throughout th s chapter to dep oy the SQL Server Database Project to SQL Azure

Note  Our discussion in this section assumes you already have an available SQL Azure server instance that you can publish to. SQL Azure server names always begin with a unique identifier randomly assigned just to you, followed by .database.windows.net. Chapter 12 (which is dedicated to SQL Azure) explains how to use the Azure Management Portal to create your own cloud databases on SQL Azure, after setting up a Windows Azure account. R ght-c ck the SampleDb project n So ut on Exp orer and choose Propert es In the Project ­Sett ngs tab, you’ not ce that the Target P atform s current y set to SQL Server 2012 Change t to SQL Azure as shown n F gure 1-22, press Ctrl+S to save the propert es, and then c ose the propert es w ndow

Figure 1-22  Chang ng the target p atform of a SQL Server Database Project to SQL Azure.



Chapter 1  ntroduc ng SQL Server Data Too s   39

Now press F5 to once aga n bu d the project and dep oy t to Loca DB The bu d fa s, and the ­Error L st pane shows the fo ow ng error SQL71560: Table [dbo].[OrderHeader] does not have a clustered index. required for inserting data in this version of SQL Server.

Clustered indexes are

Th s error nforms you that the OrderHeader tab e s m ss ng a c ustered ndex The astute reader m ght have not ced back n L st ng 1-1 that the OrderHeaderId co umn n th s tab e does not spec fy PRIMARY KEY ( ke the Customer tab e does on ts CustomerId co umn), and so OrderHeader has no c ustered ndex Th s was an overs ght that m ght not have been caught so eas y because tab es n onprem se ed t ons of SQL Server do not requ re a c ustered ndex But SQL Azure databases ­abso ute y requ re a c ustered ndex on every tab e, so now that you’re target ng the c oud spec fica y, the probem s brought to your attent on ns de the project Th s s a qu ck and easy fix to make us ng the tab e des gner Back n the SQL Server Database ­ roject ( n So ut on Exp orer), doub e-c ck the OrderHeader.sql tab e (under the dbo and Tab es P fo ders) to open the project’s defin t on of the tab e n the des gner R ght-c ck the OrderHeaderId co umn, choose Set Pr mary Key, save, and then c ose the tab e des gner The pr mary key defin t on resu ts n the creat on of a c ustered ndex on the tab e Th s reso ves the ssue, and you’ see the error d sappear from the Error L st pane mmed ate y Now that you know the database s c oud-compat b e, you’re ready to dep oy t to SQL Azure R ght-c ck the SQL Server Database Project n So ut on Exp orer and choose Pub sh to d sp ay the Pub sh Database d a og C ck Ed t, enter the server and og n nformat on for your SQL Azure ­database, and c ck OK F gure 1-23 shows the Pub sh Database d a og w th the target connect on str ng po nt ng to a SQL Azure database

Figure 1-23  The Pub sh Database d a og set to dep oy the project to a SQL Azure target nstance.

As we’ve been not ng a a ong, you can scr pt the dep oyment w thout execut ng t by c ck ng Generate Scr pt But you’re ready to dep oy to SQL Azure r ght now C ck Pub sh, and V sua Stud o 40  Part I  Core SQL Server Development

sp ns up the same fam ar process It performs a schema compare between the source SQL Server ­ atabase Project and target SQL Azure nstance, and then generates and executes the resu t ng D change scr pt on the target As w th your very first bu d to Loca DB, the database does not ex st yet on the target, so the change scr pt creates the who e database n the c oud from scratch ­Subsequent dep oyments w generate ncrementa change scr pts that spec fy just the act ons needed to synchron ze the SQL Azure database w th the project Dur ng the dep oyment process, the Data Too s Operat ons w ndow n V sua Stud o prov des a dynam c d sp ay of what’s happen ng F gure 1-24 shows the Data Too s Operat ons w ndow after the pub sh process s comp ete

Figure 1-24  The Data Too s Operat ons pane reports a the act ons taken to dep oy to SQL Azure.

A rea y n ce feature of the Data Too s Operat ons pane s the ab ty to see the scr pts that were just executed ns de query w ndows and v ew the r execut on resu ts C ck the var ous nks (V ew Prev ew, V ew Scr pt, and V ew Resu ts) to rev ew the dep oyment you just ran After dep oy ng, SSDT automat ca y adds your SQL Azure server nstance to the SQL Server Object Exp orer, as shown n F gure 1-25 You can dr down on SQL Azure databases n SQL Server Object Exp orer and work w th them us ng the very same deve opment too s and techn ques that we’ve shown throughout th s chapter It’s exact y the same mode -based, buffered exper ence you have w th connected deve opment of on-prem se databases, on y now t’s a SQL Azure database ­back ng the mode Thus, SQL Server Object Exp orer funct ons as a s ng e access po nt for connected ­deve opment aga nst any SQL Server database, wherever t’s ocated You’ve used SSDT to successfu y mp ement a the tasks to fu fi your requ rements Before conc ud ng your work, take another snapshot R ght-c ck the project n So ut on Exp orer one ast t me and choose Snapshot Project SSDT ser a zes the database mode (based on the ­project’s current state) nto another .dacpac fi e n the Snapshots fo der, wh ch you shou d rename to ­Version1Complete.dacpac Now your project has two snapshots, Version1Baseline.dacpac and Version1Complete.dacpac, and each represents the database structure at two d fferent po nts n t me The co ect on w grow over t me as you take new snapshots dur ng future deve opment, and thus your project accumu ates an h stor ca account of ts database structure as t changes w th each new vers on And because any snapshot can serve as e ther the source or target mode of a schema compare operat on, t’s very easy



Chapter 1  ntroduc ng SQL Server Data Too s   41

to d fference between any two po nts n t me, or between any s ng e po nt n t me and e ther a ve database (on-prem se or SQL Azure) or an offl ne SQL Server Database Project

Figure 1-25  A SQL Azure database connected n SQL Server Object Exp orer.

Adopting SSDT No too s perfect, and SSDT s no except on Yet even as we ca out those areas where the too s ack ng, we’ st emphas ze what b g be evers we are n th s new techno ogy, and that we great y encourage SSDT adopt on over trad t ona database deve opment methods The SSDT team has done a fantast c job w th the mode -based des gn, and there s a ot more too ng st that can be prov ded by everag ng the mode ’s r ch metadata, such as database d agrams and query des gners that are not yet prov ded There s a so no spat a v ewer to graph ca y d sp ay spat a query resu ts, such as the one prov ded n SQL Server Management Stud o (we cover spat a quer es and the spat a v ewer n Chapter 9) A though SSDT s nt mate y aware of database schema, t does not prov de data-or ented f­ unct ona ty So t can’t generate data or compare data n the database, nor does t support ­database un t test ng These are mportant features supported by the V sua Stud o Database Profess ona ­ed t on (DbPro) that are st m ss ng from SSDT Th s means that, a though SSDT s pos t oned to ­obso esce ­DbPro, that won’t happen unt t ach eves par ty w th key components of the DbPro feature set

42  Part I  Core SQL Server Development

Summary Th s chapter began w th a h gh- eve overv ew descr b ng the many cha enges deve opers face ­work ng w th databases dur ng the deve opment process Through hands-on exerc ses, you then saw how the new SQL Server Data Too s (SSDT) prov des features that can he p you tack e those ­cha enges You worked through a number of scenar os us ng SSDT for connected deve opment, offl ne ­ eve opment w th the oca database runt me (Loca DB), source contro , debugg ng, and test ng—and d then dep oyed to a oca env ronment as we as the c oud—a from w th n V sua Stud o A ong the way, you earned how to use many mportant SSDT features, such as schema compare, refactor ng, snapshot vers on ng, and mu t -p atform target ng A though you can a ways earn more about the new too s just by us ng them, th s chapter has prepared you to use SSDT as you encounter the many cha eng ng SQL Server deve opment tasks that e ahead



Chapter 1  ntroduc ng SQL Server Data Too s   43

C hapter 2

T-SQL Enhancements —Leonard Lobel

A

s SQL Server evo ves, M crosoft cont nues to mprove and enhance Transact SQL (T-SQL)—the nat ve procedura programm ng anguage that deve opers have been us ng s nce the b rth of the product Today, T-SQL ma nta ns ts ro e as the pr mary anguage for programm ng the re at ona database eng ne, even as numerous other capab t es and programmab ty po nts are added to the SQL Server stack w th each new re ease

It s true that SQL Server Common Language Runt me ntegrat on (SQL CLR) a ows you to use C# or ­V sua Bas c (VB) NET as an a ternat ve to T-SQL for wr t ng stored procedures, tr ggers, and other ­database objects Th s mportant capab ty was added n SQL Server 2005, and Chapter 3 covers SQL CLR n deta You w see that SQL CLR s a great so ut on when the r ch, managed env ronment ­afforded by the NET Framework can be everaged advantageous y on the database server ­Interna y, SQL Server everages SQL CLR to mp ement the hierarchyid (Chapter 7) and geometry/­geography (Chapter 9) data types Pr or to SQL CLR, extended stored procedures were the pr mary means for ­programm ng funct ona ty that cou d not be ach eved us ng T-SQL a one Thus, SQL CLR stored ­procedures are certa n y recommended as a safe rep acement for extended stored procedures (wh ch have the potent a to crash SQL Server f they encounter an error) But n a most a other cases, you w st want to use T-SQL as your pr mary anguage for ­ rogramm ng the most effic ent set-based quer es and stored procedures at the database eve In p th s chapter, we’ beg n w th n-depth coverage of these T-SQL features first ntroduced n SQL Server 2008 ■

Tab e-va ued parameters (TVPs)

■

Date and t me data types

■

The MERGE statement

■

The INSERT OVER DML syntax

■

The GROUPING SETS operator

We’ then exp ore these new T-SQL features n SQL Server 2012 ■

W ndow ng enhancements w th the OVER c ause

■

New T-SQL funct ons

45

■

Error hand ng mprovements w th the THROW statement

■

Server-s de resu t set pag ng w th OFFSET and FETCH NEXT

■

Sequent a number generat on w th the SEQUENCE object

■

Metadata d scovery

Note  You can use either SQL Server Management Studio (SSMS) or SQL Server Data Tools (SSDT), which we cover in Chapter 1, to run the code in this chapter.

Table-Valued Parameters As the name mp es, a tab e-va ued parameter (TVP) ets you pass an ent re set of rows as a s ng e parameter to T-SQL stored procedures and user-defined funct ons (UDFs) Th s s extreme y usefu n and of tse f, but arguab y the most compe ng facet of TVPs s the r ab ty to marsha an ent re set of rows across the network, from your NET c ent to your SQL Server database, w th a s ng e stored procedure ca (one roundtr p) and a s ng e tab e-va ued parameter Pr or to SQL Server 2008, deve opers were forced to resort to c ever hacks n an effort to reduce mu t p e roundtr ps nto one when nsert ng mu t p e rows—us ng techn ques such as XML, de m ted or encoded text, or even (gasp) accept ng hundreds (up to 2100!) of parameters But spec a og c then needs to be mp emented for packag ng and unpackag ng the parameter va ues on both s des of the w re Worse, the code to mp ement that og c s often gnar y and unma nta nab e None of those techn ques even come c ose to the e egance and s mp c ty of us ng TVPs, wh ch offer a nat ve ­so ut on to th s prob em

More Than Just Another Temporary Table Solution A TVP s based on a user-defined table type, wh ch you create to descr be the schema for a set of rows that can be passed to stored procedures and UDFs It’s he pfu to beg n understand ng TVPs by first compar ng them to s m ar “set” constructs, such as tab e var ab es, temp tab es, and Common Tab e Express ons (CTEs) A of these prov de a source of tabu ar data that you can query and jo n aga nst, so you can treat a TVP, tab e var ab e, temporary tab e, or CTE just ke you wou d an ord nary tab e or v ew n v rtua y any scenar o CTEs and tab e var ab es store the r row data n memory—assum ng reasonab y s zed sets that don’t overflow the RAM cache a ocated for them, n wh ch case, they do push the r data nto tempdb In ­contrast, a TVP’s data s a ways stored n tempdb When you first popu ate a TVP, SQL Server creates a tab e n tempdb to back that TVP as t gets passed from one stored procedure (or UDF) to another Once the stack unw nds and the TVP fa s out of scope n your T-SQL code, SQL Server c eans up tempdb ­automat ca y You never nteract d rect y w th tempdb, because TVPs prov de a tota abstract on over t

46  Part 1  Core SQL Server Development

The true power of TVPs es n the ab ty to pass an ent re tab e (a set of rows) as a s ng e ­ arameter from c ent to server, and between your T-SQL stored procedures and user-defined p ­funct ons Tab e var ab es and temporary tab es, on the other hand, cannot be passed as parameters CTEs are m ted n scope to the statement fo ow ng the r creat on and are therefore nherent y ­ncapab e of be ng passed as parameters Reusab ty s another s de benefit of TVPs The schema of a TVP s centra y ma nta ned, wh ch s not the case w th tab e var ab es, temporary tab es, and CTEs You define the schema once by creat ng a new user-defined type (UDT) of type table, wh ch you do by app y ng the AS TABLE c ause to the CREATE TYPE statement, as shown n L st ng 2-1 Listing 2-1  Defin ng the schema for a user defined tab e type.

CREATE TYPE CustomerUdt AS TABLE (Id int, CustomerName nvarchar(50), PostalCode nvarchar(50))

Th s statement creates a new user-defined tab e type named CustomerUdt w th three co umns TVP var ab es of type CustomerUdt can then be dec ared and popu ated w th rows of data that fit th s schema, and SQL Server w store and manage those rows n tempdb beh nd the scenes These var ab es can be passed free y between stored procedures—un ke regu ar tab e var ab es, wh ch are stored n RAM beh nd the scenes and cannot be passed as parameters When TVP var ab es dec ared as CustomerUdt fa out of scope and are no onger referenced, the under y ng data n tempdb support ng the TVP s de eted automat ca y by SQL Server You can see that, n fact, a TVP s essent a y a user-defined tab e type Popu ated nstances of th s type can be passed on as parameters to stored procedures and user-defined funct ons—someth ng you st can’t do w th a regu ar tab e var ab e Once the tab e type s defined, you can create stored procedures w th parameters of that type to pass an ent re set of rows us ng TVPs TVP types are d sp ayed n V sua Stud o’s SQL Server Object Exp orer n the User-Defined ­Table Types node beneath Programmability Types, as shown n F gure 2-1 (as t does n SQL Server ­Management Stud o’s Object Exp orer) There are many pract ca app cat ons for pass ng ent re sets of data around as parameters, and we’ exp ore a number of them n the rest of th s sect on

Submitting Orders A typ ca scenar o n wh ch TVPs can be app ed s an order entry system When a customer p aces an order, a new order row and any number of new order deta rows must be created n the database Trad t ona y, th s m ght be accomp shed by creat ng two stored procedures—one for nsert ng an order row and one for nsert ng an order deta row The app cat on wou d nvoke a stored p ­ rocedure



Chapter 2  T SQL Enhancements   47

ca for each nd v dua row, so for an order w th 20 deta s, there wou d be a tota of 21 stored ­procedure ca s (1 for the order and 20 for the deta s) There cou d of course be even arger orders w th many more than 20 deta s As a resu t, numerous roundtr ps are made between the app cat on and the database, each one carry ng on y a s ng e row of data

Figure 2-1  User defined tab e types that can be used for TVPs d sp ayed n SQL Server Object Exp orer.

Enter TVPs Now you can create a s ng e stored procedure w th just two TVPs, one for the order row and one for the order deta s rows The c ent can now ssue a s ng e ca to th s stored procedure, pass ng to t the ent re order w th a ts deta s, as shown n L st ng 2-2

Note  The code in Listing 2-2 assumes that the Order and OrderDetail tables already exist, and that the OrderUdt and OrderDetailUdt table types have already been created with a column schema that matches the tables. Listing 2-2  Creat ng a stored procedure that accepts TVPs.

CREATE PROCEDURE uspInsertNewOrder (@OrderTvp AS OrderUdt READONLY, @OrderDetailsTvp AS OrderDetailUdt READONLY)

48  Part 1  Core SQL Server Development

AS INSERT INTO [Order] SELECT * FROM @OrderTvp INSERT INTO [OrderDetail] SELECT * FROM @OrderDetailsTvp

As you can see, th s code nserts nto the Order and OrderDetail tab es d rect y from the rows passed n through the two TVPs You are essent a y perform ng a bu k nsert w th a s ng e ca , rather than nd v dua nserts across mu t p e ca s wrapped n a transact on We’ now take ook at the bu k nsert poss b t es for TVPs and how to create, dec are, popu ate, and pass TVPs n T-SQL Then we’ demonstrate how to popu ate TVPs and pass them across the ­network from NET c ent app cat on code to stored procedures us ng ADO NET

Using TVPs for Bulk Inserts and Updates Here’s an examp e of a stored procedure that you can create n the AdventureWorks2012 database that accepts a TVP and nserts a of the rows that get passed n through t nto the Product.Location tab e By creat ng a user-defined tab e type named LocationUdt that descr bes the schema for each row passed to the stored procedure, any code can ca the stored procedure and pass to t a set of rows for nsert on nto Product.Location us ng a s ng e parameter typed as LocationUdt F rst, create the user-defined tab e data type LocationUdt, as shown n L st ng 2-3 Listing 2-3  Creat ng the LocationUdt tab e type to be used for bu k operat ons w th TVPs.

CREATE TYPE LocationUdt AS TABLE( LocationName varchar(50), CostRate int)

Now a TVP var ab e of th s type can be dec ared to ho d a set of rows w th the two co umns ­LocationName and CostRate These rows can be fed to a stored procedure by pass ng the TVP var ab e nto t The stored procedure can then se ect from the TVP just ke a regu ar tab e or v ew and thus use t as the source for an INSERT INTO…SELECT statement that appends each row to the Product. Location tab e Rows added to Product.Location requ re more than just the two fie ds for the ocat on name and cost rate The tab e a so needs va ues for the ava ab ty and mod fied date fie ds, wh ch you’ et the stored procedure hand e What you’re do ng here s defin ng a schema that can prov de a subset of the requ red Product.Location fie ds (Name and CostRate), for pass ng mu t p e rows of data to a stored procedure that prov des va ues for the rema n ng requ red fie ds (Availability and ­ModifiedDate) In the examp e, the stored procedure sets Availability to 0 and ModifiedDate to the



Chapter 2  T SQL Enhancements   49

GETDATE funct on on each row of data nserted from the TVP (passed n as the on y parameter) that prov des the va ues for Name and CostRate, as shown n L st ng 2-4 Listing 2-4  Creat ng a stored procedure to perform a bu k nsert us ng a TVP dec ared as the LocationUdt

tab e type.

CREATE PROCEDURE uspInsertProductionLocation (@TVP LocationUdt READONLY) AS INSERT INTO [Production].[Location] ([Name], [CostRate], [Availability], [ModifiedDate]) SELECT *, 0, GETDATE() FROM @TVP

You now have a stored procedure that can accept a TVP conta n ng a set of rows w th ocat on names and cost rates to be nserted nto the Production.Location tab e, and that sets the ava ab ty quant ty and mod fied date on each nserted row—a ach eved w th a s ng e parameter and a s ng e INSERT INTO…SELECT statement! The procedure doesn’t know or care how the ca er popu ates the TVP before t s used as the source for the INSERT INTO…SELECT statement For examp e, the ca er cou d manua y add one row at a t me, as fo ows DECLARE @LocationTvp AS LocationUdt INSERT INTO @LocationTvp VALUES('UK', 122.4) INSERT INTO @LocationTvp VALUES('Paris', 359.73) EXEC uspInsertProductionLocation @LocationTvp

Or the ca er cou d bu k nsert nto the TVP from another source tab e us ng INSERT INTO…SELECT, as n the next examp e You w fi the TVP from the ex st ng Person.StateProvince tab e us ng the tab e’s Name co umn for LocationName and the va ue 0 for CostRate Pass ng th s TVP to the stored procedure w resu t n a new set of rows added to Production.Location w th the r Name fie ds set ­accord ng to the names n the Person.StateProvince tab e, the r CostRate and Availability va ues set to 0, and the r ModifiedDate va ues set by GETDATE, as shown here DECLARE @LocationTVP AS LocationUdt INSERT INTO @LocationTVP SELECT [Name], 0.00 FROM [Person].[StateProvince] EXEC uspInsertProductionLocation @LocationTVP

The TVP cou d a so be popu ated on the c ent us ng ADO NET, as you’ earn n the next sect on Bu k updates (and de etes) us ng TVPs are poss b e as we You can create an UPDATE statement by jo n ng a TVP (wh ch you must a as) to the tab e you want to update The rows updated n the tab e are determ ned by the matches jo ned to by the TVP and can be set to new va ues that are a so conta ned n the TVP For examp e, you can pass a TVP popu ated w th category IDs and names for

50  Part 1  Core SQL Server Development

updat ng a Category tab e n the database, as shown n L st ng 2-5 By jo n ng the TVP to the Category tab e on the category ID, the uspUpdateCategories stored procedure can update a the match ng rows n the Category w th the new category names passed n the TVP Listing 2-5  Bu k updates us ng TVPs.

CREATE TABLE Category (Id int PRIMARY KEY, Name nvarchar(max), datetime2(0) DEFAULT SYSDATETIME()) CreatedAt -- Initialize with a few INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id,

categories Name) VALUES(1, Name) VALUES(2, Name) VALUES(3, Name) VALUES(4, Name) VALUES(5, Name) VALUES(6,

'Housewares') 'Maternity') 'Mens Apparel') 'Womens Apparel') 'Bath') 'Automotive')

-- View the list of categories SELECT * FROM Category -- Will be used by uspUpdateCategories to pass in a set of category updates CREATE TYPE EditedCategoriesUdt AS TABLE (Id int PRIMARY KEY, Name nvarchar(max)) GO -- Receive multiple rows for the change set and update the Category table CREATE PROCEDURE uspUpdateCategories(@EditedCategoriesTVP AS EditedCategoriesUdt READONLY) AS BEGIN -- Update names in the Category table by joining on the TVP by ID UPDATE Category SET Category.Name = ec.Name FROM Category INNER JOIN @EditedCategoriesTVP AS ec ON Category.Id = ec.Id END -- Load up a few changes into a new TVP instance (to categories 1 and 5) DECLARE @Edits AS EditedCategoriesUdt INSERT INTO @Edits VALUES(1, 'Gifts & Housewares') INSERT INTO @Edits VALUES(5, 'Bath & Kitchen') -- Call the stored procedure EXECUTE uspUpdateCategories @Edits -- View the updated names for categories 1 and 5 in the Category table SELECT * FROM Category



Chapter 2  T SQL Enhancements   51

Passing TVPs Using ADO.NET We’ conc ude our d scuss on of TVPs w th the new SqlDbType.Structured enumerat on n ADO NET You w earn how easy t s to marsha mu t p e rows of data from your c ent app cat on to SQL S­ erver, ­w thout requ r ng mu t p e roundtr ps or mp ement ng spec a og c on the server for p ­ rocess ng the data S mp y prepare an ord nary SqlCommand object (Chapter 10 exp a ns the SqlCommand ­object and ADO NET n deta ), set ts CommandType property to CommandType.­StoredProcedure, and popu ate ts Parameters co ect on w th SqlParameter objects These are the rout ne steps for ­sett ng up a ca to any stored procedure Then, to mark a SqlParameter as a TVP, set ts ­SqlDbType property to ­SqlDbType.Structured Th s ets you spec fy any DataTable, DbDataReader, or IEnumerable object as the parameter va ue to be passed to the stored procedure n a s ng e ca to the server In L st ng 2-6, a new customer order s stored n separate Order and OrderDetail DataTable objects w th n a DataSet The two tab es are passed to the SQL Server stored procedure you saw ear er ­(L st ng 2-2 n the sect on “Subm tt ng Orders”), wh ch accepts them as TVPs for nsert on nto the Order and OrderDetail database tab es Note that the C# code st ngs n th s chapter are fragments of a arger so ut on, abbrev ated to conserve space A fu y funct ona so ut on s supp ed w th the samp e code for th s chapter that you can down oad from the book’s compan on webs te (see the “Introduct on” for deta s) The code for L st ng 2-6 s part of the TVPsW thDataTab e project n the TSq Enhancements so ut on Listing 2-6 Pass ng DataTable objects as TVPs to a SQL Server stored procedure from ADO.NET.

// Assumes conn is an open SqlConnection object and ds is // a DataSet with an Order and OrderDetails table using(conn) { // Create the command object to call the stored procedure SqlCommand cmd = new SqlCommand("uspInsertNewOrder", conn); cmd.CommandType = CommandType.StoredProcedure; // Create the parameter for passing the Order TVP SqlParameter headerParam = cmd.Parameters.AddWithValue ("@OrderHeaderTvp", ds.Tables["OrderHeader"]); // Create the parameter for passing the OrderDetails TVP SqlParameter detailsParam = cmd.Parameters.AddWithValue ("@OrderDetailsTvp", ds.Tables["OrderDetail"]); // Set the SqlDbType of the parameters to Structured headerParam.SqlDbType = SqlDbType.Structured; detailsParam.SqlDbType = SqlDbType.Structured; // Execute the stored procedure cmd.ExecuteNonQuery(); }

52  Part 1  Core SQL Server Development

Th s code ca s a SQL Server stored procedure and passes to t an order header and the comp ete set of order deta s n a s ng e roundtr p Remarkab y, t’s just as s mp e as that You just need to ­ensure that the schema of the DataTable objects match the correspond ng TVP-tab e-type schema You can a so send a set of rows d rect y to a parameter zed SQL statement w thout creat ng a stored procedure Because the SQL statement s dynam ca y constructed on the c ent, there s no stored procedure s gnature that spec fies the name of the user-defined tab e type for the TVP ­Therefore, you need to te ADO NET what the type s by sett ng the TypeName property to the name of the tab e type defined on the server For examp e, the code n L st ng 2-7 (part of the ­T VPAdd t ona Samp es project n the TSq Enhancements so ut on for th s chapter) passes a DataTable to a parameter zed SQL statement Listing 2-7  Pass ng TVPs to a parameter zed SQL statement from ADO.NET.

// Define the INSERT INTO...SELECT statement to insert into Categories const string TSqlStatement = @" INSERT INTO Categories (CategoryID, CategoryName) SELECT nc.CategoryID, nc.CategoryName FROM @NewCategoriesTvp AS nc"; // Assumes conn is an open SqlConnection object and ds is // a DataSet with a Category table using(conn) { // Set up the command object for the statement SqlCommand cmd = new SqlCommand(TSqlStatement, conn); // Add a TVP specifying the DataTable as the parameter value SqlParameter catParam = cmd.Parameters.AddWithValue ("@NewCategoriesTvp", ds.Tables["Category"]); catParam.SqlDbType = SqlDbType.Structured; catParam.TypeName = "dbo.CategoriesUdt"; // Execute the command cmd.ExecuteNonQuery(); }

Sett ng the TypeName property to dbo.CategoriesUdt n th s code means that you have a ­ ser-defined tab e type by that name on the server, created us ng the CREATE TYPE…AS TABLE u ­statement that defines the CategoryID and CategoryName co umns You can a so use any object der ved from DbDataReader (that s, a connected data source) to stream rows of data to a TVP The examp e shown n L st ng 2-8 (part of the ­T VPAdd t ona Samp es project n the TSq Enhancements so ut on for th s chapter) ca s an Orac e stored procedure to se ect data from an Orac e database nto a connected OracleDataReader The reader object gets passed as a s ng e tab e-va ued nput parameter to a SQL Server stored



Chapter 2  T SQL Enhancements   53

­ rocedure, wh ch can then use the Orac e data n the reader as the source for add ng new rows p nto the Category tab e n the SQL Server d ­ atabase (for th s code to work, you must have the ADO NET Prov der for Orac e nsta ed and a ­project reference set to System.Data.OracleClient, as we as access to an Orac e database server) Listing 2-8  Pass ng a connected OracleDataReader source as a TVP to SQL Server.

// Set up command object to select from Oracle OracleCommand selCmd = new OracleCommand ("SELECT CategoryID, CategoryName FROM Categories;", oracleConn); // Execute the command and return the results in a connected // reader that will automatically close the connection when done OracleDataReader rdr = selCmd.ExecuteReader(CommandBehavior.CloseConnection); // Set up command object to insert to SQL Server SqlCommand insCmd = new SqlCommand("uspInsertCategories", connection); insCmd.CommandType = CommandType.StoredProcedure; // Add a TVP specifying the reader as the parameter value SqlParameter catParam = cmd.Parameters.AddWithValue("@NewCategoriesTvp", rdr); catParam.SqlDbType = SqlDbType.Structured; // Execute the stored procedure insertCommand.ExecuteNonQuery();

Passing Collections to TVPs Using Custom Iterators What f you’re work ng w th co ect ons popu ated w th bus ness objects rather than DataTable ­objects popu ated w th DataRow objects? You m ght not th nk at first that bus ness objects and TVPs cou d work together, but the fact s that they can—and qu te gracefu y, too A you need to do s ­ mp ement the IEnumerable nterface n your co ect on c ass Th s nterface requ res your co ect on c ass to supp y a custom terator method named GetEnumerator, wh ch ADO NET w ca for each object conta ned n the co ect on when you nvoke ExecuteNonQuery Let’s demonstrate w th the same order entry examp e as L st ng 2-6, on y now you’ use ord nary bus ness object co ect ons (rather than DataTable and DataRow objects) as the source for the TVPs You have OrderHeader and OrderDetail c asses w th propert es to match the correspond ng TVP-tab e-type schemas, as shown n L st ng 2-9 Of course, a rea mp ementat on cou d nc ude much more than th s s mp e data transfer object, such as encapsu ated bus ness og c The code n L st ngs 2-9 and 2-10 s part of the TVPsW thB zCo ect on project n the TSq Enhancements so ut on for th s chapter

54  Part 1  Core SQL Server Development

Listing 2-9  Defin ng c asses for pass ng bus ness objects as TVPs.

public class OrderHeader { public int OrderId { get; set; } public int CustomerId { get; set; } public DateTime OrderedAt { get; set; } } public class OrderDetail { public int OrderId { get; set; } public int LineNumber { get; set; } public int ProductId { get; set; } public int Quantity { get; set; } public decimal Price { get; set; } }

Ord nar y, work ng w th List and List objects m ght be a ­su tab e opt on for conta n ng co ect ons of OrderHeader and OrderDetail objects n your app cat on But n these co ect ons they won’t suffice on the r own as nput va ues for TVPs because List does not mp ement IEnumerable The so ut on s to create OrderHeaderCollection and ­OrderDetailCollection c asses that nher t List and List respect ve y, and then mp ement IEnumerable n order to “TVP-enab e” them as shown n L st ng 2-10 Listing 2-10  Defin ng co ect on c asses w th custom terators for pass ng bus ness objects as TVPs.

public class OrderHeaderCollection : List, IEnumerable { IEnumerator IEnumerable.GetEnumerator() { var sdr = new SqlDataRecord( new SqlMetaData("OrderId", SqlDbType.Int), new SqlMetaData("CustomerId", SqlDbType.Int), new SqlMetaData("OrderedAt", SqlDbType.Date)); foreach (OrderHeader oh in this) { sdr.SetInt32(0, oh.OrderId); sdr.SetInt32(1, oh.CustomerId); sdr.SetDateTime(2, oh.OrderedAt); yield return sdr; } } }



Chapter 2  T SQL Enhancements   55

public class OrderDetailCollection : List, IEnumerable { IEnumerator IEnumerable.GetEnumerator() { var sdr = new SqlDataRecord( new SqlMetaData("OrderId", SqlDbType.Int), new SqlMetaData("LineNumber", SqlDbType.Int), new SqlMetaData("ProductId", SqlDbType.Int), new SqlMetaData("Quantity", SqlDbType.Int), new SqlMetaData("Price", SqlDbType.Money)); foreach (OrderDetail od in this) { sdr.SetInt32(0, od.OrderId); sdr.SetInt32(1, od.LineNumber); sdr.SetInt32(2, od.ProductId); sdr.SetInt32(3, od.Quantity); sdr.SetDecimal(4, od.Price); yield return sdr; } } }

We’ on y exp a n the OrderHeaderCollection c ass; you’ be ab e to nfer how the ­ rderDetailCollection c ass—or any of your own co ect on c asses— mp ements the custom terator O needed to support TVPs F rst, aga n, t nher ts List, so an OrderHeaderCollection object s everyth ng that a List object s Th s means mp c t y, by the way, that t a so mp ements IEnumerable, wh ch s what makes any sequence “foreach-ab e” or “LINQ-ab e ” But to the heart of our d scuss on, t exp c t y mp ements IEnumerable, wh ch means t has a custom iterator method for ADO NET to consume when an nstance of th s co ect on c ass s ­ass gned to a SqlDbType.Structured parameter for p p ng over to SQL Server w th a TVP Every enumerable c ass requ res a match ng enumerator method, so not surpr s ng y ­mp ement ng IEnumerable requ res prov d ng a GetEnumerator method that returns an IEnumerator Th s method first n t a zes a new SqlDataRecord object w th a ­schema that matches the tab e-type schema that the TVPs are dec ared as It then enters a oop that ­terates a the e ements n the co ect on (poss b e because List mp c t y ­mp ements IEnumerable) On the first terat on, t sets the co umn property va ues of the ­SqlDataRecord object to the property va ues of the first OrderHeader e ement, and then ssues the mag c yield return statement By defin t on, any method ( ke th s one) wh ch returns IEnumerator and has a yield return statement n t s a custom terator method; t s expected to return a ­sequence of T objects unt the method execut on path comp etes ( n th s case, when the foreach oop fin shes)

56  Part 1  Core SQL Server Development

The crux of th s s that you are never ca ng th s method d rect y Instead, when you ­nvoke ExecuteNonQuery to run a stored procedure w th a SqlDbType.Structured parameter (that s, a TVP), ADO NET expects the co ect on passed for the parameter va ue to mp ement IEnumerable so that IEnumerable.GetEnumerator can be ca ed ­nterna y to fetch each new record for p p ng over to the server When the first e ement s fetched from the co ect on, GetEnumerator s entered, the SqlDataRecord s n t a zed, and s then ­popu ated w th va ues us ng the SetInt32 and SetDateTime methods (there’s a SetXXX method for each data type) That SqlDataRecord “row” s then pushed nto the p pe ne to the server by yield return When the next e ement s fetched from the co ect on, the GetEnumerator method resumes from the po nt that t yield returned the prev ous e ement, rather than enter ng GetEnumerator aga n from the top Th s means the SqlDataRecord gets n t a zed w th schema nformat on on y once, wh e ts popu at on w th one e ement after another s orchestrated by the contro ng ADO NET code for ­E xecuteNonQuery that actua y sh ps one SqlDataRecord after another to the server The actua code to ca the stored procedure s 100 percent dent ca to the code n L st ng 2-6 that uses a DataTable rather than a co ect on Subst tut ng a co ect on for a DataTable object requ res no code changes and works flaw ess y, prov ded the co ect on mp ements IEnumerator Th s mean the co ect on has a GetEnumerator method that feeds each object nstance to a ­SqlDataRecord and maps each object property to a co umn defined by the user-defined tab e type that the TVP s dec ared as

TVP Limitations TVPs have severa noteworthy m tat ons F rst and foremost, once TVPs are n t a y popu ated and passed, they are read-on y structures The READONLY keyword must be app ed to TVPs n the ­s gnatures of your stored procedures, or they w not comp e S m ar y, the OUTPUT keyword cannot be used You cannot update the co umn va ues n the rows of a TVP, and you cannot nsert or de ete rows If you must mod fy the data n a TVP, you need to mp ement a workaround, such as nsertng data from the TVP nto a temporary tab e or nto a tab e var ab e to wh ch you can then app y changes There s no ALTER TABLE…AS TYPE statement that supports chang ng the schema of a TVP tab e type Instead, you must first drop a stored procedures and UDFs that reference the type before you can drop the type, re-create t w th a new schema, and then re-create the stored procedures Indexng s m ted as we , w th support on y for PRIMARY KEY and UNIQUE constra nts A so, stat st cs on TVPs are not ma nta ned by SQL Server It’s a so mportant to note that the Ent ty Framework (wh ch we cover n Chapter 10) does not support TVPs However, you can eas y send any co ect on of Ent ty Framework ent t es to a stored procedure that accepts TVPs by cod ng a custom terator just ke the one you created n the ­sect on “Pass ng Co ect ons to TVPs Us ng Custom Iterators ” Th s techn que s demonstrated n the ­T VPsW thEF project of the TSq Enhancements so ut on that you can down oad from the book’s ­compan on webs te



Chapter 2  T SQL Enhancements   57

Date and Time Data Types The date, time, datetime2, and datetimeoffset types are four date and t me data types that were ntroduced n SQL Server 2008 These types shou d be used for a new database deve opment n eu of the trad t ona datetime and smalldatetime data types The newer types are a gned w th the NET Framework, M crosoft W ndows, and the SQL standard—un ke datetime and smalldatetime—and there are mprovements n range, prec s on, and storage as we

Separation of Dates and Times We’ beg n by ook ng at the separate date and time types If you need to store on y a date va ue (for examp e, a date of b rth), use the date type S m ar y, use the time type for stor ng just a t me va ue (for examp e, a da y med cat on t me), as shown here DECLARE @DOB date DECLARE @MedsAt time

The datetime and smalldatetime types, wh ch were the on y prev ous y ava ab e opt ons, each nc ude both a date and a t me port on In cases where on y the date or on y the t me s needed, the extraneous port on consumes storage need ess y, wh ch resu ts n wasted space n the database In ­add t on to sav ng storage, us ng date rather than datetime y e ds better performance for date-on y man pu at ons and ca cu at ons, because there s no t me port on to be hand ed or cons dered And the time type s prov ded for those ess common cases when you requ re a t me w thout a date; for ­examp e, feed ng t me at the zoo

More Portable Dates and Times To store both a date and a t me as a s ng e va ue, use the datetime2 data type Th s data type supports the same range of va ues as the DateTime data type n the NET Framework, so t can store dates from 1/1/0001 (DateTime.MinValue n NET) to 12/31/9999 (DateTime.MaxValue n NET) n the Gregor an ca endar Contrast th s w th the a owab e date va ues for the o d datetime type, wh ch range on y from 1/1/1753 to 12/31/9999 Th s means that dates n NET from 1/1/0001 through 12/31/1752 can’t be stored at a n SQL Server’s datetime type, a prob em so ved by us ng e ther the date, datetime2, or datetimeoffset type Because the supported range of dates s the same n both NET and SQL Server, any date can be safe y passed between these c ent and server p atforms w th no concern You are strong y encouraged to d scont nue us ng the o der datetime and smalldatetime data types and to use on y date and datetime2 types for new deve opment (or the new datetimeoffset type for t me zone awareness, wh ch s d scussed next) There has a so been a need for greater prec s on of fract ona seconds n t me va ues The ­datetime type s accurate on y w th n rough y 3 33 m seconds, whereas t me va ues n W ndows and NET have a s gn ficant y greater 100-nanosecond (10-m onth of a second) accuracy (The ­smalldatetime type doesn’t even support seconds and s accurate on y to the m nute ) Stor ng t mes n the ­database therefore resu ts n a oss of prec s on L ke the expanded range of supported dates, the new time, datetime2, and datetimeoffset types are now more a gned w th NET and other p atforms by a so 58  Part 1  Core SQL Server Development

­ rov d ng the same 100 nanosecond accuracy As a resu t, you no onger ncur any data oss of p ­fract ona second accuracy between p atforms when record ng t me va ues to the database Of course, there are storage mp cat ons that come w th greater t me prec s on, and we’ d scuss those ­momentar y

Time Zone Awareness The fourth and ast data type n th s category s datetimeoffset Th s type defines a date and t me w th the same range and prec s on that datetime2 prov des but a so nc udes an offset va ue w th a range of –14 00 to +14 00 that dent fies the t me zone In the past, the on y pract ca approach for ­g oba zat on of dates and t mes n the database has been to store them n Coord nated Un versa T me (UTC) format Do ng th s requ res back-and-forth convers on between UTC and oca t me that must be hand ed at the app cat on eve , and that means wr t ng code Us ng the datetimeoffset type, you can store va ues that represent the oca date and t me n ­ fferent reg ons of the wor d and nc ude the appropr ate t me zone offset for the reg on n each d va ue Because the t me zone offset embedded n the date and t me va ue s spec fic to a part cu ar ­oca e, SQL Server s ab e to perform date and t me compar sons between d fferent oca es w thout any c­ onvers on efforts requ red on your part A though datetimeoffset va ues appear to go n and come out as dates and t mes oca to a part cu ar reg on, they are nterna y converted, stored, and treated n UTC format for compar sons, sort ng, and ndex ng, wh e the t me zone “tags a ong ” Ca cu at ons and compar sons are therefore performed correct y and cons stent y across a dates and t mes n the database regard ess of the d fferent t me zones n var ous reg ons By s mp y ­append ng the t me zone offset to datetimeoffset va ues, SQL Server hand es the convers ons to and from UTC for you automat ca y n the background Even better, you can obta n a datetimeoffset va ue e ther as UTC or oca t me For those of you bu d ng databases that need to store var ous oca t mes (or even just dates) n d fferent reg ons of the wor d, th s s an extreme y conven ent feature The ­database hand es a the deta s, so the app cat on deve oper doesn’t have to T me zone funct ona ty s s mp y ava ab e for free r ght at the database eve For examp e, the database knows that 9 15 AM n New York s n fact ater than 10 30 AM n Los Ange es f you store the va ues n a datetimeoffset data type w th appropr ate t me zone ­offsets ­Because the New York t me spec fies a t me zone offset of –5 00 and the Los Ange es t me has an offset of –8 00, SQL Server s aware of the three-hour d fference between the two t me zones and accounts for that d fference n a date/t me man pu at ons and ca cu at ons Th s behav or s ­demonstrated by the code n L st ng 2-11 Listing 2-11  T me zone ca cu at ons us ng datetimeoffset.

DECLARE @Time1 datetimeoffset DECLARE @Time2 datetimeoffset DECLARE @MinutesDiff int



Chapter 2  T SQL Enhancements   59

SET @Time1 = '2012-02-10 09:15:00-05:00' SET @Time2 = '2012-02-10 10:30:00-08:00'

-- NY time is UTC -05:00 -- LA time is UTC -08:00

SET @MinutesDiff = DATEDIFF(minute, @Time1, @Time2) SELECT @MinutesDiff

If you run th s code, you w see that the DATEDIFF ca cu at on returns 255 Th s nd cates that SQL Server s c ear y ab e to account for the three-hour d fference n the t me zones Because 10 30 AM n Los Ange es s actua y 1 30 PM n New York, a d fference of 255 m nutes (4 hours and 15 m nutes) between that t me and 9 15 AM New York t me was ca cu ated correct y

Note  Time zone names are not supported, nor is there support for daylight savings time. Unfortunately, these features did not make it into the final release of the product, but they are on the list for the next version of SQL Server. Time zones can be expressed only by hour/minute offsets, and you must continue to handle daylight savings time considerations on your own.

Date and Time Accuracy, Storage, and Format Date va ues stored n date, datetime2, and datetimeoffset types are compacted nto a fixed storage space of 3 bytes They use an opt m zed format that s 1 byte ess than the 4 bytes consumed by the date port on of the o der datetime type (support ng a greater range n a sma er space) T me va ues stored n time, datetime2, and datetimeoffset types, by defau t, consume 5 bytes of storage to support the same 100-nanosecond accuracy as W ndows and NET However, you can spec fy a ower degree of prec s on for even more compacted storage by prov d ng an opt ona sca e parameter when dec ar ng time, datetime2, and datetimeoffset var ab es The sca e can range from 0 to 7, w th 0 ­offer ng no fract ona -second prec s on at a consum ng 3 bytes, and 7 (the defau t) offer ng the g ­ reatest fract ona -second prec s on (100 nanoseconds) consum ng 5 bytes The sca e ­essent a y d ctates the ­number of d g ts supported after the dec ma po nt of the seconds va ue, where a sca e va ue of 7 supports a fract ona prec s on of 100 nanoseconds (each 100 nanoseconds be ng 0 0000001 second) The defau t sca e s 7, wh ch offers the greatest prec s on (to 100 nanoseconds) n the argest space (5 bytes) Th s means that dec ar ng a var ab e as time, datetime2, or datetimeoffset s the same as ­dec ar ng t as time(7), datetime2(7), or datetimeoffset(7), mak ng the fo ow ng two statements equ va ent DECLARE @StartDateTime datetime2 DECLARE @StartDateTime datetime2(7)

If you don’t requ re any fract ona prec s on at a , use a sca e of 0, as n the fo ow ng statement, wh ch consumes on y 3 bytes to store a t me n @FeedingTime DECLARE @FeedingTime time(0)

60  Part 1  Core SQL Server Development

Two t me va ues w th d ffer ng sca es are perfect y compat b e w th each other for compar son SQL Server automat ca y converts the va ue w th the ower sca e to match the va ue w th the greater sca e and compares the two safe y V rtua y a ndustry-standard str ng tera formats are supported for conven ent y represent ng dates and t mes For examp e, the date May 15, 2012, can be expressed n any of the formats shown n Tab e 2-1 Table 2-1  Common va d date and t me str ng tera formats Format

Example

Numer c

5/15/2012, 15 05 2012, 05.15.2012

A phabet ca

May 15, 2012

SO8601

2012 05 15, 201205153

ODBC

{d 2012 05 15 }

W3C XML

2012 05 15Z

You have s m ar flex b ty for represent ng t mes For examp e, the same t me va ue can be ­expressed as 23 30, 23 30 00, 23 30 00 0000, or 11 30 00 PM T me zone offsets are expressed mere y by ­append ng a p us or m nus s gn fo owed by the UTC hours and m nutes for the zone—for ­examp e, +02 00 for Jerusa em

Note  There are even more possible formatting variations than those we’ve indicated here, where the purpose has been to convey how accommodating SQL Server is with respect to variations in date and time syntax, rather than to duplicate content from SQL Server Books Online (which you can easily refer to on your own for a complete specification of supported string literal formats for dates and times). In addition, SQL Server 2012 provides several new T-SQL functions (covered later in the chapter in the section "New T-SQL Functions in SQL Server 2012") for parsing and creating culture-specific and culture-neutral dates. You can use CAST or CONVERT to extract just the date or t me port ons of a datetime2 co umn for search ng When you perform such a convers on on a datetime2 co umn that s ndexed, SQL Server does not need to resort to a sequent a tab e scan and s ab e to perform the much faster ndex seek to ocate the spec fic date or t me For examp e, the fo ow ng code defines a tab e w th a datetime2 type that has a c ustered ndex Se ect ng by date or t me on y can be ach eved us ng CONVERT, wh e st us ng the c ustered ndex for effic ent search ng, as shown n L st ng 2-12 Listing 2-12 Us ng CONVERT to extract the date and t me port on from a datetime2 co umn.

CREATE TABLE DateList(MyDate datetime2) CREATE CLUSTERED INDEX idx1 ON DateList(MyDate)



Chapter 2  T SQL Enhancements   61

-- Insert some rows into DateList INSERT INTO DateList VALUES ('2011-10-10 12:15:00'), ('2012-04-07 09:00:00'), ('2012-04-07 10:00:00'), ('2011-10-10 09:00:00') SELECT MyDate FROM DateList WHERE MyDate = '2011-10-10 12:15:00' SELECT MyDate FROM DateList WHERE CONVERT(time(0), MyDate) = '09:00:00' SELECT MyDate FROM DateList WHERE CONVERT(date, MyDate) = '2012-04-07'

If you request the est mated execut on p an for th s code, you w see that the c ustered ndex s ­everaged when us ng CONVERT to query e ther the date or t me port on of the ndexed datetime2 ­co umn

Date and Time Functions A of the trad t ona date-re ated and t me-re ated funct ons, nc ud ng DATEADD, DATEDIFF, ­DATEPART, and DATENAME, of course fu y support the newer date and t me data types, and severa ­funct ons have been added as we We conc ude our d scuss on of the SQL Server 2008 date and t me data types by exp or ng the T-SQL extens ons added to support them A few add t ona handy date and t me funct ons were added n SQL Server 2012, wh ch we cover n the sect on on new SQL Server 2012 funct ons ater n the chapter The SYSDATETIME and SYSUTCDATETIME funct ons return the date and t me on the server as ­datetime2 types (w th fu seven-sca e prec s on accuracy w th n 100 nanoseconds), just as the ­GETDATE and GETUTCDATE funct ons cont nue to return the current date and t me as datetime types Another new funct on, SYSDATETIMEOFFSET, returns the date and t me on the server as a datetimeoffset type, w th a t me zone offset reflect ng the reg ona sett ngs estab shed on the server, wh ch nc udes awareness of oca day ght sav ngs t me The code n L st ng 2-13 shows the contrast between the var ous s m ar server date and t me funct ons Listing 2-13  Compar ng server date and t me funct ons.

SET NOCOUNT ON SELECT GETDATE() AS 'GETDATE() datetime' SELECT GETUTCDATE() AS 'GETUTCDATE() datetime' SELECT SYSDATETIME() AS 'SYSDATETIME() datetime2' SELECT SYSUTCDATETIME() AS 'SYSUTCDATETIME() datetime2' SELECT SYSDATETIMEOFFSET() AS 'SYSDATETIMEOFFSET() datetimeoffset'

Runn ng th s code just after 8 20 PM on February 10, 2012 n New York resu ts n the fo ow ng output GETDATE() datetime ----------------------2012-02-10 20:21:19.380

62  Part 1  Core SQL Server Development

GETUTCDATE() datetime ----------------------2012-02-11 01:21:19.380 SYSDATETIME() datetime2 --------------------------2012-02-10 20:21:19.3807984 SYSUTCDATETIME() datetime2 --------------------------2012-02-11 01:21:19.3807984 SYSDATETIMEOFFSET() datetimeoffset ---------------------------------2012-02-10 20:21:19.3807984 -05:00

The TODATETIMEOFFSET and SWITCHOFFSET funct ons a ow you to perform t me zone offset man pu at ons TODATETIMEOFFSET w convert any date or t me type (that has no t me zone offset) to a datetimeoffset type by app y ng whatever t me zone offset you prov de ­SWITCHOFFSET makes t easy to find out what the same t me s n two d fferent t me zones You prov de the ­datetimeoffset for a source ocat on and a t me zone offset for a target ocat on, and S­ WITCHOFFSET returns a ­datetimeoffset represent ng the equ va ent date and t me n the target ­ocat on, as shown n L st ng 2-14 Listing 2-14  Perform ng t me zone offset man pu at ons us ng TODATETIMEOFFSET and SWITCHOFFSET.

DECLARE @TheTime datetime2 DECLARE @TheTimeInNY datetimeoffset DECLARE @TheTimeInLA datetimeoffset -- Hold a time that doesn't specify a time zone SET @TheTime = '2012-02-10 7:35PM' -- Convert it into one that specifies time zone for New York SET @TheTimeInNY = TODATETIMEOFFSET(@TheTime, '-05:00') -- Calculate the equivalent time in Los Angeles SET @TheTimeInLA = SWITCHOFFSET(@TheTimeInNY , '-08:00') SELECT @TheTime AS 'Any Time' SELECT @TheTimeInNY AS 'NY Time' SELECT @TheTimeInLA AS 'LA Time'

Here s the output resu t Any Time --------------------------2012-02-10 19:35:00.0000000 NY Time ---------------------------------2012-02-10 19:35:00.0000000 -05:00



Chapter 2  T SQL Enhancements   63

LA Time ---------------------------------2012-02-10 16:35:00.0000000 -08:00

You can use TODATETIMEOFFSET w th INSERT INTO…SELECT to bu k- nsert date and t me va ues w th no t me zone nformat on from a source tab e nto a target tab e and to app y a t me zone offset to produce datetimeoffset va ues n the target tab e For examp e, the fo ow ng code cop es a the row va ues from the dt2 co umn n tab e test1 (of type datetime2, wh ch has no t me zone nformat on) nto the dto co umn n test2 (of type datetimeoffset) and app es a t me zone offset of –05 00 to each cop ed va ue INSERT INTO test2(dto) SELECT TODATETIMEOFFSET(dt2, '-05:00') FROM test1

The next examp e retr eves a the datetimeoffset va ues from the dto co umn n the test2 tab e, wh ch can nc ude va ues across a var ety of d fferent t me zones Us ng a SWITCHOFFSET funct on that spec fies an offset of –05 00, the va ues are automat ca y converted to New York t me from ­whatever t me zone s stored n the test2 tab e SELECT SWITCHOFFSET(dto, '-05:00') FROM test2

Last, both the ex st ng DATEPART and DATENAME funct ons have been extended to add support for m croseconds (mcs), nanoseconds (ns), and t me zone offsets (tz) n the h gher-prec s on and t me zone-aware types, as shown n L st ng 2-15 Listing 2-15  Us ng the new date port ons n SQL Server 2008 w th DATEPART and DATENAME.

SET NOCOUNT ON DECLARE @TimeInNY datetimeoffset SET @TimeInNY = SYSDATETIMEOFFSET() -- Show the current time in NY SELECT @TimeInNY AS 'Time in NY' -- DATEPART with tz gets the time zone value SELECT DATEPART(tz, @TimeInNY) AS 'NY Time Zone Value' -- DATENAME with tz gets the time zone string SELECT DATENAME(tz, @TimeInNY) AS 'NY Time Zone String' -- Both DATEPART and DATENAME with mcs gets the microseconds SELECT DATEPART(mcs, @TimeInNY) AS 'NY Time Microseconds' -- Both DATEPART and DATENAME with ns gets the nanoseconds SELECT DATEPART(ns, @TimeInNY) AS 'NY Time Nanoseconds'

Runn ng th s code returns the fo ow ng output Time in NY ---------------------------------2012-02-10 20:50:55.7851424 -05:00

64  Part 1  Core SQL Server Development

NY Time Zone Value ------------------300 NY Time Zone String ------------------------------05:00 NY Time Microseconds -------------------785142 NY Time Nanoseconds ------------------785142400

The MERGE Statement The MERGE statement does just what ts name says It comb nes the norma nsert, update, and de ete operat ons nvo ved n a typ ca merge scenar o, a ong w th the se ect operat on that prov des the source and target data for the merge Essent a y, that means t comb nes four statements nto one In fact, you can comb ne five statements nto one us ng the OUTPUT c ause, and even more than that w th INSERT OVER DML (a spec a T-SQL feature syntax, wh ch we cover next) Pr or to SQL Server 2008, separate, mu t p e statements were requ red to ach eve what can now be accomp shed w th a s ng e MERGE statement Th s statement has a flex b e syntax that a ows you to exerc se fine contro over source and target match ng, as we as the var ous set-based DML act ons carr ed out on the target The resu t s s mp er code that’s eas er to wr te and ma nta n (and a so runs faster) than the equ va ent code us ng separate statements to ach eve the same resu t Th s first examp e uses MERGE to manage stocks and trades Beg n by creat ng the two tab es to ho d stocks that you own and da y trades that you make, as shown n L st ng 2-16 Listing 2-16  Creat ng the Stock and Trade tab es.

CREATE TABLE Stock(Symbol varchar(10) PRIMARY KEY, Qty int CHECK (Qty > 0)) CREATE TABLE Trade(Symbol varchar(10) PRIMARY KEY, Delta int CHECK (Delta 0))

You start off w th 10 shares of Adventure Works stock and 5 shares of B ue Yonder A r nes stock These are stored n the Stock tab e INSERT INTO Stock VALUES ('ADVW', 10) INSERT INTO Stock VALUES ('BYA', 5)

Dur ng the day, you conduct three trades You buy 5 new shares for Adventure Works, se 5 shares of B ue Yonder A r nes, and buy 3 shares for your new nvestment n Northw nd Traders These are stored n the Trade tab e, as fo ows INSERT INTO Trade VALUES('ADVW', 5)



Chapter 2  T SQL Enhancements   65

INSERT INTO Trade VALUES('BYA', -5) INSERT INTO Trade VALUES('NWT', 3)

Here are the contents of the two tab es SELECT * FROM Stock GO Symbol ---------ADVW BYA

Qty ----------10 5

(2 row(s) affected)

SELECT * FROM Trade GO Symbol ---------ADVW BYA NWT

Delta ----------5 -5 3

(3 row(s) affected)

At the c os ng of the day, you want to update the quant t es n the Stock tab e to reflect the trades of the day you recorded n the Trade tab e Your Adventure Works stock quant ty has r sen to 15, you no onger own any B ue Yonder A r nes (hav ng so d the on y 5 shares you owned), and you now own 3 new shares of Northw nd Traders stock That’s go ng to nvo ve jo n ng the Stock and Trade tab es to detect changes n stock quant t es resu t ng from your trades, as we as nsert, update, and de ete operat ons to app y those changes back to the Stock tab e A th s og c and man pu at on can be performed w th a s ng e statement us ng MERGE, as shown n L st ng 2-17 Listing 2-17  App y ng trades to stocks us ng MERGE.

MERGE Stock USING Trade ON Stock.Symbol = Trade.Symbol WHEN MATCHED AND (Stock.Qty + Trade.Delta = 0) THEN -- delete stock if entirely sold DELETE WHEN MATCHED THEN -- update stock quantity (delete takes precedence over update) UPDATE SET Stock.Qty += Trade.Delta WHEN NOT MATCHED BY TARGET THEN -- add newly purchased stock INSERT VALUES (Trade.Symbol, Trade.Delta);

66  Part 1  Core SQL Server Development

Let’s d ssect th s statement It beg ns of course w th the MERGE keyword tse f, fo owed by ­USING and ON keywords that respect ve y dent fy the target and source of the merge operat on, and the joining keys used to re ate the source and target to each other for the merge Three merge clauses then fo ow w th the WHEN…THEN syntax, and the statement s then fina y term nated w th a ­sem co on (;)

Important  The statement-terminating semicolon (part of the SQL standard) is usually ­unnecessary in SQL Server. However, the MERGE statement absolutely requires it, and you will receive an error if you omit it.

Defining the Merge Source and Target MERGE has a very e egant mp ementat on n SQL Server At ts core, t operates on a jo n between the source and target of the merge no d fferent y than the way any standard JOIN pred cate n the FROM c ause of any SELECT statement works As you’ see short y when you exam ne SQL Server’s query p an for MERGE, the source, the target, and the jo n between them are hand ed nterna y n exact y the same manner as for a regu ar SELECT The parts of the MERGE syntax that express th s se ect ­operat on nc ude the MERGE keyword tse f, a ong w th USING and ON, wh ch respect ve y spec fy the target, source, and jo n pred cate, as shown n th s sn ppet from L st ng 2-17 MERGE Stock USING Trade ON Stock.Symbol = Trade.Symbol

The target can be any tab e or updateab e v ew and s spec fied mmed ate y fo ow ng the MERGE keyword It s the rec p ent of the changes resu t ng from the merge, wh ch can nc ude ­comb nat ons of new, changed, and de eted rows In th s examp e, the Stock tab e s the target, and t rece ves changes merged nto t from da y trade nformat on (the source) The source s the prov der of the data, wh ch s the Trade tab e n th s examp e, and s spec fied w th the USING keyword r ght after the target The source can be v rtua y anyth ng Th s ­nc udes not on y regu ar tab es and v ews, but subquer es, CTEs, tab e var ab es, remote tab es, ­tab e-va ued ­funct ons (TVFs) and TVPs a ke, and even text fi es accessed w th OPENROWSETBYTES ­Fundamenta y, anyth ng that s va d n the FROM c ause of a SELECT statement s va d as the source of a MERGE statement—noth ng more, noth ng ess The jo n pred cate spec fied by the ON keyword that fo ows defines the co umn key or keys r­ e at ng the source and target to each other, no d fferent y than a standard tab e jo n Aga n, a ­ nyth ng you can put n a SELECT jo n can be spec fied for a MERGE jo n w th ON The jo n defines wh ch records are cons dered match ng or nonmatch ng between the source and target In th s examp e, source and target tab es are re ated by the Symbol co umn The jo n pred cate te s SQL Server what stocks ex st and don’t ex st n both tab es so that you can nsert, update, and de ete data n the target tab e accord ng y The type of jo n ( nner, eft outer, r ght outer, or fu outer) s determ ned by wh ch of the var ous merge c auses are then app ed next n the MERGE statement

Chapter 2  T SQL Enhancements   67

The WHEN MATCHED Clause The prev ous examp e uses three merge c auses two WHEN MATCHED c auses and one WHEN NOT MATCHED BY TARGET c ause Let’s ook at each of them c ose y The first WHEN MATCHED c ause executes when a match ng stock symbo s found n both the Stock and the Trade tab es, as shown n th s code sn ppet WHEN MATCHED AND (Stock.Qty + Trade.Delta = 0) THEN -- delete stock if entirely sold DELETE

A match wou d norma y mean updat ng the quant ty va ue n the Stock tab e by the de ta va ue (amount bought or so d) n the Trade tab e However, n th s scenar o, you want to phys ca y de ete the row n the Stock tab e f ts updated va ue resu ts n 0, because that means you don’t rea y own that part cu ar stock at a anymore (as s the case w th B ue Yonder A r nes) You can code for that scenar o by qua fy ng the WHEN MATCHED c ause w th an add t ona pred cate that tests whether the stock quant ty resu t ng from the trade y e ds 0 Th s g ves you flex b ty to prov de your own cr ter a as pred cates to your merge c auses, and app y custom bus ness og c as fi ters to the var ous match ng cond t ons In th s part cu ar case, you want to remove a row from the Stock tab e us ng the DELETE statement rather than chang ng ts va ue to 0 The next merge c ause s another WHEN MATCHED c ause, but th s second one has no pred cate qua fy ng the match cond t on, as shown n th s code sn ppet WHEN MATCHED THEN -- update stock quantity (delete takes precedence over update) UPDATE SET Stock.Qty += Trade.Delta

Th s second c ause hand es a the other trades of preex st ng stock that have not resu ted n 0 and changes the stock quant ty accord ng y us ng the UPDATE statement (that s, the Stock.Qty va ues w go up or down depend ng on the pos t ve or negat ve number n Trade.Delta) In th s examp e, you want the Adventure Works stock quant ty to go up to 15, reflect ng the 5 shares purchased on top of the 10 you a ready owned

Note  An error would occur if you tried to sell more than you owned. In fact, an ­error would also occur if you tried to sell everything that you owned without first catching that c­ ondition by deleting the stock in the earlier merge clause. That’s because a check ­constraint on the Qty column (defined when you created the Stock table at the beginning of the example) instructs the database not to tolerate any zero or negative Qty values. SQL Server has very part cu ar ru es govern ng the use of mu t p e merge c auses You are ­ erm tted to have one or two WHEN MATCHED c auses—but no more If there are two WHEN p MATCHED c auses, the first one must be qua fied w th an AND cond t on, as th s examp e has shown

68  Part 1  Core SQL Server Development

Furthermore, one c ause must spec fy an UPDATE, and the other must spec fy a DELETE As demonstrated, MERGE w choose one of the two WHEN MATCHED c auses to execute based on whether the AND ­cond t on eva uates to true for any g ven row

The WHEN NOT MATCHED BY TARGET Clause The ast merge c ause s WHEN NOT MATCHED BY TARGET, as shown n th s sn ppet WHEN NOT MATCHED BY TARGET THEN -- add newly owned stock INSERT VALUES (Trade.Symbol, Trade.Delta);

Th s c ause hand es rows found n the source but not n the target Th s refers to stocks that are be ng traded for the first t me, wh ch s the new Northw nd Traders stock that doesn’t yet ex st n the target Stock tab e The c ause has no pred cate (a though t cou d), and so there are no add t ona ­cond t ons for the c ause Here, you s mp y add the new data found n the Trade tab e to the Stock tab e us ng the INSERT statement

Note The BY TARGET keywords are optional for this clause. WHEN NOT MATCHED is equivalent to WHEN NOT MATCHED BY TARGET. On y one WHEN NOT MATCHED BY TARGET c ause s perm tted n a s ng e MERGE statement It can be qua fied w th an AND cond t on, as you saw ear er w th WHEN MATCHED (There s no ­purpose for an AND cond t on on the WHEN NOT MATCHED BY TARGET c ause n th s examp e ) After execut ng the MERGE statement n L st ng 2-17, the Stock tab e s updated to reflect a the trades of the day merged nto t, as shown here SELECT * FROM Stock GO Symbol ---------ADVW NWT

Qty ----------15 3

(2 row(s) affected)

Just as des red and expected, B ue Yonder A r nes s gone, Northw nd Traders has been added, and Adventure Works has been updated Th s s a rather mpress ve resu t for just one statement! It took ess code to wr te and w take ess effort to ma nta n than the equ va ent operat ons wr tten as separate statements wou d, and t a so runs faster because t s comp ed and executed as a s ng e statement No add t ona overhead s ncurred for the s mp e reason that th s statement operates on the same fundamenta pr nc p es as your bas c SELECT statement’s FROM and JOIN c auses



Chapter 2  T SQL Enhancements   69

Using MERGE for Table Replication Let’s move on to another examp e that shows how MERGE can be used as a too for ach ev ng s mp e rep cat on between two tab es F rst, define the tab es Original and Replica w th dent ca schemas Then create a stored procedure w th a MERGE statement that rep cates changes made n the Original tab e over to the Replica tab e, as shown n L st ng 2-18 Listing 2-18  Creat ng two tab es and a stored procedure that uses MERGE to synchron ze them.

CREATE TABLE Original(PK int primary key, FName varchar(10), Number int) CREATE TABLE Replica(PK int primary key, FName varchar(10), Number int) GO CREATE PROCEDURE uspSyncReplica AS MERGE Replica AS R USING Original AS O ON O.PK = R.PK WHEN MATCHED AND (O.FName != R.FName OR O.Number != R.Number) THEN UPDATE SET R.FName = O.FName, R.Number = O.Number WHEN NOT MATCHED THEN INSERT VALUES(O.PK, O.FName, O.Number) WHEN NOT MATCHED BY SOURCE THEN DELETE;

The MERGE statement n th s stored procedure hand es the rep cat on task by jo n ng the two t­ ab es on the r pr mary keys (PK) and prov d ng three merge c auses The first c ause processes ­updates, as shown here WHEN MATCHED AND (O.FName != R.FName OR O.Number != R.Number) THEN UPDATE SET R.FName = O.FName, R.Number = O.Number

Here, WHEN MATCHED s used to find a the records that ex st n both the or g na and the rep ca, and then the UPDATE statement updates the match ng rows on the rep ca s de w th the atest or g na data Perform ng such an update when no data has actua y changed s wastefu , so the pred cate qua fies the merge c ause to app y on y when a row change s detected n any of the nonkey va ues between the or g na and the rep ca The second merge c ause hand es nsert ons, as fo ows WHEN NOT MATCHED THEN INSERT VALUES(O.PK, O.FName, O.Number)

As ment oned ear er, WHEN NOT MATCHED s equ va ent to WHEN NOT MATCHED BY TARGET, wh ch returns a the or g na rows not found n the rep ca tab e These records represent new rows added to the or g na tab e s nce the ast merge, wh ch are now added to the rep ca as we us ng the INSERT statement n th s c ause

70  Part 1  Core SQL Server Development

The WHEN NOT MATCHED BY SOURCE Clause The th rd, and ast, merge c ause hand es de et ons, as shown here, fo owed by the requ red ­sem co on term nator WHEN NOT MATCHED BY SOURCE THEN DELETE;

The WHEN NOT MATCHED BY SOURCE c ause serves as the exact reverse of WHEN NOT MATCHED BY TARGET and returns target (rep ca) rows not found n the source (or g na ) tab e Th s scenar o wou d occur as the resu t of remov ng rows from the or g na tab e after they have been rep cated Rows removed from the or g na tab e need to be removed from the rep ca tab e as we , wh ch s accomp shed by the s mp e DELETE statement n th s c ause Regard ng the use of mu t p e c auses, WHEN NOT MATCHED BY SOURCE has very same ru es as WHEN MATCHED You are perm tted up to two WHEN NOT MATCHED BY SOURCE c auses If you have two, the first one must be qua fied w th an AND cond t on Furthermore, one c ause must spec fy an UPDATE, and the other must spec fy a DELETE If two WHEN NOT MATCHED BY SOURCE c auses are spec fied, MERGE w choose one of them to execute for any g ven row based on whether the AND cond t on eva uates to true Let’s get started w th some data Both tab es are empty at th s po nt (wh ch does n fact mean that they are a ready n sync) Start by fi ng the or g na tab e w th a few rows, as shown here INSERT Original VALUES(1, 'Sara', 10) INSERT Original VALUES(2, 'Steven', 20)

Now ca the uspSyncReplica stored procedure to br ng the rep ca tab e up to date The first t me you ca the stored procedure, on y the WHEN NOT MATCHED c ause w execute It w execute tw ce, actua y, once for each of the two rows to be added to rep ca tab e from the or g na tab e The rows affected count d sp ayed after runn ng the stored procedure conveys th s, as shown here EXEC uspSyncReplica GO (2 row(s) affected)

Exam n ng both tab es now ver fies that the rep ca s synchron zed w th the or g na , as shown here SELECT * FROM Original SELECT * FROM Replica GO PK ----------1 2

FName ---------Sara Steven

Number ----------10 20

(2 row(s) affected)



Chapter 2  T SQL Enhancements   71

PK ----------1 2

FName ---------Sara Steven

Number ----------10 20

(2 row(s) affected)

Now perform some more changes to the or g na tab e A m x of nsert, update, and de ete ­operat ons br ngs the two tab es out of sync aga n INSERT INTO Original VALUES(3, 'Andrew', 100) UPDATE Original SET FName = 'Stephen', Number += 10 WHERE PK = 2 DELETE FROM Original WHERE PK = 1 GO SELECT * FROM Original SELECT * FROM Replica GO PK ----------2 3

FName ---------Stephen Andrew

Number ----------30 100

(2 row(s) affected) PK ----------1 2

FName ---------Sara Steven

Number ----------10 20

(2 row(s) affected)

Invok ng the stored procedure once aga n br ngs the two tab es back n sync by rep cat ng changes from the or g na tab e to the rep ca, as fo ows EXEC uspSyncReplica GO (3 row(s) affected)

Th s t me, you can see that three rows were affected as the resu t of one nsert (Andrew), one update (Stephen), and one de ete (Sara) Exam n ng both tab es ver fies that the rep ca s once aga n synchron zed w th the or g na , as shown here SELECT * FROM Original SELECT * FROM Replica GO PK ----------2 3

FName ---------Stephen Andrew

Number ----------30 100

72  Part 1  Core SQL Server Development

(2 row(s) affected) PK ----------2 3

FName ---------Stephen Andrew

Number ----------30 100

(2 row(s) affected)

MERGE Output The MERGE statement supports the same OUTPUT c ause ntroduced n SQL Server 2005 for the INSERT, UPDATE, and DELETE statements Th s c ause returns change nformat on from each row ­affected by DML operat ons n the same INSERTED and DELETED pseudo-tab es exposed by tr ggers Be ng ab e to capture th s nformat on n the OUTPUT c ause s often a better cho ce than captur ng t n tr ggers, because tr ggers ntroduce nondeterm n st c behav or—that s, you cannot guarantee that mu t p e tr ggers on the same tab e w cons stent y fire n the same order every t me (Th s s often the cause of subt e bugs that can be very d fficu t to track down, and thus tr ggers shou d genera y be avo ded when poss b e ) In add t on to the co umns of the INSERTED and DELETED pseudo-tab e, another v rtua co umn named $action s supported when OUTPUT s used w th the MERGE statement The $action ­co umn w return one of the three str ng va ues—‘INSERT’, ‘UPDATE’, or ‘DELETE’—depend ng on the ­act on taken for each row L st ng 2-19 shows a s ght y mod fied vers on of the uspSyncReplica stored ­procedure that nc udes an OUTPUT c ause on the MERGE statement The OUTPUT c ause se ects the v rtua $action co umn and a the co umns of the INSERTED and DELETED pseudo-tab es so that the MERGE statement can prov de a deta ed report of every DML operat on t performs If you have been fo ow ng a ong w th the steps for the prev ous vers on, you’ need to drop everyth ng (the tab es and the stored procedure) first Then, subst tute the mp ementat on of the stored procedure n L st ng 2-18 w th the code shown n L st ng 2-19 and repeat the same steps Listing 2-19  Us ng the OUTPUT c ause and $action v rtua co umn w th MERGE.

CREATE PROCEDURE uspSyncReplica AS MERGE Replica AS R USING Original AS O ON O.PK = R.PK WHEN MATCHED AND (O.FName != R.FName OR O.Number != R.Number) THEN UPDATE SET R.FName = O.FName, R.Number = O.Number WHEN NOT MATCHED THEN INSERT VALUES(O.PK, O.FName, O.Number) WHEN NOT MATCHED BY SOURCE THEN DELETE OUTPUT $action, INSERTED.*, DELETED.*;



Chapter 2  T SQL Enhancements   73

Runn ng th s mod fied vers on of the MERGE statement for the ast set of changes n the rep cat on examp e produces the fo ow ng output $action ------DELETE UPDATE INSERT

PK ----NULL 2 3

FName ------NULL Stephen Andrew

Number -----NULL 30 100

PK ----1 2 NULL

FName ------Sara Steven NULL

Number -----10 20 NULL

(3 row(s) affected)

Th s output shows a the act ons taken by the merge and a the before-and-after data nvo ved Of course, you can use OUTPUT…INTO nstead of just OUTPUT ke you can w th the INSERT, UPDATE, and DELETE statements to send th s nformat on to another tab e for h story ogg ng or add t ona process ng And conc ud ng our treatment of MERGE, you’ earn about the INSERT OVER DML syntax ntroduced n SQL Server 2008 that prov des even greater flex b ty

Choosing a Join Method SQL Server s very smart about exam n ng MERGE statements, ana yz ng d fferent comb nat ons of merge c auses, and automat ca y perform ng the appropr ate type of jo n to return the source and target data needed on both s des of the operat on There are four types of jo ns, and SQL Server ­c ever y p cks the r ght one for the job based on the merge c auses that t finds n the MERGE ­statement, as shown n Tab e 2-2 Table 2-2  Source-to-target tab e jo n types chosen by SQL Server based on your merge c ause(s) Merge clause

Join type

Returns

Valid actions

WHEN MATCHED

nner

Match ng data n both source and target

Update, de ete

WHEN NOT MATCHED [BY TARGET]

Left outer

Source data not found n target

nsert

WHEN NOT MATCHED BY SOURCE

R ght outer

Target data not found n source

Update, de ete

WHEN NOT MATCHED [BY TARGET] comb ned w th any other c ause

Fu outer

Source data not found n target, and other match ng or nonmatch ng target data

nsert, update, de ete

The WHEN MATCHED c ause returns rows that are found n both tab es (that s, rows n both tab es w th match ng va ues n the jo n ng key co umns defined w th ON) The query processor treats th s as an nner jo n, because you are retr ev ng on y rows that ex st on both s des E ther an UPDATE or a DELETE operat on on match ng target rows s perm tted n the WHEN MATCHED c ause WHEN NOT MATCHED [BY TARGET] s treated as a eft outer jo n, because t retr eves on y source rows that do not ex st n the target Therefore, on y an INSERT can be performed n the WHEN NOT MATCHED [BY TARGET] c ause to “fi the ho e” n the target w th the m ss ng source data

74  Part 1  Core SQL Server Development

The WHEN NOT MATCHED BY SOURCE c ause s the comp ement to WHEN NOT MATCHED [BY TARGET] and resu ts n a r ght outer jo n to retr eve on y target rows that do not ex st n the source As w th WHEN MATCHED, you can perform an UPDATE or a DELETE operat on on those target rows m ss ng from the source Last, comb n ng WHEN NOT MATCHED [BY TARGET] w th e ther or both of the other two c auses retr eves source data not found n the target as we as target data found or not found n the source In th s case, a fu outer jo n s performed between the two tab es You can perform an INSERT n the WHEN NOT MATCHED [BY TARGET] c ause to add source data m ss ng from the target and an ­UPDATE or DELETE n the other c ause(s) to mod fy or remove target data

Important  The only valid statement in WHEN MATCHED or WHEN NOT MATCHED BY SOURCE is UPDATE or DELETE, and the only valid statement in WHEN NOT MATCHED [BY TARGET] is INSERT. No other T-SQL statements (including stored procedure calls) are ­allowed in any of these merge clauses. However, the INSERT and UPDATE statements in a merge clause are permitted to reference user-defined functions.

MERGE DML Behavior As you’ve just earned, the MERGE statement comb nes the four separate DML statements (SELECT, INSERT, UPDATE, and DELETE) nvo ved n a merge operat on The actua nterna ­mp ementat on of MERGE s the same as the d st nct DML statements t encapsu ates Th s was a good ca by ­M crosoft, because t means that deve opers can beg n us ng MERGE r ght away to everage ts mproved ­performance and ower ma ntenance benefits, w thout concern for break ng any ex st ng code A AFTER and INSTEAD OF tr ggers that have a ready been defined n ex st ng tab es or ­ pdateab e v ews cont nue to fire when those tab es or updateab e v ews are des gnated as the u target of a MERGE statement For examp e, the WHEN NOT MATCHED THEN…INSERT c ause fires any nsert tr ggers defined for the target; s m ar y, the WHEN MATCHED THEN…DELETE c ause fires de ete ­tr ggers There s no concept of a “merge tr gger,” so t’s safe for you to mmed ate y start us ng MERGE n your T-SQL code Tr ggers w fire just the same as f you used separate statements nstead of MERGE Ex st ng bus ness og c, constra nts, and ru es a cont nue to funct on as they d d before The same opt ons and features you are accustomed to us ng w th the separate DML statements are a so ava ab e w th MERGE For examp e, you can use a CTE for the source of a merge by defin ng t us ng the WITH c ause before the MERGE statement and then referenc ng t w th the USING c ause The TOP c ause s s m ar y supported, as are trad t ona query h nts And of course, both source and target tab es can be a ased w th AS just ke tab es n regu ar quer es can You can see how SQL Server nterna y mp ements the MERGE statement for the stored procedure that hand es tab e rep cat on by exam n ng the database eng ne’s query execut on p an, shown n F gure 2-2



Chapter 2  T SQL Enhancements   75

Clustered Index Merge [Replica] . [PK—Replica—321507874DE... Cost: 45%

Merge Join (Full Outer Join) Cost: 25%

Clustered Index Scan (Clustered) [Original] . [PK—Original—321507874... Cost: 15%

Clustered Index Scan (Clustered) [Replica] . [PK—Replica—321507874DE... Cost: 15%

Figure 2-2  Query p an for a MERGE operat on to synchron ze a rep ca w th an or g na (screen mage has been cropped to fit the pr nted page).

The work beg ns by scann ng the source and target tab e (query p ans are read from r ght to eft) At the r ght of F gure 2-2, you can see the advantage of hav ng created ndexes on the co umns ­jo n ng the Original and Replica tab es for the merge Because the jo n pred cate uses the pr mary keys of both tab es, the query opt m zer uses a c ustered ndex scan for the best poss b e read ­performance You shou d therefore a ways create an ndex (poss b y c ustered and dea y un que) on the jo ned co umns used for merg ng two tab es The p an a so revea s that a fu outer jo n s performed between the two tab es, as shown n the center of F gure 2-2 As ment oned ear er, the type of jo n used depends on wh ch WHEN…THEN merge c auses are (or aren’t) spec fied n the MERGE statement In th s case, SQL Server performs a fu outer jo n because the MERGE statement uses a three of the poss b e merge c auses The c ustered ndex merge operat on that fo ows (at the eft of F gure 2-2) mp ements the pred cate on the first merge c ause that se ects match ng rows on y f any of the nonkey co umns have changed between the or g na and rep ca tab es The c ustered ndex merge consumes the stream de vered by the fu outer jo n and dec des on a row-by-row bas s whether the row be ng passed to th s operat on shou d be processed as an update, an nsert, or a de ete, based on the syntax of the MERGE statement and the data n the source and target tab es

The INSERT OVER DML Syntax As you just saw, the MERGE statement supports the OUTPUT c ause to access the INSERTED and DELETED pseudo-tab es, as we as a spec a $action str ng va ue that returns ‘INSERT’, ‘UPDATE’, or ‘DELETE’ accord ng to the act on performed for each row processed by the merge operat on The resu ts generated by the OUTPUT c ause can be captured n another tab e or tab e v­ ar ab e us ng OUTPUT…INTO, mak ng t poss b e to eas y ma nta n h stor ca records of changes n the database However, because us ng OUTPUT…INTO dumps every row captured by the c ause nto the dest nat on tab e or tab e var ab e, t s not poss b e to fi ter on th s data

76  Part 1  Core SQL Server Development

A Filterable Alternative to OUTPUT…INTO INSERT OVER DML refers to a spec a syntax n wh ch you wrap an INSERT INTO…SELECT statement around any data man pu at on anguage (DML) statement (INSERT, UPDATE, DELETE, or MERGE) that has an OUTPUT c ause, rather than us ng OUTPUT…INTO on the DML statement tse f The subt e but cruc a d fference here s that OUTPUT…INTO captures the changes, wh e the INSERT OVER DML s­ yntax consumes the changes captured by OUTPUT By treat ng the OUTPUT changes as the source for a standard INSERT INTO…SELECT statement, you can app y any WHERE c ause to the INSERT INTO…SELECT statement that you want Beyond that, there sn’t anyth ng more you can do w th INSERT OVER DML than what you can w th OUTPUT…INTO But to casua y over ook the s gn ficance of fi ter ng change data s to m ss the po nt of INSERT OVER DML ent re y By be ng ab e to fi ter change data w th a WHERE c ause, you can ­better contro wh ch data changes captured are sh pped to the dest nat on tab e and wh ch aren’t The ­benefits of th s capab ty are best rea zed w th an examp e In th s scenar o, you are ma nta n ng an aud t of a changes that are posted from one tab e to another You have a master st of book pr ces and she f ocat ons n the Book tab e where each book s dent fied by ts ISBN number Every week, you rece ve a new tab e named WeeklyChanges that ­conta ns updates to the book data, a so keyed by ISBN number In add t on to pr ce and she f changes for ex st ng books, the week y update tab e can a so nc ude new books You earned a ready that MERGE can hand e th s scenar o ( t was made to, n fact), and can effect ve y app y the appropr ate updates and nserts from the WeeklyChanges tab e to the Book tab e by jo n ng on ISBN and us ng the WHEN MATCHED and WHEN NOT MATCHED c auses You a so know that the OUTPUT…INTO c ause can be used to capture and store a data changes performed by the merge nto a h story tab e To work through the scenar o, start first by us ng OUTPUT…INTO to dump these changes to the ­BookHistory tab e Then you’ mod fy the approach to use INSERT OVER DML nstead, and ga n better contro of the change data arch va process Create the Book and WeeklyChange tab es as shown n L st ng 2-20 Listing 2-20  Creat ng the Book and WeeklyChange tab es.

CREATE TABLE Book( ISBN varchar(20) PRIMARY KEY, Price decimal, Shelf int) CREATE TABLE WeeklyChange( ISBN varchar(20) PRIMARY KEY, Price decimal, Shelf int)

Next, create the BookHistory tab e Th s w be the rec p ent of the changes captured by the ­OUTPUT INTO c ause of the MERGE statement, as shown n L st ng 2-21



Chapter 2  T SQL Enhancements   77

Listing 2-21  Creat ng the BookHistory tab e.

CREATE TABLE BookHistory( Action nvarchar(10), NewISBN varchar(20), NewPrice decimal, NewShelf int, OldISBN varchar(20), OldPrice decimal, OldShelf int, ArchivedAt datetime2)

Now create the uspUpdateBooks stored procedure to perform the week y update merge operat on The stored procedure records a data changes to the BookHistory tab e, as shown n L st ng 2-22 Listing 2-22 Us ng OUTPUT…INTO w th MERGE.

CREATE PROCEDURE uspUpdateBooks AS BEGIN MERGE Book AS B USING WeeklyChange AS WC ON B.ISBN = WC.ISBN WHEN MATCHED AND (B.Price WC.Price OR B.Shelf WC.Shelf) THEN UPDATE SET B.Price = WC.Price, B.Shelf = WC.Shelf WHEN NOT MATCHED THEN INSERT VALUES(WC.ISBN, WC.Price, WC.Shelf) OUTPUT $action, inserted.*, deleted.*, SYSDATETIME() INTO BookHistory; END

Th s code shou d requ re no deta ed exp anat on, as we just covered MERGE thorough y n the prev ous sect on What’s d fferent here, however, s the OUTPUT…INTO c ause Th s c ause records the act on, new va ues, o d va ues, and current server date and t me n the BookHistory tab e, from the $action v rtua co umn, INSERTED pseudo-tab e co umns, DELETED pseudo-tab e co umns, and SYSDATETIME funct on (In th s part cu ar scenar o, $action can return on y the str ng ‘INSERT’ or ­‘UPDATE’, because de et ons are not be ng processed ) Now popu ate the Book and WeeklyChange tab es to set the stage for the update, as shown here INSERT INSERT INSERT INSERT

INTO INTO INTO INTO

Book VALUES('A', 100, 1) Book VALUES('B', 200, 2) WeeklyChange VALUES('A', 101, 1) WeeklyChange VALUES('C', 300, 3)

The WeeklyChange tab e shows a change n pr ce for book A from $100 to $101 and a so adds a new book C pr ced at $300 on she f 3 When you run the uspUpdateBooks stored procedure, those 78  Part 1  Core SQL Server Development

two operat ons take p ace as you’d expect Th s s confirmed by the number of rows affected message, as shown here EXEC uspUpdateBooks GO (2 row(s) affected)

In add t on, the act ons and data changes made by those nsert and update operat ons (and the dates and t mes that they occurred) have been saved to the BookHistory tab e as the resu t of the OUTPUT…INTO c ause, as shown here SELECT * FROM BookHistory GO Action -----UPDATE INSERT

NewISBN ------A C

NewPrice -------101 300

NewShelf -------1 3

OldISBN ------A NULL

OldPrice -------100 NULL

OldShelf -------1 NULL

ArchivedAt --------------------------2012-02-25 14:47:23.9907552 2012-02-25 14:47:23.9907552

So far, so good, r ght? Unt , after further cons derat on, you rea ze that most of your operat ons w be nserts, and there w be very few updates Thus, an unacceptab e amount of storage s go ng to be requ red for captur ng the nserts that have no o d va ues and that therefore don’t prov de any mean ngfu h stor ca va ue beyond the creat on date and t me (wh ch you cou d store n the Book tab e f you wanted to) G ven that every book must be nserted but most books are not typ ca y ­updated, the resu t w be a d sproport onate amount of nserts over updates n the BookHistory tab e That’s a ot of storage to ho d dup cate data that has tt e or no va ue Unfortunate y, there’s s mp y no way to fi ter out the nserts and save just the updates us ng OUTPUT…INTO In our rev sed scenar o, you are nterested n captur ng on y updates w thout nserts Th s requ res fi ter ng, and INSERT OVER DML prov des the so ut on You w use INSERT OVER DML n the next examp e, and—just to keep th ngs nterest ng—the h stor ca data w be appended to the Book tab e tse f, rather than be ng stored to a separate ­h story tab e In the rev sed examp e, there s an ArchivedAt co umn n the Book tab e that conta ns the server date and t me for h story records added when books are updated Thus, a NULL n th s ­co umn ­dent fies the rows ho d ng the current va ues, whereas a other rows w th non-NULL va ues n the ­ArchivedAt co umn represent prev ous va ues n h story rows Start ke you d d the ast t me by creat ng the tab es, as shown n L st ng 2-23 Listing 2-23  Creat ng new vers ons of the Book and WeeklyChange tab es for use w th NSERT OVER DML.

-- Cleanup from previous example DROP TABLE Book DROP TABLE WeeklyChange DROP TABLE BookHistory DROP PROCEDURE uspUpdateBooks GO



Chapter 2  T SQL Enhancements   79

CREATE TABLE Book( ISBN varchar(20), Price decimal, Shelf int, ArchivedAt datetime2) CREATE UNIQUE CLUSTERED INDEX UI_Book ON Book(ISBN, ArchivedAt) CREATE TABLE WeeklyChange( ISBN varchar(20) PRIMARY KEY, Price decimal, Shelf int)

Not ce th s t me that a un que ndex s created on the comb ned ISBN and ArchivedAt co umns A pr mary key cannot be created on just the ISBN co umn as before, because the Book tab e w now conta n mu t p e records w th the same ISBN number (because the tab e ho ds h stor ca change data as we as current data) On y one row per ISBN number w have a NULL va ue n the ­ArchivedAt ­co umn, wh ch s the row that conta ns the current va ues for a book A other rows w th the same ISBN number w have non-NULL va ues n ArchivedAt, wh ch serves as the t me stamp of a ­prev ous update and w conta n the h stor ca va ues arch ved for that book at that t me By ­ensur ng ­un queness between both ISBN and ArchivedAt, you are guaranteed that on y one current row for each book s ever stored n the Book tab e, because the tab e w not to erate two rows w th the same ISBN numbers and NULL va ues for ArchivedAt

Consuming CHANGES You w now mp ement a mod fied vers on of the uspUpdateBooks n L st ng 2-24 that uses the new INSERT OVER DML syntax to capture and store h stor ca data for updates that are appended to the Book tab e Listing 2-24 Us ng MERGE w th NSERT OVER DML.

CREATE PROCEDURE uspUpdateBooks AS BEGIN INSERT INTO Book(ISBN, Price, Shelf, ArchivedAt) SELECT ISBN, Price, Shelf, GETDATE() FROM (MERGE Book AS B USING WeeklyChange AS WC ON B.ISBN = WC.ISBN AND B.ArchivedAt IS NULL WHEN MATCHED AND (B.Price WC.Price OR B.Shelf WC.Shelf) THEN UPDATE SET Price = WC.Price, Shelf = WC.Shelf WHEN NOT MATCHED THEN INSERT VALUES(WC.ISBN, WC.Price, WC.Shelf, NULL) OUTPUT $action, WC.ISBN, Deleted.Price, Deleted.Shelf ) CHANGES(RowAction, ISBN, Price, Shelf) WHERE RowAction = 'UPDATE'; END

80  Part 1  Core SQL Server Development

Because there s h stor ca data n the Book tab e that s rre evant for the merge operat on, c­ r ter a has been added to the jo n pred cate after the ON keyword that tests for NULL va ues n the ­ArchivedAt co umn Th s te s MERGE to cons der on y current books as match ng target rows aga nst the source of week y changes and to comp ete y gnore a the arch ved h story records The CHANGES keyword s what makes INSERT OVER DML poss b e CHANGES exposes the ­co umns of the OUTPUT c ause defined on the nner MERGE statement to the WHERE c ause of the outer INSERT INTO…SELECT statement In L st ng 2-24, th s nc udes the v rtua $action co umn, the ISBN number, and the o d va ues for pr ce and she f be ng rep aced by the update operat on By expos ng the v rtua $action through the CHANGES keyword as RowAction, the INSERT INTO… SELECT statement can app y a WHERE c ause to fi ter out the act ons of new y nserted books and ­append on y the act ons of changed books back nto the Book tab e As exp a ned, th s s someth ng that cannot be ach eved us ng the OUTPUT…INTO c ause on the MERGE statement tse f, but s ­poss b e us ng th s very spec fic INSERT OVER DML syntax The code fi ters out nsert act ons w th the s mp e cr ter a WHERE RowAction = ‘UPDATE’ You cou d of course app y even more soph st cated og c than that f you wanted to For examp e, you may w sh to save h story data for certa n users, reg ons, or any other changed data co umns returned by the OUTPUT c ause and exposed by the CHANGES keyword And that’s exact y the key to the power of INSERT OVER DML Wa k through the scenar o step by step Start w th two books (A and B) and one pr ce change (book A from $100 to $110), as shown here INSERT INTO Book VALUES('A', 100, 1, NULL) INSERT INTO Book VALUES('B', 200, 2, NULL) INSERT INTO WeeklyChange VALUES('A', 110, 1) SELECT * FROM Book SELECT * FROM WeeklyChange GO ISBN ---A B

Price ----100 200

Shelf ----1 2

ArchivedAt ----------------------NULL NULL

(2 row(s) affected) ISBN Price Shelf ---- ----- ----A 110 1 (1 row(s) affected)

When you execute uspUpdateBooks, the nner MERGE statement w update the pr ce for book A n the Book tab e and send ts or g na va ues to the OUTPUT c ause The OUTPUT c ause va ues are then consumed by the outer INSERT INTO…SELECT statement v a the CHANGES keyword The outer statement can therefore use those or g na book va ues for nsert ng h stor ca data back nto the

Chapter 2  T SQL Enhancements   81

Book tab e w th an ArchivedAt va ue set to the current server date and t me (just before 5 00 PM on 2/25/2012, n th s examp e), as fo ows EXEC uspUpdateBooks SELECT * FROM Book GO ISBN ---A A B

Price ----110 100 200

Shelf ----1 1 2

ArchivedAt --------------------------NULL 2012-02-25 16:57:19.8600000 NULL

(3 row(s) affected)

You can see the current data for book A (the row w th an ArchivedAt date of NULL) at $110, and you a so see the prev ous data for book A, wh ch was changed from $100 at about 5 00 PM on 2/25/2012 Somet me ater, you rece ve a new set of changes Th s t me, book A has changed from she f 1 to she f 6, and a new book C has been added, as shown here DELETE FROM WeeklyChange INSERT INTO WeeklyChange VALUES('A', 110, 6) INSERT INTO WeeklyChange VALUES('C', 300, 3) GO

Just ke the first t me you ran the stored procedure, the current row for book A s updated, and a snapshot of the prev ous contents of the row s added to the tab e w th the current server date and t me, as shown here EXEC uspUpdateBooks SELECT * FROM Book GO ISBN ---A A A B C

Price ----110 100 110 200 300

Shelf ----6 1 1 2 3

ArchivedAt --------------------------NULL 2012-02-25 16:57:19.8600000 2012-02-25 16:58:36.1900000 NULL NULL

(5 row(s) affected)

Now book A has two h story records show ng the va ues saved from two ear er updates ­dent fied by date and t me va ues n the ArchivedAt co umn There s a so a new row for book C, wh ch was nserted by the WHEN NOT MATCHED c ause of the MERGE statement However, not ce that there s no h story row for book C, because the nsert act on that was ­actua y captured by the MERGE statement’s OUTPUT c ause was subsequent y fi tered out n the WHERE c ause of the outer INSERT INTO…SELECT statement Had you not fi tered out nsert act ons n that WHERE c ause, another h story record wou d have a so been added to the tab e for book C w th mean ng ess NULL va ues for both Price and Shelf Therefore, by fi ter ng OUTPUT act ons by us ng 82  Part 1  Core SQL Server Development

INSERT OVER DML, you avo d the pro ferat on of h story rows that wou d get generated w th each new book You a so e m nated the extra co umns for the new va ues that were used n the ­prev ous examp e’s BookHistory tab e, as they are stored n each updated vers on arch ved n the Book tab e tse f And you e m nated the need to store an Action co umn, because on y update act ons are ­be ng captured In the end, a ot of benefit was der ved by e m nat ng need ess storage, and t was a ­accomp shed w th a s ng e INSERT OVER DML statement As you can see, the comb nat on of the MERGE and INSERT OVER DML features n SQL Server s a very powerfu add t on to T-SQL A fu y oaded MERGE (hand ng nserts, updates, and de etes) wrapped up us ng INSERT OVER DML de vers an enormous amount of funct ona ty n a s ng e, ­manageab e statement We recommend us ng these new features n eu of the trad t ona mu t -statement approaches n whatever future deve opment scenar os you find t poss b e to do so In add t on to mproved performance, you’ apprec ate the greater manageab ty that resu ts from conso dat ng mu t p e statements nto one

The GROUPING SETS Operator The GROUP BY c ause has been part of the SELECT statement syntax s nce the ear est d a ects of T-SQL You use GROUP BY to create quer es that co apse mu t p e rows be ong ng to the same group nto a s ng e summary row and perform aggregate ca cu at ons (such as SUM and AVG) across the ­nd v dua rows of each group SQL Server 6 5 ater extended the GROUP BY c ause by add ng the WITH CUBE and WITH ROLLUP operators These operators perform add t ona group ng and ­aggregat on of data n standard re at ona quer es, s m ar to what s prov ded by on ne ana yt ca process ng (OLAP) quer es that s ce and d ce your data nto Ana ys s Serv ces cubes, but w thout ever eav ng the re at ona database wor d SQL Server 2008 added the GROUPING SETS operator that further extends the capab t es of the GROUP BY c ause for summar z ng and ana yz ng your data In th s sect on, you w exam ne GROUP BY n many of ts var ant forms You’ start w th the bas c GROUP BY c ause, and then you’ earn how the WITH CUBE and WITH ROLLUP operators can be used to enhance those summary resu ts Then you’ exp ore the GROUPING SETS operator added n SQL Server 2008 Start w th a s mp e nventory tab e that conta ns quant t es for var ous tems n d verse co ors that are ava ab e at d fferent store ocat ons, as shown n L st ng 2-25 Listing 2-25  Creat ng the Inventory tab e.

CREATE TABLE Inventory( Store varchar(2), Item varchar(20), Color varchar(10), Quantity decimal)



Chapter 2  T SQL Enhancements   83

Next, add some nventory data There are 13 rows that conta n nventory for tab es, cha rs, and sofas ava ab e n b ue, red, and green, at NY, NJ, and PA ocat ons, as shown n L st ng 2-26 Listing 2-26  Popu at ng the Inventory tab e.

INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT

INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO

Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory

VALUES('NY', VALUES('NJ', VALUES('NY', VALUES('NJ', VALUES('PA', VALUES('NY', VALUES('PA', VALUES('NY', VALUES('NJ', VALUES('NY', VALUES('NJ', VALUES('PA', VALUES('NJ',

'Table', 'Blue', 124) 'Table', 'Blue', 100) 'Table', 'Red', 29) 'Table', 'Red', 56) 'Table', 'Red', 138) 'Table', 'Green', 229) 'Table', 'Green', 304) 'Chair', 'Blue', 101) 'Chair', 'Blue', 22) 'Chair', 'Red', 21) 'Chair', 'Red', 10) 'Chair', 'Red', 136) 'Sofa', 'Green', 2)

Now use a bas c GROUP BY c ause to query on th s data SELECT Item, Color, SUM(Quantity) AS TotalQty, COUNT(Store) AS Stores FROM Inventory GROUP BY Item, Color ORDER BY Item, Color

As mp ed by ts syntax, th s query groups a the nventory records by tem and then by co or w th n each tem The resu t set therefore nc udes one summary row for each un que comb nat on of tems and co ors The store ocat on s not nc uded n the group ng, and so the resu ts returned by the query app y to a stores Each summary row nc udes a TotalQty co umn ca cu ated by the SUM aggregate funct on as the tota quant ty for a rows of the same tem and co or across a stores The ast co umn, Stores, s ca cu ated by the COUNT aggregate funct on as the number of store ocat ons at wh ch each un que comb nat on of tems and co ors s ava ab e, as shown here Item -------------------Chair Chair Sofa Table Table Table

Color ---------Blue Red Green Blue Green Red

TotalQty -----------------------------123 167 2 224 533 223

Stores -----2 3 1 2 2 3

(6 row(s) affected)

These resu ts show that SQL Server grouped the nventory records shar ng the same tem and co or nto a s ng e summary row The store ocat on s not nc uded n the breakdown, because you d d not group by t, and so each summary row app es to a stores For each tem, the tota quant ty s ­ca cu ated as the sum of the nd v dua quant ty va ues for the tem and co or comb nat ons n each group, and the store count s ca cu ated as the number of store ocat ons at wh ch each tem and 84  Part 1  Core SQL Server Development

co or comb nat on s ava ab e W th GROUP BY, every co umn returned by the query must be e ther one of the co umns actua y be ng grouped by (such as the Store, Item, and Color co umns) or an aggregate funct on that operates across a the comb ned member rows for the group [such as the SUM(Quantity) and COUNT(Store) funct ons]

Rolling Up by Level Th s query demonstrates the most bas c app cat on of the GROUP BY c ause, wh ch s mp y groups and aggregates It answers the quest on “How many tems per co or are n each store ocat on?” by ­group ng tems and co ors The WITH ROLLUP and WITH CUBE operators (wh ch were ntroduced n SQL Server 6 5) can be used to answer more quest ons than that Each of these operators supp ements the resu ts of an ord nary GROUP BY c ause w th add t ona summary aggregat ons on the under y ng data Here s the same query you ran before, on y th s t me us ng WITH ROLLUP SELECT Item, Color, SUM(Quantity) AS TotalQty, COUNT(Store) AS Stores FROM Inventory GROUP BY Item, Color WITH ROLLUP ORDER BY Item, Color GO Item -------------------NULL Chair Chair Chair Sofa Sofa Table Table Table Table

Color ---------NULL NULL Blue Red NULL Green NULL Blue Green Red

TotalQty -----------------------------1272 290 123 167 2 2 980 224 533 223

Stores -----13 5 2 3 1 1 7 2 2 3

(10 row(s) affected)

Th s t me, you rece ve the same s x grouped resu ts as before, supp emented w th four add t ona rollup rows (the ones w th NULL va ues for Item or Color, h gh ghted here n bo d) Ro up rows ­conta n add t ona h gher- eve summary nformat on that essent a y “groups the groups” of the query resu ts Any row w th NULL va ues n t s a ro up row, and the NULL shou d be thought of as “a ­va ues” n th s context In these resu ts, the first row s the top- eve ro up, as nd cated by NULL va ues for both Item and Color Th s top- eve ro up reports a grand tota quant ty of 1,272 for the ent re set (a tems n a co ors) n a store ocat ons (where the ent re set cons sts of the 13 un que tem/co or comb nat ons across a ocat ons) The next resu t s an tem- eve ro up for cha rs It reports a tota quant ty of 290 for cha rs n a co ors across 5 store ocat ons The two resu ts that fo ow are the same summary rows for cha rs ­returned by the first “p a n” GROUP BY query and that were just ro ed up They show 123 b ue cha rs n 2 ocat ons and 167 red cha rs n 3 ocat ons The next resu t s an tem- eve ro up for sofas On y

Chapter 2  T SQL Enhancements   85

one store ocat on carr es sofas, and they’re ava ab e on y n green The sofa ro up therefore conta ns the same va ues as the one and on y summary row for 2 green sofas ava ab e n 1 ocat on The ast set of rows report on tab es n the same way that the cha r and sofa data was returned Th s nc udes an tem- eve ro up show ng 980 tab es across 7 ocat ons fo owed by the summary rows ­show ng 224 b ue tab es n 2 ocat ons, 533 green tab es n 2 ocat ons, and 223 red tab es n 3 ocat ons ­returned So by s mp y add ng WITH ROLLUP, you can answer a second quest on that the first ord nary GROUP BY query cou dn’t “How many cha rs, tab es, and sofas are n stock, regardless of co or?”

Rolling Up All Level Combinations Us ng WITH CUBE now nstead of WITH ROLLUP takes th s resu t set to the next eve , as shown here SELECT Item, Color, SUM(Quantity) AS TotalQty, COUNT(Store) AS Stores FROM Inventory GROUP BY Item, Color WITH CUBE ORDER BY Item, Color GO Item -------------------NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table Table Table

Color ---------NULL Blue Green Red NULL Blue Red NULL Green NULL Blue Green Red

TotalQty -----------------------------1272 347 535 390 290 123 167 2 2 980 224 533 223

Stores -----13 4 3 6 5 2 3 1 1 7 2 2 3

(13 row(s) affected)

You now have the same resu t set returned by WITH ROLLUP, on y th s t me three more ro up rows have been added (aga n, nd cated n bo d here) Let’s ook at exact y what SQL Server d d By ­app y ng WITH CUBE, you nstructed the database eng ne to create a mu t d mens ona representat on of the data on the fly, wh ch s oose y referred to as a cube The number of d mens ons n the cube s based on the number of group ng co umns Th s nventory examp e has on y two d mens ons, but a query cou d have many more d mens ons f t spec fies more group ng co umns A cube conta ns ro ups for a the poss b e permutat ons of d mens on va ues, not just the comb nat ons of one va ue within another, as per the nest ng eve s defined by group ng co umns sted n the GROUP BY c ause So WITH CUBE returns the same ro ups returned by WITH ROLLUP—wh ch nc udes a tems regard ess of co or—p us add t ona ro ups for a co ors regard ess of tem As a resu t, you can now answer a th rd quest on that the ear er GROUP BY quer es cou dn’t “How many tems of any type n 86  Part 1  Core SQL Server Development

a part cu ar co or are n stock?” That means that you can now a so see how many b ue, green, or red tems you have n nventory regard ess of whether they are cha rs, sofas, or tab es Because a cube ro s up every poss b e comb nat on of d mens on va ues ndependent of the order of eve s expressed w th GROUP BY, each add t ona group ng eve ncreases the s ze of the resu t set exponent a y For examp e, f you mod fy the query to group by store ocat on as we , SQL Server returns 44 rows nc ud ng ro ups for every poss b e comb nat on of va ues across the three group ng co umns Store, Item, and Color, as fo ows SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY Store, Item, Color WITH CUBE ORDER BY Store, Item, Color GO Store ----NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NY NY NY NY NY NY NY NY NY NY NY PA



Item -------------------NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table Table Table NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table Table NULL NULL NULL NULL Chair Chair Chair Table Table Table Table NULL

Color ---------NULL Blue Green Red NULL Blue Red NULL Green NULL Blue Green Red NULL Blue Green Red NULL Blue Red NULL Green NULL Blue Red NULL Blue Green Red NULL Blue Red NULL Blue Green Red NULL

TotalQty -----------------------------1272 347 535 390 290 123 167 2 2 980 224 533 223 190 122 2 66 32 22 10 2 2 156 100 56 504 225 229 50 122 101 21 382 124 229 29 578

Chapter 2  T SQL Enhancements   87

PA PA PA PA PA PA PA

NULL NULL Chair Chair Table Table Table

Green Red NULL Red NULL Green Red

304 274 136 136 442 304 138

(44 row(s) affected)

These resu ts can now answer nventory quest ons for every conce vab e comb nat on of group ng eve s For examp e, across a ocat ons, there are 347 b ue tems (tab es, cha rs, and sofas), 290 cha rs (a co ors), and 533 green tab es, whereas n NY spec fica y, there are 50 red tems, 382 tab es, and 124 b ue tab es, and so on Every permutat on of store ocat on, tem, and co or—and the r ro ups —are returned by th s s ng e query

Returning Just the Top Level The ast var at on on GROUP BY s the GROUPING SETS operator ntroduced n SQL Server 2008 Th s operator returns just the top- eve ro up rows for each group ng eve and does not nc ude the actua group eve summary nformat on that was returned by ear er vers ons of the query, as fo ows SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS (Store, Item, Color) ORDER BY Store, Item, Color GO Store ----NULL NULL NULL NULL NULL NULL NJ NY PA

Item -------------------NULL NULL NULL Chair Sofa Table NULL NULL NULL

Color ---------Blue Green Red NULL NULL NULL NULL NULL NULL

TotalQty -----------------------------347 535 390 290 2 980 190 504 578

(9 row(s) affected)

GROUPING SETS s mere y another var at on on GROUP BY that you can use when you requ re on y top- eve ro ups for each of your group ng eve s (that s, one set of group ro ups per eve ) In th s case, you get a tota quant ty report for a co ors, a tems, and a store ocat ons w thout ­nc ud ng the summary rows for each of the comb nat ons of group ng eve s, just as ear er vers ons of the query d d

88  Part 1  Core SQL Server Development

Mixing and Matching But the GROUPING SETS story doesn’t end here, of course Un ke WITH ROLLUP and WITH CUBE— wh ch are mutua y exc us ve n the same query—ro up and cube operat ons can be used together and w th GROUPING SETS n any comb nat on Th s means that you can compose one query that returns on y top- eve ro ups for certa n group ng eve s and a so returns the ower- eve ro ups and summary rows for other group ng eve s, just ke you get us ng WITH ROLLUP or WITH CUBE n ­separate quer es To ach eve th s, SQL Server prov des an a ternat ve syntax for WITH ROLLUP and WITH CUBE that makes these operators capab e of be ng expressed w th one another n the same GROUP BY c ause Th s syntax s actua y qu te s mp e drop the WITH keyword, p ace the ROLLUP or CUBE keyword ­before the group ng co umns rather than after, and enc ose the group ng co umns n parentheses For examp e, the fo ow ng two GROUP BY c auses are nterchangeab e GROUP BY Item, Color WITH ROLLUP GROUP BY ROLLUP(Item, Color)

S m ar y, these two c auses are a so equ va ent GROUP BY Item, Color WITH CUBE GROUP BY CUBE(Item, Color)

A though the two vers ons are nterchangeab e when used on the r own, you must use the newer syntax f you want to comb ne them w th one another or w th GROUPING SETS n a s ng e query Here s another vers on of the nventory query that does just that SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), CUBE(Item, Color) ORDER BY Store, Item, Color

The GROUP BY c ause n th s query nc udes both a GROUPING SETS operator on Store and a CUBE operator on Item and Color Th s te s SQL Server to return top- eve ro ups on y on the Store co umn and fu summar es w th mu t d mens ona ro ups on the Item and Color co umns Here are the resu ts Store ----NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ



Item -------------------NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table

Color ---------NULL Blue Green Red NULL Blue Red NULL Green NULL Blue

TotalQty -----------------------------190 122 2 66 32 22 10 2 2 156 100

Chapter 2  T SQL Enhancements   89

NJ NY NY NY NY NY NY NY NY NY NY NY PA PA PA PA PA PA PA PA

Table NULL NULL NULL NULL Chair Chair Chair Table Table Table Table NULL NULL NULL Chair Chair Table Table Table

Red NULL Blue Green Red NULL Blue Red NULL Blue Green Red NULL Green Red NULL Red NULL Green Red

56 504 225 229 50 122 101 21 382 124 229 29 578 304 274 136 136 442 304 138

(31 row(s) affected)

The rows w th NULL va ues for both Item and Color (h gh ghted here n bo d) are the top- eve ro ups for Store returned by the GROUPING SETS(Store) operator These rows report just the t­ota s for each store (a tems, a co ors) A of the other rows are the mu t d mens ona ro up and ­summary resu ts returned by CUBE(Item, Color) These rows report aggregat ons for every ­comb nat on of Item and Color Because Store s returned by GROUPING SETS and not by CUBE, you don’t see ­comb nat ons that nc ude a stores You can use GROUPING SETS, ROLLUP, and CUBE n any comb nat on you want w th the GROUP BY c ause As a resu t, you ga n tremendous flex b ty for group ng, aggregat ng, and ana yz ng your data just the way you need to The on y restr ct on n usage s the same one that app es when us ng GROUP BY on ts own co umns returned by the query must be spec fied e ther n the GROUP BY c ause ( n any of the GROUPING SETS, ROLLUP, or CUBE operators) or n an aggregate funct on that operates across a the comb ned rows for the group (such as SUM, COUNT, MIN, MAX, and so on)

Handling NULL Values We’ conc ude the d scuss on of GROUPING SETS by d scuss ng NULL va ues As you’ve seen, SQL Server returns NULL va ues to represent a va ues n h gh- eve ro up rows If you’re fortunate enough to be work ng w th data that s guaranteed not to conta n NULL va ues, fe s good for you But th s s far more often not the case, and thus a prob em ar ses d st ngu sh ng between “rea ” NULL va ues and the NULL va ues represent ng “a va ues” n ro up rows To demonstrate, add two more rows to the Inventory tab e for amps that have no co or ­assoc at on These rows store NULL va ues n the Color co umn, as shown n L st ng 2-27

90  Part 1  Core SQL Server Development

Listing 2-27  ntroduc ng NULL va ues nto the Inventory tab e.

INSERT INTO Inventory VALUES('NY', 'Lamp', NULL, 36) INSERT INTO Inventory VALUES('NJ', 'Lamp', NULL, 8)

Now run the exact same query you ran before SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), CUBE(Item, Color) ORDER BY Store, Item, Color GO Store ----NJ NJ NJ NJ : NJ NJ NY NY NY NY : PA PA

Item -------------------NULL NULL NULL NULL

Color ---------NULL NULL Blue Green

TotalQty -----------------------------8 198 122 2

Table Table NULL NULL NULL NULL

Blue Red NULL NULL Blue Green

100 56 36 540 225 229

Table Table

Green Red

304 138

(37 row(s) affected)

These are very confus ng resu ts Because both the “a co ors” ro up co umns and the amp co umns w th “no co or” have a NULL va ue for Color, t s mposs b e to d st ngu sh between the two when ana yz ng the query resu ts For examp e, the first row returns the ro up for a tems w th no co or n NJ (that’s the 8 amps), and the second row returns the ro up for a tems n all co ors n NJ, but there s no way to d scern that d fference because NULL s used to represent both “no co or” and “a co ors ” The same prob em occurs aga n further down n the resu ts for NY, where there are a so co or ess amps n stock Once aga n, because “no co or” and “a co ors” are both represented by NULL va ues, the resu ts are noth ng short of perp ex ng The so ut on to th s prob em s to use the GROUPING funct on n your query The GROUPING funct on returns a b t va ue of 1 (true) f the co umn passed to t represents an “a va ues” ro up, and t returns 0 (fa se) otherw se It s therefore poss b e to d st ngu sh between “a va ues” ro up co umns (wh ch are a ways NULL) and regu ar data (wh ch might be NULL, as s the case for the amps, wh ch have no co or va ues) Here s a rev sed vers on of the query that uses the GROUPING funct on n conjunct on w th CASE to produce a better resu t set that c ears up the confus on between “a va ues” and “no va ue”



Chapter 2  T SQL Enhancements   91

SELECT CASE WHEN GROUPING(Store) = 1 CASE WHEN GROUPING(Item) = 1 CASE WHEN GROUPING(Color) = 1 SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), ORDER BY Store, Item, Color

THEN '(all)' ELSE Store END AS Store, THEN '(all)' ELSE Item END AS Item, THEN '(all)' ELSE Color END AS Color,

CUBE(Item, Color)

The CASE construct tests each group ng co umn returned by the query us ng the GROUPING funct on If t returns 1 (true), that means that the co umn represents an “a va ues” ro up In th s case, the str ng (all) s returned, rather than the NULL va ue that wou d have otherw se been returned If t returns 0 (fa se), the co umn conta ns regu ar data, wh ch m ght or m ght not be NULL A though there are NULL va ues on y n for the Color co umn for amps, app y the same CASE and GROUPING to the Store and Item co umns as we Th s s a defens ve cod ng measure aga nst the poss b ty of the Store or Item co umn a so conta n ng NULL va ues n the future Tak ng th s approach now reso ves the confus on w th respect to NULL va ues and ro ups n the query resu ts, as shown here Store ----NJ NJ NJ NJ : NJ NJ NY NY NY NY : PA PA

Item -------------------(all) (all) (all) (all)

Color ---------NULL (all) Blue Green

TotalQty -----------------------------8 198 122 2

Table Table (all) (all) (all) (all)

Blue Red NULL (all) Blue Green

100 56 36 540 225 229

Table Table

Green Red

304 138

(37 row(s) affected)

It’s perfect y understandab e now that the first row returns the ro up for a tems w th no co or n NJ, whereas the second row returns the ro up for a tems n a co ors ( nc ud ng no co or) n NJ The same s true farther down n the NY resu ts, where there are a so co or ess amps n stock Therefore, to avo d any potent a confus on concern ng NULL va ues n your group ng quer es, you shou d a ways use the ­GROUPING funct on n th s manner to trans ate the NULL va ues that mean “a va ues” for the user As ong as you’re mod fy ng the query to produce more readab e resu ts, enhance t one more t me In the same way that you trans ated the NULL for “a va ues” to the text (all), you can trans ate the NULL va ues for regu ar “m ss ng” data to (n/a) Th s s easy to do by add ng an ELSE c ause to the CASE construct that uses the ISNULL funct on on the co umn, as shown here SELECT CASE WHEN GROUPING(Store) = 1 THEN '(all)' ELSE ISNULL(Store, '(n/a)') END AS Store, CASE WHEN GROUPING(Item) = 1 THEN '(all)'

92  Part 1  Core SQL Server Development

ELSE ISNULL(Item, '(n/a)') END AS Item, CASE WHEN GROUPING(Color) = 1 THEN '(all)' ELSE ISNULL(Color, '(n/a)') END AS Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), CUBE(Item, Color) ORDER BY Store, Item, Color

The ELSE c ause n each CASE construct runs f the GROUPING funct on returns 0 (fa se) Th s means that the co umn s not an “a va ues” ro up, but regu ar co umn data You want regu ar co umn data to be returned as s, except for NULL va ues that shou d be returned as the str ng (n/a) The ISNULL funct on tests for NULL va ues and performs the trans at on on them, as shown n the resu ts returned by the query Store ----NJ NJ NJ NJ : NJ NJ NJ NJ NJ NJ NY NY NY : NY NY NY NY NY : PA PA

Item -------------------(all) (all) (all) (all)

Color ---------(all) (n/a) Blue Green

TotalQty -----------------------------198 8 122 2

Sofa Stool Stool Table Table Table (all) (all) (all)

Green (all) (n/a) (all) Blue Red (all) (n/a) Blue

2 8 8 156 100 56 540 36 225

Chair Stool Stool Table Table

Red (all) (n/a) (all) Blue

21 36 36 382 124

Table Table

Green Red

304 138

(37 row(s) affected)

The query now returns no NULL va ues at a , wh ch s much better for your users who don’t rea y know or care exact y what NULL means anyway By trans at ng these va ues appropr ate y to (all) and (n/a), you have produced a far more usab e report for them

Windowing (OVER Clause) Enhancements The first windowing capab t es appeared n SQL Server 2005 w th the ntroduct on of the OVER c ause and a set of four rank ng funct ons ROW NUMBER, RANK, DENSE RANK, and NTILE In our ­d scuss on, the term “w ndow” refers to the scope of v s b ty from one row n a resu t set r­ e at ve

Chapter 2  T SQL Enhancements   93

to ne ghbor ng rows n the same resu t set By defau t, OVER produces a s ng e w ndow over the ­ent re resu t set, but ts assoc ated PARTITION BY c ause ets you d v de the resu t set up nto ­mu t p e groups, each conta ned ns de the r own w ndow The row sequence w th n each w ndow s ­determ ned by an assoc ated ORDER BY c ause, and based on th s sequence, the rank ng funct ons ass gn an accumu at ng va ue to the rows n the w ndow In add t on to the rank ng funct ons, the OVER c ause can be used w th the trad t ona ­aggregate funct ons SUM, COUNT, MIN, MAX, and AVG When do ng so, you do not spec fy the GROUP BY c ause that’s norma y requ red w th the aggregate funct ons Instead, each row ca cu ates an ­aggregat on based on the w ndow of rows defined w th OVER, opt ona y grouped us ng ­PARTITION BY Th s s certa n y usefu , because t a ows you to obta n aggregat ons w thout be ng forced to ­conso date (and ose) deta rows w th a GROUP BY c ause But unfortunate y (unt now), the ­aggregate funct ons cou d not a so use ORDER BY n the OVER c ause (as s requ red when us ng OVER w th the rank ng funct ons), mak ng t mposs b e to ca cu ate cumulative aggregat ons at the row eve w th n each w ndow For examp e, you cou d use AVG w th OVER (and, opt ona y PARTITION BY), but w thout an assoc ated ORDER BY, there s no des gnated sequence to the rows n each w ndow, ­mak ng t mposs b e for SQL Server to compute a runn ng average from one row to the next w th n the w ndow Thus, the best that AVG w th OVER cou d do s compute the average for all the rows n the w ndow ( ndependent of row sequence), and then return that va ue for every row SQL Server 2012 fina y addresses th s shortcom ng In the fo ow ng code samp es, you’ see how OVER/ORDER BY can now be used w th a the trad t ona aggregate funct ons to prov de running calculations w th n ordered w ndows You’ a so earn how to frame w ndows us ng the ROWS and RANGE c ause, wh ch adjusts the s ze and scope of the w ndow to enab e sliding calculations And fina y, SQL Server 2012 ntroduces e ght new ana yt c funct ons (covered n the next sect on) that are des gned spec fica y to work w th ordered (and opt ona y part t oned) w ndows us ng OVER w th ORDER BY (and opt ona y PARTITION BY)

Note  All remaining topics in this chapter are new features in SQL Server 2012. The code n L st ng 2-28 creates a tab e popu ated w th financ a transact ons from severa d fferent accounts Tangent a y, note the use of the new DATEFROMPARTS funct on (a so covered n the next sect on), wh ch s used to construct a date va ue from year, month, and day parameters Listing 2-28  Prepar ng samp e transact on data for query ng w th w ndow funct ons.

CREATE TABLE TxnData (AcctId GO INSERT INTO TxnData (AcctId, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4,

int, TxnDate date, Amount decimal) TxnDate, Amount) VALUES 10), 500), -- 5 transactions for acct 1 22), 250), 24), 75), 26), 125), 28), 175),

94  Part 1  Core SQL Server Development

(2, (2, (2, (2, (2, (2, (2, (2, (3, (3, (3, (3,

DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012,

4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,

11), 15), 22), 25), 27), 27), 29), 30), 14), 15), 22), 23),

500), -- 8 transactions for acct 2 50), 5000), 550), 105), 95), 100), 2500), 500), -- 4 transactions for acct 3 600), 25), 125)

Running Aggregations In SQL Server 2012, an ORDER BY c ause may be spec fied w th OVER to produce runn ng ­aggregat ons w th n each w ndow, as L st ng 2-29 demonstrates Listing 2-29 Us ng OVER w th ORDER BY to produce runn ng aggregat ons.

SELECT AcctId, TxnDate, Amount, RAvg = AVG(Amount) OVER (PARTITION RCnt = COUNT(*) OVER (PARTITION RMin = MIN(Amount) OVER (PARTITION RMax = MAX(Amount) OVER (PARTITION RSum = SUM(Amount) OVER (PARTITION FROM TxnData ORDER BY AcctId, TxnDate

AcctId -----1 1 1 1 1 2 2 2 :

TxnDate ---------2012-02-10 2012-02-22 2012-02-24 2012-02-26 2012-02-28 2012-02-11 2012-02-15 2012-02-22

Amount -----500 250 75 125 175 500 50 5000

RAvg ----------500.000000 375.000000 275.000000 237.500000 225.000000 500.000000 275.000000 1850.000000

BY BY BY BY BY

RCnt ---1 2 3 4 5 1 2 3

AcctId AcctId AcctId AcctId AcctId

RMin ---500 250 75 75 75 500 50 50

ORDER ORDER ORDER ORDER ORDER

RMax ---500 500 500 500 500 500 500 5000

BY BY BY BY BY

TxnDate), TxnDate), TxnDate), TxnDate), TxnDate)

RSum ---500 750 825 950 1125 500 550 5550

The resu ts of th s query are part t oned (w ndowed) by account W th n each w ndow, the ­ ccount’s runn ng averages, counts, m n mum/max mum va ues, and sums are ordered by transact on a date, show ng the chrono og ca y accumu ated va ues for each account No ROWS c ause s spec fied (we’ exp a n how to use the ROWS c ause next), so ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW s assumed by defau t Th s y e ds a w ndow frame s ze that spans from the beg nn ng

Chapter 2  T SQL Enhancements   95

of the part t on (the first row of each account) through the current row When the account ID changes, the prev ous w ndow s “c osed” and new ca cu at ons start runn ng for a new w ndow over the next account ID

Sliding Aggregations You can a so narrow each account’s w ndow by fram ng t w th a ROWS c ause n the OVER c ause Th s enab es s d ng ca cu at ons, as demonstrated n L st ng 2-30 Listing 2-30 Us ng OVER w th ORDER BY and PRECEDING to produce s d ng aggregat ons.

SELECT AcctId, TxnDate, Amount, SAvg = AVG(Amount) OVER (PARTITION BY AcctId ORDER ROWS BETWEEN 2 PRECEDING SCnt = COUNT(*) OVER (PARTITION BY AcctId ORDER SMin = MIN(Amount) OVER (PARTITION BY AcctId ORDER SMax = MAX(Amount) OVER (PARTITION BY AcctId ORDER SSum = SUM(Amount) OVER (PARTITION BY AcctId ORDER FROM TxnData ORDER BY AcctId, TxnDate

AcctId -----1 1 1 1 1 2 2 2 :

TxnDate ---------2012-02-10 2012-02-22 2012-02-24 2012-02-26 2012-02-28 2012-02-11 2012-02-15 2012-02-22

Amount -----500 250 75 125 175 500 50 5000

SAvg ----------500.000000 375.000000 275.000000 150.000000 125.000000 500.000000 275.000000 1850.000000

SCnt ---1 2 3 3 3 1 2 3

SMin ---500 250 75 75 75 500 50 50

SMax ---500 500 500 250 175 500 500 5000

BY TxnDate AND CURRENT ROW), BY TxnDate ROWS 2 BY TxnDate ROWS 2 BY TxnDate ROWS 2 BY TxnDate ROWS 2

PRECEDING), PRECEDING), PRECEDING), PRECEDING)

SSum ---500 750 825 450 375 500 550 5550

Th s s ght y mod fied vers on of the prev ous query spec fies ROWS BETWEEN 2 PRECEDING AND CURRENT ROW n the OVER c ause for the RAvg co umn, overr d ng the defau t w ndow s ze ­Spec fica y, t frames the w ndow w th n each account’s part t on to a max mum of three rows the current row, the row before t, and one more row before that one Once the w ndow expands to three rows, t stops grow ng and starts s d ng down the subsequent rows unt a new part t on (the next ­account) s encountered The BETWEEN…AND CURRENT ROW keywords that spec fy the upper bound of the w ndow are assumed defau t, so to reduce code c utter, the other co umn defin t ons spec fy just the ower bound of the w ndow w th the shorter var at on ROWS 2 PRECEDING Not ce how the w ndow “s des” w th n each account For examp e, the s d ng max mum for ­account 1 drops from 500 to 250 n the fourth row, because 250 s the argest va ue n the w ndow of three rows that beg ns two rows ear er—and the 500 from the very first row s no onger v s b e

96  Part 1  Core SQL Server Development

n that w ndow S m ar y, the s d ng sum for each account s based on the defined w ndow Thus, the s d ng sum of 375 on the ast row of account 1 s the tota sum of that row (175) p us the two ­preced ng rows (75 + 125) on y—not the tota sum for a transact ons n the ent re account, as the runn ng sum had ca cu ated

Using RANGE versus ROWS F na y, RANGE can be used nstead of ROWS to hand e “t es” w th n a w ndow A though ROWS treats each row n the w ndow d st nct y, RANGE w merge rows conta n ng dup cate ORDER BY va ues, as demonstrated by L st ng 2-31 Listing 2-31 Compar ng ROWS and RANGE for ca cu at ng w ndow funct ons.

SELECT AcctId, TxnDate, Amount, SumByRows = SUM(Amount) OVER (ORDER BY TxnDate ROWS UNBOUNDED PRECEDING), SumByRange = SUM(Amount) OVER (ORDER BY TxnDate RANGE UNBOUNDED PRECEDING) FROM TxnData WHERE AcctId = 2 ORDER BY TxnDate

AcctId -----2 2 2 2 2 2 2 2

TxnDate ---------2012-02-11 2012-02-15 2012-02-22 2012-02-25 2012-02-27 2012-02-27 2012-02-29 2012-02-30

Amount -----500 50 5000 550 105 95 100 2500

SumByRows --------500 550 5550 6100 6205 6300 6400 8900

SumByRange ---------500 550 5550 6100 6300 6300 6400 8900

In th s resu t set, ROWS and RANGE both return the same va ues, w th the except on of the fifth row Because the fifth and s xth rows are both t ed for the same date (2/27/2012), RANGE returns the comb ned runn ng sum for both rows The seventh row (for 2/29/2012) breaks the t e, and ROWS “catches up” w th RANGE to return runn ng tota s for the rest of the w ndow

New T-SQL Functions in SQL Server 2012 The atest vers on of SQL Server augments T-SQL w th many new funct ons In th s sect on, we cover the 22 new funct ons (and 1 changed funct on) ntroduced n SQL Server 2012 We’ start by cover ng the new T-SQL ana yt c funct ons, because they operate us ng the same w ndow ng pr nc p es we were just d scuss ng



Chapter 2  T SQL Enhancements   97

New Analytic Functions E ght new ana yt c funct ons have been added to T-SQL A of them work n conjunct on w th an ordered w ndow defined w th an assoc ated ORDER BY c ause that can be opt ona y part t oned w th a PARTITION BY c ause and framed w th a BETWEEN c ause The new funct ons are ■

FIRST VALUE

■

LAST VALUE

■

LAG

■

LEAD

■

PERCENT RANK

■

CUME DIST

■

PERCENTILE CONT

■

PERCENTILE DISC

In L st ng 2-32, the FIRST VALUE, LAST VALUE, LAG, and LEAD funct ons are used to ana yze a set of orders at the product eve Listing 2-32  Us ng the FIRST VALUE, LAST VALUE, LAG, and LEAD funct ons.

DECLARE @Orders AS table(OrderDate date, ProductID int, Quantity int) INSERT INTO @Orders VALUES ('2012-03-18', 142, 74), ('2012-04-11', 123, 95), ('2012-04-12', 101, 38), ('2012-05-21', 130, 12), ('2012-05-30', 101, 28), ('2012-07-25', 123, 57), ('2012-07-28', 101, 12) SELECT OrderDate, ProductID, Quantity, WorstOn = FIRST_VALUE(OrderDate) OVER(PARTITION BY ProductID ORDER BY Quantity), BestOn = LAST_VALUE(OrderDate) OVER(PARTITION BY ProductID ORDER BY Quantity ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING), PrevOn = LAG(OrderDate, 1) OVER(PARTITION BY ProductID ORDER BY OrderDate), NextOn = LEAD(OrderDate, 1) OVER(PARTITION BY ProductID ORDER BY OrderDate) FROM @Orders ORDER BY OrderDate

OrderDate ---------2012-03-18 2012-04-11

ProductID --------142 123

Quantity -------74 95

WorstOn ---------2012-03-18 2012-07-25

98  Part 1  Core SQL Server Development

BestOn ---------2012-03-18 2012-04-11

PrevOn ---------NULL NULL

NextOn ---------NULL 2012-07-25

2012-04-12 2012-05-21 2012-05-30 2012-07-25 2012-07-28

101 130 101 123 101

38 12 28 57 12

2012-07-28 2012-05-21 2012-07-28 2012-07-25 2012-07-28

2012-04-12 2012-05-21 2012-04-12 2012-04-11 2012-04-12

NULL NULL 2012-04-12 2012-04-11 2012-05-30

2012-05-30 NULL 2012-07-28 NULL NULL

In th s query, four ana yt c funct ons spec fy an OVER c ause that part t ons the resu t set by ­ProductID The product part t ons defined for FIRST VALUE and LAST VALUE are sorted by Quantity, whereas the ­product part t ons for LAG and LEAD are sorted by OrderDate The fu resu t set s sorted by OrderDate, so you need to v sua ze the sorted part t on for each of the four funct ons to understand the output—the resu t set sequence s not the same as the row sequence used for the w ndow ng funct ons The WorstOn and BestOn co umns use FIRST VALUE and LAST VALUE, respect ve y, to return the “worst” and “best” dates for the product n each part t on Performance s measured by ­quant ty, so sort ng each product’s part t on by quant ty w pos t on the worst order at the first row n the ­part t on and the best order at the ast row n the part t on FIRST VALUE and LAST VALUE can return the va ue of any co umn (OrderDate, n th s case), not just the aggregate co umn tse f For LAST VALUE, t s a so necessary to exp c t y define a w ndow over the ent re part t on w th ROWS BETWEEN ­UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING Otherw se, as exp a ned n our ­ear er d scuss on about runn ng and s d ng aggregat ons, the defau t w ndow s ROWS ­BETWEEN ­UNBOUNDED PRECEDING AND CURRENT ROW Th s defau t behav or frames (constra ns) the ­w ndow, and does not cons der the rema n ng rows n the part t on that are needed to obta n the h ghest quant ty for LAST VALUE In the output, not ce that OrderDate, LowestOn, and HighestOn for the first order (product 142) are a the same va ue (3/18) Th s s because product 142 was on y ordered once, so FIRST VALUE and LAST VALUE operate over a part t on that has on y th s one row n t, w th an OrderDate va ue of 3/18 The second row s for product 123, quant ty 95, ordered on 4/11 Four rows ahead n the resu t set (not the part t on) there s another order for product 123, quant ty 57, p aced on 7/25 Th s means that, for th s product, FIRST VALUE and LAST VALUE operate over a part t on that has these two rows n t, sorted by quant ty Th s pos t ons the 7/25 order (quant ty 57) first and the 4/11 (quant ty 95) ast w th n the part t on As a resu t, rows for product 123 report 7/25 for WorstDate and 4/11 for BestDate The next order (product 101) appears two more t mes n the resu t set, creat ng a part t on of three rows Aga n, based on the Quantity sort of the part t on, each row n the part t on reports the product’s worst and best dates (wh ch are 7/28 and 4/12, respect ve y) The PrevOn and NextOn co umns use LAG and LEAD to return the prev ous and next date that each product was ordered They spec fy an OVER c ause that part t ons by ProductId as before, but the rows n these part t ons are sorted by OrderDate Thus, the LAG and LEAD funct ons exam ne each product’s orders n chrono og ca sequence, regard ess of quant ty For each row n each part t on, LAG s ab e to access prev ous ( agg ng) rows w th n the same part t on S m ar y, LEAD can access subsequent ( ead ng) rows w th n the same part t on The first parameter to LAG and LEAD spec fies the co umn va ue to be returned from a agg ng or ead ng row, respect ve y The second ­parameter spec fies the number of rows back or forward LAG and LEAD shou d seek w th n each part t on, ­re at ve to the current row L st ng 2-32 passes OrderDate and 1 as parameters to LAG and LEAD, ­us ng



Chapter 2  T SQL Enhancements   99

product part t ons that are ordered by date Thus, the query returns the most recent past date, and nearest future date, that each product was ordered Because the first order’s product (142) was on y ordered once, ts s ng e-row part t on has no ­agg ng or ead ng rows, and so LAG and LEAD both return NULL for PrevOn and NextOn (note that both funct ons a so opt ona y accept a th rd parameter to be used nstead of NULL when no ­agg ng or ead ng row ex sts) The second order (on 4/11) s for product 123, wh ch was ordered aga n on 7/25, creat ng a part t on w th two rows sorted by OrderDate, w th the 4/11 order pos t oned first and the 7/25 order pos t oned ast w th n the part t on The first row n a mu t -row w ndow has no agg ng rows, but at east one ead ng row S m ar y, the ast order n a mu t -row w ndow has at east one ­agg ng row, but no ead ng rows As a resu t, the first order (4/11) reports NULL and 7/25 for PrevOn and NextOn, respect ve y, and the second order (7/25) reports 4/11 and NULL for PrevOn and ­NextOn, respect ve y Product 101 was ordered three t mes, wh ch creates a part t on of three rows In th s part t on, the second row has both a agg ng row and a ead ng row Thus, the three ­orders r­ eport PrevOn and NextOn va ues for product 101, respect ve y nd cat ng NULL-5/30 for the first (4/12) order, 4/12-7/28 for the second (5/30) order, and 5/30-NULL for the th rd and ast order The ast funct ons to exam ne are PERCENT RANK (rank d str but on), CUME DIST ­(cumu at ve ­d str but on, or percent e), PERCENTILE CONT (cont nuous percent e), and PERCENTILE DISC ­(d screte percent e) The quer es n L st ng 2-33 demonstrate these funct ons, wh ch are a c ose y re ated, by query ng sa es figures across each quarter of two years Listing 2-33  Us ng the PERCENT RANK, CUME DIST, PERCENTILE CONT, and PERCENTILE DISC funct ons.

DECLARE @Sales table(Yr int, Qtr int, Amount money) INSERT INTO @Sales VALUES (2010, 1, 5000), (2010, 2, 6000), (2010, 3, 7000), (2010, 4, 2000), (2011, 1, 1000), (2011, 2, 2000), (2011, 3, 3000), (2011, 4, 4000) -- Distributed across all 8 quarters SELECT Yr, Qtr, Amount, R = RANK() OVER(ORDER BY Amount), PR = PERCENT_RANK() OVER(ORDER BY Amount), CD = CUME_DIST() OVER(ORDER BY Amount) FROM @Sales ORDER BY Amount -- Distributed (partitioned) by year with percentile lookups SELECT Yr, Qtr, Amount, R = RANK() OVER(PARTITION BY Yr ORDER BY Amount), PR = PERCENT_RANK() OVER(PARTITION BY Yr ORDER BY Amount),

100  Part 1  Core SQL Server Development

CD = CUME_DIST() OVER(PARTITION BY Yr ORDER BY Amount), PD5 = PERCENTILE_DISC(.5) WITHIN GROUP (ORDER BY Amount) PD6 = PERCENTILE_DISC(.6) WITHIN GROUP (ORDER BY Amount) PC5 = PERCENTILE_CONT(.5) WITHIN GROUP (ORDER BY Amount) PC6 = PERCENTILE_CONT(.6) WITHIN GROUP (ORDER BY Amount) FROM @Sales ORDER BY Yr, Amount

Yr ---2011 2011 2010 2011 2011 2010 2010 2010

Qtr --1 2 4 3 4 1 2 3

Amount ------1000.00 2000.00 2000.00 3000.00 4000.00 5000.00 6000.00 7000.00

R 1 2 2 4 5 6 7 8

PR ----------------0 0.142857142857143 0.142857142857143 0.428571428571429 0.571428571428571 0.714285714285714 0.857142857142857 1

CD ----0.125 0.375 0.375 0.5 0.625 0.75 0.875 1

Yr ---2010 2010 2010 2010 2011 2011 2011 2011

Qtr --4 1 2 3 1 2 3 4

Amount ------2000.00 5000.00 6000.00 7000.00 1000.00 2000.00 3000.00 4000.00

R 1 2 3 4 1 2 3 4

PR ----------------0 0.333333333333333 0.666666666666667 1 0 0.333333333333333 0.666666666666667 1

CD ---0.25 0.5 0.75 1 0.25 0.5 0.75 1

PD5 ------5000.00 5000.00 5000.00 5000.00 2000.00 2000.00 2000.00 2000.00

OVER(PARTITION OVER(PARTITION OVER(PARTITION OVER(PARTITION

PD6 ------6000.00 6000.00 6000.00 6000.00 3000.00 3000.00 3000.00 3000.00

PC5 ---5500 5500 5500 5500 2500 2500 2500 2500

BY BY BY BY

Yr), Yr), Yr), Yr)

PC6 ---5800 5800 5800 5800 2800 2800 2800 2800

The new funct ons are a based on the RANK funct on ntroduced n SQL Server 2005 So both these quer es a so report on RANK, wh ch w a d both n our exp anat on and your understand ng of each of the new funct ons In the first query, PERCENT RANK and CUME DIST (a ased as PR and CD respect ve y) rank quarter y sa es across the ent re two-year per od Look at the va ue returned by RANK (a ased as R) It ranks each row n the unpart t oned w ndow (a e ght quarters) by do ar amount Both 2011Q2 and 2010Q4 are t ed for $2,000 n sa es, so RANK ass gns them the same va ue (2) The next row breaks the t e, so RANK cont nues w th 4, wh ch accounts for the “empty s ot” created by the two prev ous rows that were t ed Now exam ne the va ues returned by PERCENT RANK and CUME DIST Not ce how they reflect the same nformat on as RANK, w th dec ma va ues rang ng from 0 and 1 The on y d fference between the two s a s ght var at on n the way the r formu as are mp emented, such that PERCENT RANK a ways starts w th 0 wh e CUME DIST a ways starts w th a va ue greater than 0 Spec fica y, PERCENT RANK returns (RANK – 1) / (N – 1) for each row, where N s the tota number of rows n the w ndow Th s a ways returns 0 for the first (or on y) row n the w ndow CUME DIST returns RANK / N, wh ch a ways



Chapter 2  T SQL Enhancements   101

returns a va ue greater than 0 for the first row n the w ndow (wh ch wou d be 1, f there’s on y row) For w ndows w th two or more rows, both funct ons return 1 for the ast row n the w ndow w th dec ma va ues d str buted among a the other rows The second query exam nes the same sa es figures, on y th s t me the resu t set s part t oned by year There are no t es w th n each year, so RANK ass gns the sequent a numbers 1 through 4 to each of the quarters, for 2010 and 2011, by do ar amount You can see that PERCENT RANK and CUME DIST perform the same RANK ca cu at ons as exp a ned for the first query (on y, aga n, part t oned by year th s t me) Th s query a so demonstrates PERCENTILE DISC and PERCENTILE CONT These very s m ar f­ unct ons each accept a percent e parameter (the des red CUME DIST va ue) and “reach n” to the w ndow for the row at or near that percent e We demonstrate by ca ng both funct ons tw ce, once w th a percent e parameter va ue of 5 and once w th 6, return ng co umns a ased as PD5, PD6, PC5, and PC6 Both funct ons exam ne the CUME DIST va ue for each row n the w ndow to find the one nearest to 5 and 6 The subt e d fference between them s that PERCENTILE DISC w return a ­prec se (d screte) va ue from the row w th the match ng percent e (or greater), wh e PERCENTILE CONT nterpo ates a va ue based on a cont nuous range Spec fica y, PERCENTILE CONT returns a va ue rang ng from the row match ng the spec fied percent e—or, f there s no exact match, a ca cu ated va ue based on the spec fied percent e f there s no exact match—and the row w th the next h gher percent e n the w ndow Th s exp a ns the va ues returned by these funct ons n th s query For the year 2010, the 5 percent e (CUME DIST va ue) s ocated exact y on quarter 1, wh ch had $5,000 n sa es Thus PERCENTILE DISC( 5) returns 5000 There s no row n the w ndow w th a ­percent e of 6, so PERCENTILE DISC( 6) matches up aga nst the first row w th a percent e greater than or equa to 6, wh ch s the row for quarter 2 w th $6,000 n sa es, and thus returns 6000 In both cases, PERCENTILE DISC returns a d screte va ue from a row n the w ndow at or greater than the spec fied percent e The same ca cu at ons are performed for 2011, return ng 2000 for ­­PERCENTILE DISC( 5) and 3000 for PERCENTILE DISC( 6), correspond ng to the $2,000 n sa es for quarter 2 ­(percent e 5) and the $3,000 n sa es for quarter 3 (percent e 75) As we stated, PERCENTILE CONT s very s m ar It takes the same percent e parameter to find the row n the w ndow match ng that percent e If there s no exact match, the funct on ca cu ates a va ue based on the sca e of percent es d str buted across the ent re w ndow, rather than ook ng ahead to the row hav ng the next greater percent e va ue, as PERCENTILE DISC does Then t returns the ­med an between that va ue and the va ue found n the row w th the next greater percent e For 2010, the 5 percent e matches up w th 5000 (as before) The next percent e n the w ndow s for 75 for 6000 The med an between 5000 and 6000 s 5500 and thus, PERCENTILE CONT(.5) returns 5500 There s no row n the w ndow w th a percent e of 6, so PERCENTILE CONT(.6) ca cu ates what the va ue for 6 wou d be (somewhere between 5000 and 6000, a b t c oser to 5000) and then ca cu ates the med an between that va ue and the next percent e n the w ndow (aga n, 75 for 6000) Thus, ­PERCENTILE CONT(.6) returns 5800; s ght y h gher than the 5500 returned for PERCENTILE CONT(.5).

102  Part 1  Core SQL Server Development

More Info The PERCENTILE DISC and PERCENTILE CONT functions define their window ordering ­using ORDER BY in a WITHIN GROUP clause rather than in the OVER clause. Thus, you do not (and cannot) specify ORDER BY in the OVER clause. The OVER clause is still required, however, so OVER (with empty parentheses) must be specified even if you don’t want to partition using PARTITION BY.

New Conversion Functions These three funct ons are des gned to ass st you w th pars ng and convert ng between d fferent data types and cu ture-sens t ve str ngs ■

TRY CONVERT

■

PARSE

■

TRY PARSE

The TRY CONVERT funct on s the ong-awa ted “safe” vers on of the CONVERT funct on TRY CONVERT works exact y ke ts ub qu tous counterpart, except that t w return NULL rather than ra se an error f the supp ed data cannot be converted to the spec fied data type For examp e, ­cons der the code n L st ng 2-34 Listing 2-34 Us ng TRY CONVERT for va d and nva d convers ons.

SELECT TRY_CONVERT(money, 'test') AS BadResult SELECT TRY_CONVERT(money, '29.5') AS GoodResult

BadResult ---------------NULL GoodResult ---------------29.50

As you can see, the TRY CONVERT funct on has the same syntax and usage as CONVERT, except that t w never ra se a convers on error The first attempt to convert the str ng ‘test’ to a money data type s nva d of course, but TRY CONVERT s mp y returns NULL n that case and a ows execut on to cont nue The second attempt to convert the str ng ’29.5’ to a money data type s perfect y va d, and performs just as CONVERT wou d The new PARSE funct on understands d fferent date, t me, and currency formats that are ­compat b e w th the var ous cu tures supported by the NET Framework Th s funct on s pa red w th



Chapter 2  T SQL Enhancements   103

ts counterpart TRY PARSE that returns NULL f the spec fied str ng cannot be parsed as requested (rather than throw ng an error as PARSE wou d) L st ng 2-35 demonstrates Listing 2-35 Us ng PARSE to ana yze and convert cu ture spec fic date, t me, and currency str ngs.

SELECT PARSE('Monday, 13 December 2010' AS datetime2 USING 'en-US') AS USResult SELECT PARSE('€345,98' AS money USING 'nl-NL') AS NLResult

USResult -----------------------2010-12-13 00:00:00.000 NLResult -----------------------345.98

The USING c ause te s SQL Server how t shou d nterpret the nput str ng w th respect to cu ture You don’t need to spec fy a cu ture w th PARSE; the cu ture w be assumed by defau t based on the current anguage f USING s om tted (the current anguage can be set us ng the SET LANGUAGE statement) The first statement parses an en-US (U S Eng sh) date-formatted str ng nto the datetime2 data type, and the second statement parses an nl-NL (Nether ands) Euro-formatted currency str ng nto the money data type Because both str ngs conta n punctuat on appropr ate for the spec fied cu tures, the PARSE funct on correct y nterprets the nput str ng n both cases However, the fo ow ng statement ra ses an error SELECT PARSE('$345,98' AS money USING 'nl-NL') AS NLResult

Because the do ar s gn symbo s not va d for a Nether ands-based currency str ng that expects and accepts the euro symbo (€), the PARSE funct on fa s Msg 9819, Level 16, State 1, Line 39 Error converting string value '$345,98' into data type money using culture 'nl-NL'.

To hand e cases where the nput str ng hasn’t been va dated for the spec fied cu ture, use TRY PARSE nstead SELECT TRY_PARSE('$345,98' AS money USING 'nl-NL') AS NLResult

Th s statement returns NULL rather than ra s ng an error Thus, you can use TRY PARSE to va date, and a you need to do s test ts resu t for NULL to determ ne f the va dat on s successfu

New Date and Time Functions A set of new funct ons et you construct dates and t mes based on d screte va ues you supp y for the var ous parts of the overa va ue There s a vers on of th s funct on for each of the supported date and t me data types n SQL Server ( nc ud ng the o der datetime and smalldatetime, a though th s shou d not encourage the r cont nued use) 104  Part 1  Core SQL Server Development

■

DATEFROMPARTS

■

TIMEFROMPARTS

■

DATETIME2FROMPARTS

■

DATETIMEOFFSETFROMPARTS

■

DATETIMEFROMPARTS

■

SMALLDATETIMEFROMPARTS

There s a so a new funct on to return the number of days n the month of a spec fied date ■

EOMONTH

The xxxFROMPARTS funct ons a work the same way; they each expect parameters that spec fy each part of the va ue to be generated Us ng these funct ons, you can create exp c t date and t me va ues w thout the formatt ng and cu ture amb gu ty concerns so common w th str ngs L st ng 2-36 demonstrates these date and t me construct on funct ons Listing 2-36  Construct ng date and t me va ues us ng the new xxxFROMPARTS funct ons.

SELECT SELECT SELECT SELECT SELECT SELECT

DATEFROMPARTS(2010, 12, 31) AS ADate TIMEFROMPARTS(23, 59, 59, 1234567, 7) AS ATime DATETIME2FROMPARTS(2010, 12, 31, 23, 59, 59, 1234567, 7) AS ADateTime2 DATETIMEOFFSETFROMPARTS(2010, 12, 31, 14, 23, 36, 5, 12, 0, 1) ADateTimeOff DATETIMEFROMPARTS(2010, 12, 31, 23, 59, 59, 123) AS ADateTime SMALLDATETIMEFROMPARTS(2010, 12, 31, 23, 59) AS ASmallDateTime

ADate ----------------------2010-12-31 ATime ----------------------23:59:59.1234567 ADateTime2 --------------------------2010-12-31 23:59:59.1234567 ADateTimeOff ---------------------------------2010-12-31 14:23:36.5 +12:00 ADateTime ----------------------2010-12-31 23:59:59.123 ASmallDateTime ----------------------2010-12-31 23:59:00



Chapter 2  T SQL Enhancements   105

The EOMONTH funct on s a handy way to determ ne the ast day of the month of any g ven date As you’d expect, eap years are accounted for as we , as L st ng 2-37 demonstrates Listing 2-37  Us ng the EOMONTH funct on to obta n the number of days n each month.

SELECT SELECT SELECT SELECT SELECT

EOMONTH('1/1/2011') EOMONTH('2/1/2011') EOMONTH('3/1/2011') EOMONTH('4/1/2011') EOMONTH('2/1/2012')

AS LastDayOfMonth UNION ALL -- 31 UNION ALL -- 28 UNION ALL -- 31 UNION ALL -- 30 -- 29 (leap year)

LastDayOfMonth ----------------------2011-01-31 2011-02-28 2011-03-31 2011-04-30 2012-02-29

By comb n ng EOMONTH w th DATEPART, t’s easy to get the day count for any g ven month For examp e, the fo ow ng query returns 29 for the number of days n February 2012 (a eap year) SELECT DATEPART(day, EOMONTH('2/1/2012')) AS DaysInFeb2012

New Logical Functions SQL Server 2012 a so ntroduces these two new funct ons for cond t ona operat ons ■

CHOOSE

■

IIF

These two funct ons are conven ent a ternat ves to more verbose CASE statements, and work ke the same-named V sua Bas c for App cat ons (VBA) funct ons n M crosoft Access CHOOSE accepts an nteger parameter (wh ch wou d need to be a var ab e n any usefu scenar o), fo owed by any number of parameters of any data type (wh ch can be e ther constants or var ab es) The nteger acts as an ndex (one-based) nto the st of parameters to return the tem at a spec fic pos t on, as demonstrated by the L st ng 2-38 Listing 2-38  Us ng the CHOOSE funct on to se ect from a st of tems.

DECLARE @CardTypeId int = 2 -- Master card SELECT CHOOSE(@CardTypeId, 'Amex', 'Master', 'Visa', 'Discover') AS CardType

106  Part 1  Core SQL Server Development

CardType -------Master

When dec d ng between one of two return va ues based on a Boo ean (true/fa se) cond t on, the new IIF (Immed ate IF) funct on offers a conc se way to express your og c The funct on accepts three parameters the first s the Boo ean cond t on to test, the second s the va ue to return f the cond t on s true, and the th rd s the va ue to return f the cond t on s fa se L st ng 2-39 demonstrates Listing 2-39  Us ng the IIF funct on to perform cond t ona test ng.

DECLARE @Num1 int = 45 DECLARE @Num2 int = 40 SELECT IIF(@Num1 > @Num2, 'larger', 'not larger' ) AS Result

Result ---------larger

New String Functions These two new str ng funct ons make t eas er to bu d and format str ngs n T-SQL ■

CONCAT

■

FORMAT

The CONCAT funct on concatenates str ngs just ke the concatenat on operator (+), but s to erant of NULL va ues When you concatenate us ng the + operator, the fina resu t s a ways NULL f any of the str ngs be ng concatenated are NULL Th s common annoyance s often dea t w th by wrapp ng ISNULL convers ons on a the va ues be ng concatenated CONCAT offers a c eaner approach s mp y by treat ng NULL va ues as empty str ngs Essent a y, t mp c t y converts a the spec fied va ues to str ngs before t concatenates them together Cons der th s ne of code SELECT CONCAT('Happy', ' Birthday ', 8, '/', NULL, '30') AS Greeting

Th s statement returns the str ng ‘Happy Birthday 8/30’ by str ng ng together a the va ues ­supp ed to CONCAT The funct on converts the va ues 8 and NULL to the str ngs ‘8’ and ‘’ (empty) respect ve y before concatenat on The new FORMAT funct on br ngs the fu power of NET formatt ng to T-SQL W th t, you can ­ as y express date, t me, and currency va ues n v rtua y any des red format Supp y the va ue to e format n the first parameter, spec fy the formatt ng code n the second parameter, and spec fy the cu ture n the th rd parameter (as w th PARSE and TRY PARSE, the th rd parameter s opt ona , and defau ts to the cu ture for the current y set anguage) Tab e 2-3 shows the var ous supported codes to format dates, t mes, and currenc es n any cu ture



Chapter 2  T SQL Enhancements   107

Table 2-3  Common date and t me formatt ng codes Size

Action

d

Genera date format

t

Genera t me format

D

Long date format

T

Long t me format

Ddd

Abbrev ated day of week

Dddd

Day of week

C

Currency (w th opt ona dec ma pos t ons; e.g., c2)

The code n L st ng 2-40 popu ates a tab e var ab e w th a handfu of d fferent cu tures, and then se ects from the cu tures to demonstrate var ous formatt ng capab t es of the FORMAT funct on Listing 2-40  Us ng the FORMAT funct on to perform cu ture spec fic date, t me, and currency formatt ng.

DECLARE @Cultures table(Culture varchar(10), Lang varchar(50)) INSERT INTO @Cultures VALUES ('en', 'English'), ('nl', 'Dutch'), ('ja', 'Japanese'), ('ru', 'Russian'), ('no', 'Norwegian') DECLARE @d datetime2 = DATETIME2FROMPARTS(2011, 2, 1, 16, 5, 0, 0, 7) DECLARE @m money = 199.99 SELECT Lang, Date = TimeOnly = LongDate = LongTime = Dow = Currency = FROM @Cultures

Lang --------English Dutch Japanese Russian Norwegian

FORMAT(@d, FORMAT(@d, FORMAT(@d, FORMAT(@d, FORMAT(@d, FORMAT(@m,

Date ---------2/1/2011 1-2-2011 2011/02/01 01.02.2011 01.02.2011

'd', Culture), 't', Culture), 'D', Culture), 'T', Culture), 'ddd', Culture), 'c2', Culture)

TimeOnly -------4:05 PM 16:05 16:05 16:05 16:05

LongDate -------------------------Tuesday, February 02, 2011 dinsdag 1 februari 2011 2011年2月1日 1 февраля 2011 г. 1. februar 2011

108  Part 1  Core SQL Server Development

LongTime ---------4:05:00 PM 16:05:00 16:05:00 16:05:00 16:05:00

Dow Currency --- -------Tue $199.99 di € 199,99 火 ¥199.99 BT 199,99p. ti kr 199,99

You can a so express custom date formats us ng masks (a comb nat on of formatt ng codes and punctuat on), as demonstrated by L st ng 2-41 Exerc se caut on w th th s approach, however, as you are hardcod ng formats that may not adapt proper y to d fferent cu tures Listing 2-41  Us ng the FORMAT funct on w th a mask to produce custom date and t me formatt ng.

DECLARE @d datetime2 = DATETIME2FROMPARTS(2011, 2, 1, 16, 5, 0, 0, 7) SELECT FORMAT(@d, 'ddd M/d/yyyy h:mm tt') AS DateAndTime

DateAndTime --------------------------Tue 2/1/2011 4:05 PM

In th s examp e, the M/d/yyyy n the mask pos t ons the month before the day Th s s correct n the Un ted States but reversed n European countr es S mp y us ng D (for ong date) or d (for short date) n the mask avo ds th s prob em, as SQL Server automat ca y adjusts the date d sp ay for the spec fied cu ture

Changed Mathematical Function There s on y one changed funct on n SQL Server 2012 It’s extreme y m nor, but we cover t for ­comp eteness ■

LOG

Th s funct on now a ows you to spec fy any des red base w th wh ch to compute the ogar thm Prev ous y, you cou d use LOG on y to compute the natura ogar thm, or use the LOG10 funct on to compute the ogar thm w th a base of 10 To compute the ogar thm of a number us ng any other base, t was necessary to d v de the natura ogar thm of the des red number by the natura ogar thm of the des red base For examp e, the base 2 ogar thm of 11 can be computed as fo ows SELECT LOG(11) / LOG(2) AS Base2LogOf11

In SQL Server 2012, the LOG funct on now accepts a second opt ona parameter to des gnate a spec fic base for the a gor thm Thus, the prev ous express on can now be coded as fo ows SELECT LOG(11, 2) AS Base2LogOf11

The THROW Statement Error hand ng n T-SQL was very d fficu t to mp ement proper y before SQL Server 2005 ntroduced the TRY/CATCH construct, a feature oose y based on NET’s try/catch structured except on hand ng mode The CATCH b ock g ves you a s ng e p ace to code error hand ng og c n the event that a prob em occurs anywhere ns de the TRY b ock above t Before TRY/CATCH, t was necessary to ­a ways check for error cond t ons after every operat on by test ng the bu t- n system funct on ­

Chapter 2  T SQL Enhancements   109

@@ERROR Not on y d d code become c uttered w th the many @@ERROR tests, deve opers (be ng humans) wou d too often forget to test @@ERROR n every needed p ace, caus ng many unhand ed except ons to go unnot ced In SQL Server 2005, TRY/CATCH represented a vast mprovement over constant y test ng ­ @­ERROR, but RAISERROR has (unt now) rema ned as the on y mechan sm for generat ng your @ own errors In SQL Server 2012, the new THROW statement (aga n, borrowed from throw n the NET ­mode ) s the recommended a ternat ve way to ra se except ons n your T-SQL code (a though ­R AISERROR does reta n severa capab t es that THROW acks, as we’ exp a n short y)

Re-Throwing Exceptions The new THROW statement can be used n two ways F rst, and as we just stated, t can serve as an a ternat ve to RAISERROR, a ow ng your code to generate errors when t detects an unreso vab e ­cond t on n process ng Used for th s purpose, the THROW statement accepts parameters for the ­error code, descr pt on, and state, and works much ke RAISERROR A more spec a zed use of THROW takes no parameters, and can appear on y ns de a CATCH b ock In th s scenar o, an unexpected error occurs n the TRY b ock above, tr gger ng execut on of the CATCH b ock Ins de the CATCH b ock, you can perform genera error hand ng (for examp e, ogg ng the error, or ro ng back a transact on), and then ssue a THROW statement w th no parameters Th s w re-throw the or g na error that occurred—w th ts code, message, sever ty, and state ntact—back up to the c ent, so the error can be caught and hand ed at the app cat on eve as we Th s s an easy and e egant way for you to mp ement a segmented except on hand ng strategy between the database and app cat on ayers In contrast, RAISERROR a ways ra ses a new error Thus, t can on y s mu ate re-throw ng the ­ r g na error by captur ng the ERROR MESSAGE, ERROR SEVERITY, and ERROR STATE n the CATCH o b ock and us ng the r va ues to ra se a new error Us ng THROW for th s purpose s much more s mp e and d rect, as demonstrated n L st ng 2-42 Listing 2-42 Us ng THROW n a CATCH b ock to re throw an error.

CREATE TABLE ErrLog(ErrAt datetime2, Severity varchar(max), ErrMsg varchar(max)) GO BEGIN TRY DECLARE @Number int = 5 / 0; END TRY BEGIN CATCH -- Log the error info, then re-throw it INSERT INTO ErrLog VALUES(SYSDATETIME(), ERROR_SEVERITY(), ERROR_MESSAGE()); THROW; END CATCH

110  Part 1  Core SQL Server Development

In th s code’s CATCH b ock, error nformat on s recorded to the ErrLog tab e and then the or g na error (d v de by zero) s re-thrown for the c ent to catch Msg 8134, Level 16, State 1, Line 4 Divide by zero error encountered.

To confirm that the error was ogged by the CATCH b ock as expected before be ng re-thrown, query the ErrLog tab e SELECT * FROM ErrLog GO ErrAt --------------------------2012-02-30 14:14:35.3361250

Severity -------16

ErrMsg -----------------------------------------Divide by zero error encountered.

Comparing THROW and RAISERROR The fo ow ng tab e summar zes the notab e d fferences that ex st between THROW and RAISERROR Table 2-4 Compar ng THROW and RAISERROR THROW

RA SERROR

Can on y generate user except ons (un ess re throw ng n CATCH b ock)

Can generate user (>= 50000) and system (< 50000) except ons

Supp es ad hoc text; doesn t ut ze sys.messages

Requ res user messages defined n sys.­messages (except for code 50000)

Doesn t support token subst tut ons

Supports token subst tut ons

A ways uses sever ty eve 16 (un ess ­re throw ng n a CATCH b ock)

Can set any sever ty eve

Can re throw or g na except on caught n the TRY b ock

A ways generates a new except on; the or g na ­except on s ost to the c ent

Error messages are buffered, and don t appear n rea t me

Supports WITH NOWAIT to mmed ate flush ­buffered output on error

A user except on s an error w th a code of 50000 or h gher that you define for your app cat on’s use System except ons are defined by SQL Server and have error codes ower than 50000 You can use the new THROW statement to generate and ra se user except ons, but not system except ons On y RAISERROR can be used to throw system except ons Note, however, that when THROW s used n a CATCH b ock to re-throw the except on from a TRY b ock as exp a ned n the prev ous sect on, the actua or g na except on—even f t’s a system except on—w get thrown (th s was demonstrated n the prev ous “d v de by zero” examp e) When RAISERROR s used w thout an error code, SQL Server ass gns an error code of 50000 and expects you to supp y an ad-hoc message to assoc ate w th the error The THROW statement a ways expects you to supp y an ad-hoc message for the error, as we as a user error code of 50000 or h gher Thus, the fo ow ng two statements are equ va ent THROW 50000, 'An error occurred querying the table.', 1; RAISERROR ('An error occurred querying the table.', 16, 1);



Chapter 2  T SQL Enhancements   111

Both these statements ra se an error w th code 50000, sever ty 16, state 1, and the same ­ essage text Compat b ty between the two keywords ends there, however, as vary ng usages m mpose d ­ fferent ru es (as summar zed n Tab e 2-4) For examp e, on y RAISERROR supports token ­subst tut on RAISERROR ('An error occurred querying the %s table.', 16, 1, 'Customer'); Msg 50000, Level 16, State 1, Line 22 An error occurred querying the Customer table.

THROW has no s m ar capab ty A so, wh e RAISERROR ets you spec fy any sever ty eve , THROW w a ways generate an error w th a sever ty eve of 16 Th s s s gn ficant, as eve 11 and h gher nd cates more ser ous errors than eve 10 and ower For examp e RAISERROR ('An error occurred querying the table.', 10, 1); An error occurred querying the table.

Because the sever ty s 10, th s error does not echo the error code, eve , state, and ne number, and s d sp ayed n b ack rather than the usua red that s used for sever ty eve s h gher than 10 In contrast, THROW cannot be used to s gna a non-severe error The ast mportant d fference between the two keywords s the RAISERROR assoc at on w th sys. messages In part cu ar, RAISERROR requ res that you ca sys.sp addmessage to define error ­messages assoc ated w th user error codes h gher than 50000 As exp a ned, the RAISERROR syntax n our ­ear er examp es uses an error code of 50000, and s the on y supported syntax that ets you supp y an ­ad-hoc message nstead of ut z ng sys.messages The fo ow ng code demonstrates how to define customer user error messages for ­R AISERROR F rst (and on y once), a token zed message for user error code 66666 s added to sys.­messages ­Thereafter, RAISERROR references the error by ts code, and a so supp es va ues for token ­rep acements that are app ed to the message’s text n sys.messages EXEC sys.sp_addmessage 66666, 16, 'There is already a %s named %s.'; RAISERROR(66666, 16, 1, 'cat', 'morris'); Msg 66666, Level 16, State 1, Line 34 There is already a cat named morris.

The THROW statement has no such requ rement You supp y any ad-hoc message text w th THROW You don’t need to separate y manage sys.messages, but th s a so means that THROW can’t (d rect y) everage centra y managed error messages n sys.messages ke RAISERROR does ­Fortunate y, the FORMATMESSAGE funct on prov des a workaround f you want to take advantage of the same capab ty w th THROW You just need to take care and make sure that the same error code s spec fied n the two p aces that you need to reference t (once for FORMATMESSAGE and once for THROW), as shown here

112  Part 1  Core SQL Server Development

DECLARE @Message varchar(max) = FORMATMESSAGE(66666, 'dog', 'snoopy'); THROW 66666, @Message, 1; Msg 66666, Level 16, State 1, Line 40 There is already a dog named snoopy.

Server-Side Paging Return ng paged query resu ts was d fficu t to ach eve pr or to SQL Server 2005 That re ease of SQL Server ntroduced a ser es of rank ng funct ons, nc ud ng the ROW NUMBER funct on that made t poss b e to return one page at a t me from your query Let’s first see how to use ROW NUMBER ­added n SQL Server 2005 to mp ement server-s de pag ng, and then you w earn how much eas er t s to ach eve the same goa us ng the new OFFSET/FETCH NEXT syntax ntroduced n SQL Server 2012

Using ROW_NUMBER The ROW NUMBER funct on, as ts name mp es, generates a sequent a number for each row n the resu t set returned by your query The va ue returned by the ROW NUMBER funct on can then be used n an outer query’s WHERE c ause to m t the resu t set to just the des red page L st ng 2-43 shows an examp e us ng the AdventureWorks2012 database that demonstrates th s techn que It ­returns “page 3” of the query resu ts, wh ch (assum ng a page s ze of 10) are rows 21 through 30 Listing 2-43  Return ng paged resu ts us ng a nested subquery and the ROW NUMBER funct on.

USE AdventureWorks2012 GO -- Using ROW_NUMBER introduced in SQL Server 2005 DECLARE @PageNum int = 3 DECLARE @PageSize int = 10 DECLARE @FirstRow int = ((@PageNum - 1) * @PageSize) + 1 DECLARE @LastRow int = @FirstRow + @PageSize - 1 SELECT * FROM (SELECT RowNum = ROW_NUMBER() OVER (ORDER BY LastName, FirstName), Title, FirstName, LastName FROM Person.Person ) AS a WHERE RowNum BETWEEN @firstRow AND @lastRow ORDER BY LastName, FirstName



Chapter 2  T SQL Enhancements   113

And here are the resu ts RowNum -------21 22 23 24 25 26 27 28 29 30

Title -------NULL Mr. NULL Ms. NULL NULL NULL NULL NULL NULL

FirstName ----------Bailey Ben Blake Carla Carlos Charles Chloe Connor Courtney Dalton

LastName ---------Adams Adams Adams Adams Adams Adams Adams Adams Adams Adams

(10 row(s) affected)

Now th s certa n y works, but there are two undes rab es here F rst, t requ res you to ­manufacture the row number as an add t ona co umn n your resu t set, whether or not you want or need t ­Second, the syntax s somewhat contorted; the requ red use of a nested SELECT statement and ­ u t p e ORDER BY c auses (as we as the requ red a as “AS a”) s both awkward and un ntu t ve m

Using OFFSET/FETCH NEXT Now take a ook at how the same resu t can be ach eved n SQL Server 2012 us ng the code shown n L st ng 2-44 Listing 2-44  Return ng paged resu ts us ng the OFFSET/FETCH NEXT syntax.

DECLARE @PageNum int = 3 DECLARE @PageSize int = 10 DECLARE @Offset int = (@PageNum - 1) * @PageSize SELECT Title, FirstName, LastName FROM Person.Person ORDER BY LastName, FirstName OFFSET @Offset ROWS FETCH NEXT @PageSize ROWS ONLY

The syntax cou dn’t be any s mp er—just spec fy your start ng row w th OFFSET and your page s ze w th FETCH NEXT Here are the resu ts Title -------NULL Mr. NULL Ms. NULL NULL

FirstName ----------Bailey Ben Blake Carla Carlos Charles

LastName ---------Adams Adams Adams Adams Adams Adams

114  Part 1  Core SQL Server Development

NULL NULL NULL NULL

Chloe Connor Courtney Dalton

Adams Adams Adams Adams

(10 row(s) affected)

You can see that th s query returns the same “page” as the prev ous vers on of the query that used the ROW NUMBER funct on, yet t wasn’t necessary to manufacture a row number co umn to do t, nor was t necessary to code a nested subquery Furthermore, OFFSET/FETCH NEXT performs s ght y faster than us ng ROW NUMBER, and s s gn ficant y faster than the pre–SQL Server 2005 hacks ­nvo v ng stored procedures and temp tab es If you st want row numbers returned n your resu t set, you can comb ne OFFSET/FETCH w th ROW NUMBER f des red by rep ac ng the SELECT statement n L st ng 2-44 w th SELECT RowNum = ROW_NUMBER() OVER (ORDER BY LastName, FirstName), Title, FirstName, LastName FROM Person.Person ORDER BY LastName, FirstName OFFSET @Offset ROWS FETCH NEXT @PageSize ROWS ONLY

The resu ts of th s query are dent ca to the resu ts shown for L st ng 2-43, but doesn’t requ re ­cod ng the subquery seen n that st ng A so note that OFFSET can be spec fied w thout FETCH NEXT to sk p a spec fied number of rows and return a rema n ng rows

The SEQUENCE Object H stor ca y, one notab e d fference between SQL Server and other database p atforms (such as Orac e and DB2) has been the manner n wh ch you mp ement automat ca y ass gned nteger va ues for ­pr mary keys when nsert ng new rows SQL Server prov des the IDENTITY attr bute for the int and bigint data types, whereas other p atforms requ re you to create an ndependent “sequence ­generator” object that feeds ncrement ng va ues to new rows n the tab e These are two d fferent ways to ach eve the same th ng, and SQL Server’s IDENTITY s arguab y s mp er to use, but sequence ­generators offer the r own un que advantages as we SQL Server 2012 now a so supports sequence generators as a powerfu a ternat ve to us ng the IDENTITY attr bute Sequences can be created on nteger types (both bu t- n and user-defined), and you can spec fy the m n mum, max mum, start, and ncrement (or decrement) va ues for the sequence You can a so cyc e back around to the m n mum va ue when the max mum va ue s reached (or v ce versa) Sequences are most common y used to ass gn new pr mary key va ues on INSERT operat ons as an a ternat ve to us ng IDENTITY-attr buted key co umns But because they ex st as objects ­ndependent of the data tab es that they feed new pr mary keys to, they offer more flex b ty and pose fewer ­ m tat ons For examp e, you can obta n the next va ue n the sequence before you perform the ­INSERT, and you can nsert any (un que) va ue nto the pr mary key co umn w thout requ r ng SET

Chapter 2  T SQL Enhancements   115

IDENTITY INSERT ON/OFF statements The NEXT VALUE FOR syntax can be used n any SELECT, INSERT, or UPDATE statement to request and ncrement the next va ue n the sequence Let’s see how sequences work by exam n ng the code n L st ng 2-45 Listing 2-45  Us ng a sequence to generate new pr mary keys.

CREATE TABLE Customer (Id int PRIMARY KEY, FirstName varchar(max), LastName varchar(max)) -- Create the sequence with a start, increment, and min/max settings CREATE SEQUENCE CustomerSequence AS int START WITH 1 INCREMENT BY 1 MINVALUE 0 NO MAXVALUE -- Use INSERT (NEXT (NEXT (NEXT

the sequence for new primary key values INTO Customer (Id, FirstName, LastName) VALUES VALUE FOR CustomerSequence, 'Bill', 'Malone'), VALUE FOR CustomerSequence, 'Justin', 'Thorp'), VALUE FOR CustomerSequence, 'Paul', 'Duffy')

The Customer tab e uses an nteger pr mary key, but does not spec fy the IDENTITY ­attr bute Instead, you use the new CREATE SEQUENCE statement n SQL Server 2012 to create a ­CustomerSequence object that feeds new ntegers, start ng w th one and ncrement ng by one, w th no spec fied upper m t (beyond the max mum s ze for the nteger data type; a 32-b t int data type n th s examp e) You then INSERT three new rows, each of wh ch spec fies NEXT VALUE FOR ­CustomerSequence as the new Id va ue (note the row constructor syntax, ntroduced n SQL Server 2008, that nserts the three rows w th a s ng e statement) The resu t, as you’d expect, appears ke th s SELECT * FROM Customer GO Id ----1 2 3

FirstName --------Bill Justin Paul

LastName ------------Malone Thorp Duffy

(3 row(s) affected)

Most ke y, you’ want to emu ate the exper ence of us ng the IDENTITY attr bute; that s, you may w sh to om t the pr mary key va ues from the INSERT statement and put SQL Server n charge of ­ass gn ng them to new rows Th s s eas y done by estab sh ng NEXT VALUE FOR as a DEFAULT ­constra nt on the Id co umn, as shown n L st ng 2-46

116  Part 1  Core SQL Server Development

Listing 2-46 Emu at ng IDENTITY attr buted pr mary key co umns w th sequences us ng a DEFAULT constra nt.

-- Set the default for IDENTITY-behavior ALTER TABLE Customer ADD DEFAULT NEXT VALUE FOR CustomerSequence FOR Id -- Generates customer ID 4 INSERT INTO Customer (FirstName, LastName) VALUES('Jeff', 'Smith')

W th the DEFAULT constra nt n p ace, the INSERT statement ooks and works just the same as f you were us ng an IDENTITY-attr buted pr mary key co umn Customer Jeff Sm th s ­automat ca y a ­ ss gned an Id va ue of 4 But by us ng sequences, you can enjoy a few add t ona benefits For ­examp e, you can peek at the current va ue w thout consum ng t by query ng the sys.sequences cata og v ew Among the many parameters for each sequence object n the database exposed by th s v ew, the current value co umn revea s the current y ass gned va ue SELECT current_value FROM sys.sequences WHERE name='CustomerSequence' GO current_value ------------4

You can a so use the ALTER SEQUENCE statement to change the behav or of an ex st ng sequence object, as shown n L st ng 2-47 Listing 2-47  Chang ng a sequence object us ng ALTER SEQUENCE.

ALTER SEQUENCE CustomerSequence RESTART WITH 1100 MINVALUE 1000 MAXVALUE 9999 CYCLE

Now the next va ue returned by CustomerSequence w be 1100, and t w cont nue to ­ncrement by one from there When t tops 9999, t w start aga n from 1000 and cont nue ncrement ng norma y Natura y, g ven the un queness of pr mary keys, the sequence can no onger be used to popu ate the Customer tab e 100 rows after t recyc es to 1000, because you a ready have customers w th pr mary keys start ng at 1100

Sequence Limitations Of course, sequences have some restr ct ons that need to be ca ed out Sequences cannot be used n a subquery, CTE, TOP c ause, CHECK CONSTRAINT defin t on, or as an argument to an aggregate funct on They a so can’t be used n v ews, computed co umns, and user-defined funct ons, as those object types are not a owed to cause the s de effects of sequence number generat on F na y, you

Chapter 2  T SQL Enhancements   117

cannot ssue a DROP SEQUENCE statement to de ete a sequence wh e you have tab es w th ­ex st ng DEFAULT constra nts that reference the sequence; you’ need to drop the referenc ng constra nts before you can drop the sequence

Metadata Discovery Modern deve opment too s de ver cr t ca product v ty features, such as graph ca des gners and code generators, and those capab t es re y heav y on database metadata d scovery Indeed, t has a ways been poss b e to nterrogate SQL Server for metadata nformat on You can eas y ­d scover a the ­objects n a database (tab es, v ews, stored procedures, and so on) and the r types by d rect y ­query ng system tab es (not recommended, as they can change from one vers on of SQL Server to a ­ nother) or nformat on schema v ews (wh ch are cons stent n each SQL Server vers on) It s s­gn ficant y more cha eng ng, however, to d scover the resu t set schema for T-SQL statements or stored procedures that conta n cond t ona og c Us ng SET FMTONLY ON/OFF has been the common techn que n the past for d scover ng the schema of a query’s resu t set w thout actua y execut ng the query tse f To demonstrate, execute the code shown n L st ng 2-48 Listing 2-48  Us ng the o d (now deprecated) SET FMTONLY ON/OFF techn que for d scover ng a tab e s schema.

USE AdventureWorks2012 GO SET FMTONLY ON SELECT * FROM HumanResources.Employee; SET FMTONLY OFF

Th s SELECT statement, wh ch wou d norma y return a the rows from the HumanResources.­ Employee tab e, returns no rows at a It just revea s the co umns The SET FMTONLY ON statement prevents quer es from return ng rows of data so that the r schemas can be d scovered, and th s ­behav or rema ns n effect unt SET FMTONLY OFF s encountered SQL Server 2012 ntroduces severa new system stored procedures and tab e-va ued funct ons (TVFs) that prov de s gn ficant y r cher metadata d scovery than what can be d scerned us ng the re at ve y ne egant (and now deprecated) SET FMTONLY ON/OFF approach These new procedures and funct ons are ■

sys sp descr be first resu t set

■

sys sp descr be undec ared parameters

■

sys dm exec descr be first resu t set

■

sys dm exec descr be first resu t set for object

118  Part 1  Core SQL Server Development

The sys.sp describe first result set accepts a T-SQL statement and produces a h gh y deta ed schema descr pt on of the first poss b e resu t set returned by that statement The fo ow ng code retr eves schema nformat on for the same SELECT statement you used ear er to get nformat on on a the co umns n the HumanResources.Employee tab e EXEC sp_describe_first_result_set @tsql = N'SELECT * FROM HumanResources.Employee'

F gure 2-3 shows the wea th of nformat on that SQL Server returns about each co umn n the resu t set returned by the T-SQL statement

Figure 2-3  Deta ed metadata returned by sp describe first result set.

A data management funct on named sys.dm exec describe first result set works very s m ar to sys.sp describe first result set But because t s mp emented as a TVF, t s easy to query aga nst t and m t the metadata returned For examp e, the query n L st ng 2-49 exam nes the same T-SQL statement, but returns just the name and data type of nu ab e co umns Listing 2-49  Query ng the new data management funct on to d scover nu ab e co umns n a tab e.

SELECT name, system_type_name FROM sys.dm_exec_describe_first_result_set( 'SELECT * FROM HumanResources.Employee', NULL, 1) WHERE is_nullable = 1



Chapter 2  T SQL Enhancements   119

name ----------------OrganizationNode OrganizationLevel

system_type_name ---------------hierarchyid smallint

Parameter zed quer es are a so supported, f you supp y an appropr ate parameter s gnature after the T-SQL The T-SQL n the prev ous examp e had no parameters, so t passed NULL for the “­parameters parameter ” The code n the L st ng 2-50 d scovers the schema of a parameter zed query Listing 2-50  D scover ng the schema returned by a parameter zed query.

SELECT name, system_type_name, is_hidden FROM sys.dm_exec_describe_first_result_set(' SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader WHERE SalesOrderID = @OrderID', '@OrderID int', 1)

name --------------OrderDate TotalDue SalesOrderID

system_type_name ---------------datetime money int

is_hidden --------0 0 1

You’d be qu ck to quest on why the SalesOrderID co umn s returned for a SELECT statement that returns on y OrderDate and TotalDue The answer es n the ast parameter passed to the data ­management funct on A bit va ue of 1 (for true) te s SQL Server to return the dent fy ng ­Sa­lesOrderID co umn, because t s used to “browse” the resu t set Not ce that t s marked true (1) for is hidden Th s nforms the c ent that the SalesOrderID co umn s not actua y revea ed by the query, but can be used to un que y dent fy each row n the query’s resu t set What f mu t p e resu t sets are poss b e? There’s no prob em w th th s as ong as they a have the same schema In fact, SQL Server w even try to forg ve cases where mu t p e poss b e schemas are not exact y dent ca For examp e, f the same co umn s nu ab e n one resu t set and non-nu ab e n the other, schema d scovery w succeed and nd cate the co umn as nu ab e It w even to erate cases where the same co umn has a d fferent name (but same type) between two poss b e resu t sets, and nd cate NULL for the co umn name, rather than arb trar y choos ng one of the poss b e co umn names or fa ng a together The code n L st ng 2-51 demonstrates th s w th a T-SQL statement that has two poss b e resu t sets depend ng on the va ue passed n for the @SortOrder parameter Because both resu t sets have ­compat b e schemas, the data management funct on succeeds n return ng schema nformat on

120  Part 1  Core SQL Server Development

Listing 2-51  D scover ng the schema returned from a T SQL statement w th mu t p e compat b e resu t sets.

SELECT name, system_type_name FROM sys.dm_exec_describe_first_result_set(' IF @SortOrder = 1 SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader ORDER BY SalesOrderID ASC ELSE IF @SortOrder = -1 SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader ORDER BY SalesOrderID DESC’, '@SortOrder AS tinyint', 0)

name ----------OrderDate TotalDue

system_type_name ---------------datetime money

D scovery won’t succeed f SQL Server detects ncompat b e schemas For examp e, n L st ng 2-52, the ca to the system stored procedure spec fies a T-SQL statement w th two poss b e resu t sets, but one returns three co umns but the other returns on y two co umns Listing 2-52  Attempt ng to d scover schema nformat on returned from a T SQL statement w th mu t p e ­ncompat b e resu t sets.

EXEC sys.sp_describe_first_result_set @tsql = N' IF @IncludeCurrencyRate = 1 SELECT OrderDate, TotalDue, CurrencyRateID FROM Sales.SalesOrderHeader ELSE SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader'

In th s case, the system stored procedure ra ses an error that c ear y exp a ns the prob em Msg 11509, Level 16, State 1, Procedure sp_describe_first_result_set, Line 53 The metadata could not be determined because the statement 'SELECT OrderDate, TotalDue, CurrencyRateID FROM Sales.SalesOrderHeader' is not compatible with the statement 'SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader'.

It s noteworthy to ment on that the data management funct on copes w th th s scenar o much more pass ve y G ven confl ct ng resu t set schemas, t s mp y returns NULL and does not ra se an ­error



Chapter 2  T SQL Enhancements   121

The data management funct on sys.dm exec describe first result set for object can be used to ach eve the same d scovery aga nst any object n the database It accepts just an object ID and the Boo ean “browse” flag to spec fy f h dden ID co umns shou d be returned You can use the OBJECT ID funct on to obta n the ID of the des red object The examp e n L st ng 2-53 demonstrates th s by return ng schema nformat on for the stored procedure GetOrderInfo Listing 2-53  D scover ng metadata returned from a stored procedure dent fied by ts object D.

CREATE PROCEDURE GetOrderInfo(@OrderID AS int) AS SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader WHERE SalesOrderID = @OrderID GO SELECT name, system_type_name, is_hidden FROM sys.dm_exec_describe_first_result_set_for_object(OBJECT_ID('GetOrderInfo'), 1)

name --------------OrderDate TotalDue SalesOrderID

system_type_name ----------------datetime money int

is_hidden --------0 0 1

F na y, the sys.sp describe undeclared parameters stored procedure parses a T-SQL statement to d scover type nformat on about the parameters expected by the statement, as L st ng 2-54 ­demonstrates Listing 2-54  D scover ng parameters (and the r data types) expected by a T SQL statement.

EXEC sys.sp_describe_undeclared_parameters N'IF @IsFlag = 1 SELECT 1 ELSE SELECT 0'

parameter_ordinal name suggested_system_type_id suggested_system_type_name ... ----------------- ------- ------------------------ -------------------------- ------1 @IsFlag 56 int ...

In th s examp e, SQL Server detects the @IsFlag parameter, and suggests the int data type based on the usage n the T-SQL statement t was g ven to parse

Summary We covered a ot of ground n th s chapter As M crosoft cont nues to nvest n T-SQL, deve opers are rewarded w th many powerfu enhancements to the T-SQL eng ne You’ve earned how to use tab e-va ued parameters (TVPs) to pass sets of rows across c ent, server, stored procedures, and userdefined funct ons (UDFs) The date and t me data types were a so covered, as we as the capab t es 122  Part 1  Core SQL Server Development

of MERGE, INSERT OVER DML, and GROUPING SETS You a so earned how to use the atest features n SQL Server 2012, nc ud ng a host of new T-SQL funct ons, the w ndow ng (OVER c ause) enhancements, the THROW statement, server-s de pag ng, SEQUENCE objects, and metadata d scovery mprovements T-SQL s st the ma n way you w program aga nst your data When your operat ons requ re you to nteract w th the operat ng system or use the NET Framework, you can use the CLR to wr te your quer es n V sua Bas c NET or V sua C# In the next chapter, you’ earn how



Chapter 2  T SQL Enhancements   123

C hapter 3

Exploring SQL CLR —Andrew Brust

T

he banner head ne for M crosoft SQL Server 2005 was ts ntegrat on of the M crosoft NET ­common anguage runt me (CLR) Th s arch tectura enhancement gave SQL Server the ab ty to use certa n NET c asses as bas c data types, as we as accommodate the use of NET anguages for the creat on of stored procedures, tr ggers, funct ons, and even user-defined aggregates

Th s capab ty was carr ed forward and enhanced n SQL Server 2008 and s the under y ng enab er of var ous “beyond re at ona ” SQL Server data types, nc ud ng hierarchyid (covered n Chapter 7), and geometry and geography (covered n Chapter 9) SQL CLR techno ogy rema ns mportant n SQL Server 2012 and we w cover the techno ogy, nc ud ng ts deve opment and dep oyment us ng SQL Server Data Too s (SSDT), n th s chapter

Note  Throughout th s chapter, we w refer to the CLR ntegrat on n SQL Server as SQL CLR ­features, funct ona ty, or ntegrat on, and we w refer to SQL CLR stored procedures, tr ggers, funct ons, ­aggregates, and user-defined types (UDTs) as the five bas c SQL CLR ent t es

In th s chapter, you w

earn

■

How to enab e (or d sab e) SQL CLR ntegrat on on your SQL Server

■

How SQL Server accommodates CLR code through the oad ng of NET assemb es

■

■

■

■

How to use SQL Server 2012, M crosoft V sua Stud o 2010, and SSDT together to wr te SQL CLR code and dep oy t, s mp y and qu ck y How to dep oy SQL CLR code ndependent y of V sua Stud o, us ng T-SQL statements, w th or w thout the he p of SQL Server Management Stud o (SSMS) How to create s mp e CLR stored procedures, tr ggers, funct ons, aggregates, and UDTs, use them n your databases, and ut ze them from Transact-SQL (T-SQL) How both the standard SQL Server c ent prov der and the new server-s de brary can be ­comb ned to mp ement SQL CLR funct ona ty

■

How SQL CLR secur ty works and how to configure secur ty perm ss ons for your assemb es

■

When to use SQL CLR funct ona ty, and when to opt to use T-SQL nstead

125

Getting Started: Enabling CLR Integration Before you can earn how to use SQL CLR features, you need to know how to enab e them As w th many new products n the M crosoft W ndows Server system fam y, most advanced features of SQL Server are d sab ed by defau t The reason ng beh nd th s s sound each add t ona feature that s ­enab ed prov des extra “surface area” for attacks on secur ty or ntegr ty of the product, and the added exposure s s mp y not just fied f the feature goes unused The SQL CLR features of SQL Server 2012 are soph st cated and can be very usefu , but they are a so, techn ca y, nonessent a It s poss b e to bu d h gh-performance databases and server-s de programm ng og c w thout SQL CLR ntegrat on, so t s turned off by defau t Don’t be d scouraged, though; turn ng on the feature s easy M crosoft prov des a system stored procedure for enab ng or d sab ng SQL CLR ntegrat on Connect to the server you’d ke to configure n SSDT or SSMS Then, from a query w ndow, type the fo ow ng statements, and execute the scr pt sp_configure 'clr enabled', 1  GO  RECONFIGURE  GO

That’s a there s to t! To d sab e SQL CLR ntegrat on, just use a va ue of 0, nstead of 1, as the second parameter va ue n the sp configure ca

Tip  Don’t forget that this will work from any tool that can connect to SQL Server, not just SSDT and SSMS. In fact, you could issue the previous command text from your own code using the ADO.NET SqlCommand object’s ExecuteNonQuery method as long as your code can connect to your server and your server can authenticate you as a user in the sysadmin server role. W th SQL CLR ntegrat on enab ed, you’re ready to get started wr t ng SQL CLR code Before you d ve n though, we need to d scuss V sua Stud o/SQL Server ntegrat on and when to use t

Visual Studio/SQL Server Integration SSDT and SQL Server 2012 ntegrate t ght y n a number of ways It’s mportant to rea ze, however, that the use of SSDT s comp ete y opt ona and the use of T-SQL s a suffic ent subst tute W th the re ease of SQL Server 2005, T-SQL was enhanced w th new data defin t on anguage (DDL) ­commands for ma nta n ng CLR assemb es, types, and aggregates, and ts ex st ng commands for stored ­procedures, tr ggers, and funct ons were enhanced to recogn ze code w th n dep oyed assemb es V sua Stud o can execute those commands on your beha f It can a so make wr t ng nd v dua SQL CLR c asses and funct ons eas er

126  Part I  Core SQL Server Development

U t mate y, we th nk a deve opers shou d be aware of both SSDT–ass sted and more manua c­ od ng and dep oyment methods You m ght dec de to use one method most of the t me, but n some s tuat ons you’ probab y need the other as we , so we want to prepare you As we cover each major area of SQL CLR programm ng, we w d scuss dep oyment from both po nts of v ew We’ cover some genera po nts about V sua Stud o ntegrat on now, and then we’ move on to cover SQL CLR ­deve opment

SQL Server Database Projects in Visual Studio The SSDT SQL Server Database Project type defines temp ates for the five bas c SQL CLR ent t es These temp ates nject spec fic code attr butes and funct on stubs that a ow you to create SQL CLR code eas y The attr butes are used by SSDT to dep oy your assemb y and ts stored procedures, ­tr ggers, and so on to your database Some of them are a so used by SQL Server to acknow edge and proper y use your funct ons, UDTs, and aggregates To test out the new project type and temp ates, fo ow th s procedure 1. Start V sua Stud o 2010, and then create a new project by choos ng F e New Project,

c­ ck ng New Project on the too bar, press ng Ctrl+Shift+N, or c ck ng the New Project… nk on the V sua Stud o Start Page 2. In the New Project d a og box, shown n F gure 3-1, c ck the expand g yph to the eft of the

Other Languages node n the Insta ed Temp ates tree v ew on the eft, c ck that node’s SQL Server ch d node, and then c ck SQL Server Database Project n the m dd e pane Enter your own project name f you want, and then c ck OK

Figure 3-1  The V sua Stud o 2010 New Project d a og box w th the SQL Server Database Project type

se ected.

You can eas y add preconfigured c asses for the five bas c SQL CLR ent t es to your project, but you must first dec de whether you w sh to use C# or V sua Bas c NET as the programm ng anguage for your SQL CLR Assemb y



Chapter 3  Exp or ng SQL CLR   127

3. Doub e-c ck the Properties node n the So ut on Exp orer, and then c ck the SQLCLR tab n the

resu t ng property sheet des gner Once ns de the SQLCLR tab, se ect C# from the Language combo box, as shown n F gure 3-2

Figure 3-2  The SQL CLR property sheet w th C# se ected as the programm ng anguage.

4. Now you’re ready to add a CLR ent ty to your project You can do th s from the Add New Item

d a og box, wh ch you d sp ay by se ect ng Project Add New Item from the ma n menu or by choos ng Add New Item from the project node’s shortcut menu n So ut on Exp orer If you se ect the SQL CLR C# (or SQL CLR VB) temp ate type from the “Insta ed Temp ates” st on the eft of the Add New Item d a og box, t shou d appear as shown n F gure 3-3

Figure 3-3  The V sua Stud o SQL Server Database Project Add New tem d a og box, w th SQL CLR C# temp ates d sp ayed.

128  Part I  Core SQL Server Development

After se ect ng an ent ty type, a c ass temp ate for that type w be added to your project and opened n the code ed tor w ndow Add t ona y, references to the System, System.Data, and ­System.Xml assemb es w be added to the project These references are requ red by the stubbed code that appears n the SQL CLR c ass temp ates

Automated Deployment Once opened, the use of the SQL Server Database Project temp ate adds a Pub sh opt on to the Bu d opt on of the V sua Stud o ma n menu, wh ch can be used to dep oy the assemb y and the SQL CLR ent t es w th n t SSDT can do a ot of dep oyment work for you But as you’ earn, you can perform the same tasks on your own and, n certa n c rcumstances, have more prec se contro over the dep oyment process when you do so

SQL CLR Code Attributes A number of NET code attr butes are prov ded for SQL CLR deve opers; these are conta ned n the Microsoft.SqlServer.Server namespace Many of them are nserted n your code when you use the ­var ous temp ates n the SQL Server Database Project type, as s a using statement (or an Imports statement n V sua Bas c) to a as the Microsoft.SqlServer.Server namespace tse f If you choose to ­deve op code w thout these temp ates, you must add the appropr ate attr butes, and opt ona y the using (or Imports) statement yourse f A though a these attr butes are prov ded n the same namespace, some are used exc us ve y by SSDT and others are used by both SSDT and SQL Server More SQL CLR attr butes are ava ab e for you to use than we w be ab e to cover n th s ­chapter Spec fica y, we w prov de coverage of the SqlProcedure, SqlFunction, SqlTrigger, ­SqlUser­DefinedAggregate, and SqlUserDefinedType attr butes We w not cover the SqlFacet and SqlMethod attr butes Just as certa n attr butes are not covered here, we cover on y some of the parameters accepted by the attr butes that we do cover And n some cases, we cover on y certa n va ues that can be passed to these attr butes of the many poss b e For examp e, SqlFunction accepts severa parameters, but the on y ones we w cover are Name, FillRowMethodName, and TableDefinition For SqlUserDefinedAggregate and SqlUserDefinedType, we w cover on y a s ng e va ue sett ng for the Format parameter and w not cover the severa other parameters those two attr butes accept The coverage we prov de w be more than suffic ent for you to mp ement bas c, ­ntermed ate, and certa n advanced funct ona ty w th a of the five bas c SQL CLR ent t es The attr butes and parameters that we won’t cover are usefu most y for opt m z ng your SQL CLR code, and they are we documented n SQL Server Books On ne and art c es on MSDN



Chapter 3  Exp or ng SQL CLR   129

About the Sample Code The samp e NET code for th s chapter s prov ded on the book’s compan on webs te (see the “Introduct on” for nstruct ons to down oad the samp e code) n two vers ons The pr mary mater a s supp ed as an SSDT SQL Server Database Project, access b e by open ng the so ut on fi e SQLCLRDemo.sln n the SQLCLRDemo fo der under the VS (V sua Stud o) subfo der of the samp e code fo der We a so supp y the code as a standard C ass L brary project, access b e by open ng the so ut on fi e SQLCLRDemoManual.sln n the SQLCLRDemoManua subfo der The code n each project s v rtua y dent ca , a though the C ass L brary project’s comp ed assemb y cannot be auto-dep oyed to a SQL Server database As we cover each SQL CLR feature, we’ d scuss how automated dep oyment takes p ace from the SSDT SQL Server Database Project and how command-dr ven dep oyment shou d be performed for the C ass L brary project We’ a so d scuss execut ng test scr pts As a compan on to those d scuss ons, we prov de a SQL Server Management Stud o project, access b e by open ng SQLCLRDemo.ssmssln n the SQLCLRDemo fo der under the samp e code fo der’s SSMS subfo der Th s project cons sts of a number of SQL scr pts used for test ng the samp e SQL CLR code, and a scr pt for c ean ng up everyth ng n the database created by the samp e code and tests The project a so conta ns a scr pt fi e named CreateObjects.sql, wh ch dep oys the C ass L brary assemb y and the SQL CLR ent t es w th n t

Your First SQL CLR Stored Procedure A though SQL CLR programm ng can get qu te comp ex and nvo ved, n rea ty t offers a s mp e mode that any NET deve oper can use w th h gh product v ty n re at ve y short order That’s ­because the crux of SQL CLR funct ona ty s noth ng more than the ab ty of SQL Server 2012 to oad NET ­assemb es nto your database and then a ow you to use the funct ons and types w th n the ­assemb y as you define your co umns, v ews, stored procedures, tr ggers, and funct ons To ga n a good understand ng of SQL CLR ntegrat on, you must exam ne ts features and t­ echn ques carefu y Before do ng so, however, et’s qu ck y wa k through an end-to-end scenar o for creat ng and execut ng a SQL CLR stored procedure Th s w make t eas er for you to understand the nd v dua features as we descr be them Str ct y speak ng, any NET c ass brary assemb y ( n certa n cases, us ng appropr ate NET code a ­ ttr butes n ts c asses and funct ons) can be oaded nto your database w th a s mp e T-SQL ­statement To see how eas y th s works, open a query w ndow n SSDT or SSMS us ng a ­connect on to the AdventureWorks2012 samp e database In the samp e code fo der, confirm that the fi e ­SQLCLRDemo.dll s ocated n the VS\SQLCLRDemoManua \SQLCLRDemo\b n\Debug ­subfo der If the samp e code parent fo der were C \Demos, you wou d oad the assemb y nto the AdventureWorks2012 database w th the fo ow ng T-SQL command CREATE ASSEMBLY SQLCLRDemo FROM 'C:\Demos\VS\SQLCLRDemoManual\SQLCLRDemo\bin\Debug\SQLCLRDemo.dll'

130  Part I  Core SQL Server Development

There are other syntax opt ons for the CREATE ASSEMBLY command, but for now we’ focus on the preced ng m ted usage Funct ons n an assemb y that res de w th n a c ass and perform oca computat ona tasks and certa n types of data access can be eas y exposed as SQL Server stored procedures, tr ggers, or ­funct ons As w th convent ona T-SQL stored procedures, tr ggers, and funct ons, a t takes s a s mp e T-SQL CREATE PROCEDURE, CREATE TRIGGER, or CREATE FUNCTION statement to make th s happen We’ go through each of these opt ons n th s chapter, but for now et’s create a s mp e CLR stored procedure You can v ew the source code for the SQLCLRDemo assemb y by open ng the so ut on fi e VS\SQLCLRDemoManua \SQLCLRDemoManual.sln n th s chapter’s samp e code fo der W th n the project, the fi e Sprocs.cs conta ns the fo ow ng code using System.Data.SqlClient; using Microsoft.SqlServer.Server; public partial class Sprocs {   public static void spContactsQuick()   {     SqlContext.Pipe.ExecuteAndSend(new SqlCommand("SELECT * FROM Person.Person"));   } }

The spContactsQuick method s des gned to connect to the database n wh ch ts assemb y has been oaded (AdventureWorks2012), perform a SELECT * aga nst the Person.Person tab e, and then use spec a server-s de objects to send the data back to the c ent app cat on To make th s CLR code ava ab e v a SQL Server as a stored procedure, a so ca ed spContactsQuick, you s mp y execute the fo ow ng command from an SSMS or SSDT query w ndow CREATE PROCEDURE spContactsQuick AS EXTERNAL NAME SQLCLRDemo.Sprocs.spContactsQuick

Important  Be sure to enter the Sprocs.spContactsQuick portion of the command verbatim. This phrase is case sensitive. To test the SQL CLR stored procedure, run t from an SSMS or SSDT query w ndow as you wou d any convent ona stored procedure, as shown here EXEC spContactsQuick

Or s mp y spContactsQuick

When the execut on comp etes, you shou d see the contents of the Person.Person tab e n the Resu ts tab of the query w ndow

Chapter 3  Exp or ng SQL CLR   131

As you can see from th s rather tr v a examp e, wr t ng a CLR stored procedure can be very easy and s a ot ke wr t ng c ent-s de or m dd e-t er code that performs data access us ng ADO NET The b ggest d fferences nvo ve the prov s on of a database connect on and the fact that the data must be “p ped” back to the c ent rather than mere y oaded nto a SqlDataReader and returned, man pu ated, or d sp ayed through a user nterface (UI) The presence of the ­SqlContext object a so d fferent ates SQL CLR code from convent ona ADO NET data access code We’ cover the use of the SqlContext object and ts Pipe property n the next sect on The b ts of T-SQL and C# code just shown certa n y don’t te the who e SQL CLR story The use of the ExecuteAndSend method a owed us to sk p over a number of otherw se mportant concepts There are three ways to dep oy assemb es, and you’ve seen on y a s mp fied vers on of one of those ways Secur ty cons derat ons must be taken nto account, and we haven’t even begun to ook at ­tr ggers, funct ons, aggregates, or UDTs So a though the examp e showed how easy SQL CLR ­programm ng can be, we’ now take our t me and show you the nooks and crann es

CLR Stored Procedures and Server-Side Data Access Our prev ous “qu ck and d rty” samp e ooked at CLR stored procedure deve opment, but we need to cover that top c more thorough y now We’ve a ready covered the mechan cs of wr t ng and ­dep oy ng a stored procedure, but et’s back up a b t to try and understand how CLR stored ­procedures work from a conceptua standpo nt SQL CLR stored procedure code runs n an nstance of the NET CLR that s hosted by SQL Server tse f; t s not ca ed as an externa process, as Component Object Mode (COM)–based extended stored procedures (XPs) wou d be Because SQL CLR code runs n the context of the server, t treats objects n the database as nat ve, local objects, more or ess As such, t must treat the c ent that ca s t as remote Th s contextua env ronment s, n effect, the oppos te of that under wh ch c ent and m dd e-t er ADO NET code runs There, commun cat ng w th the database requ res a remote ­connect on (even f the database s phys ca y on the same computer) and the ADO NET code runs oca y The SQL CLR reversa of th s takes a tt e gett ng used to, but once you’ve mastered th nk ng about th ngs th s way, SQL CLR code becomes easy to wr te and understand Meanwh e, as NET has no ntr ns c way of access ng oca objects on the server or transm tt ng data and messages to the c ent, you must use a spec a set of c asses to perform these tasks These c asses are conta ned n the Microsoft.SqlServer.Server namespace

Note  As an aside, it is interesting and important to note that the Microsoft.SqlServer.Server namespace is actually supplied by the System.Data.dll .NET Framework assembly. This means that you don’t need to worry about adding a reference to your project to use this namespace. The namespace’s location within System.Data.dll also further emphasizes the tight integration between .NET and SQL Server.

132  Part I  Core SQL Server Development

If you want, you can th nk of Microsoft.SqlServer.Server as a he per brary for System.Data.SqlClient It supp es the SQL CLR code attr butes we a ready ment oned, a few enumerat ons, an except on c ass, an nterface, and five other c asses SqlContext, SqlPipe, SqlTriggerContext, SqlMetaData, and SqlDataRecord We’ cover SqlMetaData and SqlDataRecord at the end of th s sect on, and we’ cover SqlTriggerContext when we d scuss CLR tr ggers ater n th s chapter We’ cover the SqlContext and SqlPipe objects r ght now At a h gh eve , the SqlContext object, wh ch s static, prov des a hand e to the server-s de context n wh ch your code runs It a so has a channe to the c ent through wh ch you can return data and text ts Pipe property, wh ch n turn prov des access to a proper y opened and n t a zed SqlPipe object A SqlPipe object can send data and messages to the ca ng c ent though severa methods Send, SendResultsStart, SendResultsRow, SendResultsEnd, and ExecuteAndSend In the preced ng code samp e, you used the SqlPipe object’s ExecuteAndSend method to mp c t y open a connect on, ca ExecuteReader on a SqlCommand object that uses that connect on, and transm t the contents of the resu t ng SqlDataReader back to the c ent A though the mp c t work done by ExecuteAndSend m ght have been conven ent for us to get started qu ck y, t’s mportant to avo d such shortcuts n deta ed d scuss ons of SQL CLR programm ng In genera , SQL CLR stored procedure code that quer es tab es n the database must open a ­connect on to that database, use the SqlCommand object’s ExecuteReader method to query the data, and then use one or a comb nat on of the Send methods to send t back The Send methods do not accept DataSet objects; they accept on y SqlDataReader objects, str ngs, and spec a SqlDataRecord objects L st ng 3-1, wh ch shows the mp ementat on of the funct on spContacts from spTest.cs n the SQLCLRDemo samp e project, s a representat ve examp e of how th s s done Listing 3-1  spContacts from spTest.cs.

[SqlProcedure] public static void spContacts() {     SqlConnection conn = new SqlConnection("context connection=true");     SqlCommand cm = new SqlCommand("SELECT * FROM Person.Person", conn);     conn.Open();     SqlDataReader dr = cm.ExecuteReader();     SqlContext.Pipe.Send("Starting data dump");     SqlContext.Pipe.Send(dr);     SqlContext.Pipe.Send("Data dump complete");     dr.Close();     conn.Close(); }



Chapter 3  Exp or ng SQL CLR   133

Note  The implementation of spContacts in the SQLCLRDemoManual project is identical to that shown in Listing 3-1, but is not decorated with the SqlProcedure code attribute. Because the Person.Person table includes xml columns, and because such columns can cause a slowdown in SQL CLR stored procedures, the query in the sample code for ­spContacts uses a column list (which excludes the xml columns) in the SELECT clause rather than the * wildcard. We retained the SELECT * syntax in the printed code for simplicity and terseness. For th s code to work, you need to use both the Microsoft.SqlServer.Server and System.Data.Sql­Client namespaces (and f you ook n the chapter’s samp e project rather than L st ng 3-1, you’ see that we have a ased both of those namespaces w th using statements) Th s s because any ­convent ona ADO NET objects you m ght use, such as SqlConnection, SqlCommand, and SqlDataReader, are supp ed from System. Data.SqlClient, just as they wou d be n a convent ona c ent app cat on or m dd e-t er assemb y And as a ready d scussed, you need the Microsoft.SqlServer.Server namespace n order to use objects such as SqlContext and SqlPipe The stored procedure temp ate n the SSDT SQL Server Database Project temp ate nc udes the using statement for Microsoft.SqlServer.Server and System.Data.SqlClient automat ca y A though server-s de code uses SqlClient objects, t does so n a spec a zed way For examp e, ­ ot ce that the context connection=true connect on str ng passed to the SqlConnection object’s n ­constructor Th s essent a y nstructs ADO NET to open a new connect on to the database n wh ch the CLR assemb y res des Not ce a so the second ca to the SqlContext.Pipe object’s Send method Here, the SqlDataReader parameter over oad of the SqlPipe object’s Send method s used to push the contents of the SqlDataReader back to the c ent You can th nk of th s method as perform ng a while (dr.Read()) oop through the SqlDataReader and return ng the va ues of each co umn for each ­terat on of the oop But nstead of hav ng to do that work yourse f, the Send method does t for you Before and after the SqlDataReader s p ped (returned to the consumer), the String ­parameter over oad of the Send method s used to send status messages to the c ent When th s stored ­procedure s run, the p ped text appears on the Resu ts tab of the query w ndow when you use the Resu ts To Text opt on n SSMS (Resu ts As Text n SSDT) and on the Messages tab when you use the Resu ts To Gr d (Resu ts As Gr d n SSDT) opt on The rest of the st ng conta ns typ ca ADO NET code, a of t us ng objects from the SqlClient ­prov der And that ustrates we the overa theme of SQL CLR programm ng do what you’d norma y do from the c ent or m dd e t er, and use a few spec a he per objects to work w th n the context of SQL Server as you do so

Piping Data with SqlDataRecord and SqlMetaData We ment oned that the SqlPipe object’s Send method can accept an object of type SqlDataRecord, and we ment oned prev ous y that Microsoft.SqlServer.Server prov des th s object as we as an object named SqlMetaData You can use these two objects together n a CLR stored procedure to return a resu t set one row at a t me, nstead of hav ng to supp y the SqlPipe object’s Send method w th a 134  Part I  Core SQL Server Development

SqlDataReader. Th s a ows (but does not requ re) you to nspect the data before send ng t back to the c ent Send ng SqlDataReader objects prevents nspect on of the data w th n the stored ­procedure because SqlDataReader objects are forward-on y resu t set structures Us ng the ExecuteAndSend method and a SqlCommand object has the same m tat on The SqlDataRecord object perm ts NET code to create an nd v dua row to be returned to the ca ng c ent Its constructor accepts an array of SqlMetaData objects, wh ch n turn descr bes the metadata for each co umn n the row L st ng 3-2, wh ch shows the mp ementat on of the spContactCount funct on from spTest.cs n the SQLCLRDemo samp e project, ustrates how to use SqlPipe.Send together w th SqlDataRecord and SqlMetaData objects to return a s ng e-co umn, s ng e-row resu t set from a stored procedure Listing 3-2  spContactCount from spTest.cs.

[SqlProcedure] public static void spContactCount() {   SqlConnection conn = new SqlConnection("context connection=true");   SqlCommand cm = new SqlCommand("SELECT COUNT(*) FROM Person.Person", conn);   SqlDataRecord drc = new SqlDataRecord(new SqlMetaData("ContactCount",  SqlDbType.Int));   conn.Open();   drc.SetInt32(0, (Int32)cm.ExecuteScalar());   SqlContext.Pipe.Send(drc);   conn.Close(); }

The code dec ares var ab e drc as a SqlDataRecord object and passes ts constructor a s ng e S­ qlMetaData object (Pass ng a s ng e object rather than an array s perm ss b e f the ­SqlDataRecord object w have on y a s ng e co umn ) The SqlMetaData object descr bes a co umn named ­ContactCount of type SqlDbType.Int

Note The SqlDbType enumeration is contained within the System.Data.SqlTypes namespace. The SQL Server Stored Procedure template inserts a using statement for this namespace. If you are creating SQL CLR code without using this template, you will need to add the using statement yourself. The rest of the code s rather stra ghtforward F rst, a context connect on and command are opened and a SELECT COUNT(*) query s performed aga nst the Person.Person tab e Because the query returns a s ng e sca ar va ue, t s run us ng the SqlCommand object’s ExecuteScalar method Next, the va ue returned by ExecuteScalar s cast nto a NET Int32 and that va ue s oaded nto ­co umn 0 (the on y returned co umn) of the SqlDataRecord object us ng ts SetInt32 method The ­SqlDataRecord s then p ped back to the c ent us ng the SqlPipe object’s Send method

Chapter 3  Exp or ng SQL CLR   135

Note  If you wanted to send back multiple SqlDataRecord objects, you would send the first object using the SqlContext object’s SendResultsStart method and then send all s­ ubsequent SqlDataRecord objects using the SendResultsRow method. You would call the SendResultsEnd method after all SqlDataRecord objects had been sent. Once the stored procedure has been dep oyed (the techn ques for wh ch we w d scuss short y), you can execute t just as you wou d any other stored procedure A though the resu t s a s ng e va ue, t s presented as a co umn and the co umn name ContactCount s shown on the Resu ts tab of the query w ndow Keep n m nd that th s COUNT(*) query resu t cou d have been returned w thout us ng the SqlMetaData and SqlDataRecord objects; the samp e s prov ded to demonstrate the use of these objects as an a ternat ve to p p ng SqlDataReader objects and text to the c ent

CLR Stored Procedure Usage Guidelines It’s mportant to understand how to perform data access and retr eva n CLR stored ­procedures As a NET deve oper, you a ready know how to perform computat ona tasks w th n your code, so our samp es ustrate server-s de data access more than anyth ng e se As proofof-concept code, these samp es are comp ete y adequate Meanwh e, you shou d avo d wr t ng CLR stored procedures that mere y perform s mp e “CRUD” (Create, Read, Update and De ete) operat ons Such tasks are better eft to ­convent ona T-SQL stored procedures, wh ch typ ca y perform these operat ons more effic ent y than ADO NET can CLR stored procedures work we when you need to perform computat ons on (or ­us ng) your data, and you requ re the express veness of a NET anguage or the r ch ­funct ona ty prov ded by the NET base c ass brar es to do so (where such express veness and base c ass brary support are m ss ng from T-SQL) For examp e, mp ement ng a “fuzzy search” us ng bus ness og c embedded n NET ­ ssemb es to determ ne wh ch data has an affin ty to other data s a good use of SQL CLR a stored procedures Regu ar-express on-based data va dat on n an update or nsert stored procedure s another good app cat on of SQL CLR ntegrat on As a genera ru e, stra ght data access shou d be eft to T-SQL “H gher-va ued” computat ons are good cand dates for SQL CLR ntegrat on We’ rev s t the SQL CLR usage quest on at var ous po nts n th s chapter

Deployment Before you can test your SQL CLR code, you must dep oy the assemb y conta n ng t and reg ster the nd v dua funct ons that you want recogn zed as stored procedures A number of dep oyment ­methods are at your d sposa ; we w pause to cover them n th s sect on, before d scuss ng test ng of your stored procedures and the other four bas c SQL CLR ent t es 136  Part I  Core SQL Server Development

Getting Ready If you’re us ng a SQL Server Database Project, you’ dep oy your assemb y us ng SSDT’s Pub sh ­funct on But before do ng so, you must be ab e to bu d the project successfu y A bu d w not on y comp e the SQL CLR assemb y tse f, but w a so check the code for var ous errors, nc ud ng ­unreso ved references to database objects Go ahead and bu d the project now by se ect ng the SQLCLRDemo project node n the So ut on Exp orer, and then se ect ng Bu d Bu d SQLCLRDemo from the ma n menu Equ va ent UI se ect ons (after se ect ng the project node) are the Bu d opt on from the project node’s shortcut menu or the Sh ft+F6 keyboard shortcut If you’re us ng the SQLCLRDemo samp e code demo project, after your attempt to bu d the p ­ roject you shou d see a message that the bu d fa ed, w th the fo ow ng error d sp ayed n the Error L st w ndow SQL71501: Trigger: [Person].[trgUpdatePerson] has an unresolved reference to object [Person]. [Person].

If, n the Error L st w ndow, you doub e-c ck the error, a code ed tor w ndow w open w th a T-SQL dep oyment scr pt generated by SSDT In t s a CREATE TRIGGER command for the trgUpdatePerson tr gger and a red squ gg y under the name of the tab e to wh ch the tr gger s be ng app ed, Person. Person The T-SQL s actua y va d, so you may wonder why SSDT m ght flag t w th an error SQL Server Database projects are meant to m rror fu y the databases to wh ch the r contents w be dep oyed A though you can create projects that conta n just a subset of a database—for examp e, the assets for a SQL CLR assemb y—SSDT w b ock you as soon as any T-SQL n the project references an object n the database that s not a so n your project In the case of your project, the dep oyment scr pt references the tab e Person.Person (because the SqlTrigger attr bute app ed to the trgUpdatePerson CLR tr gger references t) and yet there s no T-SQL scr pt n your project that defines that tab e Therefore, by SSDT’s cr ter a, the T-SQL s mproper You cou d so ve th s by add ng a scr pt for the Person.Person tab e and then add ng further scr pts, n a p ecemea fash on, for any objects upon wh ch the tab e s dependent Wh e do ng so wou d ­a ow the project to bu d, t wou d st be a work-around, and thus someth ng you ought to avo d The rea so ut on s to mport the AdventureWorks2012 database nto the project; that way your ­project w reflect a fu defin t on of the database “Import ng” the database rea y trans ates nto hav ng SSDT d scover the database’s structure and create correspond ng scr pts for each object Let’s do the mport now As a preparatory step, c ose V sua Stud o, open W ndows Exp orer and nav gate to the source code d rectory conta n ng the project fi e (not the so ut on fi e) and the ­project’s contents (e g C:\Demos\VS\SQLCLRDemo\SQLCLRDemo) Look for a fi e ca ed SQLCLRDemo.dbmdl and de ete t, then re-open the so ut on n V sua Stud o Th s preparatory step s necessary to enab e the Import Database funct ona ty that we need to use; f the step s not ­performed after a fa ed bu d, the Import Database opt on w be grayed-out Let’s now perform the mport operat on Once aga n, se ect the SQLCLRDemo project node n the So ut on Exp orer, then r ght-c ck that node and se ect Import Database from the shortcut menu (you can a so se ect Project Import Database from the ma n menu) Th s se ect on w br ng up the Import Database d a og box, wh ch s shown n F gure 3-4

Chapter 3  Exp or ng SQL CLR   137

Figure 3-4 The mport Database d a og box.

Se ect the AdventureWorks2012 database from the Source Database Connect on combo box, or c ck the New Connect on button, define a connect on to the AdventureWorks2012 database on ­localhost, and c ck OK Now c ck Start The Summary screen of the mport database d a og box w appear and the mport process w proceed When the F n sh button becomes enab ed, c ck t to ­fin sh the mport process Now bu d the project once more Th s t me the bu d shou d succeed, n wh ch case you are ready to dep oy your assemb y

Deploying Your Assembly Hav ng comp eted a successfu bu d, you’re now ready to pub sh the database, and dep oy the ­assemb y n the process Se ect Bu d Pub sh SQLCLRDemo from the ma n menu now (or r ght-c ck the SQLCLRDemo project node n the So ut on Exp orer and se ect Pub sh from the shortcut menu); th s w br ng up the Pub sh Database d a og box, shown n F gure 3-5

Figure 3-5 The Pub sh Database d a og box.

138  Part I  Core SQL Server Development

C ck the Ed t button next to the Target Database Connect on read-on y text box to br ng up the standard Connect on Propert es d a og box Configure the connect on to po nt to your target server (e g localhost), to use W ndows Authent cat on and to po nt to the AdventureWorks2012 database (make sure not to accept the defau t database name of SQLCLRDemo), then c ck OK Back n the ­Pub sh Database d a og box, c ck the Pub sh button to dep oy your CLR assemb y to the database

Note  Before clicking Publish, you can optionally click the Save Profile As button to save this publish profile. When you next publish the database, you can use the Load Profile button to retrieve the profile and avoid configuring the target database connection manually once again. For dep oy ng the C ass L brary project vers on, assum ng C \Demos s the samp e code parent d rectory, you can execute the fo ow ng T-SQL statement from w th n a query w ndow CREATE ASSEMBLY SQLCLRDemo AUTHORIZATION dbo FROM 'C:\Demos\VS\SQLCLRDemoManual\SQLCLRDemo\bin\Debug\SQLCLRDemo.dll' WITH PERMISSION_SET = SAFE GO

The AUTHORIZATION c ause a ows you to spec fy a name or ro e to wh ch ownersh p of the ­assemb y s ass gned The defau t author zat on s that of the current user, and because you are most ke y ogged n as dbo for AdventureWorks2012, n th s case the c ause s unnecessary (wh ch s why we om tted t from our prev ous examp e) The mean ng and effect of the WITH PERMISSION SET c ause s d scussed n the “Secur ty” ­sect on near the end of th s chapter For now, just note that th s c ause a ows you to spec fy the secur ty ­perm ss ons w th wh ch your assemb y runs As w th the AUTHORIZATION c ause, n th s case the WITH ­PERMISSION SET c ause s techn ca y unnecessary because SAFE s the defau t PERMISSION SET va ue used when a CREATE ASSEMBLY command s executed Regard ess, t’s a good pract ce to nc ude both c auses If your assemb y has dependenc es on other assemb es, SQL Server ooks to see whether those assemb es have a ready been oaded nto the database and, f so, confirms that the r ownersh p s the same as that of the spec fied assemb y If the dependent assemb es have not yet been oaded nto the database, SQL Server ooks for them n the same fo der as the spec fied assemb y If t finds a ­dependent assemb es n that ocat on, t oads them and ass gns them the same ownersh p as the pr mary assemb y If t does not find the dependent assemb es n that fo der or n the g oba assemb y cache (GAC), the CREATE ASSEMBLY command w fa You can supp y a str ng express on nstead of a tera n the FROM c ause, a ow ng for some ­nterest ng data-dr ven poss b t es For examp e, you cou d fetch an assemb y path reference from a tab e n your database It s a so poss b e to supp y an assemb y d rect y n ne by prov d ng a ­b nary stream n the FROM c ause nstead of a fi e spec ficat on You do th s by spec fy ng a ­varbinary tera va ue or express on (or a comma-de m ted st of varbinary va ues or express ons, when

Chapter 3  Exp or ng SQL CLR   139

­ ependent assemb es must be spec fied) that conta ns the actua b nary content of your assemb y d (or ­assemb es) Th s a ows the creat on of a database, nc ud ng any CLR assemb es t conta ns, to be comp ete y scr pted, w thout requ r ng d str but on of actua assemb y fi es The b nary stream can be embedded n the scr pt tse f or, us ng an express on, t cou d be fetched from a tab e n a database In add t on to us ng V sua Stud o dep oyment and the T-SQL CREATE ASSEMBLY statement, you can up oad the assemb y nto your database nteract ve y from SSMS S mp y r ght-c ck the ­servername/Databases/AdventureWorks2012/Programmability/Assemblies node n the Object ­Exp orer (where servername s the name of your server), and then choose New Assemb y from the shortcut menu The New Assemb y d a og box, shown n F gure 3-6, appears

Figure 3-6 The SSMS New Assemb y d a og box.

Type the assemb y path and fi e name n the Path To Assemb y text box, or use the Browse button to spec fy t nteract ve y You can spec fy AUTHORIZATION and WITH PERMISSION SET deta s n the Assemb y Owner text box (us ng the e ps s button, f necessary) and the Perm ss on Set combo box, respect ve y Regard ess of the dep oyment method you use, once your assemb y has been added to your ­ atabase, t becomes an ntegra part of that database and ts under y ng mdf fi e Th s means that f d your database s backed up and restored, or dep oyed, any assemb es w th n t move a ong w th the data tse f and need not be added manua y as a subsequent step

140  Part I  Core SQL Server Development

Deploying Your Stored Procedures In the SQL Server Database Project vers on of the samp e code, dep oyment of a the stored ­procedures s hand ed by V sua Stud o when the assemb y tse f s dep oyed Th s s due to the ­app cat on of the SqlProcedure attr bute to the methods n the StoredProcedures c ass (found n the spTest.cs fi e) The SqlProcedure attr bute accepts an opt ona Name parameter, the va ue of wh ch s the actua ca ab e stored procedure name If you do not supp y a va ue for the Name parameter, the name of the NET method s used as the stored procedure name The SqlProcedure attr bute s used on y by V sua Stud o when auto-dep oy ng SQL CLR ­assemb es Therefore, t does not appear n the source code n the C ass L brary project Dep oy ng the stored procedures from that vers on of the source code requ res ssu ng a CREATE PROCEDURE T-SQL ­command us ng the EXTERNAL NAME c ause to spec fy the assemb y, the fu y qua fied c ass name spec fier, and the method name For examp e, to oad the C ass L brary vers on of spContacts, you wou d ssue the fo ow ng command CREATE PROCEDURE spContacts AS EXTERNAL NAME SQLCLRDemo.StoredProcedures.spContacts

The preced ng command spec fies that the spContacts method, found n the c ass named StoredProcedures, n the oaded assemb y w th T-SQL name SQLCLRDemo, shou d be reg stered as a CLR stored procedure ca ab e under the name spContacts

Note  All necessary CREATE PROCEDURE commands for the Class Library project version of the sample code are contained in the CreateObjects.sql script in the SSMS project supplied with the chapter’s sample code on the book’s companion website. You will need to run that script in order to utilize the various SQL CLR entities implemented in the Class Library ­project. Before running the script, you will need to edit the CREATE ASSEMBLY command at the top so that the assembly’s file specification points to the location on your system where the assembly has been installed. By default, the script looks for the assembly in the C:\Demos\VS\SQLCLRDemoManual\SQLCLRDemo\bin\Debug folder on your PC. Note that f the CLR stored procedure had been wr tten n M crosoft V sua Bas c NET rather than C#, the c ass name spec fier wou d change to SQLCLRDemo.StoredProcedures Th s wou d necess tate a change to the dep oyment T-SQL code as fo ows CREATE PROCEDURE spContacts AS EXTERNAL NAME SQLCLRDemo.[SQLCLRDemo.StoredProcedures].spContacts

In V sua Bas c projects, the defau t namespace for a project tse f defau ts to the project name, as does the assemb y name The c ass w th n the project must be referenced us ng the defau t namespace as a prefix Because the c ass spec fier s a mu t part dot-separated name, t must be ­enc osed w th n square brackets so that SQL Server can dent fy t as a s ng e nd v s b e name Because C# projects hand e the defau t namespace sett ng a tt e d fferent y, the namespace prefix s not used n the c ass spec fier for C# assemb es

Chapter 3  Exp or ng SQL CLR   141

One ast po nt before we d scuss how to test your now-dep oyed SQL CLR stored procedures It s mportant to rea ze that the c ass spec fier and method name n the EXTERNAL NAME c ause are case sensitive, and that th s s true even for assemblies developed in Visual Basic .NET. A though th s po nt may seem perp ex ng at first, t does make sense SQL Server searches for your methods w th n your assemb es, not w th n your source code In other words, t’s ook ng w th n M crosoft Intermed ate Language (MSIL) code, not V sua Bas c NET or C# source code Because MSIL s case sens t ve ( t has to be to support case-sens t ve anguages ke C#), SQL Server must be as we as t searches w th n an assemb y for a spec fic c ass and method The fact that SQL Server s not case sens t ve by defau t (even though t once was) and that V sua Bas c NET s not a case-sens t ve anguage s of no mportance! If you attempt to reg ster a method and you rece ve an error that t cannot be found w th n the assemb y, doub e-check that the case ­usage n your command matches that of your source code

Testing Your Stored Procedures The TestStoredProcs.sql scr pt fi e n the SSMS project supp ed w th th s chapter’s samp e code w run both CLR stored procedures (spContactCount and spContacts) Open the fi e n SSMS, and then c ck the Execute button on the SQL Ed tor too bar, choose Execute from the Query menu, press F5, or press Ctrl-E (You can a so r ght-c ck anywhere ns de the query w ndow and se ect Execute from the shortcut menu ) When the scr pt runs, you shou d see the s ng e-va ued resu t of the spContactCount stored ­ rocedure appear first, as shown n F gure 3-7 Not ce that the co umn name ContactCount ­appears p on the Resu ts tab and reca that th s s a d rect resu t of your us ng the SqlMetaData object n the CLR code Be ow the spContactCount resu t, you w see the resu ts from the spContacts stored ­procedure come n Because the Person.Person tab e has a most 20,000 rows, on certa n mach nes, these resu ts m ght take some t me to retr eve

Figure 3-7  TestStoredProcs.sql scr pt code and resu ts.

142  Part I  Core SQL Server Development

Even wh e the resu ts are com ng n, the “Start ng Data Dump” status message shou d be v s b e on the Messages tab (or on the Resu ts tab f you’re us ng the Resu ts To Text opt on n SSMS) After a the rows have been fetched, you shou d see the “Data Dump Comp ete” message appear as we We have yet to cover CLR funct ons, tr ggers, aggregates, and UDTs, but you have a ready earned most of the sk s you need to deve op SQL CLR code You have earned how to create a V sua Stud o SQL Server Database Project and use SSDT’s CLR assemb y auto-dep oyment features You have a so earned how to deve op SQL CLR code n standard C ass L brary projects and how to use T-SQL ­commands to dep oy the code for you You’ve earned about the subt e d fferences between ­dep oy ng C#- and V sua Bas c NET-based SQL CLR assemb es, and we’ve covered the case-sens t ve requ rements of T-SQL–based dep oyment W th a th s under your be t, we can cover the rema n ng four bas c SQL CLR ent t es re at ve y qu ck y

CLR Functions Let’s take everyth ng we’ve d scussed about SQL CLR stored procedures and dep oyment and app y t to SQL CLR funct ons As any programmer knows, a function s a procedure that returns a va ue (or an object) Ma nstream NET funct ons typ ca y return NET types SQL CLR funct ons, on the other hand, must return a SqlType So to start w th, you need to make sure that your c asses that mp ement SQL CLR funct ons mport the System.Data.SqlTypes namespace w th a using or Imports statement The SQL Server Database Project C# temp ate for user-defined funct ons (UDFs) conta ns the ­appropr ate using statement by defau t; you w need to add the using statement manua y to standard C ass L brary code Once the namespace s mported, you can wr te the funct ons themse ves In V sua Stud o SQL Server Database Projects, they shou d be decorated w th the SqlFunction attr bute; th s attr bute accepts an ­opt ona name parameter that works dent ca y to ts SqlProcedure counterpart In our samp e code, a va ue s not supp ed for th s parameter SqlFunction s used by V sua Stud o SQL Server Database ­Projects for dep oyment of your SQL CLR funct ons For sca ar-va ued funct ons n C ass L brary projects, the SqlFunction attr bute s opt ona , so t appears n the C ass L brary samp e code on y for the tab e-va ued funct on (TVF) descr bed ater n th s sect on L st ng 3-3, wh ch shows the code for funct on fnHelloWorld from fnTest.cs n the SQLCLRDemo samp e code project, mp ements a s mp e “He o Wor d” funct on that returns a va ue of type ­SqlString Listing 3-3  fnHelloWorld from fnTest.cs.

[SqlFunction] public static SqlString fnHelloWorld() {     return new SqlString("Hello World"); }



Chapter 3  Exp or ng SQL CLR   143

Not ce that SqlType objects (such as SqlString) requ re exp c t nstant at on and constructor va ue pass ng; you cannot s mp y dec are and ass gn va ues to them The code n L st ng 3-3 nstant ates a SqlString object n ne w th n the return statement to avo d var ab e dec arat on A funct on that accepts no parameters and returns a hard-coded va ue s of tt e pract ca use ­ yp ca y, funct ons are passed va ues and perform ca cu at ons w th those va ues, and they are often T used from w th n T-SQL statements, n effect as extens ons to the funct ons bu t nto the T-SQL anguage tse f L st ng 3-4, wh ch shows the code for funct on fnToCelsius n fnTest.cs n the SQLCLRDemo samp e project, mp ements a Fahrenhe t-to-Ce s us convers on funct on Listing 3-4  fnToCelsius from fnTest.cs.

[SqlFunction] public static SqlDecimal fnToCelsius(SqlInt16 fahrenheit) {     return new SqlDecimal((((Int16)fahrenheit) - 32) / 1.8); }

The funct on accepts a Fahrenhe t temperature (as a SqlInt16), converts t to Ce s us, and returns t (as a SqlDecimal) Not ce that the code casts the nput parameter from a SqlInt16 to a NET Int16, app es a Fahrenhe t-to-Ce s us convers on formu a, and then passes the resu t to the constructor of a new SqlDecimal object Dep oyment of these funct ons s automat c n the SSDT SQL Server Database Project vers on of our samp e code For the C ass L brary vers on, use the T-SQL CREATE FUNCTION statement n a ­s m ar fash on to how the CREATE PROCEDURE statement was used n the prev ous sect on, but nc ude a data type spec ficat on for the return va ue For examp e, to dep oy the fnHelloWorld funct on, you wou d use th s statement CREATE FUNCTION fnHelloWorld() RETURNS nvarchar(4000) WITH EXECUTE AS CALLER AS EXTERNAL NAME SQLCLRDemo.UserDefinedFunctions.fnHelloWorld

Not ce the use of data type nvarchar(4000) to correspond w th the SqlString type used n the funct on’s mp ementat on The WITH EXECUTE AS CALLER c ause spec fies that the SQL CLR funct on shou d execute under the ca er’s dent ty

Tip  You can enter the CREATE FUNCTION command yourself, but note that all such ­necessary commands for the sample code SQL CLR functions are available in the CreateObjects.sql script file in the SSMS project supplied with this chapter’s sample code. You can test these funct ons n SSDT or SSMS Use the fo ow ng query to test the two funct ons (You can a so run the TestScalarFunctions.sql scr pt fi e n the SSMS samp e project ) SELECT    dbo.fnHelloWorld() AS HelloWorld,    dbo.fnToCelsius(212) AS CelsiusTemp

144  Part I  Core SQL Server Development

T-SQL funct ons can return resu t sets as we as sca ar va ues Funct ons wh ch return resu t sets are ca ed table-valued functions (TVFs) Wr t ng SQL CLR TVFs s poss b e, a though you do so ­d fferent y than you wou d SQL CLR sca ar-va ued funct ons or SQL CLR stored procedures SQL CLR TVFs must return a type that mp ements the NET nterface IEnumerable, and they must a so prov de a “fi row” method that converts an e ement of that type to a co ect on of sca ar va ues that compr se the ­correspond ng tab e row L st ng 3-5, wh ch shows the code for funct ons fnPortfolioTable and FillTickerRow n fnTest.cs n the SQLCLRDemo samp e project, mp ements a TVF named fnPortfolioTable Listing 3-5  fnPortfolioTable and FillTickerRow from fnTest.cs.

[SqlFunction(      FillRowMethodName="FillTickerRow",      TableDefinition="TickerSymbol nvarchar(5), Value decimal")] public static System.Collections.IEnumerable fnPortfolioTable(SqlString  tickersPacked) { string[] tickerSymbols; object[] rowArray = new object[2]; object[] compoundArray = new object[3]; char[] parms = new char[1]; parms[0] = ';'; tickerSymbols = tickersPacked.Value.Split(parms); rowArray[0] = tickerSymbols[0]; rowArray[1] = 1; compoundArray[0] = rowArray; rowArray = new object[2]; rowArray[0] = tickerSymbols[1]; rowArray[1] = 2; compoundArray[1] = rowArray; rowArray = new object[2]; rowArray[0] = tickerSymbols[2]; rowArray[1] = 3; compoundArray[2] = rowArray; return compoundArray; } public static void FillTickerRow(object row, ref SqlString tickerSymbol, ref  SqlDecimal value) { object[] rowArray = (object[])row; tickerSymbol = new SqlString((string)rowArray[0]); value = new SqlDecimal(decimal.Parse(rowArray[1].ToString())); }



Chapter 3  Exp or ng SQL CLR   145

Rather than mp ement ng ts own IEnumerable-compat b e type, fnPortfolioTable uses an array Th s s perfect y ega because arrays n NET mp ement IEnumerable The fnPortfolioTable ­funct on accepts a sem co on-de m ted st of stock t cker symbo s and returns a tab e w th each t cker ­symbo appear ng n a separate row as co umn TickerSymbol and a va ue for the t cker as co umn Value The structure of the returned tab e s dec ared n the TableDefinition parameter of the SqlFunction ­attr bute n SQL Server projects and n the CREATE FUNCTION T-SQL command for C ass L brary ­projects The ass gned va ues are hard-coded, and on y three rows are returned, regard ess of how many t cker symbo s are passed n As w th our other samp es, th s one s more usefu as a teach ng too than as a pract ca app cat on of TVFs Arrays are the name of the game here F rst, the Split method s used to crack the de m ted t cker st nto an array of s ng e t cker str ngs Then the TVF structures the data so that each e ement n the return va ue array (compoundArray) s tse f a two-e ement array stor ng a s ng e t cker symbo and ts va ue The funct on code needs on y to return compoundArray Next, the FillTickerRow funct on (named n the FillRowMethodName parameter of the SqlFunction attr bute) takes each ­t wo-e ement array (passed n as the first parameter) and converts ts members to nd v dua sca ars These ­sca ars are then returned v a ref parameters that start n the second parameter pos t on; the parameter names are not s gn ficant, but the r pos t ons must correspond to those of the co umns n the ­TableDefinition argument of the SqlFunction attr bute Because the FillRowMethodName parameter of the SqlFunction attr bute s requ red by SQL Server, the C ass L brary vers on of the fnPortfolioTable funct on s decorated w th that attr bute, supp y ng a va ue for that one parameter In the SQL Server Database Project vers on, a va ue s a so supp ed for the TableDefinition parameter to enab e auto-dep oyment of the TVF As w th the other funct ons, dep oyment of th s funct on s performed by SSDT n the SQL Server Database Project samp e code For the C ass L brary vers on, you can dep oy the funct on us ng the fo ow ng T-SQL command (a so conta ned n the CreateObjects.sql scr pt fi e) CREATE FUNCTION fnPortfolioTable(@TickersPacked nvarchar(4000)) RETURNS table (  TickerSymbol nvarchar(5),  Value decimal ) WITH EXECUTE AS CALLER AS EXTERNAL NAME SQLCLRDemo.UserDefinedFunctions.fnPortfolioTable

As w th fnHelloWorld, the SqlString data type s mapped to an nvarchar(4000), th s t me for one of the nput parameters Because fnPortfolioTable s a TVF, ts return type s dec ared as table, w th n ne spec ficat ons for the tab e’s defin t on Use the fo ow ng query n SSDT or SSMS to test the TVF (or run the TestTableValuedFunction.sql scr pt fi e n the SSMS samp e project) SELECT * FROM fnPortfolioTable('IBM;MSFT;ORCL') ORDER BY TickerSymbol

146  Part I  Core SQL Server Development

The fo ow ng data shou d be returned TickerSymbol Value ------------  -----IBM 1 MSFT 2 ORCL 3

Note that as of SQL Server 2008, t s poss b e to prov de a “h nt” to the database that your TVF returns data n a part cu ar order Th s can opt m ze quer es or creat on of ndexes ordered on the same express on To take advantage of th s, you must first ensure that og c n your CLR TVF code returns data n a spec fic order and, second, ensure that you spec fy that order n an ORDER c ause n the CREATE FUNCTION T-SQL command For examp e, mag ne that the SQL CLR TVF code ordered ts resu ts by the TickerSymbol co umn In that case, you cou d mod fy the T-SQL code that creates the funct on as fo ows CREATE FUNCTION fnPortfolioTable(@TickersPacked nvarchar(4000)) RETURNS table ( TickerSymbol nvarchar(5), Value decimal ) WITH EXECUTE AS CALLER ORDER (TickerSymbol) AS EXTERNAL NAME SQLCLRDemo.UserDefinedFunctions.fnPortfolioTable

A though th s code does not actually return data n TickerSymbol order, you can st use the preced ng T-SQL command for the C ass L brary vers on of the TVF If you re-executed the query n the TestTableValuedFunction.sql scr pt fi e, everyth ng wou d work correct y, because the nput data s co nc denta y supp ed n TickerSymbol order (‘IBM;MSFT;ORCL’) If, however, you mod fied the nput str ng to, for examp e, ‘IBM;ORCL;MSFT’, you wou d rece ve the fo ow ng error The order of the data in the stream does not conform to the ORDER hint specified for the CLR TVF 'fnPortfolioTable'. The order of the data must match the order specified in the ORDER hint for a CLR TVF. Update the ORDER hint to reflect the order in which the input data is ordered, or update the CLR TVF to match the order specified by the ORDER hint.

We po nt out th s error text because t s rea y the on y outward proof (other than ana ys s of ­query execut on p ans) that th s new ordered SQL CLR TVF feature s n fact supported, and we a ­ ssume that you wou d prefer not to take th s nformat on mere y on fa th P ease note that th s feature can on y be taken advantage of, pract ca y speak ng, when reg ster ng a SQL CLR TVF us ng one’s own T-SQL code That s because there s no parameter n the SqlFunction attr bute for spec fy ng an ORDER c ause, so SSDT’s auto-dep oyment cannot em t that c ause Th s means that you must use the ORDER c ause on y w th a C ass L brary project or e se om t the SqlFunction attr bute from your SQL CLR TVF code n a SQL Server Database Project and reg ster the TVF “manua y,” e ther from an ­externa scr pt or by add ng a scr pt to your SQL Server Database Project that performs the ­dep oyment



Chapter 3  Exp or ng SQL CLR   147

CLR Triggers T-SQL tr ggers are rea y just stored procedures that are ca ed by SQL Server at spec fic t mes and that can query va ues n the DELETED and INSERTED pseudo-tab es (wh ch expose “before and after” snapshots of data changed by the statement that fired the tr gger) SQL CLR tr ggers are s m ar to SQL CLR stored procedures, and they can be created for a data man pu at on anguage (DML) ­act ons that mod fy data (that s, updates, nserts, and de etes) SQL Server 2005 ntroduced the concept of data defin t on anguage (DDL) tr ggers, wh ch can ntercept and hand e act ons such as CREATE TABLE and ALTER PROCEDURE L ke DML tr ggers, DDL tr ggers can be mp emented n T-SQL or SQL CLR code We w cover both SQL CLR DML and DDL tr ggers n th s sect on SQL CLR DML tr ggers, ke the r T-SQL counterparts, have access to the DELETED and INSERTED pseudo-tab es and must be dec ared as hand ng one or more spec fic events for a spec fic tab e or, under certa n c rcumstances, a spec fic v ew A so, they can make use of the SqlTriggerContext object (through the SqlContext object’s TriggerContext property) to determ ne wh ch part cu ar event ­(update, nsert, or de ete) caused them to fire and wh ch co umns were updated Once you understand these concepts, wr t ng SQL CLR DML tr ggers s rea y qu te s mp e ­ L st ng 3-6, wh ch shows the code for the trgUpdatePerson funct on from trgTest.cs n the ­SQLCLRDemo samp e project, shows the SQL CLR code for the trgUpdatePerson DML tr gger, wh ch s des gned to funct on as a FOR UPDATE tr gger on the Person.Person tab e n the AdventureWorks2012 database Listing 3-6  trgUpdatePerson from trgTest.cs.

[SqlTrigger(Target="Person.Person", Event="for UPDATE")] public static void trgUpdatePerson() {     SqlTriggerContext context = SqlContext.TriggerContext;     string oldName = string.Empty;     string newName = string.Empty;     SqlConnection conn = new SqlConnection("context connection=true");     SqlCommand cmOld = new SqlCommand( "SELECT FirstName FROM DELETED", conn);     SqlCommand cmNew = new SqlCommand( "SELECT FirstName FROM INSERTED", conn);     conn.Open();     SqlDataReader drOld = cmOld.ExecuteReader();     if (drOld.Read())     {         oldName = (string)drOld[0];     }     drOld.Close();     SqlDataReader drNew = cmNew.ExecuteReader();     if (drNew.Read())     {

148  Part I  Core SQL Server Development

        newName = (string)drNew[0];     }     drNew.Close();     conn.Close();     SqlContext.Pipe.Send("Old Value of FirstName:" + oldName);     SqlContext.Pipe.Send("New Value of FirstName:" + newName);     for (int i = 0; i  ‘ eve -separat on str ng In th s manner, the code obta ns the parent, grandparent, great-grandparent, and so on, a the way up to the root When the oop term nates, @DisplayPath s returned to the ca er Here s the output from the fnGetFullDisplayPath funct on when run aga nst the emp oyee tab e SELECT NodeId, NodeId.ToString() AS NodeIdPath, dbo.fnGetFullDisplayPath(NodeId) AS NodeIdDisplayPath, EmployeeName FROM Employee ORDER BY NodeLevel, NodeId GO NodeId -----0x 0x58 0x68 0x78 0x5AC0 0x5B40 0x6AC0 0x7AC0 0x5AD6 0x5ADA

NodeIdPath ---------/ /1/ /2/ /3/ /1/1/ /1/2/ /2/1/ /3/1/ /1/1/1/ /1/1/2/

NodeIdDisplayPath ----------------------------Dave Dave > Amy Dave > John Dave > Jill Dave > Amy > Cheryl Dave > Amy > Wanda Dave > John > Mary Dave > Jill > Kevin Dave > Amy > Cheryl > Richard Dave > Amy > Cheryl > Jeff

EmployeeName -----------Dave Amy John Jill Cheryl Wanda Mary Kevin Richard Jeff

(10 row(s) affected)

The NodeIdDisplayPath n th s resu t set shows the emp oyee names a ong the h erarch ca paths n each row Popu at ng a h erarch ca tab e does requ re some effort up front, as you’ve seen However, once your data s n p ace, you can everage the power of the hierarchyid data type by nvok ng ts methods to eas y query and reorgan ze the tree structure We’re a most ready to d ve nto that, but first et’s take a moment to ta k about the ndex ng opt ons that you need to know about

Hierarchical Table Indexing Strategies You can create a depth-first ndex or a breadth-first ndex (or both) on your h erarch ca tab es The two types d ffer n how SQL Server phys ca y stores node references n the ndex Defin ng ­depth-first and breadth-first ndexes can have a s gn ficant mpact on performance for access ng data n ­h erarch ca tab es

Chapter 7  H erarch ca Data and the Re at ona Database   313

As the r names mp y, depth-first ndex ng stores parent and ch d node references near each other, whereas breadth-first ndex ng stores references for nodes at the same h erarch ca eve near each other Therefore, you w choose the appropr ate type of ndex based on an understand ng of how your h erarch ca data s shaped n the tab e, how t w grow, and how t w be typ ca y quer ed by ­c ent app cat ons You can create both types of ndexes as we , for effic ent access both vert ca y and hor zonta y across the tree

Note  Regardless of what indexing strategy you employ, comparison operations are a ­ lways performed in depth-first order. Thus, A < B means that A always comes before B in a ­depth-first traversal of the tree (or A is to the left of B if they are both at the same level), even if breadth-first indexing is used.

Depth-First Indexing Defin ng e ther a pr mary key ndex or a un que ndex on a hierarchyid co umn resu ts n a d ­ epth-first ndex Because the NodeId co umn (of type hierarchyid) s des gnated as the pr mary key n the ­emp oyee tab e, you have a ready created a depth-first ndex W th depth-first ndex ng (dep cted n F gure 7-3), SQL Server tr es to phys ca y store references to nodes n a subtree as near to each other as poss b e By “near each other,” we mean that SQL Server records them on d sk n the same page, f poss b e—or n the same extent, f not, and so on— n order to max m ze query performance Th s strategy y e ds h gh query performance f your h erarchy runs very many eve s deep Creat ng a depth-first ndex for such a h erarchy w resu t n very fast vert ca search ng on the tree (that s, query ng ancestors and descendants up and down a potent a y ong cha n)

Figure 7-3  Phys ca row storage when opt m zed for depth first search ng.

Breadth-First Indexing W th breadth-first ndex ng, ustrated n F gure 7-4, reference to nodes at the same eve of the h erarchy are phys ca y stored as near to each other as poss b e Th s type of ndex y e ds h gh query performance for trees that grow very broad If there are many ch dren beneath the parents n your h erarchy, you w want to create a breadth-first ndex to enab e fast hor zonta search ng across a potent a y arge number of nodes at the same eve 314  Part II  Going Beyond Relational

Figure 7-4  Phys ca row storage when opt m zed for breadth first search ng.

To define a breadth-first ndex, you create a compos te ndex on your h erarch ca tab e that nc udes two co umns the nteger co umn that ho ds the level of the node w th n the h erarchy (such as the NodeLevel co umn defined n the emp oyee tab e, based on the GetLevel method aga nst the NodeId co umn) and the hierarchyid co umn tse f (NodeId) So to create a breadth-first ndex, run the fo ow ng code CREATE UNIQUE INDEX IX_EmployeeBreadth ON Employee(NodeLevel, NodeId)

As we ment oned a ready, one tab e can have both depth-first and breadth-first ndexes by c­ reat ng one pr mary key or un que ndex on the hierarchyid co umn and another compos te ­ndex on the node- eve co umn and the hierarchyid Th s w carry s ght y more overhead for data ­man pu at on anguage (DML) act ons, because updates w need to be performed n both ndexes; however, query performance w be very fast for both hor zonta and vert ca search ng across arge h erarch es that are both very broad and very deep

Querying Hierarchical Tables W th a h erarch ca tab e popu ated and ndexed, you’re ready to start wr t ng effic ent quer es aga nst t us ng some more methods ava ab e on the hierarchyid type

The IsDescendantOf Method The IsDescendantOf method s nvoked on one hierarchyid va ue and accepts another hierarchyid va ue as ts parameter It returns a bit (Boo ean) va ue of 1 (true) f the hierarchyid that the method s nvoked on s a descendant (e ther d rect y or nd rect y) of the hierarchyid that s passed n as the parameter Thus, th s method essent a y returns a subtree whose root s the node spec fied by the parameter Because of ts vert ca traversa , t de vers very fast performance for tab es that have a depth-first ndex You can eas y query the emp oyee tab e to return a the descendants (ch d rows, grandch d rows, and so on) of a part cu ar emp oyee us ng the IsDescendantOf method For examp e, the fo ow ng query sts a of Amy’s descendants

Chapter 7  H erarch ca Data and the Re at ona Database   315

DECLARE @AmyNodeId hierarchyid SELECT @AmyNodeId = NodeId FROM Employee WHERE EmployeeId = 46 SELECT FROM WHERE ORDER

NodeId.ToString() AS NodeIdPath, * Employee NodeId.IsDescendantOf(@AmyNodeId) = 1 BY NodeLevel, NodeId

Th s query se ects a rows whose NodeId va ues are descendants of Amy (emp oyee ID 46) Here s the resu t NodeIdPath ---------/1/ /1/1/ /1/2/ /1/1/1/ /1/1/2/

NodeId -----0x58 0x5AC0 0x5B40 0x5AD6 0x5ADA

NodeLevel --------1 2 2 3 3

EmployeeId ---------46 269 389 87 90

EmployeeName -----------Amy Cheryl Wanda Richard Jeff

Title ---------------------Marketing Specialist Marketing Assistant Business Assistant Business Intern Business Intern

(5 row(s) affected)

Not ce that the resu t nc udes Amy n add t on to a of her descendants Amy s returned because she s cons dered to be her “own descendant” at the 0 eve of the subtree Thus, you have se ected a subtree that nc udes a of Amy’s descendants, no matter how many eve s deep they m ght ex st beneath her, w th Amy at the root To se ect only Amy’s mmed ate ch d rows that are just one eve beneath her, you can use the GetAncestor method (You used th s method ear er to create the uspAddEmployee stored procedure ) For examp e SELECT NodeId.ToString() AS NodeIdPath, * FROM Employee WHERE NodeId.GetAncestor(1) = (SELECT NodeId FROM Employee WHERE EmployeeId = 46) ORDER BY NodeLevel, NodeId GO NodeIdPath ---------/1/1/ /1/2/

NodeId -----0x5AC0 0x5B40

NodeLevel --------2 2

EmployeeId ---------269 389

EmployeeName -----------Cheryl Wanda

Title ---------------------Marketing Assistant Business Assistant

(2 row(s) affected)

Th s t me, the resu ts st on y Chery and Wanda, but not Amy nor any of Amy’s deeper ­ escendants That’s because you are request ng just the rows whose one- eve -up ancestor s Amy d (emp oyee ID 46)—that s, just Amy’s mmed ate ch dren

316  Part II  Going Beyond Relational

If you wanted to retr eve a the emp oyees exact y two eve s down from a part cu ar manager, you cou d pass the va ue 2 to the GetAncestor method For examp e, to se ect the emp oyees that report to the emp oyees beneath Dave (that s, to see a the emp oyees two eve s beneath h m), you cou d request rows whose grandparent s Dave (emp oyee ID 6; that s, just Dave’s grandch dren), as shown here SELECT NodeId.ToString() AS NodeIdPath, * FROM Employee WHERE NodeId.GetAncestor(2) = (SELECT NodeId FROM Employee WHERE EmployeeId = 6) ORDER BY NodeLevel, NodeId GO NodeIdPath ---------/1/1/ /1/2/ /2/1/ /3/1/

NodeId -----0x5AC0 0x5B40 0x6AC0 0x7AC0

NodeLevel --------2 2 2 2

EmployeeId ---------269 389 272 291

EmployeeName -----------Cheryl Wanda Mary Keven

Title ---------------------Marketing Assistant Business Assistant Marketing Assistant Marketing Intern

(4 row(s) affected)

Th s query returned a emp oyees that are two eve s be ow Dave The fact that some of them are cous ns and not s b ngs (that s, some of them have d fferent d rect parents) s rre evant Be ng exact y two eve s down beneath Dave qua fies them a for se ect on by th s query And because of ts hor zonta traversa , t de vers very fast performance w th a breadth-first ndex defined on the tab e To find the root node n the h erarchy, s mp y nvoke the GetRoot method on the hierarchyid data type tse f us ng the doub e-co on syntax, as shown n the fo ow ng code Th s s the same method you used to create the first emp oyee at the top of the tree (Dave, as shown n L st ng 7-2) SELECT NodeId.ToString() AS NodeIdPath, * FROM Employee WHERE NodeId = hierarchyid::GetRoot() GO NodeIdPath NodeId NodeLevel EmployeeId EmployeeName Title ---------- ------ --------- ---------- ------------ ---------------------/ 0x 0 6 Dave Marketing Manager (1 row(s) affected)

Reordering Nodes within the Hierarchy Reorgan z ng the nodes w th n the tree structure s a common ma ntenance task when work ng w th h erarch es You m ght need to a ter the tree structure by adjust ng the parent-ch d ­re at onsh ps w th n the h erarchy The hierarchyid type prov des a GetReparentedValue method that makes t easy to hand e th s k nd of ma ntenance You w start by first re ocat ng just a s ng e node w thout



Chapter 7  H erarch ca Data and the Re at ona Database   317

­ sturb ng any other nodes n the h erarchy Then you’ re ocate an ent re subtree (that s, a of a d node’s descendant nodes)

The GetReparentedValue Method You nvoke the GetReparentedValue method on the node you want to move, pass ng n two ­parameters The first spec fies the or g na parent (the source), and the second spec fies the new parent (the target) Wanda former y reported to Amy but s now report ng to J nstead (a ongs de Kev n) You therefore want to move Wanda’s current pos t on as a ch d of Amy to be a ch d of J nstead The fo ow ng code uses the GetReparentedValue method to perform that change DECLARE @EmployeeToMove hierarchyid DECLARE @OldParent hierarchyid DECLARE @NewParent hierarchyid SELECT @EmployeeToMove = NodeId FROM Employee WHERE EmployeeId = 389 -- Wanda SELECT @OldParent = NodeId FROM Employee WHERE EmployeeId = 46 -- Amy SELECT @NewParent = NodeId FROM Employee WHERE EmployeeId = 119 -- Jill -- Wanda now reports to Jill and no longer to Amy UPDATE Employee SET NodeId = @EmployeeToMove.GetReparentedValue(@OldParent, @NewParent) WHERE NodeId = @EmployeeToMove GO

The h erarchy now ooks ke F gure 7-5 Dave (6) /

Amy (46) /1/

Mary (272) /2/1/

Cheryl (269) /1/1/

Richard (87) /1/1/1/

John (271) /2/

Jeff (90) /1/1/2/

Figure 7-5  Reparent ng a s ng e node.

318  Part II  Going Beyond Relational

Jill (119) /3/

Kevin (291) /3/1/

Wanda (389) /3/2/

V ew ng the resu ts of the fo ow ng query confirms that Wanda’s NodeIdPath has changed from /1/2/ (ch d of Amy) to /3/2/ (ch d of J ) SELECT NodeId, NodeId.ToString() AS NodeIdPath, NodeLevel, dbo.fnGetFullDisplayPath(NodeId) AS NodeIdDisplayPath, EmployeeName FROM Employee ORDER BY NodeLevel, NodeId GO NodeId -----0x 0x58 0x68 0x78 0x5AC0 0x6AC0 0x7AC0 0x7B40 0x5AD6 0x5ADA

NodeIdPath ---------/ /1/ /2/ /3/ /1/1/ /2/1/ /3/1/ /3/2/ /1/1/1/ /1/1/2/

NodeLevel --------0 1 1 1 2 2 2 2 3 3

NodeIdDisplayPath ----------------------------Dave Dave > Amy Dave > John Dave > Jill Dave > Amy > Cheryl Dave > John > Mary Dave > Jill > Kevin Dave > Jill > Wanda Dave > Amy > Cheryl > Richard Dave > Amy > Cheryl > Jeff

EmployeeName -----------Dave Amy John Jill Cheryl Mary Kevin Wanda Richard Jeff

(10 row(s) affected)

Chang ng the pos t on of a s ng e node w th the GetReparentedValue method as you just d d does not affect any of that node’s ch dren (s nce on y one row was updated) Th s means that f Wanda had any ch d nodes, they wou d not move a ong w th Wanda as new descendants of J , now wou d they move up one eve n the h erarchy as d rect descendants of Amy to occupy the pos t on ­vacated by Wanda The same wou d be true f Wanda were s mp y de eted rather than moved Former ch d nodes of Wanda wou d therefore end up hav ng no parent at a , but wou d rather have on y a ­grandparent (wh ch n th s case wou d be Amy, Wanda’s former parent) The resu t s a “ho e” n your h erarchy SQL Server w not enforce ntegr ty checks on the h erarchy to catch th s cond t on, so t’s your job to hand e orphaned nodes proper y One way to ensure aga nst orphaned nodes s to ­transp ant an ent re subtree at one t me Let’s earn how to do just that

Transplanting Subtrees You can eas y move a arger number of peop e at one t me by reparent ng an ent re subtree Because you’ be mov ng a of a node’s descendants as a s ng e b ock, no orphaned nodes can resu t from th s operat on In the next update, you w move a of Amy’s subord nates to the r new manager, Kev n Th s s ach eved by reparent ng a nodes w th hierarchyid va ues that beg n w th /1/ (Amy) to /3/1/ (Kev n), except for Amy You use the IsDescendantOf method to return a of Amy’s descendants for the update Reca from our ear er d scuss on of the IsDescendantOf method that Amy s her own



Chapter 7  H erarch ca Data and the Re at ona Database   319

descendant at the 0 eve (that s, she’s the root of her own subtree) You must therefore exc ude her (emp oyee ID 46) from the rows to be updated, as shown n the fo ow ng code DECLARE @OldParent hierarchyid DECLARE @NewParent hierarchyid SELECT @OldParent = NodeId FROM Employee WHERE EmployeeId = 46 -- Amy SELECT @NewParent = NodeId FROM Employee WHERE EmployeeId = 291 -- Kevin UPDATE Employee SET NodeId = NodeId.GetReparentedValue(@OldParent, @NewParent) WHERE NodeId.IsDescendantOf(@OldParent) = 1 AND EmployeeId 46 -- This excludes Amy from the move. GO

Note  The employee ID values are hard-coded for Amy and Kevin in this code. This was done for demonstration purposes. In a real-world implementation, this logic should be in a stored procedure that accepts any two nodes as input parameters and wraps the SELECT and UPDATE statements in a transaction. Runn ng th s update moves a of Amy’s descendants to be descendants of Kev n, as shown n F­ gure 7-6 Not ce that the nodes have moved not on y across the tree, but down one eve as we (s nce Kev n s one eve deeper than Amy) Dave (6) /

Amy (46) /1/

Jill (119) /3/

John (271) /2/

Mary (272) /2/1/

Kevin (291) /3/1/

Wanda (389) /3/2/

Cheryl (269) /3/1/1/

Richard (87) /3/1/1/1/ Figure 7-6  Reparent ng an ent re subtree.

320  Part II  Going Beyond Relational

Jeff (90) /3/1/1/2/

Se ect ng a the emp oyees one more t me shows how SQL Server updated the hierarchyid va ues n NodeId for Chery , R chard, and Jeff to reflect the r new pos t ons beneath Kev n For the first t me n th s scenar o, the h erarchy now runs five eve s deep (count ng from 0 to 4), as shown here SELECT NodeId, NodeId.ToString() AS NodeIdPath, NodeLevel, dbo.fnGetFullDisplayPath(NodeId) AS NodeIdDisplayPath, EmployeeName FROM Employee GO NodeId --------0x 0x58 0x68 0x78 0x6AC0 0x7AC0 0x7B40 0x7AD6 0x7AD6B0 0x7AD6D0

NodeIdPath ---------/ /1/ /2/ /3/ /2/1/ /3/1/ /3/2/ /3/1/1/ /3/1/1/1/ /3/1/1/2/

NodeLevel --------0 1 1 1 2 2 2 3 4 4

NodeIdDisplayPath -------------------------------------Dave Dave > Amy Dave > John Dave > Jill Dave > John > Mary Dave > Jill > Kevin Dave > Jill > Wanda Dave > Jill > Kevin > Cheryl Dave > Jill > Kevin > Cheryl > Richard Dave > Jill > Kevin > Cheryl > Jeff

EmployeeName -----------Dave Amy John Jill Mary Kevin Wanda Cheryl Richard Jeff

(10 row(s) affected)

More hierarchyid Methods Three more hierarchyid methods—Parse, Read, and Write—are prov ded by the hierarchyid data type, a though they are ess often used Parse s essent a y the reverse of ToString It accepts the same s ash-de m ted str ng ­representat on that s returned by ToString and returns the equ va ent compacted varbinary hierarchyid va ue that SQL Server uses nterna y to represent the nodes Th s va ue cou d then be passed to ToString, n wh ch case you’d get the or g na s ash-de m ted str ng back It s the on y other stat c method bes des GetRoot, and so t uses the doub e-co on syntax, as fo ows SELECT hierarchyid::Parse('/2/1/1/') AS NodeId GO NodeId ---------0x6AD6 (1 row(s) affected)

Together, Parse and ToString enab e ser a zat on and deser a zat on to and from hierarchyid b nary va ues and the r str ng representat ons A though you cou d manage the str ng representat ons on

Chapter 7  H erarch ca Data and the Re at ona Database   321

your own and use Parse to convert them for storage as hierarchyid va ues, the hierarchyid va ues are best ass gned by ca ng the GetDescendant method, as we’ve demonstrated throughout our ­examp es The ast two methods, Read and Write, are the on y methods not ava ab e n T-SQL, and can be used on y n NET code aga nst the Microsoft.SqlServer.Types.SqlHierarchyId type These two methods are used to pass hierarchyid va ues nto and out of BinaryReader and BinaryWriter objects They are a so used nterna y by SQL Server as necessary, such as when read ng from or wr t ng to a hierarchyid co umn or for convers ons between varbinary and hierarchyid These methods are prov ded to enab e such nterna operat ons, as we as for ntegrat on w th your own NET code runn ng under SQL CLR They don’t otherw se serve any rea s gn ficant funct on n terms of the actua mp ementat on of a h erarch ca structure

Summary In th s chapter, we showed how to use the hierarchyid data type to cast a tree structure over data stored n ord nary database tab es You can now work w th h erarch ca structures n SQL Server, such as the new F eTab e ntroduced n SQL Server 2012 (you’ earn a about F eTab e the next ­chapter) The hierarchyid data type prov des a set of methods and ndex ng strateg es to enab e effic ent search ng and man pu at on of h erarchy nodes at the database eve Th s enab es you to app y a h erarch ca mode to trad t ona tab e data rather than resort ng to XML, n scenar os where t makes better sense to keep the data stored re at ona y than to redes gn the data for XML storage

322  Part II  Going Beyond Relational

C hapter 8

Native File Streaming —Leonard Lobel

T

he ongo ng pro ferat on of d g ta content n today’s wor d s generat ng unstructured data at a ­bew der ng rate Consequent y, modern app cat ons must cope w th a mu t tude of b nary fi es and formats You m ght have to store and process photos, v deos, documents, spreadsheets, ema ­messages, and other re ated unstructured art facts w th your database records Th s unstructured nformat on—wh ch s rea y just a b nary stream— s common y referred to as BLOB (B nary Large Object) data BLOB data needs to be assoc ated w th the structured data that ves n a re at ona database, and t must stream effic ent y nto and out of your app cat on Trad t ona y, re at ona database management systems store a the r data n tables, and that mp es structure Re at ona database systems were not ­or g na y des gned to hand e the mass ve amount of unstructured data that ex sts today To adapt, M crosoft SQL Server 2008 ntroduced FILESTREAM, an nnovat ve feature that ntegrates the re at ona database eng ne w th the NTFS fi e system In th s chapter, we’ start by exam n ng ­trad t ona BLOB so ut ons, and then exp a n how FILESTREAM prov des mportant advantages over the prev ous BLOB techn ques You w bu d some rea app cat ons and serv ces w th FILESTREAM, and then earn how the new F eTab e feature n M crosoft SQL Server 2012 de vers a og ca fi e ­system— mp emented as a FILESTREAM-enab ed tab e n your database—on top of your BLOBs

Traditional BLOB Strategies Pr or to SQL Server 2008, there were two trad t ona so ut ons for comb n ng structured tab e data w th unstructured BLOB data e ther keep BLOBs n the database (w th a your structured tab e data), or store them outs de the database ( n the fi e system) In the former case, BLOBs are stored r ght ns de the database In the atter case, the database mere y ho ds references (or, poss b y, references are der ved from other va ues n the database) that po nt to ocat ons n the fi e system where the BLOBs actua y ve Each of these strateg es has pros and cons w th respect to storage, manageab ty, performance, and programm ng comp ex ty that we’ d scuss—but ne ther of them are ntr ns ca y nat ve to the core database eng ne

323

BLOBs in the Database You can, of course, s mp y store BLOB data d rect y n the co umns of your database tab es You do th s by dec ar ng a co umn as a varbinary(max) data type, wh ch w a ow t to store a s ng e BLOB up to 2 g gabytes (GB) n s ze

Important  You should no longer use the image data type that was used to store BLOBs prior to Microsoft SQL Server 2005. The varbinary(max) data type should be used instead of image, which has been deprecated and may be removed in a future version of SQL Server. When BLOB data s stored d rect y ns de of tab es th s way, t s very t ght y ntegrated w th the database It’s easy to access BLOB data, because the BLOB s r ght there n the tab e’s co umn Because everyth ng s conta ned n a s ng e database un t, management s a so s mp fied Backup, restore, detach, copy, and attach operat ons on the database nc ude structured and BLOB data together Transact ona cons stency s another mportant benefit that you enjoy w th th s approach Because BLOB data s a phys ca part of the tab es n the database, t s e g b e to part c pate n transact ons If you beg n a transact on, update some data, and then ro back the transact on, any BLOB data that was updated s a so ro ed back Overa , the m xture of structured and BLOB data s hand ed very seam ess y w th th s mode Desp te a these advantages, however, phys ca y stor ng BLOBs n the database s pract ca on y for sma -sca e app cat ons hav ng very few/sma BLOBs Because BLOB content s stored n- ne w th structured data, you can severe y mpa r sca ab ty by b oat ng your fi egroups Query performance w degrade rap d y as a resu t, because the query processor must s ft through much arger amounts of data n your tab es that are consumed w th BLOB content The BLOBs a so don’t stream near y as ­effic ent y when backed by SQL Server varbinary(max) co umns as they wou d f they were he d externa y n the fi e system or on a ded cated BLOB store F na y, there s a so a 2 GB m t on the varbinary(max) data type

Tip  If you have modest BLOB requirements (that is, you are dealing with very few or very small BLOBs) you should store them in the database using the varbinary(max) data type ­instead of using the file system (either directly, or via FILESTREAM). Furthermore, you should consider caching small, frequently accessed BLOBs rather than repeatedly retrieving them from the database or the file system.

BLOBs in the File System To counter these concerns, you can store BLOBs outs de the database and n the fi e system nstead W th th s approach, the structured data n your re at ona tab es mere y conta ns path nformat on to the unstructured BLOB data, wh ch s he d externa y as ord nary fi es n the fi e system ­(a ternat ve y, path nformat on can be der ved from other structured data n a row) App cat ons use th s path nformat on as a reference for ocat ng and track ng the BLOB content assoc ated w th rows n the 324  Part II  Going Beyond Relational

database tab es Because they are phys ca y he d n the fi e system, a BLOB’s s ze s m ted on y by the host fi e system and ava ab e d sk space Th s approach a so de vers much better stream ng ­performance, because the fi e system s a nat ve env ronment that’s h gh y opt m zed for stream ng And because the phys ca database s much sma er w thout the BLOBs ns de t, the query processor can cont nue to de ver opt ma performance A though phys ca y separat ng structured and unstructured content th s way does address the ­ erformance concerns of BLOBs, t a so ra ses new ssues because the data s now separated not on y p phys ca y, but og ca y as we SQL Server has abso ute y no awareness of the assoc at on between data n the database and fi es stored externa y n the fi e system The r coup ng ex sts so e y at the app cat on eve Backup, restore, detach, copy, and attach operat ons on the database fi es therefore nc ude on y structured tab e data w thout any of the BLOB data that’s n the fi e system You won’t get comp ete backups, un ess you back up the fi e system as we , so now t’s another adm n strat ve burden to separate y manage the fi e system App cat on deve opment aga nst th s mode s a so more comp ex because of the extra effort r­ equ red for nk ng between the database and the fi e system It’s up to the deve oper to ­estab sh and ma nta n the references between structured data and externa BLOBs on the r own, and ­accord ng to the r own custom des gn Last, a though perhaps most s gn ficant, there s no un fied transact ona contro across both the database and the fi e system Natura y, ro ng back a database transact on won’t undo changes you’ve made n the fi e system

Introducing FILESTREAM Both of the trad t ona BLOB so ut ons present tough cha enges, so what do you do? W th FILESTREAM, SQL Server offers a way out of th s conundrum F rst, make sure you understand that FILESTREAM s techn ca y not a SQL Server data type Rather, t s mp emented as an attribute that you app y to the varbinary(max) data type—the same data type trad t ona y used to store BLOBs d rect y ns de structured tab es However, mere y app y ng th s attr bute te s SQL Server to store the BLOB n the fi e system rather than the tab e’s structured fi egroup W th the FILESTREAM attr bute app ed, you cont nue to treat the varbinary(max) co umn as though ts contents were stored n- ne w th your structured tab e data Under the covers, however, SQL Server stores and ma nta ns the data n the server’s oca NTFS fi e system, separate y from the structured content of the database that rema ns n the norma fi egroups W th FILESTREAM, structured and unstructured data are logically connected wh e physically separated The unstructured data s configured as a spec a fi egroup n the database, so t’s actua y cons dered part of the database— t s ava ab e n a og ca database operat ons, nc ud ng quer es, transact ons, and backup/restore On d sk, however, the BLOBs are stored as nd v dua phys ca fi es n the NTFS fi e system that are created and managed automat ca y beh nd the scenes SQL Server estab shes and ma nta ns the nk references between the structured fi e groups and the fi e ­system It knows about the unstructured BLOB data n the fi e system and cons ders the fi es ho d ng BLOB data to be an ntegra part of the overa database But the unstructured data does not mpede query ­performance because t s not phys ca y stored n- ne w th tab e data It’s stored n the fi e

Chapter 8  Nat ve F e Stream ng   325

s­ ystem, wh ch s a nat ve BLOB env ronment (and where t ostens b y be ongs) Log ca y, however, the ­database encompasses both the re at ona tab es and the nd v dua BLOB fi es n the fi e system You therefore cont nue to treat BLOB data as though you were stor ng t n- ne, from both a deve opment and an adm n strat ve perspect ve

Tip  Backing up the database includes all the BLOB data from the file system in the backup automatically. However, because the BLOB data is contained in its own database filegroup, you can easily exclude it from backups if desired or as needed. The end resu t s that SQL Server uses the appropr ate mechan sm for structured and unstructured data—stor ng re at ona (structured) data n tab es and BLOB (unstructured) data n ord nary fi es—so t can de ver the best poss b e performance a around Because t does th s comp ete y transparent y, you enjoy ntegrated management benefits over the database You a so enjoy s mp fied app cat on deve opment as you are no onger burdened w th the add t ona comp ex t es of manua y assoc at ng the database w th the fi e system and keep ng the two n sync Last, by everag ng the transact ona capab t es of the NTFS fi e system, BLOB updates part c pate seam ess y w th database transact ons If you’re start ng to get exc ted by a th s, that’s the dea! You’re now ready to d ve n to some rea code that puts FILESTREAM to work for you

Enabling FILESTREAM L ke many other features, FILESTREAM s d sab ed by defau t n SQL Server 2012, and you must first enab e t before the feature can be used Enab ng FILESTREAM s s ght y more nvo ved than enab ng other SQL Server features because t requ res two d st nct steps F rst, the feature needs to be enab ed for the mach ne, and then t needs to be enab ed for the server nstance These two ndependent FILESTREAM configurat on ayers are by des gn, and they draw a separat on of secur ty concerns ­between the ro e of W ndows adm n strator and database adm n strator

Enabling FILESTREAM for the Machine The first step s to enab e FILESTREAM for the mach ne by sett ng an access eve Th s step can ­actua y be performed at the t me that SQL Server s n t a y nsta ed by choos ng a FILESTREAM ­access eve dur ng setup To enab e FILESTREAM for the mach ne after SQL Server has been nsta ed, use the SQL Server Configurat on Manager to set the access eve (Th s too can be aunched from the Configurat on Too s fo der of the M crosoft SQL Server 2012 program group on the Start menu ) The SQL Server Configurat on Manager opens w th a treev ew on the eft In the treev ew, c ck SQL Server Serv ces to d sp ay the st of ava ab e serv ces n the ma n pane R ght-c ck the SQL Server nstance that you want to enab e FILESTREAM for, choose Propert es, and n the Propert es d a og box, se ect the FILESTREAM tab The three check boxes on the FILESTREAM tab a ow you to se ect the var ous eve s of FILESTREAM access F gure 8-1 shows the Propert es d a og box w th a three check boxes se ected 326  Part II  Going Beyond Relational

Figure 8-1  Enab ng F LESTREAM for the mach ne to support fi e /O stream ng access by remote c ents.

When a three check boxes are c eared, FILESTREAM s comp ete y d sab ed Se ect ng the first check box enab es FILESTREAM, but on y for Transact-SQL (T-SQL) access Th s prov des for ­utter transparency, where SQL Server w store BLOBs conta ned ns de varbinary(max) FILESTREAM co umns n the fi e system beh nd the scenes just as we’ve d scussed But t won’t a ow you to take advantage of d rect fi e stream ng between the database and your c ent app cat ons us ng more advanced FILESTREAM features, such as SqlFileStream and F eTab e that you’ be earn ng about soon The rea power of FILESTREAM comes nto p ay when you enab e d rect fi e I/O stream ng, wh ch de vers the best poss b e BLOB performance w th SQL Server You enab e d rect fi e I/O stream ng access by se ect ng the second check box Streamed fi e access a so creates a W ndows share name that s used to construct og ca Un versa Nam ng Convent on (UNC) paths to BLOB data dur ng FILESTREAM access, as you’ see further on when you use SqlFileStream and F eTab e The share name s spec fied n a text box after the second check box and s set by defau t to the same name as the server nstance (MSSQLSERVER, n F gure 8-1) In most cases, c ent app cat ons w not be runn ng on the same mach ne as SQL Server, and so you w usua y a so need to se ect the th rd check box to enab e FILESTREAM for remote c ent fi e I/O stream ng access w th SqlFileStream In add t on, the new F eTab e feature n SQL Server 2012 requ res that you check th s opt on n order to expose ts data as a fi e system through the W ndows share created by FILESTREAM (you’ exam ne F eTab e after you fin sh earn ng about FILESTREAM) So pract ca y, you must check a three checkboxes to get the most out of FILESTREAM (and to support a the exerc ses n th s chapter) An except on to th s genera pract ce m ght be when us ng M crosoft SQL Server 2012 Express ed t on as a oca data store for a c ent app cat on w th everyth ng runn ng

Chapter 8  Nat ve F e Stream ng   327

on the same mach ne In th s case, you cou d use the more secure sett ng and eave the th rd check box c eared Do ng so wou d enab e SqlFileStream and F eTab e access for the oca c ent app cat on but deny such access to remote c ents If you are us ng SQL Server Configurat on Manager to change th s sett ng after SQL Server has a ready been nsta ed, you must remember to restart the SQL Server nstance for the sett ng change to take effect

More Info  There is no T-SQL equivalent script that can set the FILESTREAM access level for the machine. However, Microsoft posts a VBScript file available over the Internet that allows you to enable FILESTREAM from the command line as an alternative to using SQL Server Configuration Manager. At press time, the download page for this script is http://www. codeplex.com/SQLSrvEngine/Wiki/View.aspx?title FileStreamEnable&referringTitle Home.

Enabling FILESTREAM for the Server Instance The second step s to enab e FILESTREAM for the server nstance The concept here s s m ar to what we just descr bed Vary ng eve s of access correspond to the checkboxes n F gure 8-1 Natura y, the access eve defined for the server nstance must be supported by the access eve defined for the mach ne Typ ca y, therefore, the mach ne and server nstance access eve s shou d be set to match one another FILESTREAM can be enab ed for the server nstance w th a s mp e ca to the sp configure system stored procedure, as fo ows EXEC sp_configure filestream_access_level, n RECONFIGURE

In the preced ng code, rep ace n w th a number from 0 to 2 to set the access eve A va ue of 0 ­ sab es the FILESTREAM feature comp ete y Sett ng the access eve to 1 enab es FILESTREAM for d T-SQL access on y, and sett ng t to 2 enab es FILESTREAM for fu access (wh ch nc udes oca or remote fi e I/O stream ng access as enab ed for the mach ne n the first step) To support the samp e SqlFileStream NET c ent app cat on that you’ be bu d ng ater n th s chapter, you must se ect eve 2 (fu access) Th s corresponds to the th rd checkbox n F gure 8-1 You can a so set the FILESTREAM access eve for the server nstance n SQL Server ­Management Stud o (SSMS) from the Advanced Server Propert es d a og box R ght-c ck any server nstance n ­Object Exp orer, choose Propert es, and then se ect the Advanced page The var ous eve s are ­ava ab e as cho ces n the F estream Access Leve drop-down st, as shown n F gure 8-2

Note  You can use either SQL Server Data Tools (SSDT) inside Visual Studio (covered in Chapter 1) or SSMS to set the access level, but only SSMS provides the ability to set it in the graphical user interface (GUI). Using SSDT, you can view the setting’s property from the SQL Server Object Explorer, but you can only change it by running the EXEC statement in a query window.

328  Part II  Going Beyond Relational

Figure 8-2  Se ect ng the F LESTREAM configurat on eve n SQL Server Management Stud o.

Creating a FILESTREAM-Enabled Database Once FILESTREAM s enab ed for both the mach ne and the server nstance, any database ­runn ng on the server nstance can support unstructured data by defin ng a fi egroup w th the spec a ­FILEGROUP…CONTAINS FILESTREAM c ause of the CREATE DATABASE statement For examp e, the statement n L st ng 8-1 creates a PhotoLibrary database that can store p ctures us ng FILESTREAM Before you can run th s code, you need to create an empty fo der named C \Demo\PhotoL brary Listing 8-1  Creat ng a F LESTREAM enab ed database w th FILEGROUP…CONTAINS FILESTREAM.

CREATE DATABASE PhotoLibrary ON PRIMARY (NAME = PhotoLibrary_data, FILENAME = 'C:\Demo\PhotoLibrary\PhotoLibrary_data.mdf'), FILEGROUP FileStreamGroup1 CONTAINS FILESTREAM (NAME = PhotoLibrary_group2, FILENAME = 'C:\Demo\PhotoLibrary\Photos') LOG ON (NAME = PhotoLibrary_log, FILENAME = 'C:\Demo\PhotoLibrary\PhotoLibrary_log.ldf')

The FILEGROUP…CONTAINS FILESTREAM c ause n th s otherw se ord nary CREATE DATABASE statement enab es the FILESTREAM feature for the PhotoLibrary database As when creat ng any database, the d rectory (or d rector es) spec fied for the pr mary and og ­ egroups must ex st at the t me the database s created In L st ng 8-1, the CREATE DATABASE fi statement w fa f the C \Demo\PhotoL brary d rectory spec fied by FILENAME n ts ON PRIMARY and LOG ON c auses does not ex st For the new FILESTREAM group spec fied by the FILEGROUP… CONTAINS FILESTREAM c ause, a FILENAME keyword s spec fied po nt ng to a d rectory (not a fi e) that must not ex st at the t me that the database s created (a though the path ead ng up to the fina d rectory must ex st), or the CREATE DATABASE statement w fa as we SQL Server takes contro of creat ng and manag ng th s d rectory—ca ed the FILESTREAM data conta ner—much as

Chapter 8  Nat ve F e Stream ng   329

t does for creat ng and manag ng the mdf and df fi es n the other fi egroups In L st ng 8-1, SQL Server ­automat ca y creates the C \Demo\PhotoL brary\Photos fo der when the CREATE DATABASE ­statement s executed, and t w use that fo der for stor ng a BLOB data (photos, n th s examp e) n the PhotoLibrary database Execute the code n L st ng 8-1 to create the database SQL Server creates the usua mdf and df fi es for you, and a so creates the Photos subd rectory for the FILESTREAM group, as shown n F gure 8-3

Figure 8-3  F LESTREAM storage n the fi e system.

Beh nd the scenes, SQL Server w store a your p ctures as fi es n the Photos subd rectory, and w transparent y assoc ate those fi es and the re at ona tab es that they og ca y be ong to n co umns defined as varbinary(max) FILESTREAM Un ess you exp c t y exc ude the FileStreamGroup1 fi egroup from a backup or restore command, a your p cture fi es n the Photos subd rectory w be nc uded w th the re at ona database n the backup or restore operat on

Note  You can create multiple FILESTREAM filegroups, with each one pointing to a ­different file system location. Doing so helps to partition BLOB data when you need to scale up, ­because you can designate specific filegroups for each varbinary(max) FILESTREAM column you define in your tables. See Microsoft Books Online for the appropriate syntax: http://msdn.microsoft.com/en-us/library/ms176061.aspx.

Creating a Table with FILESTREAM Columns You’re now ready to create a new PhotoAlbum tab e SQL Server requ res that any tab e u ­ s ng FILESTREAM storage has a uniqueidentifier co umn that s not nu ab e and spec fies the ­ROWGUIDCOL attr bute You must a so create a un que constra nt on th s co umn On y one ­ROWGUIDCOL co umn 330  Part II  Going Beyond Relational

can be defined n any g ven tab e, a though you can then dec are any number of varbinary(max) FILESTREAM co umns n the tab e that you want for stor ng BLOB data The statement n L st ng 8-2 creates the PhotoAlbum tab e w th a Photo co umn dec ared as varbinary(max) FILESTREAM Listing 8-2  Creat ng a F LESTREAM enab ed tab e.

USE PhotoLibrary GO CREATE TABLE PhotoAlbum( PhotoId int PRIMARY KEY, RowId uniqueidentifier ROWGUIDCOL NOT NULL UNIQUE DEFAULT NEWSEQUENTIALID(), Description varchar(max), Photo varbinary(max) FILESTREAM DEFAULT(0x))

W th th s statement, you sat sfy the FILESTREAM requ rement for the ROWGUIDCOL co umn, yet you won’t actua y have to do anyth ng to ma nta n that co umn By dec ar ng the RowId co umn w th ts DEFAULT va ue set to ca the NEWSEQUENTIALID funct on, you can just gnore th s co umn when nsert ng rows—s mp y not prov d ng va ues for t w cause SQL Server to automat ca y generate the next ava ab e g oba y un que dent fier (GUID) that t needs to support FILESTREAM on the tab e The co umn s set to not accept NULL va ues and s defined w th the requ red un que constra nt You have a so dec ared an nteger PhotoId co umn for the tab e’s pr mary key va ue You’ use the PhotoId co umn to dent fy nd v dua photos n the a bum, and SQL Server w use the RowId co umn to track and cross-reference photos n the fi e system w th rows n the PhotoAlbum tab e The Photo co umn ho ds the actua BLOB tse f, be ng defined as a varbinary(max) data type w th the FILESTREAM attr bute app ed Th s means that t gets treated ke a regu ar varbinary(max) co umn, but you know that ts BLOB s rea y be ng stored under the covers n the fi e system by SQL Server The Photo co umn s a so defined w th a defau t va ue of 0x, wh ch represents a zero- ength b nary stream Th s w come nto p ay ater when you start programm ng w th SqlFileStream We’re not there yet, so you can just gnore the defau t ass gnment for now

Storing and Retrieving FILESTREAM Data To start, you’ use p a n T-SQL to cast s mp e str ng data nto and out of the varbinary(max) data type of the Photo co umn Th s s certa n y contr ved, but start ng sma ke th s s the best way to earn FILESTREAM You are go ng to mon tor and observe effects on the NTFS fi e system as SQL Server ut zes t for BLOB storage Beg n w th the fo ow ng INSERT statement that adds your first row to the PhotoAlbum tab e INSERT INTO PhotoAlbum(PhotoId, Description, Photo) VALUES(1, 'Text pic', CAST('BLOB' As varbinary(max)))

Th s INSERT statement reads no d fferent y than t wou d f you were us ng a regu ar varbinary(max) co umn for the Photo co umn w thout the FILESTREAM attr bute It appears to store

Chapter 8  Nat ve F e Stream ng   331

the unstructured Photo co umn data n ne w th the rest of the re at ona co umns, and t certa n y ­appears the same way when return ng the data back w th a SELECT query, as shown here SELECT *, CAST(Photo AS varchar) AS PhotoText FROM PhotoAlbum GO PhotoId RowId Description Photo PhotoText ------- ------------------------------------ ----------- ---------- --------1 FC7D28BC-E8C6-E011-9849-080027565B78 Text pic 0x424C4F42 BLOB (1 row(s) affected)

However, f you peek ns de the FILESTREAM data conta ner, you can ver fy that SQL Server s ­actua y stor ng the Photo co umn outs de the database n the fi e system Because SQL Server ­obfuscates the fi e system that t’s manag ng for you beh nd the scenes, you can’t understand the manner n wh ch fi es and fo ders are named, organ zed, and referenced back to the re at ona ­database But just by dr ng down and prob ng the subfo ders beneath the Photos d rectory, you w d scover that there s n fact a new fi e stored n the fi e system (W ndows w prompt for perm ss on before open ng the Photos d rectory ) Th s fi e was created as a resu t of the INSERT statement V ew t by r ght-c ck ng the fi e name and choos ng Open, as shown n F gure 8-4

Figure 8-4  Exp or ng the F LESTREAM fi e system (note that the actua fo der and fi e names w

ones shown n th s figure).

not match the

If you se ect Notepad to open the fi e, you w see c ear proof that the unstructured content of the Photo co umn s stored outs de the database and n the fi e system In th s examp e, the text BLOB that was nserted nto the Photo co umn s stored n the fi e that you’ve opened n Notepad, as shown n F gure 8-5

332  Part II  Going Beyond Relational

Figure 8-5  Exam n ng unstructured F LESTREAM content n Notepad.

Th s c ear y shows how FILESTREAM data s og ca y connected to—but phys ca y separated from—the database Because the unstructured data s stored ent re y n the fi e system, you can eas y a ter ts content by d rect y updat ng the fi e n Notepad w thout nvo v ng the database To prove the po nt once aga n, et’s change the text n the fi e from BLOB to Cool, and save the changes back to the fi e system, as d sp ayed n F gure 8-6

Figure 8-6  Chang ng F LESTREAM content d rect y n the fi e system.

The a tered FILESTREAM data s reflected n the same SELECT statement you ran ear er, as shown here SELECT *, CAST(Photo AS varchar) AS PhotoText FROM PhotoAlbum GO PhotoId RowId Description Photo PhotoText ------- ------------------------------------ ----------- ---------- --------1 FC7D28BC-E8C6-E011-9849-080027565B78 Text pic 0x436F6F6C Cool (1 row(s) affected)

You are perform ng th s exerc se to ver fy that SQL Server s us ng the fi e system to store varbinary(max) FILESTREAM data However, we must stress that the purpose here s pure y ­demonstrat ve Ord nar y, you must never tamper d rect y w th fi es n the FILESTREAM data conta ner th s way FILESTREAM prov des a total abstract on over the varbinary(max) data type, and you need to cons der the phys ca fi e system as part of the database (wh ch t s, by v rtue of the FILESTREAM group assoc ated w th t)— t gets managed by SQL Server exc us ve y Later n the chapter, you’ earn how F eTab e (new n SQL Server 2012) adds yet another ayer of abstract on by expos ng a network share over FILESTREAM-based BLOBs that can be treated and accessed just ke an ord nary fi e system

Note  Normal SQL Server column-level security permissions apply to varbinary(max) FILESTREAM columns.



Chapter 8  Nat ve F e Stream ng   333

In- n ng BLOB data n T-SQL s ted ous, but t s feas b e and can be done when you are work ng w th re at ve y sma byte streams Run the fo ow ng INSERT statement to add a second row, th s t me w th a rea mage n the Photo co umn INSERT INTO PhotoAlbum(PhotoId, Description, Photo) VALUES(2, 'Document icon', 0x4749463839610C000E00B30000FFFFFFC6DEC6C0C0C0000080000000D3121200000 000000000000000000000000000000000000000000000000000000021F90401000002002C000000000C000E0000042C9 0C8398525206B202F1820C80584806D1975A29AF48530870D2CEDC2B1CBB6332EDE35D9CB27DCA554484204003B)

Now rev s t the fi e system and you’ find another fi e was just created Th s new fi e conta ns the mage represented by the byte stream n the INSERT statement you just executed R ght-c ck the fi e and choose Open, but th s t me se ect Pa nt to open the fi e w th F gure 8-7 shows the mage open and magn fied n Pa nt

Figure 8-7  Exam n ng unstructured F LESTREAM content n Pa nt.

More ke y, the BLOB source w be an externa fi e In th s case, you can use OPENROWSET w th ts BULK and SINGLE BLOB opt ons to consume the fi e (wh ch can be ocated on a remote fi e share) nto SQL Server as a varbinary(max) type For examp e, f the BLOB s access b e as a fi e named \\public\ shared\doc.ico, then the same resu t can be ach eved w th the fo ow ng INSERT statement INSERT INTO PhotoAlbum(PhotoId, Description, Photo) VALUES(2, 'Document icon', (SELECT BulkColumn FROM OPENROWSET(BULK '\\public\shared\doc.ico', SINGLE_BLOB) AS x))

Deleting FILESTREAM Data When a row s de eted from a FILESTREAM-enab ed tab e, the row s removed from the tab e and the fi es assoc ated w th the row’s varbinary(max) FILESTREAM co umns are removed from the fi e system The fi es are de eted by the FILESTREAM garbage co ector runn ng on a separate background thread, so you m ght not ce that the phys ca fi e s not removed from the FILESTREAM data conta ner ­ mmed ate y 334  Part II  Going Beyond Relational

SQL Server tr ggers the FILESTREAM garbage co ector thread when a CHECKPOINT occurs, wh ch, for databases that are not h gh y transact ona , m ght take a ong t me to occur If you want to tr gger the garbage co ector thread yourse f, you can ssue an exp c t CHECKPOINT statement to the server

Direct Streaming in .NET with SqlFileStream You just saw how to embed and extract b nary streams of data n T-SQL A s m ar T-SQL approach can be used w th ADO NET n your NET app cat ons to embed and extract byte arrays aga nst varbinary(max) FILESTREAM co umns It’s easy, and t works, but t s not the most effic ent way to transfer BLOBs nto and out of the database Beh nd the scenes, SQL Server must use ts own memory to stream BLOBs n and out, and t exposes BLOBs to c ent app cat ons as varbinary(max) data types that are not opt m zed for stream ng The proper (and fastest) way to get data nto and out of FILESTREAM co umns s to bu d a NET ­c ent app cat on (wr tten n C# or VB NET, for examp e), and use the spec a SqlFileStream c ass ( n NET 3 5 SP1 and h gher) to access the BLOBs SqlFileStream s a NET wrapper around ­OpenSqlFilestream, a funct on prov ded by the SQL Server nat ve c ent app cat on programm ng nterface (API) that g ves your app cat on safe, d rect access to varbinary(max) FILESTREAM co umn data stored n the fi e system W th th s approach, SQL Server does not use any of ts own memory for the BLOB transfer, wh ch reduces demand on server resources Instead, your app cat on, us ng ts own resources, streams d rect y aga nst the fi e system w thout the overhead of the varbinary(max) data type abstract on The resu t s ghtn ng fast BLOB performance—a though there’s a pr ce to be pa d, of course D rect stream ng w th SqlFileStream s re at ve y more comp ex than us ng T-SQL, because you need to nteract w th the FILESTREAM data store Overa , t’s not too onerous though, and as you’re about to see, t’s not d fficu t to understand and mp ement the appropr ate cod ng pattern

Understanding SqlFileStream W th SqlFileStream, NET app cat ons can use the standard FileStream c ass n the System.IO namespace to de ver h gh-performance stream ng of BLOB data aga nst the SQL Server-managed fi e system In the next sect on, you’ earn how to use SqlFileStream to store and retr eve p ctures n the Photo co umn But before we d g nto the ntr cac es of cod ng, et’s first get a h gh- eve ­understand ng of how SqlFileStream works

Note  You can also create native-code FILESTREAM applications (written in in C++, for ­example) by calling OpenSqlFilestream directly. The handle returned by this function can then use either the ReadFile and WriteFile Microsoft Win32 API functions. Our samples are .NET (managed-code) applications written in C#, but the process we describe here is ­fundamentally the same for native-code applications. You beg n by start ng a database transact on When work ng w th SqlFileStream, you a ways work w th transact ons (even for read access) There s no way to avo d them because FILESTREAM, by

Chapter 8  Nat ve F e Stream ng   335

des gn, coord nates transact ona ntegr ty across structured and unstructured data access n the database and the NTFS fi e system After start ng the transact on, you perform standard T-SQL operat ons to query or mod fy data But you don’t actua y nc ude varbinary(max) FILESTREAM co umns (BLOBs) n your T-SQL statements (ne ther for read ng nor for wr t ng) Instead, you ask SQL Server to g ve you two key p eces of nformat on that w enab e you to access the BLOBs much more effic ent y than us ng T-SQL—spec fica y, v a d rect fi e stream ng us ng SqlFileStream F rst, you need a og ca UNC path to the fi e ho d ng the BLOB on the server, wh ch you get by ca ng the PathName method on the des red varbinary(max) FILESTREAM va ue nstance Second, you need the fi e system transact on context, wh ch you get by ca ng the GET FILESTREAM TRANSACTION CONTEXT funct on (wh ch returns NULL f a transact on has not yet been estab shed) Ne ther of these tems mean anyth ng to your code; you just need to pass them on, a ong w th your des red access (read or wr te), to the SqlFileStream constructor The constructed SqlFileStream object nher ts from System.IO.Stream, so you can then read or wr te aga nst t us ng the same NET cod ng patterns you use to access ord nary streams Th s prov des you w th a stream ng “tunne ” between your app cat on and the SQL Server nterna y ­managed fi e system for each BLOB When you start the database transact on, SQL Server nterna y n t ates an NTFS fi e system ­transact on over the BLOB data and assoc ates t w th the current database transact on SQL Server then further ensures that both transact ons w subsequent y e ther comm t or ro back together On y when you comm t the database transact on does SQL Server nterna y comm t the NTFS fi e system transact on Th s permanent y saves changes to both the structured fi egroups of the database (from the T-SQL operat ons) and the fi es n the fi e system (from the stream ng I/O operat ons) at the same t me S m ar y, ro ng back the transact on undoes changes to both the database and fi e system s mu taneous y

Note The UNC reference returned by the PathName method is not a real path to the physical file system on the server. Rather, PathName returns a fabricated path used by SqlFileStream to enable direct streaming between the file system and client a ­ pplications. This UNC path always begins with the share name specified when FILESTREAM was ­enabled for the machine (Figure 8-1) and contains the GUID value in the corresponding row’s uniqueidentifier ROWGUIDCOL column. The file system is secured on the server no ­differently than the data and transaction filegroups (.mdf and .ldf files) are secured. Users should never be granted direct access to the file system on the server (although you can grant them indirect access via a FileTable, as we’ll discuss later on). Cont nu ng w th the photo brary examp e, you’ now create a W ndows Forms app cat on n C# that conta ns a the key e ements that br ng a FILESTREAM app cat on together—noth ng more, noth ng ess Your app cat on w a ow users to create new photos n the database by stream ng BLOBs nto the Photo co umn Users can a so se ect photos, wh ch w stream BLOBs back out from the Photo co umn and nto a PictureBox contro for d sp ay

336  Part II  Going Beyond Relational

Building the Windows Forms Client You’ beg n w th the user nterface (UI), wh ch s very s mp e Start M crosoft V sua Stud o 2010 and create a new C# W ndows Forms app cat on Des gn a form w th two separate group boxes one at the top of the form for nsert ng photos and another beneath t for retr ev ng photos Prov de abe s and text boxes for enter ng a photo ID, fi e name, and descr pt on n the top group box, a ong w th a nk abe to nvoke a save operat on In the bottom group box, prov de a text box and abe for ­enter ng a photo ID and a nk abe to nvoke a oad operat on Inc ude a abe to d sp ay the descr pt on returned from the database and a p cture box to d sp ay the photo BLOB returned v a FILESTREAM After perform ng some aesthet c a gnment and formatt ng, your form shou d appear someth ng ke the one shown n F gure 8-8

Figure 8-8  A s mp e F LESTREAM W ndows U form.

You’ wr te on y a very sma amount of code beh nd th s W ndows form, and mp ement the FILESTREAM og c n a separate data access c ass Add the code beh nd the c ck events for the form’s Save and Load nk buttons, as shown n L st ng 8-3 Listing 8-3  U ca s nto F LESTREAM data access c ass for sav ng and oad ng mage fi es.

private void lnkSave_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { int photoId = int.Parse(this.txtSavePhotoId.Text); string desc = this.txtDescription.Text; string filename = this.txtFilename.Text; PhotoData.InsertPhoto(photoId, desc, filename); }



Chapter 8  Nat ve F e Stream ng   337

private void lnkLoad_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { int photoId = int.Parse(this.txtLoadPhotoId.Text); string desc; Image photo = PhotoData.SelectPhoto(photoId, out desc); this.lblDescription.Text = desc; this.picImage.Image = photo; }

When the user supp es new photo nformat on and c cks Save, the code retr eves the photo ID, ­descr pt on, and fi e name from the three text boxes and passes them to the InsertPhoto method of the PhotoData c ass When the user spec fies a photo ID and c cks Load, the code ca s the SelectPhoto method of the PhotoData c ass to retr eve the requested mage (w th ts textua descr pt on) for d sp ay

Programming SqlFileStream Data Access A the mag c happens ns de the PhotoData c ass, wh ch s shown n L st ng 8-4

Note  As with all code in this book, the full FILESTREAM demo code in this chapter is ­available on the book’s companion website. Listing 8-4  mp ement ng a F LESTREAM data access managed c ent c ass.

using using using using using using using

System; System.Data; System.Data.SqlClient; System.Data.SqlTypes; System.Drawing; System.IO; System.Transactions;

namespace PhotoLibraryApp { public class PhotoData { private const string ConnStr = "Data Source=.;Integrated Security=True;Initial Catalog=PhotoLibrary;"; #region "Insert Photo" public static void InsertPhoto(int photoId, string desc, string filename) { const string InsertTSql = @" INSERT INTO PhotoAlbum(PhotoId, Description) VALUES(@PhotoId, @Description);

338  Part II  Going Beyond Relational

SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(InsertTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; cmd.Parameters.Add("@Description", SqlDbType.VarChar).Value = desc; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } SavePhotoFile(filename, serverPath, serverTxn); } ts.Complete(); } } private static void SavePhotoFile (string clientPath, string serverPath, byte[] serverTxn) { const int BlockSize = 1024 * 512; using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read)) { using (SqlFileStream dest = new SqlFileStream(serverPath, serverTxn, FileAccess.Write)) { byte[] buffer = new byte[BlockSize]; int bytesRead; while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) { dest.Write(buffer, 0, bytesRead); dest.Flush(); } dest.Close(); } source.Close();



Chapter 8  Nat ve F e Stream ng   339

} } #endregion #region "Select Photo" public static Image SelectPhoto(int photoId, out string desc) { const string SelectTSql = @" SELECT Description, Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; Image photo; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(SelectTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); desc = rdr.GetSqlString(0).Value; serverPath = rdr.GetSqlString(1).Value; serverTxn = rdr.GetSqlBinary(2).Value; rdr.Close(); } } photo = LoadPhotoImage(serverPath, serverTxn); } ts.Complete(); } return photo; } private static Image LoadPhotoImage(string filePath, byte[] txnToken) { Image photo; using (SqlFileStream sfs = new SqlFileStream(filePath, txnToken, FileAccess.Read))

340  Part II  Going Beyond Relational

{ photo = Image.FromStream(sfs); sfs.Close(); } return photo; } #endregion } }

Let’s exp a n th s code n deta We’ start at the top w th some requ red namespace nc us ons The two using statements to take not ce of are System.Data.SqlTypes and System.Transactions The System.Data.SqlTypes namespace defines the SqlFileStream c ass that you’ be us ng to stream BLOBs No spec a assemb y reference s requ red to use th s c ass, because t s prov ded by the System.Data. dll assemb y that your project s a ready referenc ng (V sua Stud o set th s reference automat ca y when t created your project) The System.Transactions namespace defines the TransactionScope c ass that ets you code mp c t transact ons aga nst the database Th s c ass s prov ded by the System. Transactions.dll assemb y, wh ch s not referenced automat ca y You’ need to add a reference to t now, or the code w not comp e R ght-c ck the project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, and scro to find the System.Transactions component Then doub e-c ck t to add the reference

More Info  Chapter 10 provides detailed coverage of the conventional ADO.NET objects for SQL Server (SqlConnection, SqlCommand, SqlDataReader, and so on). There, we discuss explicit transactions (using the ADO.NET SqlTransaction object) versus implicit transactions (using TransactionScope). Although the former will also work with SqlFileStream, Chapter 10 explains why implicit transactions are preferred. A connect on str ng s defined at the top of the code as a hard-coded constant named ConnStr Th s s just for demonstrat on purposes; a rea -wor d app cat on wou d store the connect on str ng e sewhere (such as n a configurat on fi e, poss b y encrypted), but we’re keep ng the examp e s mp e

Important  We must reiterate that this code takes a minimalist approach for proof-ofconcept purposes only. It is not representative of best coding practices, and has no error handling. Although the using constructs in this code ensure that all objects that allocate unmanaged resources (such as database connections and file handles) are disposed of properly even if an exception occurs, your production applications must implement a more robust strategy that includes the use of try/catch/finally blocks with a mechanism for error logging and recovery.



Chapter 8  Nat ve F e Stream ng   341

The first method defined n the c ass s InsertPhoto, wh ch accepts a new photo nteger ID, str ng descr pt on, and fu path to an mage fi e to be saved to the database, as shown here public static void InsertPhoto(int photoId, string desc, string filename) { const string InsertTSql = @" INSERT INTO PhotoAlbum(PhotoId, Description) VALUES(@PhotoId, @Description); SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(InsertTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; cmd.Parameters.Add("@Description", SqlDbType.VarChar).Value = desc; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior.SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } SavePhotoFile(filename, serverPath, serverTxn); } ts.Complete(); } }

Not ce that the InsertTSql str ng constant defined at the top of the method spec fies an INSERT s­ tatement that nc udes the PhotoId and Description co umns, but not the actua Photo BLOB co umn ­Instead, the INSERT statement s fo owed mmed ate y by a SELECT statement that retr eves two p eces of nformat on you’ use to stream the BLOB nto the Photo co umn much more effic ent y than us ng ­ord nary T-SQL—name y a og ca UNC pathname to the fi e and the transact ona context token You’ reca from the d scuss on ear er that those are the two va ues needed to use SqlFileStream, and you’re about to see how exact y But a you’ve done so far s define a constant ho d ng two T-SQL statements The constant s fo owed by two var ab e dec arat ons, serverPath and serverTxn; these var ab es w rece ve the two spec a va ues when you execute those T-SQL statements ater n the code The method then creates and enters a new TransactionScope b ock Th s does not actua y beg n the ­ atabase transact on (you’ve not even connected to the database yet), but rather dec ares that a data d access w th n the b ock (and n any code ca ed from w th n the b ock) must part c pate n a ­database ­transact on Ins de the TransactionScope b ock, the code creates and opens a new S­ qlConnection ­Be ng 342  Part II  Going Beyond Relational

the first data access code ns de the TransactionScope b ock, th s a so mp c t y beg ns the database ­transact on Next, t creates a SqlCommand object assoc ated w th the open ­connect on and prepares ts command text to conta n your T-SQL statements (the INSERT fo owed by the SELECT) Invok ng the ExecuteReader method executes the T-SQL statements and returns a reader from wh ch you can retr eve the va ues returned by the SELECT statement The transact on s st pend ng at th s t me The INSERT statement does not prov de a va ue for RowId and nstead a ows SQL Server to automat ca y generate and ass gn a new uniqueidentifier ROWGUID va ue by defau t just ke before when you used T-SQL to nsert the first two rows We’ve a so po nted out that no va ue s prov ded for the Photo co umn—and th s s exact y how the defau t 0x va ue that you defined ear er for the Photo co umn comes nto p ay (we sa d we’d come back to t, and here you are) A though the row has been added by the INSERT statement, t w ro back (d sappear) f a ­ rob em occurs before the transact on s comm tted Because you d dn’t prov de a BLOB va ue for p the Photo co umn n the new row, SQL Server honors the defau t va ue 0x that you estab shed for t n the ­CREATE TABLE statement for PhotoAlbum Th s represents a zero- ength b nary stream, wh ch s c­ omp ete y d fferent than NULL Be ng a varbinary(max) co umn decorated w th the FILESTREAM attr bute, an empty fi e gets created n the fi e system that SQL Server assoc ates w th the new row At the same t me, SQL Server n t ates an NTFS fi e system transact on over th s new empty fi e and synchron zes t w th the database transact on So just ke the new row, the new fi e w d sappear f the database transact on does not comm t successfu y

Important  You cannot use SqlFileStream to populate BLOBs against NULL values in a new row’s varbinary(max) FILESTREAM column. You must always add new rows with a zero-length binary stream (0x) in the BLOB column, either by making it the default value as in this example, or by specifying it explicitly in your INSERT statement. This will result in the creation of a z­ ero-length file that can be streamed to (overwritten) by calling SqlFileStream, as described here. Immed ate y fo ow ng the INSERT statement, the SELECT statement returns Photo.PathName and GET FILESTREAM TRANSACTION CONTEXT What you’re essent a y do ng w th the WHERE c ause n th s SELECT statement s read ng back the same row you have just added (but not yet comm tted) to the PhotoAlbum tab e n order to reference the BLOB stored n the new fi e that was just created (a so not yet comm tted) n the fi e system The va ue returned by Photo.PathName s a fabr cated path to the BLOB for the se ected PhotoId The path s expressed n UNC format, and po nts to the network share name estab shed for the server nstance when you first enab ed FILESTREAM (wh ch s MSSQLSERVER n th s case, as shown n F gure 8-1) It s not a path to the fi e’s phys ca ocat on on the server, but rather conta ns nformat on SQL Server can use to der ve the fi e’s phys ca ocat on For examp e, you’ not ce that t a ways conta ns the GUID va ue n the uniqueidentifier ROWGUIDCOL co umn of the BLOB’s correspond ng row You retr eve the path va ue from the reader’s first co umn and store t n the serverPath str ng var ab e



Chapter 8  Nat ve F e Stream ng   343

Note  You’re able to read back the row you just added because your code is running ­inside the same transaction. However, with the default (and recommended) SQL Server ­transaction isolation mode set to Read Committed, no other users will be able to see this row because it has not yet been committed. That would be what is often referred to as a “dirty read,” because the data will never actually come into existence if the code fails to ­commit the pending transaction. To prevent dirty reads, SQL Server imposes a lock on the page holding the uncommitted row. It’s therefore very important for your transactions to be committed (or rolled back) as soon as possible after they are started. Any code that doesn’t absolutely need to run inside a transaction should run outside. We just exp a ned how SQL Server n t ated an NTFS fi e system transact on over the FILESTREAM data n the new row’s Photo co umn when you started the database transact on The GET FILESTREAM TRANSACTION CONTEXT funct on returns a hand e to that NTFS transact on ( f you’re not ns de a transact on, th s funct on w return NULL and your code won’t work) You obta n the transact on context, wh ch s returned by the reader’s second co umn as a SqlBinary va ue, and store t n the byte array named serverTxn Armed w th the BLOB path reference n serverPath and the transact on context n serverTxn, you have what you need to create a SqlFileStream object and perform d rect fi e access to stream the mage nto the Photo co umn You c ose the reader, term nate ts using b ock, then term nate the ­enc os ng using b ock for the SqlConnection as we Th s wou d norma y c ose the database ­connect on mp c t y, but that gets deferred n th s case because the code s st nested ns de the outer using b ock for the TransactionScope object So the connect on s st open at th s t me, and the transact on s st pend ng It s prec se y at th s po nt that you ca the SavePhotoFile method to stream the spec fied mage fi e nto the Photo co umn of the new y nserted PhotoAlbum row, overwr t ng the empty fi e just created by defau t When contro returns from SavePhotoFile, the TransactionScope object’s Complete method s nvoked and ts using b ock s term nated, s gna ng the transact on management API that everyth ng worked as expected Th s mp c t y comm ts the database transact on (wh ch n turn comm ts the NTFS fi e system transact on) and c oses the database connect on The SavePhotoFile method reads from the source fi e and wr tes to the database FILESTREAM ­storage n 512 KB chunks at a t me us ng ord nary NET stream ng techn ques, as shown here private static void SavePhotoFile (string clientPath, string serverPath, byte[] serverTxn) { const int BlockSize = 1024 * 512; using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read)) { using (SqlFileStream dest = new SqlFileStream(serverPath, serverTxn, FileAccess.Write)) { byte[] buffer = new byte[BlockSize];

344  Part II  Going Beyond Relational

int bytesRead; while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) { dest.Write(buffer, 0, bytesRead); dest.Flush(); } dest.Close(); } source.Close(); } }

The method beg ns by defin ng a BlockSize nteger constant that s set to a reasonab e va ue of 512 KB P cture fi es arger than th s w be streamed to the server n 512 KB b ocks at a t me The oca source mage fi e ( n clientPath) s then opened on an ord nary read-on y FileStream object Then the dest nat on fi e s opened by pass ng the two spec a va ues (serverPath and serverTxn), a ong w th a FileAccess.Write enumerat on request ng wr te access, nto the SqlFileStream constructor L ke the source FileStream object, SqlFileStream nher ts from System.IO.Stream, so t can be treated just ke any ord nary stream Thus, you atta n wr te access to the dest nat on BLOB on the ­database server’s NTFS fi e system Remember that th s output fi e s en sted n an NTFS transact on and ­noth ng you stream to t w be permanent y saved unt the database transact on s comm tted by the term nat ng TransactionScope b ock, after SavePhotoFile comp etes The rest of the SavePhotoFile method mp ements a s mp e oop that reads from the source FileStream and wr tes to the dest nat on SqlFileStream, one 512 KB b ock at a t me unt the ent re source fi e s processed, and then t c oses both streams That covers nsert ng new photos The rest of the code conta ns methods to retr eve ex st ng ­ hotos and stream the r content from the fi e system nto an Image object for d sp ay You’ find that p th s code fo ows the same pattern as the ast, on y now you’re perform ng read access The SelectPhoto method accepts a photo ID and returns the str ng descr pt on from the database n an output parameter The actua BLOB tse f s returned as the method’s return va ue n a System. Drawing.Image object You popu ate the Image object w th the BLOB by stream ng nto t from the database server’s NTFS fi e system us ng SqlFileStream, as shown here public static Image SelectPhoto(int photoId, out string desc) { const string SelectTSql = @" SELECT Description, Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; Image photo; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open();



Chapter 8  Nat ve F e Stream ng   345

using (SqlCommand cmd = new SqlCommand(SelectTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior.SingleRow)) { rdr.Read(); desc = rdr.GetSqlString(0).Value; serverPath = rdr.GetSqlString(1).Value; serverTxn = rdr.GetSqlBinary(2).Value; rdr.Close(); } } photo = LoadPhotoImage(serverPath, serverTxn); } ts.Complete(); } return photo; }

Once aga n, you start th ngs off by enter ng a TransactionScope b ock and open ng a c­ onnect on You then execute a s mp e SELECT statement that quer es the PhotoAlbum tab e for the record spec fied by the photo ID and returns the descr pt on and fu path to the mage BLOB, as we as the FILESTREAM transact ona context token And once aga n you use the pathname and transact ona context w th SqlFileStream to t e nto the server’s fi e system n the LoadPhotoImage method, as shown here private static Image LoadPhotoImage(string serverPath, byte[] serverTxn) { Image photo; using (SqlFileStream sfs = new SqlFileStream(serverPath, serverTxn, FileAccess.Read)) { photo = Image.FromStream(sfs); sfs.Close(); } return photo; }

Just as when you were nsert ng new photos (on y th s t me us ng FileAccess.Read nstead of ­FileAccess.ReadWrite), you create a new SqlFileStream object from the og ca pathname and ­transact on context You then pu the BLOB content d rect y from the NTFS fi e system on the server nto a new System.Drawing.Image object us ng the stat c Image.FromStream method aga nst the SqlFileStream object The popu ated mage s then passed back up to the form, where t s d sp ayed us ng the Image property of the PictureBox contro It’s t me to see a of th s n act on and g ve the app cat on a run! To nsert a new photo, spec fy a un que photo ID (you’ve a ready used 1 and 2 a ready), an mage fi e, and a descr pt on n the top group box n the PhotoForm w ndow, as shown n F gure 8-9, and then c ck Save 346  Part II  Going Beyond Relational

Figure 8-9  nsert ng a new photo nto F LESTREAM storage.

To se ect and d sp ay the photo and ts descr pt on back from the database, type ts photo ID n the bottom group box, and then c ck Load The photo s d sp ayed, as shown n F gure 8-10

Figure 8-10  Retr ev ng a photo from F LESTREAM storage.

Th s app cat on s sma , but t demonstrates everyth ng needed to everage the power of FILESTREAM n your NET c ent app cat ons The code that you need to wr te fo ows a fa r y ­stra ghtforward pattern that adapts eas y to d fferent scenar os In the next FILESTREAM app cat on, you’ stream content from a Hypertext Transfer Protoco (HTTP) serv ce and consume t n a W ndows Presentat on Foundat on (WPF) c ent us ng the very same FILESTREAM pr nc p es that you app ed n th s W ndows Forms app cat on



Chapter 8  Nat ve F e Stream ng   347

Creating a Streaming HTTP Service You’ now bu d a very s mp e serv ce to de ver photos over HTTP It w be mp emented as a norma M crosoft ASP NET Web App cat on project w th a s ng e PhotoServ ce aspx page Th s page can be ca ed by any HTTP c ent that passes n a photo ID va ue appended to the URL query str ng, and t w stream back the b nary content for the spec fied photo from the database FILESTREAM storage n SQL Server to the c ent

More Info  We use an ordinary ASP.NET page rather than a true Windows Communications Foundation (WCF) service in this example, because the ASP.NET Response object has ­built-in streaming capabilities that are relatively easy to code against. Conversely, WCF data contracts are buffered entirely in memory before sending and receiving by design, which does not work well for potentially large byte streams (such as BLOBs being served up by FILESTREAM). Two readily available techniques can be leveraged to create WCF services that stream BLOBs. First, the Message Transmission Optimization Mechanism (MTOM) protocol allows WCF data contracts to be streamed rather than buffered, using an approach very similar to the way SMTP transmits embedded email attachments. Your second option is to use WCF Data Services and implement a streaming provider, which returns BLOB data as a binary media resource separate from the normal text-based service response feed. To bu d the serv ce, fo ow these steps Start V sua Stud o and choose F e New Project Create a M crosoft V sua C# ASP NET Web App cat on project named PhotoLibraryHttpService n a so ut on named PhotoLibraryFileStreamDemo, as shown n F gure 8-11

Figure 8-11  Creat ng the stream ng HTTP serv ce project.

348  Part II  Going Beyond Relational

Add a new Web Form named PhotoService.aspx Un ke a typ ca aspx page, th s page w not return HTML content Instead, the page’s code-beh nd c ass w stream out b nary content from the database d rect y through the Response object bu t n to ASP NET Therefore, de ete a the HTML markup, eav ng on y the d rect ve at the very top that nks the aspx page w th ts ­code-beh nd c ass Now make th s the defau t startup page by r ght-c ck ng PhotoService.aspx n So ut on Exp orer and then choos ng Set As Start Page Next, sw tch to the code-beh nd c ass fi e by r ght-c ck ng aga n on the PhotoService.aspx node n So ut on Exp orer and then choos ng V ew Code Rep ace the starter code prov ded by V sua Stud o w th the code shown n L st ng 8-5 Listing 8-5  mp ement ng code for the stream ng photo serv ce.

using using using using using using using

System; System.Data; System.Data.SqlClient; System.Data.SqlTypes; System.IO; System.Transactions; System.Web.UI;

namespace PhotoLibraryHttpService { public partial class PhotoService : Page { private const string ConnStr = "Data Source=.;Integrated Security=True;Initial Catalog=PhotoLibrary;"; protected void Page_Load(object sender, EventArgs e) { int photoId = Convert.ToInt32(Request.QueryString["photoId"]); if (photoId == 0) { return; } const string SelectTSql = @" SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); string serverPath; byte[] serverTxn; using (SqlCommand cmd = new SqlCommand(SelectTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId;



Chapter 8  Nat ve F e Stream ng   349

using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior.SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } this.StreamPhotoImage(serverPath, serverTxn); } ts.Complete(); } } private void StreamPhotoImage(string serverPath, byte[] serverTxn) { const int BlockSize = 1024 * 512; const string JpegContentType = "image/jpeg"; using (SqlFileStream sfs = new SqlFileStream(serverPath, serverTxn, FileAccess.Read)) { byte[] buffer = new byte[BlockSize]; int bytesRead; Response.BufferOutput = false; Response.ContentType = JpegContentType; while ((bytesRead = sfs.Read(buffer, 0, buffer.Length)) > 0) { Response.OutputStream.Write(buffer, 0, bytesRead); Response.Flush(); } sfs.Close(); } } } }

Th s code bears a strong resemb ance to the photo retr eva code n the ear er W ndows Forms app cat on Th s nc udes the use of mp c t transact ons w th TransactionScope, so you’ once aga n need to add the appropr ate project reference R ght-c ck the project n So ut on Exp orer, choose Add Reference, c ck the NET tab, and then doub e-c ck the System.Transactions component The Page Load method first retr eves the photo ID passed n v a the photoId query str ng va ue If no va ue s passed, the method returns w thout stream ng anyth ng back (A ternat ve y, you cou d stream back an “ mage not found” mage ) Otherw se, as before, the photo fi e name and transact on context s obta ned after estab sh ng a connect on and transact on on the database, and nvok ng a SELECT statement ca ng the PathName method and GET FILESTREAM TRANSACTION CONTEXT funct on aga nst the photo ID spec fied n the WHERE c ause

350  Part II  Going Beyond Relational

W th these two key p eces of nformat on n hand, the StreamPhotoImage method s ca ed The method beg ns by defin ng a BlockSize nteger constant that s set to a reasonab e va ue of 512 KB (as before, p cture fi es arger than th s w be streamed to the c ent n 512 KB b ocks at a t me) And then once aga n, you use SqlFileStream to read the BLOB data from SQL Server

Note  Because this code is executing under the auspices of a web server, you may also need to grant access to the photo storage directory for the account executing the webpage. This might be ASPNET or NETWORK SERVICE if you’re using Internet Information Services (IIS) or your user account if you’re executing the page using Visual Studio’s development server. Before stream ng the b nary photo content, you need to change two propert es of the Response object In an aspx page, by defau t, the Response object’s BufferOutput property s set to true and the ­ContentType s set to text/html Here, you’ change BufferOutput to false (for true stream ng) and nform the c ent that you’re send ng a JPEG mage by chang ng the ContentType property to image/jpeg Us ng the SqlFileStream object, the code then reads from the database FILESTREAM storage n 512 KB chunks and streams to the c ent us ng the Reponse.OutputStream.Write and Response.Flush ­methods Th s s mp emented w th a s mp e oop that reads content from the SqlFileStream and sends t to the c ent v a the Response object unt the ent re fi e s processed Th s comp etes the serv ce app cat on Before mov ng on to bu d the WPF c ent, first test the serv ce Press F5 to start the app cat on When Internet Exp orer aunches PhotoService.aspx, t d sp ays an empty page because no photo ID s present n the URL’s query str ng In the Address bar, append ?photoId=3 to the URL and re oad the page The code-beh nd c ass retr eves photo ID 2 from the database and streams t back for d sp ay n the browser, as shown n F gure 8-12

Figure 8-12  Stream ng a photo over HTTP to nternet Exp orer.



Chapter 8  Nat ve F e Stream ng   351

You’ve created a funct on ng HTTP serv ce app cat on that streams p ctures from the database to any HTTP c ent It’s now ncred b y easy to bu d a sma WPF c ent app cat on that ca s the serv ce and d sp ays photos A that’s needed s the proper URL w th the des red photo ID ­spec fied n the query str ng, as you’ve just seen You’re us ng the ASP NET Deve opment Server prov ded by V ­ sua Stud o, wh ch by defau t random y ass gns a port number on localhost (port 1157 was ­ass gned th s t me, as nd cated n F gure 8-12) You’ need to estab sh a fixed port number ­nstead so that your WPF c ent can re ab y construct a URL for ca ng the serv ce Any unused port ­number w suffice, and so just p ck 22111 for th s app cat on To set the port number, r ght-c ck the ­PhotoLibraryHttpService project n So ut on Exp orer, and then choose Propert es Se ect the Web tab, se ect the Spec fic Port opt on, and then type 22111 for the port number, as shown n F gure 8-13

Figure 8-13  Sett ng a spec fic port number for the HTTP serv ce app cat on.

Building a WPF Client To bu d the WPF c ent, fo ow these steps In V sua Stud o choose F e New Project Create a new V sua C# WPF App cat on project named PhotoLibraryWpfClient Be sure to se ect Add To So ut on n the So ut on drop-down st, as shown n F gure 8-14, so that the project s added to the same ­PhotoLibraryFileStreamDemo so ut on that conta ns PhotoLibraryHttpService

352  Part II  Going Beyond Relational

Figure 8-14  Creat ng the stream ng WPF c ent app cat on.

V sua Stud o creates a new WPF app cat on w th a s ng e w ndow name Ma nW ndow xam , and then opens the w ndow des gner Rep ace the w ndow’s markup n the XAML w ndow w th the code shown n L st ng 8-6 Listing 8-6  XAML markup for WPF photo c ent.



Photo ID

Download Photo





Th s s s mp e markup that mere y defines Label, TextBox, Button, and MediaElement contro s for the w ndow, as shown n F gure 8-15



Chapter 8  Nat ve F e Stream ng   353

Figure 8-15  A s mp e F LESTREAM WPF U w ndow.

The MediaElement contro n WPF s a sca ed-down med a p ayer that s capab e of render ng a var ety of mu t med a types, nc ud ng mages and v deo, from any source A you need to do s set ts Source property to a URL that t can stream ts content from Doub e-c ck the Button contro , and then nsert the fo ow ng code n the button’s event hand er private void btnDownload_Click(object sender, RoutedEventArgs e) { string url = "http://localhost:22111/PhotoService.aspx?photoId=" + this.txtPhotoId.Text; this.mediaElement1.Source = new Uri(url); }

Th s code s mp y constructs a URL to the PhotoServ ce aspx page that s known to be runn ng on localhost port 22111, pass ng the des red photo ID n the query str ng When the MediaElement contro ’s Source property s set to that URL, the contro automat ca y ca s the serv ce and renders the photo that s streamed from the database to the serv ce and then from the serv ce to the WPF c ent over HTTP To see t work, run the app cat on, and request photo ID 2 for d sp ay, as shown n F gure 8-16 354  Part II  Going Beyond Relational

Figure 8-16  Stream ng a photo from the database over HTTP to a WPF c ent app cat on.

FILESTREAM Limitations and Considerations From a programm ng standpo nt, FILESTREAM offers a comp ete abstract on over the varbinary(max) data type Thus, app cat ons and frameworks ( nc ud ng Ent ty Framework) are unab e to d ­ st ngu sh between varbinary(max) co umns w th the FILESTREAM attr bute that are stored as fi es n the fi e system, and varbinary(max) co umns w thout the FILESTREAM attr bute that are stored ns de the ­re at ona tab es Th s prov des a h gh eve of compat b ty between FILESTREAM and ex st ng ­app cat ons that work w th varbinary(max) data However, FILESTREAM’s nterna ntegrat on w th the NTFS fi e system does create un que cha enges that affect compat b ty w th severa other SQL Server features It’s very mportant that you understand these compat b ty ssues before you start to use FILESTREAM, so that you can avo d unp easant p tfa s further down the road The essent a cons derat ons for FILESTREAM are summar zed be ow ■

M rror ng/HADR FILESTREAM does not work w th m rror ng Pr or to SQL Server 2012, th s was FILESTREAM’s most egreg ous m tat on However, FILESTREAM is compat b e w th the new H gh-Ava ab ty D saster Recovery (HADR, a so known as “a ways on”) feature n SQL Server 2012 HADR offers many mprovements over the m rror ng techno ogy t rep aces, and fortunate y, FILESTREAM compat b ty s one of them

■

Transparent Data Encrypt on (TDE) TDE (covered n Chapter 5) w the NTFS fi e system

■

not encrypt varbinary(max) FILESTREAM co umn data stored n

Rep cat on FILESTREAM can be used w th both transact ona rep cat on and merge rep cat on, w th ­severa restr ct ons and cons derat ons A subscr bers must be runn ng SQL Server 2008 or h gher For merge rep cat on, SQL Server ut zes the same uniqueidentifier ­ROWGUIDCOL



Chapter 8  Nat ve F e Stream ng   355

co umn that FILESTREAM uses (there can be on y one such co umn n a tab e, and both FILESTREAM and merge rep cat on requ re t) For th s co umn to be compat b e between both features, be sure to spec fy NEWSEQUENTIALID as ts defau t va ue n your tab e Refer to Books On ne for more nformat on about us ng rep cat on w th FILESTREAM ■

Log Sh pp ng Log sh pp ng fu y supports FILESTREAM, as ong as both the pr mary and secondary servers are runn ng SQL Server 2008 or h gher

■

Fu -Text Search (FTS) The FTS eng ne ndexes FILESTREAM co umns just as ord nary varbinary(max) co umns It s not on y 100% compat b e w th FILESTREAM, but actua y has greater s gn ficance n SQL Server 2012 w th the ntroduct on of F eTab e (see the sect on “Search ng Documents” at the end of th s chapter)

■

Database Snapshots SQL Server 2005 ntroduced database snapshots, a feature that ets you create a stat c v ew of the database that can be used for report ng or ro back purposes Database snapshots are not supported for FILESTREAM fi egroups However, you can st create a database snapshot of the standard fi egroups n the database (the FILESTREAM fi egroups w be marked as offl ne n those snapshots)

■

Snapshot Iso at on Leve SQL Server 2005 a so ntroduced snapshot so at on eve for transact ons (we cover snapshot so at on n Chapter 4) When FILESTREAM was first ntroduced w th SQL Server 2008, t was ncompat b e w th snapshot so at on As w th m rror ng, th s m tat on prevented many peop e from adopt ng FILESTREAM Fortunate y, th s m tat on was removed n SQL Server 2008 R2, and FILESTREAM can now be used w th snapshot so at on

■

Loca NTFS F e System FILESTREAM data can on y by stored on oca NTFS d sk vo umes However, remote BLOB ­storage (RBS) so ut ons ava ab e from M crosoft and th rd-party vendors w a ow you to ­configure a FILESTREAM-enab ed SQL Server database as a ded cated BLOB store for ­SharePo nt and other app cat ons

■

Secur ty To support SqlFileStream, the SQL Server nstance shou d be configured to use m xed-mode ( ntegrated) secur ty

■

SQL Server Express ed t on FILESTREAM s fu y supported by the free Express ed t on of SQL Server (though notab y, t s not supported by the LocalDB nstance of SQL Server supp ed w th SSDT) Furthermore, the

356  Part II  Going Beyond Relational

database s ze m t (4 GB n SQL Server 2008, or 10 GB n SQL Server 2008 R2 and 2012) does not nc ude varbinary(max) FILESTREAM co umns stored n the fi e system

Introducing FileTable The new F eTab e feature n SQL Server 2012 bu ds on FILESTREAM F eTab e comb nes FILESTREAM w th hierarchyid and the W ndows fi e system API to de ver exc t ng new BLOB capab t es n SQL Server As mp ed by the two words jo ned together n ts name, one F eTab e funct ons as two ­d st nct th ngs s mu taneous y 1. A F eTab e s an Ord nary Tab e 2. A F eTab e s an Ord nary F e System

F rst and foremost, a F eTab e s a regu ar SQL Server database tab e n every respect, w th one except on The schema of a F eTab e s fixed The co umns of a F eTab e and the r data types are ­pre-determ ned by SQL Server Spec fica y, every F eTab e conta ns the co umns shown n Tab e 8-1 Table 8-1  F xed F eTab e co umns Column Name

Data Type

Description

stream id

uniqueidentifier ROWGUIDCOL

Un que row dent fier

file stream

varbinary(max) FILESTREAM

BLOB content (NULL f d rectory)

name

varchar(255)

Name of fi e or d rectory

path locator

hierarchyid

Locat on of fi e or d rectory w th n the og ca fi e system

creation time

datetimeoffset(7)

Created

last write time

datetimeoffset(7)

Last mod fied

last access time

datetimeoffset(7)

Last accessed

is directory

bit

0=fi e, 1=d rectory

is is is is is is

bit

Storage attr butes

offline hidden readonly archive system temporary

Every F eTab e row represents an ord nary fi e or fo der n a fi e system whose og ca fo der s­ tructure s mp emented us ng the path locator co umn Th s s a hierarchyid va ue that denotes the ocat on of each fi e and fo der (row) w th n the og ca fi e system (tab e) The hierarchyid data type was ntroduced n SQL Server 2008 as a b nary va ue that, re at ve to other hierarchyid va ues n the same tree structure, dent fies a un que node pos t on It s mp emented as a system common ­anguage runt me (CLR) type, wh ch means that t’s a NET framework c ass, and has a set of ­methods you can use (such as GetAncestor, GetDescendant, GetReparentedValue, and IsDescendantOf ) to

Chapter 8  Nat ve F e Stream ng   357

traverse and man pu ate the h erarchy Thus, t’s perfect for cast ng the h erarch ca structure of a fi e system over a re at ona tab e, wh ch s prec se y what F eTab e does

More Info  A proper understanding of hierarchyid is required to query and manipulate a FileTable’s path locator column with T-SQL. Chapter 7 provides the necessary coverage. The path locator co umn s defined as the tab e’s pr mary key (w th a non-c ustered ndex) A separate key va ue s a so stored n the stream id co umn (w th a non-c ustered un que ndex) Th s s the uniqueidentifier ROWGUIDCOL va ue requ red by any tab e w th varbinary(max) FILESTREAM co umns, so F eTab e s no except on Un ke path locator, th s un que va ue w never change once t s ass gned to a new F eTab e row, even f the row s ater “reparented” (that s, ass gned to another ­ocat on n the h erarchy) You can use e ther path locator or stream id to un que y dent fy each row n a F eTab e But because the path locator va ue changes whenever the fo der or fi e represented by the row s “moved” n the fi e system, you shou d use the stream id va ue to store ong-term ­(permanent) references to rows n a F eTab e Every row n a F eTab e corresponds to e ther a s ng e fi e or fo der, as determ ned by the bit va ue n the is directory co umn The fi e or fo der name s stored n the name co umn as an nvarchar(255) str ng A of the other co umn names are se f-descr b ng, and are used to store typ ca fi e system nformat on such as var ous t mestamps and storage attr butes Rows represent ng d rector es have an IsDirectory co umn va ue of 1 (for true) D rectory rows have no BLOB content, and thus a ways conta n NULL n the file stream co umn Rows represent ng fi es have an IsDirectory va ue of 0 (for fa se), and the actua fi e content (the BLOB tse f) s stored n the file stream co umn Th s s a varbinary(max) data type decorated w th the FILESTREAM attr bute, wh ch means that the b nary content n the file stream co umn s stored n the NTFS fi e system that SQL Server s manag ng beh nd the scenes, rather than the structured fi egroups where a the other tab e data s stored ( n other words, standard FILESTREAM behav or, as exp a ned n the first part of th s chapter) In add t on to these 14 co umns, each F eTab e nc udes the three computed (read-on y) co umns shown n Tab e 8-2 Table 8-2  Computed F eTab e co umns Column Name

Data Type

Description

parent path locator

hierarchyid

Parent node (der ved from path locator)

file type

nvarchar(255)

F ename extens on der ved from name

cached file size

bigint

BLOB byte ength der ved from file stream

The parent path locator co umn returns the resu t of ca ng GetAncestor(1) on path locator to o ­ bta n the path locator to the parent fo der of any fi e or fo der n the tab e The file type co umn returns the extens on of the fi ename parsed from the str ng va ue n the name co umn And the

358  Part II  Going Beyond Relational

cached file size co umn returns the number of bytes stored n the file stream co umn (that s, the s ze of the BLOB stored beh nd the scenes n the SQL Server nterna y managed NTFS fi e system) W th th s fixed schema n p ace, every F eTab e has what t needs to represent a og ca fi e system Thus, SQL Server s ab e to fabr cate a W ndows fi e share over any F eTab e Th s mmed ate y exposes the F eTab e to any user or app cat on who can then v ew and update the tab e us ng standard fi e I/O semant cs (for examp e, drag-and-drop w th W ndows Exp orer, or programmat ca y w th System.IO.FileStream and the W ndows API) Thus ■

Creat ng a fi e or d rectory n the og ca fi e system adds a row to the tab e

■

Add ng a row to the tab e creates a fi e or d rectory n the og ca fi e system

F gure 8-17 shows the tota F eTab e p cture

Server Mach ne Name

Server nstance F LESTREAM Share Name

Database Name

Windows File I/O F eTab e Name

T-SQL

SqlFileStream

F eTab e Rows

varb nary(max) F LESTREAM

h erarchy d

FileTable ‘Documents’ in Database ‘mydb’ stream id

name

path locator

file stream

...

27D8D4AD-D100-39... ‘ Financials’

0xFF271A3562... NULL

...

78F603CC-0460-73... ‘ReadMe . docx’

0xFF59345688... 0x3B0E956636AE020B...

...

207D4A96-E854-01... ‘Budget . xlsx’

0xFD0011039A... 0xF359000EE30F29A2...

...

SQL Server Structured F e Groups Non BLOB data (pages, rows, co umn)

NTFS F e System Synchron zed database/NTFS transact on

BLOBs (f e contents) n varb nary(max) F LESTREAM co umns

Figure 8-17  F eTab e data can be accessed v a T SQL, SqlFileStream, or the W ndows fi e system.

Take a moment to d gest what’s happen ng here In add t on to the programmat c FILESTREAM access us ng T-SQL or SqlFileStream, SQL Server 2012 now offers a th rd nterface to FILESTREAM a

Chapter 8  Nat ve F e Stream ng   359

og ca fi e system In a sense, th s fi s the FILESTREAM gap n wh ch the fi e system tse f s comp ete y naccess b e Th s s not to say that F eTab e ets you d rect y access SQL Server’s nterna y managed NTFS fi e system; certa n y not That rema ns obfuscated and pr vate, as t cont nues to funct on n standard FILESTREAM fash on aga nst BLOB data that just happens to be n a F eTab e nstead of any other tab e What you do get s an abstract on ayer over the F eTab e that funct ons as a standard fi e system In th s og ca fi e system, everyth ng about each fi e and fo der—except the BLOB content of the fi es themse ves— s stored n the F eTab e’s structured fi egroup, whereas the BLOBs themse ves are phys ca y stored n the NTFS fi e system as ord nary varbinary(max) FILESTREAM data So you can see that there’s rea y noth ng new beneath the F eTab e ayer; as before, SQL Server synchron zes transact ona access between the row n the F eTab e and ts correspond ng BLOB content n the NTFS fi e system to ensure that ntegr ty s proper y ma nta ned between them As w th T-SQL access, th s synchron zat on occurs mp c t y when man pu at ng the F eTab e v a the exposed W ndows fi e share And, be ng an ord nary tab e n v rtua y every respect, you a so have the opt on to use SqlFileStream w th exp c t transact on synchron zat on for the fastest poss b e stream ng of BLOBs nto and out of a F eTab e A of th s s extreme y appea ng You now have tota flex b ty for BLOB storage n the database W th F eTab e, you can eas y m grate ex st ng app cat ons that work aga nst phys ca fi e systems w thout wr t ng any custom T-SQL or fancy SqlFileStream code Just use a F eTab e, et the ex st ng app cat ons cont nue work ng w thout mod ficat on, and enjoy the benefits of your fi es becom ng an ntegra part of the SQL Server database

Creating a FileTable F eTab e re es on FILESTREAM Therefore, FILESTREAM must be enab ed before you can create a ­F eTab e S m ar y, a F eTab e-enab ed database must be a FILESTREAM-enab ed database created w th a proper y defined FILEGROUP…CONTAINS FILESTREAM c ause In add t on, you must prov de a WITH FILESTREAM c ause n your CREATE DATABASE statement (or, f F eTab e-enab ng an ex st ng database, you must prov de a SET FILESTREAM c ause n your ALTER DATABASE statement) In th s c ause, use DIRECTORY NAME to spec fy the fo der name for th s database Th s fo der w appear n the root d rectory of the fi e share assoc ated w th and exposed by the SQL Server nstance Second, spec fy NON TRANSACTED ACCESS=FULL to enab e non-transacted access Th s exposes every ­F eTab e n the database as a subfo der beneath the database fo der spec fied by DIRECTORY NAME Do not et the non-transacted access sett ng confuse you Th s just a ows F eTab e data to be ­exposed v a the W ndows API, whose operat ons are non-transact ona n nature Interna y, ­transact ons are a ways used to synchron ze varbinary(max) FILESTREAM data between the database and NTFS fi e system—e ther mp c t y (whether us ng T-SQL or the W ndows API) or exp c t y ( f ­us ng SqlFileStream)—as ustrated n F gure 8-17 L st ng 8-7 demonstrates creat ng a F eTab e-enab ed database w th a mod fied vers on of the ­CREATE DATABASE statement you used n L st ng 8-1 The WITH FILESTREAM c ause n th s ­statement enab es ­F eTab e for the PhotoFileLibrary database, and exposes a PhotoFileLibrary d rectory for the database beneath the W ndows fi e share for the SQL Server nstance (MSSQLSERVER n our examp es) 360  Part II  Going Beyond Relational

Listing 8-7  Creat ng a F eTab e enab ed database.

CREATE DATABASE PhotoFileLibrary ON PRIMARY (NAME = PhotoFileLibrary_data, FILENAME = 'C:\Demo\PhotoLibrary\PhotoFileLibrary_data.mdf'), FILEGROUP FileStreamGroup CONTAINS FILESTREAM (NAME = PhotoFileLibrary_blobs, FILENAME = 'C:\Demo\PhotoLibrary\PhotoFiles') LOG ON (NAME = PhotoFileLibrary_log, FILENAME = 'C:\Demo\PhotoLibrary\PhotoFileLibrary_log.ldf') WITH FILESTREAM (DIRECTORY_NAME='PhotoFileLibrary', NON_TRANSACTED_ACCESS=FULL)

Creat ng the actua F eTab e s the eas est part Because SQL Server contro s the schema of every F eTab e, you just use a CREATE TABLE statement w th the new AS FileTable c ause and don’t nc ude any co umns, as shown n L st ng 8-8 Listing 8-8  Creat ng a F eTab e.

USE PhotoFileLibrary GO CREATE TABLE PhotoFiles AS FileTable GO

The PhotoFiles F eTab e s ready to use You w find a root PhotoFiles fo der for the F eTab e beneath the PhotoLibrary fo der created for the database n the W ndows fi e share for the server nstance, as shown n F gure 8-18

Figure 8-18  A W ndows Exp orer w ndow open to the fi e share exposed by a F eTab e.



Chapter 8  Nat ve F e Stream ng   361

Manipulating a FileTable As w th any FILESTREAM-enab ed tab e, you can nteract w th the F eTab e us ng the same T-SQL and SqlFileStream techn ques covered ear er n th s chapter In add t on, the F eTab e can be accessed by users and app cat ons v a the og ca fi e system exposed by the W ndows fi e share To demonstrate, run the code n L st ng 8-9 Th s code nserts rows nto the F eTab e to create a fo der w th two fi es ns de of t Listing 8-9  Us ng T SQL to create fo ders and fi es n a F eTab e.

-- Get root pathnames for the database and FileTable DECLARE @DbRootPath varchar(max) = FILETABLEROOTPATH() -- \\machine\instance\db DECLARE @TableRootPath varchar(max) = @DbRootPath + '\PhotoFiles' -- Create folder 'Mountains' in the FileTable's root DECLARE @TableRootNode hierarchyid = GETPATHLOCATOR(@TableRootPath) INSERT INTO PhotoFiles(name, path_locator, is_directory) VALUES ('Mountains', @TableRootNode.GetDescendant(NULL, NULL), 1) -- Get new folder's hierarchyid DECLARE @FolderPath varchar(max) = @TableRootPath + '\Mountains' DECLARE @FolderNode hierarchyid = GETPATHLOCATOR(@FolderPath) -- Add a text file to the folder DECLARE @TextFileNode hierarchyid = @FolderNode.GetDescendant(NULL, NULL) INSERT INTO PhotoFiles(file_stream, name, path_locator) VALUES( CONVERT(varbinary(max), 'This folder contains pictures of mountains.'), 'ReadMe.txt', @TextFileNode) -- Add an image file to the folder DECLARE @ImageFileNode hierarchyid = @FolderNode.GetDescendant(@TextFileNode, NULL) INSERT INTO PhotoFiles(file_stream, name, path_locator) VALUES( (SELECT BulkColumn FROM OPENROWSET(BULK 'C:\Demo\Ascent.jpg', SINGLE_BLOB) AS x), 'Ascent.jpg', @ImageFileNode)

Th s code first ca s the FILETABLEROOTPATH funct on to ass gn the root pathname for ­current ­database’s F eTab e share nto @DbRootPath Th s name s based on the mach ne name, SQL Server nstance share name, and database name You shou d a ways use FILETABLEROOTPATH to ­obta n th s nformat on, because hard-cod ng the mach ne, nstance, and database names w prevent your code from runn ng n d fferent env ronments For examp e, assum ng the PhotoFileLibrary ­database s runn ng on a mach ne named SQL2012DEV under an nstance named MSSQLSERVER, the ­FILETABLEROOTPATH funct on w return \\SQL2012DEV\MSSQLSERVER\PhotoFileLibrary The next ne of code concatenates the database root pathname w th \PhotoFiles (the name you ­ ss gned to the new F eTab e) to obta n the root pathname for the F eTab e, and ass gns t nto a @TableRootPath Th s s the parent for the new Mounta ns fo der you are about to create Node ­man pu at on n the F eTab e tree structure a ways nvo ves the hierarchyid va ue stored n the path locator co umn To get the hierarchyid va ue of any fo der or fi e n the F eTab e, you can ca the 362  Part II  Going Beyond Relational

GETPATHLOCATOR funct on and pass t the fu pathname to the des red fo der or fi e In L st ng 8-9, GETPATHLOCATOR obta ns the hierarchyid of the F eTab e’s root path and ass gns t nto @TableRootNode In the INSERT statement that fo ows, GetDescendant s nvoked on @TableRootNode to ­generate the appropr ate path locator va ue (a new hierarchyid) for the Mounta ns fo der As exp a ned n Chapter 7, the GetDescendant method s nvoked on a parent’s hierarchyid va ue, and returns a new hierarchyid va ue represent ng a ch d of that parent The method expects two other hierarchyid va ues as nput parameters, wh ch spec fies two ex st ng ch d nodes that the new hierarchyid va ue shou d be “sandw ched” n-between Because there are no ch d nodes yet, NULL s passed for both GetDescendant parameters n the INSERT statement, wh ch generates the proper hierarchyid va ue for the one ch d node n the F eTab e’s root the Mounta ns fo der The INSERT statement ass gns the GetDescendant method’s resu t to the path locator co umn of the new F eTab e row, wh ch dent fies th s row’s pos t on n the h erarchy as the first and on y ch d beneath the F eTab e root It a so ass gns ‘Mounta ns’ to the name co umn and 1 (for true) to the is directory co umn A the other co umns n the new F eTab e row are ass gned the r defau t va ues Th s means that NULL s stored n file stream, the varbinary(max) FILESTREAM co umn that stores the BLOBs for rows n the F eTab e that represent actua fi es Fo ders have no BLOBs, so F eTab e rows represent ng fo ders (that s, where IsDirectory s 1) w a ways conta n NULL n the file stream co umn Next, GETPATHLOCATOR s ca ed aga n, th s t me to obta n the hierarchyid va ue of the new y created Mounta ns fo der Th s hierarchyid va ue w now be used as the parent for two more ­GetDescendant ca s—one for each of the two ch d nodes represent ng the text fi e and mage fi e ­beneath Mounta ns For the first ch d, two NULL va ues are aga n passed to GetDescendant to ­generate a hierarchyid va ue for the text fi e, because as before, th s s the first ch d node ­beneath the parent node (wh ch s the Mounta ns fo der th s t me) The INSERT statement ass gns the ­GetDescendant resu t to the path locator co umn of the new F eTab e row, wh ch dent fies th s row’s pos t on n the h erarchy as the first and on y ch d beneath Mounta ns It a so ass gns ‘ReadMe.txt’ to the name co umn and the fi e’s BLOB content to the file stream co umn The BLOB for th s sma text fi e s easy to n- ne s mp y by convert ng an ord nary str ng to varbinary(max) A other co umns ­assume the r defau t va ues, nc ud ng is directory, wh ch defau ts to 0 (fa se) The mage fi e s created n a very s m ar manner The on y s gn ficant d fference s n the ca to GetDescendant Because a second ch d node s now be ng added to the Mounta ns fo der, the hierarchyid of the ast ex st ng ch d node (current y the text fi e node just added) s spec fied as the second GetDescendant parameter, rather than NULL It’s mportant to understand that th s techn que s requ red to ensure un queness across the ch d nodes beneath a parent; t’s not rea y “pos t on ng” the ch d nodes n any usefu way Pract ca y, you and your users w sort F eTab e data as des red (for examp e, ascend ng by name, or descend ng by creat on date) The th rd INSERT statement a ­ ss gns the GetDescendant resu t to path locator and ‘Ascent.jpg’ to the name co umn The BLOB for the mage fi e s mported nto the file stream co umn from an externa copy of Ascent.jpg us ng the ­OPENROWSET funct on w th ts BULK and SINGLE BLOB opt ons (th s mage fi e s supp ed w th the samp e code for th s chapter on the book’s compan on webs te, or you can just subst tute one of your own mages for the exerc se)



Chapter 8  Nat ve F e Stream ng   363

After runn ng the code n L st ng 8-9, use W ndows Exp orer to nav gate to the Mounta ns fo der n the fi e system exposed by the F eTab e A most ke mag c, W ndows Exp orer shows the fo der and fi es created by the three INSERT statements, as shown n F gure 8-19

Figure 8-19  W ndows Exp orer show ng fo ders and fi es nserted nto a F eTab e by T SQL.

Doub e-c ck Ascent.jpg to v ew the mage us ng W ndows Photo V ewer (the defau t ­app cat on assoc ated w th mage fi es) Beh nd the scenes, SQL Server finds the F eTab e row for the mage fi e, retr eves ts BLOB from the obfuscated fi e system where that row’s varbinary(max) FILESTREAM ­co umn s actua y stored, and streams t n to W ndows Photo V ewer (wh ch you can now c ose) Now doub e-c ck the text fi e to open t n Notepad (the defau t app cat on assoc ated w th text fi es) Instead of open ng as expected, you may be surpr sed to rece ve the error message “The request s not supported” nstead In fact, th s s not a bug, but rather a man festat on of the fact that the fi e system exposed by F eTab e does not support memory-mapped files Th s s a techn que that some app cat ons use to map areas of memory d rect y to the sectors on d sk where a oca fi e s phys ca y stored Thus, the app cat on doesn’t need to “ oad” the fi e nto memory, wh ch opt m zes ­performance Notepad uses memory mapp ng when ed t ng oca fi es, and so t can’t open a fi e d rect y from a F eTab e share on the oca database server Th s w norma y not present a prob em, because users and app cat ons access the F eTab e share remote y (not d rect y on the database server), and fi e memory mapp ng s d sab ed w th remote fi e share access To prove that there’s rea y noth ng wrong w th ReadMe.txt, make a copy of t from the F eTab e share (you can just drag and drop t to your desktop) and then doub e-c ck the copy You’ see that the copy opens n Notepad w thout a prob em You can a so map a dr ve etter to F eTab e share, and then access the fi es through the mapped dr ve etter Th s foo s W ndows nto th nk ng that the oca F eTab e share s a remote fi e share and, as a resu t, oca app cat ons w not attempt to use memory mapp ng for access ng F eTab e fi es You created a fo der and two fi es n the og ca fi e system by man pu at ng a F eTab e n T-SQL L kew se, you can have SQL Server nsert, update, and de ete rows n a F eTab e by man pu at ng fi es and d rector es n the og ca fi e system S m ar y, organ z ng fi es and fo ders us ng drag and drop 364  Part II  Going Beyond Relational

n W ndows Exp orer automat ca y adjusts the hierarchyid va ues n the path locator co umn of a ­ ffected rows n the F eTab e It’s easy to demonstrate th s b d rect ona capab ty; just use W ndows a Exp orer to drag and drop a few of your own fo ders and fi es nto the F eTab e share Then run a SELECT query aga nst the F eTab e You w see that SQL Server has nserted new rows to back the fo ders and fi es you added v a W ndows Exp orer

More Info There are several catalog views provided to help you manage FileTables in your database. There is also a special dynamic management view and related stored procedure that lets you monitor and kill open handles on FileTable resources. Consult Books Online for detailed usage and syntax of these system-defined objects that support FileTable.

Searching Documents You’ve seen how FILESTREAM and F eTab e a ow unstructured documents to be stored as first-c ass c t zens ns de the database w th unprecedented ease and performance So t’s on y natura to beg n wonder ng how to make better use of unstructured content that ves n the database, beyond mere y stream ng t to c ent app cat ons Th s s where Fu -Text Search (FTS) and the new Stat st ca Semant c Search n SQL Server 2012 come n The FTS eng ne was first ntroduced as an add-on component to SQL Server 2000, and t prov des much more powerfu w dcard search capab t es than the T-SQL LIKE operator FTS has stead y ­matured w th each product re ease, and s now a fu y ntegrated part of the re at ona database eng ne FTS can search varbinary(max) content, but pr or to FILESTREAM, t wasn’t feas b e to store many arge BLOBs n varbinary(max) co umns Now that do ng so s not on y feas b e but easy, FTS s a natura too for search ng content ns de many d fferent types of documents that s mp y get dropped nto a F eTab e FTS s ab e to parse many d fferent fi e types, nc ud ng M crosoft Word documents, M crosoft PowerPo nt decks, M crosoft Exce spreadsheets, Adobe PDF fi es, and a host of other popu ar formats It understands word-stemm ng and syntax ru es of over 50 anguages, and can a so perform prox m ty search ng w th the NEAR keyword In SQL Server 2012, FTS now a so supports property search ng and custom zab e d stance for prox m ty search ng w th NEAR SQL Server 2012 a so ntroduces Stat st ca Semant c Search, a new feature that bu ds on the FTS arch tecture, extend ng fu -text search capab t es beyond what s offered by FTS a one The ­Semant c Search eng ne d scovers key phrases n each document, and can therefore return documents deemed s m ar to one another based on common key phrases In add t on to enab ng FTS, you must separate y nsta , attach, and reg ster a spec a Semant c Language Stat st cs Database before you can start us ng Semant c Search In ts first re ease, Semant c Search supports fewer anguages than FTS does (on y about 15), and ndexes on y s ng e-word phrases However the ab ty to categor ze your documents automat ca y, and to dent fy documents that are s m ar or re ated, enab es powerfu new features for document management app cat ons



Chapter 8  Nat ve F e Stream ng   365

Summary For app cat ons that work w th BLOB data, FILESTREAM great y enhances the storage and ­performance of unstructured content n the database by everag ng the nat ve NTFS fi e system It does th s wh e ma nta n ng og ca ntegrat on between the database and fi e system that nc udes transact ona support As a resu t, you no onger need to comprom se between effic ency and ­comp ex ty as you d d n the past when faced w th the cho ce of stor ng BLOB data ns de or outs de the database You a so earned how to use SqlFileStream to de ver h gh-performance stream ng of BLOB content between the fi e system managed by SQL Server and your W ndows, web, and WPF app cat ons F na y, you earned how F eTab e n SQL Server 2012 comb nes FILESTREAM and hierachyid to furn sh a database-backed fi e system for users and app cat ons You can app y the techn ques you earned n th s chapter to a w de range of app cat ons that requ re ntegrat on between the re at ona database and a stream ng fi e system

366  Part II  Going Beyond Relational

C hapter 9

Geospatial Support —Leonard Lobel

L

ocat on-aware app cat ons are ub qu tous nowadays You find them on phones, tab ets, aptops, and desktops Enterpr se users and consumers a ke re y on Geograph c Informat on System (GIS) ­app cat ons (and GIS extens ons to trad t ona app cat ons) n the home, car, office, and everywhere n between GIS app cat ons—powered by G oba Pos t on ng Sate te (GPS) techno ogy—enhance the user exper ence w th soph st cated mapp ng nte gence In short, GIS s a about stor ng and p ­ rocess ng geospatial data (often s mp y ca ed spatial data), and th s chapter s a about the r ch ­support for spat a data n M crosoft SQL Server 2012

Note  We use the terms spatial and geospatial interchangeably throughout this chapter. In SQL Server 2012, spat a data support s a powerfu extens on to the core re at ona eng ne that enab es you to embed ocat on awareness nto the database Us ng the geometry and geography data types, you can mport, export, store, and process d fferent types of spat a data In th s chapter, we w qu ck y cover bas c spat a concepts, and then d ve nto more advanced examp es (a ong w th severa samp e app cat ons) to get you comfortab e work ng w th spat a data

SQL Server Spaces Out In SQL Server 2012, the database tse f s capab e of understand ng geospat a data (shapes—projected onto e ther a flat surface or the round earth) as a nat ve data type SQL Server can effic ent y store and process nstances of such data, so deve opers can enjoy r ch spat a funct ona ty at the database eve Th s feature (wh ch was first ntroduced n M crosoft SQL Server 2008 and s ava ab e n a ed t ons of SQL Server) offers an attract ve a ternat ve to perform ng spat a ca cu at ons at the app cat on eve , e ther through custom wr tten code (wh ch s far from tr v a ) or by us ng a th rd-party brary (wh ch s far from nexpens ve) The a gor thms used n geospat a operat ons are very comp ex (to say the east), and a c­ omprehens ve treatment of the top c s we beyond the scope of th s chapter At the same t me, abstract on of that comp ex ty s a key aspect of the spat a data types The geospat a support n SQL Server s based on the OpenGIS S mp e Features for SQL standard Th s standard enab es deve opers to qu ck y program aga nst spat a data w thout requ r ng a deep understand ng of the mathemat ca

367

formu ae beh nd spat a ca cu at ons In th s chapter, we’ start by first exp a n ng the two bas c spat a mode s Then we’ exam ne severa code demonstrat ons that everage the geometry and geography data types n a var ety of more advanced scenar os

Spatial Models Let’s beg n our d scuss on w th an exp anat on of the two spat a mode s These are known as the planar and geodetic mode s

Planar (Flat-Earth) Model The p anar mode s a flat surface where shapes are p otted us ng two-d mens ona x- and y-coord nates The coord nate system semant cs are ent re y up to you; measurements can be anyth ng from p xe s to nches, meters, and m es It makes sense to work w th the p anar mode when you’re dea ng w th re at ve y sma areas (such as a p ece of paper, bu d ng floor p an, or park ng ot), or even arger areas that are e ther conceptua y flat or st sma enough where the earth’s curvature does not skew the outcome of area and d stance ca cu at ons F atten ng the earth onto a two-d mens ona surface resu ts n a spat a d stort on that makes t ­mposs b e to ca cu ate measurements accurate y over arger spaces on the g obe’s actua shape You can see th s n F gure 9-1, wh ch shows a p anar mode map of the earth The flattened ­project on stretches the earth more and more the farther away ts parts are from the equator For examp e, not ce that Antarct ca (way down by the South Po e) appears to be s gn ficant y arger than North Amer ca Yet n fact, North Amer ca (at ~ 24 m on sq km) s s gn ficant y arger than Antarct ca (at ~ 13 m on sq km) Now draw a stra ght ne from New York to Amsterdam on th s map The ength of that ne does not rea y represent the accurate d stance between those two c t es, because the actua ne wou d be stretched and curved aga nst the shape of the g obe So c ear y, the p anar mode s nappropr ate for ca cu at ng spat a data over arge areas of the p anet

Geodetic (Ellipsoidal Sphere) Model If your database needs to perform spat a ca cu at ons that span s gn ficant areas of the earth, t needs to adopt the popu ar y he d be ef that the wor d s round It’s not perfect y round, however Due to ts rotat on, the earth s w der around the equator than t s around the po es, such that t forms an ellipsoidal sphere To accurate y p ot and compute shapes and ntersect ons on a p anetary sca e, the prec se curvature of the earth’s un que spher ca shape must be taken nto cons derat on To ach eve th s accuracy, the geodet c mode (dep cted n F gure 9-2) represents ocat ons on the p anet us ng the earth’s ong tude and at tude coord nate system Any g ven po nt on earth s ­represented as at tude and ong tude, where at tude spec fies the ang e north or south of the ­equator, and ong tude spec fies the ang e east or west of the Pr me Mer d an

368  Part II  Going Beyond Relational

North America (~ 24 m sq km) Equator

Africa (~ 30 m sq km)

Antarctica (~ 13 m sq km) Figure 9-1  P anar spat a mode (flat earth project on).

Figure 9-2  Geodet c spat a mode (round earth project on).



Chapter 9  Geospat a Support   369

Spatial Data Standards SQL Server prov des two data types—geometry and geography—des gned to work respect ve y w th the p anar and geodet c spat a mode s Shapes are projected onto spat a mode s us ng vector objects, wh ch are co ect ons of po nts, nes, po ygons (c osed shapes), and—new n SQL Server 2012—curves and arcs Both the geometry and geography data types support the three standard formats for consum ng spat a data We -Known Text (WKT), We -Known B nary (WKB), and Geography Markup Language (GML) WKT, WKB, and GML are standards governed by the Open Geospat a Consort um (OGC), an ­nternat ona body of more than 400 compan es M crosoft s an act ve and h gh y nfluent a member of the OGC, and SQL Server 2012 supports the OpenGIS S mp e Features for SQL standard—a spec ficat on pub shed by the OGC for the exchange of spat a data between d fferent database p atforms (you can earn more about the spec ficat on from http://www.opengeospatial.org/standards/sfs) The spat a data types are NET c asses that expose more than 90 ava ab e methods About t­ wo-th rds of those methods are named w th the prefix ST; for examp e, STArea, STDistance, and ­STArea The ST prefix nd cates spat a methods that mp ement the OGC spec ficat on, but ­M crosoft a so adds some of ts own extens ons to SQL Server that are not part of the OGC standard These are d st ngu shed w th method names that do not start w th ST; such as Parse, ToString, and GeomFromGml Some are stat c methods (mean ng that they are nvoked d rect y on the ­geometry or geography data type name us ng the spec a doub e-co on syntax), and some are nstance ­methods (mean ng that they are nvoked on an nstance of a geometry or geography data type us ng the ­standard “dot” notat on) A though the major ty of the spat a methods are ava ab e and ­funct on dent ca y w th both data types, a few of them are un que to on y geometry or geography Th s ­chapter exp a ns many, but certa n y not a , of the ava ab e spat a methods You shou d consu t M crosoft Books On ne for the comp ete spat a data type method reference (the geometry methods can be found at http://msdn.microsoft.com/en-us/library/bb933973.aspx and the geography methods at http://msdn.microsoft.com/en-us/library/bb933802.aspx)

Importing Well-Known Text (WKT) WKT s the eas est way to work w th spat a data n SQL Server It offers a very terse syntax for defin ng shapes us ng a handfu of keywords (such as POINT, LINESTRING, and POLYGON), and SQL Server mp ements a spat a data c ass for each of these shapes F gure 9-3 shows a the supported spat a c asses, a ong w th the r assoc ated WKT keywords and an examp e nstance of each shape Note that the CIRCULARSTRING, COMPOUNDCURVE, CURVEPOLYGON, and FULLGLOBE WKT shapes are new c asses n SQL Server 2012, and that we cover those and other SQL Server 2012 enhancements ater n the chapter

370  Part II  Going Beyond Relational

POINT

MULTIPOINT

LINESTRING

MULTILINESTRING

CIRCULARSTRING

COMPOUNDCURVE

POLYGON

MULTIPOLYGON

CURVEPOLYGON

GEOMETRYCOLLECTION

FULLGLOBE

Figure 9-3  Spat a c asses and the r assoc ated WKT keywords.

Every shape from the s mp est po nt to the most comp ex po ygons and co ect ons s expressed as a WKT str ng The WKT str ng syntax comb nes the var ous shape keywords w th numer c coord nates Tab e 9-1 shows some examp es of WKT str ngs Table 9-1  Examp es of WKT Str ngs



WKT String

Description

POINT(6 10)

A s ng e po nt at xy coord nates 6, 10

POINT( 111.06687 45.01188)

A s ng e po nt on the earth ( ong tude/ at tude coord nates)

LINESTRING(3 4,10 50,20 25)

A two part ne, drawn between three po nts spec fied as ­x y coord nates

POLYGON(( 75.17031 39.95601, 75.16786 39.95778, 75.17921 39.96874, 75.18441 39.96512, 75.17031 39.95601))

An enc osed shape on the earth drawn between the po nts spec fied as ong tude/ at tude coord nates

CIRCULARSTRING(1 5, 6 2, 7 3)

A curved ne, drawn between three po nts spec fied as ­x y coord nates

GEOMETRYCOLLECTION( POINT(6 10), CIRCULARSTRING(1 5, 6 2, 7 3), LINESTRING(3 4,10 50,20 25))

A co ect on w th three shapes; a po nt, a c rcu ar str ng, and a ne str ng

Chapter 9  Geospat a Support   371

As you can see, the same WKT syntax s used for project ng spat a ent t es onto both p anar and geodet c mode s A so not ce that geodet c coord nates are a ways expressed n WKT w th the ­ong tude va ue first, fo owed by the at tude va ue

More Info  In addition to the intersection of X (or latitude) and Y (or longitude) c­ oordinates, SQL Server supports two optional data values that you can associate with each point to represent elevation: the Z value (height) and M value (measure). Both of these values are user-defined and, if supplied, are stored and retrieved with the spatial data instance. However, elevation is not considered in any spatial calculations; it merely tags along as metadata in the coordinates. WKT s SQL Server’s defau t spat a format, so str ng tera s expressed us ng WKT syntax can be ass gned d rect y nto a geometry or geography data type S m ar y, spat a data s returned n WKT format when you nvoke the ToString method on a geometry or geography data type For examp e, the fo ow ng statement parses a WKT str ng and oads t nto the geometry data type stored n @line DECLARE @line geometry = 'LINESTRING(5 15, 22 10)'

Th s next statement shows the raw b nary data for the shape n @line, and a so uses ToString to convert t back nto WKT for d sp ay SELECT @line AS AsRaw, @line.ToString() AS AsWKT

Here s the output AsRaw AsWKT ---------------------------------------------------------------- -----------------------0x00000000011400000000000014400000000000002E40000000000000364... LINESTRING (5 15, 22 10)

The STGeomFromText and STxxxFromText Methods The geometry and geography data types a so prov de a set of stat c methods for exp c t y ­mport ng any shape—or spec fic shapes—expressed n WKT The STGeomFromText method accepts any ­WKT-expressed shape and mports t as a geometry or geography type In fact, SQL Server actua y ca ed STGeomFromText when you just supp ed the str ng tera n the prev ous ass gnment Thus, the fo ow ng ass gnment s equ va ent DECLARE @line geometry = geometry::STGeomFromText('LINESTRING(5 15, 22 10)', 0)

The STxxxFromText (for examp e, STPointFromText, STLineFromText, and STPolyFromText) methods add extra va dat on on the WKT str ng to ensure t represents the ntended shape The next ­statement uses the STLineFromText method to demonstrate yet another way of oad ng the same shape DECLARE @line geometry = geometry::STLineFromText('LINESTRING(5 15, 22 10)', 0)

372  Part II  Going Beyond Relational

Th s statement works just the same, but on y because the WKT str ng s ndeed a ne str ng For examp e, the fo ow ng statement s nva d DECLARE @line geometry = geometry::STLineFromText('POINT(10 100)', 0)

Th s statement fa s because a WKT str ng represent ng a po nt was passed to STLineFromText, wh ch s a method that can on y accept ne str ngs Msg 6522, Level 16, State 1, Line 19 A .NET Framework error occurred during execution of user-defined routine or aggregate "geometry": System.FormatException: 24142: Expected "LINESTRING" at position 1. The input has "POINT(10 1". System.FormatException: at Microsoft.SqlServer.Types.WellKnownTextReader.RecognizeToken(String token) at Microsoft.SqlServer.Types.WellKnownTextReader.ParseTaggedText(OpenGisType type) at Microsoft.SqlServer.Types.WellKnownTextReader.Read(OpenGisType type, Int32 srid) at Microsoft.SqlServer.Types.SqlGeometry.GeometryFromText(OpenGisType type, SqlChars text, Int32 srid)

Th s rather verbose error message a so revea s that the spat a data types are mp emented as common anguage runt me (CLR) types Th s means that SQL Server spat a support s conta ned ns de a NET assemb y that res des on the server, wh ch a so means that you can reference the very same ­assemb y and ut ze the same spat a types and methods n your C# and VB NET app cat ons You’ see exact y how that’s done n a samp e app cat on com ng up ater n the chapter

Importing WKB We -Known B nary (WKB) s the OGC-standard b nary equ va ent of WKT, wh ch s s ght y d fferent than the nat ve b nary format that SQL Server uses to store spat a data nterna y

The STGeomFromWKB and STxxxFromWKB Methods If your data source supp es spat a data n WKB, you can use STGeomFromWKB or one of the v­ ar ous STxxxFromWKB methods to mport t As w th WKT, there s one such method for each supported shape (for examp e, STPointFromWKB, STLineFromWKB, and STPolyFromWKB) For examp e, the fo ow ng statement mports a WKB va ue and oads t nto the geometry data type stored n @point DECLARE @point geometry = geometry::STGeomFromWKB(0x010100000000000000000059400000000000005940, 0)

Th s next statement shows the raw (SQL Server) b nary data for the shape n @point, and uses ToString to convert t back nto WKT for d sp ay SELECT @point AS AsRaw, @point.ToString() AS AsWKT

Here s the output AsRaw ---------------------------------------------0x00000000010C00000000000059400000000000005940



AsWKT --------------POINT (100 100)

Chapter 9  Geospat a Support   373

Not ce that the raw b nary va ue s s m ar to the mported WKB va ue, but the two are not actua y dent ca

Importing Geography Markup Language (GML) Geography Markup Language (GML) s another spat a data format, and was deve oped by the OGC after WKT GML s both a anguage and an open nterchange format for spat a nformat on ­throughout the Internet It uses an XML d a ect that s s gn ficant y r cher and more verbose than WKT

The GeomFromGml Method You can mport spat a data from GML nto SQL Server us ng the GeomFromGml method, as fo ows DECLARE @gml xml = '

100 100 20 180 180 180 ' DECLARE @line geometry = geometry::GeomFromGml(@gml, 0) SELECT @line AS AsRaw, @line.ToString() AS AsWKT

Here s the output AsRaw AsWKT --------------------------------------------------- -----------------------0x0000000001040300000000000000000059400000000000... LINESTRING (100 100, 20 180, 180 180)

We use WKT exc us ve y n the rest of th s chapter’s code GML s a wor d unto tse f (no pun ntended), and deta ed GML coverage s beyond the scope of th s book (the GML standard s posted on the OGC webs te at http://www.opengeospatial.org/standards/gml) The key po nt to understand s that the geometry and geography data types (and the r spat a methods) are comp ete y agnost c to the part cu ar OGC standard used for mport ng spat a data; whether t’s WKT, WKB, or GML

Spatial Data Types The geometry and geography data types are system CLR types, wh ch means they are mp emented by the NET CLR nterna y by SQL Server The geometry data type s prov ded to store and process spat a data us ng the p anar mode , whereas ts counterpart geography supports the geodet c mode One n ce th ng about the two spat a types s that most methods are supported and work the same for both types So f you’re work ng w th the geodet c mode , you’ be us ng ong tude and at tude coord nates w th the geography data type For the p anar mode , you’ be us ng x- and y-coord nates w th the geometry data type But n e ther case, you’ then use many of the same methods for dea ng w th your spat a ent t es

374  Part II  Going Beyond Relational

Working with geometry Our first examp e demonstrates the geometry data type n a very s mp e scenar o You w define and store shapes represent ng town d str cts and avenues, and then query that data to return nterest ng nformat on, such as a st of the avenues that run through each d str ct Th s tt e, one-horse town has on y three d str cts and two ma n avenues, and s sma enough to be expressed on a p anar (flat) surface us ng the geometry data type As you progress through the examp e, you’ earn how to use a number of common geospat a methods F gure 9-4 shows a map of the town, and the xy-coord nates for the po nts of each shape on the map The three d str cts are po ygons (c osed shapes), and the two avenues are ne str ngs You w store these shapes n a SQL Server database us ng the geometry data type, express ng the r coord nates n WKT syntax Then you’ query and man pu ate the d str cts and streets us ng var ous spat a methods 0, 0

Downtown

150, 0

Green Park

300, 0

50, 50

100, 100 Bea

ch S tree

t

0, 150

150, 150 Harborside

20, 180

First Avenue

300, 150

180, 180

150, 300

300, 300

Figure 9-4  Sma town map on a flat (p anar) surface.

The first th ng you need to do s create tab es to ho d the shapes that define the d str cts and streets The District tab e w store the po ygons represent ng Downtown, Green Park, and Harbors de n a geometry co umn named DistrictGeo S m ar y, you’ create a Street tab e to store the ne str ngs represent ng Beach Street and F rst Avenue n a geometry co umn named StreetGeo, as shown n ­L st ng 9-1



Chapter 9  Geospat a Support   375

Listing 9-1  Creat ng the District and Street tab es w th geometry co umns.

USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'MyDB') DROP DATABASE MyDB GO CREATE DATABASE MyDB GO USE MyDB GO CREATE TABLE District (DistrictId int PRIMARY KEY, DistrictName nvarchar(20), DistrictGeo geometry) CREATE TABLE Street (StreetId int PRIMARY KEY, StreetName nvarchar(20), StreetGeo geometry)

Next, popu ate the District tab e w th the three d str cts a ong w th the r shapes, s zes, and ­coord nates as po ygons n WKT, as shown n L st ng 9-2 Listing 9-2  Us ng WKT to nsert po ygons nto geometry co umns.

INSERT INTO District VALUES (1, 'Downtown', 'POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0))'), (2, 'Green Park', 'POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0))'), (3, 'Harborside', 'POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))')

Not ce how the square shapes represent ng each d str ct are conveyed n WKT format as po ygon e ements Each po nt connect ng the nes of the po ygon s expressed as an xy-coord nate ­Un ke ne str ngs (wh ch you’ use short y to define the streets), a po ygon a ways represents a c osed shape In WKT, the coord nate for the fina po nt n any po ygon must be the same coord nate used for the start ng po nt n order to c ose the shape You can see how a of the d str ct po ygons n the ­preced ng code c ose the r shapes n th s manner If you attempt to express a po ygon w thout c os ng the shape, you w rece ve a FormatException, as fo ows Msg 6522, Level 16, State 1, Line 81 A .NET Framework error occurred during execution of user-defined routine or aggregate "geometry": System.FormatException: 24119: The Polygon input is not valid because the start and end points of the exterior ring are not the same. Each ring of a polygon must have the same start and end points. System.FormatException: at Microsoft.SqlServer.Types.GeometryValidator.ValidatePolygonRing(Int32 iRing, Int32 cPoints, Double firstX, Double firstY, Double lastX, Double lastY) at Microsoft.SqlServer.Types.Validator.Execute(Transition transition) at Microsoft.SqlServer.Types.ForwardingGeoDataSink.EndFigure()

376  Part II  Going Beyond Relational

at Microsoft.SqlServer.Types.WellKnownTextReader.ParseLineStringText() at Microsoft.SqlServer.Types.WellKnownTextReader.ParsePolygonText() at Microsoft.SqlServer.Types.WellKnownTextReader.ParseTaggedText(OpenGisType type) at Microsoft.SqlServer.Types.WellKnownTextReader.Read(OpenGisType type, Int32 srid) at Microsoft.SqlServer.Types.SqlGeometry.GeometryFromText(OpenGisType type, SqlChars text, Int32 srid) at Microsoft.SqlServer.Types.SqlGeometry.Parse(SqlString s) . The statement has been terminated.

Th s rather unfr end y error message a so a udes to the fact that po ygons can have mu t p e r ngs defined ns de the exter or r ng Th s means that you can define comp ex po ygons that have one or more “ho es” ns de them us ng WKT Now popu ate the Street tab e w th ne str ngs, as shown n L st ng 9-3 Listing 9-3  nsert ng ne str ngs nto geometry co umns.

INSERT INTO Street VALUES (1, 'First Avenue', 'LINESTRING (100 -100, 20 -180, 180 -180)'), (2, 'Beach Street', 'LINESTRING (300 -300, 300 -150, 50 -50)')

The ne str ng e ements used to store the streets of the town spec fy the po nts that descr be the paths of each street on the map In th s examp e, each street has three po nts express ng the two-part ne str ngs for F rst Avenue and Beach Street Now execute the un on query n L st ng 9-4 to retr eve a the shapes from both tab es comb ned nto a s ng e resu t set Listing 9-4  Retr ev ng spat a data.

SELECT Name = StreetName, AsRaw = StreetGeo, AsWKT = StreetGeo.ToString() FROM Street UNION ALL SELECT Name = DistrictName, AsRaw = DistrictGeo, AsWKT = DistrictGeo.ToString() FROM District

Here are the resu ts of the query Name -----------First Avenue Beach Street Downtown Green Park Harborside



AsRaw ----------------0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104...

AsWKT -----------------------------------------------------------LINESTRING (100 -100, 20 -180, 180 -180) LINESTRING (300 -300, 300 -150, 50 -50) POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0)) POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0)) POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))

Chapter 9  Geospat a Support   377

If you execute th s query n SQL Server Management Stud o (SSMS), you can v sua ze these resu ts us ng the spat a v ewer (unfortunate y, the spat a v ewer s not ava ab e ns de of V sua Stud o ­us ng SQL Server Data Too s [SSDT], so we use SSMS n th s chapter for spat a deve opment) Because SSMS recogn zes that the resu t set conta ns spat a data, t adds a new tab named Spat a Resu ts ­n-­between the Resu ts and Messages tabs C ck the Spat a Resu ts tab and you w be presented w th a graph ca render ng of the town map, as shown n F gure 9-5

Figure 9-5  D sp ay ng the town map data n SSMS us ng the Spat a Resu ts v ewer.

For a resu t set that conta ns mu t p e spat a data co umns, you can graph any one of them us ng the first drop-down st on the r ght of the chart You can a so add egend abe s based on another co umn n the resu t set us ng the second drop-down st You can a so set the chart’s ­magn ficat on eve and togg e the d sp ay of gr d nes us ng the s der and check box contro s beneath the ­drop-down sts

The STBuffer Method You can nvoke STBuffer on any spat a nstance to pad a shape by any amount (d stance), and return t back as a new nstance The query n L st ng 9-5 demonstrates the use of STBuffer to w den the streets, wh ch converts the ne str ng nto a po ygon represent ng the padded street area

378  Part II  Going Beyond Relational

Listing 9-5 Padd ng LINESTRING shapes w th the STBuffer method.

SELECT Name = StreetName, AsRaw = StreetGeo.STBuffer(5), AsWKT = StreetGeo.STBuffer(5).ToString() FROM Street UNION ALL SELECT Name = DistrictName, AsRaw = DistrictGeo, AsWKT = DistrictGeo.ToString() FROM District

The resu ts of the query show that the ne str ng shapes for F rst Avenue and Beach Street are now be ng returned as po ygon shapes represent ng streets w th w dth Name -----------First Avenue Beach Street Downtown Green Park Harborside

AsRaw ----------------0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104...

AsWKT -----------------------------------------------------------POLYGON ((20.000000000000881 -184.99999999999966, 180.00... POLYGON ((300.00000000000034 -304.9999999999992, 300.243... POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0)) POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0)) POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))

In SSMS, c ck the Spat a Resu ts tab to v ew the town map w th the w dened streets, as shown n F gure 9-6

Figure 9-6  V ew ng the resu t of ca ng STBuffer n the Spat a Resu ts v ewer.



Chapter 9  Geospat a Support   379

The STCentroid and STEnvelope Methods The STCentroid method returns the po nt at the center of a g ven shape, and the STEnvelope ­method returns a rectang e that bounds a g ven shape These two methods are ava ab e on y for the ­geometry data type The query n L st ng 9-6 uses STCentroid to ocate the center po nt of each d str ct, and then uses STBuffer to pad the po nt so that t appears as a sma c rc e that s c ear y v s b e n the Spat a Resu ts v ewer (Note, however, that STBuffer does not actua y generate c rcu ar str ngs, but nstead p ots many po nts n a po ygon very c ose to one another to produce the rounded padd ng shape ) Listing 9-6  Locat ng the center of a shape w th the STCentroid method.

SELECT DistrictGeo, DistrictGeo.ToString() FROM District UNION ALL SELECT DistrictGeo.STCentroid().STBuffer(5), DistrictGeo.STCentroid().ToString() FROM District

The resu ts appear as shown n F gure 9-7

Figure 9-7 Us ng STCentroid to ocate the center of each d str ct.

380  Part II  Going Beyond Relational

The query n L st ng 9-7 uses STEnvelope to dent fy the bound ng box of each street Each street s returned (aga n, padded w th STBuffer so that streets are c ear y v s b e n the Spat a Resu ts v ewer) a ong w th a rectangu ar shape around each street generated by STEnvelope Listing 9-7  Us ng the STEnvelope method to generate a bound ng box for each street.

SELECT StreetName, StreetGeo.STBuffer(5), StreetGeo.ToString() FROM Street UNION ALL SELECT StreetName + ' Bounds', StreetGeo.STEnvelope(), StreetGeo.STEnvelope().ToString() FROM Street

The resu ts appear as shown n F gure 9-8

Figure 9-8  Resu t of ca ng STEnvelope for each street.



Chapter 9  Geospat a Support   381

The STIntersects and STIntersection Methods Now you’re go ng to wr te a query to find out wh ch streets ntersect wh ch d str cts Th s query w use a CROSS JOIN so that t returns a poss b e comb nat ons of d str cts and streets You’ qua fy the query w th a WHERE c ause to fi ter the resu ts and report on y those d str ct and street ­comb nat ons that actua y ntersect w th one another Th s s eas y ach eved w th the STIntersects method You ­ nvoke STIntersects on one spat a nstance, pass ng n another spat a nstance as ts parameter, and SQL Server returns a bit (Boo ean) va ue of 1 (true) or 0 (fa se) te ng you whether or not the two geometry shapes ntersect w th each other L st ng 9-8 demonstrates Listing 9-8  Us ng the STIntersects method to find a d str ct and street ntersect ons.

SELECT S.StreetName, D.DistrictName FROM District AS D CROSS JOIN Street AS S WHERE S.StreetGeo.STIntersects(D.DistrictGeo) = 1 ORDER BY S.StreetName

When you run th s query, the resu ts nd cate that Beach Street runs through a three d str cts, whereas F rst Avenue runs through Downtown and Harbors de, but not Green Park StreetName -------------------Beach Street Beach Street Beach Street First Avenue First Avenue

DistrictName -------------------Downtown Green Park Harborside Downtown Harborside

But you can do more than just find out whether two spat a ent t es ntersect—you can us the STIntersection method to actua y obta n a shape that represents the over app ng area of ntersect on n a new spat a ent ty L st ng 9-9 presents a mod fied vers on of the same STIntersects query that a so uses STIntersection to report wh ch pieces of each road cut through each d str ct Listing 9-9  Us ng the STIntersection method to generate road fragments for each d str ct.

SELECT S.StreetName, D.DistrictName, S.StreetGeo.STIntersection(D.DistrictGeo).STBuffer(5) AS Intersection, S.StreetGeo.STIntersection(D.DistrictGeo).ToString() AS IntersectionWKT FROM District AS D CROSS JOIN Street AS S WHERE S.StreetGeo.STIntersects(D.DistrictGeo) = 1 ORDER BY S.StreetName

The resu ts of the query show the ne str ng shapes created by STIntersection for the port on of each street that runs through each d str ct 382  Part II  Going Beyond Relational

StreetName -----------Beach Street Beach Street Beach Street First Avenue First Avenue

DistrictName -----------Downtown Green Park Harborside Downtown Harborside

Intersection -------------------0x0000000001047F0... 0x0000000001047F0... 0x000000000104870... 0x000000000104870... 0x000000000104870...

IntersectionWKT -------------------------------------------LINESTRING (50 -50, 150 -89.999999999999886) LINESTRING (150.00000000000125 -90.00000... LINESTRING (300 -150, 300 -300) LINESTRING (100 -100, 50.000000000000782... LINESTRING (180 -180, 150.00000000000071...

(5 row(s) affected)

Not ce that the port ons of F rst Avenue that do not run through any of the three d str cts are exc uded from the resu ts As F gure 9-9 shows, you can see a v sua representat on of the resu ts by c ck ng the Spat a Resu ts tab n SSMS

Figure 9-9  V ew ng the road fragments of the streets n each d str ct us ng STIntersection.

The STDimension Method The STDimension method can be nvoked on any spat a nstance, and returns a number (0, 1, or 2) that nd cates how many d mens ons are defined by the shape n the nstance S ng e po nts have 0 d mens ons; nes, c rcu ar nes, and ne str ngs have 1 d mens on; and po ygons (c osed shapes) have 2 d mens ons That means that a of the d str cts n the town are two-d mens ona objects, whereas the streets are a one-d mens ona objects The code n L st ng 9-10 d sp ays the d mens ons for a the spat a objects current y stored n the database

Chapter 9  Geospat a Support   383

Listing 9-10  Us ng the STDimension method to show the number of d mens ons for each shape.

SELECT StreetName AS Shape, StreetGeo.ToString() AS ShapeWKT, StreetGeo.STDimension() AS Dimensions FROM Street UNION ALL SELECT DistrictName AS Shape, DistrictGeo.ToString() AS ShapeWKT, DistrictGeo.STDimension() AS Dimensions FROM District

Here are the resu ts Shape ------------First Avenue Beach Street Downtown Green Park Harborside

ShapeWKT -----------------------------------------------------------LINESTRING (100 -100, 20 -180, 180 -180) LINESTRING (300 -300, 300 -150, 50 -50) POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0)) POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0)) POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))

Dimensions ----------1 1 2 2 2

The STUnion, STDifference, and STSymDifference Methods Our next examp e demonstrates some more man pu at ons w th over app ng reg ons between shapes The code n L st ng 9-11 defines two over app ng po ygons, and then ca s severa methods aga nst those shapes The code st ng s fo owed by a ser es of screen snapshots show ng the Spat a Resu ts v ewer render ng of each method’s resu t Listing 9-11  Compar ng the STUnion, STIntersection, STDifference, and STSymDifference methods.

DECLARE @S1 geometry = 'POLYGON ((60 40, 410 50, 400 270, 60 370, 60 40))' DECLARE @S2 geometry = 'POLYGON ((300 100, 510 110, 510 330, 300 330, 300 100))' SELECT SELECT SELECT SELECT SELECT SELECT

@S1 UNION ALL SELECT @S2 S1_UNION_S2 = @S1.STUnion(@S2) S1_INTERSECTION_S2 = @S1.STIntersection(@S2) S1_DIFFERENCE_S2 = @S1.STDifference(@S2) S2_DIFFERENCE_S1 = @S2.STDifference(@S1) S1_SYMDIFFERENCE_S2 = @S1.STSymDifference(@S2)

The first query comb nes (un ons) the two shapes so you can see what they ook ke and how they over ap, as shown n F gure 9-10

384  Part II  Going Beyond Relational

Figure 9-10  Two over app ng shapes.

STUnion returns a new shape based on the two source shapes (d sregard ng the over app ng ­port on), as shown n F gure 9-11

Figure 9-11  Comb n ng the two over app ng shapes nto one us ng STUnion.



Chapter 9  Geospat a Support   385

Ear er, you used the STIntersection method to break down one shape (a str ng) nto mu t p e shapes (by d str ct) Once aga n, STIntersection produces a s m ar resu t by creat ng a new shape represent ng just the area of over ap (the center p ece) between the two source shapes, as shown n F gure 9-12

Figure 9-12  dent fy ng the area of over ap between the two shapes us ng STIntersection.

The STDifference method “cuts away” from one shape the over app ng area of another shape F­ gure 9-13 shows the resu t of nvok ng STDifference to cut away the part of the first shape where the second shape over aps

Figure 9-13  D fferenc ng the second shape away from the first shape us ng STDifference.

386  Part II  Going Beyond Relational

F gure 9-14 shows the reverse, where STDifference s used to cut away the part of the second shape where the first shape over aps

Figure 9-14  D fferenc ng the first shape away from the second shape us ng STDifference.

F na y, the STSymDifference method returns the “symmetr ca ” d fference between the two shapes Th s s essent a y the oppos te of the STIntersection method—a new shape s generated that ­represents everyth ng except the area of over ap between the two shapes (everyth ng but the center p ece), as shown n F gure 9-15

Figure 9-15  Obta n ng the symmetr c d fference between two shapes us ng STSymDifference.



Chapter 9  Geospat a Support   387

Working with geography Most (but not a ) methods of the geometry type are ava ab e and funct on s m ar y for the ­geography type The pr mary d fference s that you must use ong tude and at tude va ues ­Interna y, as exp a ned, SQL Server automat ca y compensates for the earth’s curvature when perform ng ­ca cu at ons aga nst our geography data In th s sect on, you’ work w th the geography data type How do you get your hands on ong tudes and at tudes? There are many ways Coord nates for arge c t es and other major ocat ons n the wor d can be eas y obta ned on the Wor d W de Web w th a qu ck search and v a read y ava ab e web serv ce APIs, nc ud ng Yahoo! and Goog e The ­o d-fash oned way st works too, so p ots can acqu re them from sect ona maps, for examp e You can a so use M crosoft Streets & Tr ps 2008, wh ch has a “ ocat on sensor” too that w te you the ong tude and at tude coord nates for any po nt or shape drawn on the map us ng the mouse Th s Streets & Tr ps feature was used to obta n the coord nates for our next examp e

On Your Mark … You’ use a rea - fe event to earn about spat a area, ength, and d stance ca cu at ons In th s ­app cat on, you are mapp ng the Pro Cyc ng Tour he d n Ph ade ph a (wh ch entered ts 27th year n 2011) Us ng ong tude and at tude coord nates, you w bu d a database that stores d fferent areas of the b ke race Then you’ wr te quer es to ca cu ate the area and ength of d fferent reg ons us ng other new spat a methods The map for th s app cat on s shown n F gure 9-16 The ent re race area s conta ned n one arge 14-s ded po ygon W th n the race area, there are two popu ar ocat ons where spectators gather and take p ctures One s the Parkway Area to the south, where the race starts and fin shes, and the other s the Wa Area to the north In th s app cat on, you’ comb ne spat a features w th FILESTREAM Photos subm tted by s­ pectators w be stored n the database as b nary arge objects (BLOBs) n the fi e system us ng the techn ques you earned ear er, dur ng our FILESTREAM coverage n Chapter 8 Refer to the sect on “Enab ng FILESTREAM for the Mach ne” n that chapter, wh ch descr bes how to enab e FILESTREAM us ng SQL Server Configurat on Manager Then execute the code n L st ng 9-12 to create the FILESTREAM-enab ed database for the app cat on Note that the path to the database, C \Demo\ EventL brary n th s examp e, must ex st before the database can be created Listing 9-12  Creat ng the EventLibrary database.

USE master GO EXEC sp_configure filestream_access_level, 2 RECONFIGURE GO CREATE DATABASE EventLibrary ON PRIMARY

388  Part II  Going Beyond Relational

(NAME = EventLibrary_data, FILENAME = 'C:\Demo\EventLibrary\EventLibrary_data.mdf'), FILEGROUP FileStreamGroup1 CONTAINS FILESTREAM (NAME = EventLibrary_group2, FILENAME = 'C:\Demo\EventLibrary\Events') LOG ON (NAME = EventLibrary_log, FILENAME = 'C:\Demo\EventLibrary\EventLibrary_log.ldf') GO USE EventLibrary GO

POLYGON(( -75.22280 40.02387, -75.21442 40.02810, -75.21746 40.03142, -75.22534 40.02586, -75.22280 40.02387))

Wall Area One of the most interesting parts of the race route. Contains the infamous 17% grade climb known as the “Manayunk Wall.”

POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.18870 39.97789, -75.18521 39.99237, -75.18603 40.00677, -75.19922 40.01136, -75.21746 40.03142, -75.22534 40.02586, -75.21052 40.01430, -75.19192 40.00634, -75.19248 39.99570, -75.20526 39.98374, -75.19437 39.97704, -75.19087 39.96920, -75.17031 39.95601))

Race Area The entire Pro Cycling Championship race route is contained inside this irregular polygon.

POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.17921 39.96874, -75.18441 39.96512, -75.17031 39.95601))

Parkway Area The race starts and finishes here on the Benjamin Franklin Parkway.

Figure 9-16  Pro Cyc ng Tour reg on map.

Next, create the EventRegion tab e to ho d the d fferent map reg ons and then popu ate the tab e w th the po ygons represent ng the three reg ons be ng mapped, as shown n L st ng 9-13



Chapter 9  Geospat a Support   389

Listing 9-13  Creat ng the EventRegion tab e and popu at ng t w th geograph ca data.

CREATE TABLE EventRegion (RegionId int PRIMARY KEY, RegionName nvarchar(32), MapShape geography) INSERT INTO EventRegion VALUES(1, 'Parkway Area', geography::Parse('POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.17921 39.96874, -75.18441 39.96512, -75.17031 39.95601 ))')) INSERT INTO EventRegion VALUES(2, 'Wall Area', geography::Parse('POLYGON(( -75.22280 40.02387, -75.21442 40.02810, -75.21746 40.03142, -75.22534 40.02586, -75.22280 40.02387))')) INSERT INTO EventRegion VALUES(3, 'Race Area', geography::Parse('POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.18870 39.97789, -75.18521 39.99237, -75.18603 40.00677, -75.19922 40.01136, -75.21746 40.03142, -75.22534 40.02586, -75.21052 40.01430, -75.19192 40.00634, -75.19248 39.99570, -75.20526 39.98374, -75.19437 39.97704, -75.19087 39.96920, -75.17031 39.95601))'))

Now execute the statement SELECT * FROM EventRegion to query for the st of reg ons you just created n the EventRegion tab e F gure 9-17 shows how the query resu ts appear n the spat a v ewer

Figure 9-17  V ew ng spat a geography data n SSMS.

390  Part II  Going Beyond Relational

Not ce the th rd drop-down st abe ed Se ect Project on Th s drop-down st was not present on the spat a v ewer n our ear er quer es because they returned geometry (p anar) data Th s query returns geography (geodet c) spat a data, so the spat a v ewer a ows you to render the resu ts us ng one of severa d fferent project ons from the add t ona drop-down st By defau t, the v ewer uses the Equ rectangu ar project on, but you can change that to observe the d fferent ways that geodet c spat a data s skewed when flatten ng the g obe us ng d fferent flat-earth project on mode s For ­examp e, change the th rd drop-down st to use the Mercator project on As shown n F gure 9-18, the same geodet c data gets skewed to a s ght y d fferent shape when flattened use the Mercator ­project on (not ce how t’s narrower than the Equ ractangu ar project on n F gure 9-17)

Figure 9-18  V ew ng the same spat a data us ng the Mercator project on.

The STArea and STLength Methods Area and ength (per meter) ca cu at ons are eas y performed us ng the STArea and STLength ­methods The query n L st ng 9-14 d sp ays the area and ength of each reg on, sorted from argest to sma est Listing 9-14  Ca cu at ng area and ength us ng the STArea and STLength methods.

SELECT RegionName, ROUND(MapShape.STArea(), 2) AS Area, ROUND(MapShape.STLength(), 2) AS Length



Chapter 9  Geospat a Support   391

FROM EventRegion ORDER BY MapShape.STArea() DESC

Here are the resu ts RegionName -------------------------------Race Area Parkway Area Wall Area

Area ---------------------6432902.35 689476.79 334024.82

Length ---------------------22165.07 4015.39 2529.11

(3 row(s) affected)

Spatial Reference IDs The first quest on you’re bound to ask when v ew ng these resu ts s, what un t of measurement does SQL Server use to express the area and ength? The answer s, t depends on the Spat a Reference ID (SRID) of the spat a data type nstance By defau t, geography nstances use an SRID va ue of 4326, wh ch s based on the metr c system Therefore, the area and ength resu ts shown n the preced ng code are g ven n square meters and meters, respect ve y You can execute the statement SELECT * FROM sys.spatial reference systems to obta n a st of a the SRIDs supported by SQL Server Some of the SRIDs support d fferent measurement systems, nc ud ng foot, C arke’s foot, Ind an foot, U S survey foot, and German ega meter Ca cu at ons ­performed between two spat a nstances requ re both nstances to use the same SRID See Books On ne for more nformat on about SRIDs

Building Out the EventLibrary Database Now t’s t me to create the EventPhoto tab e, as shown n L st ng 9-15 The RowId and Photo ­co umns, respect ve y, prov de the ROWGUIDCOL and varbinary(max) FILESTREAM co umns requ red for FILESTREAM storage and access to b nary photo fi es, as we exp ored n depth n Chapter 8 The ­Location co umn stores the po nt ( ong tude and at tude) where the photo was taken n a geography data type Th s tab e w be popu ated w th severa photos us ng the very same techn ques you used n the photo brary FILESTREAM app cat on from Chapter 8 Listing 9-15  Creat ng the EventPhoto tab e to ho d photos and the ocat ons where they were taken.

CREATE TABLE EventPhoto ( PhotoId int PRIMARY KEY, RowId uniqueidentifier ROWGUIDCOL NOT NULL UNIQUE DEFAULT NEWSEQUENTIALID(), Description varchar(max), Location geography, Photo varbinary(max) FILESTREAM DEFAULT(0x))

392  Part II  Going Beyond Relational

You’re about to create a W ndows Forms app cat on n C# that a ows the user to se ect a reg on and retr eve a the photos taken n that reg on Once aga n, us ng the same FILESTREAM techn ques you’ve a ready earned for the PhotoLibrary app cat on n Chapter 8, the user can then se ect a photo for d sp ay n a p cture box contro To support th s app cat on, you’ wr te severa stored procedures (see L st ng 9-16) GetRegions s ca ed to retr eve a st of a the reg on IDs and names, wh ch gets bound to a combo box for the user to make a reg on se ect on GetRegionPhotos accepts the ID of the reg on se ected by the user and returns a st of a the photo IDs and descr pt ons of p ctures taken n the se ected reg on, wh ch gets bound to a st box for the user to make a photo se ect on The photo st s obta ned by u ­ s ng the STIntersects method you earned about ear er n th s chapter w th the geometry type Last, ­GetPhotoForFilestream returns the PathName and GET FILESTREAM TRANSACTION CONTEXT va ues for us ng SqlFileStream to retr eve and d sp ay the photo se ected by the user Listing 9-16  Creat ng the stored procedures for the samp e event med a app cat on.

-- Present list of regions to user CREATE PROCEDURE GetRegions AS BEGIN SELECT RegionId, RegionName FROM EventRegion END GO -- Get all the photos taken in the selected region CREATE PROCEDURE GetRegionPhotos(@RegionId int) AS BEGIN DECLARE @MapShape geography -- Get the shape of the region SELECT @MapShape = MapShape FROM EventRegion WHERE RegionId = @RegionId -- Get all photos taken in the region SELECT PhotoId, Description FROM EventPhoto WHERE Location.STIntersects(@MapShape) = 1 END GO -- Get the SqlFileStream information for retrieving the selected photo CREATE PROCEDURE GetPhotoForFilestream(@PhotoId int) AS BEGIN -- Called by ADO.NET client during open transaction to use SqlFileStream SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT()



Chapter 9  Geospat a Support   393

FROM WHERE

EventPhoto PhotoId = @PhotoId

END GO

Creating the Event Media Client Application To create the c ent app cat on, start M crosoft V sua Stud o 2010, and then create a new C# W ndows Forms app cat on named EventMediaSpatialApp Des gn a form named PhotoForm w th a combobox for se ect ng a reg on, a nk abe to search for photos by reg on, and a st box for d sp ay ng the resu ts A so nc ude a p cture box contro for d sp ay ng the photo se ected from the st box and a nk abe that you’ use to bu k oad four photo fi es nto the EventPhoto tab e After perform ng some aesthet c a gnment and formatt ng, your form shou d appear someth ng ke the one shown n F gure 9-19

Figure 9-19  EventMedia W ndows user nterface (U ) form.

L st ng 9-17 conta ns the comp ete code beh nd the form, wh ch ca s the stored procedures you created n L st ng 9-16 to popu ate the combobox w th reg ons, query for photos taken n any se ected reg on, and then d sp ay any photo se ected from that reg on When a photo s se ected, t streams the mage from the FILESTREAM co umn n the database to the p cture box contro ­us ng SqlFileStream There are a so methods that use SqlFileStream to oad geocoded mages nto the ­database The SqlFileStream code requ res no deta ed exp anat on here; t fo ows the prec se pattern we deta ed n Chapter 8 to save and oad the BLOB mages to and from the database Listing 9-17  EventMedia c ent app cat on code.

using using using using

System; System.Data; System.Data.SqlClient; System.Data.SqlTypes;

394  Part II  Going Beyond Relational

using using using using

System.Drawing; System.IO; System.Transactions; System.Windows.Forms;

namespace EventMediaSpatialApp { public partial class PhotoSearchForm : Form { private const string ConnStr = "Data Source=.;Integrated Security=True;Initial Catalog=EventLibrary;"; public PhotoSearchForm() { InitializeComponent(); } protected override void OnLoad(EventArgs e) { base.OnLoad(e); this.LoadRegions(); } private void lnkAddPhotos_LinkClicked (object sender, LinkLabelLinkClickedEventArgs e) { this.AddPhotos(); } private void lnkSearch_LinkClicked (object sender, LinkLabelLinkClickedEventArgs e) { this.FindRegionPhotos(); } private void lstPhotos_SelectedIndexChanged(object sender, EventArgs e) { this.DisplayPhoto(); } private void LoadRegions() { using (SqlDataAdapter adp = new SqlDataAdapter("GetRegions", ConnStr)) { adp.SelectCommand.CommandType = CommandType.StoredProcedure; DataSet ds = new DataSet(); adp.Fill(ds); this.cboRegions.DataSource = ds.Tables[0]; this.cboRegions.ValueMember = "RegionId"; this.cboRegions.DisplayMember = "RegionName"; } }



Chapter 9  Geospat a Support   395

private void FindRegionPhotos() { this.lstPhotos.SelectedIndexChanged -= new System.EventHandler(this.lstPhotos_SelectedIndexChanged); int regionId = (int)this.cboRegions.SelectedValue; using (SqlDataAdapter adp = new SqlDataAdapter("GetRegionPhotos", ConnStr)) { adp.SelectCommand.CommandType = CommandType.StoredProcedure; adp.SelectCommand.Parameters.AddWithValue("@RegionId", regionId); DataSet ds = new DataSet(); adp.Fill(ds); this.lstPhotos.DataSource = ds.Tables[0]; this.lstPhotos.ValueMember = "PhotoId"; this.lstPhotos.DisplayMember = "Description"; } this.lstPhotos.SelectedIndexChanged += new System.EventHandler(this.lstPhotos_SelectedIndexChanged); this.DisplayPhoto(); } private void DisplayPhoto() { int photoId = (int)this.lstPhotos.SelectedValue; this.picImage.Image = this.GetPhoto(photoId); } private Image GetPhoto(int photoId) { Image photo; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); string filePath; byte[] txnToken; using (SqlCommand cmd = new SqlCommand("GetPhotoForFilestream", conn)) { cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); filePath = rdr.GetSqlString(0).Value; txnToken = rdr.GetSqlBinary(1).Value; rdr.Close(); } }

396  Part II  Going Beyond Relational

photo = this.LoadPhotoImage(filePath, txnToken); } ts.Complete(); } return photo; } private Image LoadPhotoImage(string filePath, byte[] txnToken) { Image photo; using (SqlFileStream sfs = new SqlFileStream(filePath, txnToken, FileAccess.Read)) { photo = Image.FromStream(sfs); sfs.Close(); } return photo; } private void AddPhotos() { this.InsertEventPhoto(1, "Taken from the Ben Franklin parkway near the finish line", -75.17396, 39.96045, "bike9_2.jpg"); this.InsertEventPhoto(2, "This shot was taken from the bottom of the Manayunk Wall", -75.22457, 40.02593, "wall_race_2.jpg"); this.InsertEventPhoto(3, "This shot was taken at the top of the Manayunk Wall.", -75.21986, 40.02920, "wall_race2_2.jpg"); this.InsertEventPhoto(4, "This is another shot from the Benjamin Franklin Parkway.", -75.17052, 39.95813, "parkway_area2_2.jpg"); MessageBox.Show("Added 4 photos to database"); } private void InsertEventPhoto( int photoId, string desc, double longitude, double latitude, string photoFile) { const string InsertTSql = @" INSERT INTO EventPhoto(PhotoId, Description, Location) VALUES(@PhotoId, @Description, geography::STGeomFromText(@Location, 4326)) SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM EventPhoto WHERE PhotoId = @PhotoId";



Chapter 9  Geospat a Support   397

const string PointMask = "POINT ({0} {1})"; string location = string.Format(PointMask, longitude, latitude); string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(InsertTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; cmd.Parameters.Add("@Description", SqlDbType.NVarChar).Value = desc; cmd.Parameters.Add("@Location", SqlDbType.NVarChar).Value = location; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } this.SavePhotoFile(photoFile, serverPath, serverTxn); } ts.Complete(); } } private void SavePhotoFile (string photoFile, string serverPath, byte[] serverTxn) { const string LocalPath = @"..\..\Photos\"; const int BlockSize = 1024 * 512; using (FileStream source = new FileStream(LocalPath + photoFile, FileMode.Open, FileAccess.Read)) { using (SqlFileStream dest = new SqlFileStream(serverPath, serverTxn, FileAccess.Write)) { byte[] buffer = new byte[BlockSize]; int bytesRead; while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) { dest.Write(buffer, 0, bytesRead); dest.Flush(); } dest.Close(); } source.Close();

398  Part II  Going Beyond Relational

} } } }

The code beh nd the Add Photos nk abe c ck event nserts four photos and the r ong tude and at tude coord nates nto the database Two of the photos were taken n the Wa Area, and the other two were taken n the Parkway Area Most phone cameras and many d g ta cameras automat ca y record GPS nformat on w th each d g ta mage fi e (that s, they geocode the mage) You can use the GetPropertyItem method of the Image c ass to extract the geocoded nformat on from these d g ta mages to obta n the ocat on where they were taken Our examp e s mp y uses hard-coded ocat on va ues n the AddPhotos method

Note  As with all code in this book, the full EventMedia demo code in this chapter is ­available on the book’s companion website (see the “Introduction” for details). Included with the code are the four photo files named bike9 2.jpg, wall race 2.jpg, wall race2 2. jpg, and parkway area2 2.jpg. If you do not have access to these images, just use any .jpg ­image files you have available—doing so won’t change the point of the demo. Run the app cat on, and c ck Add Photos to popu ate the EventPhoto tab e Then se ect d fferent reg ons from the drop-down st to test out the query Based on the STIntersects method n the stored ­procedure, the app cat on sts two of the p ctures for the Wa Area, the other two p ctures for the ­Parkway Area, and a four p ctures for the Race Area (wh ch encompasses both of the other reg ons) ­Se ect ng any photo streams the mage from the database nto the p cture box, as shown n F gure 9-20

Figure 9-20  Perform ng ocat on based photo search ng and stream ng.



Chapter 9  Geospat a Support   399

The STDistance Method You use the STDistance method for d stance ca cu at ons By jo n ng the EventPhoto tab e to tse f on PhotoId, you can nvoke th s method aga nst every comb nat on of any two p ctures, as shown n ­L st ng 9-18 The va ue returned by STDistance s the exact d stance between the two p ctures expressed n meters, so d v d ng that va ue by 1000 converts the resu t to k ometers (As ment oned ear er, the defau t SRID for the geography data type uses the metr c system ) Listing 9-18 Us ng STDistance to ca cu ate the d stance between any two photos.

SELECT P1.PhotoId AS Photo1, P2.PhotoId AS Photo2, ROUND(P1.Location.STDistance(P2.location) / 1000, 2) AS Km FROM EventPhoto AS P1 JOIN EventPhoto AS P2 ON P1.PhotoId < P2.PhotoId ORDER BY P1.PhotoId

Here are the query resu ts Photo1 ----------1 1 1 2 2 3

Photo2 ----------2 3 4 3 4 4

Km ---------------------8.46 8.58 0.39 0.54 8.83 8.95

(6 row(s) affected)

Because the jo n s based on P1.PhotoId < P2.PhotoId, you don’t get resu ts for the d stance ­ etween a photo and tse f (wh ch s a ways zero), and you a so fi ter out dup cate “oppos te b ­d rect on” resu ts For examp e, the d stance from photo 1 to photo 3 s the same as the d stance from photo 3 to photo 1, so t doesn’t need to be repeated n the resu t set

Spatial Enhancements in SQL Server 2012 SQL Server 2012 adds many s gn ficant mprovements to the spat a support that was first ­ntroduced w th SQL Server 2008 In th s sect on, we’ exp ore some of these atest enhancements ­Part cu ar y notab e s support for curves (arcs), whereas SQL Server 2008 on y supported stra ght nes, or ­po ygons composed of stra ght nes As you’ see, M crosoft a so prov des methods that test for ­non-2012-compat b e (curved) shapes, and convert c rcu ar data to ne data for backward ­compat b ty w th SQL Server 2008 (as we as other mapp ng p atforms that don’t support curves)

400  Part II  Going Beyond Relational

New Spatial Data Classes The three new shapes n SQL Server 2012 are c rcu ar str ngs, compound curves, and curve po ygons A three are supported n WKT, WKB, and GML by both the geometry and geography data types, and a of the ex st ng methods work on a of the new shapes

Circular Strings A c rcu ar str ng defines a bas c curved ne, s m ar to how a ne str ng defines a stra ght ne It takes a m n mum of three coord nates to define a c rcu ar str ng; the first and th rd coord nates define the end po nts of the ne, and the second coord nate (the “anchor” po nt, wh ch es somewhere ­between the end po nts) determ nes the arc of the ne F gure 9-21 shows the shape represented by ­CIRCULARSTRING(0 1, .25 0, 0 -1)

Figure 9-21  A s mp e c rcu ar str ng.

The code n L st ng 9-19 produces four c rcu ar str ngs A of them have the same start and end po nts, but d fferent anchor po nts The nes are buffered s ght y to make them eas er to see n the spat a v ewer Listing 9-19  C rcu ar str ngs w th d fferent anchor po nts.

-- Create a "straight" circular line SELECT geometry::Parse('CIRCULARSTRING(0 8, 4 UNION ALL -- Curve it SELECT geometry::Parse('CIRCULARSTRING(0 8, 4 UNION ALL -- Curve it some more SELECT geometry::Parse('CIRCULARSTRING(0 8, 4 UNION ALL -- Curve it in the other direction SELECT geometry::Parse('CIRCULARSTRING(0 8, 4

0, 8 -8)').STBuffer(.1) 4, 8 -8)').STBuffer(.1) 6, 8 -8)').STBuffer(.1) -6, 8 -8)').STBuffer(.1)

F gure 9-22 shows the generated shapes from the prev ous code The first shape s a “stra ght c­ rcu ar” ne, because the anchor po nt s pos t oned d rect y between the start and end po nts The next two shapes use the same end po nts w th the anchor out to the r ght (4), one a b t further than



Chapter 9  Geospat a Support   401

the other (6) The ast shape a so uses the same end po nts, but spec fies an anchor po nt that curves the ne to the eft rather than the r ght (-6)

Figure 9-22  A set of c rcu ar str ngs, nc ud ng one that s perfect y stra ght.

You can extend c rcu ar str ngs w th as many curve segments as you want Do th s by defin ng another two coord nates for each add t ona segment The ast po nt of the prev ous curve serves as the first end po nt of the next curve segment, so the two add t ona coord nates respect ve y spec fy the next segment’s anchor and second end po nt Thus, va d c rcu ar str ngs w a ways have an odd number of po nts You can extend a c rcu ar str ng ndefin te y to form curves and arcs of any k nd It’s easy to form a perfect c rc e by connect ng two sem -c rc e segments For examp e, F gure 9-23 shows the c rc e produced by CIRCULARSTRING(0 4, 4 0, 8 4, 4 8, 0 4). Th s part cu ar examp e ­connects the end of the second segment to the beg nn ng of the first segment to form a c osed shape Note that th s s certa n y not requ red of c rcu ar str ngs (or ne str ngs), and that c os ng the shape by connect ng the ast segment to the first st does not resu t n a po ygon, wh ch s a ­t wo-d mens ona shape that has area Desp te be ng c osed, th s c rc e s st cons dered a one-­ d mens ona shape w th no area As you’ soon see, the curve po ygon can be used to convert c osed ne shapes nto true po ygons

Figure 9-23  Two sem c rc e segments nked together n one c rcu ar str ng.

402  Part II  Going Beyond Relational

Compound Curves A compound curve s a set of c rcu ar str ngs, or c rcu ar str ngs comb ned w th ne str ngs, that form a des red curved shape The end po nt of each e ement n the co ect on must match the start ng po nt of the fo ow ng e ement, so that compound curves are defined n a “connect-the-dots” fash on The code n ­L st ng 9-20 produces a compound curve and compares t w th the equ va ent geometry ­co ect on shape Listing 9-20  Compound curve shapes.

-- Compound curve DECLARE @CC geometry = ' COMPOUNDCURVE( (4 4, 4 8), CIRCULARSTRING(4 8, 6 10, 8 8), (8 8, 8 4), CIRCULARSTRING(8 4, 2 3, 4 4) )' -- Equivalent geometry collection DECLARE @GC geometry = ' GEOMETRYCOLLECTION( LINESTRING(4 4, 4 8), CIRCULARSTRING(4 8, 6 10, 8 8), LINESTRING(8 8, 8 4), CIRCULARSTRING(8 4, 2 3, 4 4) )' -- They both render the same shape in the spatial viewer SELECT @CC.STBuffer(.5) UNION ALL SELECT @GC.STBuffer(1.5)

Th s code creates a keyho e shape us ng a compound curve, and a so creates an dent ca shape as a geometry co ect on (though not ce that the LINESTRING keyword s not—and cannot—be spec fied when defin ng a compound curve) It then buffers both of them w th d fferent padd ng, so that the spat a v ewer c ear y shows the two dent ca shapes on top of one another, as shown n F gure 9-24

Figure 9-24  A compound curve over an dent ca y shaped geometry co ect on.



Chapter 9  Geospat a Support   403

Both the compound curve and the geometry co ect on y e d dent ca shapes In fact, the ­express on @CC.STEquals(@GC), wh ch compares the two nstances for equa ty, returns 1 (for true) The STEquals method tests for “spat a equa ty,” mean ng t returns true f two nstances produce the same shape even f they are be ng rendered us ng d fferent spat a data c asses Furthermore, reca from L st ng 9-19 and F gure 9-22 that segments of a c rcu ar str ng can be made perfect y stra ght by pos t on ng the anchor d rect y between the end po nts, mean ng that the c rcu ar str ng offers yet a th rd opt on for produc ng the very same shape So wh ch one shou d you use? Compar ng these spat a data c asses w he p you determ ne wh ch one s best to use n d fferent scenar os A geometry co ect on (wh ch was a ready supported n SQL Server 2008) s the most accommodat ng, but carr es the most storage overhead Geometry co ect ons can ho d nstances of any spat a data c ass, and the nstances don’t need to be connected to (or ntersected w th) each other n any way The ­co ect on s mp y ho ds a bunch of d fferent shapes as a set, wh ch n th s examp e just happens to be severa ne str ngs and c rcu ar str ngs connected at the r start and end po nts In contrast, the new compound curve c ass n SQL Server 2012 has the most constra nts but s the most ghtwe ght n terms of storage It can only conta n ne str ngs or c rcu ar str ngs, and each s­ egment’s start po nt must be connected to the prev ous segment’s end po nt (a though t s most certa n y not ­necessary to connect the first and ast segments to form a c osed shape as n th s e ­ xamp e) The DATALENGTH ­funct on shows the d fference n storage requ rements; DATALENGTH(@CC) returns 152 and DATALENGTH(@GC) returns 243 Th s means that the same shape requ res 38% ess storage space by us ng a compound curve nstead of a geometry co ect on A compound curve s a so more storage-effic ent than a mu t -segment c rcu ar ne str ng when stra ght nes are nvo ved Th s s because there s overhead for the mere potent a of a curve, because the anchor po nt requ res storage even when ts pos t on produces stra ght nes, whereas compound curves are opt m zed spec fica y to connect c rcu ar str ngs and (a ways stra ght) ne str ngs

Curve Polygons A curve po ygon s very s m ar to an ord nary po ygon; ke an ord nary po ygon, a curve po ygon spec fies a “r ng” that defines a c osed shape, and can a so spec fy add t ona nner r ngs to define “ho es” ns de the shape The on y fundamenta d fference between a po ygon and a curve ­po ygon s that the r ngs of a curve po ygon can nc ude c rcu ar shapes, whereas an ord nary po ygon s composed exc us ve y w th stra ght nes Spec fica y, each r ng n a curve po ygon can cons st of any comb nat on of ne str ngs, c rcu ar str ngs, and compound curves that co ect ve y define the c osed shape For examp e, the code n L st ng 9-21 produces a curve po ygon w th the same keyho e out ne that we just demonstrated for the compound curve Listing 9-21  A curve po ygon.

-- Curve polygon SELECT geometry::Parse(' CURVEPOLYGON( COMPOUNDCURVE( (4 4, 4 8),

404  Part II  Going Beyond Relational

CIRCULARSTRING(4 8, 6 10, 8 8), (8 8, 8 4), CIRCULARSTRING(8 4, 2 3, 4 4) ) )')

Th s code has s mp y spec fied the same compound curve as the c osed shape of a curve ­po ygon A though the shape s the same, the curve po ygon s a two-d mens ona object, whereas the ­compound curve vers on of the same shape s a one-d mens ona object Th s can be seen v sua y by the spat a v ewer resu ts, wh ch shades the nter or of the curve po ygon as shown n F gure 9-25

Figure 9-25  A curve po ygon.

New Spatial Methods Let’s now exp ore some of the new spat a methods n SQL Server 2012 Some of these new methods comp ement the new curved shapes, whereas others add new spat a features that work w th a shapes

The STNumCurves and STCurveN Methods These two methods can be nvoked on any geometry or geography nstance They can be used together to d scover nformat on about the curves conta ned w th n the spat a nstance The ­STNumCurves method returns the tota number of curves n the nstance You can then pass any number between 1 and what STNumCurves returns to extract each nd v dua curve, and thus terate a the curves n the nstance L st ng 9-22 demonstrates these two methods w th the same c rc e that you saw n F gure 9-23 Reca that th s s a c rcu ar str ng w th two connected segments The c rc e s defined by the two ha ves, each of wh ch s a curved ne defin ng a sem -c rc e Listing 9-22 Us ng STNumCurves and STCurveN to obta n curve nformat on from a c rcu ar str ng.

-- Create a full circle shape (two connected semi-circles) DECLARE @C geometry = 'CIRCULARSTRING(0 4, 4 0, 8 4, 4 8, 0 4)'



Chapter 9  Geospat a Support   405

-- Get the curve count (2) and the 1st curve (bottom semi-circle) SELECT CurveCount = @C.STNumCurves(), SecondCurve = @C.STCurveN(2), SecondCurveWKT = @C.STCurveN(2).ToString()

Th s query produces the fo ow ng output CurveCount SecondCurve SecondCurveWKT ---------- ----------------------------------------------- ------------------------------2 0x000000000204030000000000000000002040000000... CIRCULARSTRING (8 4, 4 8, 0 4)

You can see that STNumCurves nd cates there are two curves, and that STCurveN(2) returns the second curve If you v ew the resu ts n the spat a v ewer, you’ see just the top ha f of the c rc e Th s s the sem -c rc e defined by the second curve, wh ch s converted back to WKT as CIRCULARSTRING (8 4, 4 8, 0 4) Not ce that th s represents the second segment of the fu c rc e n L st ng 9-21

The BufferWithCurves Method You’ve a ready earned how the STBuffer method can “pad” a ne, effect ve y convert ng t nto a ­po ygon To repr se our “sma town” demo, STBuffer was used n L st ng 9-5 to w den the town streets, and the output from that code st ng c ear y showed that SQL Server generated po ygons from the ne str ngs If you ook c ose y at the resu t ng po ygon shapes n the spat a v ewer (see F gure 9-6), t appears that po nts of each ne str ng ( nc ud ng the m dpo nts) are transformed nto rounded edges n the po ygon However, the rounded edge ook s actua y produced by p ott ng many short stra ght nes that are c ustered very c ose y together, present ng the us on of a curve Th s approach s ­because curves were not prev ous y supported before SQL Server 2012 (but the STBuffer method was) C ear y, us ng nat ve curve defin t ons n a curve po ygon s more effic ent than c uster ng a ­mu t tude of stra ght nes n an ord nary po ygon For backward compat b ty, STBuffer ­cont nues to return the ( neffic ent) po ygon as before So SQL Server 2012 ntroduces a new method, ­BufferWithCurves, for th s purpose The code n L st ng 9-23 uses BufferWithCurves to pad the street nes as before, and compares the resu t w th ts stra ght- ne cous n, STBuffer Listing 9-23 Compar ng BufferWithCurves and STBuffer.

DECLARE @streets geometry = ' GEOMETRYCOLLECTION( LINESTRING (100 -100, 20 -180, 180 -180), LINESTRING (300 -300, 300 -150, 50 -50) )' SELECT @streets.BufferWithCurves(10) SELECT AsWKT = @streets.ToString(),

406  Part II  Going Beyond Relational

Bytes = DATALENGTH(@streets), Points = @streets.STNumPoints() UNION ALL SELECT @streets.STBuffer(10).ToString(), DATALENGTH(@streets.STBuffer(10)), @streets.STBuffer(10).STNumPoints() UNION ALL SELECT @streets.BufferWithCurves(10).ToString(), DATALENGTH(@streets.BufferWithCurves(10)), @streets.BufferWithCurves(10).STNumPoints()

F gure 9-26 shows the resu t ng shape (the co ect on of padded street shapes) returned by the first SELECT statement n the spat a v ewer

Figure 9-26 Us ng BufferWithCurves to pad a ne str ng w th true rounded edges.

As w th STBuffer back n F gure 9-6, the new shapes have rounded edges around the po nts of the or g na ne str ngs However, BufferWithCurves generates actua curves and, thus, produces a ­s gn ficant y sma er and s mp er po ygon The second SELECT statement n L st ng 9-23 demonstrates th s by compar ng the three shapes—the or g na ne str ng co ect on, the po ygon returned by ­STBuffer, and the curve po ygon returned by BufferWithCurves Here are the resu ts AsWKT -------------------------------------------------------------------------GEOMETRYCOLLECTION (LINESTRING (100 -100, 20 -180, 180 -180), LINESTRIN... MULTIPOLYGON (((20.000000000000796 -189.99999999999858, 179.99999999999... GEOMETRYCOLLECTION (CURVEPOLYGON (COMPOUNDCURVE ((20.000000000000796 -1...

Bytes ----151 5207 693

Points -----6 322 38

The first shape s the or g na geometry co ect on of streets ( ne str ngs) used for nput, wh ch requ res on y 151 bytes of storage, and has on y 6 po nts For the second shape, STBuffer pads the ne str ngs to produce a mu t -po ygon (a set of po ygons) that consumes 5,207 bytes and has a tota of 322 po nts—a whopp ng 3,448 percent ncrease from the or g na ne str ngs In the th rd shape, BufferWithCurves s used to produce the equ va ent padd ng us ng a co ect on of curve po ygons



Chapter 9  Geospat a Support   407

composed of compound curves, so t consumes on y 693 bytes and has on y 38 po nts—a (re at ve y) mere 458 percent ncrease from the or g na ne str ngs

The ShortestLineTo Method Th s new method exam nes any two shapes and figures out the shortest ne between them L st ng 9-24 demonstrates th s Listing 9-24  F nd ng the shortest d stance between two shapes us ng ShortestLineTo.

DECLARE @Shape1 geometry = ' POLYGON ((-20 -30, -3 -26, 14 -28, 20 -40, -20 -30))' DECLARE @Shape2 geometry = ' POLYGON ((-18 -20, 0 -10, 4 -12, 10 -20, 2 -22, -18 -20))' SELECT @Shape1 UNION ALL SELECT @Shape2 UNION ALL SELECT @Shape1.ShortestLineTo(@Shape2).STBuffer(.25)

Th s code defines two po ygons and then uses ShortestLineTo to determ ne, generate, and return the shortest stra ght ne that connects them STBuffer s a so used to pad the ne str ng so that t s more c ear y v s b e n the spat a v ewer, as shown n F gure 9-27

Figure 9-27  Us ng the ShortestLineTo method to ca cu ate the shortest stra ght ne between two shapes.

The MinDbCompatibilityLevel Method W th the added support for curves n SQL Server 2012 comes support for backward c­ ompat b ty w th prev ous vers ons of SQL Server (2008 and 2008 R2) that don’t support curves The new ­MinDbCompatibilityLevel method accepts any WKT str ng and returns the m n mum vers on of SQL Server requ red to support the shape defined by that str ng For examp e, cons der the code n L st ng 9-25

408  Part II  Going Beyond Relational

Listing 9-25  Test ng for vers on compat b ty w th the MinDbCompatibilityLevel method.

DECLARE @Shape1 geometry = 'CIRCULARSTRING(0 50, 90 50, 180 50)' DECLARE @Shape2 geometry = 'LINESTRING (0 50, 90 50, 180 50)' SELECT Shape1MinVersion = @Shape1.MinDbCompatibilityLevel(), Shape2MinVersion = @Shape2.MinDbCompatibilityLevel()

The MinDbCompatibilityLevel method returns 110 (referr ng to vers on 11 0) for the first WKT str ng and 100 (vers on 10 0) for the second one Th s s because the first WKT str ng conta ns a ­c rcu ar str ng, wh ch requ res SQL Server 2012 (vers on 11 0), whereas the ne str ng n the second WKT str ng s supported by SQL Server 2008 (vers on 10 0) and h gher

The STCurveToLine and CurveToLineWithTolerance Methods These are two methods you can use to convert curves to rough y equ va ent stra ght ne shapes Aga n, th s s to prov de compat b ty w th prev ous vers ons of SQL Server and other mapp ng ­p atforms that don’t support curves The STCurveToLine method converts a s ng e curve to a ne str ng w th a mu t tude of segments and po nts that best approx mate the or g na curve The techn que s s m ar to what we just d scussed for STBuffer, where many short stra ght nes are connected n a c uster of po nts to s mu ate a curve And, as exp a ned n that d scuss on, the resu t ng ne str ng requ res s gn ficant y more storage than the or g na curve To offer a comprom se between fide ty and storage, the CurveToLineWithTolerance method accepts “to erance” parameters to produce ne str ngs that consume ess storage space than those produced by STCurveToLine The code n L st ng 9-26 demonstrates th s by us ng both methods to convert the same c rc e from F gure 9-23 nto ne str ngs Listing 9-26  Convert ng curves to nes w th STCurveToLine and CurveToLineWithTolerance.

-- Create a full circle shape (two connected semi-circles) DECLARE @C geometry = 'CIRCULARSTRING(0 4, 4 0, 8 4, 4 8, 0 4)' -- Render as curved shape SELECT Shape = @C, ShapeWKT = @C.ToString(), ShapeLen = DATALENGTH(@C), Points = @C.STNumPoints() -- Convert to lines (much larger, many more points) SELECT Shape = @C.STCurveToLine(), ShapeWKT = @C.STCurveToLine().ToString(), ShapeLen = DATALENGTH(@C.STCurveToLine()), Points = @C.STCurveToLine().STNumPoints()



Chapter 9  Geospat a Support   409

-- Convert to lines with tolerance (not as much larger, not as many more points) SELECT Shape = @C.CurveToLineWithTolerance(0.1, 0), ShapeWKT = @C.CurveToLineWithTolerance(0.1, 0).ToString(), ShapeLen = DATALENGTH(@C.CurveToLineWithTolerance(0.1, 0)), Points = @C.CurveToLineWithTolerance(0.1, 0).STNumPoints()

The query resu ts show that the or g na c rc e consumes on y 112 bytes and has 5 po nts Invok ng STCurveToLine on the c rc e converts t nto a ne str ng that consumes 1,072 bytes and has 65 po nts That’s a b g ncrease, but the resu t ng ne str ng represents the or g na c rc e n h gh fide ty; you w not see a percept b e d fference n the two when v ew ng them us ng the spat a v ewer However, the ne str ng produced by CurveToLineWithTolerance consumes on y 304 bytes and has on y 17 po nts; a s gn ficant y sma er footpr nt, pa d for w th a not ceab e oss n fide ty As shown by the spat a v ewer resu ts n F gure 9-28, us ng CurveToLineWithTolerance produces a c rc e made up of v s b y stra ght ne segments

Figure 9-28  F de ty oss convert ng a curve to a ne us ng CurveToLineWithTolerance.

The STIsValid, IsValidDetailed, and MakeValid Methods Spat a nstance va dat on has mproved great y n SQL Server 2012 The STIsValid method ­eva uates a spat a nstance and returns a 1 (for true) or 0 (for fa se), nd cat ng f the nstance represents a va d shape (or shapes) As L st ng 9-27 demonstrates, f the nstance s nva d, the new IsValidDetailed method w return a str ng exp a n ng the reason why Listing 9-27 Us ng IsValidDetailed to determ ne why a geometry nstance s nva d.

DECLARE @line geometry = 'LINESTRING(1 1, 2 2, 3 2, 2 2)' SELECT IsValid = @line.STIsValid(), Details = @line.IsValidDetailed()

410  Part II  Going Beyond Relational

Th s ne str ng s nva d because the same po nt (2 2) s repeated, wh ch resu ts n “over app ng edges,” as revea ed by the output from IsValidDetailed IsValid ------0

Details ------------------------------------------------------------------24413: Not valid because of two overlapping edges in curve (1).

SQL Server 2012 s more to erant of nva d spat a nstances than prev ous vers ons For examp e, you can now perform metr c operat ons (such as STLength) on nva d nstances, a though you st won’t be ab e to perform other operat ons (such as STBuffer) on them The new MakeValid method can “fix” an nva d spat a nstance and make t va d Of course, the shape w sh ft s ght y, and there are no guarantees on the accuracy or prec s on of the changes made The code n L st ng 9-28 uses MakeValid to remove over app ng parts (wh ch can be caused by anoma es such as naccurate GPS traces), effect ve y convert ng the nva d ne str ng from L st ng 9-27 nto a va d spat a nstance Listing 9-28 Us ng MakeValid to convert an nva d ne str ng to a va d spat a nstance.

DECLARE @line geometry = 'LINESTRING(1 1, 2 2, 3 2, 2 2)' SELECT @line.MakeValid().ToString() AS Fixed

The WKT str ng returned by the SELECT statement shows the “fixed” ne str ng Fixed ------------------------------------------------------------------LINESTRING (3 2, 2 2, 1.0000000000000071 1.0000000000000036)

Other Enhancements The rema nder of th s sect on g ves br ef ment on to severa other noteworthy spat a enhancements added n SQL Server 2012 These nc ude better geography support, and prec s on and opt m zat on mprovements

Note  Full coverage of these enhancements (and others, such as aggregates, histograms, the nearest neighbor query plan, spatial indexes, hints, and performance improvements) is beyond the scope of this chapter. Consult SQL Server Books Online for the complete spatial reference.

Support for geography Instances Exceeding a Logical Hemisphere Prev ous vers ons of SQL Server supported geography objects as arge as (s ght y ess than) a ­og ca hem sphere (ha f the g obe) Th s m tat on has been removed n SQL Server 2012, wh ch now ­supports geography nstances of any s ze (even the ent re p anet)

Chapter 9  Geospat a Support   411

When you define a geography po ygon, the order n wh ch you spec fy the r ng’s at tude and ong tude coord nates (known as vertex order) s s gn ficant (un ke geometry, where vertex order s ns gn ficant) The coord nate po nts are a ways defined accord ng to the left-foot inside ru e; when you “wa k” the boundary of the po ygon, your eft foot s on the ns de Thus, vertex order determ nes whether you are defin ng a sma p ece of the g obe, re at ve to the arger p ece defined by the ent re g obe except for the sma p ece (that s, the rest of the g obe) Because prev ous vers ons of SQL Server were m ted to ha f the g obe, t was mposs b e to spec fy the po nts of a po ygon n the “wrong order,” s mp y because do ng so resu ted n too arge a shape (and thus, ra sed an error) That error potent a no onger ex sts n SQL Server 2012, so t’s even more cr t ca to make sure your vertex order s correct, or you’ be unw tt ng y work ng w th the exact ­“oppos te” shape If you have a geography nstance that s known to have the wrong vertex order, you can repa r t us ng the new ReorientObject method Th s method operates on y on po ygons ( t has no effect on po nts, ne str ngs, or curves), and can be used to correct the r ng or entat on (vertex order) of the po ygon, as L st ng 9-29 demonstrates Listing 9-29 Us ng ReorientObject to change the vertex order.

-- Small (less than a logical hemisphere) polygon SELECT geography::Parse('POLYGON((-10 -10, 10 -10, 10 10, -10 10, -10 -10))') -- Reorder in the opposite direction for "rest of the globe" SELECT geography::Parse('POLYGON((-10 -10, -10 10, 10 10, 10 -10, -10 -10))') -- Reorient back to the small polygon SELECT geography::Parse('POLYGON((-10 -10, -10 10, 10 10, 10 -10, -10 -10))') .ReorientObject()

Three geography po ygon nstances are defined n th s code The first geography nstance defines a very sma po ygon The second nstance uses the exact same coord nates, but because the vertex ­order s reversed, t defines an enormous po ygon whose area represents the ent re g obe except for the sma po ygon As exp a ned, such a defin t on wou d cause an error n prev ous vers ons of SQL Server, but s now accommodated w thout a prob em by SQL Server 2012 The th rd nstance ­reverses the vertex order on the same shape as the second nstance, thereby produc ng the same sma ­po ygon as the first nstance

Full Globe Support A ong w th the aforement oned support for geography nstances to exceed a s ng e og ca ­hem sphere comes a new spat a data c ass ca ed FULLGLOBE As you may have guessed, th s s a shape that represents the ent re p anet If you’ve ever wondered how many square meters there are n the ent re wor d, the query n L st ng 9-30 g ves you the answer (wh ch s 510,065,621,710,996 square meters, so you can stop wonder ng)

412  Part II  Going Beyond Relational

Listing 9-30 Us ng FULLGLOBE to ca cu ate the area of the ent re earth.

-- Construct a new FullGlobe object (a WGS84 ellipsoid) DECLARE @Earth geography = 'FULLGLOBE' -- Calculate the area of the earth SELECT PlanetArea = @Earth.STArea()

A of the common spat a methods work as expected on a fu g obe object So you cou d, for examp e, “cut away” at the g obe by nvok ng the STDifference and STSymDifference method aga nst t us ng other po ygons as cook e-cutter shapes

New “Unit Sphere” Spatial Reference ID Ear er we noted that the defau t SRID s 4326, wh ch uses the metr c system as ts un t of ­measurement Th s SRID a so represents the true e pso da sphere shape of the earth A though th s ­representat on s most accurate, t’s a so more comp ex to ca cu ate prec se e pso da mathemat cs SQL Server 2012 offers a comprom se n speed and accuracy, by add ng a new spat a reference d (SRID), 104001, wh ch uses a sphere of rad us 1 to represent a perfectly round earth You can create geography nstances w th SRID 104001 when you don’t requ re the greatest ­accuracy The STDistance, STLength, and ShortestLineTo methods are opt m zed to run faster on the un t sphere, because t takes a re at ve y s mp e formu a to compute measures aga nst a perfect y round sphere (compared to an e pso da sphere)

Better Precision Interna spat a ca cu at ons n SQL Server 2012 are now performed w th 48 b ts of prec s on, compared to 27 b ts used n SQL Server 2008 and SQL Server 2008 R2 Th s can reduce the error caused by round ng of float ng po nt coord nates for or g na vertex po nts by the nterna computat on

Integrating with Microsoft Bing Maps We’ end th s chapter w th one fina (and fun) geospat a app cat on, n wh ch you w construct a ­M crosoft B ng Maps mash-up The term “mash-up” s used to descr be a un fied presentat on ­comb n ng one set of data w th another For th s mash-up, you w create a web app cat on that ­ayers customer coord nates over B ng Maps magery to show the ocat on of customers n the ­database as pushp n cons on a map Start w th a new database Create the Customer tab e w th a geography co umn to ho d customer ocat ons and popu ate t w th some samp e data, as shown n L st ng 9-31



Chapter 9  Geospat a Support   413

Listing 9-31  Creat ng the Customer tab e and popu at ng t w th geograph ca data.

USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'MyDB') DROP DATABASE MyDB GO CREATE DATABASE MyDB GO USE MyDB GO CREATE TABLE Customer (CustomerId int PRIMARY KEY, Name varchar(50), Company varchar(50), CustomerGeo geography) GO INSERT INTO Customer VALUES (1, 'Adam', 'Coho Vineyard & Winery', 'POINT(-111.06687 45.01188)'), (2, 'John', 'ACME Corp.', 'POINT(-104.06 41.01929)'), (3, 'Paul', 'Litware, Inc.', 'POINT(-111.05878 41.003)'), (4, 'Joel', 'Tailspin Toys', 'POINT(-121.05878 41.003)'), (5, 'Martin', 'ABC Travel', 'POINT(-110.05878 43.003)'), (6, 'Remon', 'Wingtip Toys', 'POINT(-113.05878 35.003)'), (7, 'Jason', 'School of Fine Art', 'POINT(-116.05878 34.003)'), (8, 'Fred', 'Fourth Coffee', 'POINT(-114.05878 43.003)')

Now create a stored procedure to retr eve a the customers and the r ocat ons, as shown n L st ng 9-32 Listing 9-32  Creat ng a stored procedure to retr eve customers and the r ocat ons.

CREATE PROCEDURE GetCustomers AS BEGIN SELECT Name, Company, CustomerGeo FROM Customer END GO

W th the database set up, you’re ready to create your B ng Maps mash-up Start V sua Stud o, and then choose F e New Project From the temp ates sted under V sua C#, Web, create an ASP NET Empty Web App cat on project named BingMapsSpatialApp, as shown n F gure 9-29

414  Part II  Going Beyond Relational

Figure 9-29  Creat ng the BingMapsSpatialApp project.

In Default.aspx, rep ace the starter markup prov ded by V sua Stud o w th the code shown n ­L st ng 9-33 Listing 9-33  Creat ng a V rtua Earth mash up w th geography data.











Th s webpage uses Asynchronous JavaScr pt and XML (AJAX) to render a map us ng M crosoft B ng Maps, upon wh ch t draws pushp n cons for each customer There are abso ute y no page postbacks to the server; a the serv ce ca s are made d rect y from the c ent browser Let’s d ssect th s page carefu y Every AJAX-enab ed webpage requ res a ScriptManager e ement, wh ch you’ve p aced at the top of the form Nested w th n the ScriptManager e ement, two serv ce references are dec ared The first ­reference po nts to CustomerQueryService.asmx, wh ch s a W ndows Commun cat ons Foundat on (WCF) Serv ce that you w create momentar y to ca the stored procedure that retr eves ­customers and the r ocat ons The second reference s to the M crosoft B ng Maps web serv ces app cat on programm ng nterface (API) at dev.virtualearth.net Th s API prov des the B ng Maps Ajax Contro that can be used to render and man pu ate a map on any webpage The map renders w th a UI that prov des zoom and pan contro s s mp y by c ck ng and dragg ng the mouse

Note  The script reference indicates the name Virtual Earth, which is the original name for Bing Maps. Currently, the Bing Maps Ajax Control API still reflects the old name by u ­ sing objects prefixed with VE. Thus, we interchange the terms “Bing Maps” and “Virtual Earth” throughout our discussion of the code. After the ScriptManager, the page dec ares a sect on that defines the rectangu ar area of the page (640 by 400 p xe s n th s examp e) n wh ch the map shou d be d sp ayed The e ement s 416  Part II  Going Beyond Relational

ass gned an ID named divBingMap that w be referenced ater by a ca nto the B ng Maps API when the map s created Beneath the map, the page d sp ays a button named btnGetCustomers that s w red to a c ent-s de event hand er named btnGetCustomers click() The rest of the page conta ns the JavaScr pt that nteracts w th the two serv ces from the user’s browser At the top of the scr pt, a page- eve var ab e named map s dec ared Th s var ab e w ho d a reference to the VEMap (V rtua Earth map) object created n the pageLoad funct on, wh ch fires on the c ent when the browser oads the page The pageLoad funct on nstant ates the map v­ ar ab e by dec ar ng t as a new VEMap object Th s resu ts n a ca to the B ng Maps web serv ce that returns a VEMap object that s stored n the map var ab e The object s bound to the tag named divBingMap (dec ared ear er n the markup) that was passed n as a parameter nto the VEMap ­constructor As a resu t, the map oads and d sp ays tse f n the divBingMap sect on of the page when the page oads Next, the button’s c ck event hand er s defined n the btnGetCustomers click() funct on The s ng e ne of code n th s funct on ca s your own CustomerQueryService.asmx web serv ce (wh ch you’re about to create), dec ared n the preced ng ScriptManager sect on Th s resu ts n an asynchronous ca to the serv ce, wh ch runs n the background to ca your stored procedure and retr eve customers and the r ocat ons from the database When the ca s made, the page spec fies the name of the ca back funct on OnDataRetrievalComplete to be nvoked when the resu ts are returned by the serv ce F na y, the OnDataRetrievalComplete funct on s defined to rece ve and process the resu ts of the c­ ustomer query, wh ch s a st of customers and the r ocat ons that s passed to the funct on as a p ­ arameter named results The st s processed as an array of objects terated w th a for oop Each e ement n the array s an object w th propert es defined for the Customer c ass (wh ch s a WCF data contract that you’re about to create w th the serv ce) For each customer, the Latitude and Longitude propert es are used to construct a VELatLong (V rtua Earth at tude/ ong tude) object stored n a var ab e named point (Note that the VELatLong constructor requ res the at tude va ue for the first ­parameter and the ong tude va ue for the second parameter, un ke WKT, wh ch uses the reverse ­order of ong tude fo owed by at tude ) A new VEShape (V rtua Earth shape) object s then created for the po nt as a VEShapeType.Pushpin shape stored n the var ab e pin The Company and Name propert es are then used to set the pushp n’s t t e and descr pt on us ng the SetTitle and ­SetDescription methods The pushp n s then added to the map by ­ nvok ng the AddShape method on the map ­var ab e, wh ch represents the map d sp ayed n the tag named divVirtualEarthMap on the page That comp etes the webpage Your next step s to create the WCF Serv ce ca ed by the page to retr eve your customers and the r ocat ons from the database In So ut on Exp orer, r ght-c ck the VirtualEarthSpatialApp project, and se ect Add New Item In the nsta ed temp ates sted under Web, WCF Serv ce, name the serv ce CustomerGeoService.svc, as shown n F gure 9-30



Chapter 9  Geospat a Support   417

Figure 9-30  Creat ng the customer query web serv ce.

To keep the examp e s mp e, you won’t separate nterface from mp ementat on Therefore, you can de ete the ICustomerGeoService.cs fi e that V sua Stud o created for you to define the nterface Then, open the serv ce’s CustomerGeoService.svc.cs code-beh nd fi e and rep ace the starter code that V sua Stud o prov ded w th the code shown n L st ng 9-34 Listing 9-34  mp ement ng a WCF Serv ce to retr eve customer spat a data.

using using using using using using using

System; System.Collections.Generic; System.Data; System.Data.SqlClient; System.Runtime.Serialization; System.ServiceModel; System.ServiceModel.Activation;

using Microsoft.SqlServer.Types; namespace BingMapsSpatialApp { [DataContract] public class Customer { [DataMember] public string Name { get; set; } [DataMember] public string Company { get; set; } [DataMember] public double Latitude { get; set; }

[DataMember]

418  Part II  Going Beyond Relational

public double Longitude { get; set; } } [ServiceContract(Namespace = "BingMapsSpatialApp")] [AspNetCompatibilityRequirements (RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)] public class CustomerGeoService { private const string ConnStr = "Data Source=.;Initial Catalog=MyDb;Integrated Security=True;"; [OperationContract] public List GetCustomers() { var customers = new List(); using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = "GetCustomers"; cmd.CommandType = CommandType.StoredProcedure; using (var rdr = cmd.ExecuteReader()) { while (rdr.Read()) { var customer = new Customer(); customer.Name = rdr["Name"].ToString(); customer.Company = rdr["Company"].ToString(); // Get the CustomerGeo column bytes into a geography instance var geo = SqlGeography.Deserialize(rdr.GetSqlBytes(2)); customer.Latitude = (double)geo.Lat; customer.Longitude = (double)geo.Long; customers.Add(customer); } rdr.Close(); } } conn.Close(); return customers; } } } }



Chapter 9  Geospat a Support   419

The Customer data contract c ass s defined first, and s mp y exposes propert es to define the object that gets returned to the webpage Then the CustomerGeoService serv ce contract c ass s defined, wh ch n turn defines the GetCustomers operat on contract ca ed by the webpage The GetCustomers method extracts the at tude and ong tude va ues from the geography typed Location co umn returned by the GetCustomers stored procedure that you created n L st ng 9-32 In M crosoft NET, you can work w th the SQL Server system CLR types (such as geography) by us ng c asses defined n the Microsoft.SqlServer.Types namespace Th s namespace s dec ared by a using statement at the top of the code (h gh ghted n bo d) The SqlGeography c ass n th s namespace corresponds to the geography data type n SQL Server The Lat and Long propert es of a SqlGeography object expose the at tude and ong tude va ues conta ned n the geography nstance t encapsu ates (as double data types) Before us ng any types n th s namespace, you must first estab sh a reference to ts assemb y To do so, r ght-c ck the BingMapsSpatialApp project n So ut on Exp orer, and then se ect Add Reference On the NET tab, scro down and se ect Microsoft.SqlServer.Types, as shown n F gure 9-31, and then c ck OK

Figure 9-31  Add ng a reference to the Microsoft.SqlServer.Types assemb y.

After open ng a connect on to the database, the GetCustomers method prepares a SqlCommand object to ca the GetCustomers stored procedure n the database and then nvokes the ExecuteReader method to obta n a SqlDataReader object that returns the resu ts The method oops through a the rows n the reader returned by the stored procedure and popu ates a gener c List object w th the resu ts For each row, a Customer object s created and popu ated w th the Name and ­Company co umns, as we as ocat on nformat on extracted from the CustomerGeo co umn The ­ocat on va ue s obta ned by ca ng the stat c Deserialize method on the SqlGeography c ass, pass ng n the bytes from the CustomerGeo co umn of the reader, and rece v ng back a geography nstance The bytes are retr eved by ca ng the reader’s GetSqlBytes method, pass ng n the zero-based ndex of the des red co umn ( n th s case 2 refers to the th rd co umn, CustomerGeo) Then the customer’s Latitude and Longitude propert es (double va ues) are popu ated from the geography nstance’s Lat 420  Part II  Going Beyond Relational

and Long propert es, respect ve y The Customer object s then added to the gener c List object that s returned to the ca er after a the customers have been processed Your next and fina step s to configure the serv ce for asynchronous ca s, wh ch s requ red to support Ajax c ent app cat ons ke the one you are bu d ng Spec fica y, you need to ed t Web.config and define a custom behav or for the serv ce endpo nt to enab e web scr pt ng ca s, as shown n L st ng 9-35 Listing 9-35  Configur ng the WCF Serv ce for Ajax c ents.















After mod fy ng Web.config, you can g ve the app cat on a run To v ew the webpage, r ght-c ck Default.aspx n So ut on Exp orer, and choose Set as Start Page Then press F5 to run the a ­ pp cat on When the page oads, t runs the c ent-s de JavaScr pt you wrote n the pageLoad funct on and d sp ays the map n the browser C ck the Get Customers button to nvoke the database query and d sp ay the pushp ns correspond ng to the ocat ons of the customers returned by the query, as shown n F gure 9-32



Chapter 9  Geospat a Support   421

Figure 9-32  Runn ng the B ng Maps mash up aga nst geography data n SQL Server.

Th s page de vers fu B ng Maps capab ty Users can zoom, pan, and render aer a v ews of the map wh e the pushp ns rema n bound to your geograph c data, as shown n F gure 9-33

Figure 9-33  Zoom, pan, and aer a v ew capab t es of the B ng Maps Ajax Contro .

422  Part II  Going Beyond Relational

Summary The power of spat a process ng s now read y ava ab e to a deve opers us ng the geometry and geography data types bu t nto SQL Server 2012, enab ng the rap d deve opment of soph st cated GIS app cat ons In th s chapter, you saw how easy t s to express and project coord nates aga nst e ther the p anar (flat-earth) or geodet c (e pso da sphere) spat a mode s us ng the We -Known Text (WKT), ­We -Known B nary (WKB), and Geography Markup Language (GML) formats We wa ked you through numerous samp es and demos, and, a ong the way, exam ned the many methods ava ab e for ­man pu at ng spat a data You a so created severa geospat a app cat ons that ntegrate ocat on nte gence and mapp ng capab t es for end users A though th s chapter has on y scratched the surface of spat a data and spat a programm ng, t does prov de the foundat on you need to work w th the powerfu spat a support ava ab e n SQL Server 2012



Chapter 9  Geospat a Support   423

Par t I I I

Applied SQL chapter 10

The M crosoft Data Access Juggernaut

427

chapter 11

WCF Data Access Techno og es

509

chapter 12

Mov ng to the C oud w th SQL Azure

579

chapter 13

S QL Azure Data Sync and W ndows Phone Deve opment . . . . . . . . . . . . . . . .

619

chapter 14

Pervas ve Ins ght . . . . . . . . . . . . . . . . . . . . . . . . . . . .

675

chapter 15

xVe oc ty In-Memory Techno og es . . . . . . . . . . . .

701

425

C hapter 1 0

The Microsoft Data Access Juggernaut –Leonard Lobel

A

t some po nt, and n some way, you need to expose your SQL Server data so that c ent ­app cat ons can consume t From the perspect ve of the database, every consumer s a “c ent app cat on”—whether you are bu d ng an app cat on to commun cate d rect y w th M crosoft SQL Server (trad t ona c ent/server arch tecture), or you are bu d ng a m dd e-t er serv ce ayer that, n turn, s consumed by other types of c ent app cat ons (n-t er arch tecture) Regard ess of the nature of the consum ng app cat on, t s your respons b ty as a profess ona deve oper to arch tect a data access ayer that exchanges nformat on between SQL Server and your app cat on (or the t ers of your app cat on) n a re ab e, sca ab e, and ma nta nab e manner In the next ser es of chapters, you w earn how to do just that us ng a of the techn ques ­ava ab e w th the M crosoft NET Framework—and there are many A though the number of cho ces m ght seem daunt ng, don’t fee nt m dated We w take th ngs step by step so that you ga n a c ear ­understand ng of what a the p eces are, and how the ent re stack fits (or doesn’t fit) together

Th s chapter covers a your c ent/server opt ons, nc ud ng raw ADO NET objects, the DataSet, Language-Integrated Query (LINQ), and the ADO NET Ent ty Framework In the next chapter, you w earn how to extend your data’s reach by us ng two spec a W ndows Commun cat on ­Foundat on (WCF) techno og es WCF Data Serv ces, wh ch exposes your data us ng Open Data Protoco (OData), and WCF RIA Serv ces, a r cher framework that targets M crosoft S ver ght n part cu ar The two chapters that fo ow then carry the d scuss on up to the c oud, where you w earn how to ­create so ut ons us ng SQL Azure, SQL Azure Data Sync, W ndows Azure, and W ndows Phone 7 Th s ­comprehens ve coverage w he p you choose the r ght NET data access strategy for your SQL Server app cat ons

.NET Data Access Evolution ADO NET was re eased w th the very first vers on of the NET Framework n ear y 2002, and t ­offers you two ways to access the database programmat ca y us ng raw objects or DataSets (or ­comb nat ons of the two) You can use the raw ADO NET objects to nteract d rect y w th the database by exp c t y connect ng to t, execut ng d rect T-SQL or stored procedures, and pu ng data back

427

w th stream ng readers A ternat ve y, you can work at a h gher eve of abstract on w th D ­ ataSets, us ng DataAdapters as wrappers around the raw objects to fi and update the DataSet Because ­DataAdapters are mere y wrappers around the very same raw objects, you can use a comb nat on of raw objects and DataSets as des red In our d scuss on, we define both of these methods co ect ve y as “convent ona ADO NET,” to d st ngu sh them from data access techno og es added to ater vers ons of the NET Framework Convent ona ADO NET was the on y data access opt on for NET programmers from the t me the framework was first re eased n 2002 unt NET 3 5 was re eased n ate 2007, fo owed short y ­thereafter by NET 3 5 SP1 n 2008 S nce NET 3 5 (and espec a y SP1), th ngs have been acce erat ng qu te rap d y The advent of Language-Integrated Query (LINQ), n NET 3 5, rad ca y changed the way deve opers wr te code aga nst many d fferent types of back-end data stores For SQL Server n part cu ar, programmab ty opt ons have ncreased great y w th the re ease of LINQ to SQL n NET 3 5 and the ADO NET Ent ty Framework—first re eased as part of NET 3 5 SP1, and great y enhanced n NET 4 0 as of 2010 New app cat on programm ng nterfaces (APIs) and too s have a so emerged, such as WCF Data Serv ces and WCF RIA Serv ces (covered n Chapter 11) These too s promote rap d deve opment of n-t er database app cat ons by prov d ng even r cher abstract ons at the WCF ayer When techno ogy churns th s fast, t s on y natura to fee overwhe med Keep ng pace s certa n y cha eng ng, but th s chapter w he p you meet the cha enge w th m n ma pa n You w earn the necessary cod ng techn ques for work ng w th a of the NET data access opt ons as you study each w th pract ca , hands-on use cases Th s comparat ve approach w he p br ng the s m ar t es and ­d fferences between each opt on to ght, so that you effect ve y ga n a so d understand ng across the spectrum Anyone who has worked w th NET for any ength of t me s a ready qu te fam ar w th c­ onvent ona ADO NET, but may now quest on ts re evance n the wake of a the techno og es that have emerged s nce In fact, convent ona ADO NET—the raw data access objects n part cu ar, but the DataSet as we — s not on y st re evant, but rema ns core to the NET framework The newer APIs comp ement convent ona ADO NET—they do not rep ace t Th s s a vast qua ty-of- fe mprovement over standard M crosoft pract ce n the days of the Component Object Mode (COM) p atform that preceded the NET Framework In the days of COM, each new data access API wou d typ ca y render prev ous APIs obso ete Data access objects (DAO) prov ded an object mode around the Open Database Connect v ty (ODBC) API that was very powerfu , unt t was made obso ete by Remote Data Objects (RDO) Yet t was not ong after RDO that Act veX Data Objects (ADO) comp ete y obso esced RDO And of course, a of these COM-based APIs were rendered obso ete by the NET Framework, and the not on of managed code prov ders w th ADO NET One s mp y accepted obso escence w th the re ease of each new API, as part of the nev tab e pr ce to be pa d f one wanted to everage the atest techno og es As a resu t of th s pa nfu cyc e, product on code became st gmat zed as “ egacy code” very qu ck y n those days S nce NET, the s tuat on has mproved great y There s noth ng “c ass c” or “ egacy” (more ­po te terms for “obso ete”) about convent ona ADO NET The newer APIs ayer and extend, but most certa n y do not obso esce convent ona ADO NET And wh e they each have d fferent use cases and prov de d fferent eve s of abstract on, they a so typ ca y do not obso esce one another (though 428  Part III  Applied SQL

depend ng on one’s subject ve defin t on of the term “obso ete,” LINQ to SQL m ght be cons dered an except on to th s statement, as t has been essent a y ec psed by the atest vers on of the ADO NET Ent ty Framework) L ke DataAdapters w th DataSets, the atest APIs prov de vary ng degrees of abstract on beneath wh ch e the very same managed code prov ders that NET deve opers have been work ng w th s nce the b rth of the framework (Sq C ent, n the case of SQL Server) Th s new and mproved sty e of evo ut on serves as a testament to the stab ty and matur ty of the NET Framework, and s gn ficant y extends the she f- fe of your app cat ons A though keep ng pace st cont nues to pose a cha enge, the fact s that the newer APIs broaden rather than narrow your cho ces, and we be eve that’s a good th ng So regard ess of wh ch API you choose, the same ADO NET data prov der (Sq C ent) s used to send commands to SQL Server and stream data back to your app cat on Th s s true no matter wh ch API you are us ng DataSets are fi ed and updated us ng DataAdapters, wh ch wrap the raw connect on, command, and reader objects W th LINQ to SQL, there s a DataContext object that w a so ut ze Sq C ent to retr eve and update data n SQL Server If you are us ng ADO NET Ent ty Framework, the storage ayer s m ar y does the same WCF Data Serv ces and WCF RIA Serv ces (wh ch we cover n the next chapter) prov de abstract ons at the commun cat ons transport ayer further up n the stack, so c ent requests hand ed by those serv ce-or ented APIs w a so u t mate y bo down to raw ADO NET data access when the t me comes to h t the database Th s techno ogy stack s dep cted n F gure 10-1

WCF Data Services

WCF RIA Services

WCF

DataSet

LINQ to SQL

Entity Framework

Raw ADO.NET Objects

SQL Server

Figure 10-1  The .NET data access techno ogy stack over SQL Server.

Note  In our discussions, we will refer to the ADO.NET Entity Framework frequently as Entity Framework, or abbreviate it simply as EF. Convent ona ADO NET today st rema ns a perfect y v ab e opt on for data access n many ­scenar os It s not on y he pfu but prudent for even exper enced deve opers to refresh the r

Chapter 10  The M crosoft Data Access Juggernaut   429

­ nderstand ng of t Therefore, our coverage of convent ona ADO NET s recommended read ng even u f you’re a seasoned deve oper who may be tempted to sk p ahead to the newer techno og es further on n the chapter

Preparing the Sample Database Start by creat ng a bas c order entry database n SQL Server Then you’ bu d app cat on code over the database that quer es and updates us ng each of the ava ab e data access APIs Use e ther SQL Server Data Too s (SSDT) n V sua Stud o (covered n Chapter 1) or SQL Server Management Stud o (SSMS) to execute the scr pt shown n L st ng 10-1 and create the samp e database Listing 10-1  T SQL scr pt for creat ng the SampleDb database.

USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'SampleDb') DROP DATABASE SampleDb GO CREATE DATABASE SampleDb GO USE SampleDb GO CREATE TABLE Customer( CustomerId bigint IDENTITY(1,1) NOT NULL, FirstName varchar(50) NOT NULL, LastName varchar(50) NOT NULL, Balance money NOT NULL DEFAULT 0, CreatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), UpdatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), CONSTRAINT PK_Customer PRIMARY KEY CLUSTERED (CustomerId ASC)) CREATE TABLE OrderHeader( OrderHeaderId bigint IDENTITY(1,1) NOT NULL, CustomerId bigint NOT NULL, ShipVia varchar(20) NOT NULL, OrderStatus varchar(20) NULL, Notes varchar(max) NULL, CreatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), UpdatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), CONSTRAINT PK_OrderHeader PRIMARY KEY CLUSTERED (OrderHeaderId ASC)) ALTER TABLE OrderHeader WITH CHECK ADD CONSTRAINT FK_OrderHeader_Customer FOREIGN KEY(CustomerId) REFERENCES Customer(CustomerId) CREATE TABLE Employee( EmployeeId bigint IDENTITY(1,1) NOT NULL, FirstName varchar(50) NOT NULL, LastName varchar(50) NOT NULL,

430  Part III  Applied SQL

Salary decimal(18, 9) NOT NULL, CreatedAt datetime2(7) NOT NULL, UpdatedAt datetime2(7) NOT NULL, CONSTRAINT PK_Employee PRIMARY KEY CLUSTERED (EmployeeId ASC)) CREATE TABLE CustomerEmployee( CustomerId bigint NOT NULL, EmployeeId bigint NOT NULL, CONSTRAINT PK_CustomerEmployee PRIMARY KEY CLUSTERED (CustomerId ASC, EmployeeId ASC)) ALTER TABLE CustomerEmployee WITH CHECK ADD CONSTRAINT FK_CustomerEmployee_Customer FOREIGN KEY(CustomerId) REFERENCES Customer(CustomerId) ALTER TABLE CustomerEmployee WITH CHECK ADD CONSTRAINT FK_CustomerEmployee_Employee FOREIGN KEY(EmployeeId) REFERENCES Employee(EmployeeId) GO CREATE PROCEDURE SelectCustomer(@CustomerId bigint) AS SELECT CustomerId, FirstName, LastName, Balance, CreatedAt, UpdatedAt FROM Customer WHERE CustomerId = @CustomerId GO CREATE PROCEDURE SelectCustomers AS SELECT CustomerId, FirstName, LastName, Balance, CreatedAt, UpdatedAt FROM Customer ORDER BY CustomerId GO CREATE PROCEDURE InsertCustomer( @FirstName varchar(20), @LastName varchar(20), @Balance money = 0) AS DECLARE @CreatedAt datetime2 = SYSDATETIME() INSERT INTO Customer (FirstName, LastName, Balance, CreatedAt, UpdatedAt) VALUES (@FirstName, @LastName, @Balance, @CreatedAt, @CreatedAt) SELECT CustomerId = CAST(SCOPE_IDENTITY() AS bigint), CreatedAt = @CreatedAt, UpdatedAt = @CreatedAt GO CREATE PROCEDURE UpdateCustomer( @CustomerId bigint, @FirstName varchar(20),



Chapter 10  The M crosoft Data Access Juggernaut   431

@LastName varchar(20), @Balance money, @OriginalUpdatedAt datetime2) AS DECLARE @NewUpdatedAt datetime2 = SYSDATETIME() UPDATE Customer SET FirstName = @FirstName, LastName = @LastName, Balance = @Balance, UpdatedAt = @NewUpdatedAt WHERE CustomerId = @CustomerId AND UpdatedAt = @OriginalUpdatedAt IF @@ROWCOUNT = 0 THROW 50000, 'The customer does not exist, or has been updated/deleted', 1; ELSE SELECT UpdatedAt = @NewUpdatedAt GO CREATE PROCEDURE DeleteCustomer(@CustomerId bigint) AS DELETE Customer WHERE CustomerId = @CustomerId GO CREATE PROCEDURE GetCustomerBalance( @CustomerId bigint, @Balance money OUTPUT) AS SELECT @Balance = Balance FROM Customer WHERE CustomerId = @CustomerId GO

We’ve kept the database des gn sma , wh e st demonstrat ng key concepts, such as the use of stored procedures for create, retr eve, update, and de ete (CRUD) operat ons; opt m st c concurrency checks; dent ty-based pr mary keys; and many-to-many re at onsh ps The scr pt first creates the Customer and OrderHeader tab es, a ong w th a fore gn key constra nt that estab shes a one-to-many re at onsh p between them Then t creates the Employee tab e There s a many-to-many re at onsh p between customers and emp oyees, wh ch s mp emented by the CustomerEmployee junct on tab e—the fourth and ast tab e created n the scr pt The junct on tab e conta ns just the two keys re at ng customers w th emp oyees and has fore gn key constra nts to both tab es, wh ch s how you mp ement a many-to-many re at onsh p n SQL Server The three tab es Customer, OrderHeader, and Employee each fo ow a s m ar pattern The r pr mary keys are long va ues (64-b t ntegers), and they each have an IDENTITY spec ficat on that nstructs SQL Server to automat ca y ass gn the next ava ab e nteger when nsert ng new rows Each of them

432  Part III  Applied SQL

a so has a pa r of datetime2 co umns named CreatedAt and UpdatedAt that are used for aud t ng and concurrency checks n the CRUD stored procedures, created next Your first exerc se w work d rect y aga nst tab es But n rea product on env ronments, c ent app cat ons are often den ed perm ss on to execute d rect SELECT, INSERT, UPDATE, and DELETE operat ons aga nst tab es n the database Instead, typ ca y, each tab e s “wrapped” by a set of stored procedures that expose CRUD operat ons n a contro ed manner In ater examp es, you’ work w th stored procedures nstead of d rect SQL to access the Customer tab e (For brev ty, the code on y ­creates CRUD stored procedures for the Customer tab e; you can extrapo ate from that how a s m ar set of procedures wou d be mp emented for the OrderHeader and Employee tab es ) Two stored procedures retr eve customers SelectCustomer accepts a @CustomerID ­parameter and returns a s ng e row for the spec fied customer SelectCustomers takes no ­parameters, and returns every customer n the tab e sorted by ast name, first name Both stored procedures r­ eturn every co umn from the Customer tab e Of course, your typ ca app cat on wou d have many more ­parameter zed stored procedures for retr eva (such as SelectCustomersByThis and ­SelectCustomersByThat), but these two w suffice for th s exerc se Next, the scr pt defines the InsertCustomer stored procedure that creates a new customer The stored procedure accepts one parameter for each customer co umn, except for ­CustomerId, ­CreatedAt, and UpdatedAt The c ent cannot supp y a CustomerId because SQL Server w ­automat ca y ass gn the next dent ty va ue for the pr mary key and then return t back to the c ent Va ues are a so not accepted for CreatedAt and UpdatedAt because the stored procedure w ensure that both of these va ues get set to the current t me on the database server c ock, and then return them back to the c ent a ong w th CustomerId after the INSERT InsertCustomer first dec ares @CreatedAt as a datetime2 var ab e, ass gns t the current date and t me us ng the SYSDATETIME funct on, and then executes the INSERT statement that creates the new row The INSERT statement supp es the SYSDATETIME va ue captured n @CreatedAt for both the CreatedAt and UpdatedAt co umns, and supp es the ncom ng parameter va ues for a the other ­co umns The on y co umn m ss ng from the INSERT statement s the CustomerId pr mary key (th s can’t be spec fied because you’ve put SQL Server n charge of ass gn ng the next ava ab e ­IDENTITY va ue to new pr mary keys) The stored procedure returns a the co umn va ues ass gned by the server (CustomerId, CreatedAt, and UpdatedAt) to the c ent as a s ng e-row resu t set n the SELECT ­statement mmed ate y fo ow ng the INSERT Th s SELECT statement returns the SCOPE IDENTITY funct on that g ves you the new pr mary key va ue ass gned to CustomerId, and returns the @CreatedAt va ue ass gned to both CreatedAt and UpdatedAt A c ent app cat on can use th s return nformat on to refresh ts oca v ew of the new y added customer, rather than be ng forced to retr eve the ent re customer aga n from the database The UpdateCustomer stored procedure s defined next Th s stored procedure mod fies an ex st ng customer, and enforces an opt m st c concurrency check ensur ng that users do not overwr te each other’s changes It accepts a @CustomerId parameter spec fy ng the customer row to be updated, ­fo owed by one parameter for each customer co umn to be updated, except for CreatedAt and ­UpdatedAt Th s des gn prevents the CreatedAt co umn from be ng changed once t has been ass gned by InsertCustomer, and thus prov des you w th a re ab e aud t of each customer’s or g na creat on

Chapter 10  The M crosoft Data Access Juggernaut   433

t me The ast parameter taken by UpdateCustomer s @OriginalUpdatedAt, wh ch s used for the ­mu t user concurrency check Th s parameter must be passed the or g na UpdatedAt va ue of the row to be changed, wh ch mp es that you must first retr eve a customer before you can change t UpdateCustomer first dec ares @NewUpdatedAt as a datetime2 var ab e and ass gns t the ­current date and t me on the server us ng the SYSDATETIME funct on Th s va ue overwr tes the current va ue n the UpdatedAt co umn, and then gets returned to the c ent, assum ng that the opt m st c ­concurrency check doesn’t fa The UPDATE statement executes w th a WHERE c ause that finds the des red row to be updated by @CustomerId, but a so checks that the UpdatedAt co umn st ­conta ns the va ue passed n to @OriginalUpdatedAt Because UpdatedAt gets overwr tten w th each ­mod ficat on, the WHERE c ause w no onger find the row f another user se ected and updated the same customer s nce the t me you or g na y retr eved t Th s prevents you from overwr t ng changes a ready comm tted by the other user After the UPDATE statement, @@ROWCOUNT s tested to see f the spec fied customer was actua y found and updated A va ue of 0 nd cates that e ther a mu t user confl ct occurred, or the spec fied CustomerId never ex sted n the first p ace The stored ­procedure treats e ther of those cond t ons as an error, and ssues a THROW statement that w ra se an e ­ xcept on on the NET s de Otherw se, the UPDATE has succeeded, and the stored procedure returns the new UpdatedAt va ue as a s ng e-row, s ng e-co umn resu t set w th the SELECT statement f­ o ow ng the UPDATE As w th InsertCustomer, th s return nformat on enab es c ent app cat ons to refresh the r oca v ew of updated customers DeleteCustomer s the ast—and s mp est—of the CRUD stored procedures for the Customer tab e It takes just an @CustomerId parameter and ssues a DELETE statement that de etes that customer’s row from the tab e

Note  Although this delete operation does not check for multiuser conflicts, business requirements will ultimately dictate how you implement a concurrency strategy in your application. You might need to be more stringent and put checks on deletions as well as updates, by requiring an @OriginalUpdatedAt parameter and then testing it in the WHERE clause like you did for UpdateCustomer. You may also need to devise a more sophisticated conflict resolution strategy than merely throwing an error. For example, you can provide a merge facility that lets the user select which version of each column should “win” for the update. Conversely, if the probability of a multiuser collision is very low in your particular application, you may be able to do away with concurrency checks altogether (that is, if a conflict does occur, just blindly let the “last user win,” and don’t worry about the few cases where they overwrite another user’s earlier changes). There s one more stored procedure n the database that s not used for CRUD operat ons The GetCustomerBalance stored procedure demonstrates how to return a sca ar va ue us ng an output parameter (as opposed to a resu t set w th one row and one co umn) Th s stored procedure returns the ba ance for the spec fied @CustomerId n the @Balance output parameter

434  Part III  Applied SQL

That comp etes the exp anat on of the database You are ready to access t from NET (and you w start to n a moment us ng convent ona ADO NET) But first, fire up a SQL Profi er trace

Monitoring Database Activity with SQL Server Profiler SQL Server Profi er (or SQL Profi er for short) s an nd spensab e too that ets you see the d rect ­commands as they are sent to the server by ADO NET on beha f of your runn ng app cat on You w use t throughout th s chapter to observe database act v ty beh nd the scenes, so now s a good t me to start trac ng Launch SQL Server Profi er from the Performance Too s fo der n the M crosoft SQL Server 2012 group on the Start menu Press CTRL+N to create a new trace, and c ck Connect to connect to the server SQL Profi er d sp ays the Trace Propert es d a og C ck ng Run w start the trace, but don’t c ck t just yet You’ want to ta or the trace first By defau t, SQL Profi er h des transact ona events from the trace, so you won’t get to see the t­ ransact ona statements be ng sent to SQL Server by ADO NET un ess you change the defau t ­sett ngs These nc ude the BEGIN TRANSACTION and COMMIT TRANSACTION statements ssued ­mp c t y by the TransactionScope object that you w use n an upcom ng examp e to batch updates to SQL Server To mon tor these events, c ck the Events Se ect on tab, check the Show A Events checkbox, scro the Events st down, and expand the Transact ons category Then check a the “comp ete” events beg nn ng w th TM (wh ch stands for Transact on Manager), such as TM Beg n Tran comp ete, TM Comm t Tran comp ete, and so on, as shown n F gure 10-2

Figure 10-2  Configur ng a SQL Profi er trace.



Chapter 10  The M crosoft Data Access Juggernaut   435

You a so want to m t the trace to show commands sent on y from your app cat on, because t w show commands sent to SQL Server from any c ent by defau t C ck the Co umn F ters button to d sp ay the Ed t F ter d a og Choose App cat onName from the st on the eft, expand the Like node n the treev ew on the r ght to create a new L ke fi ter, and type .Net SqlClient Data Provider Then c ck OK to c ose the d a og Now the trace w on y d sp ay commands ssued by the ADO NET ­Sq C ent prov der (wh ch come from your app cat on), and fi ter out “no se” generated by other runn ng app cat ons and serv ces that ta k to SQL Server (such as SQL Server Report ng Serv ces, for examp e) W th transact ona events enab ed and the App cat onName fi ter now set, c ck Run to start the trace

Tip  You can save these settings in a template and have them loaded by default whenever you start a new trace. Click the File menu, choose Templates, and select New Template. Give the template a name, and check the “Use as a default template” checkbox. Then set the transactional events and column filter in the Events Selection tab and save the template.

Conventional ADO.NET We’ve a ready defined convent ona ADO NET as the two ava ab e data access opt ons that were ­ntroduced w th NET 1 0; name y, the raw data access objects and the DataSet/DataAdapter ­abstract on over those objects You’ now exp ore both of these approaches, beg nn ng w th the raw objects

Using the Raw Data Access Objects The raw ADO NET data access objects are based on the fo ow ng set of nterfaces n the NET Framework, wh ch are defined n the System.Data namespace ■

IDbConnection

■

IDbCommand

■

IDbParameter

■

IDbDataReader

■

IDbTransaction

These nterfaces define the operat ons that can be conducted aga nst the database (or other ­ ack-end data source) by any concrete prov der that mp ements them For SQL Server, there s a b managed ADO NET prov der (ca ed Sq C ent) that mp ements these nterfaces n the fo ow ng set of c asses, wh ch are defined n the System.Data.SqlClient namespace ■

SqlConnection

■

SqlCommand

436  Part III  Applied SQL

■

SqlParameter

■

SqlDataReader

■

SqlTransaction

As the nterface and c ass names convey, an ADO NET prov der g ves you a connect on object so you can ta k to the database, a command object w th parameters so you can express your quer es and update data, a reader object to stream query resu ts nto your app cat on, and a transact on object to batch your updates atom ca y Arguab y, these raw objects are a you need to perform any SQL Server data access n NET Any conce vab e command can be sent to the database, any conce vab e resu t set can be retr eved back, and mu t p e updates can be transact ona zed Those rea y are a the p eces you need to bu d a data access ayer n NET And that s why they cont nue to serve as the under y ng objects used by a the var ous NET data access abstract ons ava ab e today But that s a so prec se y why you need to st cons der us ng them n s tuat ons that ca for t, as opposed to us ng someth ng “ ater and greater,” m sgu ded by the not on that you must conform to a more fash onab e way of do ng th ngs n order to ma nta n your cred b ty as a deve oper Our gu dance here s to not fa nto that trap, but rather st ck to the o d adage of us ng the r ght too for the r ght job (and ho d your head up h gh) It sn’t a ways necessary to p e ayers and ayers of abstract ons over each other; n many cases, t h nders more than he ps When your needs are s mp e and ghtwe ght, you can and shou d cont nue to use the raw objects to get the job done qu ck y and effic ent y, w thout the overhead of a DataSet, LINQ to SQL mode , or Ent ty Data Mode (EDM) added to the m x There’s abso ute y noth ng wrong w th hook ng up a s mp e connect on object to a command object configured w th parameters for ca ng a stored procedure, and stream ng back the resu ts w th a reader, when that’s more or ess a you need to do And so there rea y are no hard and fast ru es e ther for or aga nst us ng raw objects d rect y versus any of the ava ab e abstract ons offered by the more advanced a ternat ve techno og es It a comes down to your understand ng of each (wh ch you’ get from th s chapter), comb ned w th the proper judgment on your part (wh ch you can ga n on y from exper ence) You must deem when t s ­appropr ate to everage the benefit of (and create a dependency on) more robust f­ rameworks, ­commensurate w th the eve of comp ex ty that your app cat on ca s for, and the degree of ­abstract on you w sh to ayer over that comp ex ty W th th s understand ng n m nd, et’s start at the very beg nn ng connect ons and commands

Creating Connections and Commands You can’t do anyth ng w thout a connect on object That s obv ous At the same t me, you can’t do anyth ng w th just a connect on object You’re go ng to need to wrap a command object around that connect on W th those m n ma requ rements met, you have what you need to execute any d rect T-SQL or stored procedure on the server Add a few parameter objects to the command, and you can pass nput parameters to the server and even retr eve sca ar va ues back v a output ­parameters ­Invok ng the ExecuteNonQuery method on the command object causes the T-SQL or stored ­procedure to execute on the server F gure 10-3 dep cts the process

Chapter 10  The M crosoft Data Access Juggernaut   437

Command

Database

Connection Parameters

Figure 10-3  The raw ADO.NET objects used to execute a bas c command n SQL Server.

You’ wr te your first b t of NET code w th just these objects Start w th a s mp e command to add a new customer to the database Launch V sua Stud o, create a new V sua C# W ndows Forms ­app cat on, and name t DemoRawDataAccess Drag a button contro from the Too box and drop t onto the form Form1 created automat ca y by V sua Stud o In the Propert es w ndow, name the button btnDirectSql, and set the Text property to Direct SQL Then doub e-c ck the button V sua Stud o w automat ca y create a c ck event hand er for the button, and then open the code ed tor so that you can mp ement the hand er Add the code shown n L st ng 10-2 Listing 10-2  Perform ng a s mp e INSERT operat on us ng ExecuteNonQuery.

private void btnDirectSql_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "INSERT INTO Customer (FirstName, LastName) VALUES ('Lukas', 'Keller')"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = TSql; cmd.ExecuteNonQuery(); } conn.Close(); } }

The comp er s unaware that the SqlConnection and SqlCommand c ass names referred to n your code can be found n the System.Data.SqlClient namespace, un ess you d rect t to ook there Do so by add ng the fo ow ng using d rect ve at the very top of the source code using System.Data.SqlClient;

The code beg ns by defin ng the ConnStr constant, and po nts t to the samp e database runn ng on the oca SQL Server nstance (you must change the Data Source accord ng y f you have c­ reated 438  Part III  Applied SQL

the database on another server or nstance) Hardcod ng the connect on str ng n th s ­manner s ­unacceptab e n product on app cat ons But n order to best ock down concepts as you p ­ rogress, you’re start ng w th abso ute y m n ma code Later on, you’ store the connect on str ng n a ­configurat on fi e so that t can be a tered w thout recomp ng the app cat on Next, the TSql constant s defined Th s str ng conta ns the INSERT statement to be executed by SQL Server (note that you can a so batch mu t p e T-SQL statements to be executed n a s ng e ca , by separat ng them w th sem co ons) Hardcod ng the first and ast name va ues Lukas and Ke er s even more unrea st c than us ng a hardcoded connect on str ng, but aga n, you’ soon graduate to more pract ca cod ng techn ques n upcom ng examp es W th the connect on str ng and T-SQL statement defined, you’re ready to start work ng w th the raw ADO NET data objects for SQL Server A new SqlConnection object s then created and ass gned to the var ab e conn at the top of a using b ock The using statement ensures that the SqlConnection object d sposes ts resources when the ­object s no onger needed (see s debar) You w see the using statement app ed frequent y n our code samp es, and we encourage you to fo ow th s best pract ce whenever you are work ng w th d sposab e objects

Disposable Objects and the using Statement D sposab e objects are objects that, nterna y, a ocate unmanaged resources (mean ng ­resources not managed by the NET Framework, such as database connect ons) Because of the nondeterm n st c nature of NET garbage co ect on, there’s no way to know for certa n when these objects w actua y be “fina zed” (destroyed) Th s s true even after no more object ­references rema n Thus, there’s no nherent mechan sm n the NET Framework for ­automat ca y re eas ng unmanaged resources ( ke database connect ons) when they are no onger needed A d sposab e object therefore mp ements the IDisposable nterface, wh ch exposes a Dispose method that can be ca ed exp c t y as soon as you know that you no onger need t Ca ng Dispose nstructs the object to re ease ts unmanaged resources at that po nt n t me, regard ess of the fact that the object tse f won’t get destroyed by NET garbage ­co ect on unt some (poss b y d stant) future t me The using statement (wh ch s va d on y w th an IDisposable object) makes t very easy to work w th d sposab e objects (Th s s not to be confused w th the comp ete y unre ated ­using d rect ve that mports a namespace, ke you d d a moment ago ) The statement dec ares a var ab e ass gned to an IDisposable object that gets scoped to a code b ock fo ow ng the ­using statement W th th s pattern, you never actua y wr te the ne of code that ca s Dispose; nstead, the framework guarantees that Dispose gets ca ed on the object once t fa s out of scope (that s, at the end of the code b ock) Th s guarantee s abso ute y re ab e, even f an unhand ed except on occurs at any po nt w th n the using code b ock wh ch wou d cause the b ock to term nate premature y

After creat ng the SqlConnection object, the ConnectionString property s set to the prev ous y defined constant and the Open method s nvoked on t to estab sh the database connect on

Chapter 10  The M crosoft Data Access Juggernaut   439

A second using statement then creates a new SqlCommand object and ass gns t to the var ab e cmd It s perfect y okay to nest using b ocks ke th s; once the second one s entered, the framework guarantees that Dispose w get ca ed on both the SqlConnection and the SqlCommand objects regard ess of f, when, or where an except on occurs The SqlCommand object’s Connection property s then assoc ated w th the open connect on n conn, and ts CommandText property s set to the TSql constant (the T-SQL you want to execute) defined ear er Then the ExecuteNonQuery method s nvoked on t, wh ch passes the command a ong to SQL Server The command, n th s case, s a s ng e INSERT statement that adds a new row to the Customer tab e Because va ues for on y two co umns (FirstName and LastName) are spec fied, a other co umns n the new Customer row (bes des CustomerId) w be ass gned the defau t va ues des gnated for them when the Customer tab e was defined at the beg nn ng of the chapter The CustomerId co umn w be ass gned automat ca y to the next ava ab e dent ty va ue, because IDENTITY was spec fied on that pr mary key co umn n the Customer tab e defin t on After the command executes on the server, the nner using b ock c oses, wh ch mp c t y d sposes the SqlCommand object and dereferences the cmd var ab e F na y, you nvoke the Close method on the connect on and the outer using b ock c oses, wh ch mp c t y d sposes the SqlConnection object and dereferences the conn var ab e Run the code and c ck the button Then se ect the new row to v ew t SELECT * FROM Customer

The output confirms that the Customer tab e now has one row n t, the one you just added for Lukas Ke er CustomerId ---------1

FirstName --------Lukas

LastName -------Keller

Balance ------0.00

CreatedAt ----------------------2012-04-06 12:34:16.50

UpdatedAt ---------------------2012-04-06 12:34:16.50

As expected, the CustomerId co umn was ass gned a va ue of 1, and the Balance, CreatedAt, and UpdatedAt co umns were ass gned the defau t va ues of 0, SYSDATETIME, and SYSDATETIME, respect ve y These va ues were ass gned automat ca y based on the IDENTITY spec ficat on and defau t va ues for the Customer tab e defin t on n L st ng 10-1

Using Parameters In pract ce, you wou d never hardcode va ues for a new customer ke you just d d The va ues wou d come from e sewhere, and then get passed nto your app cat on’s data access ayer You can certa n y use str ng concatenat on to embed the nput va ues nto a dynam ca y composed T-SQL statement, but do ng so exposes your app cat on to potent a SQL nject on attacks (a secur ty r sk that we exp a n n Chapter 5) Furthermore, your code wou d need to san t ze the nput va ues (for examp e, by doub ng up tera s ng e quotat on characters n str ngs), or you further r sk runt me except ons resu t ng from str ngs that conta n T-SQL syntax errors So the proper way to hand e var ab e va ues n T-SQL s to use parameters, as demonstrated by a mod fied vers on of the prev ous code

440  Part III  Applied SQL

Drag two text box contro s from the Too box and drop them onto the form In the Propert es w ndow, name the text boxes txtFirstName and txtLastName Now drag a button contro from the Too box and drop t onto the form, name t btnParameterizedDirectSql, and set the Text property to Parameterized Direct SQL Then doub e-c ck the button and add the code n L st ng 10-3 Listing 10-3  Us ng parameters w th a command object.

private void btnParameterizedDirectSql_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "INSERT INTO Customer (FirstName, LastName) VALUES (@FirstName, @LastName)"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = TSql; cmd.Parameters.AddWithValue("@FirstName", this.txtFirstName.Text); cmd.Parameters.AddWithValue("@LastName", this.txtLastName.Text); cmd.ExecuteNonQuery(); } conn.Close(); } }

Th s vers on of the code s a most dent ca to the prev ous vers on, but has some mportant ­ fferences You’ not ce that the hardcoded first and ast name va ues n the INSERT statement d defined by the TSql constant have been rep aced w th parameter p aceho ders The parameter names, a ong w th the r va ues wh ch you get from the text boxes on the form, are added to the Parameters co ect on of the SqlCommand object w th the AddWithValue method

More Info  AddWithValue is a very convenient method. In one line of code, it creates a new SqlParameter object with its ParameterName and Value properties set, and adds the SqlParameter object to the Parameters collection. More elaborate ways of c­ onfiguring the Parameters collection give you fine control over the data type, length, precision, direction (input or output), and source column mapping of each parameter. You shall see an example of this shortly, when you learn how to return scalar values ­using ­output parameters. But for simple input parameters such as the ones in this example, c­ alling the AddWithValue method is sufficient.



Chapter 10  The M crosoft Data Access Juggernaut   441

By us ng parameters (rather than str ng concatenat on), your code s not vu nerab e to SQL nject on attacks, nor do you need to concern yourse f w th san t z ng the first and ast name va ues ADO NET w automat ca y parse the command text and parameter va ues to produce a T-SQL statement that s both safe and va d for execut on (for examp e, t w doub e-up tera s ng e quotat on marks entered n e ther the first or ast name text box) Now go ahead and run the app cat on Type va ues n the first and ast name text boxes, and c ck the button to create a new customer row Then, to confirm, se ect and v ew the new customer as you d d after creat ng the first customer

Calling Stored Procedures So far, you’ve used the SqlCommand object to execute a d rect T-SQL statement on the server and nsert a new customer row However, t’s a better pract ce to use stored procedures rather than d rect SQL to ­perform such updates Us ng stored procedures draws a c ear separat on between NET code and T-SQL code Pr mary benefits to th s approach nc ude ma nta nab ty, reusab ty, and secur ty D rect SQL can qu ck y ead to code dup cat on and esca ate nto a ma ntenance n ghtmare, w th NET and T-SQL og c ntertw ned and t ght y coup ed n your app cat on Good arguments can be made on both s des of the great “d rect SQL versus stored procedure debate” (read the s debar n the upcom ng ADO NET Ent ty Framework coverage), but the fact s that somet mes you just don’t have the cho ce It s very common pract ce for database adm n strators n the enterpr se to ock down tab es and deny d rect access to them by c ent app cat ons The database nstead exposes a “serv ce ayer” of stored procedures that nd rect y prov de contro ed access to the under y ng tab es Th s s unquest onab y far more secure than just ­a ow ng c ents to execute any SQL that they want d rect y aga nst your tab es Argu ng the mer ts and d sadvantages between d rect SQL and stored procedures s po nt ess n such env ronments—you’re stuck w th us ng stored procedures and that’s a there s to t Now rev se the code to ca the InsertCustomer stored procedure nstead of us ng d rect SQL Drag another button contro from the Too box and drop t onto the form, name t btnStoredProcedureInsert, and set ts Text property to Stored Procedure INSERT Then doub e-c ck the button and add the code n L st ng 10-4 Listing 10-4  Ca ng a stored procedure w th parameters.

private void btnStoredProcedureInsert_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "InsertCustomer"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand())

442  Part III  Applied SQL

{ cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.AddWithValue("@FirstName", this.txtFirstName.Text); cmd.Parameters.AddWithValue("@LastName", this.txtLastName.Text); cmd.ExecuteNonQuery(); } conn.Close(); } }

Aga n, not ce the two d fferences from the prev ous examp e F rst, rather than cod ng the ­actua T ­ -SQL INSERT statement r ght ns de the C# method, you refer on y to the name of the stored ­procedure n the database, InsertCustomer The og c for add ng the new customer row, ­nc ud ng the actua ­INSERT ­statement, s conta ned n the stored procedure Second, you nform the ­SqlCommand object that the str ng you have set n ts CommandText property s the name of a stored p ­ rocedure and not a d rect T-SQL statement You do th s by sett ng ts CommandType property to C ­ ommandType.StoredProcedure ­(overr d ng the defau t va ue of CommandType.Text for d rect T-SQL) The rest ooks and works the same G ve the code a run to add a th rd customer, and then v ew the new customer row as before But the InsertCustomer stored procedure does more than just execute the INSERT statement to add the new customer row Referr ng back to L st ng 10-1, reca that t a so executes a SELECT ­statement r ght after the INSERT to return co umns va ues ass gned to the new row by the stored ­procedure In part cu ar, the SELECT statement returns a s ng e-row resu t set to the c ent that ­conta ns the un que pr mary key va ue ass gned to CustomerId (by v rtue of ts IDENTITY ­spec ficat on), and the current server date and t me va ues ass gned to CreatedAt and UpdatedAt (by v rtue of the r defau t va ues of SYSDATETIME, the bu t- n funct on that returns the server c ock’s current date and t me as a datetime2 data type) In th s manner, the c ent can obta n va ues ass gned to a new customer by SQL Server when contro s returned from the stored procedure that creates the new customer, rather than ssu ng another command (and ncurr ng another server round tr p) to retr eve them You can obta n a data reader to retr eve these va ues from the stored procedure by nvok ng ExecuteReader rather than ExecuteNonQuery on the SqlCommand object A though data readers are most common y used to retr eve mu t p e rows of data from SQL Server, they are a so used to retr eve a s ng e-row resu t set n cases ke th s In the next sect on, you’ exper ment w th both use cases

Iterating Data Readers If your T-SQL statement or stored procedure ca returns a resu t set (or mu t p e resu t sets), you retr eve t (or them) us ng a data reader (a so ca ed a connected data reader, and somet mes referred to as a “firehose cursor”) Data s streamed from SQL Server d rect y nto your app cat on as qu ck y as poss b e v a a data reader n a read-on y, forward-on y fash on F gure 10-4 dep cts the process



Chapter 10  The M crosoft Data Access Juggernaut   443

Command

Database

Connection

DataReader

Parameters

Figure 10-4  The raw ADO.NET objects used to execute a command n SQL Server that returns resu ts n a data

reader.

The SqlDataReader advances through the resu t set one row at a t me, g v ng your code the ­ pportun ty to extract the co umn va ues of each row What you do w th the row data s ent re y up o to you; you can nstant ate and popu ate objects, or you can process one row at a t me w thout ever cach ng the data (but do rema n cogn zant of the fact that the database connect on rema ns open wh e you terate the reader; you shou d genera y str ve to keep connect ons open for the shortest durat on of t me poss b e) The connect on, command, and parameter objects get configured just as you’ve been do ng a a ong, on y now you nvoke the ExecuteReader method rather than ExecuteNonQuery aga nst the SqlCommand object to execute the command Interna y, the ExecuteReader method nvokes the command on the server just ke ExectuteNonQuery does, but then a so creates a new ­SqlDataReader object pos t oned at the very beg nn ng of the resu t set to be streamed back, and returns the ­SqlDataReader for your code to terate one row at a t me Th s s the on y way to create a reader ­object The SqlDataReader c ass has no constructor, so you cannot nstant ate one w th the new ­keyword; you can on y obta n a new reader by ca ng ExecuteReader You w now use a data reader to retr eve the CustomerId, CreatedAt, and UpdatedAt va ues ­ ss gned to the new y nserted row (for a new customer, the same va ue s expected to returned for a CreatedAt and UpdatedAt) Drag another button contro from the Too box and drop t onto the form, name t btnStoredProcedureInsertWithReader, and set the Text property to Stored Procedure INSERT with Reader Then doub e-c ck the button and add the code n L st ng 10-5 Listing 10-5  Us ng a data reader to retr eve va ues ass gned to a new row added by the InsertCustomer stored

procedure.

private void btnStoredProcedureInsertWithReader_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "InsertCustomer"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) {

444  Part III  Applied SQL

cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.AddWithValue("@FirstName", this.txtFirstName.Text); cmd.Parameters.AddWithValue("@LastName", this.txtLastName.Text); using (var rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)) { rdr.Read(); var customerId = rdr.GetInt64(0); var createdAt = rdr.GetDateTime(1); var updatedAt = rdr.GetDateTime(2); rdr.Close(); MessageBox.Show( string.Format("Customer ID {0}; Created at {1}; Updated at {2}", customerId, createdAt, updatedAt)); } } } }

Everyth ng s the same unt t comes t me to execute the command, at wh ch po nt you nvoke E­ xecuteReader to obta n a SqlDataReader object ass gned to the var ab e rdr Because SqlDataReader s a d sposab e object, you ca ExecuteReader w th a using statement b ock to ensure that the reader w a ways be d sposed proper y You shou d c ose a data reader when you are done work ng w th t, after wh ch t me you shou d a so c ose the under y ng database connect on The CommandBehavior. CloseConnection enumerat on passed to ExecuteReader offers a conven ence here, by nstruct ng ADO NET to c ose the SqlConnection automat ca y when you c ose the SqlDataReader Therefore, you’ not ce that the ne conn.Close() has been removed from th s vers on of the code, because the ­connect on w be c osed automat ca y when the reader s c osed on the ne rdr.Close() When you rece ve the reader, t s pos t oned before the first row (wh ch n th s case s the on y row) n the resu t set Invok ng the Read method advances the reader to the row so you can retr eve ts ­va ues The reader exposes a set of GetXxx methods for a ava ab e data types, each of wh ch accepts an nteger parameter that spec fies the ord na pos t on of the co umn from wh ch to ­retr eve the va ue Because the row s expected to return one ong (64-b t) nteger for the new CustomerId and two date/t me va ues for CreatedAt and UpdatedAt— n that order—you ca the GetInt64 and ­GetDateTime methods pass ng n the va ues 0, 1, and 2 respect ve y You can a so extract co umn va ues by access ng the SqlDataReader object us ng ndexed notat on In th s examp e, you cou d have obta ned the three va ues by cod ng rdr[0], rdr[1], and rdr[2], rather than us ng the GetXxx methods W th th s approach, co umn va ues are returned as the type object, so you must cast or convert to the des red type If you know the co umn names, you can use a str ng ndex rather than a numer c ndex, such as rdr[“CustomerId”], rdr[“CreatedAt”], and rdr[“UpdatedAt”] (though be aware that do ng so s s ght y s ower than us ng a numer c ndex, because ADO NET needs to find the des red co umn matched by name) You w use str ng ndexes n the next data reader examp e



Chapter 10  The M crosoft Data Access Juggernaut   445

More Info  All techniques for extracting data reader values require that you know the data type of the column you are accessing. In this case, as in most, you will have that k­ nowledge at design time. However, there are some scenarios in which you may not know what a ­column’s data type is, or indeed, what the actual columns being returned are altogether. You can call the reader’s GetDataTypeName method and pass it a column index to discover the data type of any particular column. If you need to dynamically discover all the columns in the result set being returned, you can call the GetSchemaTable method on the reader to obtain a DataTable populated with schema information. You can then glean the result set schema (before streaming in from the reader) by examining the DataTable. It will contain one DataRow per result set column, populated with descriptive information about each ­column (name, data type, scale, precision, and so on). Run the examp e now to add a fourth row to the Customer tab e The MessageBox d a og w d sp ay the CustomerId (wh ch shou d be 4, f you ran the three prev ous examp es), CreatedAt, and UpdatedAt va ues returned by the stored procedure Because the InsertCustomer stored procedure returns a resu t set that s known to a ways conta n one and on y one row, the code s based on that as be ng a re ab e assumpt on That s, the Read method s ca ed on the SqlDataReader uncond t ona y, and exact y once In many other scenar os, you w not know f you are gett ng back zero, one, or more rows n the reader, and so you can’t make any assumpt ons Your code w need to test the reader for an end-of-stream cond t on, because an except on w occur when you attempt to retr eve data f there are no rows n the resu t set, or f you have a ready advanced beyond the ast row n the resu t set The next examp e demonstrates how to mp ement th s pattern You w ca the SelectCustomers stored procedure to return the ent re set of customers, and then bu d a st of d sp ay str ngs formatted as “ ast name, first name ” Drag a st box contro from the Too box and drop t onto the form In the Propert es w ndow, name the st box lstNames Now drag a button contro from the Too box and drop t onto the form, name t btnStoredProcedureWithReader, and set the Text property to Stored Procedure with Reader Then doub e-c ck the button and add the code n L st ng 10-6 Listing 10-6  Process ng a data reader.

private void btnStoredProcedureWithReader_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "SelectCustomers"; var names = new List(); using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr;

446  Part III  Applied SQL

conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; using (var rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)) { while (rdr.Read()) { var name = string.Format("{0}, {1}", rdr["LastName"], rdr["FirstName"]); names.Add(name); } rdr.Close(); } } } this.lstNames.DataSource = names; }

Th s t me, the SqlCommand object’s CommandText property s set to po nt to the SelectCustomers stored procedure Then ExecuteReader s nvoked to obta n a SqlDataReader object that returns the resu ts as before Th s t me, however, there s no way to know how many rows w come back So the code mp ements a while b ock, and tests for an end-of-stream cond t on before each ca to the reader’s Read method as t oops through the resu t set If there are no customers, Read w return true the very first t me t s ca ed, and the while b ock w never execute Otherw se, the b ock w execute once for each customer row returned by the stored procedure For each row, a str ng ndex s used aga nst the SqlDataReader to extract the LastName and ­FirstName va ues so the customer name can be formatted (th s s a more conven ent, a be t s ght y s ower a ternat ve to ca ng GetString on the reader and pass ng n the ord na co umn pos t ons of the name fie ds) Each formatted name s added to the List co ect on that s n t a zed before creat ng the SqlConnection After c os ng the reader (wh ch, because CommandBehavior.CloseConnection was spec fied, a so c oses the connect on), the DataSource property of the st box b nds t to the str ng co ect on G ve the app cat on a run, and confirm that the st box d sp ays the names of a the customers you added n the ear er exerc ses

Returning Scalar Values In the first data reader examp e, you retr eved three sca ar va ues back from ca ng the ­InsertCustomer stored procedure If you on y need to return a s ng e sca ar va ue, ADO NET offers a shortcut that ­e m nates the need to work w th a data reader a together By nvok ng the ExecuteScalar method (rather than ExecuteReader) on the SqlCommand object, ADO NET s mp y returns the va ue n the first co umn of the first row of whatever resu t set gets returned by SQL Server If there are more co umns (or rows), they are gnored, and f no resu t set at a gets returned, an except on s thrown The sca ar va ue s returned as the type object, wh ch you then cast or convert as necessary to the expected data type

Chapter 10  The M crosoft Data Access Juggernaut   447

The UpdateCustomer stored procedure s a good examp e It returns the one and on y va ue ­affected by an update operat on, name y the UpdatedAt co umn va ue ass gned by the ­SYSDATETIME funct on ( n L st ng 10-1, th s s the ne of code that reads SELECT UpdatedAt = @NewUpdatedAt) A though you can use the same data reader approach you used when ca ng InsertCustomer (where you needed to retr eve three va ues), you can retr eve the new UpdatedAt va ue returned by ­UpdateCustomer w th ess code by us ng ExecuteScalar nstead of ExecuteReader, as shown n the fo ow ng code sn ppet // This code... using (var rdr = cmd.ExecuteReader()) { rdr.Read(); var updatedAt = rdr.GetDateTime(0); rdr.Close(); } // ...is equivalent to this code var updatedAt = Convert.ToDateTime(cmd.ExecuteScalar());

The ExecuteScalar method s c ear y a cod ng conven ence, but don’t th nk that t works any faster than us ng ExecuteReader A the work st goes on beh nd the scenes to bu d and return a resu t set that could potent a y conta n mu t p e co umns and rows, and then you’re say ng that you on y want the first co umn of the first row That can amount to need ess server overhead and network transport, cons der ng that you can return output parameters to send sca ar va ues back from SQL Server W th output parameters, you don’t use a s ng e-row resu t set to return sca ar va ues (a though you can effect ve y comb ne the use of output parameters w th mu t -row resu t sets to return both sca ar and tabu ar data) Instead, you add a Parameter object correspond ng to each output parameter expected by the stored procedure and set ts Direction property to ParameterDirection.Output. Then you ca ExecuteNonQuery to run the stored procedure After the stored procedure executes, you can extract the va ue(s) set by the server from the Value property of the Parameter object(s) The GetCustomerBalance stored procedure returns the ba ance of any spec fied customer us ng an output parameter Let’s see how you wr te the ADO NET code to ca GetCustomerBalance and retr eve ts return va ue Drag another text box contro from the Too box and drop t onto the form In the Propert es w ndow, name the text box txtCustomerId Now drag a button contro from the T ­ oo box and drop t onto the form, name t btnOutputParameter, and set the Text property to Output ­Parameter Then doub e-c ck the button and add the code n L st ng 10-7 Listing 10-7  Retr ev ng an output parameter va ue returned by a stored procedure.

private void btnOutputParameter_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "GetCustomerBalance"; var customerId = Convert.ToInt64(this.txtCustomerId.Text); decimal balance;

448  Part III  Applied SQL

using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.AddWithValue("@CustomerId", customerId); var outputParam = new SqlParameter() { ParameterName = "@Balance", SqlDbType = SqlDbType.Money, Direction = ParameterDirection.Output }; cmd.Parameters.Add(outputParam); cmd.ExecuteNonQuery(); balance = (decimal)outputParam.Value; } conn.Close(); } MessageBox.Show(string.Format("Customer ID {0} balance is {1:c}", customerId, balance)); }

The stored procedure accepts two parameters The first one s the @CustomerId nput parameter (a 64-b t nteger; bigint n SQL Server, correspond ng to long n NET), and the second one s the @Balance output parameter (a money va ue n SQL Server, correspond ng to decimal n NET) The customer ID s obta ned from the va ue entered n the text box and converted nto a 64-b t nteger n customerId that w be passed for the @CustomerId nput parameter Then the balance var ab e s dec ared as a decimal that w get ass gned the va ue returned by the @Balance output parameter returned by the stored procedure, after t executes a few nes further down n the code After the connect on and command objects are configured, you add the parameters The @CustomerId nput parameter s added by ca ng AddWithValue, as you’ve been do ng thus far The @Balance output parameter, however, cannot be added n just a s ng e ne of code Instead, you need to exp c t y nstant ate the new SqlParameter nstance yourse f, set the ­appropr ate propert es, and add t to the Parameters co ect on Spec fica y, you set the ParameterName, SqlDbType, and ­Direction propert es to @Balance, SqlDbType.Money, and ParameterDirection.Output, respect ve y W th the Parameters co ect on proper y configured, ExecuteNonQuery runs the stored procedure When contro returns to the app cat on, you extract the output parameter va ue set by the stored procedure nto the balance var ab e Run the app cat on now and g ve t a try Enter any ex st ng customer ID (you shou d have four customers w th IDs 1 through 4) and c ck the button to run the stored procedure and confirm that the ba ance s be ng returned v a the output parameter A the customers entered to th s po nt have zero ba ances (because you’ve not yet exp c t y prov ded a va ue for the Balance co umn, wh ch

Chapter 10  The M crosoft Data Access Juggernaut   449

has a ­defau t va ue of 0 00) In the next sect on, you’ perform some updates that change customer ­ba ances, after wh ch you can run th s code aga n to ver fy that the stored procedure s return ng ­ba ance data as expected

Batching Updates with Transactions The ast raw ADO NET object eft to cover s a so, nterest ng y, the on y one wh ch you shou d no ­onger use, and that s SqlTransaction object ADO NET 2 0 ntroduced the TransactionScope object, wh ch s now the recommended way to manage transact ons at the app cat on eve w th ADO NET The SqlConnection object has a BeginTransaction method that, at the t me you nvoke t, ssues a BEGIN TRANSACTION command to SQL Server over the open connect on A SqlTransaction ­object s returned back for you to ho d as a hand e onto the pend ng transact on You then perform the ­necessary updates, sett ng the Transaction property of each SqlCommand object nvo ved n the ­updates to the SqlTransaction object F na y, you nvoke e ther the CommitTransaction or RollbackTransaction method on the SqlTransaction object, depend ng on the success or fa ure of a the updates These methods ssue, respect ve y, a COMMIT TRANSACTION or ROLLBACK TRANSACTION command to the server Because your NET code exp c t y ssues methods that trans ate d rect y to T-SQL statements contro ng transact ons, t s sa d to be manag ng explicit transact ons There are a number of pa n po nts w th exp c t transact ons The first d fficu ty es n the r­ equ rement that every SqlCommand object used to perform updates ns de the transact on must have ts Transaction property set to the SqlTransaction object returned by BeginTransaction Th s means that you must take care to pass a ong the SqlTransaction object to any p ace n your code that performs an update, because fa ng to ass gn t to the Transaction property of every SqlCommand object n the transact on resu ts n a runt me except on The ssue s compounded when you need to track the SqlTransaction object across mu t p e method ca s that perform updates It becomes even harder to manage th ngs when these methods need to be flex b e enough to work whether or not a ­transact on s nvo ved or requ red The prob em s worse when work ng w th any of the other techno og es we’ be cover ng ater that prov de abstract on ayers over the raw objects For examp e, because a SqlDataAdapter a ­ ctua y wraps three d st nct SqlCommand objects (for nsert, update, and de ete), you must d g beneath the data adapter and hook nto ts three under y ng command objects so that you can set the r ­Transaction propert es (We don’t genera y recommend that you m x and match d fferent data ­access APIs w th n your app cat on, but f you must transact ona ze updates across a comb nat on of ­techno og es, ­mp c t transact ons make t easy ) The TransactionScope object, ntroduced as part of a ded cated transact on management API w th NET 2 0 n 2005, ets you code transact ons mp c t y Th s s a super or approach that re eves you from a of the aforement oned burdens assoc ated w th exp c t transact ons So the gu dance here s to a ways work w th mp c t transact ons whenever poss b e You w wr te ess code that s more flex b e when you a ow the framework to hand e transact on management for you However, t s st a so mportant to understand exp c t transact ons w th the SqlTransaction object, as you m ght need to ntegrate w th and extend ex st ng code that a ready uses exp c t transact ons Therefore, we w cover both transact on management sty es to prepare you for a s tuat ons 450  Part III  Applied SQL

More Info  The transaction management API offers many more benefits besides i­mplicit transactions. For example, TransactionScope is capable of promoting a ­lightweight ­transaction (one associated with a single database) to a distributed transaction ­automatically, if and when your updates involve changes across multiple databases. Furthermore, you can enlist updates other than database changes to participate in the transaction. Chapter 4 covers these concepts in more detail. The next examp e demonstrates how to code exp c t transactons It performs a ba ance transfer between customers, a ow ng a port on of one customer’s ba ance to be subtracted, and then added to another customer’s ba ance Both update operat ons obv ous y need to succeed or fa as a who e, and so they must be transact ona zed Drag three text box contro s from the Too box and drop them onto the form In the ­Propert es w ndow, name the text boxes txtTransferFromCustomerId, txtTransferToCustomerId, and ­t xtTransferAmount Now drag a button contro from the Too box and drop t onto the form, name t btnUpdateWithExplicitTransaction, and set the Text property to Update with Explicit ­Transaction Then doub e-c ck the button and add the code n L st ng 10-8 Listing 10-8  Updat ng data us ng exp c t transact ons.

private void btnUpdateWithExplicitTransaction_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "UPDATE Customer SET Balance += @Transfer WHERE CustomerId = @CustomerId"; var fromCustomerId = Convert.ToInt64(this.txtTransferFromCustomerId.Text); var toCustomerId = Convert.ToInt64(this.txtTransferToCustomerId.Text); var transferAmount = Convert.ToDecimal(this.txtTransferAmount.Text); using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var txn = conn.BeginTransaction()) { try { using (var cmd1 = new SqlCommand()) { cmd1.Connection = conn; cmd1.CommandText = TSql; cmd1.Transaction = txn; cmd1.Parameters.AddWithValue("@CustomerId", fromCustomerId); cmd1.Parameters.AddWithValue("@Transfer", decimal. Negate(transferAmount));



Chapter 10  The M crosoft Data Access Juggernaut   451

cmd1.ExecuteNonQuery(); } using (var cmd2 = new SqlCommand()) { cmd2.Connection = conn; cmd2.CommandText = TSql; cmd2.Transaction = txn; cmd2.Parameters.AddWithValue("@CustomerId", toCustomerId); cmd2.Parameters.AddWithValue("@Transfer", transferAmount); cmd2.ExecuteNonQuery(); } txn.Commit(); } catch { txn.Rollback(); throw; } } conn.Close(); } }

Make sure that your SQL Server Profi er trace s proper y configured and runn ng, as we exp a ned ear er (see F gure 10-2) Then set a breakpo nt at the top of the method n L st ng 10-8 and run the code Enter two ex st ng customer IDs and an amount to transfer, c ck the Update with Explicit ­Transaction button, and h t the breakpo nt Then step through the code one ne at a t me as you mon tor database act v ty n the SQL Profi er trace Two SqlCommand objects are nvo ved n the ba ance transfer, both of wh ch execute the ­UPDATE statement defined by the TSql constant The UPDATE statement accepts the two ­parameters @ CustomerId and @TransferAmount and adjusts the Balance of the spec fied customer by the ­spec fied amount The statement s executed first for the source customer, pass ng n the negated transfer amount (th s subtracts from the first customer’s ba ance), and s then executed aga n for the target customer, pass ng n the actua transfer amount (th s adds to the second customer’s ba ance) C ear y, you must use a transact on to ensure that both operat ons succeed or fa together To transact ona ze these updates exp c t y, you nvoke BeginTransaction on the SqlConnection to start the transact on (the SQL Profi er trace shows that th s method sends a BEGIN TRANSACTION statement to SQL Server) and obta n a SqlTransaction object ass gned to the var ab e txn Because SqlTransaction s a d sposab e object, you ca BeginTransaction w th a using statement b ock to ensure that the transact on gets d sposed of proper y at the end of the b ock The two updates are then ­executed us ng the standard ExecuteNonQuery pattern, w th the add t ona step of sett ng the ­Transaction property of both SqlCommand objects to the SqlTransaction After the second update, you ca txn.CommitTransaction On y then are both updates ­permanent y app ed to the database (the SQL Profi er trace shows that th s method sends a ­COMMIT ­TRANSACTION statement to SQL Server) The two updates are wrapped n a try/catch b ock so 452  Part III  Applied SQL

that you can ca txn.RollbackTransaction n the event of an error (th s wou d send a ROLLBACK ­TRANSACTION statement to SQL Server) Of course, SQL Server w never comm t the transact on f txn.CommitTransaction doesn’t execute, and w eventua y ro t back anyway, so the try/catch b ock sn’t str ct y necessary But t’s a bad dea to keep transact ons pend ng any onger than necessary; you shou d nstead hand e except ons that occur ns de transact ons so you can ro back mmed ate y In th s case, you’re not rea y hand ng the except on n the catch b ock; you just want to ensure an exp c t ro back on error So the except on s then s mp y rethrown (and now needs to be hand ed further up the stack) after ca ng txn.RollbackTransaction Now cons der a more rea st c scenar o w th mu t p e updates mp emented n separate methods It then becomes a greater burden to manage the transact on, because you need to pass the SqlTransaction object around and make sure t gets ass gned to the Transaction property of every SqlCommand object execut ng w th n the transact on Th ngs are much eas er us ng mp c t transact ons, as you’ see now

Tip  Although our coverage of implicit transactions is provided here in the section on raw data access objects, it pertains to all the layered technologies as well. You should i­mplicitly transactionalize your updates using a TransactionScope block (as demonstrated here) whether you are working with raw objects, DataAdapters with DataSets, LINQ to SQL, or Entity Framework. Drag another button contro from the Too box and drop t onto the form, name t ­ tnUpdateWithImplicitTransaction, and set the Text property to Update with Implicit b ­Transaction Then doub e-c ck the button and add the code n L st ng 10-9 Listing 10-9  Updat ng data us ng mp c t transact ons.

private void btnUpdateWithImplicitTransaction_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "UPDATE Customer SET Balance += @Transfer WHERE CustomerId = @CustomerId"; var fromCustomerId = Convert.ToInt64(this.txtTransferFromCustomerId.Text); var toCustomerId = Convert.ToInt64(this.txtTransferToCustomerId.Text); var transferAmount = Convert.ToDecimal(this.txtTransferAmount.Text); using (var ts = new TransactionScope()) { using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd1 = new SqlCommand()) { cmd1.Connection = conn;



Chapter 10  The M crosoft Data Access Juggernaut   453

cmd1.CommandText = TSql; cmd1.Parameters.AddWithValue("@CustomerId", fromCustomerId); cmd1.Parameters.AddWithValue("@Transfer", decimal.Negate(transferAmount)); cmd1.ExecuteNonQuery(); } using (var cmd2 = new SqlCommand()) { cmd2.Connection = conn; cmd2.CommandText = TSql; cmd2.Parameters.AddWithValue("@CustomerId", toCustomerId); cmd2.Parameters.AddWithValue("@Transfer", transferAmount); cmd2.ExecuteNonQuery(); } conn.Close(); } ts.Complete(); } }

The key object here s TransactionScope, wh ch s not techn ca y part of ADO NET TransactionScope s prov ded by the transact on management API ntroduced w th NET 2 0, and s conta ned n the ­System. Transactions assemb y (deta ed coverage of th s API s prov ded n Chapter 4, wh ch s ded cated to ­transact ons) So before th s code w comp e, you need to reference that assemb y and mport ts namespace w th a using d rect ve R ght-c ck the project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, scro to find the System.Transactions component, and doub e-c ck t Then add the fo ow ng using d rect ve at the very top of the source code using System.Transactions;

The project shou d now bu d successfu y Once aga n, start a SQL Profi er trace and s ng estep through the code You’ now see how exp c t transact ons behave d fferent y than mp c t ­transact ons Rather than exp c t y beg nn ng, comm tt ng, and ro ng back transact ons, you nstead wrap your updates ns de of a TransactionScope b ock Then you s gna success by nvok ng the Complete method on the TransactionScope object before the end of the b ock, such that an except on occurr ng w th n the b ock means that Complete won’t get ca ed SQL Profi er revea s that the transact on does not start when you nstant ate the TransactionScope, nor does t comm t when you nvoke ts Complete method Rather, the TransactionScope b ock mere y declares that a updates ns de the b ock need to be transact ona zed Th s nc udes ca s ns de the b ock that you make to other methods wh ch may themse ves dec are add t ona TransactionScope b ocks that are nested ns de the ca er’s b ock You can nest TransactionScope b ocks as many eve s as you need to As ong as each b ock nvokes the Complete method before t ends (and no unhand ed except ons occur w th n any of the b ocks), the transact on w comm t when the outermost b ock term nates Nest ng b ocks prov des you w th great flex b ty, because the transact on s a ways created and comm tted at the outermost b ock eve Nested b ocks s mp y p ggyback off that transact on The outermost b ock creates and m ­ a nta ns 454  Part III  Applied SQL

a s ng e transact on assoc ated w th the database connect on, even f the database connect on s opened and c osed mu t p e t mes ns de of nested b ocks Th s s because ADO NET s smart enough not to actua y c ose and reopen the connect on wh e ns de of a pend ng mp c t transact on W th th s scheme, transact ons get started and comm tted automat ca y as needed at runt me So the actua database transact on beg ns mp c t y, some time after the outermost TransactionScope b ock s entered The SQL Profi er trace shows that ADO NET ssues the BEGIN TRANSACTION statement to SQL Server when the connect on s opened, severa nes nto the TransactionScope b ock S m ar y, the database transact on comm ts mp c t y, just after the outermost TransactionScope b ock s ex ted (aga n, under the cond t on that Complete has been ca ed w th n the b ock and a nested b ocks) The trace shows a COMMIT TRANSACTION statement sent to SQL Server just after the TransactionScope b ock ends, even though that occurs after you c ose the connect on (wh ch, be ng ns de a pend ng mp c t ­transact on, won’t actua y c ose the connect on) and ca the Complete method Un ke w th exp c t transact ons, there s no transact on object for you to pass around and keep track of, nor do you need to worry about s­ ett ng the Transaction property of every SqlCommand object that s part c pat ng n the update Your on y ­concern s to ca Complete before the TransactionScope b ock ends, and th s dramat ca y s mp fies the task of wr t ng transact ona code

More Info  Chapter 4 is dedicated to the subject of atomic database transactions, and includes ­detailed discussion on the various transaction isolation levels supported by SQL Server. In that chapter, we show how you can control these isolation levels from the ­application layer using ADO.NET as well. Now that you know to use the raw data access objects, you can advance to the second component of convent ona ADO NET, the DataSet

Working with DataSets Techn ca y, one can create a data access ayer ent re y on just the raw objects— ndeed, many ­successfu app cat ons have been bu t that way St , M crosoft cou d not rea st ca y have re eased the NET Framework n t a y w thout a so prov d ng the DataSet Do ng so wou d have been an unacceptab e step backward for COM-based deve opers a ready accustomed to the conven ence of the RecordSet object In c ass c ADO, the RecordSet prov des a scro ab e cursor that can random y nav gate a resu t set returned by SQL Server And so, the DataSet was ntroduced n ADO NET 1 0 as a d sconnected, mu t -tab e cache to rep ace the (usua y) connected s ng e-tab e RecordSet cursor n c ass c ADO There s by now a great dea of ex st ng DataSet-based NET code n product on that st won’t be phased out for 5 years or more (some compan es are st runn ng code today that s eas y 10 or more years o d) P us, there may st be t mes when t s appropr ate to use DataSets So deve opers w need to keep the r DataSet sk s sharp for a ong t me to come



Chapter 10  The M crosoft Data Access Juggernaut   455

To Use or Not to Use DataSets Shou d you work w th DataSets? That has been the subject of much debate s nce the dawn of NET, and s even more debatab e w th the advent of S ver ght, ADO NET Ent ty ­Framework, and the newest W nRT runt me n W ndows 8 Some deve opers d sm ss DataSets out of hand, pr mar y because—desp te the r ab ty to be strong y typed and to encapsu ate ­bus ness ­og c—they are not true bus ness objects For examp e, you need to nav gate through ­re at onsh p objects n the DataSet to connect between parent and ch d rows Th s s not ­ntu t ve to object-or ented programmers, who th nk of parent–ch d re at onsh ps n s mp er terms; each parent has as a ch d co ect on property and each ch d has a parent property Furthermore, the DataSet parad gm does not a ow for nher tance, wh ch s a so extreme y mportant to object-or ented deve opers Spec a hand ng s a so requ red for nu va ues n a DataSet Notw thstand ng these concerns, we don’t genera y advocate d sm ss ng any techno ogy out of hand Every app cat on s d fferent, and you are do ng yourse f a d sserv ce f you don’t exam ne the facets of a ava ab e cho ces on a case-by-case bas s L ke anyth ng e se, ­DataSets can be used or they can be abused, and t’s true that they do present m tat ons f you try to ut ze them as bus ness objects But f what you need s a gener c n-memory ­database mode , then that’s what a DataSet s, and that’s what a DataSet g ves you A though the ­aforement oned concerns are a va d, the fact rema ns that DataSets are very powerfu and can serve ­extreme y we as data transfer objects Furthermore, they have the un que ab ty to dynam ca y adapt the r shape accord ng to the schema of whatever data s streamed nto them, wh ch s ­someth ng that none of the newer APIs can do So to re terate, DataSets are not obso ete The DataSet s a cornerstone of the NET Framework, and t cont nues to be supported by the newest NET Framework 4 5 n the W ndows 8 W n32 ­compat b ty runt me Even when LINQ to SQL and Ent ty Framework were first re eased, M ­ crosoft was enhanc ng the DataSet For examp e, the TableAdapterManager was added n NET 3 5 to great y s mp fy h erarch ca updates, as you’ see n an examp e short y The fact s that DataSets do st have the r p ace, and you w cont nue to encounter them for a ong t me to come as you ma nta n ex st ng app cat ons Hav ng sa d that, we must a so stress that M crosoft has c ear y pos t oned ADO NET Ent ty Framework as the preferred data access techno ogy today and n the future, ec ps ng both DataSets and LINQ to SQL Furthermore, ne ther S ver ght nor the NET Framework 4 5 n the W ndows 8 nat ve W nRT runt me supports the DataSet

A DataSet conta ns one or more DataTable objects, each of wh ch n turn conta ns rows and ­co umns The DataSet has a of the bas c character st cs of a re at ona database, such as pr mary keys, defau t va ues, re at onsh ps and constra nts, but has no awareness of any part cu ar u ­ nder y ng data store In fact, t’s poss b e to use a DataSet w th mu t p e data stores, or even none at a (that s, an ent re y n-memory scenar o) It s the job of a DataAdapter to shutt e data back and forth b ­ etween 456  Part III  Applied SQL

a DataSet and a spec fic back-end data store The DataAdapter—not the DataSet—works as an ­abstract on over the raw ADO NET command objects that are used to get data nto and out of the DataSet Many ADO NET data prov ders are ava ab e today that offer a DataAdapter to br dge the D ­ ataSet w th d fferent p atforms W th the SQL Server prov der (Sq C ent), the DataSet s pa red w th the ­SqlDataAdapter F gure 10-5 dep cts these objects and shows the nteract on between the ­database, DataAdapter, and DataSet

Note  This chapter covers just the SQL Server provider, though we often refer to the SqlDataAdapter object using the more generic term DataAdapter. Similarly, we refer to data readers as SqlDataReader and DataReader interchangeably.

DataSet Table Column Column Column Column

DataAdapter Table Column Column Column

SelectCommand InsertCommand UpdateCommand DeleteCommand

Column Column

Database Figure 10-5 The DataSet and DataAdapter work together to retr eve and update data n SQL Server.

The SqlDataAdapter fac tates fi ng and updat ng (v a ts apt y named Fill and Update methods) data n the DataSet It ho ds four d st nct SqlCommand objects, exposed by the SelectCommand property (used when you ca Fill), and the InsertCommand, UpdateCommand, and DeleteCommand propert es (used when you ca Update) It s a common m sconcept on that DataSets are s ower than DataReaders, when n fact DataReaders are used nterna y when you ca Fill on a DataAdapter What t rea y comes down to s whether you are cach ng the data versus process ng t as you stream t Beh nd the scenes, the Fill method works just ke what you’d code by hand to popu ate a co ect on of objects n memory us ng the DataReader techn ques we just covered Spec fica y, ca ng Fill nvokes ExecuteReader on the SqlCommand object referenced by the SelectCommand of the DataAdapter, and thus obta ns a SqlDataReader The adapter s smart enough to open the SqlConnection (referenced by the Connection property of the SqlCommand object) first, f t sn’t a ready opened It then ca s Read to terate the resu ts one row at a t me For each row, t

Chapter 10  The M crosoft Data Access Juggernaut   457

nstant ates and popu ates a DataRow object, and adds the DataRow to a DataTable n the DataSet If mu t p e resu t sets are returned by the server, then mu t p e DataTables are automat ca y nstant ated to accommodate them After the ast row of data n the ast resu t set s processed, t c oses the reader when done F na y, f the connect on wasn’t opened before the Fill, the connect on s c osed as we At th s stage, the connect on w th the server has been severed, and the DataAdapter can be d scarded f des red Ca ng Update on the DataAdapter prov des a s m ar abstract on to push changes made n the DataSet ( nserts, updates, and de etes) back to the database Spec fica y, ADO NET exam nes the ­RowState property of each row n a DataTable to determ ne whether ExecuteNonQuery shou d be ca ed, and on wh ch SqlCommand object (InsertCommand, UpdateCommand, or D ­ eleteCommand) t shou d be ca ed Each DataRow ho ds current and or g na va ues, mak ng t easy to map both ­vers ons of any co umn to the three update commands Because Update works on y on one tab e at a t me, t can be tr cky to hand e h erarch ca updates when the database s ­enforc ng ­referent a ­ntegr ty constra nts between tab es New parent rows need to be added before the r ch d rows get added, whereas de et ons need to be coded n the reverse order Fortunate y, the ­TableAdapterManager was ntroduced n NET 3 5 to do a that work for you As you’ see n an ­upcom ng examp e, the TableAdapterManager can push a the nsert ons, mod ficat ons, and ­de et ons across an ent re ­DataSet back to SQL Server n just one ne of code The DataSet s a very powerfu data conta ner that supports both b nary and XML ser a zat on across t ers and can track ts own changes It s an eas y b ndab e, h erarch ca , d sconnected cache that features fi ter ng, sort ng, computed express ons, and even prov des transact ona capab t es (such as beg n/end ed t, and accept/reject changes) And LINQ to DataSet (wh ch we cover short y) prov des powerfu extens ons that make t even eas er to query a popu ated DataSet

Generic Versus Strongly Typed DataSets The term generic can be somewhat amb guous when used w th DataSets In one sense, and as a ready exp a ned, DataSets are a ways gener c w th respect to the r back-end data store But more ­common y, gener c DataSets refer to DataSets that have no schema defin t on, as opposed to strongly typed DataSets that do Gener c DataSets are usefu n (the very sma m nor ty of) cases when you are not aware of the schema of the data that you’re work ng w th at des gn t me In those ­s tuat ons, t’s ­actua y qu te powerfu and conven ent to have a DataSet object that starts out schema- ess (no ­defined tab es or co umns) and just dynam ca y adapts to whatever data you stream nto t (even ­Ent ty Framework can’t do that!) When you ca Fill on the SqlDataAdapter, one DataTable gets ­created for each resu t set returned by the SelectCommand and one DataColumn w th n each ­DataTable gets created for each co umn of each resu t set (the co umn data types get nferred from the resu t set) Then the rows get popu ated w th the actua data However, n the vast major ty of cases, you are aware of the schema of the data that you’re ­ ork ng w th at des gn t me, and so you want to work w th strong y typed DataSets (a so ca ed typed w DataSets) You’ start now w th examp es us ng a gener c DataSet, and then move on to create an XML Schema Defin t on (XSD) for examp es us ng a typed DataSet As you progress, you w see many benefits of work ng w th typed DataSets 458  Part III  Applied SQL

Filling and Updating a Generic DataSet Let’s get busy w th DataSets Launch V sua Stud o, create a new V sua C# W ndows Forms ­app cat on, and name t DemoDataSets Drag a button contro from the Too box and drop t onto the form Form1 created automat ca y by V sua Stud o In the Propert es w ndow, name the button btnGenericDataSet, and set the Text property to Generic DataSet Then doub e-c ck the button V sua Stud o w automat ca y create a c ck event hand er for the button, and then open the code ed tor so that you can mp ement the hand er Add the code as shown n L st ng 10-10 Listing 10-10  F

ng and updat ng w th a gener c DataSet.

private void btnGenericDataSet_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string SelectTSql = "SELECT * FROM Customer"; const string UpdateTSql = @" UPDATE Customer SET FirstName = @FirstName, LastName = @LastName, UpdatedAt = SYSDATETIME() WHERE CustomerId = @CustomerId"; var ds = new DataSet(); using (var cn = new SqlConnection()) { cn.ConnectionString = ConnStr; using (var cm = new SqlCommand()) { cm.Connection = cn; cm.CommandText = SelectTSql; using (var da = new SqlDataAdapter()) { da.SelectCommand = cm; da.Fill(ds); } } } // The self-tracking DataSet can now be serialized across the tiers var customerTable = ds.Tables[0]; if (customerTable.Rows.Count == 0) { MessageBox.Show("There are no customers"); return; } var firstCustomerRow = customerTable.Rows[0];



Chapter 10  The M crosoft Data Access Juggernaut   459

// Type casting required var customerId = (long)firstCustomerRow["CustomerId"]; var updatedAt = (DateTime)firstCustomerRow["UpdatedAt"]; MessageBox.Show(string.Format("Customer {0} was updated at {1}", customerId, updatedAt)); firstCustomerRow["FirstName"] = "Brian"; firstCustomerRow["LastName"] = "Perry"; using (var cn = new SqlConnection()) { cn.ConnectionString = ConnStr; using (var cm = new SqlCommand()) { cm.Connection = cn; cm.CommandText = UpdateTSql; cm.Parameters.Add("@FirstName", SqlDbType.VarChar, 50, "FirstName"); cm.Parameters.Add("@LastName", SqlDbType.VarChar, 50, "LastName"); cm.Parameters.Add("@CustomerId", SqlDbType.BigInt, -1, "CustomerId"); using (var da = new SqlDataAdapter()) { da.UpdateCommand = cm; da.Update(customerTable); } } } }

As w th the prev ous project, you must mport the namespace so that the comp er can recogn ze c asses n the System.Data.SqlClient namespace So a so add the fo ow ng using d rect ve at the very top of the source code using System.Data.SqlClient;

L st ng 10-10 demonstrates the fundamenta pr nc p es of data access us ng DataSets and ­ ataAdapters W th gener c DataSets, there are no XSDs, des gners, or other v sua too s nvo ved; t’s D a pure cod ng exper ence n ADO NET The code beg ns w th a hard-coded connect on str ng constant, as before Th s s the ast t me that you w fo ow th s ant -pract ce; the rema n ng examp es n the chapter store the connect on str ng n a configurat on fi e so that t can be a tered w thout recomp ng the app cat on Two str ng ­constants, SelectTSql and UpdateTSql, are then defined as d rect T-SQL statements aga nst the Customer tab e that, respect ve y, retr eve a the customers from the database and update a s ng e customer’s ­FirstName and LastName fie ds For now, you aren’t support ng de et ons and nsert ons, nor are you dea ng w th any other Customer co umns or the re ated OrderHeader tab e A new gener c DataSet s then nstant ated and ass gned to the var ab e ds The code s now go ng to fi ds w th customers, change one of the customer’s nformat on, and then save those changes back to the database Let’s start w th the fi operat on 460  Part III  Applied SQL

The SqlConnection and SqlCommand are nstant ated w th using b ocks, just as we’ve demonstrated n a the prev ous examp es Not ce, however, that the code does not actua y open the SqlConnection Th s s gna s the fi operat on to automat ca y open the connect on before t oads the resu t set and c oses t afterwards (converse y, f you d d open the connect on, the fi operat on wou d not try to open t aga n, nor wou d t c ose the connect on afterwards) Now the SqlDataAdapter object s created, and ts ­SelectCommand property s po nted to the SqlCommand object configured to SELECT the ent re customer tab e A the p eces are now n p ace to ca the Fill method When you ca the Fill method on the SqlDataAdapter, the SqlCommand object of the adapter’s S­ electCommand property s executed to fi the DataSet As exp a ned, th s happens by ca ng ­E xecuteReader and terat ng a SqlDataReader beh nd the scenes to popu ate a DataTable ns de the DataSet w th DataRows Because th s s pure y a retr eva operat on, the SqlDataAdapter doesn’t care that you haven’t set the other three SqlCommand propert es After the Fill, the DataSet (wh ch had been devo d of both schema and data) now has a s ng e popu ated DataTable n t w th a schema der ved from the resu ts Spec fica y, the Columns co ect on of the DataTable s popu ated w th ­DataColumn objects whose data types are nferred by the resu t set returned from the server At th s t me, every DataRow n the DataTable has a RowState property whose va ue s set to Unchanged The server connect on s severed at th s po nt, and the data access objects are d sposed On y the popu ated DataSet rema ns The code now makes some changes n the DataSet and updates those changes w th another DataAdapter A though everyth ng s coded n a s ng e method, th s ­demonstrates that a DataSet ex sts ent re y ndependent y of the DataAdapter, and the DataAdapter’s under y ng connect ons and commands used to shutt e data nto and out of the DataSet Between the t me that one SqlDataAdapter s used to ca Fill and the t me that another SqlDataAdapter s used to ca Update, the DataSet cou d potent a y be ser a zed across mu t p e t ers up to the ­presentat on ayer, bound to contro s n the user nterface, and then transported back down to the data access ayer to be updated

Note  We stress this point because it highlights a key difference between DataSets and the much newer and more advanced Entity Framework that is coming up later in the chapter. The independence of the DataSet as a self-tracking container makes it extremely easy to develop for n-tier scenarios. With EF, conversely, stateful entity objects rely on the ObjectContext for change tracking, posing much greater challenges in n-tier architectures. We will elaborate more on this point when discussing EF. Next the code gets a hand e onto the DataTable ho d ng the resu ts, wh ch s the first (and ­current y on y) member n the DataSet’s Tables co ect on If there are no customers t s mp y d sp ays a ­message and ex ts the method Otherw se, t gets a hand e onto the DataRow ho d ng the first customer, wh ch s the first member n the DataTable’s Rows co ect on It then uses str ng ndexes nto the DataRow to extract the CustomerId and UpdatedAt va ues of the first customer and d sp ays them n a MessageBox After the MessageBox, the customer’s name s changed to Br an Perry The code that d rect y changes the va ue of the FirstName and LastName co umns automat ca y a ters the RowState

Chapter 10  The M crosoft Data Access Juggernaut   461

­ roperty of the customer DataRow from Unchanged to Modified Th s s exact y the way the ­DataSet p wou d be affected f a user typed the changes nto a gr d on a form that was bound to the ­DataTable Furthermore, the prev ous first and ast name va ues just overwr tten are st ava ab e, so you can a ways access or g na va ues n a mod fied DataSet (across a t ers) for any purpose that your ­app cat on requ res Now the changes are ready to be saved In a rea n-t er app cat on, you can mag ne th s wou d be the po nt where the user c cks Save and the DataSet gets ser a zed back down across the t ers to the data access ayer so that t can be updated w th another SqlDataAdapter—th s one configured for ca ng Update rather than Fill Once aga n, the code nstant ates SqlConnection and SqlCommand objects And aga n, by not open ng the connect on, the adapter nterna y opens and c oses the connect on on ts own The SqlCommand w be used to update every row n the DataTable whose RowState property s set to Modified, and the Parameters co ect on of the SqlCommand object s popu ated us ng an over oad of the Add method that maps parameter names n the T-SQL statement to correspond ng co umn names n the DataTable F na y, the SqlDataAdapter object s created, ts UpdateCommand property s po nted to the SqlCommand object, and ts Update method s nvoked As exp a ned, th s causes ExecuteNonQuery to be nvoked on each DataRow n the DataTable passed to the Update method that has a RowState of Modified Start a SQL Profi er trace and g ve the code a run Watch the trace as you s ng e-step over the Fill and Update methods to see the act on as t happens beh nd the scenes (The Update method w update a rows w th a RowState of Modified, but because your code on y mod fies one customer row, you w on y see a one UPDATE n the trace ) Th s code works because t knows that the on y RowState va ue that the adapter’s Update method s go ng to encounter (other than Unchanged) w be Modified If a RowState of Added or Deleted wou d be encountered, th s code w fa , because the Update method wou d ook respect ve y to the InsertCommand and DeleteCommand propert es for the appropr ate SqlCommand to execute, and those propert es haven’t been set To support nsert ons and de et ons, you’d need to add the correct parameter zed T-SQL statements to the code, as demonsrated for updates And to support the OrderHeader tab e as we wou d requ re even more code to dea w th another tab e’s Fill and Update operat ons, as we as the extra og c to proper y hand e the order of de et ons and nsert ons across the h erarch ca compos t on of parent and ch d re at onsh ps Th s qu ck y adds up to a ot of code that you need to wr te and ma nta n for typ ca app cat ons that work w th many re ated tab es Strong y typed DataSets (a ong w th the TableAdapterManager) come to the rescue here In the next sect on you w earn how to wr te ess code and get more done us ng typed DataSets nstead of gener c DataSets

Building Strongly Typed DataSets A typed DataSet (short for strong y typed DataSet) offers numerous benefits over the gener c ­ ataSet When creat ng your app cat on, you use a graph ca des gner n V sua Stud o to v sua y D bu d the schema of your typed DataSet You ay out the DataSet by dragg ng and dropp ng tab es and re at onsh ps from the Server Exp orer, and then further custom ze ts propert es (occas ona y 462  Part III  Applied SQL

w th the ass stance of a w zard) When you perform these act ons, you are actua y ed t ng an XML Schema Defin t on (XSD) fi e n the background that descr bes a the tab es, co umns, data types, and re at onsh ps you’ve a d out for the DataSet V sua Stud o prov des a custom code-generat on too that then reads the XSD fi e and produces NET source code from t The process s tr ggered when any change s made to the XSD fi e, wh ch s any t me you make a change w th the graph ca des gner The generated code defines a c ass that nher ts from ­DataSet—so that a typed DataSet s everyth ng that a gener c DataSet s—but t a so prov des strong y typed members for a of the e ements (tab es, co umns, re at onsh ps) defined by the XSD In add t on to strong y typed DataTable schemas, NET 2 0 enhanced the comb ned v sua -des gner/ code-generat on exper ence by ntroduc ng TableAdapters These are s mp y strong y typed ­wrappers around DataAdapters that are pre-configured (as you’ see n the next examp e) w th the four ­SqlCommand objects—comp ete w th parameters, connect on str ng, and just about any other propert es you can th nk of sett ng The resu t s a drast c reduct on n the amount of code you need to wr te, because the des gner-generated code s now do ng the work to hook up a four command objects and the r ­parameters to an adapter, for each tab e Now that we’ve exp a ned some of the benefits, et’s tap nto the power of typed DataSets In S­ o ut on Exp orer, r ght-c ck the project and choose Add New Item In the Add New Item d a og, scro down to find and c ck on DataSet Name the new DataSet SampleDS.xsd and c ck Add V sua Stud o creates the fi e and opens t n the typed DataSet des gner In the center of the empty ­des gn surface, c ck the Server Exp orer nk The Server Exp orer w he p you get started qu ck y w th ­bu d ng your typed DataSet, but you’ first need to g ve t a connect on to the database In the Server Exp orer, r ght-c ck Data Connect ons and choose Add Connect on Se ect M crosoft SQL Server as the data source and c ck Cont nue In the Add Connect on d a og, type localhost for the Server name, se ect SampleDB from the database drop-down st, and c ck OK (you must choose d fferent sett ngs f you have created the database on another server or nstance) V sua Stud o ­creates the connect on beneath the Data Connections node n the Server Exp orer Expand the new connect on node n Server Exp orer, and then expand the Tables node beneath that, as shown n F gure 10-6

Figure 10-6  V ew ng data connect ons n V sua Stud o s Server Exp orer.



Chapter 10  The M crosoft Data Access Juggernaut   463

C ck the first tab e (Customer), ho d down the Sh ft key, and then c ck on the ast tab e (­ OrderHeader) to se ect a four tab es Then drag the four tab es from the Server Exp orer and drop them onto the DataSet des gn surface After rearrang ng the tab es to get a better v ew of how they re ate, your DataSet shou d ook s m ar to F gure 10-7

Figure 10-7  The typed DataSet des gn surface n V sua Stud o.

Severa very mportant th ngs have happened automat ca y at th s po nt F rst, not ce that an ­ pp cat on configurat on fi e (named app.config) has been added to the project Open ­app.config and a you’ see a e ement w th an e ement nested ns de of t The ­e ement defines a name for the connect on, and a so conta ns the comp ete connect on str ng to the database If and when you need to change the connect on str ng, you’ be ab e to ed t the ­configurat on fi e and won’t need to recomp e the app cat on as you wou d w th a hard-coded ­connect on str ng You can a so dep oy the app cat on to d fferent env ronments (deve opment, ­test ng, product on), and ed t the configurat on fi e accord ng y to adjust the connect on str ng appropr ate y for those env ronments Now c ose app.config and return to the typed DataSet des gner Next, you can see that the des gner has created tab es, co umns, and re at onsh ps n the DataSet that essent a y m m c the database structure The Server Exp orer does a very good job at d scover ng and rep cat ng schema from the database to the DataSet des gner, but t’s not perfect For examp e, t won’t p ck up on defau t va ues In the Customer tab e, c ck the Balance co umn and not ce n the Propert es w ndow that t does not have a defau t va ue, even though a defau t va ue of 0 s defined for Balance n the database You can use the Propert es w ndow to set the defau t va ue and make other tweaks as necessary after dragg ng and dropp ng tab es from Server Exp orer onto the DataSet

464  Part III  Applied SQL

F na y, the des gner d sp ays TableAdapters attached at the bottom of each tab e Be ng that there s one TableAdapter for fi ng and updat ng each DataTable, th s s a reasonab y conven ent d sp ay However, t can a so be qu te m s ead ng Remember that TableAdapters are rea y just ­DataAdapters, and that DataTables conta ned ns de of DataSets ex st ent re y ndependent y of DataAdapters So wh e you use the des gner to eas y configure each tab e together w th ts adapters, the code ­generator actua y produces d st nct DataSet and TableAdapter c asses An nstance of a typed DataSet nc udes on y typed DataTables; t does not nc ude the correspond ng TableAdapters Th s makes sense, because when the DataSet s ser a zed up to the c ent across the network, the adapter nformat on s mean ng ess TableAdapters are nstant ated ndependent y of DataSets on y on the ­app cat on server connected to the database ns de the firewa , n order to fi and update them just as you’ve been do ng a a ong By defau t, the des gner has generated a TableAdapter for each tab e, w th each one a ready configured w th the four command objects needed to fi and update the tab e It a so generated ­appropr ate d rect T-SQL statements for each command n each tab e, and configured the r parameter co ect ons Compared w th the amount of code n the prev ous gener c DataSet examp e, t s obv ous that typed TableAdapters offer a huge sav ngs n t me and effort

Mapping Stored Procedures to the Typed DataSet Now you w use the CRUD stored procedures defined for the Customer tab e nstead of the d rect SQL generated by the des gner for each of the four command objects n ts TableAdapter C ck the ­CustomerTableAdapter head ng n the des gner The Propert es w ndow shows the four command ­propert es SelectCommand, InsertCommand, UpdateCommand, and DeleteCommand Exam ne each of them and you w see engthy d rect T-SQL statements ass gned to the CommandText property and correspond ng parameters popu ated n the Parameters co ect on property You’ use the Tab eAdapter Configurat on W zard to change th s so that the CRUD stored procedures are used nstead R ght-c ck the CustomerTableAdapter head ng and choose Configure It may seem odd, but th s drops you off n the middle of the w zard, so c ck the Prev ous button to go back to the Choose a Command Type page The Use SQL Statements rad o button s se ected by defau t, wh ch means that the Customer tab e shou d use the d rect SQL statements generated automat ca y when you dragged and dropped from Server Exp orer The second rad o button w actua y create new stored ­procedures, wh ch wou d conta n the same automat ca y generated SQL statements Se ect the th rd rad o button to te the w zard that you want to use ex st ng stored procedures, as shown n F gure 10-8 C ck Next For the Se ect, Insert, Update, and De ete drop-down sts, choose SelectCustomers, InsertCustomer, UpdateCustomer, and DeleteCustomer, respect ve y When you map the Se ect, the w zard d scovers the co umns returned by the stored procedure and d sp ays them n the pane on the r ght, as shown n F gure 10-9 (the capt on “Set Se ect procedure parameters” above the co umn st s actua y ncorrect; t shou d rea y state “Se ect procedure co umns”) When you map the Insert, ­Update, and De ete, the w zard d scovers the parameters requ red by the stored procedure and ­d sp ays them n the r ght-hand pane



Chapter 10  The M crosoft Data Access Juggernaut   465

Figure 10-8  Te ng the w zard to use ex st ng stored procedures for a tab e adapter.

Figure 10-9  Mapp ng stored procedures to a tab e adapter.

Now c ck F n sh Rev s t the Propert es w ndow for the CustomerTableAdapter and you’ see someth ng very d fferent than before Rather than a engthy T-SQL statement, the four CommandText propert es mere y conta n the stored procedure name, as shown n F gure 10-10

466  Part III  Applied SQL

Figure 10-10  Tab e adapter propert es show ng mapped stored procedures.

Because stored procedures parameters are cons stent y named w th the tab e co umns, the w zard correct y mapped a the parameters to co umns It even guessed to map the parameter @OriginalUpdatedAt to the UpdatedAt co umn for the UpdateCustomer stored procedure But you st need to tweak that mapp ng In the Propert es w ndow, expand the UpdateCommand property and c ck the e pses next to Parameters to open the Parameters Co ect on Ed tor Then c ck the @OriginalUpdatedAt parameter The des gner correct y set the SourceColumn to UpdatedAt co umn even though the parameter name s prefixed w th the word Original, but the SourceVersion s st set to Current Th s w work fine as ong as the c ent never changes the UpdatedAt va ue after they retr eve t, but you can never trust what the ­c ent m ght do So nstead, you want to spec fy the or g na UpdatedAt co umn that was retr eved from the database and not worry about the c ent poss b y overwr t ng t (acc denta y or de berate y) Because each DataRow reta ns every co umn’s or g na va ue, t s easy for you to map the or g na va ue of any co umn to any stored procedure parameter that needs t (Tangent a y, t s a so very conven ent that the va dat ons ru e and ca cu at ons n your bus ness og c can access or g na va ues n a DataSet ) Change the ­SourceVersion for @OriginalUpdatedAt from Current to Original, as shown n F gure 10-11, and c ck OK Then save and bu d the project to make sure that t comp es successfu y A though the who e po nt of the des gner s to sh e d you from the deta s of the XSD and the typed DataSet c ass that gets generated from t, fam ar z ng yourse f w th both of these fi es s very en ghten ng, and w go a ong way n he p ng you understand the techno ogy So before wr t ng the code for th s fina DataSet examp e, go have a qu ck ook at the fi es beh nd the des gner



Chapter 10  The M crosoft Data Access Juggernaut   467

Figure 10-11  Mapp ng a co umn s or g na va ue to a stored procedure parameter.

R ght-c ck SampleDS.xsd n the So ut on Exp orer and choose Open W th Se ect XML (Text) Ed tor and c ck OK to v ew the XSD fi e as XML Th s is the XML that you are compos ng as you bu d your typed DataSet n the des gner w th Server Exp orer drag-and-drop, the Tab e Adapter Configurat on W zard, and the Propert es w ndow Scro through the XSD, and you w find an XML representat on of everyth ng you added to the DataSet us ng the des gner Next, expand the SampleDS.xsd fi e n the So ut on Exp orer to revea the SampleDS.Designer.cs fi e nested beneath t Doub e-c ck t to open the C# code, wh ch s generated automat ca y whenever the XSD s updated In th s code, you w see the SampleDS c ass that nher ts from DataSet, as we as tab e c asses that nher t from DataTable w th fu y descr bed co umns Scro a b t more and you w find the generated TableAdapter code, wh ch uses code s m ar to what you wrote by hand ear er to set up the four command objects and the r parameters w th an adapter for each tab e W th a of th s funct ona ty now baked nto your typed DataSet, you’re go ng get a ot more done w th a ot ess code, compared to work ng w th gener c DataSets Drag another button ­contro from the Too box and drop t onto the form In the Propert es w ndow, name the button ­btnTypedDataSet, and set the Text property to Strongly Typed DataSet Then doub e-c ck the button and add the code n L st ng 10-11 Listing 10-11  F

ng and updat ng w th a strong y typed DataSet.

private void btnTypedDataSet_Click(object sender, EventArgs e) { var ds = new SampleDS(); using (var da = new SampleDSTableAdapters.CustomerTableAdapter()) { da.Fill(ds.Customer); }

468  Part III  Applied SQL

using (var da = new SampleDSTableAdapters.OrderHeaderTableAdapter()) { da.Fill(ds.OrderHeader); } // The self-tracking DataSet can now be serialized across the tiers if (ds.Customer.Count == 0) { MessageBox.Show("There are no customers"); return; } var firstCustomerRow = ds.Customer[0]; // No type casting required var customerId = firstCustomerRow.CustomerId; var updatedAt = firstCustomerRow.UpdatedAt; var orderRows = firstCustomerRow.GetOrderHeaderRows(); MessageBox.Show( string.Format("Customer {0} was updated at {1} and has {2} order(s)", customerId, updatedAt, orderRows.Length)); // Change the customer's name firstCustomerRow.FirstName = "David"; firstCustomerRow.LastName = "Jones"; // Add a customer order ds.OrderHeader.AddOrderHeaderRow(firstCustomerRow, "Regular Mail", "Open", string.Empty, DateTime.Now, DateTime.Now); using (var tam = new SampleDSTableAdapters.TableAdapterManager()) { tam.CustomerTableAdapter = new SampleDSTableAdapters.CustomerTableAdapter(); tam.OrderHeaderTableAdapter = new SampleDSTableAdapters.OrderHeaderTableAdapter(); tam.UpdateAll(ds); } }

F rst, a new nstance of SampleDS (your typed DataSet) s created Un ke a new y ­nstant ated ­gener c DataSet, th s DataSet s fu y oaded w th schema nformat on, and has DataTable, ­DataColumn, and DataRelation objects for everyth ng defined n the XSD a ready created ns de of t the moment t s nstant ated The on y th ng m ss ng are the rows of data themse ves Next, a the customers and orders are retr eved from the database Each tab e s fi ed us ng the TableAdapter you configured n the des gner For the CustomerTableAdapter, you know that means

Chapter 10  The M crosoft Data Access Juggernaut   469

the SelectCustomers stored procedure w be ca ed For the OrderHeadersTableAdapter, the defau t des gner-generated T-SQL SELECT statement w be ca ed When you run the app cat on, s ng e step over both Fill method ca s as you exam ne the runn ng trace n SQL Profi er to ver fy that these are ndeed the statements be ng sent by ADO NET to SQL Server

Note  Realistically, your fill operations will be parameterized to filter just a subset of the data from the database. In the designer, you can create multiple data retrieval queries in addition to the one associated with the SelectCommand property. These queries can be tied to parameterized stored procedures, and the designer will generate FillByXxx methods (for example FillByNameSearch, FillByState, FillByCategory, and so on) with .NET parameters that match the T-SQL parameters of the stored procedure. This way, through its ­associated TableAdapter, each DataTable can be filled using any number of queries with varying ­parameters for controlling the subsets of data that will get retrieved from the database. Once aga n, the server connect on s now severed, the data adapters are d sposed, and on y the popu ated DataSet rema ns In an n-t er scenar o, t wou d be ser a zed across mu t p e t ers up to the presentat on ayer at th s po nt Now you start to enjoy the benefits of strong typ ng You don’t need to refer to the customer DataTable by ndex, as n ds.Tables[0] or ds.Tables[“Customer”], because the typed DataSet has a ­property ca ed Customer (that the comp er won’t et you m sspe ), and t has a Count property that te s you how many customer rows were retr eved If there are no customers, the code ­d sp ays a ­message and ex ts the method Otherw se, t gets a hand e onto the DataRow ho d ng the first ­customer, wh ch s the first member n the typed DataRow co ect on exposed by the Customer property It then extracts the CustomerId and UpdatedAt va ues of the first customer as n the ast examp e On y th s t me, t uses genu ne long and DateTime propert es exposed by the strong y typed DataRow You don’t need to cast from a generic object type or use str ng ndexes—both of wh ch are undes rab e, nconven ent, and error-prone pract ces The Inte sense feature n V sua Stud o makes t very easy to find the r ght property, and f you st manage to m sspe a property name, your project won’t bu d Because of the re at onsh p defined n the typed DataSet between Customer and OrderHeader, the typed customer DataRow s a so equ pped w th a GetOrderHeaderRows method Th s method returns an array of DataRow objects for the orders be ong ng to th s customer (wh ch s a subset of the comp ete st of orders be ong ng to a customers that you fi ed from the database) You ca ­GetOrderHeaderRows and then show a MessageBox that d sp ays the customer’s ID, updated date, and order count After the MessageBox, the customer’s name s changed to Dav d Jones Aga n, strong y typed propert es for FirstName and LastName are vast y super or to the gener c approach n the prev ous examp e, where you were constant y cast ng back and forth between object and spec fic data types You won’t be ab e to ass gn ncorrect data types or m sspe co umn names w th typed DataSets ­Mak ng these (extreme y common) m stakes w th gener c DataSets do not get caught by the ­comp er, and t w be the poor unsuspect ng user that d scovers the prob em w th an ug y error 470  Part III  Applied SQL

message d sp ayed by an unhand ed except on You shou d be gett ng the dea that our very strong ­recommendat on s to a ways use typed DataSets over gener c DataSets, except for cases where you need to d scover schema dynam ca y at runt me Gener c DataSets are the perfect too for the job n such scenar os (they’re a so great for “throw-away” apps, ke qu ck demos or proof-of-concepts) In add t on to chang ng the customer name, the code adds a new customer order Aga n, due to the re at onsh p between Customer and OrderHeader, the OrderHeader tab e property of the typed DataSet s equ pped w th an AddOrderHeaderRow method that was created automat ca y by the typed DataSet code generator Th s method accepts appropr ate y typed parameters for each co umn of a new order The first parameter dent fies the new OrderHeader’s parent Customer ­Interest ng y, th s passed as the parent Customer row, rather than the CustomerId fore gn long va ue The ­rema n ng parameters supp y va ues for the new order’s ShipVia, OrderStatus, Notes, CreatedAt, and UpdatedAt co umns The typed DataRow requ res va ues for CreatedAt and UpdatedAt, wh ch are supp ed as ­DateTime.Now (based on the app cat on’s c ock) If there were CRUD stored procedures for ­OrderHeader as there are for Customer, the nsert and update stored procedures wou d be us ng SYSDATETIME (based on SQL Server’s c ock) for these co umns, gnor ng and then rep ac ng the va ues set n the DataRow Now mag ne that the user c cks Save and the DataSet gets ser a zed back down across the t ers to the data access ayer so that ts changes can be pushed back to the database Th s t me, the ­TableAdapterManager s used, and th s hand es a the concerns of h erarch ca updates when dea ng w th compos t ona object graphs of re ated ent t es If you have referent a ntegr ty constra nts on the ­re at onsh ps n the database tab es (and we strong y recommend that you do) that aren’t enab ed for cascad ng de etes (and we strong y recommend that they aren’t), a two-phase approach needs to be taken when pers st ng nsert ons and de et ons from the DataSet back to the database In part cu ar, new parent rows need to be INSERTed before new ch d rows are, whereas removed ch d rows need to be DELETEed before removed parent rows Pr or to NET 3 5, you needed to code th s og c yourse f us ng ­nd v dua DataAdapters (or TableAdapters) But the TableAdapterManager removes a that pa n As the code demonstrates, you s mp y create an nstance of a new TableAdapterManager and prov de t w th a new TableAdapter nstance for each tab e nvo ved n the update W th a s ng e ca to the UpdateAll method, the TableAdapterManager updates both tab es ns de a transact on, automat ca y revers ng the order of nsert ons and de et ons between parent and ch d tab es as necessary Run the code and watch the SQL Profi er trace as t executes You’ve now earned the most mportant techn ques for us ng convent ona ADO NET w th SQL Server Desp te how o d convent ona ADO NET s compared w th LINQ to SQL or Ent ty Framework, these newer techno og es (wh ch are very d fferent than DataSets) are based on many of the same pr nc p es that have proven so successfu w th typed DataSets—name y the use of a des gner to author an XML document represent ng your data mode , comb ned w th a code generator to produce typed ent t es for your app cat ons to work w th programmat ca y So the firm grasp you now have on these concepts w make earn ng the newer techno og es much eas er You’ start by enter ng the wor d of LINQ



Chapter 10  The M crosoft Data Access Juggernaut   471

Language-Integrated Query (LINQ) The express veness and flex b ty of anguage- ntegrated query (LINQ), first ntroduced w th NET 3 5, brought rad ca change to the way deve opers program n C# and V sua Bas c (VB) NET That’s a bo d statement, but don’t m s nterpret t to mean that LINQ s a ways the answer There are many ­mp ementat ons of LINQ, and we w soon be exam n ng many factors that you need to cons der ­before determ n ng wh ch ones are the r ght (or wrong) cho ces for your app cat on But a LINQ mp ementat ons u t mate y rest on top of LINQ to Objects, and LINQ to Objects s always the r ght cho ce when work ng w th objects Th s means that—at the very east—you shou d be us ng LINQ ­nstead of wr t ng oops wherever poss b e to query sts, co ect ons, and arrays n your NET ­app cat on You w enjoy produc ng t ghter, more e egant code that expresses what you want to do, not how to do t

Note  LINQ is a huge topic, and this is a book on SQL Server, not LINQ. We d ­ emonstrate only a fraction of the real power of LINQ in our examples, and focus instead on how ­different LINQ implementations can be made to work with data in SQL Server. For a ­detailed and concise treatment of LINQ, we recommend the LINQ Pocket Reference (O’Reilly) by Joseph Albahari and Ben Albahari. In T-SQL, you work w th set-based operat ons It wou d be absurd to mp ement a query n T-SQL by cod ng a cursor that scans a tab e, oops through t one row at a t me, and exam nes each row for some cr ter a That’s what SELECT/WHERE s for! Yet before LINQ, that’s exact y what deve opers had been do ng w th object arrays and co ect ons How many t mes have you coded a oop that scans some enumerab e sequence of objects? You cou d be fi ter ng for a subset, ca cu at ng an aggregat on, or perhaps you just need to determ ne f a part cu ar tem s present n a co ect on LINQ was des gned to e m nate a such oop ng constructs from your code You m ght st code a foreach oop to process the resu ts of a LINQ query, but the LINQ query tse f s coded as a s ng e statement rather than a procedura oop, just ke ­set-based operat ons n the database wor d So a though you shou d a ways use LINQ to Objects to query n-memory object sequences, other LINQ mp ementat ons need to be eva uated nd v dua y When mak ng your determ nat on, the first th ng to cons der s whether you are query ng d rect y over memory-res dent data structures or externa data stores Bes des “ord nary” objects quer ed w th LINQ to Objects, memory-res dent data ­structures nc ude cached XML documents (the XDocument object, wh ch can be quer ed w th LINQ to XML) or—more re evant to th s chapter—popu ated DataSets (quer ed w th LINQ to DataSet) To work w th n-memory data structures such as these, aga n, you’ a ways want to use a spec a zed LINQ mp ementat on ( f ava ab e) over the more trad t ona , procedura approaches Because the source data res des oca y n memory, your LINQ query w be ab e to ca nto any NET method n the framework or your app cat on Then there are the other LINQ mp ementat ons des gned to query externa data stores These LINQ prov ders dynam ca y, at runt me, form an express on tree from your LINQ query to generate a query n another anguage (such as T-SQL) that w then execute aga nst an externa data store (such as SQL Server) 472  Part III  Applied SQL

Th s s the case w th both LINQ to SQL and Ent ty Framework So now you ser ous y need to quest on the w sdom of wr t ng a query n one anguage (LINQ) on y to have t trans ated nto another anguage (T-SQL) If you are a ready we versed n T-SQL, you may fee that you can ­produce better T-SQL code than the LINQ runt me Converse y, th s s a very appea ng approach when T-SQL sk s are ack ng Regard ess, t’s mportant to understand that such LINQ quer es can on y ca methods n the NET Framework that have a compat b e equ va ent n T-SQL (for examp e, the way StartsWith can be mp emented us ng LIKE), and that you obv ous y won’t be ca ng methods n your own app cat on because they don’t ex st n SQL Server We’ be exam n ng LINQ to SQL and Ent ty Framework a b t further on, but you’ start your first LINQ exerc se by extend ng the prev ous examp es to work w th LINQ to DataSet

LINQ to DataSet To re terate, LINQ to DataSet operates on a memory-res dent DataSet that s a ready popu ated Th s means that you cont nue to use convent ona ADO NET to fi the DataSet Then once you’ve fi ed the DataSet, you use LINQ to DataSet to further query over the data retr eved from the database Here s an examp e of a LINQ to DataSet query var list = (from cust in ds.Customer where !cust.LastName.StartsWith("A") select cust).ToList();

Now how do you know th s s a LINQ to DataSet query, and not LINQ to someth ng e se? We , that’s part of the dea beh nd LINQ What you’re LINQ- ng to s not supposed to be mmed ate y ­obv ous, the advantage be ng that a LINQ quer es k nd of ook the same regard ess of the ­under y ng data source So you want to zone n on what fo ows the in c ause; that’s what te s you wh ch LINQ mp ementat on s be ng used In th s examp e, the in c ause s fo owed by a DataTable n a typed DataSet, so th s s LINQ to DataSet Th s query bu ds a st of customers whose ast names don’t beg n w th the etter A, and s equ va ent to the fo ow ng code that you m ght have wr tten before LINQ to ach eve the same th ng var list = new List(); foreach (SampleDS.CustomerRow cust in ds.Currency) { if (!cust.LastName.StartsWith("A")) { list.Add(currencyCode); } }

If you’re a ready fam ar w th DataSets, you probab y know that even before LINQ, the RowFilter property of the DataView object cou d have been used to ach eve the very same th ng, ke so var dv = new DataView(ds.Customer); dv.RowFilter = "Customer NOT LIKE 'A*'";

True, the query s expressed n a s ng e statement But there are two prob ems w th th s ­approach F rst, the RowFilter property s a string property Even though t s be ng used to fi ter the strong y

Chapter 10  The M crosoft Data Access Juggernaut   473

typed Customer tab e, m sspe ng Customer n the query (or any other typo) w not get caught by the comp er Second, ask yourse f what query anguage s be ng used here, because t s most ­certa n y not T-SQL Remember that you’ve a ready h t the database to fi the Customer tab e from SQL Server Indeed, th s cou dn’t be T-SQL because the LIKE operator n T-SQL uses the percent ­symbo (%) and not the aster sk (*) symbo to denote a w dcard match So what syntax s th s? As t turns out, t’s a m n -query anguage understood on y by the RowFilter property of the DataView ­object, and a pr me examp e of why LINQ was nvented W th LINQ, you don’t need to start ­earn ng d fferent query anguages (some more obscure than others) for d fferent data sources If you use LINQ to DataSet, then you don’t need to earn th s RowFilter anguage to query DataSets; f you use LINQ to XML, then you don’t need to earn XPath to query XML documents; that’s the dea And yes, w th LINQ to SQL and Ent ty Framework, you don’t have to (necessar y) earn T-SQL to query SQL Server! You just query everyth ng w th LINQ

Querying a Generic DataSet LINQ to DataSet s easy to use, and works both w th gener c DataSets and typed DataSets Th s first ­examp e w use a gener c DataSet So drag another button contro from the Too box and drop t onto the form In the Propert es w ndow, name the button btnLinqToGenericDataSet, and set the Text property to LINQ to Generic DataSet Then doub e-c ck the button and add the code n L st ng 10-12 Listing 10-12  Query ng a gener c DataSet w th L NQ to DataSet.

private void btnLinqToGenericDataSet_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string SelectTSql = "SELECT * FROM Customer"; var ds = new DataSet(); using (var cn = new SqlConnection()) { cn.ConnectionString = ConnStr; using (var cm = new SqlCommand()) { cm.Connection = cn; cm.CommandText = SelectTSql; using (var da = new SqlDataAdapter()) { da.SelectCommand = cm; da.Fill(ds); } } } var query = from cust in ds.Tables[0].AsEnumerable()

474  Part III  Applied SQL

where !cust.Field("LastName").StartsWith("A") orderby cust.Field("LastName") descending select cust; var dv = query.AsDataView(); MessageBox.Show(string.Format( "Filtered {0} of {1} customers", dv.Count, ds.Tables[0].Rows.Count)); }

None of the code n th s st ng before the LINQ query requ res any exp anat on That’s because, as we’ve sa d, you’re st h tt ng the database us ng convent ona ADO NET just ke you d d n L st ng 10-10 In th s examp e, you’ve retr eved a the customers nto a gener c DataSet, and now you’re us ng LINQ to DataSet to query the resu t set by fi ter ng out a customers whose ast name starts w th A Let’s exam ne th s LINQ to DataSet query up c ose A though the DataTable does have a Rows co ect on property that cou d theoret ca y be ­quer ed us ng LINQ to Objects, there s a spec a NET assemb y for LINQ to DataSet that supp es a few ­extens on methods des gned spec fica y for us ng LINQ aga nst cached DataSets

More Info  Visual Studio automatically set a reference to this assembly when you created the project. The assembly is named System.Data.DataSetExtensions.dll, and you will see it listed under the References node in Solution Explorer. Because the extension methods in this assembly are all defined in the System.Data namespace, you don’t need to add any special using directive to the top of the source file to make them recognizable. The key extens on method n LINQ to DataSet s AsEnumerable, wh ch you nvoke on the ­DataTable LINQ operates over any object that mp ements IEnumerable, but the DataTable object does not ­ mp ement IEnumerable (the tab e tse f s not a co ect on) W th LINQ to DataSet, you query the DataTable tse f, not ts Rows co ect on, and so you ca AsEnumerable on the DataTable to LINQ-­enab e t The rest s an ord nary LINQ query that scans the DataTable, fi ter ng and sort ng on LastName

Note  This is the first of several examples that filter customers by last names not beginning with the letter A. To get conclusive results as you walk through these examples, you should have at least one customer in the database with a last name that starts with the letter A. If you’ve not given any of the customers from earlier examples a last name that starts with A, go ahead and add another customer now with a last name beginning with A so you can test that the filtering logic works as expected. You’re a so us ng the Field extens on method to extract the LastName va ue from each ­ ataRow be ng scanned by the query Th s method makes work ng w th gener c DataSets a tt e D b t eas er by sav ng you the need to cast between object and spec fic data types when referenc ng

Chapter 10  The M crosoft Data Access Juggernaut   475

c­ o umns n a gener c DataRow Because the LastName co umn s a string type, you use Field to obta n ts va ue as a string n the where and orderby c auses w thout hav ng to d rect y cast t So now you’ve defined the LINQ query and ass gned t to the query var ab e, wh ch s dec ared ­us ng var Just ke the nput sequences they consume, LINQ quer es a ways return an IEnumerable of something; and w th LINQ to DataSet n part cu ar, that w be some spec a zed c ass that mp ements IEnumerable defined n the extens ons assemb y But you w typ ca y not need to know or care exact y wh ch c ass that s, and so var s a conven ent way of te ng the comp er to nfer the actua return type from the context of the LINQ query A you rea y care about s the fact that t returns an IEnumerable object, mean ng that the resu ts can be retr eved as a sequence of DataRows F na y, there are two more extens on methods that are un que to LINQ to DataSet, and those are AsDataView and CopyToDataTable In th s examp e, you are us ng AsDataView to execute the query and return the resu ts n an ord nary DataView (just as f you used an o d-fash oned RowFilter) The DataView conta ns the fi tered resu ts, and ho ds ve DataRow nstances t ed to the under y ng ­DataTable Changes made to the DataView are reflected n the DataSet and w get pushed back to the database when you update the DataSet w th a DataAdapter The CopyToDataTable method s ­s m ar to AsDataView, but t makes a copy of the resu ts n a new DataTable nstead of r­ eturn ng ve references n a DataView Th s s usefu when you need to c one the resu ts so that they ex st ­ndependent y of the source DataTable n the DataSet

Querying a Strongly Typed DataSet LINQ to DataSet s even eas er to use w th typed DataSets The next examp e w perform the same f­unct on as the ast one, on y th s t me us ng a typed DataSet Drag another button contro from the Too box and drop t onto the form In the Propert es w ndow, name the button btnLinqToTypedDataSet, and set the Text property to LINQ to Strongly Typed DataSet Then doub e-c ck the button and add the code n L st ng 10-13 Listing 10-13  Query ng a strong y typed DataSet w th L NQ to DataSet.

private void btnLinqToTypedDataSet_Click(object sender, EventArgs e) { var ds = new SampleDS(); using (var da = new SampleDSTableAdapters.CustomerTableAdapter()) { da.Fill(ds.Customer); } var query = from cust in ds.Customer where !cust.LastName.StartsWith("A") orderby cust.LastName descending select cust; var dv = query.AsDataView();

476  Part III  Applied SQL

MessageBox.Show(string.Format( "Filtered {0} of {1} customers", dv.Count, ds.Customer.Count)); }

Aga n, the code to fi the Customer tab e works the same as the ear er typed DataSet examp e n L st ng 10-11 And the LINQ query that fo ows a so works the same as the prev ous LINQ to gener c DataSet examp e n L st ng 10-12 But th s vers on of the LINQ query s even neater and t ghter Not ce that there are no str ng tera s at a (other than the parameter passed to StartsWith); co umn names are strong y typed and there’s no need for any cast ng to and from object, e ther exp c t y or ­us ng the Field method A so, because one of the base c asses (ca ed TypedTableBase) between the strong y typed Customer tab e and the under y ng DataTable mp ements IEnumerable, you don’t need to use the AsEnumerable method e ther Once aga n, you see that typed DataSets offer ­numerous advantages over gener c DataSets Let’s venture forward now nto the next generat on of NET data access APIs Armed w th the ­techn ques and concepts we’ve covered thus far, grasp ng them a w be eas er than you mag ne

Object Relational Modeling (ORM) Comes to .NET There s no doubt that DataSets have served programmers we over the years And f what you need s an n-memory database object mode , or a gener c data conta ner, they st rema n usefu today The DataSet a so has many features to make t “work” ke a bus ness object You’ve seen how a typed DataSet supp es propert es for the Customer tab e s m ar to how you’d code the propert es a Customer ent ty c ass yourse f And because you can embed your own bus ness og c and va dat ons d rect y nto a typed DataSet by everag ng part a c asses, DataSets can de ver many of the same encapsu at on benefits that object-or ented des gns offer when work ng w th pure bus ness ent t es But c ose s not c ose enough Though the DataSet funct ons as an exce ent data transfer object, the fact s that DataSets are not true bus ness objects The DataSet parad gm of tab es, co umns, and re at onsh ps does not a ways trans ate we to the object-or ented deas of c asses, propert es, and co ect ons A row n the typed DataSet’s Customer tab e s just that—a DataRow, w th co umns— and not rea y a convent ona bus ness ent ty object w th ts own ord nary propert es, methods, and events Because t nher ts from DataRow, t can’t a so nher t from a base c ass of your own, nor s t poss b e to have one tab e type nher t from another Furthermore, the manner n wh ch parent-ch d h erarch es of a DataSet are nav gated v a re at onsh ps and spec a zed code-generated methods s un ntu t ve to object-or ented programmers n NET who th nk of “regu ar” objects n s mp er terms A parent ent ty has a property that ho ds a co ect on of ch d ent t es, and each ch d ent ty has a property that refers back to ts parent Man pu at ng h erarch es n a DataSet s qu te d fferent, and somewhat more cumbersome Ser ous object-or ented programmers demand that the r bus ness ent ty c asses everage ­nher tance, po ymorph sm, and des gn patterns Furthermore, they ns st that the m dd e t er ­rema ns free of dependenc es on any part cu ar techno ogy The ph osophy s that data access and

Chapter 10  The M crosoft Data Access Juggernaut   477

­ resentat on ayer techno og es w come and go, but bus ness og c n the m dd e t er—wr tten p n ent ty c asses (such as the qu ntessent a public class Customer)— s bu t to ast The term “p a n o d CLR object” (POCO) s used to refer to such bus ness c asses that are des gned to be nsu ated from ­orb t ng p atform changes Investments n m dd e-t er deve opment are thus better protected, and bus ness og c code surv ves even as new techno og es rep ace o d ones n other ayers of the ­app cat on These concerns (among other factors) have dr ven the deve opment of Object Re at ona Mapp ng (ORM) techno og es, such as LINQ to SQL and Ent ty Framework, wh ch enab e you to bu d data access ayers much more a gned w th object-or ented pr nc p es than DataSets

Defining ORM Bus ness ent t es are represented as objects at the app cat on eve and as relational data n the database, and—one way or another—you need to map the two n your app cat on The key words ta c zed n that sentence define Object Re at ona Mapp ng, or ORM Take the s mp est cases where you wr te the code yourse f to shutt e va ues, one at a t me, between a ­SqlDataReader or a DataRow and the propert es of an object Such code s mapp ng re at ona data to objects, and so there’s your ORM Typ ca y though, the term ORM refers to robust frameworks des gned spec fica y to ­e m nate th s manua abor by automat ng the process n some way Less t me and effort e ­ xpended on the mundane ntr cac es of CRUD operat ons between SQL Server and your ­objects means more t me and resources ava ab e to focus on more mportant and creat ve th ngs, ke core bus ness og c, app cat on features, and user nterface des gn Both LINQ to SQL and Ent ty Framework (as we as other non-M crosoft ORM techno og es, such as the open-source NH bernate brary) comp ete y abstract the pers stence deta s between bus ness ent t es and the database away from you But ORM abstract on can be ra sed much h gher than th s One genera y assoc ates tab es, co umns, and re at onsh ps n the database w th c asses, propert es, and co ect ons n the app cat on, and th s s mp st c v ew m ght be adequate for some bas c app cat ons But n most rea -wor d scenar os, you encounter the so-ca ed ­impedance mismatch prob em, where there sn’t a ways a one-to-one correspondence b ­ etween re at ona tab es n the database and bus ness objects n the app cat on Beyond mere y ­hand ng pers stence deta s, more advanced ORMs (such as Ent ty Framework, but notab y not LINQ to SQL) prov de r ch abstract ons over the data mode tse f that can automat ca y reso ve many mpedance m smatch scenar os A common examp e s the many-to-many re at onsh p between two tab es, mp emented n the database w th a th rd junct on tab e The junct on tab e ho ds fore gn keys that re ate ent t es n the other two tab es to one another, support ng the many-to-many re at onsh p The samp e database does th s w th the CustomerEmployee junct on tab e between Customer and Employee That’s three tab es n the database phys ca y, but on y two objects conceptua y In the app cat on, you on y work w th Customer and Employee objects Each of those objects has a co ect on property that ho ds mu t p e object nstances of the other type, wh ch estab shes the many-to-many re at onsh p at the app cat on eve Ent ty Framework automat ca y

478  Part III  Applied SQL

manages the CustomerEmployee junct on tab e beh nd the scenes, so that your app cat on s concerned on y w th the two og ca ent t es Customer and Employee You’ see th s n ­act on a b t further on w th an examp e, and we w a so ta k about other mpedance m smatch ­scenar os that Ent ty Framework can hand e

Multiple ORM Offerings from Redmond Both LINQ to SQL and Ent ty Framework were des gned to g ve deve opers someth ng better than ­convent ona ADO NET, where you can bu d a data access ayer based on much more ord nary ­objects than a DataSet These ORM techno og es ra se the eve of abstract on above the raw ADO NET c asses even more than DataSets and DataAdapters As the examp es w show, you no onger nteract w th SqlConnection and SqlCommand objects at a Instead, you w use a spec a context object that acts as a contro er for query ng and updat ng data between SQL Server and objects (true ord nary objects; not “row” objects) n your app cat on Desp te these (and other) mportant d fferences w th convent ona ADO NET, the two ORM ­techno og es de ver a V sua Stud o deve oper exper ence that s str k ng y s m ar to that of typed ­DataSets LINQ to SQL and Ent ty Framework each sport a graph ca des gner that’s used as a ­front-end to author XML markup ( n d a ects other than XSD), wh ch n turn dr ves NET code ­generat on of strong y typed objects W th the know edge you a ready have of bu d ng typed ­DataSets n V sua Stud o, the LINQ to SQL and Ent ty Framework mode ng too s w seem very ­fam ar and ntu t ve That’s the good news The ava ab ty of two techno og es so over app ng n purpose s the bad news, and requ res some exp a n ng

LINQ to SQL: Then and Now LINQ to SQL was re eased n 2007 as part of NET 3 5 W th LINQ to SQL, you wr te LINQ quer es n your NET app cat on that get trans ated nto T-SQL quer es dynam ca y at runt me LINQ to SQL prov des a DataContext object that you use as the source of your LINQ quer es, prov d ng funct ona ty s m ar to the way a DataAdapter fi s a DataSet The DataContext a so tracks any changes made to a objects returned from the query resu ts You can then nstruct the DataContext to send the appropr ate nsert, update, and de ete commands back to SQL Server, much ke the way a DataAdapter updates a DataSet Here’s a s mp e examp e of a LINQ to SQL query using (var ctx = new SampleL2SDataContext()) { var q = from customer in ctx.Customers where !customer.LastName.StartsWith("A") orderby customer.LastName descending select customer; var list = q.ToList();



Chapter 10  The M crosoft Data Access Juggernaut   479

MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); }

At runt me, LINQ to SQL w parse th s query nto an express on tree and generate a T-SQL query for SQL Server to execute The LINQ to SQL runt me mp ements the funct ona ty of the NET StartsWith method n T-SQL by append ng the percent s gn (%) w dcard symbo to a parameter zed va ue for the LIKE operator exec sp_executesql N'SELECT [t0].[CustomerId], [t0].[FirstName], [t0].[LastName], [t0]. [Balance], [t0].[CreatedAt], [t0].[UpdatedAt] FROM [dbo].[Customer] AS [t0] WHERE NOT ([t0].[LastName] LIKE @p0) ORDER BY [t0].[LastName] DESC',N'@p0 varchar(8000)',@p0='A%'

LINQ to SQL evo ved from the or g na LINQ project, and was created by the anguage ­ eve opment team at M crosoft as an abstract on ayer that prov des a s mp e, one-to-one mapp ng d between objects n your app cat on and database tab es n SQL Server The Ent ty Framework, first re eased n 2008 as part of NET 3 5 SP1, was a project of the data programmab ty team not re ated to LINQ It s substant a y more advanced and prov des many more capab t es than LINQ to SQL The EF team focused on the Ent ty Data Mode (EDM) and created Ent ty SQL as the nat ve anguage for query ng the EDM (you’ soon work hands-on w th the EDM and Ent ty SQL) W th the advent of LINQ, however, t was natura (a most compu sory) for the team to a so prov de LINQ to Ent t es as a strong y typed way of query ng the EDM, so that earn ng Ent ty Framework doesn’t (necessar y) requ re earn ng Ent ty SQL L ke typed DataSets, both LINQ to SQL and EF de ver a des gn-t me exper ence n V sua Stud o for bu d ng an object mode around the database and auto-generat ng typed c asses based on the mode In code, EF a so prov des an ObjectContext object that works s m ar y to the LINQ to SQL’s DataContext And w th LINQ to Ent t es ava ab e as an a ternat ve to Ent ty SQL, both techno og es can use LINQ to query the database mapped to the object mode It was becom ng obv ous that M crosoft had two compet ng NET data access methodo og es that targeted the same scenar os, yet they were offer ng tt e or no gu dance over wh ch one to use Deve opers became more confused, and eventua y one of these techno og es had to emerge as the v ctor Not surpr s ng y, Ent ty Framework won and LINQ to SQL ost M crosoft has fina y made ts v­ s on c ear, and (as of the second vers on of EF re eased w th NET 4 n 2010) recommends the Ent ty ­Framework over LINQ to SQL as the data access so ut on for LINQ to re at ona scenar os

More Info  A post on the ADO.NET team blog explains Microsoft’s position on the future of Entity Framework and LINQ to SQL. You can read it at http://blogs.msdn.com/b/adonet/­ archive/2008/10/29/update-on-linq-to-sql-and-linq-to-entities-roadmap.aspx. So what does th s mean for LINQ to SQL? As far s th s book s concerned, t means that our LINQ to SQL coverage s m ted to the gu dance n th s sect on You shou d not cons der LINQ to SQL for any new deve opment, and t wasn’t around 480  Part III  Applied SQL

ong enough before be ng ec psed by EF for t to have ga ned very much of a strongho d (though t has defin te y won qu te a number of fans) For a ong t me st , you w encounter most y DataSet code and re at ve y tt e LINQ to SQL n ma nta n ng ex st ng product on app cat ons Our ear er ­coverage on convent ona ADO NET prepares you for that But because EF s recommended over LINQ to SQL for new app cat ons, we have made the dec s on not to d g nto LINQ to SQL and to focus nstead on Ent ty Framework deve opment For M crosoft, gett ng the r message out has not been easy It was a tough se to recommend the first vers on of Ent ty Framework (EF1) over LINQ to SQL, because the former acked many mportant capab t es offered by the atter Th s resu ted n an oppos t on group that banded together and posted a “vote of no confidence” n the new techno ogy, d sm ss ng t as ncomp ete and c t ng a tany of comp a nts aga nst t At the top of the st was the ack of support for azy oad ng (a so known as on-demand oad ng, where ch d ent t es are oaded automat ca y as needed) and POCOs (where t sn’t necessary for ent t es to nher t from, and thus be t ed to, any base c ass) EF1 d d not support azy oad ng, and a though t d d support nher tance, ent ty c asses were st forced to der ve from EntityObject LINQ to SQL supported azy oad ng and POCO support from the beg nn ng EF1 was a so sharp y cr t c zed for do ng a poor job of dynam ca y generat ng T-SQL quer es trans ated from your LINQ quer es at runt me It was pa nfu y obv ous that Ent ty Framework was re eased before t was ready, and that t defin te y needed to catch up w th LINQ to SQL In ts n t a form, there was no way for M crosoft to even hope for w despread adopt on of Ent ty Framework Fortunate y, the second vers on of the ADO NET Ent ty Framework (wh ch was re eased w th the NET Framework 4 0 n 2010, and s actua y referred to as EF4) represents a major mprovement over the first re ease EF4 supports azy oad ng, and there are now severa ways to use POCOs so that t s no onger necessary to nher t from EntityObject Improvements were made n dynam c T-SQL code generat on as we , and EF4 w usua y generate T-SQL that’s as good as ( f not better than) what you m ght code by hand Many other cr t c sms were addressed, and new features were added as we M crosoft rea y stened th s t me, and they deserve a ot of cred t for the work they d d w th ts vast y matured second re ease, so we w over ook the fact that, offic a y, there was no EF2 or EF3 LINQ to SQL s re at ve y far more mpa red when you compare t w th EF4 LINQ to SQL can on y be used w th SQL Server, but EF4 works w th any database p atform that has a compat b e ADO NET prov der LINQ s the on y way to query w th LINQ to SQL ( m t ng ts use to C# and VB NET), whereas wh e EF offers LINQ as a strong y typed a ternat ve to the more powerfu Ent ty SQL (wh ch can be used w th any NET anguage) LINQ to SQL supports on y a one-to-one correspondence between the re at ona tab e structure and your object mode , and thus w not reso ve many of the mpedance m smatch scenar os that EF can w th ts EDM You can a so detach and attach ent t es to the ObjectContext n EF4, enab ng pract ca n-t er deve opment that was mposs b e w th EF1 (as we as LINQ to SQL) A though t’s been a rough road (and there s st more room for mprovement), the turn ng po nt has defin te y been reached where you can accept Ent ty Framework ser ous y as M crosoft’s ­ ccess strategy of cho ce A though LINQ to SQL w not evo ve, t s not deprecated e ther data a LINQ to SQL rema ns part of the NET Framework It s re at ve y stra ghtforward to m grate from LINQ to SQL to Ent ty Framework f you have good reasons to do so, but t sn’t necessary ­otherw se ­M grat on makes more sense f the app cat on s be ng extended w th funct ona ty that cou d benefit

Chapter 10  The M crosoft Data Access Juggernaut   481

from ­features ava ab e on y n EF Fortunate y, the m grat on path s fa r y stra ghtforward, as you are ­mov ng toward a w der feature set where pretty much every LINQ to SQL concept has an EF ­equ va ent

Note  In Chapter 13, you build a Windows Phone 7 application that accesses local d ­ evice storage in SQL Server Compact edition (CE) using LINQ to SQL. This represents an ­exception to the general rule of avoidance that we just advocated, and is the one scenario in which LINQ to SQL is not only appropriate but preferred. Specifically, LINQ to SQL can and should be used client-side to access against the (single-user) local SQL Server CE d ­ atabase running on a Windows Phone 7 device—which is very different than accessing a full SQL Server 2012 or SQL Azure instance running in a data center. As you will see in Chapter 13, the phone application also includes a service component that synchronizes local data on the phone with a SQL Azure database running in the cloud, and this service component uses WCF Data Services with the Entity Framework—not LINQ to SQL—to access the SQL Azure database.

Entity Framework: Now and in the Future W th a of the ear er techno og es and gu dance d scussed to th s po nt, you are now pr med and ready to d ve nto Ent ty Framework, M crosoft’s recommended data access so ut on now and for the future The very first th ng you need to earn about s the Ent ty Data Mode (EDM), wh ch es at the heart of Ent ty Framework

Building an Entity Data Model (EDM) There are three parts to the EDM F rst there s the storage schema, wh ch s a thorough d ­ escr pt on of how tab es, co umns, and re at onsh ps are phys ca y arranged n the database Then there s the ­conceptual schema, wh ch descr bes the c asses, propert es, and nav gat on paths of bus ness ­ent t es n the app cat on F na y, you have the mapping schema, wh ch defines how the storage and ­conceptua schemas re ate to one another These three p eces (co ect ve y ca ed the mode ’s ­metadata) are se f-conta ned n a s ng e .edmx fi e ns de your project You author the .edmx fi e n V sua Stud o us ng a graph ca des gner s m ar to the one used to bu d an XSD-based typed ­DataSet ear er n the chapter—w th one cr t ca d fference the EDM design surface only displays the ­conceptual schema And there n s the crux of programm ng aga nst the Ent ty Framework Your app cat on works on y w th objects defined n the conceptua schema as t appears n the des gner, and has no ­awareness of the under y ng database structure defined n the storage schema You’ soon use the Mode Browser and var ous other pane s n V sua Stud o to v ew and configure the storage schema and mapp ngs at des gn t me Then at runt me, t s the Ent ty Framework’s job to retr eve and pers st data between your app cat on and SQL Server, and to reso ve the d fferences between storage and conceptua mode s automat ca y and dynam ca y us ng the mapp ng schema Thus, a three schemas need to be present and ava ab e to the app cat on wh e t’s runn ng 482  Part III  Applied SQL

By defau t, V sua Stud o embeds the three schemas as resources w th n the app cat on to ­ uarantee that they are a ways ava ab e You can a so keep the schema defin t ons outside your g ­app cat on’s assemb y, so they can be ed ted ndependent y w thout requ r ng you to recomp e and redep oy Because the mapp ng ayer effect ve y serves as a buffer between SQL Server and your app cat on, t serves as nsu at on from database changes that are made n the future For examp e, f a co umn name s changed, you can keep the o d name n the conceptua schema by mak ng the change on y n the storage schema of the EDM, and then adjust the mapp ng schema accord ng y Your app cat on can cont nue runn ng unaffected w th the una tered conceptua schema If you have a one-to-one correspondence between every tab e n the database and every c ass n the bus ness object mode , then both the storage and conceptua schemas w match exact y, and the mapp ng schema s mp y connects them up one-to-one That s noth ng d fferent than what you get w th LINQ to SQL or typed DataSets, and the true power of the EDM sn’t rea y be ng tapped But rea -wor d app cat ons rare y have storage and conceptua schemas that match exact y There s usua y some form of impedance mismatch that needs to be reso ved (see the s debar “Defin ng ORM” ear er n the chapter), and the EDM s a powerfu too that can he p to reso ve t Many-to-many re at onsh ps are a good examp e In the samp e database, the ­CustomerEmployee tab e serves as a junct on tab e for the many-to-many re at onsh p between the Customer and ­ hys ca ­Employee tab es But th s jo n tab e s rea y on y an mp ementat on deta of the under y ng p database structure SQL Server does not nherent y support many-to-many re at onsh ps, and so you requ re CustomerEmployee to serve as an ntermed ary tab e between the Customer and Employee tab es, both of wh ch t has a one-to-many re at onsh p w th In the wor d of objects, you care on y about Customer and Employee, whose many-to-many re at onsh ps are mp emented w th ch d ­co ect ons on each s de of each ent ty nstance Through the EDM, the Ent ty Framework ­understands the phys ca tab e ayout n the storage schema and sh e ds you from t Th s makes t eas er to code your app cat on, because you stay focused on og ca ent t es and don’t get entang ed w th the phys ca database deta s As you’re about to see, EF w automat ca y jo n on the junct on tab e for quer es, and update the junct on tab e for re at onsh p changes w thout a CustomerEmployee ent ty n the conceptua schema Now that you understand the EDM, you’re ready to create one Launch V sua Stud o, create a new V sua C# W ndows Forms app cat on, and name t DemoEntityFramework In So ut on Exp orer, r ght-c ck the project and choose Add New Item In the Add New Item d a og, scro down to find and c ck on ADO.NET Entity Data Model Name the new mode SampleEF.edmx and c ck Add V sua Stud o aunches the Ent ty Data Mode W zard so that you can beg n bu d ng the mode Th s w zard s your pr mary too for creat ng or updat ng mode s from an ex st ng database, wh ch s notab y ­d fferent than the Server Exp orer drag-and-drop approach taken by the typed DataSet and LINQ to SQL des gners The database s a ready n p ace, so choose Generate From Database on the first page of the w zard and c ck Next



Chapter 10  The M crosoft Data Access Juggernaut   483

More Info  There will often be an existing database that you can base a new model on ­using the wizard, as in this scenario. But other design strategies are possible as well. EF also supports model-first development, where you start by first building the conceptual model in the Visual Studio designer. This generates Data Definition Language (DDL) statements in a T-SQL script that will create the database from the model. Code-first design is another possible strategy introduced with Entity Framework 4.2, where you start out with neither a database nor a model, and just write your entities in plain code. Both the model and ­database are then derived and generated from your code. F rst you set up the connect on If you wa ked through the ear er typed DataSet examp es, then the ­database connect on SampleDb w st be ava ab e n the Server Exp orer and the EDM W zard w se ect t by defau t Other prev ous y used connect ons w a so be ava ab e n the drop-down st for you to reca eas y, or you can c ck New Connect on to create a new one on the fly from w th n the w zard

Tip  If you didn’t follow along with the typed DataSet examples earlier, you’ll need to ­create the connection now. Click New Connection, select Microsoft SQL Server as the data source, and click Continue. In the Add Connection dialog, type localhost for the Server name, ­select SampleDB from the database drop-down list, and click OK (you must choose ­different settings if you have created the database on another server or instance). The w zard d sp ays the ent ty connect on str ng, wh ch t bases on the SampleDb connect on, as shown n F gure 10-12

Figure 10-12  Sett ng the connect on for an EDM.

484  Part III  Applied SQL

Let’s exp a n th s connect on str ng metadata=res://*/SampleEF.csdl|res://*/SampleEF.ssdl|res://*/SampleEF.msl; provider=System.Data.SqlClient; provider connection string="Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=True;"

Th s m ght ook scary at first, but you can eas y dec pher what t a means from what you’ve a ready earned of the Ent ty Framework Remember that w th the EDM, you code on y aga nst the conceptua schema, wh e Ent ty Framework figures out how to do a the database work at runt me us ng the nformat on n the storage and mapp ng schemas Therefore, a three d st nct schemas must be ava ab e at runt me, and so t makes sense that a connect on str ng n EF must po nt to where they can be found The metadata keyword n the connect on str ng spec fies a p pe-de m ted st of the three schema fi es SampleEF.csdl (conceptua schema defin t on anguage), SampleEF.ssdl (storage schema defin t on anguage), and SampleEF.msl (mapp ng schema anguage) Reca from our ear er d scuss on that these three fi es are housed n the s ng e .edmx fi e ns de your project at des gn t me, and that they (by defau t) get embedded as resources n the assemb y when you bu d the ­so ut on The res://*/ des gnat on te s the EF runt me that the fi es can be ocated n the resources of the ­runn ng assemb y (or any referenced assemb es) The rest of the connect on str ng spec fies wh ch part cu ar ADO NET prov der s be ng used The provider keyword spec fies Sq C ent, wh ch nd cates SQL Server Remember that EF can be made to work w th any database p atform that has a compat b e ADO NET Ent ty Framework prov der The ­provider connection string keyword po nts to the prov der-spec fic connect on str ng, wh ch for ­Sq C ent s the fam ar SampleDb database connect on str ng Thus, an ent ty connect on str ng a ways conta ns references to the three schemas that define the mode , p us a “nested” connect on str ng n the syntax of the part cu ar back-end prov der be ng used w th the Ent ty Framework Beneath the connect on str ng, the w zard d sp ays an a ready checked checkbox te ng you that the connect on str ng w be saved n the app cat on’s configurat on fi e as SampleDbEntities Th s s just what you want, so c ck Next The w zard now scours the database to d scover a the tab es, v ews, and stored procedures that t can pu nto the mode There are four tab es and a handfu more stored procedures, and you w mport them a C ck the checkbox next to Tab es and the one next to Stored Procedures, as shown n F gure 10-13 on the next page, and then c ck F n sh The w zard bu ds the mode , and automat ca y adds the requ red reference to System.Data.Entity (the Ent ty Framework assemb y) to your project The mode then opens up n the des gner, as shown n F gure 10-14 Are you wonder ng why you see on y three ent t es when you se ected four tab es? That’s ­a ready been answered The des gner shows on y the conceptua ent t es, and the CustomerEmployee ­junct on tab e s not a part of the conceptua mode The w zard detected the ro e of th s tab e as the phys ca connect on between two og ca ent t es and exc uded t from the conceptua schema ­CustomerEmployee has a presence on y n the EDM’s mapp ng and storage schemas



Chapter 10  The M crosoft Data Access Juggernaut   485

Figure 10-13  Se ect ng database objects to be mported nto an EDM.

The Mode Browser s one of severa w ndows and panes n V sua Stud o that you use to v ew and configure the storage and mapp ng schemas To open t, r ght-c ck on an empty area of the des gn surface and choose Mode Browser In the Mode Browser, find and expand the Entity Types and Tables / Views nodes F gure 10-14 shows the Mode Browser on the r ght

Figure 10-14  The ent ty data mode des gn surface n V sua Stud o.

486  Part III  Applied SQL

The Mode Browser sts CustomerEmployee among the four tab es n the storage schema, wh ch s denoted by the suffix n SampleEFModel.Store You are go ng to become fast fr ends w th the Mode Browser as you use t to d g nto the EDM—much deeper than we w be demonstrat ng here Take the t me to browse around and get comfortab e w th a of the e ements of the mode structure

More Info  Our coverage of Entity Framework in this chapter merely scratches the surface of this powerful API. If you really want to become an EF expert, we recommend reading Julia Lerman’s Programming Entity Framework, Second Edition (O’Reilly), for an excellent and thorough treatment of the technology. Not ce how the w zard created associations between the ent t es, wh ch t based on tab e r­ e at onsh ps detected n the storage schema The many-to-many assoc at on between Customer and Employee has an aster sk (*) symbo ( nd cat ng mu t p c ty) on both ends, whereas the ­assoc at on ­between Customer and OrderHeader nd cates the one-to-many re at onsh p between those two tab es In add t on to sca ar propert es, each ent ty a so has one or more navigation properties that are based on ts assoc at on(s) w th other ent t es These nav gat on propert es surface as ch d co ect on and parent reference propert es that your app cat on can use to access re ated ent t es n code The w zard a so s ngu ar zed or p ura zed each nav gat on property appropr ate y For examp e, ook at the one-to-many re at onsh p between Customer and OrderHeader Each Customer ent ty has an OrderHeaders property (p ura ) that ho ds a co ect on of OrderHeader (s ngu ar) ent t es ­Converse y, each OrderHeader ent ty has a Customer property (s ngu ar) that po nts back to ts parent ent ty S m ar y, Customer and Employee each have p ura zed nav gat on co ect on propert es of each other (Customer has Employees, and Employee has Customers), because those ent t es are jo ned n a many-to-many re at onsh p Th s s just the way an object-or ented programmer expects the wor d to be, and s a great dea eas er to work w th n code than the DataSet when nav gat ng between re ated ent t es As w th typed DataSets, the EDM des gner acts as a front end that you use to author an XML document transparent y n the background But nstead of us ng the XSD format, th s d a ect of XML represents the metadata for the three schemas of the EDM And rather than tr gger ng the generat on of typed DataSet and TableAdapter c asses from the XML, typed ent ty and ObjectContext c asses are generated nstead Most of the t me, the des gner w sh e d you from the deta s of the under y ng XML and the code that t generates, but t defin te y pays to become fam ar w th them f you rea y want to ga n a so d understand of EF So before wr t ng the code for your first Ent ty Framework ­examp e, et’s qu ck y ook over the two fi es R ght-c ck SampleEF.edmx n the So ut on Exp orer and choose Open W th Se ect XML (Text) E ­ d tor and c ck OK to v ew the fi e as raw XML V sua Stud o w prompt you first to c ose the mode open n the des gner before open ng the raw XML (wh ch is the very XML that you are ­compos ng as you bu d your ent ty mode n the des gner) Scro through the fi e, and you w find an XML representat on of everyth ng you’ve done us ng the des gner If you co apse the nodes beneath , you w see the h gh- eve v ew of the three sect ons for the conceptua , ­storage, and mapp ng schemas As exp a ned, th s s the metadata that s requ red at runt me and gets

Chapter 10  The M crosoft Data Access Juggernaut   487

­ mbedded as a resource n the project’s assemb y when you bu d the so ut on After the metadata, e you w see the ­sect on, wh ch s used on y by V sua Stud o at des gn t me Th s s where the des gner saves ­nformat on about the ayout of shapes (s ze, pos t on, and so on) n the mode d agram Layout nformat on s gnored by the runt me, and you can gnore t now as we When you bu d your app cat on, the sect on s not nc uded w th the metadata that gets embedded as an assemb y resource Next, expand the SampleEF.edmx fi e n the So ut on Exp orer to revea the SampleEF.Designer. cs fi e nested beneath t Doub e-c ck t to open the C# code, wh ch s generated automat ca y ­whenever the .edmx fi e s updated The code s organ zed n reg ons, so start by expand ng the ­Ent t es reg on You w find three c asses for the three ent t es Customer, Employee, and OrderHeader; and you’ see that they a nher t from EntityObject Th s s based on the defau t code generat on strategy wh ch was the on y cho ce ava ab e w th EF1 W th EF4, you can use an a ternat ve code ­generator based on T4 temp ate techno ogy to produce POCO ent t es—c asses that don’t nher t from anyth ng (bes des perhaps other c asses n the mode , f your mode defines nher tance) Expand the nested reg ons further, and you’ find the defin t ons for each property w th change not ficat on og c bu t nto the property setters In add t on to the ent t es themse ves, the generated code nc udes a context c ass for ­programm ng quer es and updates aga nst the mode Expand the Contexts reg on (at the top of the source fi e, just ­beneath the namespace dec arat on) and you’ see the SampleDbEntities c ass that nher ts from ­ObjectContext and funct ons as the pr mary access po nt to your mode at runt me Th s becomes c earer when you expand the ObjectSet Propert es reg on ns de the context c ass There you w find a of the entity sets exposed by the context (one set for each ent ty), named n the p ura (Customers, Employees, and OrderHeaders) You w conduct LINQ quer es aga nst the ent ty sets exposed by the SampleDbEntities context and mater a ze the resu ts from the database nto ve object nstances The context w a so track any changes you make to the ve objects so that the changes can be saved back to the database ater That’s enough exp or ng beh nd the scenes C ose the XML document and et’s move on to the first EF examp e us ng LINQ to Ent t es

Using LINQ to Entities Ent ty SQL s the nat ve anguage n Ent ty Framework for query ng the EDM, and ater on, we w d scuss when or why you’d choose to use t But for now, start w th the eas est way to query ent t es, wh ch s to use LINQ to Ent t es Drag a button contro from the Too box and drop t onto the form Form1 created automat ca y by V sua Stud o In the Propert es w ndow, name the button btnLinqWithDirectSql, and set the Text property to LINQ to Entities Query (Direct SQL) Then doub e-c ck the button and add the code n L st ng 10-14

488  Part III  Applied SQL

Listing 10-14  Query ng an EDM w th L NQ to Ent t es us ng d rect SQL.

private void btnLinqWithDirectSql_Click(object sender, EventArgs e) { // Retrieve via direct SQL: using (var ctx = new SampleDbEntities()) { // Define, but do not execute, the query (deferred execution) // 'q' is of type System.Data.Objects.ObjectQuery var q = from customer in ctx.Customers where !customer.LastName.StartsWith("A") orderby customer.LastName descending select customer; // Implicitly execute query on the server, returning List var list = q.ToList(); MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); } }

Before runn ng the app cat on, set a breakpo nt on the ne that dec ares the q var ab e and ­ass gns t to the LINQ query A so start a new SQL Profi er trace, f one sn’t st runn ng from the ear er examp es Now start the app cat on and c ck the button You w h t the breakpo nt r ght before the LINQ query s ass gned to q S ng e-step over the ne and sw tch mmed ate y over to the SQL Profi er trace You may be surpr sed to see that no query was sent to SQL Server Th s h gh ghts a key pr nc p e of LINQ known as deferred execution A you’ve done s define your query and ass gn t to q There s no mag ca method that you ca to actua y execute the query; execut on s mp ed, and w automat ca y k ck off anyt me you try to access a resu t from the query Th s cou d happen s mp y by try ng to foreach your way through q, by b nd ng t to a contro n the user nterface, or by ca ng a method ke ToList on t as you’re do ng here Go back to V sua Stud o, s ng e-step over the next ne, and then sw tch r ght back to the trace Sure enough, you can see the SELECT statement that was just sent to SQL Server, transformed from a LINQ to Ent t es query nto nat ve T-SQL SELECT [Extent1].[CustomerId] AS [CustomerId], [Extent1].[FirstName] AS [FirstName], [Extent1].[LastName] AS [LastName], [Extent1].[Balance] AS [Balance], [Extent1].[CreatedAt] AS [CreatedAt], [Extent1].[UpdatedAt] AS [UpdatedAt] FROM [dbo].[Customer] AS [Extent1] WHERE NOT ([Extent1].[LastName] LIKE 'A%') ORDER BY [Extent1].[LastName] DESC

It’s cr t ca to rea ze that execut ng another q.ToList, or any other attempt to “tap” at q—for ­examp e, even just gett ng a count w th q.Count—fires another query execut on n SQL Server ­Need ess round-­ tr pp ng s obv ous y someth ng to avo d, so t s mperat ve for you to rema n ­cogn zant of deferred ­execut on n your code You’ typ ca y want to query the database just once, and then ho d onto the

Chapter 10  The M crosoft Data Access Juggernaut   489

resu ts n some cached form such as an array, co ect on, or d ct onary The ­IEnumerable extens on methods ToArray, ToList, and ToDictionary are prov ded so you can do just that The generated SQL may not be pretty, but you can see that the NET StartsWith method n the LINQ query’s where c ause was correct y expressed us ng the LIKE operator n the T-SQL query’s WHERE c ause Thus, the fi ter ng and sort ng occurs n SQL Server, and on y the resu ts of nterest get returned to your app cat on EF s qu te capab e of produc ng quer es that perform at east as we as what you’d code by hand—most of the t me You’d never a as the Customer tab e as Extent1 because Extent1 s a mean ng ess word to you, but the a as n no way negat ve y mpacts the query ­performance The runt me must be flex b e enough to compose quer es far more comp ex than th s s mp e examp e, and so t w somet mes generate strange syntax that doesn’t resemb e anyth ng you’d wr te by hand, but st doesn’t necessarily represent a performance h t Of course there are except ons, so you defin te y need to keep an eye on SQL Profi er to make sure that the EF’s dynam ca y generated quer es are reasonab e Enjoy ng the conven ence offered by an abstract on ayer does not mean you can be tota y gnorant of what’s go ng on beneath the surface Th s s true even more as you ra se the eve of abstract on W th the great power that EF hands you comes the great respons b ty of understand ng how t operates and be ng aware of the ­consequences of your act ons, such as how you use deferred execut on or azy oad ng In some cases, you m ght need to refactor a LINQ query to encourage the rut me to generate a better T-SQL query In other cases, you may determ ne that t’s better to wr te the T-SQL query yourse f ns de a ­parameter zed stored procedure, and then use a s mp e LINQ query that mere y ca s the stored ­procedure That’s perfect y fine, because EF doesn’t mpose an a -or-noth ng cho ce You can a ow EF to generate d rect T-SQL from fu y expressed LINQ quer es, and a so ma nta n tota contro over T-SQL query syntax when necessary by us ng stored procedures

Note  If you have strong feelings on the subject of direct SQL and stored procedures, you are not alone. Read the sidebar “Rethinking the Great Direct SQL vs. Stored Procedure Debate” a bit further on in the chapter for a brief discussion on major factors to consider.

Mapping Stored Procedures to the EDM Even though you mported the set of CRUD stored procedures for the Customer tab e nto the mode , EF st generated a d rect T-SQL dynam ca y Import ng stored procedures mere y defines them n the storage schema, but they aren’t actua y mapped to anyth ng yet Ent ty Framework won’t know to use mported stored procedures un ess you map them to the conceptua schema us ng the des gner, wh ch you’ do next Stored procedures that retr eve data are mapped as function imports us ng the Mode Browser You w now mport a funct on to map the SelectCustomers stored procedure Doub e-c ck SampleEF.edmx n the So ut on Exp orer to open the mode n the des gner If the Mode Browser s not a ready v s b e, r ght-c ck on an empty area of the des gn surface and choose Mode Browser Beneath SampleDbModel n the Mode Browser, expand the EntityContainer: ­SampleDbEntities node R ght-c ck the Function Imports node and se ect Add Funct on Import, as shown n F gure 10-15 490  Part III  Applied SQL

Figure 10-15  mport ng a funct on nto the conceptua mode .

Name the funct on SelectCustomers, after the stored procedure you’re mapp ng the funct on to (though t certa n y doesn’t need to match) Th s s the name that you’ refer to n your next LINQ query so that the stored procedure gets ca ed nstead of d rect SQL Next, choose SelectCustomers from the drop-down st of ava ab e stored procedures n the ­storage schema F na y, because you know that th s stored procedure returns the co umns for a ­Customer ent ty, c ck the Ent t es rad o button beneath Returns A Co ect on Of, and then se ect Customer from the drop-down st Th s te s EF that t can and shou d return a co ect on of Customer ent t es when you use the SelectCustomers funct on mport n a LINQ query F gure 10-16 shows the Add Funct on Import d a og box configured to mport SelectCustomers

Figure 10-16  Mapp ng a stored procedure that returns a resu t set to a funct on that returns an ent ty co ect on.



Chapter 10  The M crosoft Data Access Juggernaut   491

Now wr te a mod fied vers on of the prev ous LINQ query that uses the SelectCustomers funct on you just mapped Drag another button contro from the Too box and drop t onto the form, name t btnLinqWithStoredProc, and set the Text property to LINQ to Entities Query (Stored Procedure) Then doub e-c ck the button and add the code n L st ng 10-15 Listing 10-15  Query ng an EDM w th L NQ to Ent t es us ng a stored procedure.

private void btnLinqWithStoredProc_Click(object sender, EventArgs e) { // Retrieve via a function import mapped to a stored procedure: using (var ctx = new SampleDbEntities()) { // Execute the stored procedure on the server immediately // 'q' is of type System.Linq.OrderedEnumerable var q = from customer in ctx.SelectCustomers() where !customer.LastName.StartsWith("A") orderby customer.LastName descending select customer; // Filter & sort stored proc results on the client, returning List var list = q.ToList(); MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); } }

Th s LINQ query s v rtua y dent ca to the prev ous one (see L st ng 10-14), except that the S­ electCustomers funct on mport, rather than the Customers ent ty set, s spec fied as the source of the query after the in keyword Everyth ng e se s exact y the same As you’re about to see, however, th s sma change makes an enormous d fference n the way the query executes Once aga n, set a ­breakpo nt on the ne dec ar ng q and make sure you’ve got a current SQL Profi er trace runn ng Start the app cat on and c ck the button When you h t the breakpo nt, s ng e-step over the LINQ query and sw tch back to the SQL Profi er trace You m ght be surpr sed aga n to find that, th s t me, EF ca s out to SQL Server mmed ate y on the ne of code that defines the LINQ query The trace shows the stored procedure ca be ng made when you s ng e-step over that ne exec [dbo].[SelectCustomers]

But wa t The SelectCustomers stored procedure returns every customer n the database You know th s—you wrote the stored procedure! You know that t’s return ng everyth ng, so that means that—un ke the prev ous vers on—the fi ter ng and sort ng operat ons spec fied n the LINQ query are be ng executed on the client Th s s occurr ng after send ng a the Customer rows across the ­network from SQL Server to your app cat on—even rows you don’t care about, even rows that w get thrown away because they don’t pass the where cr ter a There s st deferred execut on at p ay, but t’s ­mere y the n-memory fi ter ng and sort ng of the comp ete resu t set (a ready returned by SQL Server) that gets deferred unt you ca ToList on q n the next ne of code 492  Part III  Applied SQL

So t comes down to th s If you LINQ aga nst any of the ent ty sets n the mode , then t w a ways be deferred execut on of d rect SQL aga nst the tab e(s)/v ew(s) mapped to the ent ty, and resu ts w be fi tered on the server by a dynam ca y generated WHERE c ause that’s based on the LINQ query’s where c ause Th s s a true LINQ to Ent t es query But f you LINQ aga nst a funct on mport mapped to a stored procedure nstead, then the stored procedure w execute mmed ate y, a of ts resu ts w get sent back to the c ent, where the resu ts w subsequent y get fi tered by the LINQ query’s where c ause just ke an ord nary LINQ to Objects query

More Info  This behavior stems from the fact that tables and views are composable (they can be combined with a parameterized WHERE clause in T-SQL) while stored procedures are not. Table-valued functions (TVFs) offer the best of both worlds; they provide the ­encapsulation benefits of stored procedures yet they are composable like tables and views. Unfortunately, TVFs are not currently supported in EF, though Microsoft has confirmed TVF support for the next major release of the Entity Framework. Th s h gh ghts the mportance of understand ng what goes on under the covers w th EF Once you know what’s rea y happen ng, you earn how to use (and not m suse) the techno ogy Thus, f you’re pu ng n a moderate y s zed resu t set from the database w th your stored procedure, and then extract ng a reasonab e subset of that data on the c ent n the where c ause, then that’s a perfect y acceptab e use case On the other hand, f the stored procedure returns a mass ve amount of data and the where c ause n your LINQ query fi ters the resu t set on the c ent down to just a handfu of rows, then you know you are do ng someth ng wrong The SelectCustomers stored procedure n th s examp e has no parameters, but you can certa n y map parameter zed stored procedures to funct on mports as we In that case, a WHERE c ause n the stored procedure can first fi ter the resu t set on the server (us ng parameters passed to t through the funct on mport), and then the where c ause n the LINQ query can fi ter the resu ts further on the c ent Th s s qu te the same th ng as fi ng a DataSet from the database v a a parameter zed stored procedure and then further fi ter ng on that resu t set n memory us ng LINQ to DataSet U t mate y, th s begs the quest on “where’s the where?,” and the answer s that t’s rea y up to your own judgement of how to best d str bute fi ter ng og c between c ent and server n a way that makes the most sense for your app cat on

Rethinking the Great Direct SQL vs. Stored Procedure Debate Ent ty Framework (and LINQ to SQL) has good stored procedure support, but s rea y des gned to query the database w th d rect SQL Yet many database profess ona s reject and forb d the use d rect SQL, ns st ng exc us ve y on the use of stored procedures So now s a good t me to re-v s t th s heated debate, n wh ch secur ty, performance, and ma nta nab ty are the most often c ted po nts of argument



Chapter 10  The M crosoft Data Access Juggernaut   493

Security—D rect SQL s vu nerab e to SQL nject on attacks But th s s true on y f you are bu d ng T-SQL statements dynam ca y us ng str ng concatenat on Don’t forget that even stored procedures are vu nerab e n th s respect, f they generate d rect SQL by concatenat ng str ngs and embed content of an unknown and unparsed or g n The pr mary ne of defense aga nst SQL nject on attacks t to parameter ze the query, wh ch s eas y done w th d rect SQL As ev denced by observ ng a SQL Profi er trace, EF generated a parameter zed database ­command that protects aga nst SQL nject on Performance—Th s s actua y an outdated concern, based on the not on that d rect SQL doesn’t get comp ed ke stored procedures do, and so stored procedures therefore run faster It’s true that stored procedures wou d get part a y comp ed to speed mu t p e execut ons n SQL Server vers ons 6 5 (re eased n 1996) and ear er But as of SQL Server 7 0 (re eased n 1999), that s no onger the case Instead, SQL Server 7 0 (and ater) comp es and caches the query execut on p an to speed mu t p e execut ons of the same query (where on y parameter va ues vary), and that’s true whether execut ng a stored procedure or a d rect T-SQL statement Maintainability—In product on app cat ons, t s very poor pract ce to embed T-SQL ­ rect y n NET code Do ng so t ght y coup es c ent and server og c n a way that qu ck y d snowba s nto a ma ntenance n ghtmare But th s s not the case w th LINQ to Ent t es (or w th LINQ to SQL) You are on y compos ng and ma nta n ng the LINQ query n NET code; the ­trans at on to d rect T-SQL occurs on the fly at runt me when you execute your app cat on, so th s concern s mp y doesn’t app y These cons derat ons shou d change your perspect ve somewhat If you’re fortunate enough to be n tota contro over your app cat on’s arch tecture, you can enjoy a good comprom se w th a hybr d approach You can a ow d rect SQL aga nst tab es and v ews for SELECT ­quer es on y, but cont nue us ng stored procedures for INSERT, UPDATE, and DELETE operat ons W th th s strategy, updates at the app cat on eve rest on top of a stored procedure ayer that can perform cr t ca va dat ons at the database eve You can a so cons der creat ng v ews to expose a m ted or a tered subset of tab e data, and then grant ng SELECT access on y to the v ews and not the tab es Then you can map the ent t es n your data mode to the v ews rather than tab es for query ng, wh e updates get pers sted back us ng stored procedures ­ ELETE So we recommend that you st ck to us ng stored procedures for INSERT, UPDATE, and D operat ons, wh e deny ng d rect access to the under y ng tab es (except for SELECT) Do ng so a ows you to perform add t ona va dat on that cannot be bypassed by c rcumvent ng the ­app cat on ayer and commun cat ng d rect y w th the database server At the same t me, your LINQ quer es w trans ate “natura y” to d rect SQL because SELECT perm ss ons are st granted aga nst the tab e (or v ew) be ng quer ed

If you’re st paused at the breakpo nt n the prev ous examp e, cont nue execut on now and c ose the app cat on It’s t me to start updat ng some data w th Ent ty Framework

494  Part III  Applied SQL

Saving Entity Changes When t comes to update operat ons, th ngs are much more stra ghtforward EF w e ther dynam ca y generate a d rect SQL statement (wh ch s ts defau t behav or) or ca a stored procedure ( f you’ve mapped one) for each changed row E ther way, ent t es get updated one at a t me Rev s t the mode and w re up the three stored procedures for sav ng Customer ent ty changes to the database (they were created at the beg nn ng of the chapter; refer to L st ng 10-1 f you need to rev ew them) You w nstruct EF not to generate d rect SQL statements for sav ng Customer ­ent t es n the database, but to ca these stored procedures nstead Doub e-c ck SampleEF.edmx n the ­So ut on Exp orer to open t n the des gner Then r ght-c ck anywhere on the Customer ent ty and choose Stored Procedure Mapp ng to d sp ay the Mapp ng Deta s pane (p n t down to keep t n v ew) In t a y, there are no mapp ngs defined, wh ch means that EF dynam ca y generates d rect SQL statements for INSERT, UPDATE, and DELETE You’re now go ng to change that behav or and map a three stored procedures a ong w th the r parameters and return va ues C ck and choose the InsertCustomer stored procedure from the ­ rop-down st You need to map each stored procedure parameter to the correspond ng ent ty d property, but the des gner he ps out where t can Because parameters and co umns are cons stent y named, a of the nput parameters requ red by the stored procedure were correct y mapped The on y add t ona th ng you need to do s map the three va ues returned by the stored procedure back nto the ent ty n the Resu t Co umn B nd ngs Reca (from L st ng 10-1) that the InsertCustomer stored procedure returns the CustomerId, ­CreatedAt, and UpdatedAt va ues—ass gned to the new customer row n the database—as ­sca ar v­ a ues n a ­s ng e-row resu t set W th the Resu t Co umn B nd ngs, you can shove those va ues r ght back nto the c ent nstance and refresh ts v ew of the new ent ty after the stored procedure ­executes Thus, the c ent doesn’t need to retr eve the ent re ent ty after add ng t on y to obta n these three va ues You just need to type the name of each co umn n the s ng e-row resu t set and map t to the appropr ate property C ck and type CustomerId Add the other two return va ues the same way, typ ng the r ­respect ve names CreatedAt and UpdatedAt Unfortunate y, the des gner offers no he p mapp ng the resu t b nd ngs, and a three are mapped to the CustomerId property by defau t Th s happens to be a ­correct assumpt on for the first b nd ng, but the other two b nd ngs need to be changed C ck the ­Property drop-down st for each of them and se ect C ­ reatedAt and ModifiedAt, respect ve y You’re done w th the Insert funct on and can move on to the Update funct on

Important  The ADO.NET Entity Framework 4 supports stored procedures that return scalar values either as output parameters or as a single-row result set, with one glaring ­exception. The new primary key value assigned by SQL Server can only be sent back to a newly ­created entity’s key using a single-row result set; an output parameter cannot be used. Thus, if the InsertCustomer stored procedure was written to return its three server-assigned values (one of them being the newly assigned CustomerId primary key value) using output parameters rather than a single-row result set, you could not use Entity Framework to call InsertCustomer.



Chapter 10  The M crosoft Data Access Juggernaut   495

C ck and choose the UpdateCustomer stored procedure from the ­drop-down st Once aga n, a of the same-named parameters and propert es get mapped ­automat ca y by the des gner The on y parameter that doesn’t have a match ng property s @OriginalUpdatedAt C ck the Property drop-down st and se ect UpdatedAt to set the mapp ng, but also check the Use Or g na Va ue checkbox Th s s an mportant guarantee that the UpdatedAt va ue or g na y retr eved from the database s what gets sent back to UpdateCustomer n th s parameter Because the context preserves the or g na va ues of every property, the des gner a ows you to map them to any stored procedure parameters you des re The UpdateCustomer stored procedure ra ses a concurrency error f another user changes the UpdatedAt va ue between the t me that you or g na y retr eve the customer and the t me you ­attempt to save t back Pass ng n the or g na UpdatedAt va ue n the @OriginalUpdatedAt parameter you just mapped a ows the stored procedure to perform the check If the update succeeds norma y, ­UpdateCustomer returns the new UpdatedAt va ue that got ass gned to the changed customer row n the database as a sca ar va ue n a s ng e-row resu t set As w th the Insert funct on, you want to br ng the updated sca ar va ue for UpdatedAt back nto the c ent nstance to refresh ts v ew of the mod fied ent ty after the stored procedure executes C ck and type UpdatedAt Then c ck the Property drop-down st to the r ght and se ect UpdatedAt to map the return va ue back nto the ent ty That does t for the Update funct on; the Delete funct on s next and ast (and eas est) C ck and choose the DeleteCustomer stored procedure from the ­ rop-down st The one and on y @CustomerId parameter s automat ca y mapped to the ­CustomerId d property and you’re done! F gure 10-17 shows the Mapp ng Deta s pane w th a three funct ons mapped to stored procedures

Figure 10-17  Mapp ng three stored procedures for an ent ty s nsert, update, and de ete operat ons.

496  Part III  Applied SQL

W th a the CRUD stored procedures mapped, you’re ready to wr te some more code Drag ­another button contro from the Too box and drop t onto the form, name t btnUpdatingData, and set the Text property to Updating Data Then doub e-c ck the button and add the code n L st ng 10-16 Listing 10-16  Updat ng an EDM object graph.

private void btnUpdatingData_Click(object sender, EventArgs e) { using (var ctx = new SampleDbEntities()) { var customers = (from cust in ctx.Customers where !cust.LastName.StartsWith("A") orderby cust.LastName descending select cust).ToList(); if (customers.Count == 0) { MessageBox.Show("There are no customers"); return; } var firstCustomer = customers[0]; var orderCount = firstCustomer.OrderHeaders.Count; MessageBox.Show(string.Format( "Customer {0} has {1} order(s)", firstCustomer.CustomerId, orderCount)); // Change the customer's name firstCustomer.FirstName = "Keith"; firstCustomer.LastName = "Harris"; // Add a customer order firstCustomer.OrderHeaders.Add(new OrderHeader() { ShipVia = "Regular Mail", OrderStatus = "Open", }); ctx.SaveChanges(); } }

Set a breakpo nt at the top of the method (on the using statement that nstant ates the new S­ ampleDbEntities object) Make sure you’ve got a SQL Profi er trace runn ng, and then run the ­app cat on C ck the button and you’ h t the breakpo nt Step over the ne to create the context and advance to the next ne of code Th s t me, the LINQ to Ent t es query s wrapped ns de ­parentheses and ToList s ca ed on t a n one ne of code Th s effect ve y short-c rcu ts deferred execut on, and forces the query to run mmed ate y (and on y once) The ToList method mater a zes the query resu ts nto Customer objects, and returns a popu ated List co ect on n customers When you s ng e-step over the ne, the SQL Profi er trace shows that EF generates and executes the same SELECT statement you saw used w th L st ng 10-14 Aga n, d rect SQL s be ng used because you’re

Chapter 10  The M crosoft Data Access Juggernaut   497

query ng aga nst the Customers ent ty set, and not the SelectCustomers funct on mapped to the SelectCustomers stored procedure So th s s a rea LINQ to Ent t es query that executes n SQL Server and returns on y fi tered Customer rows to your app cat on Step over the test that checks for no customers You’ve a ready added severa customers n ear er examp es, so the code shou d jump r ght to the ne that gets the first customer from the st Step over that ne, and a so over the next ne that gets the customer’s order count A though you are ­perform ng s m ar operat ons as you d d w th DataSets ear er (see L st ng 10-11), th s code s s gn ficant y more flex b e and object-or ented than that examp e Let’s exp a n further to see how The Customer object n firstCustomer has an OrderHeaders co ect on wh ch, ke any ord nary co ect on, has a Count property that te s you how many orders the customer has Th s s much more ntu t ve than need ng to ca the GetOrderHeaderRows method to get an array of ch d rows so that you can get the order count w th the array’s Length property, as you d d w th the typed DataSet

Object Context Lazy Loading and Change Tracking But wa t—your LINQ to Ent t es query retr eves on y customers from the database, but none of the r orders The ear er DataSet examp e se ected both customers and orders, so GetOrderHeaderRows s mp y returned an array of n-memory order rows a ready retr eved from the database So how s EF ab e to get the count of orders that you haven’t yet retr eved? The answer s azy oad ng, wh ch s on by defau t If you s ng e-step some more and sw tch back to SQL Profi er trace, you w see that EF ssues a second T-SQL query to retr eve the orders be ong ng to just the first customer Th s was tr ggered by your request to obta n the count through the customer’s OrderHeader co ect on’s Count property Now the context s track ng a customers retr eved by the or g na query, p us the orders for th s one customer just retr eved by azy oad ng At th s t me, a of the objects be ng tracked by the context have an EntityState property whose va ue s set to EntityState.Unchanged Compare the flex b ty afforded by EF azy oad ng w th the typed DataSet approach taken n L st ng 10-11 In that examp e, you oaded every ch d ent ty, not know ng wh ch ones w rea y be needed and wh ch won’t W th EF, t’s much eas er to str ke the opt mum ba ance of what gets oaded and when, wh ch often var es n d fferent s tuat ons You can eager- oad some re ated ent t es up front (by us ng the Include method n your n t a LINQ query to generate the appropr ate T-SQL jo n), and a ow azy oad ng to automat ca y fetch other re ated ent t es when you reference them Or you can d sab e azy oad ng by sett ng the LazyLoadingEnabled property on the context object to false and contro everyth ng yourse f Then you cou d eager- oad some re ated ent t es up front, and defer the oad ng of other re ated ent t es unt you need them (by exp c t y nvok ng the Load method on a ch d co ect on or parent reference property) Step over the next ne and d sm ss the MessageBox that d sp ays the customer ID and order count Now the customer’s name s changed by sett ng ts FirstName and LastName propert es Th s n turn automat ca y changes the EntityState of the customer to EntityState.Modified A new order s then created for the customer Aga n the code s c eaner than L st ng 10-11 Rather than be ng forced to ca a spec a method ke AddOrderHeaderRow to add a new DataRow to a DataTable, you treat your ent t es as p a n objects and co ect ons The customer object has an OrderHeaders property, wh ch s an ord nary co ect on w th an ord nary Add method The new order s created n ne d rect y ns de 498  Part III  Applied SQL

the Add method, us ng object initializers to set va ues for a ts propert es Now the context starts track ng the new order, and EF sets ts EntityState property to EntityState.Added It’s now t me to push a your data changes back to SQL Server Pers stence s s mp e; just nvoke the s ng e SaveChanges method on the context EF does a the work of ssu ng transact ona zed, proper y-ordered updates to the database for a the objects be ng tracked for changes by the context Step over the SaveChanges method ca and then sw tch over to SQL Profi er to exam ne the trace You w see that EF sent two commands to SQL Server F rst t generated and ssued a d rect INSERT statement to add the new order to the OrderHeader tab e Then, because of the stored ­procedure mapp ngs you estab shed for the Customer ent ty, EF ca ed the UpdateCustomer stored procedure to save the changes made to the customer name You can a so see that EF wrapped those two operat ons up ns de BEGIN TRANSACTION and COMMIT TRANSACTION statements ­automat ca y

Note  If you did not enable transactional events for the SQL Profiler trace as we explained earlier, you will not see the BEGIN TRANSACTION and COMMIT TRANSACTION statements in the SQL Profiler trace. Stop the trace, and start a new one with transactional events ­enabled, as shown in Figure 10-4. When you rerun the code, you will see these statements appear in the trace surrounding the insert and update operations.

The EF n-Tier Challenge We’ve cr t qued the DataSet qu te sharp y as we compared t w th EF But DataSets are st eas er than EF n n-t er scenar os Reca how the ear er DataSet examp e (back n L st ng 10-10) used two adapters (and thus, two separate connect ons)—one w th a command to do the fi and another w th three commands to do the update You d dn’t need to use two adapters; you cou d just have eas y used one adapter configured w th a four commands We d rected you to do so on y to s mu ate the d sconnected state that the DataSet exper ences n an n-t er arch tecture, n wh ch t traverses the t ers between the fi and the update Wh e d sconnected, the DataSet tracked a changes n each row’s RowState property so they cou d be saved back to the database It s ent re y se f-re ant for ­track ng ts own changes, and you aren’t requ red to ho d on to the same adapter that fi ed t n order to update t after the round-tr p The Ent ty Framework s very d fferent You must perform the fi and update us ng one and the same context, or you’ be faced w th ser ous prob ems A though each ent ty does ndeed have an ­EntityState property that funct ons ke the RowState property does w th DataSets, that property va ue s mere y a reflect on of the ent ty state stored nterna y n the context object that’s actua y ­track ng the ent ty Ent ty nstances are not se f-track ng as DataSets are; they depend on the context for change track ng Yet the context s not ser a zab e, and does not trave a ong w th the ent ty nstances t s track ng as they traverse the t ers The context therefore does not surv ve the round tr p When you retr eve objects from the database and then destroy the context, those objects are no onger ab e to track changes on the r own as you pass them through the t ers W thout the context, the EntityState property s a ways set to EntityState.Detached, and w no onger update a ­ utomat ca y

Chapter 10  The M crosoft Data Access Juggernaut   499

as you app y changes to the objects That means t becomes someone e se’s job to track the r changes wh e d sconnected from a context, f you want to be ab e to pers st those changes back to the ­database ater When the t me does come to save the changes, you w a so need to re-attach the objects to a new context w th the change nformat on, so that the context can carry out the correct commands to update the database These concerns must be addressed f you want to mp ement an n-t er arch tecture bu t on the Ent ty Framework Fortunate y, there are a number of read y ava ab e so ut ons that you can everage to a d you n the task As of EF4, you can hook up detached objects rece ved from the c ent to a new context and exp c t y set change nformat on for the update You can a so overr de V sua Stud o’s defau t code generat on strategy n the EDM des gner and produce rea se f-track ng POCO ent t es from the mode WCF Data Serv ces and WCF RIA Serv ces are two other techno og es that can he p you bu d ­n-t er app cat ons w th the Ent ty Framework Each n the r own un que way, these two APIs stream ne CRUD operat ons n d sconnected n-t er scenar os, and they both work seam ess y w th the ent ty c asses produced by the defau t EDM code generator n V sua Stud o We cover WCF Data Serv ces and WCF RIA Serv ces n the next chapter

Resolving Impedance Mismatch The mapp ng ayer n the EDM s a very powerfu aspect of Ent ty Framework W th t, you can des gn many d fferent types of abstract ons over a phys ca database structure For examp e, you can define a s ng e conceptua ent ty based on two tab es n the database that have a one-to-one re at onsh p w th each other You can a so der ve mu t p e conceptua ent t es from a s ng e tab e In the samp e ­database, there s a many-to-many re at onsh p junct on tab e that s comp ete y abstracted away from you by the mapp ng ayer These are examp es are what’s common y referred to as ­impedance mismatch—where you don’t have an exact one-to-one correspondence between tab es n the ­database and ent t es n the app cat on (see the s debar “Defin ng ORM” ear er n the chapter) Rev s t the Mapp ng Deta s pane to see how t can be used to custom ze the mode and reduce mpedance m smatch Doub e-c ck SampleEF.edmx n the So ut on Exp orer to open t n the des gner Then r ght-c ck anywhere on the Customer ent ty and choose Tab e Mapp ng to d sp ay the Mapp ng Deta s pane, as shown n F gure 10-18

Figure 10-18  Mapp ng an ent ty and ts propert es w th tab es or v ews and the r co umns.

500  Part III  Applied SQL

You used the Mapp ng Deta s pane ear er to map three update stored procedures to the ­ ustomer ent ty, but stored procedure mapp ng s on y one of two purposes that th s pane serves Its C other purpose s to et you map tab es and co umns w th ent t es and propert es You get dropped nto the appropr ate mode based on whether you choose Stored Procedure Mapp ng or Tab e ­Mapp ng from the r ght-c ck context menu n the des gner, and you can a so togg e between the two modes by c ck ng the two buttons n the eft marg n of the Mapp ng Deta s pane The mapp ng shows that the Customer ent ty s mapped to the s ng e Customer tab e, and that there s a one-to-one mapp ng between same-named propert es and co umns Th s s how the w zard set th ngs up or g na y by defau t The Mapp ng Deta s pane ets you take th ngs further than th s, by a ow ng you to map add t ona tab es or v ews from the storage schema to the og ca Customer ent ty For examp e, f you had another tab e n the database that had a one-to-one re at onsh p w th Customer, you cou d c ck the drop-down st to br ng the second tab e and ts co umns nto the ent ty mapp ng You wou d then remove the second tab e from the conceptua ­schema, and the s ng e Customer ent ty wou d have a co ect ve set of propert es whose va ues are backed by two separate tab es n the database EF wou d automat ca y jo n the two tab es together when query ng for a s ng e Customer, and a so sp t a Customer update operat on nto separate ­commands to update each of the tab es nd v dua y A so not ce the drop-down st, wh ch ets you embed a row fi ter nto the mode for the mapp ng For examp e, mag ne that nstead of hav ng two separate tab es, both Customer and Employee ent t es n the mode were based on a s ng e under y ng Person tab e n the database Th s Person tab e wou d have a PersonType co umn to d st ngu sh between customers and emp oyees The PersonType co umn (ca ed the discriminator) cou d be spec fied accord ng y n the tab e mapp ng cond t ons for the Customer and Employee ent t es As a resu t, you cou d ­cont nue work ng conceptua y w th customers and emp oyees as d st nct ent ty types (or, f you w sh to ­everage nher tance, as d st nct ent ty types that der ve from the same base Person type), and a ow EF to manage the mapp ng to the Person tab e automat ca y beh nd the scenes If you were to query for a Customer or an Employee, EF wou d test PersonType n the database query accord ng y to return rows of the correct type S m ar y, EF wou d automat ca y store the correct va ue n the PersonType co umn depend ng on whether you were sav ng a customer or an emp oyee to the database At no t me wou d your code be aware of the actua Person tab e or the PersonType co umn In fact, you cou d change the database des gn n the back-end from us ng two tab es to one (and back aga n) w thout mpact ng your app cat on’s code whatsoever

Working with Many-to-Many Relationships Let’s demonstrate th s abstract on concept w th the many-to-many re at onsh p between Customer and Employee n the samp e database Drag two st box contro s from the Too box and drop them onto the form Name one of them lstCustomers and the other lstEmployees Next, drag and drop another button contro , name t btnManyToMany, and set ts Text property to Many To Many ­Relationships Now doub e-c ck the button and add the code n L st ng 10-17



Chapter 10  The M crosoft Data Access Juggernaut   501

Listing 10-17  Work ng w th many to many re at onsh ps.

private void btnManyToMany_Click(object sender, EventArgs e) { using (var ctx = new SampleDbEntities()) { // Create 3 customers and 3 employees var custClaus = new Customer() { FirstName = "Claus", LastName = "Hansen" }; var custTerry = new Customer() { FirstName = "Terry", LastName = "Adams" }; var custDan = new Customer() { FirstName = "Dan", LastName = "Park" }; ctx.Customers.AddObject(custClaus); ctx.Customers.AddObject(custTerry); ctx.Customers.AddObject(custDan); var empDavid = new Employee() { FirstName = "David", LastName = "Alexander" }; var empAndy = new Employee() { FirstName = "Andy", LastName = "Jacobs" }; var empErik = new Employee() { FirstName = "Erik", LastName = "Ryan" }; ctx.Employees.AddObject(empDavid); ctx.Employees.AddObject(empAndy); ctx.Employees.AddObject(empErik); // Create these many-to-many relationships: // Customer : Employees | Employee : Customers // Claus : Andy, Erik | Andy : Claus, Terry // Terry : Andy, David | David : Terry, Dan // Dan : David, Erik | Erik : Claus, Dan // Can either add employees to customers... custClaus.Employees.Add(empAndy); custClaus.Employees.Add(empErik); custTerry.Employees.Add(empAndy); custTerry.Employees.Add(empDavid); // ...or add customers to employees empErik.Customers.Add(custDan); empDavid.Customers.Add(custDan); // Save new customers, employees, and their relationships to the database ctx.SaveChanges(); // Get customers belonging to employee David (Terry and Dan) this.lstCustomers.Items.Clear(); foreach (var cust in empDavid.Customers) { this.lstCustomers.Items.Add( string.Format("{0} {1}", cust.FirstName, cust.LastName)); }

502  Part III  Applied SQL

// Get employees belonging to customer Terry (Andy and David) this.lstEmployees.Items.Clear(); foreach (var emp in custTerry.Employees) { this.lstEmployees.Items.Add( string.Format("{0} {1}", emp.FirstName, emp.LastName)); } } }

Th s s rea y s mp e code A t does s create three customers and three emp oyees, and then estab shes many-to-many re at onsh ps between them by add ng nto each other’s co ect ons Not ce how t doesn’t matter wh ch d rect on you work n; you can add customers to an emp oyee’s ­Customers co ect on, or you can add emp oyees to a customer’s Employees co ect on S ng e-step through the code wh e runn ng a SQL Profi er trace to watch t work When the SaveChanges method s ca ed on the context, the trace shows a the commands sent to the database As expected, the customers are created w th nd v dua ca s to the InsertCustomer stored procedure, and the emp oyees are created w th nd v dua y generated INSERT statements nto the Employee tab e But you’ a so not ce that add t ona INSERT statements nto the CustomerEmployee junct on tab e were generated to create the many-to-many re at onsh ps n the database Th s s more EF mag c at work S m ar y, f you were to break re at onsh ps by remov ng objects from co ect ons n the app cat on, EF wou d automat ca y ssue the appropr ate DELETE statements aga nst the CustomerEmployee tab e n the database Next, the code starts a foreach oop to terate a of the customers assoc ated w th emp oyee Dav d If you s ng e-step the code’s entry nto the oop, you w see n the trace that EF automat ca y ssues a query to SQL Server to retr eve the re ated customers from the Customer tab e by jo n ng on the CustomerEmployee junct on tab e Then there s another foreach oop that s s m ar, but works n the oppos te d rect on Th s one terates a of the emp oyees assoc ated w th customer Terry, and the trace shows that EF runs a query n SQL Server to retr eve the re ated emp oyees from the Employee tab e by aga n jo n ng on CustomerEmployee

Exploring the Entity SQL Alternative You w now earn about the Ent ty SQL anguage, wh ch offers a way to query the EDM w thout us ng LINQ to Ent t es Why two ways to query the EDM? We , first understand that Ent ty SQL s actua y the nat ve anguage of EF des gned spec fica y for query ng an EDM It was created as a core part of Ent ty Framework before the LINQ revo ut on, and ex sts ndependent y of LINQ Most of the t me, you’ st use LINQ to Ent t es and enjoy the benefits of strong-typ ng and object mater a zat on that t prov des However, there may be t mes when you’ need to use Ent ty SQL to wr te a query that can’t be expressed as we (or at a ) n LINQ Ent ty SQL a so offers a way to stream query resu ts nto a data reader when you don’t need to mater a ze them nto objects, as we’ demonstrate short y



Chapter 10  The M crosoft Data Access Juggernaut   503

Ent ty SQL bears a strong resemb ance to the T-SQL anguage upon wh ch t s based The key ­ fference between them s that Ent ty SQL express ons refer to conceptua ent t es and ­propert es d n the mode , rather than phys ca tab es and co umns n the database Let’s wr te a s mp e query to ­demonstrate Drag another button contro from the Too box and drop t onto the form Name the button btnEntitySql and set ts Text property to Entity SQL Then doub e-c ck the button and add the code n L st ng 10-18 Listing 10-18  Us ng Ent ty SQL nstead of L NQ to return ent ty objects.

private void btnEntitySql_Click(object sender, EventArgs e) { const string EntitySql = @" SELECT VALUE cust FROM SampleDbEntities.Customers AS cust WHERE cust.LastName NOT LIKE 'A%' ORDER BY cust.LastName"; using (var ctx = new SampleDbEntities()) { // Define, but do not execute, the Entity SQL query (deferred execution) var q = ctx.CreateQuery(EntitySql); // Implicitly execute query on the server, returning List var list = q.ToList(); MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); } }

Th s code s funct ona y equ va ent to what you wrote n L st ng 10-14 The on y d fference s that you’re us ng Ent ty SQL nstead of LINQ to Ent t es to query the database for the des red customers The EDM st cont nues to funct on as a mapp ng ayer between conceptua and phys ca mode s, and the context s st used to nvoke the query and track any changes made to the resu ts Observe how the code runs by s ng e-stepp ng through t w th a runn ng SQL Profi er trace as usua The CreateQuery method s nvoked on the context, rather than a LINQ to ­Ent t es query as n L st ng 10-14 Th s method accepts a str ng parameter that conta ns an Ent ty SQL ­express on expected to return a sequence of Customer ent t es As w th LINQ, th s method mere y defines the query but does not execute t Execut on s deferred unt the next ne of code that ca s ToList The ToList method ca k cks off the query n SQL Server as before You can see th s by ­watch ng the SQL Profi er trace as a step over the method ca w th the debugger Take a c oser ook at the syntax of th s Ent ty SQL query SELECT VALUE cust FROM SampleDbEntities.Customers AS cust WHERE cust.LastName NOT LIKE 'A%' ORDER BY cust.LastName

504  Part III  Applied SQL

As ment oned, Ent ty SQL quer es ook very s m ar to T-SQL quer es They can even be ­ arameter zed just ke T-SQL quer es are But remember that you’re dea ng w th the conceptua p mode here, not tab es In th s query, Customers refers to the ent ty set of Customer ent t es exposed by the Samp eDbEnt t es mode , not the Customer tab e n the database (even though they current y happen to be mapped one-to-one) S m ar y, LastName refers to a property of the Customer ent ty, not the LastName co umn of the Customer database tab e The VALUE keyword spec fies that the query returns a s ng e strong y typed object, wh ch s a Customer ent ty n th s case A so note that you must a as ent ty set names spec fied n the FROM c ause, un ke T-SQL where tab e a ases are opt ona In th s examp e, Customer ent t es n the Customers ent ty set are a ased as cust

Working with EntityClient We conc ude our Ent ty Framework coverage (and th s chapter) w th EntityClient, wh ch s yet another way to query the EDM Ent tyC ent on y understands Ent ty SQL; t doesn’t know anyth ng about LINQ Us ng Ent tyC ent, you w query the mode w th Ent ty SQL and stream the resu ts back us ng just a reader object Drag another button contro from the Too box and drop t onto the form Name the button ­btnEntityClient and set ts Text property to EntityClient Then doub e-c ck the button and add the code n L st ng 10-19 Listing 10-19  Us ng Ent ty SQL w th Ent tyC ent to return a stream ng reader.

private void btnEntityClient_Click(object sender, EventArgs e) { const string ConnStr = "name=SampleDbEntities"; const string EntitySql = @" SELECT VALUE cust FROM SampleDbEntities.Customers AS cust WHERE cust.LastName NOT LIKE @Pattern ORDER BY cust.LastName"; var names = new List(); using (var conn = new EntityConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new EntityCommand()) { cmd.Connection = conn; cmd.CommandText = EntitySql; cmd.Parameters.AddWithValue("Pattern", "A%"); using (var rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection | CommandBehavior.SequentialAccess)) { while (rdr.Read()) { var firstName = rdr.GetString(1);



Chapter 10  The M crosoft Data Access Juggernaut   505

var lastName = rdr.GetString(2); var name = string.Format("{0}, {1}", lastName, firstName); names.Add(name); } rdr.Close(); } } this.lstCustomers.DataSource = names; } }

Look fam ar? It shou d! Compare th s code w th L st ng 10-6 way back at the beg nn ng of the chapter You’re us ng raw ADO NET objects to query the EDM w th the Ent tyC ent prov der, just ke raw objects have been used to query SQL Server d rect y w th the Sq C ent for years It’s the exact same pattern deve opers have been fo ow ng s nce NET 1 0, though there are a few subt e ­d fferences that we’ po nt out as we exp a n the code Ent tyC ent s supp ed by the System.Data.Entity assemb y that your project s a ready ­referenc ng (reca that the EDM des gner automat ca y added that reference for you when you created the mode ) Just as w th Sq C ent at the beg nn ng of the chapter, you need to nform the comp er where t can find the EntityConnection and EntityCommand c asses referred to n your code by add ng the fo ow ng using d rect ve at the top of the source fi e using System.Data.EntityClient;

The ConnStr constant defined at the top of the method s mp y po nts to the named ­connect on SampleDbEntities Th s named connect on refers to the actua ent ty connect on str ng (stored and ma nta ned n the app cat on configurat on fi e) that fu y descr bes the metadata ocat on and prov der-spec fic connect on nformat on for the mode Next, the EntitySql constant s defined, and s set to the same Ent ty SQL express on as the ast ­examp e, except that you are parameter z ng the va ue for LIKE w th @Pattern rather than hardcod ng t as ‘A%’ After n t a z ng an empty st of str ngs n names, you create a new EntityConnection n conn, set ts ConnectionString property to the ConnsStr constant you defined ear er, and nvoke the Open method on t to estab sh a connect on to the mode Next, you create an EntityCommand object n cmd and set ts Connection property to a ­ ssoc ate the command w th the open connect on The CommandText property gets set to the Ent ty SQL ­statement defined ear er n the EntitySql constant Because the Ent ty SQL has a parameter ca ed @Pattern, you supp y the parameter name and va ue to the command’s Parameters co ect on us ng the AddWithValue method just as you d d n L st ng 10-3 w th T-SQL Note, however, that you do not prefix parameter names w th the at-s gn (@) symbo n the Parameters co ect on as you do w th the Sq C ent, and so the parameter name s spec fied as Pattern rather than @Pattern Now you nvoke ExecuteReader to nvoke the query and obta n an EntityDataReader object that returns the query resu ts The EntityDataReader works just ke the SqlDataReader, w th ­severa ­add t ona constra nts F rst, not ce the CommandBehavior.SequentialAccess sett ng that you’re 506  Part III  Applied SQL

­comb n ng w th the CommandBehavior.CloseConnection sett ng that you were us ng ear er when c­ reat ng a SqlDataReader by ca ng ExecuteReader on a SqlCommand Th s add t ona sett ng s ­requ red w th Ent tyC ent, and forces str ct sequent a consumpt on of the data stream A though a data reader s an nherent y sequent a mechan sm, the SqlDataReader w st et you access co umn va ues random y w th n each row de vered sequent a y by the resu t set The same s not true w th an EntityDataReader because the data returned by an Ent ty SQL query can nc ude ­h erarch ca resu ts, n wh ch case ch d ent t es get unpacked dur ng the reader’s stream ng process Th s mposes the requ rement for you to retr eve nd v dua co umn va ues sequent a y—once you extract a co umn va ue from the current ent ty be ng returned by the reader, you w not be ab e to extract the va ue of any prev ous co umn for the same ent ty because the reader has a ready ­advanced past t Not know ng how many ent t es you’ get back, you test for an end-of-stream cond t on w th each ca to the reader’s Read method at the top of a while b ock If there are no ent t es, Read w return true the very first t me t s ca ed, and the while b ock w never execute Otherw se, the b ock w execute once for each customer ent ty returned by the query For each ent ty, the reader’s GetString method s used to extract the first and ast name ­va ues so you can format the customer name Th s po nts out another mportant d fference w th the ­SqlDataReader—co umn va ues can on y be extracted from an EntityDataReader by ord na ­pos t on us ng the GetXxx methods supp ed for each data type (such as GetString, GetDateTime, GetInt64, and so on) You cannot use ndexed notat on ke you can w th a SqlDataReader to access co umns by name (such as rdr[“FirstName”]) A so not ce that you extract the first name before the ast name, ­because they are returned by the query n that order as co umns 1 and 2 Even though you’re ­d sp ay ng ast name first, you must extract them n the order they are returned because of the ­CommandBehavior.SequentialAccess sett ng (as just exp a ned) Each formatted name s added to the List co ect on that got n t a zed at the top of the method before you created the EntityConnection After c os ng the reader (wh ch a so c oses the ­connect on because of the CommandBehavior.CloseConnection sett ng), you set the DataSource ­property on the lstCustomers st box to b nd the str ng co ect on to t Now run the app cat on and s ng e-step through t w th a SQL Profi er trace as usua Th s s a good demonstrat on of how EF can be used to execute quer es and process the r resu ts w thout nvo v ng objects or LINQ By now you’ve earned a great dea about the Ent ty Framework—more than enough to get started bu d ng rea app cat ons w th t You bu t a funct ona Ent ty Data Mode , and n the next chapter you w earn how to use WCF Data Serv ces and WCF RIA Serv ces to extend the reach of your EDMs to c ent app cat ons across the Internet



Chapter 10  The M crosoft Data Access Juggernaut   507

Summary Th s chapter has de vered a fa r y cr t ca ana ys s across the gamut of NET data access APIs and too s that M crosoft has re eased over more than the past decade, w th gu dance to he p you ­nte gent y d st ngu sh between them We wa ked through numerous examp es and demonstrated cod ng ­techn ques for work ng w th each of them, and a so prov ded mportant gu de nes to keep n m nd as you do We started by exp a n ng the cont nued re evance of convent ona ADO NET—both raw data access obects and DataSets—a ongs de newer techno og es You started w th raw connect ons, commands, readers, adapters, and DataSets to run d rect SQL statements and execute stored procedures n SQL Server You a so earned about exp c t and mp c t transact ons and SQL Server Profi er Then we ntroduced the concept of LINQ, and ts var ous mp ementat ons, and you saw how LINQ to DataSet extends convent ona ADO NET for query ng both gener c and strong y typed DataSets We then advanced the d scuss on to Object Re at ona Mapp ng, what ORM techno og es are, and the types of prob ems they are des gned to so ve We a so c ar fied M crosoft’s pos t on on the r two ORM offer ngs LINQ to SQL and ADO NET Ent ty Framework a though LINQ to SQL w cont nue ­be ng supported n future vers ons of NET, the Ent ty Framework s pos t oned as the current and future recommended data access so ut on for deve op ng app cat ons Then we dug nto the extens ve capab t es of the Ent ty Framework, start ng by exam n ng the Ent ty Data Mode and ts under y ng conceptua , storage, and mapp ng schemas From there, you bu t funct ona NET app cat ons us ng LINQ to Ent t es to query and update data n SQL Server, ­demonstrat ng powerfu key EF features such as azy oad ng, ent ty mapp ngs, change track ng, many-to-many re at onsh ps, Ent ty SQL, and Ent tyC ent But the NET data access story does not end here In the next chapter, you w earn about WCF Data Serv ces and WCF RIA Serv ces—two frameworks you can use to bu d data access serv ces and n-t er app cat ons for c ents over the Wor d W de Web

508  Part III  Applied SQL

C hapter 1 1

WCF Data Access Technologies –Leonard Lobel

W

ndows Commun cat on Foundat on (WCF) s a commun cat ons p atform that prov des a the support you need to bu d d str buted, serv ce-or ented so ut ons n the NET Framework ­M crosoft first ntroduced WCF n ate 2006 as part of the NET Framework 3 0 re ease S nce then, WCF has estab shed tse f as a cornerstone of the framework, and s regarded today as the de facto so ut on for bu d ng any k nd of serv ce n NET

Defining Services There are many types of serv ces Some are ntended for use on y w th n an organ zat on’s pr vate ntranet, where commun cat on occurs on y beh nd the corporate firewa (w th remote access enab ed v a V rtua Pr vate Network [VPN] connect ons) These serv ces can be t ght y coup ed w th NET ­c ents v a sockets and effic ent b nary remot ng protoco s Other serv ces may be exposed pub c y over the Internet Pub c serv ces must be ab e to penetrate firewa s and support any type of ­c ent, wh ch typ ca y requ res them to use eXtens b e Markup Language (XML) or the re at ve y terser ­JavaScr pt Object Notat on (JSON) message format over a standard transport (usua y HTTP) These text-based ser a zat on formats are bu k er and ess effic ent than b nary ser a zat on, but they do support a much w der range of c ents—M crosoft and non-M crosoft a ke And there are many more var ab es Some serv ces requ re authent cat on, whereas many do not—and those that do use some authent cat on methods but not others Some serv ces operate ­synchronous y, others asynchronous y Serv ces are usua y state ess, but some support c ent sess ons ­Messag ng ­protoco s vary as we ; some serv ces use S mp e Object Access Protoco (SOAP), whereas others are based on the Representat ona State Transfer (REST) protoco and return data as an Atom Pub sh ng Protoco (AtomPub) response (an XML d a ect very s m ar to the Rea y S mp y Synd cat on [RSS] feed format) Before WCF, such part cu ars were typ ca y hard-coded nto an app cat on If you bu t a pr vate serv ce us ng b nary remot ng and then dec ded to open up the serv ce to the pub c Internet, you needed to mod fy s gn ficant code n your app cat on to support that change Th s s not norma y the case w th WCF Typ ca y, you bu d your app cat on to commun cate us ng WCF proxy c asses, and configure WCF separate y to meet the requ rements of your part cu ar network ng env ronment A though WCF configurat on can be coded us ng C# or V sua Bas c (VB) NET (wh ch essent a y embeds t nto an assemb y), t s usua y (and best) managed dec arat ve y n the app cat on’s externa

509

c­ onfigurat on fi e (th s s e ther Web.config or App.config n your M crosoft V sua Stud o project) So, for nstance, to enab e pub c access to serv ces that were former y pr vate, you can s mp y open a text fi e n Notepad and add SOAP w th HTTP n add t on to the b nary ser a zat on w th s­ ockets ­configurat on that s a ready n p ace Or, you cou d just as eas y tweak the WCF configurat on to e ­ nab e asynchronous behav or for serv ce methods that former y supported on y synchronous ­behav or These are just a few examp es of the advantages offered by WCF’s configurat on-based ­approach

WCF Data Access Options You can certa n y work w th WCF d rect y to create custom serv ces and expose data from an Ent ty Data Mode (EDM) w th the ADO NET Ent ty Framework, or any other data access ayer (DAL) of your choos ng To take th s “raw” approach, you need to start w th the bas cs, or what s common y referred to as the ABC’s of WCF Addresses, B nd ngs, and Contracts You must create serv ce, operat on, and data contracts, and then configure your serv ce mode w th appropr ate endpo nt addresses and compat b e bindings to be reachab e by c ents Serv ces are usua y state ess, so you must a so hand e c ent-s de change track ng and mu t -user confl ct reso ut on ent re y on your own The earn ng curve can be qu te steep, after wh ch you w st need to expend a great dea of effort to make t work A ternat ve y, you can turn to one of the two ater techno og es that M crosoft has bu t on top of WCF, wh ch are the focus of th s chapter These are WCF Data Serv ces and WCF RIA Serv ces, and they represent two very d fferent approaches for bu d ng data-or ented serv ces Both prov de abstract ons that sh e d you from many under y ng WCF part cu ars, so you get to spend more t me focus ng on your app cat on and ess t me on p umb ng For one th ng, you don’t need to code WCF contracts or manage change track ng on the c ent; a that gets done for you W th WCF RIA Serv ces (and M crosoft S ver ght), you don’t even need to create and update serv ce references; V sua Stud o generates code automat ca y v a a spec a nk that keeps your c ent and WCF RIA Serv ces projects n sync at a t mes In th s chapter, you w earn how to use WCF Data Serv ces and WCF RIA Serv ces by expos ng the SampleDb database’s EDM that you created n the prev ous chapter as serv ces, and then bu d ng c ents over those serv ces By deta ed compar son, you w earn how to choose nte gent y between these two frameworks for any g ven scenar o

Monitoring Network Activity with Fiddler In Chapter 10, you used SQL Profi er to mon tor the database act v ty that Ent ty Framework was conduct ng beh nd the scenes Now that you are extend ng the c ent-server mode to an n-t er serv ce-or ented arch tecture, you w s m ar y want to mon tor the network ­act v ty ­further up the stack (HTTP traffic) as you bu d separate c ent and server ­components A ­number of too s are ava ab e that w et you do th s (so-ca ed “packet sn ffers”) In our ­wa kthrough, we use F dd er (wh ch s sma , free, and easy to use) to watch and see exact y how c ent requests and serv ce responses are be ng transm tted to and from WCF serv ces on the

510  Part III  Applied SQL

other end of the w re Th s s an nva uab e too for earn ng the techno ogy as you exper ment w th the examp es n th s chapter And once you start bu d ng product on systems w th the techno ogy, t w a so prove to be an nd spensab e too for rout ne debugg ng—whether you are programm ng raw WCF, WCF Data Serv ces, or WCF RIA Serv ces Go to http://www.fiddler2.com to down oad and nsta F dd er When you start the too , t d sp ays a sp t screen w th requests on top and responses on bottom, and then mmed ate y beg ns to mon tor HTTP traffic (see F gure 11-8) You w refer to F dd er output rout ne y to observe c ent/server commun cat on as you test and debug the code n th s chapter You shou d a so beg n a SQL Profi er trace so that you can mon tor the database act v ty of your WCF serv ces as we ( nstruct ons for start ng a trace can be found n Chapter 10 under the ­sect on “Mon tor ng Database Act v ty w th SQL Server Profi er“)

WCF Data Services M crosoft des gned WCF Data Serv ces as a th n ayer over Ent ty Framework that exposes data-centr c serv ces to c ent app cat ons across the Internet You can th nk of WCF Data Serv ces as un versa Web Serv ces bu t just for data The p atform s based on open ndustry standards and protoco s, wh ch means that these serv ces are consumab e by v rtua y every type of c ent n the wor d And you can qu ck y bu d a WCF Data Serv ces over an EDM w th a most no effort whatsoever Before bu d ng your very first serv ce (and a correspond ng c ent p ece), et’s first exp a n the ndustry standards that WCF Data Serv ces s based upon—wh ch are REST and the Open Data Protoco (OData) REST prov des a un form nterface for query ng and updat ng data It s based on HTTP, ­mean ng that c ent requests are ssued n the form of GET, POST, MERGE, and DELETE act ons—standard verbs understood by a HTTP c ents Any REST query can be nvoked w th an HTTP GET request by ­express ng a the e ements of the query n a proper y formed Un form Resource Ident fier (a URI, wh ch s a more genera term than Un form Resource Locator [URL]) You can even test the serv ce w th an ord nary browser by typ ng the URI d rect y nto the address bar The POST, MERGE, and ­DELETE verbs correspond respect ve y to nsert, update, and de ete operat ons supported by the serv ce Un ke GET, the pay oad (parameters, data, and other metadata) for these operat ons s passed n HTTP headers; t s not embedded n the URI So you cannot use the browser’s address bar to nsert, update, or de ete data However, t s easy to v ew HTTP headers us ng F dd er, and you w be do ng that short y to nspect the comp ete pay oad for each c ent request ssued to the serv ce Just as REST enab es un versa data access v a HTTP, OData estab shes un versa data structure v a standard ser a zat on formats (see http://www.odata.org) A c ents can hand e p a n-text formats such as JSON and XML, and so OData defines standard response formats based on both formats JSON prov des a compact structure su tab e for many bas c types of serv ces, whereas XML forms the bas s for the more verbose AtomPub feed format AtomPub s the defau t ser a zat on format n WCF Data Serv ces, because t effect ve y everages the h erarch ca nature of XML to descr be the r ch structure of data and metadata n an EDM

Chapter 11  WCF Data Access Techno og es   511

WCF Data Services without Entity Framework M crosoft des gned WCF Data Serv ces w th a data source prov der ta ored espec a y for the Ent ty Framework, mak ng t v rtua y effort ess to create serv ces over any Ent ty Data Mode Th s s rat ona , g ven that Ent ty Framework s M crosoft’s preferred NET data access so ut on Nonethe ess, there are a ternat ve data source prov ders ava ab e to make WCF Data Serv ces work w th other data access ayers as we Th s approach requ res substant a add t ona effort, deta s of wh ch are beyond the scope of th s chapter Here s a br ef ment on of the a ternat ves to gu de you n the r ght d rect on f you need to earn more M crosoft supp es two data source prov ders that you can use to expose data sources other than Ent ty Framework w th WCF Data Serv ces These are the Reflect on and Stream ng ­prov ders To use the Reflect on prov der, your data c asses mp ement IQueryable to support query ng, and ­opt ona y mp ement IUpdateable to support updates as we You can map these c asses to memory-res dent objects or any back-end data store that you want—even LINQ to SQL or typed DataSets The Stream ng prov der g ves you a spec a IDataServiceStreamProvider nterface that you can mp ement to expose arge b nary objects (BLOBs), such as FILESTREAM data n SQL Server (wh ch we cover n Chapter 8) F na y, custom serv ce prov ders et you dynam ca y define data mode s of any type Th s approach requ res the most effort to mp ement, and shou d be used on y f the other ava ab e prov ders are nadequate For more nformat on about prov ders for WCF Data Serv ces, v s t http://msdn.microsoft.com/en-us/library/dd672591.aspx

Building a WCF Data Service To bu d the serv ce, fo ow these steps Start V sua Stud o and choose F e New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose Web Then se ect the ASP NET Empty Web App cat on temp ate to create a project named DemoRestService n a so ut on named DemoWcfDataServices, as shown n F gure 11-1

Figure 11-1  Creat ng an ASP.NET Web App cat on project for the WCF Data Serv ce.

512  Part III  Applied SQL

Every ASP NET app cat on s hosted on a web server, typ ca y Internet Informat on Serv ces (IIS) n product on env ronments, or the more ghtwe ght V sua Stud o Deve opment Web Server (a so known as “Cass n ”) on deve opment desktops and aptops But rather than serv ng up ord nary ­webpages, th s project w expose a spec a svc fi e that exposes an EDM us ng WCF Data Serv ces, ­us ng the same SampleDb EDM (and under y ng database) from the prev ous chapter

Creating the Entity Data Model The next step s to create the EDM (If you sk pped the prev ous chapter, you first need to create the ­database by runn ng the T-SQL scr pt n L st ng 10-1 of that chapter ) Then, you can repeat the same steps n the prev ous chapter (see the sect on “Bu d ng an Ent ty Data Mode “) to create the same EDM n th s project, or— f you have a ready bu t the EDM n the prev ous chapter— t s much eas er to s mp y copy t (and ts connect on str ng) from the prev ous chapter’s project To do so, open W ndows Exp orer to the prev ous chapter’s project, drag the SampleEF.edmx fi e, and drop t on the DemoRestService ­project n ­So ut on Exp orer Then copy the connectionStrings sect on n the App.config fi e from prev ous chapter’s project and paste t nto th s DemoRestService project’s Web.config fi e (just after the open ng ­ tag toward the top)



Note  The complete code—including the T-SQL script and a Visual Studio solution with the EDM already built—is available for download from the book’s companion website (see the “Introduction“ for details). Your EDM s now n p ace If you have been fo ow ng a ong w th the prev ous chapter, the EDM’s under y ng database now conta ns random samp e data from ear er exerc ses; f you have just created the database now, then t s empty In e ther case, refresh the database now w th new samp e data by runn ng the scr pt n L st ng 11-1 Listing 11-1  Popu at ng the database w th samp e data.

USE SampleDb GO DELETE FROM OrderHeader DELETE FROM CustomerEmployee DELETE FROM Customer



Chapter 11  WCF Data Access Techno og es   513

SET IDENTITY_INSERT Customer ON INSERT INTO Customer (CustomerId, FirstName, LastName, Balance) VALUES (1, 'Lukas', 'Keller', 35), (2, 'Andy', 'Jacobs', 10), (3, 'Mike', 'Ray', 40), (4, 'Josh', 'Barnhill', 5) SET IDENTITY_INSERT Customer OFF SET IDENTITY_INSERT OrderHeader ON INSERT INTO OrderHeader (OrderHeaderId, CustomerId, ShipVia, OrderStatus) VALUES (1, 1, 'Regular Mail', 'Shipped'), (2, 1, 'Express Mail', 'Pending'), (3, 2, 'Priority Mail', 'Shipped'), (4, 3, 'Priority Mail', 'Shipped'), (5, 3, 'Regular Mail', 'Cancelled'), (6, 3, 'Priority Mail', 'Shipped'), (7, 3, 'Express Mail', 'Pending'), (8, 4, 'Regular Mail', 'Shipped'), (9, 4, 'Express Mail', 'Shipped') SET IDENTITY_INSERT OrderHeader OFF

A that’s eft to mp ement the serv ce s to create an assoc ated svc fi e, wh ch w nstant y expose your EDM to any REST c ent R ght-c ck the DemoRestService project, choose Add New Item, and se ect Web from the st of temp ate categor es on the eft Then scro down the st of nsta ed temp ates and choose WCF Data Serv ce Name the fi e CustomerDataService.svc and then c ck Add, as shown n F gure 11-2

Figure 11-2  Creat ng an ASP.NET Web App cat on project for the WCF Data Serv ce.

Now rep ace the starter code that V sua Stud o creates by defau t w th the code shown n L st ng 11-2

514  Part III  Applied SQL

Listing 11-2  A s mp e WCF Data Serv ce.

using using using using using

System; System.Data.Services; System.Data.Services.Common; System.Linq; System.ServiceModel.Web;

namespace DemoRestService { public class CustomerDataService : DataService { public static void InitializeService(DataServiceConfiguration config) { config.SetEntitySetAccessRule("Customers", EntitySetRights.All); config.SetEntitySetAccessRule("OrderHeaders", EntitySetRights.All); config.SetEntitySetAccessRule("Employees", EntitySetRights.AllRead); config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2; } } }

Th s s a the code you need for create a fu y funct ona serv ce Any c ent can ca th s serv ce to query and update your EDM us ng REST and OData It s st WCF under the covers, but you d d not have to wr te any server-s de og c, create a s ng e WCF contract, or tweak a s ng e WCF configurat on sett ng—wh ch s qu te remarkab e As the code demonstrates, you need to do on y two s mp e th ngs F rst, nher t from DataService, where T s your Ent ty Data Mode ’s object context c ass (SampleDbEntities, n our ­current examp e) Second, mp ement the InitializeService method to contro wh ch parts of the EDM (that s, wh ch ent ty sets) are to be exposed by the serv ce, and what access eve s are perm tted (read-on y versus updateab e) In our examp e, a three ent ty sets n the EDM can be quer ed, but on y Customers and OrderHeaders are updateab e

Testing WCF Data Services with Internet Explorer Even a p a n web browser can supp y a proper y formatted URI to query th s serv ce and d sp ay the XML content of the AtomPub feed that gets returned n response Th s s a qu ck and easy way to test that your WCF Data Serv ces are work ng proper y before wr t ng even one ne of c ent code Try t out now by us ng Internet Exp orer 9 to ssue a few URI quer es to the serv ce



Chapter 11  WCF Data Access Techno og es   515

Important  These instructions work specifically with Internet Explorer; other browsers may exhibit different or undesirable behavior. If you have another browser set as your default, open up Internet Explorer explicitly and copy the URL into its address bar after your default browser opens. We also recommend 32-bit Internet Explorer over the 64-bit version, as the 64-bit Internet Explorer apparently crashes consistently in our test environment when ­displaying RSS feeds. R ght-c ck the CustomerDataService.svc fi e n So ut on Exp orer and choose V ew In Browser Th s mmed ate y starts the ASP NET Deve opment Web Server that hosts the serv ce on a random y ­ass gned port (1055 n th s examp e, but yours w be d fferent) and then aunches Internet Exp orer The browser nav gates to the svc fi e, and the serv ce responds w th the st of ava ab e ent ty sets As shown n F gure 11-3, these are the Customers, Employees, and OrderHeaders ent ty sets spec fied n the InitializeService method (L st ng 11-2)

Figure 11-3  Nav gate to the serv ce s .svc fi e to d scover the ava ab e ent ty sets.

Based on the “d scovery” of ava ab e ent ty sets, you can append any ent ty set name to the URI to request a data n that ent ty set to be returned For examp e, to v ew a the customers s mp y append Customers to the URL n the browser’s address bar The comp ete URI (aga n, w th a d fferent random y ass gned port number than 1055) s http://localhost:1055/CustomerDataService.svc/Customers

Go ahead and browse to th s URL Instead of d sp ay ng the raw AtomPub response feed, the feed ­read ng v ew (wh ch s on by defau t) n Internet Exp orer attempts unsuccessfu y to format the feed for d sp ay To v ew the AtomPub feed n ts raw XML form nstead, you need to turn off feed ­read ng v ew Open the Internet Opt ons d a og n Internet Exp orer, c ck the Content tab, and then c ck the Sett ngs button under Feeds and Web S ces to d sp ay the Feeds and Web S ce Sett ngs d a og Then uncheck Turn On Feed Read ng V ew, as shown n F gure 11-4, and c ck OK tw ce to d sm ss both d a ogs 516  Part III  Applied SQL

Figure 11-4  Turn off feed read ng v ew n nternet Exp orer to v ew the raw XML content of AtomPub responses.

Th s part cu ar change may not take effect unt Internet Exp orer s restarted So c ose the browser now and then once aga n r ght-c ck the CustomerDataService.svc fi e n So ut on Exp orer, choose V ew In Browser, and append Customers to the URL Th s t me Internet Exp orer d sp ays the ­AtomPub feed n readab e XML, and you can see that the URI query returns a customers n the ­database, as shown n F gure 11-5

Figure 11-5  Customer data returned as an AtomPub feed by WCF Data Serv ces.



Chapter 11  WCF Data Access Techno og es   517

To return a s ng e ent ty by pr mary key, append ts un que ID n parentheses after the ent ty set name For examp e, mod fy the URL as fo ows to return the s ng e customer w th a CustomerId va ue of 3 (M ke Ray) http://localhost:1055/CustomerDataService.svc/Customers(3L)

Not ce that the term nat ng L s requ red to convey the data type ( ong nteger, correspond ng to SQL Server’s 64-b t bigint type) of the pr mary key va ue The syntax can qu ck y grow contorted, as ad-hoc fi ter ng, sort ng, and a host of other d rect ves are supported as we For examp e, cons der the fo ow ng URI http://localhost:1055/CustomerDataService.svc/Customers?$filter=Balance gt 10&$expand=OrderHeade rs&$orderby=LastName

The $filter opt on n th s URI quer es for a customers w th a ba ance h gher than 10 The $expand opt on te s WCF Data Serv ces to return a the re ated OrderHeader ent t es w th each Customer ­ent ty n the response feed, and $orderby requests that the resu ts n the response shou d be sorted by LastName If you enter th s URI nto the browser’s address bar, the serv ce returns the two ­customers that meet the fi ter cr ter a Lukas Ke er and M ke Ray (hav ng ba ances of 35 and 40, respect ve y)

Note  An OData URI always resolves to a specific resource or set of resources. Taking this approach is often known as implementing resource-based services, in contrast with more traditional domain-oriented services based on operation methods. As you can see, you need to know the prec se OData convent ons to construct mean ngfu URI quer es (v s t http://www.odata.org/documentation/uri-conventions for the comp ete OData URI ­spec ficat on) Fortunate y, the WCF Data Serv ces c ent brar es for NET ( nc ud ng S ver ght and W ndows Phone 7) supp y a spec a LINQ prov der for th s purpose, common y known as LINQ to REST Th s prov der automat ca y trans ates c ent-s de LINQ quer es nto an equ va ent OData URI, mean ng that you rea y don’t need to earn the OData URI syntax f you are bu d ng M crosoft c ents over WCF Data Serv ces Th s s a huge benefit because, once aga n, you are us ng LINQ to avo d earn ng yet another query ng anguage (wh ch s a key object ve of LINQ)

Building Client Applications for WCF Data Services M crosoft prov des spec a brar es for desktop (W ndows and W ndows Presentat on Foundat on [WPF]), web (ASP NET and S ver ght), and W ndows Phone 7 c ent deve opment aga nst WCF Data Serv ces These brar es support LINQ to REST, wh ch converts your c ent-s de LINQ quer es nto the OData URI syntax expected by the serv ce They a so a prov de a statefu context object for track ng changes on the c ent and push ng updates back to the serv ce In th s sect on, you w create two W ndows c ents that consume the serv ce you bu t n the ­ rev ous sect on The first s a test c ent that demonstrates how LINQ to REST, OData, and change p track ng features are mp emented by the WCF Data Serv ces c ent brary Th s bare samp e has 518  Part III  Applied SQL

no user nterface ( n fact, we may just as we have mp emented t as a conso e app cat on), and s des gned to teach you how to code aga nst the WCF Data Serv ces c ent brar es You w then app y that know edge to bu d a more robust, nteract ve data entry c ent that supports h erarch ca CRUD (create, retr eve, update, and de ete) operat ons

Building a Test Client Let’s get started w th the test c ent R ght-c ck the DemoWcfDataServices so ut on n So ut on ­Exp orer and choose Add New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose W ndows Then se ect the W ndows Forms App cat on temp ate to create a new project named DemoRestClientTest, as shown n F gure 11-6

Figure 11-6  Add ng a new W ndows Forms c ent app cat on to the WCF Data Serv ces demo so ut on.

The DemoWcfDataServices so ut on now has two projects n t an ASP NET project (the serv ce) and a W ndows Forms project (the new DemoRestClientTest project w th an empty form) Your next step s to set a reference from the c ent project to the serv ce project To do th s, r ght-c ck the ­DemoRestClientTest project n So ut on Exp orer and choose Add Serv ce Reference In the Add Serv ce Reference d a og, c ck the D scover button V sua Stud o ocates the WCF Data Serv ces fi e ­CustomerDataServices.svc n the ASP NET project Expand the treev ew on the eft to revea the ­ent t es exposed by the serv ce Then type CustomerDataService for the Namespace at the bottom of the d a og, as shown n F gure 11-7 C ck OK to estab sh the WCF Data Serv ces reference These are the same steps you wou d perform for sett ng a reference to an ord nary WCF serv ce And as w th an ord nary WCF serv ce, V sua Stud o creates a c ent-s de proxy w th strong y typed data c asses that correspond to data contracts (ent ty types) exposed by the serv ce, a ong w th the p umb ng that ser a zes and deser a zes ent t es as they are sent and rece ved across the w re But a WCF Data Serv ces c ent proxy goes even further— t a so supports LINQ to REST, OData, and change track ng, as th s c ent app cat on w demonstrate

Chapter 11  WCF Data Access Techno og es   519

Figure 11-7  Creat ng a serv ce reference from a W ndows c ent project to a WCF Data Serv ces project.

Drag a button from the Too box, drop t on to the form, and name t btnTestClient Then doub ec ck the button V sua Stud o creates a hand er for the button’s Click event and opens a w ndow to the form’s code, wh ch you shou d comp ete y rep ace w th the code shown n L st ng 11-3 (remember to change the port number n the ServiceUri constant defin t on toward the top of the code w th the port number for your env ronment) Listing 11-3  A s mp e WCF Data Serv ces c ent.

using System; using System.Linq; using System.Windows.Forms; using DemoRestClientTest.CustomerDataService; namespace DemoRestClientTest { public partial class Form1 : Form { public Form1() { InitializeComponent(); } // - The "." after "localhost" is a hack to enable Fiddler packet-sniffing // - Change the port number accordingly for your dev machine public const string ServiceUri = "http://localhost.:1055/CustomerDataService.svc"; private void btnTestClient_Click(object sender, EventArgs e) {

520  Part III  Applied SQL

// Create the client context var ctx = new SampleDbEntities(new Uri(ServiceUri)); // Construct an OData URI query var q = from cust in ctx.Customers.Expand("OrderHeaders") where !cust.FirstName.StartsWith("A") orderby cust.LastName descending select cust; // Call the service (issues an HTTP GET) var customers = q.ToList(); if (customers.Count == 0) { MessageBox.Show("There are no matching customers"); return; } var firstCustomer = customers[0]; var orderCount = firstCustomer.OrderHeaders.Count; MessageBox.Show(string.Format( "Customer {0} has {1} order(s)", firstCustomer.CustomerId, orderCount)); // Change the customer's name firstCustomer.FirstName = "Cassie"; firstCustomer.LastName = "Hicks"; // Add a customer order var newOrder = new OrderHeader() { ShipVia = "Regular Mail", OrderStatus = "Open", }; firstCustomer.OrderHeaders.Add(newOrder); ctx.UpdateObject(firstCustomer); ctx.AddRelatedObject(firstCustomer, "OrderHeaders", newOrder); // Issues an HTTP MERGE to update the customer's name, // and an HTTP POST to INSERT the customer's new order ctx.SaveChanges(); } } }

You w earn exact y how th s c ent code works by s ng e-stepp ng through t w th the debugger, as you keep an eye on background HTTP conversat on w th F dd er (and back-end database ­act v ty w th SQL Profi er) Set a breakpo nt at the very top of the button’s c ck event hand er by c ck ng n the eft marg n area on the ne private void btnTestClient Click(object sender, EventArgs e). V sua ­Stud o d sp ays a red bu et n the marg n, nd cat ng that the breakpo nt s set



Chapter 11  WCF Data Access Techno og es   521

You are now ready to run the so ut on R ght-c ck DemoRestClientTest n So ut on Exp orer, choose Set As Startup Project, and press F5 V sua Stud o bu ds the so ut on and aunches the c ent ­W ndows form C ck the button on the form and you w h t the breakpo nt you just set n the event hand er Before proceed ng from th s po nt, make sure you have both F dd er and SQL Profi er runn ng ( nstruct ons for start ng F dd er are g ven n the “Mon tor ng Network Act v ty w th F dd er“ s debar ear er n th s chapter; steps for runn ng a SQL Profi er trace are prov ded n Chapter 10) Use the debugger to s ng e-step over the first ne of code (F10 s the defau t keystroke n V sua Stud o for s ng e-stepp ng) Th s ne creates the c ent-s de context and stores t n the var ab e ctx var ctx = new SampleDbEntities(new Uri(ServiceUri));

The context c ass s named SampleDbEntities, and was generated automat ca y when you created the serv ce reference V sua Stud o generated th s c ass n a namespace der ved by append ng the name CustomerDataService that you spec fied n F gure 11-7 (when creat ng the serv ce reference) to the project name DemoRestClientTest By nc ud ng a using DemoRestClientTest.CustomerDataService statement at the very top of the st ng, the comp er s ab e to recogn ze SampleDbEntities as the ­c ent context c ass generated by the serv ce reference The name of the context c ass, SampleDbEntities, s based on the EDM exposed by the serv ce But the c ass has noth ng to do w th Ent ty Framework— t runs ent re y on the c ent Spec fica y, SampleDbEntities on the c ent s a statefu context object that der ves from DataServiceContext, and t prov des strong y typed propert es based on the SampleDbEntities EDM d scovered when the ­serv ce reference s created (or updated) Th s context s capab e of query ng ent t es from the serv ce, ­track ng c ent-s de changes, and push ng updates back to the serv ce Not ce the ServiceUri passed n to the SampleDbEntities constructor Th s s a str ng constant that po nts to the serv ce, and s defined a b t further up n the code (just before the event hand er) as ­fo ows // - Change the port number accordingly for your dev machine // - The "." after "localhost" is a hack to enable Fiddler packet-sniffing public const string ServiceUri = "http://localhost.:1055/CustomerDataService.svc";

As we’ve a ready ment oned, and as nd cated n the comments, you need to change the port number from 1055 to whatever port number s random y ass gned by the V sua Stud o Deve opment Web Server on your mach ne A ternat ve y, you cou d eave the code as- s and exp c t y set port 1055 n the Web tab of the serv ce project’s propert es page In e ther case, make sure to nc ude the “dot” after localhost or F dd er w not mon tor HTTP traffic as the code executes Now step over the next ne Th s s a strong y typed LINQ query that fi ters by first name, sorts by ast name, and returns match ng customers a ong w th the r re ated orders var q = from cust in ctx.Customers.Expand("OrderHeaders") where !cust.FirstName.StartsWith("A") orderby cust.LastName descending select cust;

522  Part III  Applied SQL

The query source ctx s a DataServiceContext (that s, a WCF Data Serv ces c ent context) It ­ rov des strong y typed propert es based on the SampleDbEntities EDM exposed by the serv ce, such p as the Customers ent ty set and the FirstName and LastName propert es of the Customer ent ty n th s examp e (unfortunate y, the Expand method uses a str ng tera rather than strong typ ng to reference the OrderHeaders nav gat on property) Th s s a LINQ to REST query, wh ch may not be mmed ate y obv ous, but s c ear y ev dent the moment you hover over the q The too t p that appears shows that the LINQ query has just been transformed dynam ca y at runt me nto the fo ow ng equ va ent OData URI query str ng http://localhost.:1055/CustomerDataService.svc/Customers()?$filter=not startswith(FirstName,'A') &$orderby=LastName desc&$expand=OrderHeaders

At th s po nt, the query has been defined, but t has not been nvoked (the HTTP GET operat on has not yet been ssued) Both F dd er and SQL Profi er show that no act v ty has yet occurred beh nd the scenes Stepp ng over the next ne of code actua y executes the query var customers = q.ToList();

In genera LINQ terms, th s ne of code s mp y means “execute the query and return a st of objects ” For the LINQ to REST prov der n th s part cu ar examp e, t means “ ssue a GET request to a REST serv ce w th an OData URI query based on the LINQ query, and deser a ze the AtomPub feed returned by the serv ce nto a st of strong y typed Customer ent t es ” That’s qu te a ot to accomp sh w th just a s ng e ne of code, yet the output from F dd er and SQL Profi er confirms that th s s exact y what occurs F rst ook at F dd er Se ect the ast HTTP request n the st on the eft (the one made to ­CustomerDataService.svc) Then c ck the Inspectors tab to v ew the request and response n a sp t pane d sp ay C ck the Raw button at the top of the request and response pane s to read them n p a n text, as shown n F gure 11-8

Figure 11-8  Us ng F dd er to v ew the WCF Data Serv ces conversat on over HTTP.



Chapter 11  WCF Data Access Techno og es   523

The request pane at the top shows the GET request ssued by the c ent, w th the OData URI query str ng generated from the LINQ query Beneath, the response pane shows the AtomPub feed resu t w th a of the ent t es returned by the query (not ce the application/atom+xml va ue returned by Content-Type n the HTTP response header) The WCF Data Serv ces c ent brary ­automat ca y ­deser a zes th s response nto a st of Customer ent t es (each of wh ch nc udes OrderHeader ­ent t es popu ated n the OrderHeaders co ect on) So the very next ne of c ent code s ab e to work ­mmed ate y w th objects returned by the serv ce Of course, the serv ce s ts n front of an EDM managed by Ent ty Framework, wh ch n turn c­ ommun cates w th SQL Server (see Chapter 10 for deta ed coverage of EF and the EDM) When the serv ce rece ves the URI query, t runs an equ va ent query aga nst the EDM, wh ch generates the ­actua T-SQL statements that u t mate y h t the database That’s a ot of ayers, so et’s spe t out c ear y once more For the request ■

The c ent defines a LINQ query

■

The WCF Data Serv ces c ent brary converts the LINQ query nto OData URI syntax

■

An HTTP GET s ssued to WCF Data Serv ces w th an OData URI query str ng

■

WCF Data Serv ces generates an equ va ent query aga nst the EDM

■

Ent ty Framework generates the requ red T-SQL statement to query the database

■

The T-SQL statement s executed n SQL Server

Now sw tch over to SQL Profi er As shown n the SQL Profi er trace, EF generates and executes a d rect T-SQL statement that sat sfies the EDM query SELECT [Project1].[C1] AS [C1], [Project1].[CustomerId] AS [CustomerId], [Project1].[FirstName] AS [FirstName], [Project1].[LastName] AS [LastName], [Project1].[Balance] AS [Balance], [Project1].[CreatedAt] AS [CreatedAt], [Project1].[UpdatedAt] AS [UpdatedAt], [Project1].[C2] AS [C2], [Project1].[C3] AS [C3], [Project1].[OrderHeaderId] AS [OrderHeaderId], [Project1].[CustomerId1] AS [CustomerId1], [Project1].[ShipVia] AS [ShipVia], [Project1].[OrderStatus] AS [OrderStatus], [Project1].[Notes] AS [Notes], [Project1].[CreatedAt1] AS [CreatedAt1], [Project1].[UpdatedAt1] AS [UpdatedAt1] FROM ( SELECT [Extent1].[CustomerId] AS [CustomerId], [Extent1].[FirstName] AS [FirstName], [Extent1].[LastName] AS [LastName], [Extent1].[Balance] AS [Balance], [Extent1].[CreatedAt] AS [CreatedAt], [Extent1].[UpdatedAt] AS [UpdatedAt],

524  Part III  Applied SQL

1 AS [C1], N'OrderHeaders' AS [C2], [Extent2].[OrderHeaderId] AS [OrderHeaderId], [Extent2].[CustomerId] AS [CustomerId1], [Extent2].[ShipVia] AS [ShipVia], [Extent2].[OrderStatus] AS [OrderStatus], [Extent2].[Notes] AS [Notes], [Extent2].[CreatedAt] AS [CreatedAt1], [Extent2].[UpdatedAt] AS [UpdatedAt1], CASE WHEN ([Extent2].[OrderHeaderId] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C3] FROM [dbo].[Customer] AS [Extent1] LEFT OUTER JOIN [dbo].[OrderHeader] AS [Extent2] ON [Extent1].[CustomerId] = [Extent2].[CustomerId] WHERE NOT ([Extent1].[FirstName] LIKE 'A%') ) AS [Project1] ORDER BY [Project1].[LastName] DESC, [Project1].[CustomerId] ASC, [Project1].[C3] ASC

Not ce how th s query jo ns Customer w th OrderHeader, wh ch returns re ated order data w th each se ected customer Th s s because of the c ent-s de LINQ query spec fied the Expand method When ca ng the serv ce, the LINQ to REST prov der trans ated Expand(“OrderHeaders”) nto the equ va ent OData URI syntax $expand=OrderHeaders, wh ch EF then trans ated nto the ­appropr ate T-SQL jo n After th s T-SQL query executes n SQL Server ■

The resu ts are returned to Ent ty Framework

■

EF mater a zes the resu t set nto conceptua ent t es

■

WCF Data Serv ces ser a zes the conceptua ent t es nto AtomPub format

■

The AtomPub response feed s sent back to the c ent

■

The WCF Data Serv ces c ent brary deser a zes the AtomPub response feed nto objects

Based on the samp e data that you popu ated the database w th ( n L st ng 11-1), th s query returns the three customers whose first names do not beg n w th the etter A (a ong w th the r orders) to the ­c ent At th s t me, the c ent-s de context s aware of three Customer ent t es and the r re ated ­OrderHeader ent t es

Note  WCF Data Services can be called either synchronously (blocking the client from ­executing while waiting for a response to each request) or asynchronously (­without ­blocking). Some clients, notably Silverlight and Windows Phone 7, support only ­asynchronous service calls. This Windows desktop application is working ­synchronously, which is a bit simpler to code (and delivers a less responsive user experience) than ­asynchronous communication. Later in this chapter, you will create a Silverlight client with WCF RIA Services that works asynchronously, and in Chapter 13, you will build a Windows Phone 7 OData client that makes asynchronous calls to WCF Data Services hosted in the cloud on Windows Azure.



Chapter 11  WCF Data Access Techno og es   525

Cont nue s ng e-stepp ng After first ensur ng that there s at east one customer n the st returned by the serv ce, the code extracts the first customer from the st and d sp ays the customer ID and the number of orders for that customer var firstCustomer = customers[0]; var orderCount = firstCustomer.OrderHeaders.Count; MessageBox.Show(string.Format( "Customer {0} has {1} order(s)", firstCustomer.CustomerId, orderCount));

Next, the code mod fies the customer’s name and adds another order for them S ng e-step through t now // Change the customer's name firstCustomer.FirstName = "Cassie"; firstCustomer.LastName = "Hicks"; // Add a customer order var newOrder = new OrderHeader() { ShipVia = "Regular Mail", OrderStatus = "Open", }; firstCustomer.OrderHeaders.Add(newOrder);

Not ce how th s s the same cod ng exper ence as server-s de Ent ty Framework, because there s a v rtua y dent ca data mode defined on the c ent Of course these c ent-s de ent t es may ook and fee ke EF ent t es, but they most certa n y are not The c ent has no awareness of EF’s ObjectContext (or any other data source that may s t beh nd the serv ce); t just works b ssfu y w th oca objects that are automat ca y sent back and forth between tse f and the WCF Data Serv ces The next two nes of code nform the c ent context of the changes made to the customer ctx.UpdateObject(firstCustomer); ctx.AddRelatedObject(firstCustomer, "OrderHeaders", newOrder);

The UpdateObject method marks the Customer ent ty (w th updated FirstName and LastName propert es) as “mod fied,” and the AddRelatedObject method marks the new OrderHeader ent ty added to the customer’s OrderHeaders co ect on as “new ” Th s step s essent a —you must use these context methods to exp c t y mark ent t es as changed, or the WCF Data Serv ces c ent brary w not track them as such S ng e-step through th s code, stopp ng just before the ast ne n the method that ca s SaveChanges ctx.SaveChanges();

Before execut ng the SaveChanges method, take a qu ck ook over at F dd er and SQL Profi er They show that no act v ty (from th s app cat on) has transp red s nce the query that n t a y retr eved the three customers Th s s expected—a changes made thus far have occurred ent re y on the c ent, where those changes are be ng and tracked by the WCF Data Serv ces c ent context The SaveChanges method ssues the appropr ate HTTP requests to push a the buffered up changes on the c ent back to the serv ce and

526  Part III  Applied SQL

update the database Step over the SaveChanges method ca now, and then sw tch r ght back to F dd er and Profi er to see the resu t In F dd er, you can see that two requests were sent to the serv ce The first s an HTTP MERGE that updates the customer w th the name change, and the second s an HTTP POST that adds the new order Let’s exam ne these requests (and the r responses) n deta , start ng w th the MERGE request for the update shown n L st ng 11-4 Listing 11-4  An HTTP MERGE request that updates a Customer ent ty.

MERGE http://localhost.:1055/CustomerDataService.svc/Customers(3L) HTTP/1.1 User-Agent: Microsoft ADO.NET Data Services DataServiceVersion: 1.0;NetFx MaxDataServiceVersion: 2.0;NetFx Accept: application/atom+xml,application/xml Accept-Charset: UTF-8 Content-Type: application/atom+xml If-Match: W/"datetime'2012-02-01T12%3A32%3A18.2991875'" Host: localhost.:1055 Content-Length: 991 Expect: 100-continue





2012-02-01T19:51:38.2358239Z http://localhost.:1055/CustomerDataService.svc/Customers(3L)

40.0000 2012-02-01T12:32:18.2991875 3 Cassie Hicks 2012-02-01T12:32:18.2991875



The MERGE (update) request s made w th an OData URI that spec fies the customer to be updated as Customers(3L), fo owed by severa HTTP headers and the updated data tse f Interest ng y, the User-Agent header (wh ch dent fies the HTTP c ent mak ng the request) st reads Microsoft ADO.NET Data Services (the o d name for WCF Data Serv ces) Fortunate y, the serv ce does not m nd the c ent dent fy ng tse f

Chapter 11  WCF Data Access Techno og es   527

w th the outdated brand name and accepts the request anyway The If-Match header (a few nes down) s far more noteworthy and s gn ficant Reca from Chapter 10 that the Customer ent ty n the EDM s mapped to an UpdateCustomer stored procedure, and that UpdateCustomer expects the or g na va ue of the UpdatedAt co umn to mp ement a concurrency check aga nst mu t -user confl cts The WCF Data Serv ces c ent brary detected th s or g na va ue mapp ng n the EDM when the serv ce reference was ­estab shed, so t s aware that the or g na va ue of every Customer ent ty’s UpdatedAt property needs to be preserved It s thus ab e to return that or g na va ue n the If-Match header n the MERGE request, so that the serv ce can pass t on to EF, and EF can pass t on to the UpdateCustomer stored procedure SQL Profi er shows the actua stored procedure ca executed n SQL Server exec [dbo].[UpdateCustomer] @CustomerId=3,@FirstName='Cassie',@LastName='Hicks',@ Balance=40.0000,@OriginalUpdatedAt='2012-02-01 14:51:38.6967614'

After the update, the serv ce response s returned to the c ent It conta ns severa HTTP headers, as shown n L st ng 11-5 Listing 11-5  The new UpdatedAt concurrency va ue returned to the c ent as an ETag response header.

HTTP/1.1 204 No Content Server: ASP.NET Development Server/10.0.0.0 Date: Wed, 01 Feb 2012 19:51:38 GMT X-AspNet-Version: 4.0.30319 DataServiceVersion: 1.0; Cache-Control: no-cache ETag: W/"datetime'2012-02-01T14%3A51%3A38.6967614'" Content-Length: 0 Connection: Close

Aga n, because of the or g na UpdatedAt va ue mapp ng, the serv ce retr eves the new y ass gned va ue (set by the stored procedure after a successfu update) and returns t to the c ent n the ETag header The WCF Data Serv ces c ent brary then automat ca y refreshes the c ent-s de nstance of the ent ty w th th s updated va ue, thereby synchron z ng t w th the ent ty updated n the database Now ook at the how the new order was nserted In F dd er, c ck the second request to v ew the HTTP traffic for the POST that nserted the order The request data appears n the top pane , as shown n L st ng 11-6 Listing 11-6  An HTTP POST request that nserts a new OrderHeader ent ty.

POST http://localhost.:1055/CustomerDataService.svc/Customers(3L)/OrderHeaders HTTP/1.1 User-Agent: Microsoft ADO.NET Data Services DataServiceVersion: 1.0;NetFx MaxDataServiceVersion: 2.0;NetFx Accept: application/atom+xml,application/xml Accept-Charset: UTF-8

528  Part III  Applied SQL

Content-Type: application/atom+xml Host: localhost.:1055 Content-Length: 961 Expect: 100-continue





2012-02-01T19:51:38.8188317Z

0001-01-01T00:00:00 0

0 Open Regular Mail 0001-01-01T00:00:00



As w th the MERGE request, the POST ( nsert) request s made w th an OData URI that ­spec fies the customer orders co ect on to be nserted nto as Customers(3L)/OrderHeaders, fo owed by severa HTTP headers and the new order data And as before, the request s forwarded by WCF Data Serv ces to Ent ty Framework The OrderHeader ent ty s not mapped to stored procedure mapp ngs n the EDM ke the Customers ent ty s, so th s t me EF dynam ca y generates and executes a d rect ­parameter zed INSERT statement for the new order, as shown n the SQL Profi er trace exec sp_executesql N'insert [dbo].[OrderHeader]([CustomerId], [ShipVia], [OrderStatus], [Notes], [CreatedAt], [UpdatedAt]) values (@0, @1, @2, null, @3, @4) select [OrderHeaderId] from [dbo].[OrderHeader] where @@ROWCOUNT > 0 and [OrderHeaderId] = scope_identity()',N'@0 bigint,@1 varchar(20),@2 varchar(20),@3 datetime2(7),@4 datetime2(7)',@0=3,@1='Regular Mail',@2='Open',@3='0001-01-01 00:00:00',@4='0001-01-01 00:00:00'

After the nsert, the serv ce response s returned to the c ent The response nd cates that the ­ent ty was created, and returns the ent re new ent ty to the c ent, as shown n L st ng 11-7



Chapter 11  WCF Data Access Techno og es   529

Listing 11-7  The HTTP response for a new y nserted ent ty.

HTTP/1.1 201 Created Server: ASP.NET Development Server/10.0.0.0 Date: Wed, 01 Feb 2012 19:51:39 GMT X-AspNet-Version: 4.0.30319 DataServiceVersion: 1.0; Content-Length: 1345 Location: http://localhost.:1055/CustomerDataService.svc/OrderHeaders(10L) Cache-Control: no-cache Content-Type: application/atom+xml;charset=utf-8 Connection: Close

http://localhost.:1055/CustomerDataService.svc/OrderHeaders(10L)

2012-02-01T19:51:39Z





10 3 Regular Mail Open

0001-01-01T00:00:00 0001-01-01T00:00:00



Building a Data Entry Client Now that you have a firm understand ng of the core WCF Data Serv ces concepts (REST, OData, and the c ent brar es), you are ready to bu d a more usefu app cat on Th s next WCF Data Serv ces c ent comb nes the techn ques you just earned w th a W ndows Forms user nterface and some data b nd ng to et the user ed t customers and orders

530  Part III  Applied SQL

Note  Data binding is a vast topic, and Windows Forms, ASP.NET, and WPF/Silverlight each support very different data binding architectures that are beyond the scope of this book. The minimal data binding code in our examples is provided to demonstrate client/server data access techniques with WCF Data Services (and, in the next section, WCF RIA Services). R ght-c ck the DemoWcfDataServices so ut on n So ut on Exp orer and choose Add New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose W ndows Then se ect the W ndows Forms App cat on temp ate to create a new project named DemoRestClientDataEntry The DemoWcfDataServices so ut on now has three projects n t an ASP NET project (the ­serv ce) and two W ndows Forms projects (the prev ous DemoRestClientTest project and the new ­DemoRestClientDataEntry project w th an empty form) Set a reference from the ­DemoRestClientDataEntry project to the serv ce project as you d d for the test c ent R ght-c ck the DemoRestClientDataEntry project, choose Add Serv ce Reference, and c ck D scover Then type ­CustomerDataService for the Namespace and c ck OK Now drag severa contro s onto the empty form For v ew ng and ed t ng customers, create two buttons named btnNewCustomer and btnDeleteCustomer a ong w th a DataGridView named grdCustomers Create two more buttons named btnNewOrder and btnDeleteOrder a ong w th another DataGridView named grdOrders for v ew ng and ed t ng orders Then add two buttons named btnLoad and btnSave to query ng from the serv ce and send changes back to t F na y, set the AllowUserToAddRows and AllowUserToDeleteRows propert es of both DataGridView contro s Fa se (th s s because add ng and de et ng ent t es w be hand ed by code beh nd the respect ve New and De ete buttons) After perform ng some aesthet c a gnment and formatt ng, your form shou d appear someth ng ke the one shown n F gure 11-9

Figure 11-9  The WCF Data Serv ces data entry c ent user nterface.



Chapter 11  WCF Data Access Techno og es   531

Next, r ght-c ck on the form’s des gn surface and choose V ew Code to open a w ndow to the form’s code Rep ace t ent re y w th the code shown n L st ng 11-8 (once aga n, remember to change the port number n the serviceUri constant—th s t me defined n the CreateNewContext method— w th the port number for your env ronment) Listing 11-8  Code for the WCF Data Serv ces data entry c ent form.

using using using using using

System; System.Collections.Generic; System.Data.Services.Client; System.Linq; System.Windows.Forms;

using DemoRestClientDataEntry.CustomerDataService; namespace DemoRestClientDataEntry { public partial class Form1 : Form { private SampleDbEntities _context; private List _customers; private Customer _currentCustomer; private BindingSource _customersBindingSource; private BindingSource _ordersBindingSource; public Form1() { InitializeComponent(); } private void btnLoad_Click(object sender, EventArgs e) { this.CreateNewContext(); var query = from customer in this._context.Customers.Expand("OrderHeaders") select customer; this._customers = query.ToList(); this._customersBindingSource = new BindingSource(this._customers, null); this.BindGrid(this.grdCustomers, this._customersBindingSource); this.grdOrders.DataSource = null; this.ShowCustomerOrders(); } private void CreateNewContext() { // - The "." after "localhost" is a hack to enable Fiddler packet-sniffing

532  Part III  Applied SQL

// - Change the port number accordingly for your dev machine var serviceUri = new Uri("http://localhost.:1055/CustomerDataService.svc"); this._context = new SampleDbEntities(serviceUri); } private void grdCustomers_SelectionChanged(object sender, EventArgs e) { this.ShowCustomerOrders(); } private void ShowCustomerOrders() { if (this.grdCustomers.SelectedRows.Count == 1) { this._currentCustomer = (Customer)this.grdCustomers.SelectedRows[0].DataBoundItem; this._ordersBindingSource = new BindingSource(this._currentCustomer.OrderHeaders, null); this.BindGrid(this.grdOrders, this._ordersBindingSource); } } private void btnNewCustomer_Click(object sender, EventArgs e) { this._customersBindingSource.AddNew(); } private void grdCustomers_CellValueChanged (object sender, DataGridViewCellEventArgs e) { var customer = (Customer)this.grdCustomers.SelectedRows[0].DataBoundItem; var changeInfo = this._context.Entities.SingleOrDefault(ed => ed.Entity == customer); if (changeInfo == null) { this._context.AddToCustomers(customer); } else if (changeInfo.State == EntityStates.Unchanged) { this._context.UpdateObject(customer); } } private void btnDeleteCustomer_Click(object sender, EventArgs e) { var customer = (Customer)this.grdCustomers.SelectedRows[0].DataBoundItem; this._customersBindingSource.RemoveCurrent(); this.grdOrders.DataSource = null;



Chapter 11  WCF Data Access Techno og es   533

var changeInfo = this._context.Entities.SingleOrDefault(ed => ed.Entity == customer); if (changeInfo != null) { foreach (var orderHeader in customer.OrderHeaders) { this._context.DeleteObject(orderHeader); } this._context.DeleteObject(customer); } } private void btnNewOrder_Click(object sender, EventArgs e) { if (this.grdOrders.Columns.Count == 0) { this.ShowCustomerOrders(); this.grdCustomers.EndEdit(); } var orderHeader = (OrderHeader)this._ordersBindingSource.AddNew(); orderHeader.CustomerId = this._currentCustomer.CustomerId; orderHeader.Customer = this._currentCustomer; } private void grdOrders_CellValueChanged (object sender, DataGridViewCellEventArgs e) { var orderHeader = (OrderHeader)this.grdOrders.SelectedRows[0]. DataBoundItem; var changeInfo = this._context.Entities.SingleOrDefault(ci => ci.Entity == orderHeader); if (changeInfo == null) { this._context.AddToOrderHeaders(orderHeader); } else if (changeInfo.State == EntityStates.Unchanged) { this._context.UpdateObject(orderHeader); } } private void btnDeleteOrder_Click(object sender, EventArgs e) { var orderHeader = (OrderHeader)this.grdOrders.SelectedRows[0]. DataBoundItem; this._ordersBindingSource.RemoveCurrent(); this.grdOrders.DataSource = null;

534  Part III  Applied SQL

var changeInfo = this._context.Entities.SingleOrDefault(ed => ed.Entity == orderHeader); if (changeInfo != null) { this._context.DeleteObject(orderHeader); } } private void btnSave_Click(object sender, EventArgs e) { this.grdCustomers.EndEdit(); this.grdOrders.EndEdit(); try { this._context.SaveChanges(SaveChangesOptions.Batch); } catch (Exception ex) { MessageBox.Show (ex.Message, "Error", MessageBoxButtons.OK, MessageBoxIcon.Error); } this.grdCustomers.Refresh(); this.grdOrders.Refresh(); } private void BindGrid(DataGridView grd, BindingSource bs) { var readOnlyColumns = new string[] { "CustomerId", "OrderHeaderId", "CreatedAt", "UpdatedAt" }; var hiddenColumns = new string[] { "Customer" }; grd.DataSource = bs; foreach (DataGridViewColumn col in grd.Columns) { col.ReadOnly = readOnlyColumns.Contains(col.HeaderText); col.Visible = !hiddenColumns.Contains(col.HeaderText); } grd.AutoResizeColumns(); } } }

Your ast step s to w re up the event hand ers Return to the form des gner and c ck the Events button n the too bar at the top of the Propert es gr d Then, one at a t me, se ect nd v dua contro s and ass gn them to event hand ers n the code, as fo ows ■



Ass gn the btnNewCustomer button’s Click event to btnNewCustomer Click

Chapter 11  WCF Data Access Techno og es   535

■

Ass gn the btnDeleteCustomer button’s Click event to btnDeleteCustomer Click

■

Ass gn the grdCustomers gr d’s SelectionChanged event to grdCustomers SelectionChanged

■

Ass gn the grdCustomers gr d’s CellValueChanged event to grdCustomers CellValueChanged

■

Ass gn the btnNewOrder button’s Click event to btnNewOrder Click

■

Ass gn the btnDeleteOrder button’s Click event to btnDeleteOrder Click

■

Ass gn the grdOrders gr d’s CellValueChanged event to grdOrders CellValueChanged

■

Ass gn the btnLoad button’s Click event to btnLoad Click

■

Ass gn the btnSave button’s Click event to btnSave Click

Th s c ent code adds some bas c data b nd ng to the concepts you earned w th the test c ent Before you run the so ut on, et us exp a n the mportant aspects of the code n L st ng 11-8 F rst, not ce the pr vate SampleDbEntities form var ab e named context Th s means that the c ent context ma nta ns ts state dur ng the fet me of the form, track ng the user’s changes made from the t me they c ck Load to retr eve data and the t me they c ck Save to update t back C ck ng Load fires btnLoad Click wh ch nstant ates the c ent context by ca ng CreateNewContext As w th the test c ent, you must adjust the serv ce URL’s port number 1055 n the CreateNewContext method ­accord ng y, and ensure that there s a “dot” after localhost so that F dd er can mon tor HTTP requests After creat ng the context, the btnLoad Click method executes a LINQ to REST query that retr eves a customers and the r orders nto a st, stores the st n the pr vate form var ab e customers, and b nds the st to the top gr d (grdCustomers) us ng a BindingSource object As soon as the st s bound to the top gr d, the first customer row gets se ected automat ca y, so btnLoad ca s the ShowCustomerOrders method to b nd the first customer’s orders to the bottom gr d (grdOrders) When the user se ects any other customer n the top gr d, the grdCustomers Selection event hand er responds w th another ShowCustomerOrders ca to update the bottom gr d for the se ected customer The ShowCustomerOrders method uses the DataBoundItem property to obta n the object bound to the current y se ected row n the top gr d, wh ch t casts to the expected Customer ent ty type W th the se ected Customer ent ty n hand, the method b nds ts OrderHeaders co ect on to grdOrders us ng a second BindingSource object The next three methods hand e Customer nserts, updates, and de etes act ons made by the user n the top gr d In part cu ar, these methods respond to events that fire as the user ed ts data, and nform the c ent context to track changes as the user makes them When the user c cks New Customer, the btnNewCustomer method ca s AddNew on the c­ ustomer b nd ng source Th s nterna y creates a new Customer object and adds t to the st bound to ­grdCustomers, wh ch n turn appends a new row to the bottom of the gr d for the user to enter the new customer’s data The context s not yet aware of the new ent ty; t w be nformed about the new customer as soon as the user ed ts the first ce n the new gr d row In grdCustomers CellValueChanged, the code responds to ed ts made by the user on any customer row Th s method determ nes whether the user s ed t ng a new or ex st ng customer by query ng the 536  Part III  Applied SQL

c ent context’s Entities co ect on for the current customer Th s co ect on ho ds change ­nformat on for a of the ent t es be ng tracked on the c ent, and the SingleOrDefault method searches the ­co ect on for the part cu ar customer that the user s ed t ng If t s not found, that means that the context s not yet track ng the object, wh ch must mean that th s s a new customer (added when the user c cked New Customer) In th s case, AddToCustomers s ca ed on the context, nstruct ng t to track th s customer as new If the context s a ready track ng the object, and the change ­nformat on’s State property equa s EntityStates.Unchanged, t means that th s s an ex st ng customer that has not yet been mod fied In th s case, UpdateObject s ca ed on the context, nstruct ng t to track th s ­customer as mod fied It s mportant to remember that the grdCustomers CellValueChanged method fires for each ce changed n a gr d’s row, but the context needs to be nstructed to track that row’s ent ty state once and on y once That s why t’s cr t ca to check the State property For examp e, cons der the new ­customer scenar o The AddToCustomers method s ca ed the first t me a ce s ed ted n a new ­customer row When another ce s ed ted n the same row, the event hand er fires aga n, on y th s t me the new Customer ent ty s found n the Entities co ect on because of the prev ous ­AddToCustomers ca Therefore, the State property s checked to prevent the act on from be ng nterpreted ncorrect y as an update to be tracked rather than an nsert that’s a ready be ng tracked The State property for the new customer s EntityStates.Added, not EntityStates.Unchanged, wh ch prevents the event hand er from erroneous y track ng the same customer aga n as mod fied S m ar y, the first t me the user ed ts the ce or an ex st ng customer’s row, the State property s changed from EntityStates.Unchanged to EntityStates.Modified by the ca to UpdateObject, wh ch prevents ex st ng customer ent t es from be ng tracked as mod fied mu t p e t mes Once a customer s be ng tracked as e ther new or changed, t w have an entry n the context’s Entities co ect on w th a State other than EntityStates.Unchanged, n wh ch case the grdCustomers CellValueChanged event hand er does ­noth ng When the user c cks De ete Customer, the btnDeleteCustomer method first obta ns the customer be ng de eted and then ca s RemoveCurrent on the customer b nd ng source Th s nterna y de etes the Customer object from the st bound to the current gr d row, wh ch n turn removes the row from the gr d on the form It s mportant to perform these steps n th s order, because you w not be ab e to reference the de eted customer ent ty once t has been removed from the b nd ng source The code then attempts to ocate the change track ng entry for the de eted customer by ­query ng the context’s Entities co ect on as before If the user c cked De ete Customer mmed ate y after ­c ck ng New Customer, then the context has no change track ng nformat on for th s customer, and so no ­act on needs to be taken Otherw se, the method enters a oop that ca s DeleteObject on the c­ ustomer’s re ated OrderHeader ent t es, and then ca s DeleteObject for the parent Customer ent ty Th s nforms the c ent context to track the customer and ts orders as de eted The next three methods that fo ow hand e OrderHeader nserts, updates, and de etes act ons made by the user n the bottom gr d These methods mon tor for changes us ng the very same techn ques we just exp a ned, and keep the c ent context nformed of data that the user ed ts n the bottom gr d



Chapter 11  WCF Data Access Techno og es   537

F na y, the btnSave Click event hand er fires when the user c cks Save Th s method first ca s ­EndEdit on both gr ds to app y any pend ng changes (wh ch s mportant f the user c cks Save wh e n the m dd e of ed t ng a ce ; ca ng EndEdit forces the gr d’s CellValueChanged event to fire, whereas otherw se t wou d not) Then, SaveChanges s nvoked on the context to push a the ­c ent updates to the serv ce, just as we demonstrated w th the test c ent But not ce that th s code ­spec fies SaveChangesOptions.Batch, wh ch s required to support h erarch ca updates (that s, ­updates ­nvo v ng both parent and ch d ent t es) The SaveChangesOptions.Batch sett ng nstructs the WCF Data Serv ces c ent brary to batch up a the ent ty changes nto a s ng e request, rather than ­nd v dua requests as you saw w th the test c ent Th s means that on y one HTTP request w be ssued to WCF Data Serv ces no matter how many ent t es are nserted, updated, or de eted on the c ent To see everyth ng work, mon tor the HTTP traffic w th F dd er as you run the so ut on F rst, make sure that both F dd er and SQL Profi er are runn ng Then r ght-c ck DemoRestClientDataEntry n So ut on Exp orer, choose Set As Startup Project, and press F5 V sua Stud o bu ds the so ut on and aunches the form C ck the Load button Th s quer es the serv ce and then b nds the customer and order data ­returned by the serv ce to the user nterface, as shown n F gure 11-10

Figure 11-10  The c ent form popu ated w th data returned by WCF Data Serv ces.

Now ed t the data as fo ows ■

■ ■

Insert a new order for Andy Jacobs by se ect ng Andy’s row n the top gr d and c ck ng New Order Then enter any random text nto the ShipVia and OrderStatus co umns for the new order Update the ba ance for Cass e H cks by chang ng the va ue from 40 to 50 n the top gr d De ete customer Josh Barnh by se ect ng Josh’s row n the top gr d and c ck ng De ete ­Customer Reca from our exp anat on of the code that th s not on y marks Josh’s customer ent ty as de eted, but a the ch d orders as we

538  Part III  Applied SQL

You have just performed a m x of nsert, update, and de ete operat ons on the c ent Now c ck Save to push a those changes to the serv ce Sw tch over to F dd er to exam ne the HTTP conversat on that just took p ace You w see that two HTTP requests were made; the first one s the GET ssued by the LINQ to REST query when you first c cked the Load button The second one s a POST ssued by SaveChanges, and t conta ns a of the nd v dua ent ty changes batched up ns de a s ng e HTTP request, as shown n L st ng 11-9 Listing 11-9  A batch of c ent updates posted to WCF Data Serv ces as a s ng e request.

POST http://localhost.:1055/CustomerDataService.svc/$batch HTTP/1.1 User-Agent: Microsoft ADO.NET Data Services DataServiceVersion: 1.0;NetFx MaxDataServiceVersion: 2.0;NetFx Accept: application/atom+xml,application/xml Accept-Charset: UTF-8 Content-Type: multipart/mixed; boundary=batch_74726ae1-e572-4e88-a53fbeb080591cdd Host: localhost.:1055 Content-Length: 3509 Expect: 100-continue Connection: Keep-Alive --batch_74726ae1-e572-4e88-a53f-beb080591cdd Content-Type: multipart/mixed; boundary=changeset_d7d5bcea-d717-4c2e-813480e730c80738 --changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary POST http://localhost.:1055/CustomerDataService.svc/OrderHeaders HTTP/1.1 Content-ID: 25 Content-Type: application/atom+xml;type=entry Content-Length: 965





2012-02-02T19:43:04.539875Z

0001-01-01T00:00:00 2



Chapter 11  WCF Data Access Techno og es   539

0 Pending Regular Mail 0001-01-01T00:00:00

--changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary MERGE http://localhost.:1055/CustomerDataService.svc/Customers(3L) HTTP/1.1 Content-ID: 26 Content-Type: application/atom+xml;type=entry If-Match: W/"datetime'2012-02-02T14%3A21%3A07.89925'" Content-Length: 991





2012-02-02T19:43:04.539875Z http://localhost.:1055/CustomerDataService.svc/Customers(3L)

50.0000 2012-02-02T14:20:55.696125 3 Cassie Hicks 2012-02-02T14:21:07.89925

--changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary DELETE http://localhost.:1055/CustomerDataService.svc/OrderHeaders(8L) HTTP/1.1 Content-ID: 27 --changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary DELETE http://localhost.:1055/CustomerDataService.svc/OrderHeaders(9L) HTTP/1.1 Content-ID: 28

540  Part III  Applied SQL

--changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary DELETE http://localhost.:1055/CustomerDataService.svc/Customers(4L) HTTP/1.1 Content-ID: 29 If-Match: W/"datetime'2012-01-01T00%3A00%3A00'" --changeset_d7d5bcea-d717-4c2e-8134-80e730c80738---batch_74726ae1-e572-4e88-a53f-beb080591cdd--

If you break down th s request, you can see that t conta ns mu t p e parts, each of wh ch ­represents a s ng e ent ty change request The serv ce URI for the ma n request s term nated w th $batch, wh ch nforms the serv ce that t s actua y rece v ng a batch of requests, a packaged up ns de the ma n HTTP request A so not ce that the Content-Type header reads multipart/mixed w th a str ng defin ng the boundar es of the batch (th s str ng appears before the first request n the batch and aga n after the ast request n the batch) Each of the requests n the batch s dent fied w th a un que Content-ID header, and they are a hand ed the same as the nd v dua requests you saw n the test c ent The first s a POST to nsert the new order for Andy Jacobs That s then fo owed by the MERGE to update the ba ance for Cass e H cks F na y, three DELETE operat on requests are ssued to de ete Josh Barnh ; the first two de ete the customer’s re ated orders, and the th rd de etes the customer tse f Sw tch over now to SQL Profi er to see the database act v ty generated by these requests As shown n F gure 11-11, EF automat ca y generated INSERT and DELETE statements to update the ­OrderHeader tab e and stored procedure ca s to UpdateCustomer and DeleteCustomer that update the Customer tab e Th s s expected, aga n, because there are stored procedure mapp ngs n the EDM for the Customers ent ty set but not OrderHeaders A so not ce that a the updates are batched ns de a transact on, wh ch ensures that they a succeed or fa together

Figure 11-11  SQL Server Profi er output show ng the T SQL statements executed by the batched WCF Data

Serv ces request.



Chapter 11  WCF Data Access Techno og es   541

Now sw tch back to F dd er and exam ne the HTTP response returned to the c ent, as shown n L st ng 11-10 Listing 11-10  A batch of WCF Data Serv ces resu ts returned as a s ng e response.

HTTP/1.1 202 Accepted Server: ASP.NET Development Server/10.0.0.0 Date: Thu, 02 Feb 2012 19:43:04 GMT X-AspNet-Version: 4.0.30319 DataServiceVersion: 1.0; Content-Length: 2898 Cache-Control: no-cache Content-Type: multipart/mixed; boundary=batchresponse_f525ef65-e8fb-4cd6-810840e1212fe166 Connection: Close --batchresponse_f525ef65-e8fb-4cd6-8108-40e1212fe166 Content-Type: multipart/mixed; boundary=changesetresponse_0eb669ef-4471-477785a6-8c0ed3993cd4 --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 201 Created Content-ID: 25 Cache-Control: no-cache DataServiceVersion: 1.0; Content-Type: application/atom+xml;charset=utf-8 Location: http://localhost.:1055/CustomerDataService.svc/OrderHeaders(33L)

http://localhost.:1055/CustomerDataService.svc/OrderHeaders(33L)

2012-02-02T19:43:04Z





33 2 Regular Mail Pending

542  Part III  Applied SQL

0001-01-01T00:00:00 0001-01-01T00:00:00

--changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 26 Cache-Control: no-cache DataServiceVersion: 1.0; ETag: W/"datetime'2012-02-02T14%3A43%3A04.9305'" --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 27 Cache-Control: no-cache DataServiceVersion: 1.0; --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 28 Cache-Control: no-cache DataServiceVersion: 1.0; --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 29 Cache-Control: no-cache DataServiceVersion: 1.0; --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4---batchresponse_f525ef65-e8fb-4cd6-8108-40e1212fe166--

You can see that the response s a so batched Not ce that nd v dua responses ns de the batch are each tagged w th Content-ID va ues so that the WCF Data Serv ces c ent brary can corre ate each ­response that comes back w th the or g na request that went out As exp a ned for the test c ent, each nd v dua response’s content s used to refresh the c ent-s de nstances of each updated ent ty



Chapter 11  WCF Data Access Techno og es   543

Extending WCF Data Services We have conducted a fa r y deta ed study of WCF Data Serv ces St , the examp es thus far have been based on very s mp e c ent-s de quer es and stra ghtforward updates A though OData URI quer es can be qu te flex b e, they do not support the more advanced LINQ query operat ons ­Therefore, some LINQ to REST quer es s mp y cannot be converted nto an equ va ent OData URI str ng ­Furthermore, you may need to supp ement the defau t query/update funct ona ty prov ded by the serv ce w th your own custom bus ness og c that fi ters or va dates data Fortunate y, WCF Data Serv ces has severa extens b ty features ava ab e that a ow your serv ces to overcome the m tat ons of OData URI quer es and prov de much more funct ona ty than stra ghtforward CRUD operat ons We conc ude our WCF Data Serv ces coverage by d scuss ng custom serv ce operat ons, serv ce method overr des, and query nterceptors

Creating Custom Service Operations It s very easy to expose your own custom methods as add t ona serv ce operat ons S mp y define any pub c method that returns IQueryable (where T s any ent ty type n your EDM), and decorate the method w th the WebGet attr bute Then add one ne of code to the InitializeService method to expose the method as a serv ce operat on You can use th s techn que to mp ement quer es on the serv ce that are d fficu t or mposs b e to express on the c ent us ng LINQ to REST and an OData URI It s a so a great way to extend the serv ce w th custom bus ness og c To demonstrate, return to the CustomerDataService.cs fi e n the DemoRestService project (L st ng 11-2) Doub e-c ck the fi e n So ut on Exp orer and add the code shown n L st ng 11-11, just beneath the ­InitializeService method Listing 11-11  Add ng a custom operat on to WCF Data Serv ces.

[WebGet] public IQueryable GetCustomersByBalance(decimal minBalance) { return from customer in base.CurrentDataSource.Customers where customer.Balance > minBalance select customer; }

Then add th s one ne to the InitializeService method (you can add t anywhere ns de the method) config.SetServiceOperationAccessRule("GetCustomersByBalance", ServiceOperationRights.All);

Now rebu d the so ut on and WCF Data Serv ces w serv ce operat on

mmed ate y expose the method as a custom

Th s s mp e examp e may not do much, but t does ustrate a the essent a s po nts The WebGet attr bute exposes the pub c method GetCustomersByLastName as a serv ce operat on The serv ce operat on accepts a dec ma parameter, wh ch t uses n the where c ause of a LINQ query—on y 544  Part III  Applied SQL

th s s a true LINQ to Ent t es query, us ng Ent ty Framework on the server The LINQ query obta ns access to the ObjectContext v a the CurrentDataSource property defined n the serv ce’s base c ass, DataService Here s the OData URI syntax for ca ng th s serv ce operat on that returns those customers whose ba ance exceeds 15 (the M suffix n 15M represents the money data type n SQL Server, correspond ng to decimal n NET) http://localhost.:1055/CustomerDataService.svc/GetCustomersByBalance()?minBalance=15M

Any t me you mod fy your serv ce, you need to update the serv ce reference on the c ent to regenerate the proxy c asses So you wou d be correct to assume that your next step s to do just that, but unfortunate y updat ng the serv ce reference w not expose GetCustomersByBalance on the c ent s de (as t shou d) Th s s due to a m tat on n V sua Stud o’s WCF Data Serv ces c ent proxy generator too that fa s to detect custom serv ce operat ons ( t on y recogn zes ent ty sets exposed by the InitializeService method) The workaround s to manua y extend the automat ca y generated c ent-s de context c ass ­SampleDbEntities w th a pub c method that wraps the serv ce operat on Essent a y, th s means ­add ng the code that the proxy generator shou d have added for you The generated ­SampleDbEntities context c ass s marked partial, mak ng t very easy for you to fi n the m ss ng ­funct ona ty by add ng the wrapper method Try th s out w th the ear er DemoRestClientTest project by add ng the code shown n L st ng 11-12 ­ nywhere to the project You can s mp y append the code to the bottom of the form’s code you added a from L st ng 11-3 A ternat ve y, you can create a new c ass fi e for custom proxy wrapper extens ons and add the code there; t rea y doesn’t matter Just be sure you don’t add t d rect y to the generated proxy code, as that code gets overwr tten whenever you update the serv ce reference Listing 11-12  Extend ng the c ent context c ass w th a custom serv ce operat on wrapper method.

namespace DemoRestClientTest.CustomerDataService { using System.Data.Services.Client; using System.Collections.Generic; public partial class SampleDbEntities { public DataServiceQuery GetCustomersByBalance(decimal minBalance) { return base .CreateQuery("GetCustomersByBalance") .AddQueryOption("minBalance", string.Format("{0}M", minBalance)); } } }



Chapter 11  WCF Data Access Techno og es   545

Th s code extends the c ent context c ass by g v ng t a GetCustomersByBalance method The method accepts a m n mum ba ance parameter, and nterna y ca s the base c ass CreateQuery and AddQueryOption methods to produce a DataServiceQuery object Embedd ng th s code nto the context tse f makes t comp ete y reusab e, so th s p umb ng og c does not need to be dup cated anywhere e se The context c ass now makes the GetCustomersByBalance method as easy to nvoke as LINQ to REST quer es aga nst ord nary ent ty sets For examp e, th s ne of code ca s the serv ce operat on to get the customers w th a ba ance exceed ng 15 and popu ates a st w th the resu ts var list = ctx.GetCustomersByBalance(15).ToList();

You can eas y test th s out as fo ows Create another button on the form and add th s ne of code to the button’s Click event hand er Set a breakpo nt on the ne, r ght-c ck ­DemoRestClientTest n So ut on Exp orer, choose Set As Startup Project, and press F5 Then c ck the button to h t the breakpo nt, and s ng e-step over the ne of code so you can mon tor the act v ty w th F dd er and SQL Profi er to confirm the resu ts returned n list

Overriding Service Methods The serv ce’s base c ass, DataService, has two v rtua protected methods that you can overr de (ak n to, but not exact y the same as, an event) to nject custom og c that executes on each request or whenever an except on occurs Respect ve y, these are named ­OnStartProcessingRequest and HandleException, and each of them accepts an arguments parameter w th nformat on you can use n hand ng these “events ” Both of them shou d nvoke the base c ass mp ementat on before add ng extended funct ona ty L st ng 11-13 be ow demonstrates how to mp ement these method overr des Listing 11-13  Overr d ng the OnStartProcessingRequest and HandleException base c ass methods.

protected override void OnStartProcessingRequest(ProcessRequestArgs args) { base.OnStartProcessingRequest(args); // add code here... } protected override void HandleException(HandleExceptionArgs args) { base.HandleException(args); // add code here... }

The OnStartProcessingRequest overr de executes w th each ncom ng c ent request In th s method, you can exam ne the ProcessRequestArgs parameter to d scover and act upon a host of nformat on, such as the request URI, method, and headers, as we as the response headers, and an IsBatchRequest flag nd cat ng that th s s a batch (mu t -part) request

546  Part III  Applied SQL

The HandleException overr de effect ve y serves as a g oba “catch b ock” for the serv ce The HandleExceptionArgs parameter has an Exception property that hands you the unhand ed except on object The arguments parameter a so exposes a UseVerboseErrors property that contro s whether deta ed nformat on about the except on s returned to the c ent, p us severa usefu read-on y response-re ated propert es (ResponseContentType, ResponseStatusCode, and ResponseWritten)

Writing Interceptors F na y, you can wr te spec a methods to ntercept each query or change request at the ent ty set eve Th s prov des you w th a great dea of contro over serv ce behav or There are two types of nterceptors Query nterceptors a ow you to fi ter data be ng returned out of an ent ty set that s sat sfy ng any query n the serv ce Change nterceptors a ow you to exam ne each ent ty change just before t occurs, so you can take spec a act on f des red Both types of nterceptors operate at the ent ty set eve At runt me, the framework uses ­Reflect on to d scover nterceptors n the serv ce by ocat ng methods decorated w th the QueryInterceptor or ChangeInterceptor attr bute The ent ty set name s spec fied as a str ng parameter to the method ­attr bute, whereas the method name tse f s ns gn ficant Interceptor methods must be coded w th the correct s gnatures; query nterceptors are expected to return a ambda that fi ters the ent ty set, wh e change nterceptors are expected to accept two parameters conta n ng nformat on about the ent ty be ng changed, as L st ng 11-14 demonstrates Listing 11-14  mp ement ng a query nterceptor and change nterceptor for the Customers ent ty set.

[QueryInterceptor("Customers")] public Expression OnQueryCustomers() { return c => c.Balance > 15; } [ChangeInterceptor("Customers")] public void OnChangeCustomers(Customer customer, UpdateOperations op) { if (op == UpdateOperations.Delete) { throw new DataServiceException("Customers cannot be deleted"); } }

The first method dec ares a query nterceptor for the Customers ent ty As such, t returns a ambda express on that w be used to fi ter customer data returned for a quer es Th nk of t as a g oba “where c ause” for the Customers ent ty set A ambda express on s mere y a method po nter, wh ch n the case of Func refers to a method that accepts a Customer object and returns a bool resu t In th s examp e, the ambda returns true f the customer ba ance s greater than 15, effect ve y hardcod ng a fi ter that exc udes a customers w th a ba ance of 15 and ower from every



Chapter 11  WCF Data Access Techno og es   547

query that gets serv ced Note that to use the actua return data type Expression, you must add the fo ow ng using statement at the top of the code using System.Linq.Expressions;

The second method mp ements a change nterceptor, a so for the Customers ent ty Change ­nterceptors return no data, and must accept two parameters The first s an object of the ent ty set type (wh ch s Customer n th s examp e), and the second s an UpdateOperations enumerat on You can nspect the ent ty be ng updated as we as the type of update be ng attempted, and take any custom act on requ red by your serv ce Th s prov des you w th a much finer degree of contro over secur ty than the g oba read-on y versus wr tab e contro prov ded by the InitializeService method In th s examp e, the serv ce s mp y does not perm t customers to be de eted WCF Data Serv ces supports authent cat on, a though t does not mp ement authent cat on on ts own Instead, t re es on authent cat on opt ons supported by the host (typ ca y IIS), ­nc ud ng ­anonymous, bas c, d gest, W ndows, ASP NET forms, and c a ms-based authent cat on W th ­authent cat on enab ed, you can perform user-based secur ty checks n your change nterceptors, as we as user-based fi ter ng n your query nterceptors For examp e, f your WCF Data Serv ces are hosted as an ASP NET Web App cat on ( ke those n th s chapter are), you can retr eve the ­pr nc pa (secur ty nformat on) of the request by exam n ng the HttpContext.Current.User property Then you can everage that nformat on (for examp e, user name, or user ID) to mp ement user-spec fic ­nterceptors n your WCF Data Serv ces

WCF RIA Services WCF RIA Serv ces s, we , richer than WCF Data Serv ces (and a so newer) Indeed, the R n RIA means r ch, a though the fu TLA (Three-Letter Acronym) can stand for R ch Internet App cat on or R ch Interactive App cat on—depend ng on who you’re ta k ng to S nce ts ear est days, WCF RIA Serv ces was des gned to work best w th S ver ght, a though t now a so supports OData, SOAP, and JSON to reach a w der range of c ents You can bu d WCF RIA Serv ces over any data access ayer, ­nc ud ng Ent ty Framework, LINQ to SQL, or P a n O d CLR Objects (POCOs), n wh ch case you hand e the ­pers stence yourse f us ng any data access techn que you want, nc ud ng convent ona ADO NET Both WCF Data Serv ces and WCF RIA Serv ces so ve many of the same prob ems, so t s on y natura to quest on wh ch one to use The answer extends a b t beyond the standard “ t depends on your scenar o” response, s nce WCF RIA Serv ces offers a ot more than just data access funct ona ty It a so features c ent-s de se f-track ng ent t es, c ent-s de va dat on, automat c server-to-c ent code generat on, and more A fu treatment of WCF RIA Serv ces s we beyond the scope of th s sect on, where our object ve s to demonstrate how WCF RIA Serv ces fac tates data access for funct ona ty s m ar to the WCF Data Serv ces so ut on n the prev ous sect on After work ng through a WCF RIA Serv ces vers on of the same so ut on, the chapter conc udes w th a s de-by-s de compar son of the two frameworks Th s approach w g ve you a c ear understand ng of when t s better to use one over the other (or perhaps, when t m ght be best to use ne ther)

548  Part III  Applied SQL

Establishing a WCF RIA Services Link When WCF RIA Serv ces s used w th S ver ght, V sua Stud o prov des a spec a nk between your c ent and serv ce projects L ke a serv ce reference, th s nk b nds the two projects together, on y a WCF RIA Serv ces nk coup es them much more t ght y than an ord nary serv ce reference does Pub c changes on the serv ce s de are reflected automat ca y n correspond ng c asses on the c ent s de every t me you perform a bu d—you never need to worry about work ng aga nst an outdated proxy n the c ent project s mp y because you forgot to manua y update a serv ce reference You can create a V sua Stud o so ut on w th a S ver ght c ent project and a WCF RIA Serv ces project a ready nked to each other n a s ng e step You w do that r ght now to bu d the demo

Important  The complete WCF RIA Services framework is distributed as two separate ­downloads that you must install on top of Visual Studio 2010. Before you can follow along with the samp e application, download and install WCF RIA Services SP2 for Silverlight 4 and 5 from http://www.microsoft.com/download/en/details.aspx?id 28357, and then go to http://www.microsoft.com/download/en/details.aspx?id 26939 for the WCF RIA Services Toolkit. You may also need the latest Silverlight Developer Runtime, which you can download from http://go.microsoft.com/fwlink/?LinkId 146060. If you have a non-developer version of Silverlight installed, be sure to uninstall it before attempting these installations. Start V sua Stud o and choose F e New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose S ver ght Then se ect the S ver ght App cat on temp ate to create a ­project named DemoSL n a so ut on named DemoWcfRiaServices, as shown n F gure 11-12

Figure 11-12  Creat ng a S ver ght App cat on project.



Chapter 11  WCF Data Access Techno og es   549

After you c ck OK, V sua Stud o d sp ays the New S ver ght App cat on d a og Check the Enab e WCF RIA Serv ces opt on at the bottom of the d a og as shown n F gure 11-13, and c ck OK

Figure 11-13  Check ng the Enab e WCF R A Serv ces opt on to nk the c ent and serv ce projects.

When you c ck OK, V sua Stud o creates a so ut on w th two projects One s the S ver ght c ent ­ roject, and s named DemoSL just as you spec fied n the New Project d a og The other s the serv ce p counterpart, wh ch s an ASP NET Web App cat on Project named DemoSL.Web, as spec fied n the New S ver ght App cat on d a og Ord nary S ver ght so ut ons a so beg n at th s start ng po nt, but because you enab ed Enab e WCF RIA Serv ces, V sua Stud o a so performs two mportant add t ona steps F rst, t adds a number of spec a references to the S ver ght c ent project In add t on to the ghtwe ght WebRequest and WebResponse c asses prov ded by System.Net n ord nary S ver ght projects, V sua Stud o sets a reference to the core WCF assemb es System.Runtime.Serialization and System.ServiceModel, as we as the WCF RIA Serv ces c ent-s de runt me assemb es System.ServiceModel.DomainServices.Client, System.ServiceModel.DomainServices.Client.Web, and System.ServiceModel.Web.Extensions As suggested by these names, WCF RIA Serv ces s a about bu d ng domain services, and these assemb es support the framework’s r ch c ent-s de exper ence by consum ng those doma n serv ces Second, t estab shes the WCF RIA Serv ces nk that b nds the two projects together Forget s­ erv ce references; there are none The WCF RIA Serv ces nk prov des far better synchron zat on than an ord nary WCF serv ce reference (or a WCF Data Serv ces reference) W th the nk estab shed, V sua Stud o cont nuous y regenerates the c ent-s de prox es to match the doma n serv ces, every t me you bu d your so ut on It auto-generates c ent-s de cop es of shared app cat on og c you define n the serv ces project, s mp y by ook ng for c asses you’ve defined n fi es named * shared cs, or * shared vb (rather than s mp y cs or vb) The nk a so enforces automat c c ent-s de va dat on and keeps va dat on ru es n sync between the doma n serv ces and the c ent at a t mes The WCF RIA Serv ces nk great y s mp fies the n-t er pattern, and makes trad t ona n-t er deve opment fee more ke the c ent/server exper ence After V sua Stud o creates the so ut on, r ght-c ck DemoSL (the c ent project) n So ut on Exp orer and choose Propert es The WCF RIA Serv ces nk s shown n the drop-down st on the bottom of the 550  Part III  Applied SQL

c ent project’s S ver ght propert es page, and the add t ona referenced assemb es are v s b e n the References node n So ut on Exp orer, as shown n F gure 11-14

Figure 11-14  A S ver ght so ut on w th a WCF R A Serv ces nk.

Creating the Entity Data Model The next step s to create the EDM, just as you d d for the WCF Data Serv ces project n the prev ous ­sect on You can e ther repeat the same steps to bu d the EDM from scratch as exp a ned n Chapter 10, or copy the SampleEF.edmx fi e (and the connectionStrings sect on n Web.config) from the WCF Data Serv ces project as fo ows Open W ndows Exp orer to the WCF Data Serv ces project, drag the SampleEF.edmx fi e, and drop t on the DemoSL.Web project n So ut on Exp orer Then copy the connectionStrings sect on n the Web.config fi e from WCF Data Serv ces project and paste t nto th s DemoSL.Web project’s Web.config fi e (just after the open ng tag toward the top)





Chapter 11  WCF Data Access Techno og es   551

Tip  This is the second time that we have instructed you to copy the EDM from one project to another. For your production applications, it is best practice to maintain a single v­ ersion of the EDM in a class library and compile it into an assembly. Then all you need to do is ­reference the assembly and add the connection string to the configuration file in all the projects that need to share the EDM. Bu d the so ut on (press Ctrl+Shift+B) before proceed ng If you don’t bu d the so ut on now, then the EDM w not be recogn zed when you attempt to bu d doma n serv ces c asses for t n the next step

Building the Domain Service and Metadata Classes R ght-c ck the DemoSL.Web project, choose Add New Item, and se ect Web from the st of temp ate categor es on the eft Then scro down the st of nsta ed temp ates and choose Doma n Serv ce C ass Name the fi e SampleRiaService.cs and then c ck Add, as shown n F gure 11-15

Figure 11-15  Creat ng a WCF R A Serv ces doma n serv ce c ass.

When you c ck Add, V sua Stud o d sp ays the Add New Doma n Serv ce C ass d a og, shown n F gure 11-16 The purpose of th s d a og s to generate temp ate code that he ps you get started wr t ng the ­ oma n serv ce c ass and assoc ated metadata c asses for ent t es n your EDM But we w take a d s ght y d fferent approach, and have you wr te the code for these c asses from scratch nstead Th s method w he p you better understand what these c asses are a about So don’t check off any ­ent t es n th s d a og, and just c ck OK

552  Part III  Applied SQL

Figure 11-16  The Add New Doma n Serv ce C ass d a og.

Because you d d not se ect any ent t es, V sua Stud o generates an empty c ass fi e So why go through th s process nstead of s mp y add ng an empty c ass fi e? The answer s that the r­ equ red WCF RIA Serv ces assemb y references were not automat ca y added to the serv ce project when the so ut on was created, as they were for the c ent project V sua Stud o (for some reason) ­defers ­automat ca y sett ng the references n the serv ce project unt the very first t me you use th s ­d a og—and t w set those references at that t me even f you don’t se ect any ent t es Now that the references are set, you can create doma n serv ce c asses and assoc ated metadata c asses ­s mp y by add ng ord nary c asses to your serv ce project (or you cou d have a so manua y added the ­references, but gett ng V sua Stud o to add them for you s a neat tt e t me-sav ng tr ck) The on y reason to use th s d a og aga n n the future s f you want to get a head start on wr t ng the CRUD code n your doma n serv ces You w now p ug n the code for the both the doma n serv ce c ass and the metadata c asses Then we w exp a n the mean ng of these c asses, and wa k through the code n deta Rep ace the ­generated empty SampleRiaService c ass w th the code shown n L st ng 11-15 Listing 11-15  The WCF R A Serv ces doma n serv ce.

using using using using



System; System.Data; System.Linq; System.ServiceModel.DomainServices.EntityFramework;

Chapter 11  WCF Data Access Techno og es   553

using System.ServiceModel.DomainServices.Hosting; using System.ServiceModel.DomainServices.Server; using System.Transactions; namespace DemoSL.Web { [EnableClientAccess()] public class SampleRiaService : LinqToEntitiesDomainService { public IQueryable GetCustomers() { var q = from cust in base.ObjectContext.Customers.Include("OrderHeaders") orderby cust.LastName, cust.FirstName select cust; return q; } public void InsertCustomer(Customer addedCustomer) { base.ObjectContext.Customers.AddObject(addedCustomer); } public void UpdateCustomer(Customer customer) { var originalCustomer = base.ChangeSet.GetOriginal(customer); if (originalCustomer == null) { base.ObjectContext.Customers.Attach(customer); } else { base.ObjectContext.Customers.AttachAsModified(customer, originalCustomer); } var orderChanges = this.ChangeSet.GetAssociatedChanges (customer, o => o.OrderHeaders); foreach (OrderHeader order in orderChanges) { var changeOperation = this.ChangeSet.GetChangeOperation(order); switch (changeOperation) { case ChangeOperation.Insert: base.ObjectContext.ObjectStateManager.ChangeObjectState (order, EntityState.Added); break; case ChangeOperation.Update: base.ObjectContext.OrderHeaders.AttachAsModified

554  Part III  Applied SQL

(order, base.ChangeSet.GetOriginal(order)); break; case ChangeOperation.Delete: base.ObjectContext.OrderHeaders.Attach(order); base.ObjectContext.DeleteObject(order); break; } } } public void DeleteCustomer(Customer removedCustomer) { base.ObjectContext.Customers.Attach(removedCustomer); base.ObjectContext.DeleteObject(removedCustomer); var orderChanges = this.ChangeSet.GetAssociatedChanges (removedCustomer, o => o.OrderHeaders); foreach (OrderHeader order in orderChanges) { base.ObjectContext.OrderHeaders.Attach(order); base.ObjectContext.DeleteObject(order); } } public override bool Submit(ChangeSet changeSet) { var result = false; using (var ts = new TransactionScope()) { var commit = true; result = base.Submit(changeSet); if (this.ChangeSet.HasError) { commit = false; } // Other out-of-band updates run in the same transaction scope if (commit) { ts.Complete(); } } return result; } } }



Chapter 11  WCF Data Access Techno og es   555

Not ce the TransactionScope object used n the Submit method Us ng TransactionScope enab es your serv ce to have other (non-doma n serv ce) updates part c pate n the doma n serv ce update The ­TransactionScope object s prov ded by the transact on management API n the System.Transactions assemb y, wh ch you need to reference n order for th s code to comp e R ght-c ck the DemoSL.Web project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, scro to find the System.Transactions component, and doub e-c ck t Next, r ght-c ck the DemoSL.Web project aga n, and choose Add C ass Name the new c ass fi e SampleRiaService.metadata.cs and c ck Add Then rep ace the empty c ass w th the code shown n L st ng 11-16 Listing 11-16  The WCF R A Serv ces metadata c asses.

using using using using

System; System.ComponentModel.DataAnnotations; System.Data.Objects.DataClasses; System.ServiceModel.DomainServices.Server;

namespace DemoSL.Web { [MetadataType(typeof(Customer.CustomerMetadata))] public partial class Customer { internal sealed class CustomerMetadata { private CustomerMetadata() { } [Composition] [Include] public EntityCollection OrderHeaders { get; set; } [RoundtripOriginal] public DateTime UpdatedAt { get; set; } } } [MetadataType(typeof(OrderHeader.OrderHeaderMetadata))] public partial class OrderHeader { internal sealed class OrderHeaderMetadata { private OrderHeaderMetadata() { } [RoundtripOriginal] public DateTime UpdatedAt { get; set; } } } }

w

Let’s exp a n what th s doma n serv ce c ass does and the purpose of these metadata c asses We start w th the metadata c asses first, and then c rc e back around to the doma n serv ce c ass

556  Part III  Applied SQL

The Metadata Classes Metadata s “data about data,” mean ng that metadata c asses conta n spec fic nstruct ons and h nts for the WCF RIA Serv ces framework to recogn ze about your data source You don’t actua y code any og c n metadata c asses Instead, the framework ooks for metadata by exam n ng your assemb y (us ng Reflect on) for types w th spec a attr butes that you are expected to supp y to better descr be your data source The framework searches first for a MetadataType attr bute on each pub c ent ty c ass exposed by the data source—wh ch s the EDM n your current DemoSL.Web project The pub c ent ty c asses are generated automat ca y by a custom too n V sua Stud o whenever the EDM changes, so add ng the MetadataType attr bute d rect y to those c asses s not a v ab e opt on (the attr bute wou d get ost the next t me the c asses are overwr tten by the EDM des gner) Instead, you create a separate code fi e and everage the part a c ass feature to add the MetadataType attr bute The auto-generated ent ty c asses are a defined w th the partial keyword by des gn, so that you can eas y and safe y extend them n th s manner L st ng 11-16 demonstrates the techn que Th s code extends the des gnergenerated Customer and OrderHeader ent ty c asses by defin ng part a c asses w th the same names and n the same namespace as those n the EDM, and decorat ng the c asses w th the MetadataType attr bute At bu d t me, the comp er merges th s code w th the auto-generated code to produce a s ng e og ca ent ty c ass The MetadataType attr bute takes a s ng e type parameter that spec fies the metadata c ass for the ent ty That s, the MetadataType attr bute tse f s mere y a po nter that te s the framework wh ch c ass conta ns the actua metadata These c asses can be defined anywhere, but to keep code neat y organ zed, they are defined as nested c asses r ght ns de of the ent ty c asses themse ves For ­examp e, the Customer c ass has a MetadataType attr bute that po nts to the metadata c ass ­ Customer.CustomerMetadata, wh ch s nested ns de the Customer c ass In L st ng 11-16, not ce how both metadata c asses Customer.CustomerMetadata and OrderHeader.OrderHeaderMetadata mp ement a pr vate constructor that does noth ng Th s suppresses the defau t pub c constructor, wh ch prevents the metadata c asses from ever be ng nstant ated Aga n, no og c goes nto these c asses, so there s never a reason to nstant ate them They just need to be defined, so that they can be reflected Once t finds the actua metadata c ass, the c ass s exam ned for attr butes that attach spec a mean ng and behav ors to nd v dua propert es of the ent ty That s, for each pub c property defined n the data source’s ent ty c ass, the framework ooks for correspond ng propert es w th the same name and type n the metadata c ass The propert es have no og c n the r get and set b ocks— nstead, they are dec ared on y so that they can be dec ared w th attr butes These attr butes are very often re ated to va dat on For examp e, there s a Required attr bute that des gnates a property as mandatory, and there s a StringLength attr bute to constra n a str ng property w th a max mum ength Va dat on ru es bubb e up to the c ent auto-mag ca y, and are enforced by databound contro s n the S ver ght app cat on w thout any cod ng effort on your part ­ efin te A though WCF RIA Serv ces supports data sources other than Ent ty Framework, there are d advantages to us ng an EDM as the data source One of them s that you don’t need to dec are ­va dat on attr butes n metadata c asses for every s ng e ent ty property—they surface ­automat ca y from the metadata n the EDM tse f Mean ng for examp e, propert es des gnated as requ red or

Chapter 11  WCF Data Access Techno og es   557

­ av ng a max mum ength n the EDM w be recogn zed as such by the S ver ght c ent automat h ca y, w thout defin ng Required and StringLength attr butes n the metadata c asses Th s avo ds dup cat ng va dat on ru es between the EDM and the metadata c asses, wh ch s why you see so few propert es n the Customer.CustomerMetadata and OrderHeader.OrderHeaderMetadata c asses The on y propert es that you need to define are those you want to decorate w th attr butes that attach spec a mean ng to them (beyond va dat on) The metadata c asses n L st ng 11-16 demonstrate the Composition, Include, and RoundtripOriginal attr butes The Composition attr bute s app ed to the OrderHeaders property n the Customer.­CustomerMetadata c ass OrderHeaders s a nav gat on property; t “nav gates” from a parent Customer object to ts ­re ated co ect on of OrderHeader objects When the WCF RIA Serv ces framework sees Composition on a ­nav gat on property, t treats the parent and ch d objects as part of the same “compos t on”—two parts of a arger ent ty to be treated as a who e Compos t ons can nc ude add t ona re ated ent t es s mp y by mark ng nav gat on propert es w th the Composition attr bute n each parent’s metadata c ass Even w th th s attr bute, however, ch d ent t es w not be retr eved automat ca y w th parent ent t es un ess you a so spec fy the Include attr bute You w therefore typ ca y a ways spec fy Include w th Composition, as shown for the OrderHeader property n L st ng 11-16 On the c ent, a compos t on s cons dered changed whenever any of ts part c pat ng ent t es are changed, and the compos t on as a who e s then transm tted back to the serv ce for updat ng In the current demo, th s means that the “customer” s cons dered changed f any order data changes— even f the Customer ent ty tse f was not mod fied—and that the ent re compos t on ( nc ud ng the unmod fied customer) s sent to the serv ce for updat ng A s ng e doma n serv ce method s then respons b e for updat ng the parent ent ty and a of ts re ated ch d ent t es There are obv ous advantages and d sadvantageous to compos t ons, so be jud c ous w th the r use There s certa n y a conven ence factor n pass ng a the ent t es one at a t me, but you a so ncur arger data transfers by pass ng nd v dua ent t es that have no changes Next, not ce the RoundtripOriginal attr bute on the UpdatedAt propert es n both the Customer.CustomerMetadata and OrderHeader.OrderHeaderMetadata c asses Th s attr bute fac tates the opt m st c concurrency checks that re y on the or g na UpdatedAt va ue to detect mu t -user confl cts Reca from Chapter 10 that th s funct ona ty s mp emented for the Customer ent ty n the EDM by the UpdateCustomer stored procedure

Note  Although we did not walk you through the same process for the OrderHeader ­entity, you would similarly implement an UpdateOrderHeader stored procedure and map it in the EDM if this were a production application. The EDM, as it is currently configured, ­generates direct DML statements to update the OrderHeader table rather than using stored ­procedures, and makes no use of the original UpdatedAt value. You can preserve and return the or g na va ue of any property s mp y by mark ng t w th ­ oundtripOriginal, wh ch s he pfu for more than just concurrency checks Th s capab ty can R ­e m nate the need for bus ness ru es to re oad data before app y ng an update Imag ne that the ­Balance property was a so marked w th RoundtripOriginal Upon update, the doma n serv ce c ass 558  Part III  Applied SQL

wou d have access to the or g na va ue so that t cou d ensure, just for examp e, that the new b ­ a ance does not exceed the prev ous ba ance by 10 percent W thout the or g na va ue round-tr pped, the ­update method n the doma n serv ce c ass wou d be forced to go back to the database for the ­or g na ba ance n order to test for the same cond t on

The Domain Service Class Now we can sh ft focus to the doma n serv ce c ass n L st ng 11-15, SampleRiaService Th s c ass has a the pub c methods for query ng and updat ng the data source, and can be extended w th any other custom operat ons you may w sh to expose to the c ent The c ass s decorated w th the ­EnableClientAccess attr bute, wh ch s what causes the c ent-s de code for the c ass to be generated automat ca y by the WCF RIA Serv ces nk each t me you bu d the so ut on The SampleRiaService c ass nher ts from LinqToEntitiesDomainService, mean ng that t uses Ent ty Framework as the data source The T spec fies the EDM’s object context c ass, SampleDbEntities A WCF RIA Serv ces doma n serv ce c ass can a so nher t from LinqToSqlDomainService to expose LINQ to SQL mode s, or nher t from DomainService to expose POCOs (ord nary bus ness objects based on any data access ayer) The first method s mp y retr eves a customers and the r orders Not ce that no attr bute s requ red on the method; s mp y defin ng pub c methods that return IQueryable exposes those methods as query operat ons that can be ca ed by the c ent The other three methods work to nsert, update, and de ete Customer ent t es rece ved back from the c ent, and because of the Composition attr bute n the Customer.CustomerMetadata c ass, they hand e ch d OrderHeader ent t es as we W thout the Composition attr bute n the metadata c ass, another three methods wou d be needed n the doma n serv ce c ass to separate y nsert, update, and de ete OrderHeader ent t es When the c ent subm ts changes back to the serv ce, a new EF object context for the EDM s nstant ated beh nd the scenes on the server Th s context s ma nta ned across ca s that the WCF RIA Serv ces runt me makes to the set of nsert, update, and de ete methods that you must supp y n the doma n serv ce c ass These methods are respons b e for attach ng ncom ng objects to the context, wh ch the base c ass exposes through ts ObjectContext property These methods a so set the proper state nformat on ( nserted, updated, or de eted) for the objects t attaches to the context, so EF b ­ ehaves as though th s context or g na y retr eved the data and has been track ng ts changes a a ong The InsertCustomer method s s mp est It has on y one ne of code n t that ca s the AddObject method on the new Customers ent ty W th compos t on, a new “customer” s stra ghtforward t’s a ways go ng to nvo ve a new parent Customer ent ty and a co ect on of new ch d OrderHeader ­ent t es So add ng the parent ent ty automat ca y adds the ch d ent t es and the object context tracks the ent re compos t on as new The UpdateCustomer method s a b t more nvo ved Th s one method must account for both ­ ustomer and OrderHeader ent t es separate y, and nstruct the object context to track the r C changes accord ng y The Customer ent ty must be attached first, but remember that t may or may not ­actua y be mod fied as part of the overa compos t on If on y order data changed, then the

Chapter 11  WCF Data Access Techno og es   559

­ ustomer ent ty needs to be attached by ca ng Attach; otherw se, t needs to be attached by ca ng C AttachAsModified The GetOriginal method s ca ed on the ChangeSet property exposed by the base c ass to make th s determ nat on Th s method returns null f there s no or g na vers on of the ent ty n the changeset, wh ch means that the Customer was not mod fied, and Attach s ca ed ­Otherw se, AttachAsModified s ca ed to nform the context that the Customer was mod fied, and what ts ­or g na va ues were The Customer object returned by GetOriginal conta ns or g na va ues on y for those propert es marked w th RoundtripOriginal n the metadata c ass (such as UpdatedAt); a other propert es of the “or g na ” customer are n t a zed to defau t va ues (nu s and zeros) Next, GetAssociatedChanges s ca ed on the ChangeSet to return the co ect on of OrderHeader changes For each e ement n the co ect on, GetChangeOperation s ca ed to obta n a Change­ Operation enumerat on va ue that te s you f the OrderHeader was nserted, updated, or de eted For an nsert, the ChangeObjectState method s nvoked on the context’s ObjectStateManager to attach the ch d ent ty as added For an update, AttachAsModified s ca ed to attach the ch d ent ty w th ts or g na va ues, just as AttachAsModified was ca ed to attach the parent Customer ent ty when t was detected as mod fied F na y, ch d de et ons work by first ca ng Attach on the OrderHeaders ent ty set, and then ca ng DeleteObject on the ObjectContext The DeleteCustomer method performs a stra ghtforward de ete of the ent re compos t on—parent ent ty, and a re ated ch d ent t es As you saw w th de eted ch d ent t es n UpdateCustomer, ent t es are de eted by first ca ng Attach and then ca ng DeleteObject After de et ng the parent Customer ent ty, GetAssociatedChanges s ca ed to return the co ect on of ch d OrderHeader ent t es There s no need to eva uate the ChangeOperation of each ent ty; t s a ways go ng to nd cate a de et on Therefore, each OrderHeader ent ty s s mp y de eted by ca ng Attach and then DeleteObject F na y, the doma n serv ce base c ass a so exposes a Submit method that you can overr de to ­ntercept that po nt n t me just before a the changes are wr tten back to the database Ins de th s method, you can add any custom process ng og c Th nk of the Submit method as the doma n ­serv ce’s “g oba update tr gger” that you can everage to nspect, adjust, or reject pend ng data changes, or take any other des red act on before comm tt ng changes to the database As shown n the Submit method overr de n L st ng 11-15, you can a so start a T ­ ransactionScope b ock and then ca other methods to perform separate, out-of-band updates W th n the ­TransactionScope b ock, ca ng the base Submit method saves the changes from the doma n serv ce update request as part of a broader database transact on that the other updates contr bute to In th s manner, a of the updates e ther succeed or fa as a who e

More Info  Chapter 10 explains how to use TransactionScope in detail, and Chapter 4 is dedicated to the topic of database transactions.

560  Part III  Applied SQL

Building the Silverlight Client W th the doma n serv ce c ass and metadata c asses n p ace n the DemoSL.Web serv ce project, you are ready to create the S ver ght user nterface n the DemoSL c ent project Th s user nterface sha support s m ar ad-hoc data entry capab t es as the W ndows Forms c ent you created w th WCF Data Serv ces n the prev ous sect on As w th the W ndows Forms c ent, th s user nterface (UI) w nc ude gr d ­contro s for customer and order data But V sua Stud o does not automat ca y br ng n a of the supported ­S ver ght contro s ( nc ud ng the DataGr d contro ) when you create your project So before you can use the D ­ ataGr d to bu d the UI, you must add a reference to System.Windows.Controls.Data R ght-c ck the ­DemoSL project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, and scro to find and se ect the System.Windows.Controls.Data component, as shown n F gure 11-17 Then c ck OK

Figure 11-17  Sett ng a reference to System.Windows.Controls.Data prov des a DataGr d contro for S ver ght.

Now open MainPage.xaml, and rep ace the markup w th the code shown n L st ng 11-17 Listing 11-17  The S ver ght c ent XAML.





Chapter 11  WCF Data Access Techno og es   561











The des gn v ew of your S ver ght user contro shou d ook s m ar to F gure 11-18

Figure 11-18  The WCF R A Serv ces data entry c ent user nterface.

Your ast step s to add the code-beh nd for the user contro R ght-c ck anywhere on the des gn ­surface and choose V ew Code Then rep ace the empty MainPage c ass w th the code shown n L st ng 11-18

562  Part III  Applied SQL

Listing 11-18  Code for the WCF R A Serv ces data entry c ent S ver ght form.

using using using using using

System; System.ServiceModel.DomainServices.Client; System.Text; System.Windows; System.Windows.Controls;

using DemoSL.Web; namespace DemoSL { public partial class MainPage : UserControl { private SampleRiaContext _ctx = new SampleRiaContext(); public MainPage() { InitializeComponent(); } private void btnLoad_Click(object sender, RoutedEventArgs e) { this.btnLoad.IsEnabled = false; var lo = this._ctx.Load( this._ctx.GetCustomersQuery(), this.LoadOperationComplete, null); this.grdCustomers.ItemsSource = lo.Entities; } private void LoadOperationComplete(LoadOperation lo) { if (lo.HasError) { MessageBox.Show(lo.Error.Message, "Load failed", MessageBoxButton.OK); lo.MarkErrorAsHandled(); } this.btnLoad.IsEnabled = true; } private void btnSave_Click(object sender, RoutedEventArgs e) { this.btnSave.IsEnabled = false; this._ctx.SubmitChanges(this.SaveOperationCompleted, null); } private void SaveOperationCompleted(SubmitOperation so) { this.btnSave.IsEnabled = true; if (!so.HasError)



Chapter 11  WCF Data Access Techno og es   563

{ return; } var sb = new StringBuilder(); sb.AppendLine(so.Error.Message); foreach (var badEntity in so.EntitiesInError) { sb.AppendFormat(string.Format("Entity {0} Error", badEntity. GetIdentity())); sb.AppendLine(); foreach (var ve in badEntity.ValidationErrors) { sb.AppendLine(" " + ve.ErrorMessage); } } MessageBox.Show(sb.ToString(), "Submit failed", MessageBoxButton.OK); so.MarkErrorAsHandled(); } private void grdCustomers_SelectionChanged (object sender, SelectionChangedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer; if (cust != null) { this.grdOrders.ItemsSource = cust.OrderHeaders; } } private void btnNewCustomer_Click(object sender, RoutedEventArgs e) { var cust = new Customer(); this._ctx.Customers.Add(cust); this.grdCustomers.ItemsSource = this._ctx.Customers; } private void btnDeleteCustomer_Click(object sender, RoutedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer; if (cust != null) { this._ctx.Customers.Remove(cust); this.grdCustomers.ItemsSource = this._ctx.Customers; } } private void btnNewOrder_Click(object sender, RoutedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer;

564  Part III  Applied SQL

if (cust != null) { var order = new OrderHeader(); cust.OrderHeaders.Add(order); this.grdOrders.ItemsSource = cust.OrderHeaders; } } private void btnDeleteOrder_Click(object sender, RoutedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer; if (cust != null) { var order = new OrderHeader(); cust.OrderHeaders.Remove(order); this.grdOrders.ItemsSource = cust.OrderHeaders; } } } }

Th s comp etes the ent re so ut on Go ahead and bu d t, but don’t run t just yet We’ first exp a n the c ent code you just added, and then you can run the app cat on to g ve t a test-dr ve The c ent S ver ght project knows a about the pub c serv ces and ent t es defined n the serv ce project W thout estab sh ng a serv ce reference, how does t obta n th s know edge? The answer s that V sua Stud o automat ca y generates c ent prox es every t me you bu d the so ut on Th s s tr ggered by the WCF RIA Serv ces nk that ex sts between the two projects The code s wr tten to a source fi e and added to the Generated Code fo der n your S ver ght project These tems are ­h dden by defau t, but you can v ew them by se ect ng the DemoSL project and then c ck ng Show A F es (the second too bar button at the top of So ut on Exp orer), as shown n F gure 11-19

Figure 11-19  The h dden generated code fi e conta n ng WCF R A Serv ces c ent prox es.



Chapter 11  WCF Data Access Techno og es   565

In add t on to the generated proxy code n DemoSL.Web.g.cs, the WCF RIA Serv ces nk ­ utomat ca y cop es any code fi e n the serv ce project w th a name end ng n shared cs (or a shared vb) nto the ­Generated Code fo der For examp e, you cou d define a serv ce project c ass n a fi e named CommonHelpers.shared.cs, and that code fi e wou d be nstant y cop ed and updated to the c ent project for use there as we Th s automated shar ng of metadata and og c between the c ent and serv ce projects s what the WCF RIA Serv ces nk s a about Exam n ng the proxy code n DemoSL.Web.g.cs s very en ghten ng It w he p you better ­understand the WCF RIA Serv ces framework and the c ent code n L st ng 11-18 Take some t me now to exp ore t, and you w find severa nterest ng c asses n there As you probab y expect, there are Customer and ­OrderHeader c asses that correspond to the same-named ent t es exposed by the serv ce’s data source These c ent-s de c asses were generated because of the MetadataType attr bute added to the ent ty ­metadata c asses n the serv ce project The most mportant generated c ass s SampleRiaContext, wh ch nher ts from DomainContext Th s s a statefu c ent context object that fac tates data access (quer es and updates) w th the serv ce, much the way the SampleDbEntities c ent context was used n the WCF Data Serv ces ­vers on of th s so ut on ear er n the chapter However, there s an mportant d fference here that makes ­c ent ­deve opment much eas er w th WCF RIA Serv ces than WCF Data Serv ces You do not need to ­manua y not fy the SampleRiaContext object of every change to every ent ty, as you are requ red to do w th the WCF Data Serv ces c ent brary These Customer and OrderHeader c asses are comp ete y se f-track ng ent t es Because these ent t es track the r own changes, your c ent code does not need to be concerned at a w th change track ng—you just query the context, and then ssue an update to t whenever you want to save changes back to the serv ce At the top of the MainPage c ass n L st ng 11-18, a new SampleRiaContext object s dec ared and nstant ated n ctx Th s creates the statefu DomainContext for the UI when the MainPage c ass s constructed, wh ch s access b e by a the code n the c ass v a the ctx var ab e If you compare L st ng 11-18 w th the equ va ent c ent code for the WCF Data Serv ces vers on of th s so ut on n L st ng 11-8, you can see there s s gn ficant y ess code and comp ex ty requ red, thanks to the se f-track ng capab t es of WCF RIA Serv ces c ent-s de ent t es At the same t me, th s S ver ght code works asynchronous y, wh ch ntroduces a new cha enge The W ndows Forms c ents you bu t for the WCF Data Serv ces ear er n the chapter made synchronous ca s, wh ch b ocks ­c ent execut on (and the UI) wh e awa t ng a response from the serv ce Th s s mp fies cod ng, but s ­express y proh b ted n S ver ght and W ndows Phone 7 app cat ons These c ent p atforms requ re the UI to rema n respons ve at a t mes, and thus, they on y perm t asynchronous (non-b ock ng) ­serv ce ca s Fortunate y, the necessary ca back mechan sm s re at ve y easy to mp ement, as we exp a n next Four methods mp ement the data access; the first two hand e oad ng and the other two hand e sav ng Every asynchronous operat on a ways nvo ves a pa r of methods; the first method nvokes the operat on and the second method (the ca back) hand es the response Code that fo ows the nvocat on n the first method cont nues to execute mmed ate y after ssu ng the serv ce ca , ­before the response has come back for t Th s keeps the UI respons ve, but a so means that you ­cannot know f or when the serv ce ca comp etes successfu y The second method gets ca ed ­automat ca y when 566  Part III  Applied SQL

the serv ce comp etes, and rece ves nformat on that you use to determ ne the success or fa ure of the operat on The btnLoad Click event hand er fires when the user c cks Load, and mmed ate y d sab es the Load button Th s prevents the user from c ck ng Load aga n, because the UI w et them do so wh e the first request s st process ng It then nvokes the Load method on the context to nvoke the query request asynchronous y var lo = this._ctx.Load( this._ctx.GetCustomersQuery(), this.LoadOperationComplete, null);

The first parameter passed to Load spec fies the method to be ca ed n the doma n serv ce c ass Every query method ava ab e n the serv ce has a correspond ng context method that dent fies t on the c ent Th s c ent-s de method s named the same as the method n the doma n serv ce c ass, fo owed by Query Thus, the first parameter to Load ca s the context’s GetCustomersQuery method, wh ch references the GetCustomers method n the doma n serv ce The next Load parameter spec fies the ca back; that s, the method to be ca ed when the serv ce method comp etes Th s method s expected to match a part cu ar s gnature; spec fica y, t can be any method that returns void and accepts a LoadOperation object, where T s the ent ty type returned by the serv ce (Customer, n th s case) The LoadOperationComplete method uses th s exact s gnature, wh ch a ows t to be used as the ca back to rece ve contro when the serv ce operat on comp etes The ne of code fo ow ng the Load method ca

s most nterest ng

this.grdCustomers.ItemsSource = lo.Entities;

Th s code b nds the ItemsSource property of the customer’s gr d to the Entities property of the LoadOperation object returned by the Load method In trad t ona asynchronous ­programm ng, data b nd ng does not occur unt the ca back method rece ves the serv ce response But th s ne of code executes r ght after nvok ng the asynchronous ca , wh ch s before the resu ts have been returned to the c ent At the t me the ItemsSource property s set on the customer’s gr d, the c ent s aware of the schema of the data that t expects to rece ve, but t hasn’t actua y rece ved the data tse f yet Instead, the gr d w auto-mag ca y d sp ay the data as soon as t arr ves Th s ­behav or s ustrated when you run the app cat on; you w not ce that the co umn headers of the gr d appear the nstant Load s c cked, whereas the rows are popu ated w th data a moment ater The LoadOperationComplete method has the requ red s gnature expected for the ca back; t r­ eturns void and accepts a LoadOperation. B nd ng s a ready taken care of by the ­framework as just exp a ned, so the pr mary purpose of the ca back s to re-enab e the Load button and check f an error occurred dur ng the oad operat on The HasError property returns true f there was an error, and the Error property exposes the Exception object that was thrown on the serv ce Because the UI hand es the error, t a so ca s MarkErrorAsHandled on the LoadOperation object If th s method s not ca ed, the framework re-throws the except on at runt me to make sure the prob em does not s p by unnot ced



Chapter 11  WCF Data Access Techno og es   567

The asynchronous pattern of the two save methods works exact y the same The btnSave Click event hand er fires when the user c cks Save It d sab es the Save button (prevent ng the user from c ck ng Save aga n wh e a prev ous request s process ng), and then nvokes the SubmitChanges method on the context to k ck off the asynchronous save request this._ctx.SubmitChanges(this.SaveOperationCompleted, null);

The c ent does not ca the nd v dua pub c methods for nsert, update, and de ete defined n doma n serv ce d rect y As exp a ned when d scuss ng the doma n serv ce c ass mp ementat on, the WCF RIA Serv ces framework manages a server-s de EF object context and orchestrates the ca s to the doma n serv ce methods nterna y as appropr ate A you need to do s ca SubmitChanges The SubmitChanges method takes a parameter that spec fies the ca back, wh ch s expected to be any method that returns void and accepts a SubmitOperation object The SaveOperationComplete method uses th s exact s gnature, and s spec fied as the ca back to rece ve contro when the serv ce operat on comp etes The SaveOperationComplete ca back method re-enab es the Save button and checks for ­errors that m ght have occurred dur ng the save operat on Just ke the LoadOperation object, there s a HasError property to te you f there was an error, and an Error property ho d ng the Exception thrown on the serv ce The SubmitOperation object’s EntitiesInError property exposes a co ect on that reports the spec fic deta s of each ent ty that has va dat on prob ems If an error s detected, these deta s are comb ned us ng a StringBuilder and d sp ayed to the user F na y, MarkErrorAsHandled s gna s that the error was hand ed, wh ch prevents an except on from be ng thrown

More Info  We presented each operation as two distinct methods to help you grasp the idea of asynchronous programming. In practice, you will find it common instead to e ­ mbed the callback as an anonymous method inside of the calling method using a Lambda. You will learn the Lambda technique in Chapter 13, when you build a Windows Phone 7 ­application that calls WCF Data Services hosted in Windows Azure. The same a ­ synchronous programming concepts apply whether you are “inlining” the callback method using a Lambda or coding a separate callback method, as shown in Listing 11-18. The rest of the c ent code s re at ve y s mp e In grdCustomers SelectionChanged, the bottom gr d s bound to the OrderHeaders property to show the orders for customers se ected n the top gr d The four rema n ng event hand ers respond to New Customer, De ete Customer, New Order, and De ete Order button c cks The btnNewCustomer Click and btnDeleteCustomer Click methods respect ve y add and remove Customer ent t es to and from the context’s Customers co ect on And the btnNewOrder Click and btnDeleteOrder Click methods respect ve y add and remove OrderHeader ent t es to and from the r parent Customer object’s OrderHeaders co ect on None of the e aborate change track ng code (see L st ng 11-8) that was needed w th the WCF Data Serv ces c ent brary s requ red here, because a of the ent t es track the r own changes When the user ed ts data n the UI, the se f-track ng ent t es bound to the UI automat ca y nform the context about the change, so you don’t have to 568  Part III  Applied SQL

Inspecting the .NET Framing Protocol with Fiddler The so ut on s ready to run, and as before, you w want to mon tor background act v ty w th ­F dd er and SQL Profi er But un ess you first nsta a spec a F dd er “ nspector,” you won’t be ab e to read the raw HTTP requests n c ear text as you d d w th the REST/OData packets n the ear er WCF Data Serv ces so ut on Th s s because, by defau t, WCF RIA Serv ces uses the NET Fram ng protoco wh ch ­comb nes XML and b nary e ements Us ng F dd er’s “raw” nspector, the b nary e ements are ­unreadab e ­Fortunate y, Joe Zhou of the WCF RIA Serv ces team at M crosoft has wr tten a spec a ­nspector that dec phers the b nary e ements so that you can v ew the traffic as raw (readab e) XML text Insta ng the nspector s a s mp e matter of copy ng BinaryMessageFiddlerExtension.dll nto the I­nspectors fo der beneath F dd er’s program d rectory Th s s typ ca y C:\Program Files (x86)\Fiddler2\Inspectors, on 64-b t mach nes You can get the d fi e by down oad ng BinaryMessageFiddlerExtension.zip from http://archive.msdn.microsoft.com/Project/Download/FileDownload.aspx?ProjectName=silverlightws&Down loadId=12007 The z p fi e nc udes a comp ete V sua Stud o so ut on, but you on y need to copy the BinaryMessageFiddlerExtension.dll fi e from the b n\debug fo der We a so supp y the d fi e n the L b fo der of the so ut on for th s chapter’s code, wh ch you can down oad from the book’s compan on webs te (see the ”Introduct on” for deta s) Once the d fi e s cop ed nto ts Inspectors fo der, F dd er d sp ays a new WCF S ver ght opt on n the request and response pane s of the Inspectors tab You w use th s opt on to mon tor NET Fram ng requests and responses that are passed over HTTP by th s WCF RIA Serv ces so ut on

Testing the Complete WCF RIA Services Solution Make sure you have nsta ed the NET Fram ng nspector for F dd er, and that both F dd er and SQL Profi er are runn ng before runn ng the so ut on Then press F5 to bu d and run the so ut on The browser aunches to the URL for the ma n page, wh ch s runn ng off of oca host However, F dd er won’t mon tor traffic as you use the app cat on because there s no “dot” n the URL So use the same tr ck exp a ned ear er, and add a (dot) mmed ate y after localhost n the browser’s address bar and press Enter to re oad the page Now F dd er w recogn ze the HTTP traffic n the app cat on, and d sp ay t for your nspect on C ck the Load button If you watch very c ose y as you do, you w not ce that co umn headers appear nstant y on the customer gr d, and then rows of data appear a br ef moment ater Th s s because of the asynchronous datab nd ng behav or n L st ng 11-18 that we exp a ned ear er C ck on the first customer (Cass e H cks) n the top gr d to d sp ay the customer orders n the bottom gr d, as shown n L st ng 11-20 Now ed t the data to exerc se the app cat on as you d d w th the W ndows Forms c ent n the WCF Data Serv ces so ut on F rst nsert a new order for Andy Jacobs by se ect ng Andy’s row n the top gr d and c ck ng New Order Then enter any random text nto the ShipVia and OrderStatus co umns for the new ­order As you enter the new order row, you mmed ate y exper ence the auto-mag ca va dat on features of the WCF RIA Serv ces framework W thout wr t ng any code, the gr d s aware of requ red fie ds As shown n F gure 11-21, the gr d d sp ays error messages as the user enters data, and then c ears those ­messages as the user sat sfies the va dat on ru es Where d d the ru es come from? They bubb ed

Chapter 11  WCF Data Access Techno og es   569

up from the ­metadata n the serv ce project and surfaced automat ca y n the c ent project And as exp a ned ­ear er, the m ­ etadata der ves va dat on ru es from the conceptua mode n the under y ng EDM ­automat ca y as we

Figure 11-20  The c ent form popu ated w th data returned by WCF R A Serv ces.

Figure 11-21  The c ent form popu ated w th data returned by WCF R A Serv ces.

570  Part III  Applied SQL

After supp y ng va ues for OrderStatus and ShipVia, the error messages d sappear from the gr d and the new order for Andy s accepted Now update the ba ance for Cass e H cks by chang ng the va ue from 50 to 60 n the top gr d F na y, de ete customer Lukas Ke er (and a h s orders) by ­se ect ng h s row n the top gr d and c ck ng De ete Customer Now c ck Save to push a the changes back to the serv ce, and sw tch over to F dd er to v ew the HTTP traffic that just transp red On the eft, c ck the ast HTTP request The raw v ew n the Inspectors tab shows that a POST was ssued to a SubmitChanges operat on, but the content s not readab e Not ce the Content-Type ­header that reads application/msbin1, wh ch refers to the NET Fram ng protoco Now sw tch away from raw v ew by c ck ng on the WCF S ver ght opt ons at the top of the request and response ­pane s The NET Fram ng protoco nspector you added to F dd er now presents the content n a perfect y readab e format L st ng 11-19 shows the SubmitChanges request ( rre evant port ons of the request are om tted to conserve space) Listing 11-19  The WCF R A Serv ces SubmitChanges request.



0001-01-01T00:00:00 2

0 Pending Priority Mail 0001-01-01T00:00:00

false 0 Insert

:

OrderHeaders

8 9 10 11 12



Chapter 11  WCF Data Access Techno og es   571



60 2012-02-10T11:55:39.667375 3 Cassie Hicks 2012-02-10T12:00:23.261125

true 2 Update

OrderHeaders

8 9 10 11 12



0 0001-01-01T00:00:00 3

2012-02-10T12:00:23.261125



35.0000 2012-02-10T11:55:39.667375 1 Lukas Keller 2012-02-10T11:55:39.667375

572  Part III  Applied SQL

false 3 Delete

OrderHeaders

4 5



2012-02-10T11:55:39.667375 0

1 Shipped Regular Mail 2012-02-10T11:55:39.667375

true 4 Delete

0001-01-01T00:00:00 1

1

2012-02-10T11:55:39.667375



2012-02-10T11:55:39.667375 0

2 Pending Express Mail



Chapter 11  WCF Data Access Techno og es   573

2012-02-10T11:55:39.667375

true 5 Delete

0001-01-01T00:00:00 1

2

2012-02-10T11:55:39.667375

:

You can see that each mod fied Customer and OrderHeader ent ty n the compos t on s represent n the request w th an e ement that nd cates Insert, Update, or Delete In the case of the Update, not ce that the request nc udes an e ement that supp es the ­or g na va ues of the customer ent ty be ng updated If you ook at the or g na customer ent ty, you w not ce that t on y conta ns the or g na va ue for the UpdatedAt property Th s s because that s the on y property des gnated w th the RoundtripOriginal attr bute n the metadata; a the other (non-key) propert es are n t a zed to the defau t va ues of the r respect ve data types Now sw tch over to Profi er to see the database act v ty, as shown n F gure 11-22

Figure 11-22  SQL Server Profi er output show ng the T SQL statements executed by the WCF R A Serv ces update

request.

574  Part III  Applied SQL

The appropr ate T-SQL commands were generated by EF based on the ent t es n the ­changeset r­ ece ved from the c ent that were attached to the ObjectContext by the doma n serv ce c ass ­methods The updates are wrapped up ns de of a database transact on because of the TransactionScope b ock n the Submit method overr de added to the doma n serv ce c ass (see L st ng 11-15) Now sw tch back to F dd er and exam ne the response returned to the c ent, shown n L st ng 11-20 You can see that WCF RIA Serv ces returns updated v ews of each ent ty back to the c ent In the case of the Insert operat on for Andy’s new order, the OrderHeaderId va ue 13 nd cates the pr mary key va ue ­ass gned by the SQL Server IDENITY co umn n the under y ng OrderHeader tab e Listing 11-20  The WCF R A Serv ces SubmitChangesResponse.

:

0001-01-01T00:00:00 2

13 Pending Priority Mail 0001-01-01T00:00:00

false 0 Insert

60 2012-02-10T11:55:39.667375 3 Cassie Hicks 2012-02-10T18:18:39.2757928

true 2 Update

:



Chapter 11  WCF Data Access Techno og es   575

35.0000 2012-02-10T11:55:39.667375 1 Lukas Keller 2012-02-10T11:55:39.667375

false 3 Delete

0001-01-01T00:00:00 1

1

2012-02-10T11:55:39.667375

true 4 Delete

0001-01-01T00:00:00 1

2

2012-02-10T11:55:39.667375

true 5 Delete



576  Part III  Applied SQL

Congratu at ons! You have earned how to bu d n-t er app cat ons us ng the atest and most ­mportant data techno og es ava ab e n the NET stack Ent ty Framework, WCF Data Serv ces, and WCF RIA Serv ces

Making the Right WCF Data Access Choice Our coverage n th s chapter has h gh ghted severa key d fferences between WCF Data Serv ces and WCF RIA Serv ces, and you have probab y a ready made some educated guesses about when to se ect one over the other To better gu de you n the dec s on-mak ng process, we conc ude the chapter w th a more focused compar son of these two frameworks Table 11-1  Compar ng WCF Data Serv ces and WCF RIA Serv ces WCF Data Services

WCF RIA Services

Supported C ents

Resource based AP , supports a c ents v a deep REST and OData support.

Doma n based AP , most ta ored for use w th S ver ght, but supports other c ents v a SOAP, JSON, and OData support.

Supported Data Access Layers

Targets EF. Other DALs are supported, but greater effort s requ red.

Supports EF, L NQ to SQL, and POCO (custom pers stence ayer).

C ent Deve opment

Requ res you to not fy the context for change track ng.

Supports se f track ng ent t es, ­synchron zed c ent/server og c, and much more ­(part cu ar y w th S ver ght).

Serv ce Deve opment

nstant, code ess, extens b e REST serv ces out of the box (w th EF); “free CRUD.”

Requ res you to code CRUD operat ons manua y n doma n serv ce c asses.

Severa casua observat ons can be made from the compar son n Tab e 11-1 WCF RIA Serv ces s more attract ve for S ver ght c ents than non-S ver ght c ents—regard ess of wh ch data access ayer s used And WCF Data Serv ces s more appropr ate for use w th Ent ty Framework than t s w th other data access ayers—regard ess of wh ch c ent s used If your scenar o uses EF on the back end and targets S ver ght on the front end, then you are n the best pos t on Both WCF frameworks pack a huge w n over wr t ng trad t ona WCF serv ces “by hand ” Your dec s on at th s po nt s based on whether you s mp y requ re serv ces to prov de data ­access (that s, you pr mar y need CRUD support), or f you are seek ng to everage add t ona ­benefits Another cons derat on s whether you are target ng S ver ght as the c ent exclusively or not WCF Data Serv ces s re at ve y ghtwe ght, and requ res a most no effort to get up and runn ng w th t So t’s the better cho ce f you on y need data access funct ona ty n your serv ces, ­part cu ar y f you want to keep your serv ce open to non-S ver ght c ents as we WCF RIA Serv ces s more ­robust, and offers numerous add t ona features Th s makes t a very compe ng cho ce for the ­deve opment of r ch c ent app cat ons A though t began as a p atform a most exc us ve y des gned for S ver ght, support s stead y emerg ng for other c ent p atforms v a OData, SOAP, and JSON, as we as se f-track ng ent ty brar es now ava ab e for JavaScr pt and jQuery Converse y, t st requ res the add t ona effort to create doma n serv ce c asses



Chapter 11  WCF Data Access Techno og es   577

F na y, both frameworks are extens b e, and both can be secured by trad t ona authent cat on and author zat on techn ques They are a so both capab e of ntegrat ng w th the ASP NET Membersh p prov der for ro e management and persona zat on What f you are us ng ne ther S ver ght nor Ent ty Framework? We , then your work w be cut out for you whatever cho ce you make W th WCF Data Serv ces, you w need to mp ement e ther the Reflect on or Stream ng prov der, or wr te your own custom prov der And w th WCF RIA Serv ces, you w not get to fu y enjoy a the benefits of the framework, p us you w st need to wr te doma n serv ces and metadata c asses After carefu cons derat on, you may we conc ude that ne ther cho ce s appropr ate, and dec de nstead to st ck w th tr ed and true WCF serv ces, cod ng your own serv ce contracts, data contracts, b nd ng configurat ons, change track ng, va dat ons, and so on

Summary Th s chapter bu t on the Ent ty Framework coverage of our prev ous chapter, and taught you how to expose your Ent ty Data Mode (EDM) as serv ces us ng two flex b e W ndows Commun cat on ­Foundat on techno og es WCF Data Serv ces and WCF RIA Serv ces We began by exp or ng the protoco s under y ng WCF Data Serv ces, nc ud ng Representat ona State Transfer (REST) and Open Data Protoco (OData) You earned about the OData URI query ­syntax, and how they expose resource-based serv ces over your EDM We a so descr bed many WCF Data Serv ces extens b ty features, such as custom serv ce operat ons, base c ass overr des, and nterceptors Then you earned how to use the WCF Data Serv ces c ent brary to bu d front-end ­app cat ons After wr t ng a s mp e test c ent, you bu t a comp ete W ndows Forms data entry c ent to consume the serv ce You got an up-c ose ook at the act on beh nd the scenes by mon tor ng HTTP traffic w th F dd er and observ ng database act v ty w th SQL Profi er Then you bu t a s m ar so ut on us ng WCF RIA Serv ces and S ver ght You saw how the spec a nk between c ent and server projects n your V sua Stud o so ut on enab es r ch c ent ­deve opment aga nst doma n serv ces Then you earned how to create a doma n serv ce c ass to support ­quer es and updates, as we as metadata c asses for your ent t es to custom ze serv ce behav or After ­comp et ng the serv ce, you bu t the S ver ght front end us ng the c ent-s de context and ­se f-track ng ent t es automat ca y generated by the WCF RIA Serv ces nk F na y, we conc uded by focus ng on the s m ar t es and d fferences between WCF Data Serv ces and WCF RIA Serv ces, w th a ­compar son to he p you make the r ght cho ces n any g ven scenar o These past two chapters have covered a of the data access techn ques ava ab e n NET for c ent/server and n-t er deve opment Today, n-t er deve opment extends nto the c oud, and the next chapter s a about M crosoft’s c oud database, SQL Azure The chapter that fo ows comb nes techno og es n a three preced ng chapters, and adds synchron zat on and mob e deve opment In that chapter, you w bu d an end-to-end so ut on w th SQL Azure, SQL Azure Data Sync, WCF Data Serv ces hosted on W ndows Azure, and W ndows Phone 7 deve opment too s

578  Part III  Applied SQL

C hapter 1 2

Moving to the Cloud with SQL Azure —Andrew Brust

I

n techno ogy, th ngs typ ca y start out as exc us ve, expens ve, and comp ex, before ­eventua y ­becom ng ma nstream, affordab e, and s mp e As comput ng advances, barr ers to entry are ­owered, and that’s good for everyone Th s has happened w th most techno og es, nc ud ng ­M crosoft SQL Server How so? A though t’s pretty easy to get SQL Server up and runn ng these days, t wasn’t a ways that way When SQL Server first came out for the W ndows operat ng system, t ran on y on W ndows NT—and on y on a server The very determ ned cou d use NT Server as a workstat on OS and thus have SQL Server runn ng on the r deve opment mach ne But for a ntents and purposes, us ng SQL Server requ red sett ng up a rea server, wh ch nh b ted the product’s adopt on by deve opers Today though, SQL Server runs fine on c ent operat ng systems, so deve op ng for SQL Server has far fewer obstac es than t once d d But what about production? What f you want to dep oy and ma nta n a SQL Server database for a substant a number of users? If you have an IT department ­support ng you, then you’re a set Ask to have a server prov s oned and managed, and away you go But f you don’t have an IT department, or you have one, but t’s overburdened or unab e to ass st you, then you’re on your own You can certa n y buy and set up a beefy PC, or even a server, nsta SQL Server on t and adm n ster t yourse f, and you cou d make that workab e for a wh e But are you ready to proper y manage network traffic, backups, warm spares, and the ke? Probab y not, wh ch means the do- t-yourse f approach s r sky, eav ng you and your users vu nerab e For examp e, f more users come on ne, you w have a b g prob em mak ng your one-person operat on sca e

The “c oud” can he p As a database deve oper, what wou d probab y su t you we wou d be a ­ osted database serv ce, where network operat ons, server management and configurat on, ­database h ­m rror ng/rep cat on, backups, and other nfrastructure-re ated tasks are taken care of for you ­Actua y, what wou d probab y work even better s an arrangement whereby you have to manage just the database (or a c­ o ect on of databases) Everyth ng e se—even a the adm n strat ve tasks ­assoc ated w th the ­database, other than des gn- eve opt m zat ons—wou d dea y be hand ed for you, just as f you had an IT d ­ epartment support ng you and even a part-t me database adm n strator (DBA)

579

Th s serv ce wou d be d fferent than a s mp e SQL Server offer ng from a host ng prov der And t wou d be d fferent from nfrastructure as a serv ce (IaaS) offer ngs, where a v rtua zed server s ­prov ded, but the nsta at on, care, and feed ng of SQL Server wou d st be your respons b ty The dea serv ce we have descr bed wou d be a c oud-based database p atform as a serv ce (PaaS) ­offer ng, where you wou d need on y to prov s on and operate a database; the rest wou d be taken care of by the serv ce SQL Azure is that dea database PaaS offer ng Even better, n funct ona terms, t s arge y equ va ent to the on-prem ses vers on of SQL Server For readers of th s book, that means the earn ng curve for sett ng up a pub c-fac ng re at ona database w th A- st Internet peer ng and nfrastructure management s very sha ow So sha ow, n fact, that we can cover what you need to know to get up and runn ng w th SQL Azure n th s one chapter a one! In th s chapter, you w ■ ■

■

■

■

earn

How to s gn up for a SQL Azure subscr pt on, then prov s on a SQL Azure server and database How to manage, des gn, and ma nta n your database us ng the W ndows Azure Management Porta , the SQL Azure Management Porta , SQL Server Management Stud o (SSMS), and SQL Server Deve oper Too s (SSDT) n V sua Stud o How to dep oy and m grate databases to, and between, SQL Server and SQL Azure us ng ­Data-T er App cat ons (DACPAC and BACPAC fi es) What SQL Azure Federat ons are, when to use them, and how to do so n the SQL Azure ­Management Porta and SSMS How to get up and runn ng w th SQL Azure Report ng, us ng the W ndows Azure Management Porta , Report ng Serv ces projects n V sua Stud o, and Report Bu der

In short, you’ earn the procedures and approaches un que to SQL Azure for gett ng up and r­ unn ng w th the product, and you w then see how you can keep go ng w th th ngs you a ready know

Important One big difference between on-premises software and cloud services is that the latter can be updated and enhanced much more frequently than the former, given that no installation on customer infrastructure is required in the cloud case. Cloud services are ­subject to frequent changes in pricing as well. As such, costs, limitations, features, the tooling user interface or even the branding of SQL Azure and/or SQL Azure Reporting, as described in this chapter, may have evolved by the time you read it. (For example, while this chapter was being written, SQL Azure’s pricing was reduced—by as much as 75% for certain configurations—and this chapter was updated to reflect the change.) Regardless of the potential for such changes, the principles and techniques covered here should help you achieve comfort with and mastery of SQL Azure. Just make sure to review online what features may have been added or limitations removed from SQL Azure and SQL Azure Reporting since the timeframe of SQL Server 2012’s release, so you can read this chapter in the proper context. 580  Part III  Applied SQL

History When we wrote the ast ed t on of th s book, we prov ded br ef coverage of a c oud serv ce from M crosoft ca ed SQL Server Data Serv ces (SSDS), wh ch was n beta A though that serv ce used SQL Server as ts nfrastructure, t presented an nterface to the deve oper that was comp ete y ­d fferent from the re at ona structure of SQL Server SSDS used a mode ca ed “ACE,” wh ch stood for ­Author ty-Conta ner-Ent ty, and was essent a y a key-va ue store NoSQL database (a though we d dn’t have the vocabu ary at the t me to ca t that) Subsequent to the beta re ease of SSDS, two th ngs happened (1) M crosoft aunched Azure Tab e Storage (among other serv ces), wh ch had a mode very s m ar to ACE, and (2) feedback came n from SQL Server customers that they wanted a true SQL Server re at ona database serv ce, and weren’t sat sfied w th the ACE mode M crosoft stened, d scont nued SSDS, and offered n ts p ace SQL Azure Database, n wh ch t offered a PaaS flavor of SQL Server that used T-SQL, worked w th SQL Server’s Tabu ar Data Stream (TDS) protoco , and was compat b e w th ex st ng SQL Server database dr vers and APIs That meant that NET deve opers, and espec a y Azure deve opers us ng NET, cou d access SQL Azure n the same way they d d on-prem ses SQL Server, from on-prem ses app cat ons, chang ng on y the r ­connect on str ng and avo d ng certa n SQL Server features that SQL Azure d d not support The pub c beta per od for SQL Azure Database ­began n 2009 and SQL Azure was re eased as a commerc a serv ce n 2010 SQL Azure started out w th a number of m tat ons, pr m t ve too ng, and a max mum database s ze of 10 GB, wh ch m ted ts use to on y certa n app cat on scenar os But the serv ce s now robust and feature-r ch, has much r cher too ng, offers a max mum database s ze of 150 GB, and supports Federat ons, a sca e-out mode wh ch a ows deve opers to transcend even that phys ca m tat on

But What Is SQL Azure? The best way to th nk of SQL Azure s as a bank of SQL Server boxes that someone e se s ­manag ng, and because someone e se s manag ng those boxes, you have to ab de by some mportant ­restr ct ons and requ rements For examp e ■ ■

■ ■



Every tab e must have a c ustered ndex The abso ute argest database you can have s 150 GB (but Federat ons offer a work-around for b gger databases) There s no fu -text search Wh e the XML data type s supported, there s no support for XML ndexes, XML schemas, or schema co ect ons

■

On y the SQL Server secur ty authent cat on mode s supported

■

There s no support for convent ona SQL Server part t on ng Chapter 12  Mov ng to the C oud w th SQL Azure    581

Note More comprehensive inventories of SQL Azure restrictions, relative to the feature set of the full SQL Server product, can be found at http://msdn.microsoft.com/en-us/library/ windowsazure/ff394115.aspx and http://msdn.microsoft.com/en-us/library/windowsazure/ ee336241.aspx. SQL Azure features on y the re at ona database component of SQL Server There s a separate, m ted mp ementat on of SQL Server Report ng Serv ces (wh ch we cover ater n th s chapter), but there s current y no c oud mp ementat on of Integrat on Serv ces, Ana ys s Serv ces, Master Data Serv ces, Data Qua ty Serv ces, StreamIns ght, or any other d screte component of the “greater” SQL Server fam y Keep ng the above restr ct ons and m tat ons n m nd summons an mportant paradox You can cons der SQL Azure to be just ke SQL Server, except that t’s somet mes d fferent Is that a tt e obtuse of us to say? Perhaps But the more you work w th SQL Azure, the more you w come to ­apprec ate th s observat on’s accuracy Meanwh e, here are a few th ngs the two M crosoft databases have n common ■ ■

■

■

Both can be managed from SSMS and SSDT Both can serve as data sources for Report ng Serv ces, Integrat on Serv ces, PowerP vot, or Ana ys s Serv ces The SQL Server Nat ve C ent dr ver and the SqlClient ADO NET prov der can connect to, query, and perform data man pu at on anguage (DML) operat ons on both And, a though the two code bases d ffer somewhat, SQL Azure is SQL Server

Why the Limitations? Let’s ta k about that ast po nt If SQL Azure s rea y just SQL Server, then why doesn’t t support ts fu feature set? The th ng to keep n m nd s that SQL Azure databases run on shared servers and yet are automat ca y rep cated and transparent y fau t-to erant As such, they can respons b y on y offer features that keep the serv ce sca ab e and the databases “fit and tr m” enough to be dup cated and ma nta ned very qu ck y That’s why the databases have s ze m tat ons, and that’s why ess sca ab e features, ke fu -text search, are not supported Other features are unsupported ess for reasons of sca ab ty and more because the serv ce s st evo v ng Not a SQL Server features have yet been fu y mp emented and/or tested n SQL Azure’s mu t -tenant env ronment But SQL Azure s be ng mproved a the t me, and by the t me you read th s, some of the restr ct ons enumerated here may no onger ex st

582  Part III  Applied SQL

Pricing C oud serv ces’ feature sets and pr c ng schemes tend to be refreshed at a greater frequency than are pr nted books That’s a ong-w nded way of say ng that any d scuss on of SQL Azure pr c ng n th s book s subject to fa ng out-of-date However, t’s hard to judge the efficacy of a c oud-based ­vers on of SQL Server w thout some understand ng of ts costs and b ng tenets So et’s proceed w th a d scuss on of SQL Azure’s pr c ng as t stands at press t me, and et’s treat th s d scuss on as a contextua too for understand ng when and where mp ementat on of SQL Azure m ght make sense Make sure to check on ne for up-to-date pr c ng nformat on and comb ne that w th the concepts prov ded n th s sect on to sketch out for yourse f the fu bus ness case for SQL Azure SQL Azure s ava ab e n two “ed t ons” the Web ed t on nc udes databases of up to 1 GB or up to 5 GB The Bus ness ed t on nc udes databases wh ch can be any of the fo ow ng max mum s zes 10 GB, 20 GB, 30 GB, 40 GB, 50 GB, 100 GB, or 150 GB Rates range from $10/GB/month for a 1 GB database to as tt e as $1 51/GB/month for a 150 GB database If abso ute cost s more mportant to you than per-GB rates, there s a so a 100 MB database opt on for $4 995/month Inbound data— nc ud ng data sent for INSERT, UPDATE, or subm tted v a Bu k Copy Program (BCP) operat ons— s not charged for Outbound data ( nc ud ng data quer ed or requested v a BCP ­operat ons) however, s b ab e, current y at $0 12/GB from North Amer can and European data ­centers, and $0 19/GB from As an data centers Data qua fies for egress ( e , outbound) charges not when t eaves SQL Azure per se, but when t eaves the data center and goes over the Internet ­Therefore, data quer ed by a W ndows Azure app cat on runn ng n the same data center as the SQL Azure server s not b ed (a though outbound data transfer charges from the W ndows Azure ­app cat on w app y) These rates are for “pay-as-you-go” serv ce, and are based on actua metered usage Th s s ­ spec a y effic ent n cases where you are uncerta n of how much or tt e capac ty you w ­rea y e need However, n cases where you have an awareness of certa n m n mum usage needs, and can comm t to those serv ce eve s for s x months, you can pay a subscr pt on rate for the serv ce as fo ows you purchase “un ts,” each one of wh ch covers $100 worth of month y SQL Azure serv ces (based on pay-as-you go rates) but costs on y $79 99/month Th s s a tt e b t ke buy ng m nutes on a mob e ca ng p an w thout a ro over mechan sm just as you m ght ose unused m nutes, you w ose any unused SQL Azure serv ces Therefore the un ts you purchase shou d be based on your m n ma needs Serv ce over and above what your subscr pt on covers w be b ed at the regu ar ­pay-as-you-go rates

The First One’s Free Un ke runn ng SQL Server on your deve opment mach ne, work ng w th SQL Azure requ res sett ng up an account, and pay ng for t That creates a barr er to entry that cou d d ssuade some deve opers from try ng the product M crosoft s aware of th s and, at press t me, s offer ng a so ut on free tr a subscr pt ons to W ndows Azure and SQL Azure



Chapter 12  Mov ng to the C oud w th SQL Azure    583

As ong as you fee comfortab e prov d ng a cred t card number, and you don’t need to exceed the a owances prov ded n the tr a , you’ be ab e to s gn up for SQL Azure mmed ate y and work w th t extens ve y, free-of-charge, for three months To g ve th s chapter some rea -wor d context, we’ d scuss how to s gn up for the tr a subscr pt on

Note  Even if the trial offer is withdrawn at some point, the end-to-end coverage in this c­ hapter should make things more concrete and make you more comfortable with the rather small ­investment necessary to work with SQL Azure at the regular rates (as little as $4.995/month for a 100 MB database) already discussed. If you are a V sua Stud o Profess ona , Prem um, or U t mate w th MSDN subscr ber, you are ­a ready ent t ed to a month y a owance of Azure serv ces, nc ud ng SQL Azure, at no add t ona charge A though the free tr a subscr pt on s ava ab e to you as we , you may prefer s mp y to act vate the Azure subscr pt on that s nc uded w th your MSDN subscr pt on A though the s gn-up process w d ffer somewhat from what tr a offer users w exper ence, everyth ng from prov s on ng of a server and beyond ( e , the vast major ty of what we cover n th s chapter) w a gn perfect y

Getting Set Up Now that we have a good overv ew of what SQL Azure s, what t nc udes, and what t om ts, and we have estab shed that you can work w th t for tt e or no money up-front, et’s ook at how to s gn up for a free tr a account To s gn up, you w need a W ndows L ve ID Most readers of th s book w a ready have one but f you don’t, or f you’d ke to create a spec a L ve ID for use w th the Azure tr a subscr pt on, then go to http://signup.live.com and create one For the purposes of th s chapter, we set up a spec a L ve ID under the ema address programm ngmssq 2012@ ve com The account was deact vated before th s book went to press, but we ment on th s so that you w recogn ze th s address as a L ve ID n some of the screenshots n th s chapter

Note If you have an Xbox Live account, a Zune account, a Hotmail account, a Windows Live Messenger account, or an activated Windows Phone, your login credentials for those ­services are in fact Live IDs and can be used to provision a Windows Azure trial account.

Tip You will have separate accounts/login credentials for your Live ID, your SQL Azure server, and your SQL Azure Reporting account. Make sure to record each set of credentials carefully, in a place that is secure yet accessible.

584  Part III  Applied SQL

Beyond the Prerequisites Once you have your L ve ID credent a s, perform the fo ow ng steps to prov s on your tr a account 1. Po nt your browser to http://www.windowsazure.com. 2. C ck the Free Tr a

nk on the upper-r ght hand of the page Th s w take you to a page w th, among other content, a descr pt on of the usage a owances prov ded w th the tr a

Info  As of this writing, the relevant allowance information for SQL Azure is that the trial includes a 1 GB Web edition SQL Azure database, unlimited inbound data ­transfer, and 20 GB of outbound data transfer, per month. 3. C ck the “Try It Free” button n the m dd e of the new page 4. Log n w th your L ve ID credent a s and then c ck Next (the r ght-arrow button) on the first

pop-up page 5. Enter a mob e phone number at wh ch to rece ve an act vat on code v a text message, and

c ck Send Text Message 6. Once you rece ve the text message, enter the act vat on code t conta ns and then c ck Ver fy

Code 7. Once the code s va dated, c ck Next (the r ght-arrow button) 8. Enter the requested cred t card, account, and b

ng nformat on Check or uncheck the ­acknow edgment and opt- n checkboxes at the bottom of the form and then c ck Next

9. After a br ef pause, your account w

be prov s oned and your browser w

be red rected to a

we come page

Info  Although sign-up for the trial subscription requires entry of a valid credit card number, you can still rest easy about unintended and unwanted charges. Trial ­accounts are configured by default to disable automatically for the remainder of any month where and if a monthly usage limit is reached, and to expire a ­ utomatically at the end of the three-month trial period. Should you reach your usage limit ­during month 1 or 2, although your account will be disabled, your database will not be ­deleted, and your account will automatically re-enable at the beginning of month 2 or 3, respectively. If you wish to remove this mechanism and keep your ­account ­active, you can disable the spending limit. If you do so, all usage beyond the ­monthly allowance in months 1–3 and any usage at all after month 3 will be billed to your credit card at the SQL Azure standard pay-as-you-go rates already described.



Chapter 12  Mov ng to the C oud w th SQL Azure    585

Provisioning Your Server Once your account s set up, c ck the Porta nk toward the upper-r ght of the new page ­Do ng so w br ng you nto the W ndows Azure Management Porta ( f you don’t have the atest ­vers on of S ver ght on your mach ne, you’ need to nsta t first) C ck the Database button n the ­ower- eft-hand Out ook-sty e nav gat on menu and you’ be taken to a screen where you can create SQL Azure servers and databases C ck the 3 Month Free Tr a node under the Subscr pt ons fo der, above the Out ook-sty e nav gat on bar Th s w enab e the Create button n the r bbon’s Server group, as shown n F gure 12-1

Figure 12-1  The Database page n W ndows Azure Management Porta .

C ck the Create button n the Server group of the r bbon (h gh ghted n F gure 12-1), and you’ aunch the Create Server w zard The w zard w first ask you to se ect a geograph c reg on Th s reg on—wh ch can be set to South Centra US, West US, East US, North Centra US, North Europe, West Europe, East As a, or Southeast As a—corresponds to the phys ca ocat on of the data center on whose ­nfrastructure your database w be ocated Th s cho ce shou d be made carefu y; there are a few reasons why (a coup e of wh ch we’ve a ready a uded to) ■

As a ru e, performance w customers

be better f you p ck a data center that s phys ca y c ose to your

■

There may be certa n regu atory cons derat ons mportant to you that mpact your se ect on

■

Data transfer fees for certa n data centers are s ght y h gher than for others

586  Part III  Applied SQL

■

If you w be pa r ng your SQL Azure database w th a W ndows Azure app cat on, or SQL Azure Report ng, you w want to ocate them n the same reg on/data center to avo d ­outgo ng data transfer fees from the database to the app or report You can on y assure that your W ndows Azure hosted serv ce and/or your SQL Azure Report ng server w be n the same reg on as your SQL Azure server f you keep track of what reg on you se ected for SQL Azure n the first p ace!

Note  At press time, SQL Azure Reporting services were not offered in the East US or West US data centers. Therefore, users wishing to locate both SQL Azure and SQL Azure Reporting services in a single US data center would be required to select South Central US or North Central U.S. as the location for their SQL Azure server.

Configuring the Administrative User and Firewall After se ect ng a reg on, c ck Next and you’ be asked to prov de a user name and password for the server-w de adm n strat ve user (s m ar to the sa user n SQL Server) Enter that nformat on and c ck Next Your next step w be to spec fy n t a firewa ru es By defau t, your SQL Azure server w b ock a network traffic If you want to use the server from your own deve opment mach ne, you’ need to unb ock ts IP address You’ probab y want to a ow access by W ndows Azure serv ces as we The Create Server w zard ets you take care of th s It prov des a checkbox that, when se ected, w open up SQL Azure’s firewa to W ndows Azure serv ces; t a so prov des an Add button wh ch a ows you to configure nd v dua IP addresses, or IP address ranges, that w be a owed through C ck the Add button and the Add F rewa Ru e d a og box w appear At the bottom, the IP address from wh ch you’re connect ng w be shown You can copy that address (by h gh ght ng t w th the mouse and press ng Ctrl+C) and paste t (w th Ctrl+V) nto the IP Range Start and IP Range End ed t contro s, and then enter a name n the Ru e Name textbox, as shown n F gure 12-2

Figure 12-2  Add ng a SQL Azure firewa ru e.



Chapter 12  Mov ng to the C oud w th SQL Azure    587

Tip  You don’t need to use the SQL Azure Management Portal to set firewall rules; you can use T-SQL instead. With the master database selected, simply use the sp set firewall rule system stored procedure supplying the rule name, IP range start address, and IP range end address as parameters (each as quoted strings). To do this, you will need to connect to your SQL Azure server from SSMS, SSDT, or another client—this will be covered later in this chapter. When you’re done, c ck OK Back n the Create Server w zard, enter add t ona firewa ru es f you’d ke, and then c ck the F n sh button to return to the W ndows Azure Management Porta When you get there, you’ see that your server has been created and g ven a random y ass gned 10-character ­a phanumer c un que name Th s name w be d sp ayed n the tree v ew at the upper- eft hand corner of the screen, under the node for your subscr pt on, once you expand the atter If you c ck your new server’s node n the tree v ew, the name w a so be d sp ayed at the topcenter of the screen, a ong w th a summary of your firewa ru es The server name, comb ned w th the database.windows.net doma n name, const tutes an access b e host name that you can use when creat ng connect ons or connect on str ngs for your SQL Azure database If you expand the server’s tree v ew node, you w see that a master database for your server has been created for you There’s a so a button n the m dd e of the r bbon that w a ow you to create a new database A of th s s shown and h gh ghted n F gure 12-3

Figure 12-3  V ew ng the nformat on page for a new SQL Azure server.

588  Part III  Applied SQL

Provisioning Your Database C ck the Create button n the Database group at the center of the r bbon to br ng up the Create Database d a og Enter a database name, and make sure Web s se ected for Ed t on, and 1 GB for Max mum S ze Now c ck OK to return to the W ndows Azure Management Porta Your database shou d appear n the tree v ew at the upper eft of the screen Se ect t and you shou d see a summary of your database d sp ayed A button n the r bbon w appear that a ows you to test connect v ty to the database You’ a so see Manage buttons that et you manage the server or the database n a separate too ca ed the SQL Azure Management Porta At the bottom-r ght of the screen you w a so see an https URL that can be used to nav gate to the SQL Azure Management Porta d rect y A of th s s shown and h gh ghted n F gure 12-4

Figure 12-4  SQL Azure Management Porta server page after prov s on ng a database.

Managing Your Database At th s po nt, you’re done w th the W ndows Azure Management Porta and can use the SQL Azure Management Porta nstead C ck the Manage button n the r bbon’s Database group to br ng up the SQL Azure Management Porta Use your SQL Azure server-w de adm n strat ve user credent a s (not your L ve ID and ts password) to og n, and then a ow the porta a few moments to oad up



Chapter 12  Mov ng to the C oud w th SQL Azure    589

The SQL Azure Management Porta , wh ch s re at ve y new as of th s wr t ng, s a separate ­ anagement too for adm n ster ng your SQL Azure server and databases, as we as des gn ng, m ma nta n ng data n, and query ng your databases The too features a Metro-sty e UI us ng the same e ements of L ve T es, semant c zoom, unc uttered des gn, and emphas s on typography as s used n the W ndows Phone, W ndows 8, and Xbox dashboard env ronments C ck the Des gn tab (at the bottom eft of the screen) and the Des gn screen w come up It has pages for ma ntenance of Tab es, V ews, and Stored Procedures The Tab es page s se ected by ­defau t and shown n F gure 12-5

Figure 12-5  Tab es page of database Des gn screen.

Creating Tables and Entering Data Not ce the “breadcrumb bar” at the top of the screen and the screen tab we at the upper eft The e ements n each are a c ckab e to a ow you to eas y nav gate to a pr or screen, or to a d fferent ma ntenance screen n the server > database > object co ect on h erarchy C ck the New Tab e nk (near the center of the screen) to br ng up the tab e des gner, and then ­ nter a tab e name and deta s for ts co umns Your screen shou d appear somewhat ke the one e shown n F gure 12-6

590  Part III  Applied SQL

Figure 12-6  Co umns page of the tab e Des gn screen.

When you’re done, c ck the Save button n the r bbon (h gh ghted n F gure 12-6), then se ect the Data screen and enter a few rows of data, as shown n F gure 12-7

Figure 12-7  Data page of the tab e Des gn screen w th two new rows pend ng.



Chapter 12  Mov ng to the C oud w th SQL Azure    591

C ck the Save button n the r bbon and your data w be d sp ayed, n a non-ed tab e state, n a gr d From here, you can execute ad hoc SQL quer es aga nst the database

Querying in the Browser C ck New Query and you’ be p aced n the SQL Azure Management Porta ’s T-SQL query too , ­comp ete w th co or syntax h gh ght ng (a though w thout Inte Sense or squ gg y error nd cators) Try enter ng a s mp e SELECT * query aga nst your new tab e and c ck the Run button n the r bbon Your screen shou d appear s m ar to that shown n F gure 12-8

Figure 12-8  Query screen w th resu ts.

Not ce the sp tter bar between the query ed tor and the response pane A so not ce the separate nks for Messages and Resu ts, just ke the correspond ng tabs n SSDT and SSMS If you hover your mouse over the number-of-rows d sp ay, a drop-down arrow w appear wh ch can be used to sw tch between d fferent resu t sets, f more than one s returned by your query

Index Design You can return to the tab e des gn v ew by c ck ng the Des gn tab at the bottom eft and ed t ng the tab e or c ck ng the tem n the tab we at the upper eft, correspond ng to your tab e name Once there, you can c ck the Indexes And Keys nk at the top to go to the ndex ed tor, shown n F gure 12-9

Note  Figure 12-9 depicts a different table than that shown in Figures 12-6, 12-7, and 12-8, in order to illustrate greater index complexity. 592  Part III  Applied SQL

Figure 12-9  ndexes And Keys page of the tab e Des gn screen.

Management and Visualizations The SQL Azure Management Porta a so prov des a great server- eve dashboard To see t, c ck the Log Off nk at the top r ght of the screen and then og n aga n, th s t me eav ng the Database ed t contro b ank Your screen shou d appear s m ar to the one shown n F gure 12-10

Figure 12-10  The Overv ew page n SQL Azure Management Porta .



Chapter 12  Mov ng to the C oud w th SQL Azure    593

W th on y one database, there’s not a ot to see here yet, but once you have a few, you’ apprec ate the server- eve contro that th s screen g ves you Not ce the Se ect a Database nk at the top eft C ck ng th s wou d a ow you to se ect from a st of recent y opened databases, as we as from a st of a databases on the server The tab we r ght be ow the Se ect a Database nk a ows you to sw tch between screens connected to d fferent databases on your server A co or-cod ng scheme wou d prov de v sua cues as to wh ch tabs were nked w th wh ch database In the s tuat on where you have mu t p e screens open, c ck ng the My Work nk w br ng you to a thumbna v ew of those screens, as shown n F gure 12-11

Figure 12-11  The My Work screen, w th three screen thumbna s d sp ayed.

The SQL Azure Management Porta even prov des features that the SQL Server too set does not One examp e of th s s an exce ent Metro-sty e v sua zat on of query execut on p ans F gure 12-12 shows a part a v ew of one such p an, v sua zed, w th a deta s pane d sp ayed for a Clustered Index Seek node n the p an The execut on p an v sua zer can be used to show both est mated and actua p ans (us ng c­ orrespond ng y abe ed buttons n the r bbon) and can even be used on SQL Server execut on p ans, by us ng the Open button n the r bbon and open ng up a sq query fi e or a sq p an p an fi e from a oca dr ve or network share The v sua zer a so supports the Metro concept of semant c zoon For the query p an v sua zat on shown n F gure 12-12, you cou d s mp y zoom out, us ng the s der contro at the ower eft of the v sua zat on, or the whee on your mouse, and a more summar zed v ew of the p an, shown n F gure 12-13, wou d appear

594  Part III  Applied SQL

Figure 12-12  Query P an page of Query screen, w th deta s pane d sp ayed.

Figure 12-13  Query P an page, zoomed out to summary v ew.



Chapter 12  Mov ng to the C oud w th SQL Azure    595

Another handy feature s the database Adm n strat on screen’s Query Performance page wh ch can show you, at a g ance, the quer es that are most expens ve n terms of CPU ut zat on, durat on of execut on, and d sk access, as shown n F gure 12-14

Figure 12-14  Query Performance page of database Adm n strat on screen.

Connecting from Down Below Now that you understand what SQL Azure s about, how to get started w th t, and how to m ­ anage t from browser-based too s, t’s t me for us to descend from the c ouds and understand how to ntegrate SQL Azure w th the too s that we use and the database des gn work that we carry out w th SQL Server Let’s take a ook at how to manage SQL Azure w th too s ke SSDT and SSMS, and how to move databases and database des gns up to SQL Azure w th those too s as we F rst th ngs first Let’s d scuss the bas cs of connect ng to SQL Azure from any c ent For the most part, you do so the same way as you wou d w th SQL Server you can use the ADO NET SQL Server prov der (SqlClient) or connect v a Open Database Connect v ty (ODBC) us ng the SQL Server Nat ve C ent dr ver You’ configure your connect on (or connect on str ng, f you’re cod ng) n a ke manner as you wou d w th SQL Server too, as ong as you keep the fo ow ng n m nd ■

Set the server name to the fu host name for your server, n the form servername.database.windows.net In the examp es we’ve been us ng, the server name wou d be set to og7ztx092p.database.windows.net

596  Part III  Applied SQL

■

■

■

■

Set authent cat on to SQL Server Authent cat on (or set Integrated Security=False n your ­connect on str ng, f you’re cod ng) Then set the og n and password credent a s to those for a va d SQL Azure og n n your database (not your L ve ID credent a s) Some c ents w requ re you to add an @ s gn and the server name after the user name (e g , sqlazadmin@og7ztx092p) Keep n m nd that SQL Azure a ways operates over TCP port 1433 SQL Server uses the same port by defau t, but t a ows t to be configured to a d fferent port, whereas SQL Azure does not As such, any oca or network firewa s must a ow outbound traffic on TCP port 1433 You need to set firewa ru es on your SQL Azure server n the W ndows Azure Management Porta for a IP addresses from wh ch a c ent may connect If you don’t do th s, you may find yourse f unab e to connect to your SQL Azure database, and you may ncorrect y assume that one of the above sett ngs s n error, when t’s s mp y a matter of need ng to author ze the ­c ent’s IP address

In add t on to the connect on t ps shared so far, there rea y s one other we shou d d scuss You shou d set Encrypt=True and TrustServerCertificate=False, whether n your connect on str ng f you’re cod ng, or n the advanced propert es of a connect on d a og f you’re not Th s w prevent ­“man- n-the-m dd e” attacks on your SSL connect on to SQL Azure

Tip  If your client application allows you to select SQL Azure specifically (rather than SQL Server generically) as your data source, then the Encrypt and TrustServerCertificate settings may be configured correctly by default. But you should check to make sure, and set them yourself if necessary. From SSMS and SSDT, sett ng these connect on str ng parameters can be a tt e tr cky A though an encrypted connect on can be turned on by c ck ng the Connect to Server d a og’s Opt ons button and then check ng the Encrypt Connect on check box n the Connect on Propert es tab, there s no such GUI opt on to set TrustServerCertificate=False Therefore, the eas est way to set both of these s to enter TrustServerCertificate=False;Encrypt=True n the Add t ona Connect on Parameters tab F gure 12-15 shows the proper y configured Log n and Add t ona Connect on Parameters tabs of the Connect to Server d a og box

Note  A sample ASP.NET application, SARASPNETTest, is included with this chapter. The code connects to a SQL Azure database, performs a generic SELECT * query on it, and binds the ­result set to an ASP.NET GridView control. The code uses a generic SQL Azure connection string in the web.config file. Just edit the connection string’s server name, user id, and password in web.config, and the SQL query in default.aspx.cs, to be appropriate for your own SQL Azure ­database, and run the application for a complete practical demo.



Chapter 12  Mov ng to the C oud w th SQL Azure    597

Figure 12-15  Connect ng to SQL Azure from SSMS and SSDT.

Note  You may very well prefer not to use the server-wide administrative login to c­ onnect to your SQL Azure database from the SQL Server tools. If that’s the case, you’ll need to ­connect as the server-wide administrative user once and then, either in SSMS or SSDT, ­create a new login and database user with the T-SQL templates both tools provide for these tasks. You may wish to make this user a member of the db owner role. Once you have entered the correct connect on nformat on, c ck the Connect button F gure 12-16 shows the resu t ng appearance of the Object Exp orer n SSMS (top) and the SQL Server Object ­Exp orer n V sua Stud o/SSDT (bottom) (The server and Databases nodes are co apsed by defau t and were manua y expanded to revea the under y ng database)

Tip  After creating a database-level user, you can disconnect and then reconnect using that user’s credentials. However, your login will fail unless you set Default Database in the Connection Properties tab of the Connect To Server dialog box to a database that user has ­permissions on. A though our database has a ready been created, you cou d nstead connect to the server and ­create a database v a T-SQL For examp e, to create a 40 GB Bus ness ed t on database, you’d connect to your SQL Azure server and ssue the fo ow ng command CREATE DATABASE MyNewDB (EDITION='BUSINESS', MAXSIZE=40GB)

598  Part III  Applied SQL

Figure 12-16  SQL Azure connect on d sp ayed n SSMS Object Exp orer and V sua Stud o/SSDT SQL Server ­Object Exp orer.

The EDITION and MAXSIZE parameters a so app y to the ALTER DATABASE command, thereby ­ ow ng you to change the max mum s ze of your SQL Azure database (to be e ther b gger or a sma er), shou d the need ar se Note the server g yph n both too s dep cts a ght-b ue (azure, n fact) database drum con as a v sua queue that the server connect on s to SQL Azure Other d fferences w man fest themse ves based on the de ta n funct ona ty between SQL Server and SQL Azure For examp e, there are no Server Objects or Replication ch d nodes for a SQL Azure connect on as there wou d be for a SQL Server connect on A though th ngs d ffer, there s a so a ot of par ty If you dr down on a database’s node n the SSDT SQL Server Object Exp orer, you can see, des gn, and query ts tab es, v ews, stored procedures, funct ons, and tr ggers The Object Exp orer n SSMS prov des ess des gner support for these objects and requ res you to work more at the T-SQL eve , but because SSDT constructs the same n-memory database mode for SQL Azure databases that t does for SQL Server databases, v rtua y a the ­too ng support for the atter s ava ab e for the former

Migrating and Syncing Between Earth and Cloud The danger n wr t ng a d screte chapter on SQL Azure n a vast book on SQL Server s that we w prepare you to use both products, but n a rather segregated fash on If that happened t wou d be a huge fa ng though, because the ntersect on between the two products s rather arge Not on y can you take an agnost c approach to too s, query, and deve opment that works w th SQL Azure and SQL Server re at ve y nterchangeab y, but you can a so move databases and database des gns from one to the other very e egant y In pract ca terms, what does th s mean? You can move a database from SQL Server to SQL Azure, and v ce versa; you can a so export database schemas from one p atform to the other, nterchangeab y; you can compare and sync schemas; and sync and rep cate data

Chapter 12  Mov ng to the C oud w th SQL Azure    599

For the task of sync ng data, M crosoft prov des a powerfu Azure-based too ca ed Data Sync, wh ch s covered n depth n the next chapter A ongs de Data Sync, though, es SQL Server Data-T er App cat ons (DACs), and support for them n SSDT, SSMS, the W ndows Azure Management P ­ orta , and the SQL Azure Management Porta In th s sect on, we’ d scuss what DACs are, how they’re used by var ous SQL Server too s, and how SQL Azure ut zes them for var ous mport/export and ­m grat on tasks Data-T er App cat ons were ntroduced n SQL Server 2008 R2 as both a too ng approach for database des gn and a package format for database dep oyment Restor ng backups, attach ng MDF fi es, or just runn ng T-SQL scr pts a so prov de ways of dep oy ng databases on servers other than where they were des gned But none of the atter three techn ques tru y prov des a s ng e un t of ­database dep oyment, nor the ab ty to create or dep oy them n a s ng e adm n strat ve operat on

DACPACs to the Rescue The fi es that store the contents of a Data-T er App cat on do supp y such a dep oyment un t though They have a DACPAC extens on but are actua y z p fi e arch ves conta n ng a number of const tuent xm fi es SSMS has the ab ty to extract or dep oy DACPAC fi es each through a menu opt on The SQL Server Database Projects n SSDT generate DACPACs as a matter of course, and SSDT can create a project from a DACPAC as we G ven a th s DACPAC ng n the SQL Server too set, t shou d come as no surpr se that SQL Azure uses the DACPAC format for ts own dep oyment-re ated features, too Data-T er App cat ons and SQL Azure a so support a BACPAC format, wh ch s used as a conta ner for both schema and data (DACPACs conta n schema on y), and can thus be used for mport/export funct ons to m grate a database, rather than mere y dep oy a new (and n t a y empty) database nstance Let’s take a ook now at the mechan cs of mov ng and updat ng database schemas and data, w th Data-T er App cat ons, between SQL Server and SQL Azure us ng SSMS, SSDT, and the W ndows Azure and SQL Azure Management Porta s

Extract, Deploy, Export, and Import DAC files Let’s beg n by document ng the mechan cs of how and where we can create and open DACPAC and BACPAC fi es Afterwards we’ d scuss how and where we can use them Prov d ng th s nventory of menu and r bbon UI techn ques s, honest y, a rather dry document ng process, so we w cover t n tab e form, rather than n our narrat ve

SQL Server Management Studio SSMS s the most og ca too to start w th as t can work b d rect ona y w th both DACPAC and BACPAC fi es SSMS conta ns a number of w zards that ass st n the generat on and use of ­Data-T er App cat ons A of these are aunched from context menus on the Object Exp orer Tab e 12-1 ­summar zes your opt ons 600  Part III  Applied SQL

Table 12-1  Work ng w th DACPAC and BACPAC fi es n SSMS Action

Result

R ght c ck a server s Databases ch d node, and then se ect Dep oy Data T er App cat on from the context menu

Creates a database from a DACPAC fi e se ected n the Dep oy Data T er App cat on W zard

R ght c ck a database node, and then c ck Tasks Dep oy Database to SQL Azure

Creates a BACPAC fi e from the se ected database and dep oys t d rect y to SQL Azure

R ght c ck a database node, and then c ck Tasks Export Data T er App cat on

Creates a BACPAC fi e from the se ected database wh ch can be manua y mported, subsequent y

R ght c ck a database node, and then c ck Tasks Extract Data T er App cat on

Creates a DACPAC fi e from the se ected database

R ght c ck a database node, and then c ck Tasks Reg ster As Data T er App cat on

Creates a DACPAC from, and embeds t n, the se ected database

R ght c ck a server s Databases ch d node, and then se ect mport Data T er App cat on from the context menu

Creates a database, and mports ts data, from a BACPAC fi e on oca d sk or n W ndows Azure B ob Storage.

R ght c ck a BACPAC fi e from an Azure Storage connect on, and then c ck mport Data T er App cat on

Creates a database, and mports ts data, from a BACPAC fi e n W ndows Azure B ob Storage. A W ndows Azure Storage deta s n the mport Sett ngs screen of the resu t ng mport Data T er W zard w be pre popu ated.

SQL Server Data Tools SQL Server Database Projects n V sua Stud o are themse ves prem sed on the generat on of DACPAC fi es There are severa other ntegrat on po nts for Data-T er App cat on fi es n SSDT as we , ­nc ud ng, but not m ted to, ts Snapshot funct ona ty A summary of these appears n Tab e 12-2 Table 12-2  Work ng w th DACPAC fi es n SSDT Action

Result

R ght c ck the project node n So ut on Exp orer and se ect mport Data T er App cat on (*.dacpac) from the context menu

Creates SQL Server Database Project from a DACPAC fi e

R ght c ck the project node n So ut on Exp orer and se ect Snapshot Project from the context menu

Generates DACPAC fi e ( n Snapshots fo der), reflect ng current state of SQL Server Database Project

Bu d Bu d So ut on or Bu d Bu d project

Generates DACPAC fi e n b n\debug or b n\re ease fo der (depend ng on bu d configurat on)

Windows Azure Management Portal The W ndows Azure Management Porta hosts a spec a SQL Azure Import/Export serv ce that reads and wr tes BACPAC fi es from and to W ndows Azure B ob Storage, requ r ng credent a s for both SQL Azure and W ndows Azure Storage as t does so Tab e 12-3 documents the opt ons



Chapter 12  Mov ng to the C oud w th SQL Azure    601

Table 12-3  Work ng w th the BACPAC fi es n the W ndows Azure Management Porta Action

Result

C ck the mport button n the r bbon s mport And Export group

Creates a database, and mports ts data, from spec fied BACPAC fi e n W ndows Azure B ob Storage

C ck the Export button n the r bbon s mport And Export group

Exports a database, nc ud ng ts data, to spec fied BACPAC fi e n W ndows Azure B ob Storage

SQL Azure Management Portal Un ke the W ndows Azure Management Porta , wh ch mports and exports us ng BACPAC fi es n B ob Storage, the SQL Azure Management Porta , much ke SSMS, features Dep oy, Extract, and Upgrade buttons n ts r bbon that work w th DACPAC fi es on a oca d sk or network share Tab e 12-4 deta s a the opt ons Table 12-4  Work ng w th DACPAC fi es n the SQL Azure Management Porta Action

Result

n Overv ew tab, c ck Dep oy button n r bbon or c ck the Dep oy A Data T er App cat on t e

Creates a database from a DACPAC fi e on oca d sk or network share

n Adm n strat on tab, n the Databases or Tasks page, or n the Des gn tab, c ck Dep oy button n r bbon

Creates a database from a DACPAC fi e on oca d sk or network share

n Adm n strat on tab, n the Databases page, c ck the Extract button

Creates a DACPAC fi e from the current y open database and saves t to a oca d sk or network share

n Adm n strat on tab, n the Databases page, c ck the Upgrade button

Updates schema of current y open database from a DACPAC fi e on a oca d sk or network share

Scenarios Now you know what to c ck to work w th DACPAC and BACPAC fi es, but do you know when? It’s one th ng to st out a the p aces where DACPACs and BACPACs can be created and consumed, but now et’s d scuss some SQL Azure–re ated scenar os where these capab t es wou d be most he pfu

New Deployment to SQL Azure If you’ve been des gn ng a database spec fica y for SQL Azure, be t n SSMS or SSDT, then at some po nt you w need to dep oy t There are severa ways to do th s Let’s ook at a few From SSDT For readers of th s book, the dea dep oyment s tuat on wou d nvo ve des gn ng the database n SSDT, spec fica y for SQL Azure, and “pub sh ng” the (empty) database from there Des gn ng your database n th s way ensures that you w not nadvertent y use a SQL Server feature that SQL Azure does not support, because SSDT wou dn’t a ow t Just make sure that you se ect SQL Azure as the target p atform n the Project Sett ngs tab of the Propert es sheets

602  Part III  Applied SQL

To dep oy the database to SQL Azure, you’ then need to se ect the Bu d Pub sh projectname ma n menu opt on or r ght-c ck the project node n So ut on Exp orer and se ect the Pub sh opt on from the resu t ng context menu Th s w br ng up the Pub sh Database d a og box In that d a og box, c ck the Ed t button next to the Target Database Connect on (read-on y) text box and n the resu t ng Connect on Propert es d a og box, supp y connect on nformat on s m ar to that shown n the Log n tab n F gure 12-15 Make sure to c ck the Advanced button and set the Encrypt and TrustServerCertificate propert es, as shown n F gure 12-17

Figure 12-17  Advanced Propert es d a og box for SSDT database connect on to SQL Azure.

When you’re done, just c ck OK through the Advanced Propert es and Connect on Propert es d a og boxes From there, you may w sh to c ck the Save Profi e As button to save out these sett ngs for easy retr eva dur ng a subsequent pub sh operat on (you wou d re- oad those sett ngs by c ck ng the Load Profi e button) When you’re done, c ck Pub sh, and SSDT w generate and execute the ­appropr ate T-SQL scr pt on your SQL Azure server From SSMS You cou d a so des gn the database on SQL Server w th SSMS and then use that too ’s Dep oy to SQL Azure feature, or use ts Export Data-T er App cat on funct on coup ed w th the Import feature n the W ndows Azure Management Porta For the ­atter techn que, you wou d need to up oad the BACPAC fi e generated by SSMS nto B ob Storage n a W ndows Azure Storage ­account A though there are a number of ways to do th s, one opt on s us ng Neudes c’s free Azure Storage ­Exp orer, a desktop app cat on ava ab e at http://azurestorageexplorer.codeplex.com/ Keep n m nd that the SSMS Export Data-T er App cat on feature a ows you to export se ected ­ bjects from your database, rather than requ r ng you to export a of them The key here s to c ck o the Advanced tab n the Export Sett ngs screen of the Export Data-T er App cat on w zard Th s w revea a co apsed tree v ew Dr down on the Select All and schema name nodes to revea a st of ­database objects, a of wh ch w be se ected by defau t You’ need to uncheck the objects you w sh to om t from the BACPAC fi e F gure 12-18 shows the EmployeeTerritories tab e om tted from an export of the Northwind database

Chapter 12  Mov ng to the C oud w th SQL Azure    603

Note  To upload your BACPAC with the Azure Storage Explorer (or using any Azure Storage upload technique) you’ll need to create a new storage account in the Windows Azure Management Portal. Do this by clicking on the Hosted Services, Storage Accounts & CDN tab in the Outlook-style left-hand navigation bar. Use the New Storage Account and View Access Keys buttons in the ribbon to create the account and fetch the keys needed to ­access it. Supply the name of the storage account and an access key to the Azure Storage Explorer, and then use that application to create a “Container” (a folder, essentially) into which you can upload your BACPAC file. From there you can use the Import facility in the Windows Azure Management Portal to import the BACPAC into SQL Azure. Be careful ­during this process because the file name must be entered in a case-sensitive manner.

Figure 12-18  Exc ud ng a tab e from a BACPAC fi e w th the SSMS Export Data T er App cat on w zard.

From the SQL Azure Management Portal F na y, you cou d use the Dep oy feature n the SQL Azure Management Porta to create a new SQL Azure database based on a DACPAC The fi e cou d be sourced from a DACPAC fi e created n SSDT through the Bu d or Snapshot processes, or a DACPAC created n SSMS w th the Extract Data-T er App cat on feature

Deploy Update to SQL Azure DACPACs and the SSDT Pub sh operat ons are not just for n t a database dep oyments; they’re for nter m schema update dep oyments too For th s scenar o, you cou d perform a subsequent Pub sh o ­ perat on from SSDT You cou d a so use the Upgrade feature n SQL Azure Management Porta a ong w th a DACPAC fi e created n SSDT, or one created n SSMS w th the Extract Data-T er App cat on feature 604  Part III  Applied SQL

Migration from SQL Server to SQL Azure One common scenar o w th the c oud s a deve oper or corporate dec s on to m grate a database from an on-prem ses env ronment to the c oud env ronment As such, the scenar o n wh ch an on-prem ses SQL Server database s m grated to SQL Azure w be a common one SSDT  One way to accommodate th s scenar o s to use SSDT as fo ows 1. Create a SQL Server Database Project w th the target p atform set to SQL Azure 2. Import an ex st ng SQL Server database by r ght-c ck ng on the project node n So ut on

­Exp orer and then c ck ng Import From Database n the resu t ng context menu 3. Pub sh the database to SQL Azure fo ow ng the steps descr bed prev ous y

After mport ng the database, SSDT w warn you of anyth ng n the database’s des gn that s not compat b e w th SQL Azure One such ssue that can be common s the ack of c uster ng ndexes on spec fic tab es F gure 12-19 dep cts some other errors for a SQL Server Database Project target ng SQL Azure nto wh ch the AdventureWorksLT samp e database has been mported

Figure 12-19  SSDT flagg ng SQL Azure compat b ty errors.

SSMS The on-prem ses-to-c oud m grat on scenar o can a so be accommodated n var ous ways w th SSMS For examp e, the SSMS Dep oy Database to SQL Azure feature cou d be used to m grate the database n one step In th s case, the Resu ts page of the Dep oy Database w zard w ensure compat b ty of the cand date SQL Server database w th SQL Azure In case of ncompat b t es, t w report errors and prevent the dep oyment, as shown n F gure 12-20

Chapter 12  Mov ng to the C oud w th SQL Azure    605

Figure 12-20  SSMS flagg ng SQL Azure compat b ty errors.

Another cho ce s to use the SSMS Export Data-T er App cat on feature to generate a BACPAC fi e wh ch cou d then be brought nto SQL Azure us ng the Import feature n W ndows Azure ­Management Porta A though var ous methods of dep oy ng DACPAC fi es to SQL Azure cou d be used as we , these wou d m grate the schema on y, requ r ng other means to transport the data

Migration from SQL Azure to SQL Server A though m grat ng to the c oud s the more common scenar o, there are certa n y nstances when ­m grat ng from SQL Azure to SQL Server cou d take p ace as we SSDT can mport from SQL Azure ­databases; t needn’t on y be used to pub sh to them The Export feature n the W ndows Azure ­Management Porta coup ed w th the SSMS Import Data-T er App cat on feature cou d be used as we Aga n, var ous methods of dep oy ng DACPAC fi es to SQL Server that were extracted from SQL Azure cou d be used as we , but these wou d m grate the schema on y, requ r ng other means to transport the data We have now covered scenar os of work ng exc us ve y w th SQL Azure and ts too s, as we as s­ tuat ons where SQL Server too s and databases are added to the m x W th these bas cs estab shed, et’s take a ook at an advanced feature of SQL Azure

606  Part III  Applied SQL

SQL Azure Federations We’ve spent a ot of t me so far d scuss ng certa n features n SQL Azure that are not supported n SQL Server But as t turns out, one converse case ex sts as we SQL Azure offers a very nterest ng feature ca ed Federat ons, wh ch s not offered by SQL Server And because the Federat ons feature s extreme y usefu n the k nd of mu t -tenancy scenar os ( e , coex stence of mu t p e customers’ data) that c oud ­products are extreme y su tab e for, t’s mportant that we cover Federat ons here n th s chapter Federat ons are the SQL Azure mp ementat on of a part t on ng pattern ca ed sharding Shard ng s a very common feature n NoSQL (non-re at ona ) databases, but s not often seen n the re at ona wor d In a shard ng scheme, each part t on s n fact a separate phys ca database and/or res dent on a separate phys ca server When SQL Azure was first ntroduced, t had the restr ct on of a 10 GB max mum database s ze Many customers expressed surpr se and skept c sm towards th s m tat on, and M crosoft responded by say ng customers cou d mp ement a shard ng scheme to combat t In effect, M crosoft was say ng that f you needed a database of x GB, then d v de x by 10 and create that number of 10 GB ­databases From there, the gu dance suggested you mp ement your own programm ng og c for sp tt ng up the data amongst these mu t p e databases when wr t ng data, or for gather ng data from each appropr ate database when query ng Qu te frank y, many customers found that prescr pt ve gu dance to be a b t g b, and so d d we Perhaps n response to th s, M crosoft mp emented the Federat ons feature, wh ch keeps customers from hav ng to mp ement a shard ng nfrastructure of the r own M crosoft a so mp emented very strong too ng support for Federat ons n the SQL Azure Management Porta , and the SQL Server 2012 vers on of SSMS prov des good support as we To understand how Federat ons work, we’ d scuss Federat ons concepts and vocabu ary, as we as the correspond ng T-SQL syntax and the SQL Azure Management Porta techn ques needed to create and use the objects d scussed

A SQL Azure Federations Lexicon In SQL Azure, a Federat on s the co ect on of phys ca databases that together compr se a s ng e, og ca , federated database The first database to jo n the Federat on s ca ed the Federat on Root; t stores the Federation object tse f, and t can be thought of as the “home base” or d rectory node for the Federat on The other phys ca databases that compr se the Federat on are ca ed Federat on Members, and they store the federated data The Federat on Key, a so known as the D str but on, defines a data key wh ch s used to determ ne how d fferent data s mapped to spec fic Federat on members A data for a g ven Federat on Key va ue s conta ned w th n a s ng e Atom c Un t, so named ­because t s never to be sp t across Federat on Members That sa d, no other spec fic correspondence between Atom c Un ts and Federat on Members ex sts One Federat on Member may, and ke y w , conta n mu t p e Atom c Un ts, a though t cou d, theoret ca y, conta n just one But any Atom c Un t conta ned by a Federat on Member w be conta ned n ts ent rety n that Federat on Member Th s supports mu t -tenancy scenar os n ce y, as we w cover at the end of th s sect on

Chapter 12  Mov ng to the C oud w th SQL Azure    607

Creating a Federation Federat ons are created w th the CREATE FEDERATION T-SQL command, and requ re that the ­database to be used as the Federat on Root s current y se ected Here’s an examp e CREATE FEDERATION MyFed (MyDist BIGINT RANGE)

The above examp e assumes MyFed s the Federat on be ng created and MyDist s the ­D str but on name It further assumes that the D str but on Key w be of type bigint (the other cho ces are int, uniqueidentifier, and varbinary) and that the D str but on w be defined such that each Federat on Member maps to a RANGE of va ues (wh ch, as of th s wr t ng, s the on y cho ce) In the SQL Azure Management Porta , you create a Federat on by se ect ng an ex st ng database and c ck ng the New button on the far r ght of the r bbon A form then pops up wh ch essent a y prov des a UI for the CREATE FEDERATION command, ask ng you to supp y the Federat on name, D str but on name, D str but on Key data type, and D str but on type (where, aga n, RANGE s the on y cho ce) C ck ng Save on th s form, and then wa t ng for the UI to update tse f, w resu t n the Federat on be ng sted n the database’s summary screen C ck ng the Federat on’s r ght-arrow g yph then br ngs you to a Federat on Members map screen, as shown n F gure 12-21

Figure 12-21  Federat on Member map n SQL Azure Management Porta .

608  Part III  Applied SQL

Federated Tables A D str but on defines how th ngs are sp t, and Federat on Members house spec fic const tuent data for each so-ca ed Federated Tab e ( e , each tab e that you dec de ought to be sp t) Federated Tab es must have a co umn that stores the Federat on Key va ues, wh ch cannot be NULL Imag ne a d str but on where one Federat on Member s des gnated to conta n data for the range of Federat on Key va ues between 1 and 99 and another w conta n data for Federat on Key ­co umn va ues of 100 and above In th s hypothet ca , any data for any Federated Tab e whose Federat on Key co umn’s va ue s 99 or ess w go n that tab e n the first Federat on Member and data for ­Federat on Key co umn va ues of 100 and h gher w go n that tab e n the other Federat on Member Creat ng a Federated Tab e s done us ng a most the same T-SQL that s used for a convent ona tab e, but a Federat on Member must be se ected first and a FEDERATED ON c ause, spec fy ng the D str but on name and the name of the co umn n the tab e conta n ng the Federat on Key, must be supp ed n the CREATE TABLE command For examp e USE FEDERATION [MyFed] ([MyDist] = -9223372036854775808) WITH FILTERING = OFF, RESET GO CREATE TABLE MyFederatedTable( ID bigint NOT NULL, lastname varchar(50), firstname varchar(50), CONSTRAINT PKFederated PRIMARY KEY CLUSTERED (ID ASC) ) FEDERATED ON (MyDist=ID)

The above T-SQL code se ects the first Federat on Member ( dent fied w th a max ma ­negat ve number as the Federat on Key va ue) and assumes that MyFed s the Federat on, MyDist s the ­D str but on, MyFederatedTable s the Federated Tab e, and ID s the co umn wh ch w conta n the Federat on Key va ues The tab e s g ven a c ustered pr mary key because a SQL Azure tab es must have a c ustered ndex and the co umn ID s configured as NOT NULL because that s a requ rement for the Federat on Key co umn Execut ng the above T-SQL code assures that the spec fied ­Federat on Member wou d have ts own nstance of MyFederatedTable, conta n ng rows w th va ues of ID ­correspond ng to the range of va ues defined for that Federat on Member If your federated database has mu t p e Federat on Members when th s tab e s created, then the data defin t on anguage (DDL) query must be executed aga nst each such Federat on Member to have that tab e represented n each If you create the tab e when the federated database has on y one Federat on Member, then the query need on y be executed once, of course If you ater sp t the ­Federat on Member, the tab e w automat ca y be present n the new Federat on Members that resu t



Chapter 12  Mov ng to the C oud w th SQL Azure    609

Using a Federation Member When t comes t me to nsert, update, de ete, or query data, the appropr ate Federat on Member(s) must be se ected w th the T-SQL USE command Wh e that may seem nconven ent, the good news s that rather than need ng to know what data es where and se ect ng the correspond ng Federat on Member by name, you nstead spec fy a key va ue n the USE command to se ect the ­correspond ng Federat on Member (as was shown n the prev ous T-SQL sn ppet) For examp e, to se ect the ­Federat on Member conta n ng data w th a Federat on Key va ue of 100, you wou d ssue the ­fo ow ng T-SQL USE command USE FEDERATION [MyFed] ([MyDist] = 100) WITH RESET

In the SQL Azure Management Porta , you can br ng up a query w ndow conta n ng the necessary T-SQL n a temp ate to create or query a Federated Tab e, preceded by the appropr ate USE FEDERATION query To do so, c ck a Federat on Member n the Federat on Members map screen shown n F gure 12-21, se ect Query from the resu t ng pop-up menu, and then c ck Create Federated Tab e or New Query, respect ve y

Splitting and Dropping Federation Members The pop-up menu a so conta ns opt ons for sp tt ng or dropp ng a Federat on Member These opt ons prov de UIs for the ALTER FEDERATION T-SQL command, us ng the SPLIT or DROP c ause, respect ve y These commands et you add or remove a Federat on Member by add ng or remov ng a sp t, defined by a spec fic key va ue For examp e, your federated database w start off hav ng on y a s ng e Federat on Member, conta n ng a va ues for the Federat on Key Us ng our ear er examp e, you cou d use the ALTER FEDERATION…SPLIT command (or the Sp t menu opt on on the Federat on Member n the SQL Azure Management Porta ) to sp t the Federat on Member at a boundary va ue of 100, wh ch wou d create a second Federat on Member and transfer a data w th key va ues of 100 and h gher to t You a so cou d ater dec de to remove one of those two Federat on Members us ng the ALTER FEDERATION… DROP command (or the Drop Federat on Member menu opt on n the SQL Azure Management ­Porta ) Regard ess of whether you sp t or drop, and whether you do so w th a T-SQL command or the SQL Azure Management Porta too ng, your federated database w rema n on ne and query-ab e wh e the operat on takes p ace Th s s an exce ent capab ty and one wh ch s unprecedented among other databases that support shard ng

Central Tables and Reference Tables Not a tab es need to be federated ( e , sp t) Those wh ch are not federated and ex st so e y n the Federat on Root are ca ed Centra Tab es Other tab es, wh ch are ne ther federated nor centra , and nstead are duplicated across Federat on Members, are ca ed Reference Tab es Typ ca y, Reference Tab es conta n ookup data wh ch can be used to opt m ze quer es on any Federat on Member

610  Part III  Applied SQL

Fan-Out Queries and Multi-Tenancy SQL Azure Federat ons requ re exp c t use of Federat on Members n order to query the r data For quer es that span mu t p e Federat on Members, mu t p e quer es must be performed, one for each Federat on Member whose data s n scope The SQL Azure Federat ons feature does not support the concept of fan-out quer es, where a s ng e SQL query spann ng mu t p e Federat on Members s ­ssued and a data s returned n a s ng e operat on A though th s may seem nconven ent, keep n m nd that the most appropr ate use of ­Federat ons nvo ves arch tect ng your database so that many, or even most, quer es can be sat sfied ent re y from a s ng e Federat on Member That may seem far-fetched unt you cons der that the Federat ons feature s present most y to support mu t -tenancy, where data correspond ng to various work oads s com ng ed n one database But, f the Federat on sp ts are des gned correct y, data for any given work oad w be comp ete y conta ned n a s ng e Federat on Member For examp e, mag ne a software as a serv ce (SaaS) system for outsourced web-based nvo ce ookups A though such a serv ce cou d ma nta n separate databases for each customer’s records, t wou d be far more effic ent to keep them n a s ng e database, and manage the data n a way where each customer’s records were v rtua y segregated Th s cou d be eas y ach eved us ng Federat ons, us ng the customer ID as the Federat on Key In such a scenar o, most quer es wou d be w th n the scope of a s ng e Federat on Key va ue ( e , a spec fic customer ID), and therefore a s ng e Atom c Un t S nce Atom c Un ts are never sp t over Federat on Members, support for fan-out quer es wou d become moot Quer es cou d be carr ed out as f aga nst a convent ona database as ong as they were preceded by a USE FEDERATION command referenc ng the customer’s ID as the Federat on Key va ue

Federations Support in SSMS and SSDT SSMS a so prov des too ng support for Federat ons The SSMS Object Exp orer w show the Federat on Root, and a ows you to v ew one or many Federat on Members by spec fy ng a Federat on Key va ue There are a so context menu operat ons for creat ng Federat ons and Federated Tab es, as we as sp tt ng and dropp ng Federat on Members These commands open query w ndows conta n ng T-SQL ­temp ates for the correspond ng task If you prefer a UI to a command-dr ven approach, then you’ be more ­comfortab e n the SQL Azure Management Porta The SSDT SQL Server Object Exp orer n V sua Stud o (VS) does not exp c t y support SQL Azure Federat ons The Federat on Root w appear under the federated database’s name The ­Federat on Members, however, appear under the r system-generated phys ca database names F gure ­12-22 shows both Object Exp orers, each d sp ay ng the Federat on Root, Federat on Members, and ­Federated Tab es of the same SQL Azure database, a of wh ch are h gh ghted



Chapter 12  Mov ng to the C oud w th SQL Azure    611

Figure 12-22  Federated databases n SSMS Object Exp orer and SSDT SQL Server Object Exp orer n VS.

Federations Make Sense in the Cloud SQL Azure Federat ons both m t gate the SQL Azure max mum database s ze (wh ch, at press t me, was 150 GB) and prov de a hor zonta sca e-out strategy that works we n many c oud scenar os The T-SQL and too ng support s strong, and the feature s re at ve y ntu t ve Support for fan-out quer es wou d be a we come add t on, but f the sp ts and D str but on for a federated database are we ­des gned, the ack of fan-out query shou d be ess mpactfu Federat ons are a powerfu SQL Azure feature that SQL Azure customers shou d become fam ar w th

SQL Azure Reporting We thought t sens b e and usefu , before we c ose th s chapter, to cover SQL Azure Report ng (SAR), a c oud-hosted vers on of SQL Server Report ng Serv ces (SSRS) Just as SQL Azure can be thought of as a m ted vers on SQL Server, SAR can be thought of as a m ted vers on of SSRS In th s sect on, we’ cover how to prov s on a SQL Azure Report ng server, how to create reports for t, and what the m tat ons are around do ng so In rea ty, SAR s more eas y presented n terms of ts on-prem ses counterpart than s SQL Azure, because SAR s a true subset of SSRS It doesn’t even offer ts own too ng; nstead, you’ use ­Report ng Serv ces projects n V sua Stud o or the stand-a one Report Bu der app cat on to des gn your reports, and then dep oy them to SAR As we d d w th SQL Azure, et’s enumerate mportant restr ct ons of SAR, compared to ts ­fu -fledged, on-prem ses counterpart 612  Part III  Applied SQL

■

■

■

■

■

The only va d data source for SAR reports s SQL Azure No other data source, not even a pub c-fac ng SQL Server nstance, s supported SAR has ts own (forms-based) authent cat on scheme It does not support W ndows ntegrated secur ty, nor does t support SQL Server authent cat on Therefore, users (or app cat ons) must authent cate separate y nto both SAR and the SQL Azure database(s) wh ch serve as the data source(s) for reports SAR runs on y n ts own stand-a one mode; t does not offer a SharePo nt- ntegrated mode, be t w th SharePo nt Server or SharePo nt On ne/Office 365 SAR supports the SOAP Web Serv ces nterface of SSRS, but does not offer the standard ­n-browser Report Manager UI W ndows Azure Management Porta offers a SAR UI, but n ­ av gat on to the SAR server URL n a browser prov des on y a d rectory brows ng-sty e ­nterface SAR does not support OData render ng of reports and therefore s not a va d data source for PowerP vot or SQL Server Ana ys s Serv ces Tabu ar mode

As ong as you are m ndfu of the above restr ct ons, you can deve op SAR reports and ntegrate them nto app cat ons (whether those app cat ons are c oud-based or run on-prem ses) Of course, you can’t do that unt you prov s on a SAR server, so et’s qu ck y cover how to do so

Provisioning The process for sett ng yourse f up n SAR s s m ar to that for SQL Azure Once you are ogged nto the W ndows Azure Management Porta , c ck the Report ng button n the ower eft-hand Out ooksty e nav gat on menu, and then c ck the Create button n the Server group on the r bbon to br ng up the ­Create Server w zard, shown n F gure 12-23

Figure 12-23  Subscr pt on and Reg on spec ficat on for a SQL Azure Report ng server.

Se ect the appropr ate subscr pt on and reg on and, as w th other reg on se ect ons, cons der configur ng your SAR server to be n the same Azure data center as your SQL Azure server, n order to m n m ze data egress charges C ck Next and then supp y credent a s to be used n the creat on of an

Chapter 12  Mov ng to the C oud w th SQL Azure    613

adm n strat ve user on the server After you c ck F n sh, a ow the prov s on ng process to take p ace, and then dr down through the tree v ew nodes on the upper eft to revea the node for your server ­Se ect that node and the Report ng page shou d d sp ay the web serv ce URL, name, and reg on of your SAR server You can cons der the Report ng page to be equ va ent to the Report ng Manager nterface ­offered by SSRS n stand-a one mode (you can sk p ahead to F gure 12-26 to see what t ooks ke) The ower ha f of the screen d sp ays a st of reports, data sources, and fo ders (wh ch s n t a y empty, of course) and the r bbon features a button for up oad ng RDL fi es from a oca d sk or network share There are a so buttons for creat ng fo ders and shared data sources, as we as sett ng perm ss ons on the former The Manage button a ows you to create and ma nta n SAR user accounts (reca that th s must be tended to separate y from SQL Azure account management) Once you are done w th these adm n strat ve tasks, you shou d h gh ght the web serv ce URL and copy t nto the c pboard, us ng the Ctrl+C keyboard shortcut Now paste t somewhere safe, as you w need t to configure the dep oyment server’s address when you pub sh your authored reports Th s w enab e you to dep oy your reports d rect y to SAR from the SSRS author ng too s, rather than manua y up oad ng your RDL fi es n the W ndows Azure Management Porta

Report Authoring Now a you have to do s create your report! But remember, n order for your report to be SARcompat b e, a of ts data sources must be SQL Azure databases As such, when you define your data source, make sure to se ect SQL Azure, rather than SQL Server If you’re us ng the Report W zard to create your report, then n the Se ect The Data Source page, se ect SQL Azure from the Type dropdown st, as shown n F gure 12-24

Figure 12-24  Se ect ng SQL Azure as a report s data source type.

614  Part III  Applied SQL

After se ect ng SQL Azure as your data source type, c ck the Ed t button (or the Bu d button n ­ eport Bu der) and configure the data source as shown prev ous y F gure 12-15 (on the eft-hand R s de) and F gure 12-17 Make sure to spec fy a database, n add t on to the server deta s After your data source s defined, bu d out your Dataset as you wou d norma y Next, n the Choose the ­Dep oyment Locat on page of the Report W zard, se ect SQL Server 2008 R2 Or Later n the Report Server Vers on drop-down st and paste the web serv ce URL you cop ed ear er from the W ndows Azure Management Porta nto the Report Server ed t contro F na y, spec fy a dep oyment fo der that has been created, or w be created pr or to dep oyment (reca that you can create a fo der us ng the Create button n the Fo der group of the r bbon n W ndows Azure Management Porta ) A of th s s shown and h gh ghted n F gure 12-25

Figure 12-25  Dep oyment deta s for SQL Azure Report ng.

Note that, un ke n the typ ca SSRS case, the SAR server web serv ce works over an SSL ­connect on, us ng a URL beg nn ng w th https:// When you’re fin shed us ng the Report W zard, des gn your report as you norma y wou d for SSRS

Deploying Reports When t comes t me for dep oyment, th ngs are og ca y s mp e, but the mechan ca steps may not be obv ous at first If you’re us ng the Report ng Serv ces project too ng n V sua Stud o, r ght-c ck the project node n So ut on Exp orer and se ect Propert es from the context menu Confirm the TargetServerURL and TargetReportFolder propert es are set as they were configured n the Report W zard ( f you d dn’t use the w zard, then set them manua y now), and that a other TargetxxxFolder propert es are e ther b ank or set to appropr ate, ex st ng SAR fo ders Now dep oy the report(s)



Chapter 12  Mov ng to the C oud w th SQL Azure    615

If you’re us ng Report Bu der, the dep oyment procedure s a b t more s mp e In the Save As Report d a og box, just paste the web serv ce URL n the Name text box, c ck Save, nav gate to the des red dep oyment fo der, enter a report name n the Name text box, and then c ck Save aga n Regard ess of wh ch too you use, you w be requ red to supp y a user name and password, n a pop-up og n d a og box, to connect to the SAR server Log n w th your SAR credent a s (not your SQL Azure or L ve ID credent a s) and c ck OK When fin shed, v s t the Report ng screen n the W ndows Azure Management Porta ( f t’s a ready open, then refresh your browser) You shou d see your report(s) sted n the fo der(s) to wh ch t was (they were) dep oyed, as shown n F gure 12-26

Figure 12-26  Brows ng SQL Azure Report ng reports n the W ndows Azure Management Porta .

C ck the nk for your report, and t w

be d sp ayed n the browser

For a NET app cat on, you can use standard report ng contro s, but you w prov de the authent cat on credent a s necessary to og n to the SAR server

616  Part III  Applied SQL

need spec a code to

Note  The sample ASP.NET project provided with this chapter demonstrates this technique using the ASP.NET Report Viewer control. The code fetches the SQL Azure Reporting server URL, report path, user id, and password from the appSettings section of web.config. Just edit these to be appropriate for your own SQL Azure Reporting server and report, then run the application, and click the SQL Azure Reporting link at the top of the page for a complete practical demo. The code of interest is in Report.aspx.cs and ReportServerCredentials.cs.

Getting Your Bearings The use of SSRS too s to author SAR reports can be a b t d sor ent ng, n terms of what needs to be c oud-based, and what does not To make th s a b t more c ear, here are some th ngs to keep n m nd ■

■

■

Convent ona SSRS reports can be bu t for SQL Azure databases You don’t have to use SAR (Just make sure to configure your SQL Azure firewa ru es to perm t your SSRS server access to the database ) Such convent ona reports can be converted to SAR reports s mp y by dep oy ng them to your SAR server or us ng the Up oad feature n W ndows Azure Management Porta Just make ­certa n that any shared data sources used by the reports are dep oyed as we SAR reports can be ntegrated nto any app cat on, be t on-prem ses or n the c oud, as ong as the SAR URL and credent a s are proper y app ed and the proper SQL Azure author zat on sett ngs are configured

W th th s coverage of SQL Azure Report ng, you now have a comprehens ve set of nformat on for bu d ng app cat ons whose database features and, opt ona y, report ng features are c oud-based

Summary C oud comput ng, n genera , s ga n ng s gn ficant tract on n the IT marketp ace, but even as t does so, hybr d approaches that comb ne c oud serv ces w th on-prem ses techno ogy, are ­emerg ng as the most common pattern n c oud techno ogy adopt on And because SQL Azure s at once a ­c oud-centr c serv ce and a product featur ng s gn ficant commona ty w th on-prem ses SQL Server, t s very we su ted for th s market env ronment We’ve seen n th s chapter that SQL Azure can be used comp ete y on ts own, us ng browser-based too ng for prov s on ng, manag ng, des gn ng, and query ng We have d scussed how the ­Federat ons feature he ps SQL Azure databases sca e and accommodates the k nd of mu t -tenancy scenar os that occur frequent y n the c oud We even earned how SQL Azure Report ng prov des a compan on ­c oud-based report ng serv ce for data n SQL Azure databases But we a so saw how on-prem ses SQL Server too ng, nc ud ng SSMS and SSDT, can be used w th SQL Azure and how databases can rather eas y be m grated from on-prem ses env ronments to the

Chapter 12  Mov ng to the C oud w th SQL Azure    617

c oud, and v ce versa Reports aga nst SQL Azure data are des gned w th on-prem ses too s and can be dep oyed to on-prem ses SQL Server Report ng Serv ces nsta at ons or to SQL Azure Report ng App cat ons that work w th SQL Azure, and even SQL Azure Report ng, can a so run on-prem ses, n the c oud, or both SQL Azure ets you take your SQL Server sk set and move t to the c oud The product emp oys ­ va ab ty and part t on ng strateg es that are sens t ve to the c oud, but makes them easy for SQL Server a deve opers to use or, n some cases, mp ements them beh nd the scenes, present ng d ­ eve opers w th zero add t ona earn ng requ rements In genera , SQL Azure prov des an ­approachab e, econom ca , and ­flex b e on-ramp to c oud comput ng for SQL Server profess ona s The goa of th s ­chapter has been to convey that w th c ar ty and nsp re you to try the serv ce or even adopt t for ­product on use

618  Part III  Applied SQL

C hapter 1 3

SQL Azure Data Sync and Windows Phone 7 Development —Paul Delcogliano

O

ne major reason for the r se of mob e app cat ons n recent years s the r ab ty to prov de ­users ub qu tous access to data App cat ons for smartphones and tab ets de ver a seem ng y end ess stream of data to consumers The one requ rement for “data everywhere” s a re ab e connect on to e ther a corporate network or the Internet

There are severa downs des to need ng an ever-present connect on For one, the W -F (­w re ess networks) and 3 and 4G (ce phone networks) rad os on smartphones and tab ets qu ck y dra n the dev ce’s battery Second, a though we are gradua y gett ng c oser to g oba network coverage, we s mp y aren’t there yet Re ab e network connect ons are st hard to come by n certa n parts of the wor d Th rd, data p ans from mob e prov ders today genera y have a month y cap to the amount of data consumed App cat ons that constant y transm t data eat away at the month y a owance ke a ­contestant at a hot dog eat ng contest F na y, performance suffers due to the ncreased atency of the network connect on An occas ona y connected system s a so ut on arch tected to operate w thout depend ng on a constant network connect on The c ent app cat on stores data oca y and transm ts t to a server whenever a connect on s obta nab e, wh ch addresses near y a of the aforement oned downs des of a constant y connected app cat on By e m nat ng the need for a connect on, the remote user can turn off the W -F and 3G rad os on the r dev ces, wh ch ncreases battery fe The ack of a connect on s rre evant to the c ent ­component of an occas ona y connected system; the app cat on works whether a connect on s ­present or not Reduc ng the frequency of transm ss ons he ps to keep data rates under the a owance, ead ng to ower costs It a so eads to reduced atency, wh ch ncreases performance as we Occas ona y connected systems prov de benefits for consumer app cat ons and ne-of-bus ness app cat ons a ke Enterpr ses are extend ng trad t ona desktop app cat ons beyond the workstat on by bu d ng mob e counterparts Mob z ng an app cat on untethers emp oyees from the r desks and a ows them to work at remote ocat ons wh e away from the office

619

Characteristics of an Occasionally Connected System Bu d ng an app cat on that operates whether connected or not requ res a d fferent arch tecture than a “trad t ona ” app cat on An app cat on des gned to work w thout a cont nuous network c­ onnect on has severa character st cs that are not present n a typ ca desktop app cat on An occas ona y ­connected system must be ab e to manage changes made wh e d sconnected, reso ve data confl cts that ar se dur ng synchron zat on, and update the oca data store appropr ate y Co ect ve y, these processes are known as data synchronization The system’s ab ty to synchron ze data s ts pr mary character st c A secondary character st c stems from the m tat ons of the dev ces the system runs on The ­dev ces are typ ca y smartphones, tab ets, and aptops w th s gn ficant y ess storage, memory, battery fe, and process ng power, than the r b g cous n, the desktop PC The r sma er d sp ay area trans ates to ess screen rea estate ava ab e for v ew ng the app cat on An occas ona y connected system must be capab e not on y of manag ng dev ce resources, but a so d sp ay ng nformat on effic ent y g ven the constra ned screen space

Data Management The very nature of work ng wh e d sconnected means the data on the dev ce s a copy of data from some centra database In an occas ona y connected scenar o, users rece ve cop es of the data they need for the app cat on to funct on wh e they are d sconnected from the network When a ­connect on becomes ava ab e, data synchron zat on occurs dur ng wh ch the c ent up oads any changes made on the dev ce to the server, and down oads any updated data from the server back to the dev ce When the network connect on s severed, a data needed to run the app cat on ­product ve y must be ava ab e on the dev ce Data synchron zat on can transp re n one of two modes, one-way and two-way One-way s­ ynchron zat on sends data from server to c ent or from c ent to server It s typ ca y used for ookup or read-on y data, ke a st of products or mov e names One-way synchron zat on s a so a great so ut on for creat ng data cache scenar os where data s co- ocat ng around the g obe Putt ng data c oser to your users eads to better app cat on respons veness A two-way synchron zat on (often referred to as b d rect ona synchron zat on) s one n wh ch the data changes on the server are sent to the c ent, and data changes on the c ent are sent to the server Two-way synchron zat on requ res the c ent app cat on to store changes on the dev ce unt they can be up oaded to the server For examp e, a mob e worker m ght down oad a customer st to the r W ndows Phone and then update customer phone numbers throughout the day A of the changed customer nformat on s stored n the phone’s oca database When an Internet connect on s estab shed at the end of the day, a changed customer data s up oaded to the centra server, and any new customer nformat on s down oaded to the dev ce In a two-way scenar o, there s a h gher chance for data confl cts to occur, because data on the dev ce has a tendency to turn sta e Changes made to a part cu ar customer contact record may be made by two d fferent users on two d fferent dev ces When these changes are up oaded to the server a confl ct may occur between the d fferent changes Any confl cts can be hand ed and reso ved by the synchron z ng protoco 620  Part III  Applied SQL

Typ ca y, the synchron zat on prov der s respons b e for coord nat ng changes to data between the c ent and the server In the next sect on, we’ ntroduce a new c oud-based synchron zat on ­prov der M crosoft s mak ng ava ab e to deve opers ca ed SQL Azure Data Sync

Getting to Know SQL Azure Data Sync SQL Azure Data Sync (Data Sync) s a new synchron zat on serv ce bu t on top of the M crosoft Sync Serv ces Framework Data Sync s part of the Azure fam y of serv ces and s ava ab e to W ndows Azure subscr bers It s capab e of perform ng synchron zat ons n a var ety of scenar os nc ud ng SQL Azure to SQL Azure, SQL Azure to SQL Server, or SQL Server to SQL Azure

Important  Microsoft frequently updates its cloud services. As such, costs, limitations, ­features, the tooling user interface, or even the branding of SQL Azure and/or SQL Azure Data Sync, as described in this chapter, may have evolved by the time you read it. You manage and configure Data Sync v a the W ndows Azure Management Porta The porta ’s w zards and graph ca d a ogs fac tate creat ng, schedu ng, and mon tor ng synchron zat ons The porta a so prov des too s for group ng the data to be synchron zed, and estab sh ng the servers part c pat ng n a synchron zat on job Data Sync s capab e of b d rect ona synchron zat on of data between two or more geograph ca y d str buted SQL Azure and SQL Server databases The SQL Azure databases cou d be hosted n any of the W ndows Azure data centers around the g obe The SQL Server databases may a so be geograph ca y ­d str buted Data Sync must have at east one SQL Azure database wh ch acts as the hub for synchron zat ons (we d scuss Data Sync hubs and Data Sync requ rements ater n the chapter) As of th s wr t ng, Data Sync s prov ded as a “prev ew” re ease ava ab e to a SQL Azure subscr bers, and so nformat on n th s chapter s subject to change A though t s a prev ew re ease, Data Sync shou d not be used for product on app cat ons In ts current ncarnat on, Data Sync cannot perform mob e to c oud data synchron zat on d rect y—so th s chapter demonstrates the use of Data Sync to synchron ze between SQL Server (on-prem se) and SQL Azure (c oud), comb ned w th the Open Data Protoco (OData) and WCF Data Serv ces to commun cate between the c ent component runn ng on the mob e dev ce and SQL Azure

Capabilities and Features Data Sync supports three d fferent sync d rect on opt ons B -D rect ona , Sync From The Hub, and Sync To The Hub A sync d rect on determ nes the flow of data between databases dur ng synchron zat on The ­B -D rect ona opt on perm ts the changes to be synchron zed between the se ected database and the hub database n both d rect ons The Sync From The Hub opt on perm ts the changes to be synced from the hub database to the se ected database on y Changes n the se ected database are not up oaded to the hub And the Sync To The Hub opt on perm ts the changes to be synced from the se ected database to the hub database on y Changes on the hub database are not down oaded to the se ected database

Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    621

Each sync d rect on opt on s appropr ate for a d fferent set of scenar os The Sync From The Hub scenar o s appropr ate for rep cat ng data for fa over or sca ng, or to mprove app cat on response t me by co- ocat ng data around the g obe The Sync To The Hub opt on can be used to gather data from remote ocat ons and aggregate that data as a store for a data warehouse or on ne ana yt ca process ng (OLAP) The B -D rect ona opt on s appropr ate for scenar os where data confl cts can occur and need to be reso ved In these scenar os, Data Sync can be configured to manage the data confl cts us ng a confl ct reso ut on po cy In b -d rect ona synchron zat on scenar os, confl ct reso ut on can detect and reso ve confl cts between mu t p e ocat ons attempt ng to mod fy the same records Confl ct reso ut on po c es are configured to determ ne what act ons to take when a confl ct s detected Data Sync’s opt ona row fi ter ng feature a ows sma er sets of data to be transm tted ­dur ng s­ ynchron zat on A row fi ter works essent a y as a T-SQL WHERE c ause; t se ects a sub-set of rows from a tab e F ters mprove performance and he p to keep costs down by synchron z ng on y ­re evant data, not an ent re tab e

Data Sync Terminology Before d v ng n, et’s estab sh some bas c Data Sync terms that we use n th s chapter’s Data Sync coverage Some of these terms w be descr bed n greater deta throughout the chapter Tab e 13-1 defines the term no ogy Table 13-1  Common Data Sync terms Term

Definition

Sync Group

A Sync Group s a group of SQL Azure and SQL Server databases configured for synchron zat on w th each other.

Hub Database

The hub database s the centra database n a Sync Group. t must be a SQL Azure database.

Member Database

A member database s any SQL Azure database, exc ud ng the hub, or on prem se SQL Server that s part of the Sync Group.

C ent Sync Agent

The SQL Azure C ent Sync Agent s a W ndows serv ce nsta ed on the on prem se database server. t enab es commun cat on between the on prem s SQL Server database and the SQL Azure hub database.

Sync Schedu e

A Sync Schedu e can be created to execute a data synchron za t on at defined nterva s. The schedu ed synchron zat on task s ca ed a Sync Job.

Sync Loop

A Sync Loop s a cond t on that occurs when one Sync Group s synchron zat on tr ggers the synchron zat on of another Sync Group.

Synchron zat on Confl ct

A synchron zat on confl ct occurs when changes take p ace on the same p ece of data n two or more databases n the Sync Group; for examp e, when two users update the same customer record at the same t me.

622  Part III  Applied SQL

Sync Groups A Sync Group s a co ect on of databases grouped together for the purpose of synchron z ng shared data A Sync Group s compr sed of severa e ements Co ect ve y, the e ements define ■

the databases n the group

■

the data to be synchron zed

■

the confl ct reso ut on po cy

■

an opt ona synchron zat on schedu e

The databases n a Sync Group are arranged n a hub-spoke topo ogy Data Sync mandates that a SQL Azure database serves as the hub Once the hub s se ected, t cannot be changed to a d fferent database or removed Any comb nat on of SQL Azure and SQL Server databases make up the spokes, wh ch are ca ed members, n the group A member database can be added to or removed from a Sync Group at any t me as ong as data synchron zat on s not act ve y tak ng p ace The hub/spoke arrangement a ows SQL Azure Data Sync to support d fferent synchron zat on scenar os, nc ud ng c oud-to-c oud synchron zat ons and synchron zat ons nvo v ng both c oud and on-prem se SQL Server databases

Synchronization The data to be synchron zed n a Sync Group s ca ed a dataset. Datasets are composed of tab es, ­co umns, and any fi tered rows When a synchron zat on job runs, on y the se ected tab es, co umns, and rows are synchron zed For examp e, you can se ect the Movies tab e a ong w th ts Name and Genre co umns, and app y the fi ter Theater = ‘Roxy’ so on y mov es from the Roxy theater get ­synchron zed Once the Sync Group s dep oyed, the tab es n a dataset cannot be added or removed The Sync Group must be re-created to change a dataset Data Sync supports most common SQL Server data types, nc ud ng a the str ng, date/t me, and numer c types, as we as sql variant, table, uniqueidentifier, and xml However, some of the more ­spec a zed types are not supported For a comp ete st, see http://msdn.microsoft.com/library/ hh667319.aspx) Data n a Sync Group can be synchron zed manua y or accord ng to a schedu e Perform ng a manua synchron zat on s a s mp e matter of c ck ng a button on the W ndows Azure ­Management Porta The Sync Group s capab e of sett ng up a schedu e for a recurr ng synchron zat on; th s s known as configur ng a Sync Schedu e A Sync Schedu e creates a Sync Job wh ch s executed at a spec fied nterva The nterva s set n un ts of m nutes, hours, days, or months and spec fies the e apsed t me between synchron zat on jobs The nterva va ue must fa between 5 m nutes and 1 month The nterva does not spec fy the oca or nternat ona t me that the Sync Job executes You can a ways nvoke a manua synchron zat on, even f you have configured a Sync Schedu e Synchron zat on occurs on y among the databases n the assoc ated Sync Group Dur ng ­synchron zat on, the rows n each of the tab es part c pat ng n the Sync Group are updated accord ng



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    623

to a part cu ar workflow F rst, the changes n the member databases are synchron zed w th the hub database Then, the changes n the hub database are synchron zed w th the member databases Data confl cts can occur n any synchron zat on scenar o n wh ch changes occur n two or more databases Imag ne someone updat ng a row n a tab e n the hub database, and another user ­updat ng that same row n a member database Under th s cond t on, Data Sync must determ ne the appropr ate vers on of the record to reta n A Sync Group makes th s dec s on us ng a confl ct ­reso ut on po cy

Conflict Resolution The Sync Group’s confl ct reso ut on po cy determ nes the rows to be reta ned and the rows to be d scarded whenever the same record s changed n d fferent databases The confl ct reso ut on po cy cannot be changed once the first synchron zat on has taken p ace If you need to change the po cy, you have to re-create the Sync Group Data Sync prov des two opt ons for reso v ng confl cts Hub W ns and C ent W ns If the po cy s set to Hub W ns, changes n the hub record a ways overwr te changes n the member database(s) Under th s sett ng, the first row change wr tten to the hub s kept Subsequent attempts to wr te to the same row n the hub are gnored Because a other changes to the row are gnored, the first change wr tten to the hub s the change that s d str buted to a members n the group If the po cy s set to C ent W ns, changes to a record n the member database are wr tten to the hub If mu t p e members have changed the record, the ast member’s change s wr tten to the hub, and th s vers on of the record s the one that gets synchron zed to a member databases

The Client Sync Agent The SQL Azure C ent Sync Agent s a ghtwe ght W ndows serv ce that s respons b e for commun cat ng w th the Data Sync serv ce The agent s nsta ed to a server on your network and prov des the Data Sync server d rect access to your on-prem se SQL Server databases The agent s respons b e for commun cat on between the on-prem se database and the SQL Azure hub database It grants access to your SQL Server database w thout the need to open the firewa Each Data Sync server requ res ts own agent, and an agent can connect to mu t p e SQL Server databases from d fferent servers n your enterpr se Through the agent, Data Sync can access the tab es n your SQL Server database The C ent Sync Agent supports encrypted, b d rect ona commun cat on between the on-prem se SQL Server database and the SQL Azure hub database A sma W ndows app cat on s nc uded w th the agent nsta at on wh ch you can use to manage the agent The app cat on’s user nterface (UI) ets you ma nta n reg stered databases and test connect v ty to the Data Sync serv ce We d scuss sett ng up the agent n the “Creat ng the Sync Group” sect on ater n the chapter

624  Part III  Applied SQL

SQL Azure Data Sync Considerations Whenever eva uat ng a new p ece of software or p atform, you need nformat on on certa n aspects such as secur ty, performance, and costs Th s sect on touches on these top cs as they perta n to Data Sync Th s nformat on w he p you dec de whether or not Data Sync s an appropr ate so ut on for your un que scenar o

Data Sync Security Data Sync prov des secur ty n two ways, us ng encrypt on and authent cat on Data s encrypted whenever t s stored n the c oud or transm tted between components Stored data nc udes tems such as og n credent a s for SQL Azure connect ons, configurat on data for the c ent agent, and ­serv ce credent a s for both SQL Azure system databases and W ndows Azure system storage Data Sync a so enforces authent cat on whenever connect ons are made In the Data Sync mode , data s transm tted between severa components; for nstance, between SQL Azure and SQL Server Whenever a connect on s made between two components, the connect on se ­ ncrypted Encrypted connect ons are estab shed between the Data Sync serv ce and the fo ow ng ­components ■

the SQL Azure system database

■

the W ndows Azure system storage

■

the c ent agent

■

the W ndows Azure Management Porta

Data Sync emp oys authent cat on across a connect on po nts to prevent unauthor zed users and serv ces from access ng the system and data The W ndows Azure Management Porta authent cates ­users us ng W ndows L ve IDs and the W ndows Azure subscr pt on database Connect ons made between ­components are authent cated us ng cert ficates The c ent agent respons b e for synchron z ng data ­between the hub and member database performs authent cat on us ng a un que token generated when the agent s nsta ed On y a user w th Adm n strator pr v eges can manage the agent

Performance and Costs Synchron zat on performance s affected by many factors When us ng Data Sync, there are severa th ngs you can do to ensure you ga n the best poss b e performance, start ng w th the ocat on of your SQL Azure databases The geograph c ocat on where your SQL Azure databases are hosted can mpact both the effic ency and cost of your synchron zat ons To m n m ze atency, ocate your SQL Azure databases n data centers as c ose to your SQL Server databases as poss b e Next, m t your datasets to nc ude just the tems you need to sync Data Sync doesn’t requ re the ent re database to part c pate n a Sync Group, so you shou d a ways se ect the fewest tab es and co umns poss b e and app y an app cab e row fi ter when you configure your dataset Th s pract ce mproves performance by reduc ng the overa pay oad of a Sync Job



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    625

Another cons derat on to bear n m nd s the frequency w th wh ch a Sync Job occurs If a Sync Job attempts to synchron ze a Sync Group that has not yet comp eted a pr or synchron zat on, the attempt fa s When p ann ng a Sync Schedu e, take care that you set the nterva suffic ent y arge enough to ensure that synchron zat on comp etes before the next Sync Job s started The Sync Schedu e can affect your SQL Azure costs as we A though M crosoft current y offers Data Sync as a free serv ce, SQL Azure fees are charged accord ng to the amount of data moved nto and out of a data center To m n m ze costs, you shou d cons der d v d ng data nto separate Sync Groups accord ng to the frequency w th wh ch the data changes Vo at e data shou d be synchron zed at a h gher frequency than stat c or ookup data Part t on ng Sync Groups n th s way a ows you to ­configure an opt ma schedu e that he ps reduce costs by send ng data ess frequent y One p tfa to avo d when sett ng up mu t p e Sync Groups s a cond t on known as a Sync Loop A Sync Loop occurs when a change n a record n one Sync Group s re-wr tten to the same record by a second Sync Group, s m ar to a c rcu ar reference Th s h gh y undes rab e cond t on can potent a y enter an ­nfin te oop and consume enough resources to s gn ficant y degrade performance Furthermore, you w pay fees for mov ng data nto and out of SQL Azure unnecessar y You can avo d Sync Loops n a few ways ■

■

■

Des gn your Sync Groups such that a oop cannot ex st; that s, don’t et the same tab e be synchron zed by two d fferent Sync Groups A so, avo d add ng the same database more than once to the same Sync Group, even f the database s reg stered w th a d fferent C ent Sync Agent If mu t p e Sync Groups synchron ze the same tab e, use row fi ter ng to prevent the same rows from be ng updated by a of the Sync Groups Spec fy a sync d rect on such that a oop cond t on cannot ex st

Creating an Occasionally Connected System At the t me of th s wr t ng, Data Sync does not support d rect synchron zat on between mob e dev ces and SQL Server or SQL Azure A though th s funct ona ty may be added to a future vers on of Data Sync, for now, you s mp y cannot bu d synchron zat on capab t es d rect y nto the phone ­app cat on us ng Data Sync But cons der as we that you may not want to expose your SQL Server database v a the Internet d rect y to the mob e app cat on Add ng a SQL Azure database to the des gn to act as a buffer between mob e c ents and the on-prem se “back office” database ntroduces an extra ayer of secur ty to the so ut on W th th s arch tecture, you protect SQL Server beh nd the corporate firewa and nsta the agent on a pub c fac ng server or per meter network (a so known as DMZ, or ­dem tar zed zone) Such a configurat on a ows the agent to commun cate w th the database for sync ng w th SQL Azure, and restr cts a other access to your SQL Server Th s chapter demonstrates how to bu d such a so ut on by tak ng a deep d ve nto SQL Azure Data Sync, W ndows Azure, and the W ndows Phone SDK The so ut on a so ncorporates two add t ona techno og es, SQL Azure and WCF Data Serv ces, each of wh ch we have ded cated ent re chapters to n th s book 626  Part III  Applied SQL

The FlixPoll Application F xPo s the fict t ous survey app cat on that demonstrates how a these var ous techno og es work together n an occas ona y connected system A F xPo user nterv ews patrons as they ex t a mov e theater and records the r responses on a W ndows Phone 7 dev ce The c ent app cat on on the phone requ res no network connect on to store data oca y Once a connect on s ava ab e, the c ent up oads data to, and down oads data from, SQL Azure Th s s ach eved by ca ng the F xPo serv ce, mp emented w th WCF Data Serv ces and hosted n W ndows Azure The serv ce, n turn, obta ns data from the SQL Azure hub database At that po nt, Data Sync s respons b e for synchron z ng changes between SQL Azure and the on-prem se SQL Server database Data Sync updates the on-prem se ­database and reso ves any confl cts between data up oaded from the phone and data entered by ­users n the back office

Note  This application is merely a learning tool that demonstrates how to build an ­occasionally connected system with WCF Data Services and SQL Azure Data Sync. FlixPoll uses basic WCF Data Services that do not handle conflict resolution between the phone ­client and SQL Azure. Changes uploaded by FlixPoll users are propagated out to the ­member database, and these changes overwrite any changes made to the on-premise SQL Server. Conflict resolution between SQL Azure and on-premise SQL Server is handled by policies defined in SQL Azure Data Sync, as we explain later in the chapter. When the F xPo c ent app cat on starts, t checks the oca dev ce for prev ous y down oaded mov e data that s requ red to run the app cat on If the data s ava ab e, t s retr eved and d sp ayed; otherw se, a connect on s made to the serv ce and the data s down oaded The user then ­nterv ews mov e-goers, stor ng the gathered po ng nformat on oca y on the dev ce The user can a so ­update mov e nformat on oca y Then the user n t ates a connect on w th the serv ce to up oad the r changes to SQL Azure

Microsoft Sync Framework Toolkit The prev ous ed t on of th s book showed how to create an occas ona y connected system for SQL Server 2008 us ng the M crosoft Sync Serv ce Framework S nce then, the M crosoft Sync Serv ce Framework has gone through severa transformat ons, and M crosoft has deferred new deve opment of that framework n favor of SQL Azure Data Sync The ast re ease of the ­framework was vers on 4 0, wh ch rema ns n beta The Sync Serv ce Framework can st be used to add synchron zat on support to your ­app cat ons In fact, M crosoft has re eased the M crosoft Sync Framework Too k t, based on Sync Framework vers on 4 0, to support that not on However, g ven M crosoft’s dec s on to suspend ­deve opment of the framework, we have wr tten th s brand new chapter that gu des you n ­M crosoft’s current d rect on for data synchron zat on, SQL Azure Data Sync To find out more about the M crosoft Sync Framework Too k t, v s t http://go.microsoft.com/fwlink/p/?LinkId=235330)



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    627

The rema nder of th s chapter s d v ded nto three major sect ons, and each sect on d ves nto the deta s of one major component of the app cat on Th s first sect on wa ks you through sett ng up the app cat on database and configur ng SQL Azure Data Sync In the second sect on, you w use WCF Data Serv ces to create a serv ce ( tse f hosted n W ndows Azure) that exposes the SQL Azure data In the fina sect on, you w bu d the W ndows Phone app cat on that consumes the data serv ce

Creating the FlixPoll Databases Your first object ve s to bu d the databases There are two databases n the F xPo app cat on, one on-prem se SQL Server database and a second one hosted n SQL Azure The on-prem se database s named FlixPollOnPrem Th s s the app cat on’s member database The SQL Azure database s the hub, and s named FlixPollOnCloud For demonstrat on purposes, we kept both databases very s mp e St , we represent tab es ke those you see n a typ ca database there are ookup tab es as we as tab es for stor ng transact ona data To create the FlixPollOnPrem database, open SQL Server Data Too s (SSDT) n V sua Stud o or SQL Server Management Stud o (SSMS) to connect to your oca SQL Server 2012 nstance Then open a new query w ndow and run the scr pt n L st ng 13-1 to create the database and popu ate ts tab es w th samp e data Listing 13-1  Creat ng the FlixPollOnPrem database and tab es, and nsert ng data.

CREATE DATABASE FlixPollOnPrem GO USE FlixPollOnPrem GO CREATE TABLE Movie( MovieId int IDENTITY(1,1) NOT NULL CONSTRAINT PK_Movie PRIMARY KEY, Name varchar(100) NOT NULL, YearReleased smallint NOT NULL) GO CREATE TABLE Inquiry( InquiryId int IDENTITY(1,1) NOT NULL CONSTRAINT PK_Inquiry PRIMARY KEY, Question varchar(100) NOT NULL) GO CREATE TABLE Response( ResponseId int IDENTITY(1,1) NOT NULL CONSTRAINT PK_Response PRIMARY KEY, MovieId int NOT NULL, InquiryId int NOT NULL, Answer bit NOT NULL CONSTRAINT DF_Answer DEFAULT(0) ) GO INSERT INTO Movie (Name, YearReleased) VALUES ('SQL Wars', 2011), ('Like Water for SQL', 2011),

628  Part III  Applied SQL

('SQL Mission Impossible', 2011), ('Amazing SQL-man', 2012), ('Chipmonks, The SQL', 2012), ('The SQL Avengers', 2012) GO INSERT INTO Inquiry (Question) VALUES ('Did you like the movie?'), ('Is the movie a potential award nominee?'), ('Did you pay a fair ticket price?') GO

Now create an empty FlixPollOnCloud database The tab es w be created and popu ated ater as part of the first synchron zat on job Open a connect on to your SQL Azure server and execute the fo ow ng statement CREATE DATABASE FlixPollOnCloud

SQL Azure s covered n Chapter 12 Refer to that chapter for a deta ed d scuss on exp a n ng how to use the Azure Management Porta to create your own c oud databases on SQL Azure The ­“Gett ng Set Up” sect on n that chapter descr bes how to prov s on your server and create the firewa ru e necessary to access a SQL Azure server When creat ng your SQL Azure server for use w th the samp e app cat on, be sure you check the opt on A ow Other W ndows Azure Serv ces To Access Th s Server Do ng so sets up the firewa ru e that a ows access to your SQL Azure server You w need th s access ater when runn ng your WCF Data Serv ces If you do not create th s firewa ru e, ne ther Data Sync nor WCF Data Serv ces w have access to the SQL Azure server

Note  MSDN subscribers receive free access to Windows Azure. The level of access you receive is dependent upon your subscription level. See https://www.windowsazure.com/en-us/pricing/member-offers/msdn-benefits for details.

Prerequisites To bu d th s so ut on, you w



need the add t ona software and subscr pt ons sted be ow

■

A W ndows L ve ID

■

A W ndows Azure subscr pt on, w th W ndows Azure and SQL Azure accounts

■

W ndows Azure SDK for NET

■

W ndows Azure Too s For M crosoft V sua Stud o

■

W ndows Phone 7 1 SDK

■

S ver ght Too k t for W ndows Phone Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    629

Configuring SQL Azure Data Sync Sett ng up SQL Azure Data Sync s a two-step process that you accomp sh us ng the W ndows Azure Management Porta The first step s to prov s on the SQL Azure Data Sync server Then you create a Sync Group These steps are further d v ded nto many sma er steps, as we exp a n ahead

Provisioning the SQL Azure Data Sync Server Your first step to comp ete n sett ng up SQL Azure Data Sync s to prov s on a SQL Azure server The prov s oned server s referred to as the Data Sync server A Data Sync server can host mu t p e Sync Groups, but you can on y prov s on one SQL Azure server as the Data Sync server per W ndows Azure subscr pt on Prov s on ng a server requ res two tems, an act ve W ndows Azure p atform account and a SQL Azure server To prov s on a server now, og n to the W ndows Azure porta at http://windows.azure.com us ng your W ndows L ve ID On the eft vert ca menu, c ck Data Sync Then start the prov s on ng w zard by ­e ther c ck ng on the Prov s on button n the too bar or the arge Prov s on Data Sync Prev ew Server ­button shown n F gure 13-1

Figure 13-1  The Prov s on Data Sync Prev ew Server button n the W ndows Azure porta .

Prov s on ng a server s a stra ghtforward, three-step process Step one s to accept the terms of use agreement Read through the agreement, check the I Agree To The Terms Of Use Statement Above check box, and then c ck Next to cont nue to step two Step two asks you to se ect the SQL Azure subscr pt on you want to prov s on Expand the S­ ubscr pt on drop-down st and se ect a subscr pt on from the st C ck Next to move on to the fina step In the fina step, you se ect a reg on where the Data Sync server w be hosted Azure uses ­reg ons to manage the phys ca ocat on of your app cat on and data A reg on represents a M crosoft data center where your app cat on or database can be dep oyed Expand the Reg on drop-down st and se ect an appropr ate reg on S nce you are bu d ng a samp e app cat on, the reg on you se ect doesn’t matter much For product on app cat ons however, you shou d cons der se ect ng a reg on where most of your app cat on’s users are ocated, or n c ose prox m ty to your data center As we ment oned, the ma n benefit n do ng so s reduced network atency Prov s on ng a server c osest to your users ncreases your app cat on’s response t me by decreas ng the d stance the data must trave to ts dest nat on C ck the F n sh button after se ect ng the reg on

630  Part III  Applied SQL

Creating the Sync Group W th the Data Sync server prov s oned, you are ready to create the Sync Group A Sync Group s ­created n s x, h gh- eve steps 1. Supp y a name for the Sync Group 2. Add the SQL Server database to the group 3. Add the SQL Azure hub database 4. Set the Sync Schedu e and confl ct reso ut on po cy 5. Define the tab es, co umns, and rows that make up the dataset 6. Dep oy the Sync Group to the Data Sync server

F xPo uses a Sync Group composed of one SQL Azure database named FlixPollOnCloud (the hub) and one SQL Server database named FlixPollOnPrem (the member) From w th n the W ndows Azure Management Porta c ck on the Data Sync button on the eft-hand s de of the porta Se ect the subscr pt on under wh ch you want to create the Sync Group from the tree v ew n the porta ’s eft pane ( f you don’t se ect the subscr pt on now, you w be prompted to se ect one as the first step n the w zard) Then c ck the arge Sync between On-Prem se And SQL Azure Databases button shown n F gure 13-2

Figure 13-2  Th s button n the porta s center pane starts the Sync Group w zard to create a Sync Group ­composed of SQL Azure and SQL Server databases.

More Info  The portal actually provides three different ways to create a Sync Group. The Sync Between On-Premise And SQL Azure Databases button starts a wizard with steps for creating a SQL Azure to SQL Server Sync Group, as in our FlixPoll solution. The Sync Between SQL Azure Databases starts a different wizard for creating a Sync Group consisting only of SQL Azure databases. Clicking the Create button on the toolbar allows you to add any combination of SQL Azure and SQL Server databases to a Sync Group, in any order.



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    631

Step 1: Name the Sync Group A good ru e of thumb s to prov de a name that s mean ngfu and descr bes the purpose of the Sync Group, wh e a so be ng un que Go ahead and name the Sync Group FlixPollSyncGroup After ­enter ng the Sync Group name, move on to the next step by c ck ng the arrow above and to the r ght of the Sync Group Name textbox or by press ng the Enter key

Step 2: Add the On-Premise Database A w zard now gu des you through the process of add ng a SQL Server member database to the group As you advance through the w zard, you w ■

Add a new SQL Server database to the group

■

Insta and configure a new C ent Sync Agent

■

Reg ster your SQL Server database w th the agent

■

Se ect the SQL Server database for nc us on n the group

Add a new SQL Server database to the group  Start the process of add ng the member ­database by c ck ng on the “Add SQL Server database” con ( t’s the mage of the database w th a p us s gn ­embedded ns de) Th s opens the Add Database To Sync Group d a og box Th s s the first Sync Group you are creat ng, so se ect the Add A New SQL Server Database To The Sync Group opt on, and then se ect the B -D rect ona opt on n the Sync D rect on drop-down st Th s opt on spec fies that data changes flow n both d rect ons between the hub and member databases C ck Next to cont nue Install and configure a new Client Sync Agent  Now the w zard ets you nsta a new C ent Sync Agent, wh ch you must do because the prov s oned Data Sync server does not a ready have an agent nsta ed Choose the Insta a New Agent opt on Th s br ngs up the Insta A New Agent d a og box, as shown n F gure 13-3 C ck the Down oad button to beg n down oad ng the nsta er

Note  Be sure to review the prerequisites for using the agent on the Client Sync Agent download page.

632  Part III  Applied SQL

Figure 13-3  nsta ng a new C ent Sync Agent.

Run the nsta er when t fin shes down oad ng Accept the cense agreement and defau t ocat on for the nsta at on fo der You w then be prompted for a user account w th Adm n strator pr v eges Be sure to use a W ndows Serv ce account that has perm ss ons to connect to a of the SQL Server databases you want to reg ster Register your SQL Server database with the agent  After the agent software s nsta ed, return to the W ndows Azure Management Porta and enter FlixPollAgent for the agent name Next, c ck the Generate Agent Key button Th s generates and d sp ays an encrypted str ng that s requ red by the C ent Sync Agent C ck the Copy button to copy the key to the c pboard and then c ck Next to advance the w zard to the next step W th the agent key cop ed, you’ temporar y eave the porta to configure the agent Don’t c ose the porta , as you w return to t momentar y C ck Start A Programs M crosoft SQL Azure Data Sync and M crosoft SQL Azure Data Sync Agent Prev ew to start the agent UI, as shown n F gure 13-4



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    633

Figure 13-4  The C ent Sync Agent user nterface.

F rst, c ck the Subm t Agent Key button and paste the cop ed key nto the Agent Key textbox Then c ck OK You are now ready to reg ster the FlixPollOnPrem database w th the agent C ck the Reg ster button to open the SQL Server Configurat on d a og box When the d a og box opens, supp y the necessary og n credent a s Next, enter the name of your oca database server and enter FlixPollOnPrem for the database name Do not se ect the opt on of us ng SSL w th th s connect on, but cons der enab ng t n a product on env ronment Now c ck the Test Connect on button to ver fy your credent a s are entered proper y and can access the database server Then c ck the Save button to cont nue You can confirm that the FlixPollOnPrem database s reg stered when you see t appear n the agent UI form’s Status pane The ast th ng to do before c os ng the app cat on s to c ck on the P ng Sync Serv ce button to test your connect on to the Data Sync serv ce When you rece ve a message stat ng your p ng was successfu , c ose the agent UI app cat on

Note  If you cannot ping the sync service and your enterprise uses a network proxy, you may need to set the appropriate firewall rules to allow outbound connections to the sync service. In some cases you can resolve the problem by simply toggling the proxy server ­setting in Internet Explorer. Select the SQL Server database for inclusion in the group  Return back to the W ndows Azure porta You shou d see the form shown n F gure 13-5

634  Part III  Applied SQL

Figure 13-5  The fina form n the w zard adds a SQL Server database to a Sync Group.

V sua y ver fy that the agent name shown n the second sect on reads FlixPollAgent, then c ck the Get Database L st button Expand the Se ect Database drop-down st n the th rd sect on, and you shou d see the FlixPollOnPrem database n the st Se ect t and c ck the F n sh button

Step 3: Select the SQL Azure Hub Database To comp ete th s step, you w need your SQL Azure server’s adm n strator og n credent a s and ts fu y qua fied DNS Name (wh ch a ways beg n w th a un que dent fier random y ass gned just to you, fo owed by .database.windows.net) You can get the SQL Azure server’s DNS name from the porta by c ck ng on the Database tab Once you have the DNS name and og n credent a s, c ck on the “Add A SQL Azure Database” con ( t’s the mage of the database w th the cr sscross ng nes embedded ns de) You w be prompted to supp y your SQL Azure server name (th s s the DNS name), user ID, and password After enter ng your server and og n credent a s, c ck on the Database Name drop-down st, se ect the FlixPollOnCloud database, and then c ck Add If you don’t see the FlixPollOnCloud database n the drop-down st, type t manua y



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    635

Step 4: Configuring Sync Schedule and Conflict Resolution Step four cons sts of two tasks The first s to create a Sync Schedu e, wh ch s opt ona and enab ed by defau t Enter 10 as the frequency and se ect M nutes from the nterva drop-down st to have F xPo nvoke a Sync Job every 10 m nutes The second task s to se ect the confl ct reso ut on po cy, wh ch s a requ rement for a Sync Group Go ahead and set the po cy to Hub W ns so that the data up oaded to the hub from the F xPo UI overwr tes changes made to the on-prem se database

Step 5: Define the Dataset Th s s the fina step before dep oy ng the Sync Group to the Data Sync server C ck the Ed t Dataset button to open the Define Sync Dataset d a og The FlixPollOnCloud database s a ready se ected when the d a og appears Reca that you created th s database ear er, and that t s comp ete y devo d of schema or data Because the FlixPollOnCloud database s empty, no tab es are shown as ava ab e for the dataset Sw tch to the FlixPollOnPrem database by se ect ng t from the drop-down st Th s s a fu y popu ated database whose tab es now appear n the Tab es st contro under the drop-down st Tab es must meet spec fic requ rements to be e g b e to part c pate n a Sync Group For examp e, tab es w thout pr mary keys cannot be added

Tip  To discover incompatibilities earlier in the process, it is advantageous to add the source database to the Agent prior to provisioning. You can then resolve issues such as ­unsupported data types or missing primary key constraints by modifying the offending table’s schema. Then just click the Refresh Schema button to reload and revalidate the schema. Se ect the Movie, Inquiry, and Response tab es from the st After se ect ng each tab e, not ce that the st contro to the r ght of the tab es shows a of the co umns ava ab e for synchron zat on Each tab e shou d have a co umns se ected Now set up a fi ter on the Movie tab e so that on y the mov es from the year 2012 w be ­nc uded n the dataset Th s exc udes rows for years other than 2012 from be ng synchron zed Se ect the Movie tab e In the st of fie ds, p ace a check n the Filter co umn next to the YearReleased fie d At the bottom sect on of the form, you w see a ser es of contro s for bu d ng a fi ter c ause The ­YearReleased fie d shou d be pre-se ected; f t sn’t, se ect t now from the drop-down st Set the Operator to Equa s (=) and enter 2012 n the Va ue textbox The Define Sync Dataset d a og shou d ook ke F gure 13-6 C ck OK to comp ete step five The porta d sp ays a message nstruct ng you to dep oy the F xPo Sync Group Before you dep oy, qu ck y rev ew the deta s d sp ayed n the porta and shown n F gure 13-7 Ver fy that the va ues for the sync schedu e and confl ct reso ut on po cy are correct Confirm the tab es you nc uded n the dataset are found n the Synced Tab es drop-down st a ong w th each tab e’s assoc ated co umns and the fi ter on the YearReleased co umn In the ma n pane, you’ not ce that ne ther the FlixPollOnCloud nor the FlixPollOnPrem databases are dep oyed 636  Part III  Applied SQL

Figure 13-6  The comp eted Define Sync Dataset d a og box.

Figure 13-7  The W ndows Azure Management Porta ma n pane after creat ng the Sync Group.



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    637

A tt e mag c s about to happen To watch t, open a connect on to your SQL Azure server us ng SSDT or SSMS Expand the Tables node under the FlixPollOnCloud database and ver fy that no tab es ex st

Step 6: Deploying the Sync Group From w th n the W ndows Azure Management Porta , c ck the Dep oy button n the too bar Var ous cons w beg n to update the r status to keep you nformed that someth ng s occurr ng wh e the dep oyment process s runn ng After dep oyment s comp ete, you shou d see a status of “good” next to each database n the group

Note  The dataset cannot be changed once the Sync Group is deployed. Go back to SSDT/SSMS and refresh the Tables node under the FlixPollOnCloud database Th s t me you w see the three tab es from the FlixPollOnPrem database Movie, Inquiry, and Response Now ssue these s mp e quer es SELECT * FROM Inquiry SELECT * FROM Movie

The resu ts revea that the data from the FlixPollOnPrem database s now ava ab e n the ­FlixPollOnCloud database Furthermore, f you ook c ose y at the Movie tab e, you w see on y rows w th the YearReleased of 2012 Rows for other years n the Movie tab e were not synchron zed Your enterpr se database can be synchron zed w th SQL Azure w thout wr t ng a s ng e ne of code, and we ca that mag ca ! Upon further rev ew of the tab es n both databases, you w see new tab es added that were not part of the scr pt to create the databases These tab es prov de change track ng capab t es and are used to determ ne the de tas n the data, and were added automat ca y dur ng the dep oyment process

Note  Although the deployment process executes DDL to build the tables in the sync group’s databases, schema changes are not synchronized during subsequent sync jobs. Also, no other database objects, such as stored procedures, triggers, and constraints are created as part of deployment or synchronization. Therefore it is a best practice to ­provision the destination database with these types of objects already in place before you run your initial synchronization. Now that you have an on-prem se SQL Server database synchron zed w th a SQL Azure database, take a moment to get fam ar w th the synchron zat on process Use the scr pts n L st ng 13-2 and L st ng 13-3 to exerc se synchron zat on a b t

638  Part III  Applied SQL

Listing 13-2 Mod fy ng Movie records n the FlixPollOnPrem database.

-- Issue these queries in the FlixPollOnPrem database INSERT INTO Movie (Name, YearReleased) VALUES ('OnPrem New Movie One', 2012), ('OnPrem New Movie Two', 2012), ('OnPrem New Movie Three', 2011) -- this insert will not be sync'd -- since it isn't for year 2012 UPDATE Movie SET Name = 'Chipmonks, The Three-quel' WHERE Name = 'Chipmonks, The SQL' -- this is a conflict w/ the hub and will be over written during sync UPDATE Movie SET Name = 'Amazing SQL-man 2' WHERE Name = 'Amazing SQL-man'

Listing 13-3 Mod fy ng Movie records n the FlixPollOnCloud database.

-- Issue these queries in the FlixPollOnCloud database INSERT INTO Movie (Name, YearReleased) VALUES ('OnCloud New Movie One', 2012), ('OnCloud New Movie Two', 2012), ('OnCloud New Movie Three', 2011) -- this insert will not be sync'd -- since it isn't for year 2012 -- this update over writes the members update to the same record UPDATE Movie SET Name = 'Revenge of the Amazing SQL-man' WHERE Name = 'Amazing SQL-man'

The FlixPollSyncGroup s configured to propagate changes out to the databases n the group every 10 m nutes, but you don’t have to wa t to see the resu ts of execut ng these scr pts You can force a sync job by c ck ng the Sync Now button n the W ndows Azure Management Porta The porta a so prov des fac t es to check the hea th and status of the Sync Group You can see the status of the ast Sync Job by se ect ng the agent from the porta Status nformat on s shown n the porta ’s ma n pane You can a so v ew deta s by c ck ng the nk prov ded to exam ne the og After runn ng the scr pts above and comp et ng a synchron zat on job, ssue the fo ow ng SELECT query aga nst the Movie tab e n both the FlixPollOnPrem and FlixPollOnCloud databases SELECT * FROM Movie

The comb ned resu ts of both quer es are shown n Tab e 13-2 The resu ts show the records from the FlixPollOnPrem have been synchron zed w th the Movie tab e n the FlixPollOnCloud database



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    639

Table 13-2  Database contents after synchron zat on Records from the FlixPollOnCloud Movie Table Mov e d

Name

YearRe eased

4

Revenge of the Amaz ng SQL man

2012

5

Ch pmonks, The Three que

2012

6

The SQL Avengers

2012

7

OnC oud New Mov e One

2012

8

OnC oud New Mov e Two

2012

9

OnC oud New Mov e Three

2011

17

OnPrem New Mov e One

2012

18

OnPrem New Mov e Two

2012

Records from the FlixPollOnCloud Movie Table Mov e d

Name

YearRe eased

1

Squ d Wars

2011

2

L ke Water for SQL

2011

3

SQL M ss on mposs b e

2011

4

Revenge of the Amaz ng SQL man

2012

5

Ch pmonks, The Three que

2012

6

The SQL Avengers

2012

7

OnC oud New Mov e One

2012

8

OnC oud New Mov e Two

2012

17

OnPrem New Mov e One

2012

18

OnPrem New Mov e Two

2012

19

OnPrem New Mov e Three

2011

Note  Because the MovieId column is defined as an Identity, the values in the column on your system will probably differ from those shown in Table 13-2. Not ce how the change made to the “Amaz ng SQL-man” record (Mov e ID 4) n the member database was overwr tten by the change made to the same record n the hub database The record s now “Revenge of the Amaz ng SQL-man” n both databases, because you set the confl ct reso ut on strategy to Hub W ns ear er n step 4 A few scenar os w cause synchron zat on to fa To ensure that everyth ng runs smooth y, keep the fo ow ng po nts n m nd ■

A pr mary key s requ red on every tab e be ng synchron zed

640  Part III  Applied SQL

■

■

■

Changes to a pr mary key va ue w prevent the record from be ng synchron zed Th s s not cons dered a best pract ce n genera , but w th regards to Data Sync, after chang ng the ­pr mary key va ue the keys no onger match and therefore the row cannot be synchron zed App y ng a fi ter to a non-nu ab e co umn w th no defau t va ue w a so present a prob em In th s scenar o, c ents that have the fi tered vers on w never be ab e to sync up the r changes, and w a ways get sync fa ures V s t http://msdn.microsoft.com/library/hh667303.aspx for more nformat on on known Data Sync ssues

Other authors may have stopped here, but not us In the next sect on we are go ng to forge onward to bu d a WCF Data Serv ce for the FlixPollOnCloud database hosted n W ndows Azure The serv ce w be consumed by the F xPo UI U t mate y, the UI w update data n FlixPollOnCloud v a the WCF Data Serv ce where the data w be synchron zed w th the FlixPollOnPrem database

Hosting WCF Data Services in Windows Azure In Chapters 10 and 11, you were ntroduced to the Ent ty Framework (EF) and WCF Data Serv ces, respect ve y Those chapters descr be how to create an Ent ty Data Mode (EDM) over a SQL Server database, and how to query and update the EDM w th any HTTP c ent us ng Representat ona State Transfer Protoco (REST) and OData In th s sect on, you are go ng to bu d on the know edge you obta ned from those chapters to create WCF Data Serv ces dep oyed to W ndows Azure—M crosoft’s c oud-based server operat ng system Then, n the fina sect on, you w consume these serv ces from a F xPo c ent app cat on bu t for W ndows Phone 7

About Windows Azure W ndows Azure (or s mp y, Azure) s the core of M crosoft’s c oud p atform Azure fac tates the ­creat on, dep oyment, and management of app cat ons across M crosoft-managed datacenters around the g obe W ndows Azure prov des the ab ty to qu ck y sca e your app cat ons up or down to meet demand Th s capab ty s referred to as “Compute capac ty,” wh ch s determ ned us ng “Compute resources ” Compute resources are ass gned to your app cat on and are added to, or ­removed from, your app cat on dynam ca y to adjust for ncreas ng or decreas ng demand Compute resources are ut zed through one or more Compute conta ners ca ed “ro es ” There are three d fferent types of ro es ava ab e Web, Worker, and V rtua Mach ne (VM) A ro e s essent a y a v rtua mach ne pre-configured for a spec fic purpose For examp e, the Web ro e s a VM configured w th Internet Informat on Serv ces (IIS) 7 enab ed For the FlixPoll app cat on you are on y concerned w th the Web ro e type You w ass gn the ro e n a moment when you create the WCF Data Serv ce

More Info Visit http://www.windowsazure.com/en-us/home/tour/overview to learn more about Windows Azure.



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    641

Windows Azure Best Practices W ndows Azure s a great c oud p atform, but host ng an app cat on n the c oud produces new concerns that you need to be aware of Here are a few gu de nes to he p you m n m ze cost and consume serv ces secure y on W ndows Azure W ndows Azure fees are based on the amount of outgo ng data There are a coup e of ­benefits to m n m z ng that data ■

Less data equa s ower costs,

■

Less data equa s reduced atency and ncreased app cat on performance

To he p reduce the amount of outgo ng data, you shou d cons der us ng compress on w th XML formats If your c ent app cat on s not a W ndows Phone c ent, another opt on s to use JSON Unfortunate y the W ndows Phone WCF Data Serv ce L brary current y does not ­support ser a z ng to JSON format However, the DataContractJsonSerializer c ass s ava ab e on the ­dev ce to ser a ze a JSON response nto objects on the c ent If you go th s route, you are ­trad ng off ease of deve opment for operat ng ower costs As a deve oper, t s your respons b ty to m t gate secur ty r sks when mp ement ng a c oud serv ce n your app cat ons To make your hosted app cat ons more secure, use an ­authent cat on mechan sm such as Access Contro Serv ces (ACS) or ASP NET Forms-Based ­Authent cat on Both are supported on W ndows Phone c ents ACS s a c oud-based serv ce that enab es c ents to offload the task of authent cat on to a trusted prov der, ke W ndows L ve ID ASP NET Web ro es a so support ASP NET Forms-Based Authent cat on Both of these authent cat on mechan sms are ava ab e w th the W ndows Azure Too k t for W ndows Phone In a cases, a ways use SSL when exchang ng og n credent a s w th an Azure serv ce

Creating the FlixPoll Solution Azure deve opment n V sua Stud o 2010 requ res e evated pr v eges, so you need to start V sua ­Stud o as an Adm n strator R ght-c ck the M crosoft V sua Stud o 2010 shortcut, choose Run As Adm n strator from the context menu, and then c ck Yes n the User Account Contro d a og when prompted Now create a new b ank so ut on Expand the Other Project Types node n the temp ate st and se ect V sua Stud o So ut on From the st of project types se ect B ank So ut on and name t FlixPoll Walkthru Add a new project to the so ut on From the st of temp ate categor es on the eft, beneath V sua C#, choose C oud If th s s your first t me creat ng a project us ng the C oud temp ate, you may be surpr sed to see a project type opt on ca ed Enab e W ndows Azure Too s Th s s an nd cat on that you first need to nsta the W ndows Azure Too s for V sua Stud o F gure 13-8 shows V sua Stud o’s New Project d a og box prompt ng you to nsta the too s 642  Part III  Applied SQL

Figure 13-8  nsta ng W ndows Azure Too s.

Choose Enab e W ndows Azure Too s to start the nsta at on Depend ng on your V sua Stud o configurat on, th s may start the Web P atform Insta er (WPI) If t does, c ck through the d a ogs to nsta WPI Once nsta ed, WPI prompts you w th a d a og d sp ay ng a of the software and configurat on changes nsta ng W ndows Azure Too s makes to your computer C ck I Accept to accept the cens ng terms for the software be ng nsta ed Then fo ow the prompts to comp ete the nsta at on

Adding the FlixPoll Data Service You w now create a W ndows Azure project to host the serv ce, a though you won’t actua y dep oy the serv ce to W ndows Azure unt a b t ater Once the too s are nsta ed, nav gate back to the Insta ed Temp ates project temp ates and se ect the C oud temp ate under V sua C# Th s t me you shou d see the W ndows Azure Project type, shown n F gure 13-9 Type FlixPollWindowsAzure n the Name textbox and c ck OK

Figure 13-9  V sua Stud o s W



ndows Azure Project project type. Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    643

Next, you w be asked to choose the ro es for the F xPo serv ce as shown n F gure 13-10 To host the F xPo serv ce, the on y ro e you need to add s the WCF Serv ce Web Ro e Under the Visual C# node, se ect WCF Serv ce Web Ro e and c ck the arrow button to move the ro e to the r ght s de of the d a og Hover ng over the ro e on the r ght s de revea s two buttons C ck the button that ooks ke a penc to rename the ro e Set the name to FlixPollWcfServiceWebRole and then c ck the d a og’s OK button

Figure 13-10  Se ect ng the appropr ate W ndows Azure ro e for the F xPo WCF Data Serv ce.

V sua Stud o creates a W ndows Azure so ut on w th two projects, one named FlixPollWindowsAzure and a second named FlixPollWCFServiceWebRole The FlixPollWindowsAzure project s just for Azure server configurat on fi es and the app cat on’s web ro es You w use th s project ater to package the serv ce for dep oyment to the c oud The FlixPollWCFServiceWebRole project s an ASP NET app cat on that you can program aga nst as you wou d any ord nary ASP NET app cat on

Note  Windows Azure, or any other cloud platform, is not a requirement for an occasionally connected system. You could host the FlixPoll service on your own IIS servers. W th both projects set up, you can put those menta y as de for a moment and focus on creat ng the serv ce You are creat ng a data serv ce wh ch a ows you to query data n the FlixPollOnCloud database To beg n w th, you are go ng to create the ent ty mode

Adding the Entity Data Model In Chapter 10, the sect on ”Bu d ng an Ent ty Data Mode (EDM)” shows you how to generate an EDM from an ex st ng database For th s project, you w create a F xPo mode from the FlixPollOnCloud database Before you bu d the mode you need to know the fo ow ng nformat on ■

Fu y qua fied DNS name

■

Log n credent a s for your SQL Azure database

■

Your computer’s IP address

644  Part III  Applied SQL

A of these tems can be v ewed on the W ndows Azure Management Porta The server’s DNS name and og n credent a s are requ red to make a connect on both to bu d the mode and for execut ng quer es Before dep oy ng the serv ce to W ndows Azure, you w be ­runn ng the serv ce oca y on your deve opment mach ne Runn ng the serv ce on your computer means your computer needs d rect access to the SQL Azure server Therefore, you a so need to know your IP ­address so you can create a firewa ru e n Azure to a ow your computer access to SQL Azure Log nto the porta , nav gate to the Database, and se ect the Azure server host ng your FlixPollOnCloud database Then copy the fu y qua fied DNS name to the c pboard Next, rev ew the firewa ru es, ook ng for a ru e grant ng access to your IP address If t s not n the firewa ru e st, c ck the Add button to open the Add F rewa Ru e d a og box, as shown n F gure 13-11

Figure 13-11  The Add F rewa Ru e d a og box.

Your IP address s d sp ayed toward the bottom of the d a og box Enter a va ue for the ru e name and type your IP address nto both the IP Range Start and IP Range End textboxes Then c ck OK to create the ru e Now go back to V sua Stud o and expand the FlixPollWCFServiceWebRole project n So ut on Exp orer De ete the IService.cs and Service.cs fi es from the project and then add a new ent ty mode to the project (see the sect on “Bu d ng an Ent ty Data Mode ” n Chapter 10 for deta ed nformat on about th s process) Name the mode FlixPollModel and, when prompted, se ect the Generate From Database opt on Create a new connect on to your SQL Azure server by supp y ng the server’s fu y qua fied DNS name and your og n credent a s Then se ect the FlixPollOnCloud database from the drop-down st, enab e the Save My Password opt on, then c ck Next Acknow edge that you want to save sens t ve nformat on n the connect on str ng S nce th s s a demo app cat on, t s okay to forgo

Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    645

the best pract ce of sav ng the connect on str ng w thout user credent a s Se ect the Inquiry, Movie, and Response tab es for the mode Be sure not to nc ude any of the change track ng tab es added by Data Sync, such as Movie dss tracking Enter FlixPollModel nto the Mode Namespace textbox, and then c ck F n sh The comp eted mode shou d ook s m ar to F gure 13-12

Figure 13-12  The FlixPollModel ent ty mode .

Now you are ready to add a new serv ce to the project Under Insta ed Temp ates, expand the Visual C# node and choose the Web temp ate Then se ect the WCF Data Serv ce tem and name the serv ce ­FlixPollDataService.svc Change the serv ce’s c ass defin t on so that t nher ts from DataService and rep ace the serv ce’s InitializeService method as shown n L st ng 13-4 Listing 13-4  mp ement ng the WCF Data Serv ce.

public class FlixPollDataService : DataService { public static void InitializeService(DataServiceConfiguration config) { config.UseVerboseErrors = true; config.SetEntitySetAccessRule("Movies", EntitySetRights.All); config.SetEntitySetAccessRule("Inquiries", EntitySetRights.AllRead); config.SetEntitySetAccessRule("Responses", EntitySetRights.AllWrite); config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2; } }

646  Part III  Applied SQL

Th s code exposes serv ce endpo nts for the Movies, Inquiries, and Responses ent ty sets It a so sets the c ass’s UseVerboseErrors property to true Enab ng th s property n the deve opment e ­ nv ronment s cruc a for debugg ng, espec a y after you dep oy the serv ce to W ndows Azure If t s not set, errors w return gener c messages, mak ng t near mposs b e to debug prob ems At the same t me, the extra deta s returned by verbose error messages often nc ude sens t ve nterna nformat on that shou d rema n h dden from c ents Therefore, UserVerboseErrors shou d be set to fa se once your code s runn ng n product on env ronments That’s a you need to do to create the serv ce You can now turn your attent on to the F xPo UI

Creating the FlixPoll Client The F xPo c ent s a W ndows Phone 7 app cat on w th a s ng e page named Ma nPage Th s page has contro s that et users se ect and ed t mov es, and enter responses to nterv ew quest ons

Note  You need to install the Windows Phone 7.1 SDK to create the FlixPoll client. You can download the SDK from http://msdn.microsoft.com/en-us/library/ff637516.aspx. Open V sua Stud o and add a new project to your so ut on Under the project temp ates se ect the S ver ght For W ndows Phone temp ate and then the W ndows Phone Databound App cat on project type Name the project FlixPoll and c ck OK, and you w be prompted to se ect the target W ndows Phone Operat ng System (OS) vers on Se ect W ndows Phone OS 7 1 from the drop-down st and c ck OK

Creating the View Th s project temp ate creates pages and c asses for bu d ng a phone app cat on us ng data b nd ng F xPo bu ds on th s temp ate You w not use the DetailsPage.xaml fi e so go ahead and de ete t from the project

Note  Space constraints prevent us from walking you through the complete FlixPoll c­ lient code. Instead, the Movie entity serves as the canonical example for building FlixPoll. All of the walkthroughs for Movies related to data binding, consuming OData with LINQ to REST, and accessing the local SQL Server Compact edition (SQL CE) database storage ­using LINQ to SQL can be applied to the remaining entities in the application. The c­ omplete source code can be downloaded from the book’s companion website (see the book’s “Introduction” for details). Open the MainPage.xaml page n the des gner Then add the fo ow ng attr bute to the ­PhoneApplicationPage e ement at the top of the XAML to ga n access to the W ndows Phone contro s xmlns:toolkit="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone.Controls.Toolkit" 



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    647

Locate the StackPanel named TitlePanel Change the Text property of the ApplicationTitle TextBlock to FlixPoll and then change the Text property of the PageTitle TextBlock to Movies Next, rep ace the markup ns de of the Grid contro named ContentPanel w th the XAML shown n L st ng 13-5 Th s XAML adds ListPicker, ListBox, and TextBox contro s to the page and sets up the data b nd ng temp ate for d sp ay ng mov es and nqu r es on the page

Note The MainPage.xaml file uses controls from the SilverLight Toolkit for Windows Phone. You will need to install that library to compile this code. Download the toolkit from CodePlex at http://silverlight.codeplex.com. Listing 13-5  The XAML markup for the F xPo user nterface.



















648  Part III  Applied SQL

Towards the bottom of the XAML, you w see des gn t me code for the App cat onBar contro By defau t t s commented out F xPo uses the App cat onBar contro on the Ma n page Rep ace the commented out App cat onBar contro w th the fo ow ng XAML





Th s XAML produces a resu t s m ar to F gure 13-13

Figure 13-13  Des gn t me ook at the comp eted Ma n page.

Now open the page’s code-beh nd fi e MainPage.xaml.cs and rep ace ts contents w th the code shown n L st ng 13-6



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    649

Listing 13-6  Code beh nd og c for the F xPo U page.

using System; using System.Windows; using System.Windows.Controls; using Microsoft.Phone.Controls; namespace FlixPoll { public partial class MainPage : PhoneApplicationPage { // Constructor public MainPage() { InitializeComponent(); DataContext = App.ViewModel; this.Loaded += new RoutedEventHandler(MainPage_Loaded); } // Handle selection changed on ListBox private void MovieListPicker_SelectionChanged (object sender, SelectionChangedEventArgs e) { // If selected index is -1 (no selection) do nothing if (movieListPicker.SelectedIndex == -1) return; // bind the textbox to the movie MovieNameTextBox.DataContext = (LocalData.MovieTable)movieListPicker.SelectedItem; } // Load data for the ViewModel Items private void MainPage_Loaded(object sender, RoutedEventArgs e) { if (!App.ViewModel.IsDataLoaded) { App.ViewModel.LoadData(); } } private void ApplicationBarIconButton_SaveClick(object sender, EventArgs e) { // save changes to the movie's name App.ViewModel.SaveMovieToLocalDb(); } private void ApplicationBarIconButton_SyncClick(object sender, EventArgs e) { App.ViewModel.SyncDone += new EventHandler(Cloud_UpdateDone); App.ViewModel.UpdateToCloud(); }

650  Part III  Applied SQL

private void ApplicationBarIconButton_NewClick(object sender, EventArgs e) { // save the answers and reset the quiz for the next person App.ViewModel.NewQuiz ((movieListPicker.SelectedItem as LocalData.MovieTable).MovieId); } // called after upload to cloud has taken place to refresh local db private void Cloud_UpdateDone(object sender, SyncDoneEventArgs e) { // reset the data stored locally. App.ViewModel.DeleteResponsesAfterSync(); App.ViewModel.ResetMoviesAfterSync(); // show the message box on the UI thread Deployment.Current.Dispatcher.BeginInvoke(() => { MessageBox.Show("sync is complete"); }); } } }

Th s code s re at ve y m n ma and un nterest ng, because the ma n data access code s ­actua y ­mp emented separate y (we w get to that code n a moment) In th s code-beh nd fi e, the ­constructor sets the DataContext property to the App c ass’s ViewModel property to fac tate data b nd ng After the constructor, the MovieListPicker SelectionChanged event hand er fires when the user taps on a mov e n the st, and cop es the mov e name nto the TextBox where t can be ed ted Next, the MainPage Loaded event hand er fires when the page oads (th s occurs after the constructor code executes) The code n th s method ca s the V ewMode ’s LoadData method F na y, there are three event handers for hand ng button c cks on the App cat onBar

Understanding the Model-View-ViewModel (MVVM) Pattern The page code s on y concerned w th nav gat on and presentat on of data; t conta ns no other og c Data access and bus ness og c are abstracted away and encapsu ated n separate c asses as per the popu ar Mode -V ew-V ewMode (MVVM) pattern mp emented by the W ndows Phone DataBound ­App cat on project temp ate MVVM s a user nterface des gn pattern that separates concerns among code and re es on data b nd ng as a mechan sm for d sp ay ng data In a W ndows Phone a ­ pp cat on, the Mode s the data or bus ness og c, the V ew s the co ect on of UI pages and v sua e ements (contro s), and the ­V ewMode s a spec a c ass s tt ng between the V ew and the Mode The ­V ewMode exposes propert es that the V ew uses for data b nd ng and a so nteracts w th the Mode to process data In F xPo , MainPage.xaml s the V ew The ent t es ava ab e v a WCF Data Serv ces define the Mode The V ewMode s are c asses w th propert es correspond ng to fie ds from the Mode s; for examp e, MovieId and Name The F xPo V ewMode s are respons b e for convert ng between ent ty mode types and custom types stored n the oca database The convers on process s d scussed a b t ater when we ta k about query ng the oca database

Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    651

The F xPo V ewMode s can connect to the serv ce to retr eve and save data The W ndows Phone 7 1 SDK nc udes a WCF Data Serv ces c ent brary you can use to create a serv ce reference to the FlixPollDataService you created ear er To use the c ent brary you must add a reference to the System.Data.Services.Client assemb y The assemb y s ocated under the Extensions node of the Assemb es tab n the Add Reference d a og box The process of creat ng the serv ce reference for a W ndows Phone 7 app cat on s exact y the same as for a W ndows desktop app cat on In So ut on Exp orer, r ght-c ck the FlixPoll project and choose Add Serv ce Reference Then c ck D scover to find the WCF serv ces n your so ut on The FlixPollDataService shou d appear n the Address drop-down st Take note of the URL as you w use t ater when sett ng up the App c ass In the Namespace ­textbox, enter FlixPollODataService Th s w be the namespace you use n c ent code to reference the proxy c asses that V sua Stud o generates for the serv ce C ck OK to create the serv ce reference F xPo has one V ewMode named MovieViewModel, wh ch s the V ewMode used by the Ma n page To create the V ewMode c ass, add a new c ass to the V ewMode s fo der named ­MovieViewModel, and supp y the code shown n L st ng 13-7 Listing 13-7  The comp ete MovieViewModel c ass.

using using using using using using

System; System.Collections.Generic; System.Collections.ObjectModel; System.ComponentModel; System.Data.Services.Client; System.Linq;

namespace FlixPoll { public class MovieViewModel : INotifyPropertyChanged { private List _movies; private List _flixPollQuiz; public EventHandler SyncDone; public List Movies { get { if (this._movies == null) this._movies = new List(); return this._movies; } set { if (this._movies != value) { this._movies = value; RaisePropertyChanged("Movies"); }

652  Part III  Applied SQL

} } public List FlixPollQuiz { get { if (this._flixPollQuiz == null) this._flixPollQuiz = new List(); return this._flixPollQuiz; } set { if (this._flixPollQuiz != value) { this._flixPollQuiz = value; RaisePropertyChanged("FlixPollQuiz"); } } } private void RaisePropertyChanged(string propertyName) { PropertyChangedEventHandler temp = this.PropertyChanged; if (temp != null) temp(this, new PropertyChangedEventArgs(propertyName)); } public event PropertyChangedEventHandler PropertyChanged; public bool IsDataLoaded { get; private set; } // Get data and display on the page public void LoadData() { if (MoviesInLocalDb()) { GetMoviesFromLocalDb(); } else { GetMoviesFromService(); } if (InquiriesInLocalDb()) { GetInquiriesFromLocalDb(); }



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    653

else { GetInquiriesFromService(); } this.IsDataLoaded = true; } private bool MoviesInLocalDb() { // see if there are any movies in the local CE db using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { var movies = from m in flixPollDc.Movies select m.MovieId; return (movies.Count() > 0); } } private bool InquiriesInLocalDb() { // see if there are any questions in the local CE db using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { var inquiries = from i in flixPollDc.Inquiries select i.InquiryId; return (inquiries.Count() > 0); } } private void GetMoviesFromLocalDb() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // populate the movie list var movies = from m in flixPollDc.Movies select m; this.Movies = movies.ToList(); } } private void GetInquiriesFromLocalDb() { // create a new quiz. When saved later, we'll add the movie id. // this model is bound to the form. using (var flixPollDB = new LocalData.FlixPollDataContext(App.DBConnectionString))

654  Part III  Applied SQL

{ var q = from i in flixPollDB.Inquiries select new LocalData.Quiz { Answer = false, InquiryId = i.InquiryId, Question = i.Question }; this.FlixPollQuiz = q.ToList(); } } public void GetMoviesFromService() { // create a client context for the service var ctx = new FlixPollODataService.FlixPollOnCloudEntities( new Uri(App.FlixPollServiceUrl)); // define the query (deferred execution) var q = from m in ctx.Movies select m; // load the movies asynchronously var movies = new DataServiceCollection(); movies.LoadCompleted += (o, e) => { // Executes when the query completes if (e.Error != null) throw e.Error; // once the data is retrieved, save it to local db CacheInLocalDb(movies); // once we have it locally, populate the list GetMoviesFromLocalDb(); }; // begin the async query movies.LoadAsync(q); } private void GetInquiriesFromService() { // create a client context for the service var ctx = new FlixPollODataService.FlixPollOnCloudEntities( new Uri(App.FlixPollServiceUrl)); // deferred execution var q = from i in ctx.Inquiries select i;



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    655

var inquiry = new DataServiceCollection(ctx); inquiry.LoadCompleted += (o, e) => { if (e.Error != null) throw e.Error; // once the data is retrieved, save it to local db CacheInLocalDb(inquiry); // once we have it locally, populate the list GetInquiriesFromLocalDb(); }; // begin the async query inquiry.LoadAsync(q); } private void CacheInLocalDb(IEnumerable movies) { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // first remove old movies from local db flixPollDc.Movies.DeleteAllOnSubmit(flixPollDc.Movies); // next, cache the movies in local db foreach (FlixPollODataService.Movie m in movies) { LocalData.MovieTable newMovie = new LocalData.MovieTable(); newMovie.MovieId = m.MovieId; newMovie.Name = m.Name; flixPollDc.Movies.InsertOnSubmit(newMovie); } flixPollDc.SubmitChanges(); } } private void CacheInLocalDb(IEnumerable inquiries) { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // first remove old questions from local db flixPollDc.Inquiries.DeleteAllOnSubmit(flixPollDc.Inquiries); // next, cache the questions in local db foreach (FlixPollODataService.Inquiry q in inquiries) { LocalData.InquiryTable newQuestion = new LocalData.InquiryTable(); newQuestion.InquiryId = q.InquiryId; newQuestion.Question = q.Question;

656  Part III  Applied SQL

flixPollDc.Inquiries.InsertOnSubmit(newQuestion); } flixPollDc.SubmitChanges(); } } public void NewQuiz(int movieId) { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // determine the next new response id. start from -1 and work backwards int newId = (flixPollDc.Responses.Count() > 0 ? flixPollDc.Responses.Min(r => r.ResponseId) : 0); // insert each response into the SQL CE db foreach (LocalData.Quiz quiz in this.FlixPollQuiz) { newId--; LocalData.ResponseTable response = new LocalData.ResponseTable(); response.Answer = quiz.Answer; response.InquiryId = quiz.InquiryId; response.MovieId = movieId; response.ResponseId = newId; flixPollDc.Responses.InsertOnSubmit(response); // reset the questions to be asked again quiz.Answer = false; } flixPollDc.SubmitChanges(); } } public void SaveMovieToLocalDb() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // save any movie name changes List changes = this.Movies.Where(m => m.MovieChanged == true).ToList(); if (changes != null && changes.Count > 0) { foreach (LocalData.MovieTable c in changes) { var localMovieRecord = (from m in flixPollDc.Movies where m.MovieId == c.MovieId select m).Single();



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    657

localMovieRecord.Name = c.Name; localMovieRecord.UpdateToCloud = true; // flag the record as changed. // (will be updated to SQL Azure during sync) // reset the ischanged flag so record is not constantly updated c.MovieChanged = false; } } // save the changes to local db. // changes will be uploaded to SQL Azure during sync process flixPollDc.SubmitChanges(); } } private void RaiseSyncDone(SyncDoneEventArgs e) { EventHandler temp = this.SyncDone; if (temp != null) { temp(this, e); } } public void UpdateToCloud() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // Any changed movies stored locally need to be updated to SQL Azure var q = from m in flixPollDc.Movies where m.UpdateToCloud == true select m; foreach (LocalData.MovieTable localMovieRecord in q.ToArray()) { UpdateMovieInCloud(localMovieRecord); } } SendResponsesToCloud(); } private void UpdateMovieInCloud(LocalData.MovieTable localMovieRecord) { // create a client context for the service var ctx = new FlixPollODataService.FlixPollOnCloudEntities( new Uri(App.FlixPollServiceUrl)); // get the movie from the service var q = from m in ctx.Movies where m.MovieId == localMovieRecord.MovieId select m;

658  Part III  Applied SQL

var dsq = q as DataServiceQuery; dsq.BeginExecute(r => { var result = r.AsyncState as DataServiceQuery; var cloudMovieRecord = result.EndExecute(r).Single(); if (cloudMovieRecord != null) { if (localMovieRecord.MovieId == cloudMovieRecord.MovieId) { // Update the movie w/ the new name cloudMovieRecord.Name = localMovieRecord.Name; ctx.UpdateObject(cloudMovieRecord); // push the changes back up to the service ctx.BeginSaveChanges(CloudSync_Completed, ctx); } } }, dsq); } private void SendResponsesToCloud() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // We are only storing NEW responses; just insert into the db var q = from r in flixPollDc.Responses select r; InsertToCloud(q.ToList()); } } private void InsertToCloud(List responses) { // create a client context for the service var ctx = new FlixPollODataService.FlixPollOnCloudEntities( new Uri(App.FlixPollServiceUrl)); bool hasChanges = (responses.Count() > 0); foreach (LocalData.ResponseTable localResponse in responses) { FlixPollODataService.Response newResponse = new FlixPollODataService.Response() { Answer = localResponse.Answer, InquiryId = localResponse.InquiryId,



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    659

MovieId = localResponse.MovieId }; ctx.AddToResponses(newResponse); } if (hasChanges == true) { // async call to insert the new responses. perform insert as a batch ctx.BeginSaveChanges(SaveChangesOptions.Batch, CloudSync_Completed, ctx); } } private void CloudSync_Completed(IAsyncResult result) { var ctx = result.AsyncState as FlixPollODataService.FlixPollOnCloudEntities; DataServiceResponse response = ctx.EndSaveChanges(result); // TODO: a better idea is to use a progress bar and check result for errors this.RaiseSyncDone(new SyncDoneEventArgs { Success = true }); } public void ResetMoviesAfterSync() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // reset UpdateToCloud flag in local db after updating cloud foreach (LocalData.MovieTable m in flixPollDc.Movies) { m.UpdateToCloud = false; } flixPollDc.SubmitChanges(); } } public void DeleteResponsesAfterSync() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // remove responses from local db after updating cloud flixPollDc.Responses.DeleteAllOnSubmit(flixPollDc.Responses); flixPollDc.SubmitChanges(); } } } }

At the top of the code, you can see that the MovieViewModel c ass mp ements the I­NotifyPropertyChanged nterface Th s nterface s cruc a to data b nd ng, as t s respons b e for ­not fy ng bound UI e ements to a change n property va ues A ert ng bound c ents that a va ue has changed s 660  Part III  Applied SQL

done n the property’s setter method For examp e, the MainPage b nds the movieListPicker’s ItemsSource property to the MovieViewModel’s Mov es co ect on ­After the MovieViewModel oads data, severa events occur F rst, sett ng the Movies property ca s the RaisePropertyChanged(“Movies”) method to ra se the PropertyChanged event Then the ­movieListPicker rece ves a not ficat on that the st of tems t s d sp ay ng has changed F na y, the contro refreshes the tems n ts st and d sp ays the updated mov es data to the user

Modifying the App Class Ear er, you saw that the Ma n page’s code-beh nd sets the App c ass’s ViewModel property The App c ass s der ved from the Application c ass, wh ch n turn encapsu ates a S ver ght app cat on, and prov des app cat on- eve serv ces such as unhand ed except on hand ng and fecyc e management The App c ass extends Application to prov de the start ng po nt for a W ndows Phone app cat on The ViewModel property s a stat c property on the App c ass, and s prov ded by the project t­ emp ate so you can mp ement the MVVM pattern By defau t, the ViewModel property s spec fied as ­MainViewModel, wh ch s a c ass that V sua Stud o created automat ca y as a start ng po nt for MVVM deve opment F xPo uses ts own MovieViewModel nstead of bu d ng out MainViewModel, so you w now tweak code n the App c ass to change ts ViewModel property dec arat on from MainViewModel to MovieViewModel In So ut on Exp orer, find and open the App.xaml.cs fi e, and then ook for the fo ow ng code toward the top of the fi e private static MainViewModel viewModel = null; /// /// A static ViewModel used by the views to bind against. /// /// The MainViewModel object. public static MainViewModel ViewModel { get { // Delay creation of the view model until necessary if (viewModel == null) viewModel = new MainViewModel(); return viewModel; } }

Rep ace a nstances of MainViewModel n th s code w th MovieViewModel, and then save your changes Then you can de ete the unneeded MainViewModel c ass from the project’s V ewMode s fo der

Wh e you have the app.xaml.cs fi e open, go ahead and make a few more code ed ts, as shown n L st ng 13-8 Th s defines constants for the serv ce URL and oca SQL Server Compact ed t on (SQL CE) database connect on str ng Add these constants just after the c ass dec arat on



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    661

Listing 13-8  Constants added to the App c ass for the serv ce URL and oca database connect on str ngs.

// Define a string constant pointing to FlixPoll WCF Data Services public static string FlixPollServiceUrl = ""; // example when running the app against local hosted WCF Data Service // http://localhost:49448/FlixPollDataService.svc/ // example when running the app against Windows Azure hosted WCF Data Service // http://e26a38c22af74bdc825e4306ffeb91dd.cloudapp.net/FlixPollDataService.svc/ // Define a string constant pointing to the phone's local SQL Server CE database public static string DBConnectionString = "Data Source=isostore:/FlixPoll.sdf";

When add ng th s code, rep ace the FlixPollServiceUrl str ng w th the URL of your oca y hosted WCF Data Serv ce (the second examp e URL above, w th the port number rep aced by the port ­number ass gned n your deve opment env ronment) Th s s the same one you spec fied when you set the serv ce reference ear er Last y, add the code shown n L st ng 13-9 to the end of the constructor method to create the oca SQL CE database on the phone when the user runs F xPo for the very first t me Listing 13-9  Code added to the App c ass constructor n t a zes the oca SQL CE database.

// Create the local database if it does not exist. using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { if (flixPollDc.DatabaseExists() == false) { flixPollDc.CreateDatabase(); } }

F xPo everages SQL CE to pers st re at ona data retr eved by the serv ce on the oca dev ce Th s code, SQL CE, and the ro e of oca dev ce storage are d scussed n greater deta short y But first, we sh ft focus back to the MovieViewModel c ass

Consuming OData on Windows Phone The MovieViewModel c ass uses v rtua y the same pattern for ca ng WCF Data Serv ces that the ­W ndows desktop c ents use n Chapter 11 In part cu ar, t uses the W ndows Phone 7 vers on of the WCF Data Serv ces c ent brary to convert LINQ to REST quer es nto OData URI str ngs, and mater a zes the AtomPub response feed returned by the serv ce nto c ent-s de ent t es exposed by the ViewModel The pr mary d fference from the W ndows c ents n Chapter 11 s that F xPo ssues 662  Part III  Applied SQL

asynchronous ca s to the serv ce rather than synchronous ca s As we exp a ned n Chapter 11, WCF Data Serv ces supports both synchronous and asynchronous operat on, but W ndows Phone 7 c ents support on y asynchronous ca s If you try to execute a synchronous query ke the ones n Chapter 11, your code w comp e but you w rece ve an except on w th the fo ow ng message at runt me Silverlight does not enable you to directly enumerate over a data service query. This is because enumeration automatically sends a synchronous request to the data service. Because Silverlight only supports asynchronous operations, you must instead call the BeginExecute and EndExecute methods to obtain a query result that supports enumeration.

In Chapter 11, you earned how to code asynchronous ca s between a S ver ght c ent and WCF RIA Serv ces A though the code syntax s d fferent w th WCF Data Serv ces, you w see that the ­concept s exact y the same ssue a serv ce request and wa t for the response n a ca back method Take a c oser ook at the GetMoviesFromService method n the MovieViewModel to see how the ­F xPo c ent makes asynchronous ca s to ts WCF Data Serv ces counterpart public void GetMoviesFromService() { // create a client context for the service var ctx = new FlixPollODataService.FlixPollOnCloudEntities( new Uri(App.FlixPollServiceUrl)); // define the query (deferred execution) var q = from m in ctx.Movies select m; // load the movies asynchronously var movies = new DataServiceCollection(); movies.LoadCompleted += (o, e) => { // Executes when the query completes if (e.Error != null) throw e.Error; // once the data is retrieved, save it to local db CacheInLocalDb(movies); // once we have it locally, populate the list GetMoviesFromLocalDb(); }; // begin the async query movies.LoadAsync(q); }

At the top of the method, the c ent context s nstant ated and stored n the var ab e ctx (not ce that the URL po nt ng to the serv ce s passed nto the constructor as a Uri object) Then a LINQ to REST query s defined n q Th s defines the query to retr eve mov es from the serv ce, but does not nvoke t



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    663

Next, the movies co ect on s defined as a DataServiceCollection, where T spec fies F­ lixPollODataService.Movie as the type of object that w be conta ned n co ect on Th s type s ­exposed on the c ent, and reflects the actua Movie ent ty type defined by the EDM on the serv ce s de Reca that you des gnated FlixPollODataService as the namespace for generated c ent proxy c asses when you estab shed the serv ce reference The DataServiceCollection c ass has a LoadAsync method that you nvoke to k ck off the query on the server W th asynchronous behav or, your app cat on does not wa t for serv ce ­operat ons to comp ete Instead, c ent code can cont nue execut ng and the UI rema ns respons ve wh e ­serv ce ­operat ons run n the background To a ow the c ent to respond once the serv ce operat on s ­comp ete, th s spec a co ect on c ass a so prov des a LoadCompleted event Th s event accepts a ­funct on po nter to another method on the c ent, known as the callback method. The ca back ­executes as soon as the serv ce has comp eted the request In F xPo , th s ca back funct on po nter s expressed as a ambda us ng the => (goes to) syntax, wh ch essent a y embeds the event hand er n ne Methods defined th s way have no name, and are hence often ca ed anonymous methods The (o, e) syntax refers to the standard NET event hand er s gnature, where o (object) s the sender (the object ra s ng the event) and e conta ns event arguments that the c ent can nspect to determ ne the success or fa ure of the serv ce ca These arguments are passed nto the anonymous ca back method when t rece ves contro upon comp et on of the serv ce operat on W th the context, query, and ca back n p ace, the c ent ca s the serv ce by nvok ng the L­ oadAsync method on the movies co ect on and pass ng n the query to execute (defined n q) As you earned n Chapter 11, the WCF Data Serv ces c ent brary automat ca y converts the LINQ to REST query nto an OData URI str ng, and then nvokes an HTTP GET aga nst the serv ce to run the query When the serv ce returns the resu ts, the ca back method on the c ent s nvoked ­automat ca y The first th ng the ca back does s ra se an except on f the serv ce nd cates that an ­ therw se, the data retr eved from the serv ce s ava ab e n the movies co ect on error occurred O var ab e, wh ch the ca back then stores n the oca SQL Server CE database by ca ng CacheInLocalDb F na y, mov e records are retr eved from the oca database and d sp ayed on the page by ca ng ­GetMoviesFromLocalDb We w exp a n the nnards of that method n a moment, but et’s first have a ook at some more asynchronous programm ng techn ques n F xPo When the user taps the Sync button on the Ma n page, changes stored n the oca database are up oaded to the SQL Azure database The UpdateMovieInCloud method s ca ed for each changed mov e, and t sends the changes to the data serv ce asynchronous y private void UpdateMovieInCloud(LocalData.MovieTable localMovieRecord) { // create a client context for the service var ctx = new FlixPollODataService.FlixPollOnCloudEntities( new Uri(App.FlixPollServiceUrl)); // get the movie from the service var q = from m in ctx.Movies where m.MovieId == localMovieRecord.MovieId select m; var dsq = q as DataServiceQuery;

664  Part III  Applied SQL

dsq.BeginExecute(r => { var result = r.AsyncState as DataServiceQuery; var cloudMovieRecord = result.EndExecute(r).Single(); if (cloudMovieRecord != null) { if (localMovieRecord.MovieId == cloudMovieRecord.MovieId) { // Update the movie w/ the new name cloudMovieRecord.Name = localMovieRecord.Name; ctx.UpdateObject(cloudMovieRecord); // push the changes back up to the service ctx.BeginSaveChanges(CloudSync_Completed, ctx); } } }, dsq); }

The first few nes of code set up the context and LINQ query that ooks for a mov e record n the c oud match ng the mov e ID of the oca changed mov e Th s s s m ar to the code n GetMoviesFromService, but UpdateMovieInCloud now takes a d fferent approach to nvoke the query The GetMoviesFromService method used LoadAsync and LoadComplete w th DataServiceCollection to oad an ent re co ect on asynchronous y Th s method needs to retr eve just a s ng e ent ty, so t nstead uses BeginExecute and EndExecute w th DataServiceQuery Spec fica y, t casts the LINQ query to a DataServiceQuery object n dsq and nvokes the BeginExecute method to start the asynchronous ­serv ce operat on Th s method takes two parameters; the first s the ca back method (wh ch s coded n ne us ng the ambda syntax as before) and the second s an object that gets passed nto the ca back method Th s s the query object tse f (dsq n th s examp e), that gets passed a ong to the ca back as the var ab e r (for resu t) To comp ete the query, the ca back obta ns the AsyncState property from r and passes t n to the EndExecute method The Single method s then nvoked to obta n the s ng e Movie ent ty The Single method s used to retr eve the one and on y one ent ty returned by the query, and w fa f the query returns mu t p e (or zero) ent t es Once the ca back has obta ned the mov e from the serv ce, the code s mp y updates t by ­sett ng ts Name property to the new va ue n the Name property of the oca vers on of th s mov e ­record passed n to UpdateMovieInCloud The mod fied Movie ent ty s then passed to UpdateObject, wh ch nforms the context to track th s ent ty as changed F na y, the BeginSaveChanges ssues an ­asynchronous serv ce ca to update the database n the c oud As w th BeginExecute, the BeginSaveChanges method accepts two parameters one for the ca back and one for the context object tse f so that t gets passed to the ca back Th s t me, the ca back s mp emented n a “norma ” method ca ed CloudSync Completed, nstead of us ng the ambda ­approach that mp ements an anonymous method We d d th s mere y to he p ustrate how ambdas work to define anonymous methods just ke ord nary methods Certa n y, the ca back cou d nstead have been embedded n ne as an anonymous method us ng a ambda, as you’ve seen done severa t mes by now Whether you mp ement the ca back as an anonymous method or not, t s expected to

Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    665

rece ve the s ng e IAsyncResult parameter that s used to “end” the operat on As w th BeginExecute/ EndExecute, the CloudSync Completed ca back spec fied n BeginSaveChanges comp etes the save operat on by ca ng EndSaveChanges private void CloudSync_Completed(IAsyncResult result) { var ctx = result.AsyncState as FlixPollODataService.FlixPollOnCloudEntities; DataServiceResponse response = ctx.EndSaveChanges(result); // TODO: a better idea is to use a progress bar and check result for errors this.RaiseSyncDone(new SyncDoneEventArgs { Success = true }); }

You are now ready to start deve op ng the part of F xPo that stores data oca y on the phone

SQL Server on the Phone The or g na W ndows 7 Phone OS a ows you to store data oca y on the dev ce us ng e ther Iso ated Storage or oca resource fi es Both of these methods are des gned for a spec fic, ­non-re at ona ­storage scenar o Iso ated Storage s appropr ate for key/va ue pa r data ke app cat on s­ ett ngs, whereas oca resource fi es are usefu for embedd ng mages or oca zed str ngs nto your ­app cat on But ne ther approach s su tab e for stor ng re at ona data

Note  Phone resources have limited physical storage capacity. Your applications should minimize the amount of data stored on a phone to help conserve resources. Fortunate y, the W ndows Phone 7 1 re ease (codenamed “Mango”) ntroduces a new way for s­ tor ng data oca y on the phone M crosoft SQL Server Compact ed t on (SQL CE) W th SQL CE, you can create a re at ona database d rect y on the dev ce There are three ways to use SQL CE The first way s to create the database separate y from the phone us ng the SQL Server Compact Too box The second way s to use the command ne ut ty ExportSqlCe to export the database to a fi e Us ng e ther of these approaches, you can then embed the SQL CE database as a resource nto your phone app cat on

Important  If you choose to develop an application by embedding your own SQL CE ­database as a resource file, you must use SQL Server CE version 3.5 Service Pack 2. The th rd way s to programmat ca y create the database and tab es from code executed ns de your app cat on When you create a database th s way, t ex sts as a fi e stored n so ated storage F xPo creates the database us ng th s “pure code” approach

666  Part III  Applied SQL

LINQ to SQL Strikes Back Regard ess of how you create the SQL CE database, you use LINQ to SQL to access ts data Reca from Chapter 10 that Ent ty Framework s the preferred NET data access API, and that the use of LINQ to SQL s d scouraged for new deve opment Th s rema ns true for c ent/server or n-t er mu t -user app cat ons that are bu t over SQL Server However, for s ng e-user scenar os on a oca dev ce (such as a W ndows Phone), LINQ to SQL ves on as the preferred data access method for access ng SQL CE The LINQ to SQL mp ementat on on W ndows Phone does have a few m tat ons For one th ng, on y SQL CE data types are supported For a comp ete st of LINQ to SQL features not supported on W ndows Phone, v s t http://msdn.microsoft.com/library/hh202872.aspx Leverag ng SQL CE as the offl ne storage mechan sm enab es F xPo to operate wh e d sconnected from a network or the Internet Th s s the pr mary character st c of any occas ona y connected system

Securing Local Data By the very nature of the r use, mob e c ent dev ces typ ca y conta n sens t ve data Here are some steps you can take to ensure that nformat on does not fa nto the wrong hands f a user’s dev ce s ost or sto en ■

■

Do not hard code connect on user names and passwords n the app cat on or config fi es Th s techn que s on y appropr ate for demos and samp e code In pract ce, always prompt the user for connect on nformat on upon first use, and be sure to encrypt credent a s before stor ng them on the dev ce SQL Server Compact 3 5 supports data encrypt on n the dev ce’s oca database Use th s ­encrypt on whenever poss b e to protect pr vate data from unwanted exposure

Create the SQL CE Database Creat ng a SQL CE database s very stra ghtforward F rst add a reference SQL.Data.Linq, wh ch s the assemb y for LINQ to SQL The assemb y s ocated under the Assemb es Framework tab n the Add Reference d a og You use a LINQ to SQL DataContext c ass to connect to and nteract w th a SQL CE nstance DataContext s respons b e for br dg ng the app cat on’s object mode w th the data n the database You create a context c ass for your own ent t es by nher t ng from DataContext and ­expos ng tab es and co umns as propert es Create a new fo der n the FlixPoll project named Loca Data, and add a new c ass to the fo der named FlixPollDataContext Then mp ement the c ass w th the code shown n L st ng 13-10 Listing 13-10 The FlixPollDataContext c ass exposes the SQL CE database for L NQ to SQL.

using System; using System.Data.Linq; namespace FlixPoll.LocalData



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    667

{ public class FlixPollDataContext : DataContext { // Pass the connection string to the base class. public FlixPollDataContext(string connectionString) : base(connectionString) { } public Table Movies; public Table Inquiries; public Table Responses; } }

The FlixPollDataContext c ass exposes three fie ds Movies, Inquiries, and Responses Each fie d s of type Table type where T s a separate c ass that you w create for each tab e—MovieTable, InquiryTable, and ResponseTable These tab e c asses, n turn, are marked w th spec a attr butes that define schema nformat on you wou d typ ca y find n a database tab e, such as co umns, data types, and pr mary key constra nts Add a new c ass named MovieTable to the Loca Data fo der Then mp ement the c ass w th the code shown n L st ng 13-11 Listing 13-11 The MovieTable c ass defines the schema for the Movie tab e n the SQL CE database.

using using using using

System; System.ComponentModel; System.Data.Linq; System.Data.Linq.Mapping;

namespace FlixPoll.LocalData { // Define the table as a class marked with the [Table] attribute [Table] public class MovieTable : INotifyPropertyChanged, INotifyPropertyChanging { public bool MovieChanged { get; set; } // Define columns as properties marked with the [Column] attribute private int _movieId; [Column(IsPrimaryKey = true, IsDbGenerated = false, DbType = "INT NOT NULL", CanBeNull = false)] public int MovieId { get

668  Part III  Applied SQL

{ return _movieId; } set { if (_movieId != value) { NotifyPropertyChanging("MovieId"); _movieId = value; NotifyPropertyChanged("MovieId"); } } } private string _name; [Column] public string Name { get { return _name; } set { if (_name != value) { // used to track if update has taken place if (this._name != null) this.MovieChanged = true; NotifyPropertyChanging("Name"); this._name = value; NotifyPropertyChanged("Name"); } } } private bool _updateToCloud; [Column] public bool UpdateToCloud { get { return _updateToCloud; } set { if (_updateToCloud != value) { NotifyPropertyChanging("IsChanged"); _updateToCloud = value;



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    669

NotifyPropertyChanged("IsChanged"); } } } // Notify the page that a data context property changed public event PropertyChangedEventHandler PropertyChanged; private void NotifyPropertyChanged(string propertyName) { if (PropertyChanged != null) { PropertyChanged(this, new PropertyChangedEventArgs(propertyName)); } } // Notify the data context that a data context property is about to change public event PropertyChangingEventHandler PropertyChanging; private void NotifyPropertyChanging(string propertyName) { if (PropertyChanging != null) { PropertyChanging(this, new PropertyChangingEventArgs(propertyName)); } } } }

The SQL CE tab es are des gned to store data down oaded from the serv ce, and so the r schema c ose y matches that of the tab es n the FlixPollOnCloud database As you can see, MovieTable s an ord nary c ass decorated w th the [Table] attr bute Th s te s LINQ to SQL that the MovieTable c ass represents the schema for creat ng a tab e n the database W th n the c ass, the tab e’s co umns are defined as propert es marked w th the [Column] attr bute Not ce that the [Column] attr bute for the MovieId property has extra nformat on that dent fies t as the pr mary key The code you added ear er to the App c ass (L st ng 13-9) creates the database once, the very first t me the user aunches F xPo on the r W ndows Phone That code ca s the CreateDatabase method on the DataContext to bu d the database fi e and save t n Iso ated Storage The fi e ocat on s determ ned from the connect on str ng parameter passed to the DataContext’s constructor The ­connect on str ng was a so added ear er to the App c ass (L st ng 13-8), and po nts to the database fi e n Iso ated Storage Data Source=isostore:/FlixPoll.sdf

SQL CE uses the LINQ to SQL c asses you created ear er to create the database Us ng Reflect on, t ooks for the [Table] and [Column] attr butes to determ ne the database schema After the ca to CreateDatabase comp etes, you w have a new database popu ated w th just schema nformat on— no data 670  Part III  Applied SQL

Note  A word about using connection strings on the Windows Phone. The local folder on the phone where the SQL CE database is stored determines the type of the access ­permitted to the database. SQL CE databases stored in the special app data folder are read-only, whereas those stored in Isolated Storage are read-write. For more information about connection strings for SQL CE on Windows Phone, see http://msdn.microsoft.com/ library/hh202861.aspx.

Issuing LINQ to SQL Queries Against SQL CE Retr ev ng data from SQL CE s accomp shed us ng LINQ to SQL quer es You first create a new ­ nstance of FlixPollDataContext to ssue quer es aga nst Then you ssue LINQ quer es that get ­converted dynam ca y at runt me nto the T-SQL statements that execute aga nst SQL CE The ­GetMoviesFromLocalDb method n the MovieViewModel c ass (L st ng 13-7) demonstrates th s process It uses LINQ to SQL to get mov e records from the oca SQL CE database private void GetMoviesFromLocalDb() { using (var flixPollDc = new LocalData.FlixPollDataContext(App.DBConnectionString)) { // populate the movie list var movies = from m in flixPollDc.Movies select m; this.Movies = movies.ToList(); } }

Running Against a Local WCF Data Service To enforce th s chapter’s concepts, the wa kthrough thus far has focused just on the Movie ent ty, and you have a part a y comp eted c ent app cat on at th s po nt Down oad the code ocated at the book’s compan on webs te to comp ete the code for F xPo (see the book’s “Introduct on” for the URL) You need to add the InquiryTable, Quiz, and ResponseTable c asses n the Loca Data fo der and the SyncDoneEventArgs c ass n the project’s ma n fo der Once you have added the rema n ng code, you can run F xPo aga nst your oca WCF Data Serv ce F gure 13-14 shows you what F xPo ooks ke runn ng n the W ndows Phone Emu ator The serv ce s st runn ng oca y at th s po nt Before dep oy ng t to W ndows Azure, test the c ent app cat on for a b t To se ect a new mov e, tap on the mov e name n the movieListPicker on the top of the page Once you se ect a mov e, you can change the mov e’s t t e by tapp ng the mov e name n the TextBox Th s br ngs up the keyboard After you mod fy a mov e name, move off of the TextBox contro and you w see the name change n the movieListPicker Tap the Save button to save the change to the mov e t t e



Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    671

Figure 13-14  F xPo runn ng n the W ndows Phone emu ator.

To add responses, you tap each s der contro to the r ght or eft, r ght for “yes,” eft for “no ” After answer ng a quest ons, tap the New button to save the responses and refresh the qu z so you can enter new responses Then c ck the Sync button to send the data to the SQL Azure database The Sync Schedu e you set up toward the beg nn ng of th s chapter s st n effect After ­mod fy ng a few mov es and enter ng some responses, og n to the W ndows Azure Management Porta and ­perform a manua sync job Then (us ng e ther SSMS or SSDT) ssue SELECT statements aga nst the Movie and Response tab es n both the FlixPollOnPrem and FlixPollOnCloud databases The ­resu ts ­returned reflect changes to the data as a resu t of the update from phone to c oud, and the ­subsequent Sync Job to update the on-prem se database

Deploying to Windows Azure When sat sfied w th the oca y runn ng serv ce, you are ready to dep oy t to W ndows Azure We ­conc ude th s chapter by wa k ng you through the steps to set up and host the F xPo WCF Data ­Serv ces n W ndows Azure In So ut on Exp orer, r ght-c ck on the FlixPollWindowsAzure project and choose Package to beg n the dep oyment process A d a og appears w th severa opt ons Accept the defau t sett ngs and c ck the Package button At some po nt dur ng the dep oyment process, a W ndows Exp orer w ndow w

672  Part III  Applied SQL

open, show ng two fi es n a fo der on your oca computer Make a copy of the path to the fo der as you w need t n a subsequent step C ose the w ndow after you have cop ed the fo der path Log nto the W ndows Azure Management Porta and c ck on the New Hosted Serv ce button n the too bar The Create A New Hosted Serv ce d a og box appears, shown n F gure 13-15

Figure 13-15  Creat ng a hosted serv ce n W ndows Azure.

Name your serv ce FlixPollService When enter ng a URL prefix for the serv ce, remember that the prefix name must be un que If you enter a va ue that has a ready been used by anyone, you w be prompted to enter a d fferent prefix Se ect a reg on from the drop-down st and then se ect the ­Dep oy To Stage Env ronment rad o button Ensure that the Start After Successfu Dep oyment ­checkbox s checked Then enter the name FlixPollPackage for the Dep oyment Name Spec fy the Package Locat on fi e by c ck ng the Browse Loca y button In the W ndows Exp orer w ndow that opens, paste the path you cop ed ear er from V sua Stud o when you created the ­package nto the W ndows Exp orer address bar and press Enter Se ect the Serv ce Package fi e and c ck the Open button Repeat the same steps to spec fy the C oud Serv ce Configurat on fi e Then c ck OK to start the dep oyment Now W ndows Azure beg ns to va date the package ( f you rece ve a warn ng regard ng “… a ro e w th on y one nstance…” you can gnore the warn ng and cont nue the dep oyment) When the ­dep oyment successfu y comp etes, the FlixPollService w be shown n the management porta w th a status of Ready

Chapter 13  SQL Azure Data Sync and W ndows Phone 7 Deve opment    673

That’s a t takes s to dep oy WCF Data Serv ces for F xPo to W ndows Azure Your ast step s to configure F xPo to run aga nst the hosted serv ce Copy the DNS name from the porta and then open the App.xaml.cs fi e n the F xPo W ndows Phone project Ed t the ne of code that defines the FlixPollServiceUrl constant by past ng n the URL va ue w th the DNS va ue you cop ed from the porta For examp e public static string FlixPollServiceUrl = "http://e26a38c22af74bdc825e4306ffeb91dd.cloudapp.net/FlixPollDataService.svc/";

Run F xPo aga n and mod fy some data You shou d not not ce any d fference, but F xPo s now runn ng aga nst a serv ce hosted n the c oud Arch tectura y speak ng, you have a ot of flex b ty w th an app cat on ke th s You cou d dec de to host the serv ce yourse f or n the c oud Sw tch ng from one env ronment to the other s s mp y a matter of choos ng the proper URL str ng

Summary You have many dec s ons to make before bu d ng an occas ona y connected system Choos ng the r ght synchron zat on framework s on y the beg nn ng SQL Azure Data Sync offers an attract ve so ut on for bu d ng an app cat on w th robust synchron zat on features As you have earned n th s chapter, SQL Azure Data Sync requ res m n ma cod ng and effort and works n a var ety of scenar os The data synchron zat on story us ng SQL Azure Data Sync s st tak ng shape at M crosoft Even at th s ear y stage, t packs enough features to meet a w de array of synchron zat on scenar os ­However, unt t supports synchron zat on w th mob e c ents, you can use WCF Data Serv ces as we’ve ­demonstrated n th s chapter to fi that gap We exp ored the concepts of occas ona y connected systems, and bu t such a system us ng SQL Azure Data Sync serv ces coup ed w th a W ndows Phone c ent and WCF Data Serv ce hosted n W ndows Azure You saw how to bu d app cat ons for W ndows Phone 7 dev ces that consume REST serv ces and work n both offl ne and on ne modes You a so earned how to synchron ze data between the c oud and your on-prem se databases W th the fundamenta s covered n th s chapter, you can now extend your bus ness systems w th a sorts of new and exc t ng “occas ona y connected” capab t es

674  Part III  Applied SQL

C hapter 1 4

Pervasive Insight —Andrew Brust

D

esp te th s book’s deta ed focus on the re at ona database eng ne, and transact ona use, of M crosoft SQL Server, t’s mperat ve to rea ze and d scuss the product’s w de range of bus ness nte gence (BI) capab t es, some of wh ch are arguab y the best n the ndustry Depend ng on the ed t on of SQL Server you purchase, you w have access to some or a of the fo ow ng serv ces ■

Data Warehous ng

■

Report ng and data v sua zat on

■

On ne ana yt ca process ng (OLAP)

■

Co umn store, n-memory ana yt cs

■

Data M n ng

■

Extract, Transform and Load (ETL)

■

Comp ex event process ng (CEP)

■

Master Data Management (MDM)

■

Data Qua ty

Note  We’ll cover SQL Server editions, and their included BI feature sets, at the end of this chapter. Cover ng these techno og es n depth wou d requ re a separate book But even f you are ­pr mar y focused on the re at ona capab t es of SQL Server, t s mportant to have an understand ng of the BI components nc uded w th SQL Server and, at a h gh- eve , what each BI component does In th s chapter, therefore, we w g ve you that nformat on We w descr be each component, prov de screenshots that w he p make the concepts more concrete, and present those concepts from the context of re at ona database techno ogy so that you won’t fee you need to sh ft gears too much

675

Note  Microsoft Press has an excellent book — Smart Business Intelligence Solutions with Microsoft® SQL Server® 2008 by Langit, Goff, et al. — that covers much of the SQL Server BI stack in great detail. Each of the BI components n SQL Server covers part of a arger data strategy, and part c pates n a fu fe cyc e around the treatment and ana ys s of data, after that data has been captured by ­transact ona systems We w present the BI components of SQL Server n a sequence that fo ows that workflow; that way you can understand each component n the context of the others, as we as n the context of the re at ona eng ne We’ a so br efly d scuss e ements of M crosoft Exce and SharePo nt that comp ement these capab t es and nterface w th SQL Server BI techno og es In th s chapter, you w ■

■

■ ■

■ ■

■

■

■

■

earn

How SQL Server manages, c eans, and m grates data w th Master Data Serv ces, Data Qua ty Serv ces, and Integrat on Serv ces The const tuent parts of the SQL Server data warehouse p atform, nc ud ng SQL Server ­Enterpr se ed t on, Fast Track Data Warehouse, and Para e Data Warehouse ed t on Data Warehouse term no ogy The capab t es of SQL Server Ana ys s Serv ces (SSAS), nc ud ng ts mu t d mens ona , Tabu ar, and data m n ng capab t es OLAP, co umn store, and n-memory database term no ogy and capab t es Why Data V sua zat on and Ana ys s are mportant and how Power V ew estab shes a new precedent n mak ng these easy and fun How and why SQL Server Report ng Serv ces (SSRS) s a true BI component and not mere y an operat ona report ng too How M crosoft Office and SharePo nt fit nto the p cture w th the BI capab t es of Exce , ­PowerP vot, Exce Serv ces, and PerformancePo nt Serv ces What StreamIns ght s, how t fits n w th SQL Server, and how t funct ons as a stand-a one deve oper too The range of SQL Server ed t ons, and the BI features of each

The Microsoft BI Stack: What’s It All About? The po nt of BI s to move beyond the tact ca and operat ona aspects of databases, wh ch nvo ve the fast co ect on, updat ng, and retr eva of data BI a ms to take the “raw mater a ” that s the data co ected from such act v t es and refine t nto the “fin shed goods” of nformat on about the data

676  Part III  Applied SQL

s­ ummar es, trends, patterns, v sua zat ons, and even pred ct ons These art facts can he p dr ve ­bus ness dec s ons, and n fact, BI used to be ca ed “Dec s on Support ” Do ng th s k nd of work requ res mak ng sure the data s c ean and cons stent; transform ng ts schema to be more opt m zed for query ng; hav ng too s that v sua ze summar zed v ews of the data; and us ng spec a zed database eng nes that make ana yt ca quer es fast and pa n ess The M crosoft BI stack, wh ch s anchored by SQL Server, a ms to prov de you w th a of these Let’s d scuss how, by ook ng at each component n the stack

Master Data Services It’s a b t unorthodox to beg n our d scuss on of SQL Server BI w th the Master Data Serv ces (MDS) component, because t was on y added to SQL Server n the 2008 R2 re ease Often t’s eas er to ta k about o der components first and then sketch out the nuances that make the newer ­components ­usefu and necessary Nonethe ess, s nce we’re cover ng the BI stack n og ca workflow order, MDS s the r ght p ace to start MDS s M crosoft’s entry nto the area of BI known as Master Data ­Management (MDM), wh ch concerns tse f w th the ma ntenance of author tat ve reference data ke sts of customers, store ocat ons, product categor es, personne t t es, and even more gener c ent t es ke countr es, states, and prov nces, or honor fics ke Mr , Mrs , and Ms If you th nk about t from an app cat on database perspect ve, master data tends to be re-­created for each app cat on, resu t ng n var ous tab es appear ng n numerous databases that conta n ­dent ca , or near y dent ca , data And t’s the nearly dent ca part that s the prob em, because when t comes t me to ook at data from more than one of these databases n a conso dated way, s ght m sa gnments n master data can make the conso dat on process d fficu t For examp e, how cou d users dr down revenue numbers and sa es project ons by store ­ocat on f the forecast ng system and the order system don’t work form the same st of stores, or f they do but use d fferent key va ues for the same ocat on? MDM a ms to so ve th s prob em by prov d ng too s and a repos tory for author ng, refin ng, and stor ng the author tat ve set of master data needed n an ­organ zat on In the SQL Server arena, Master Data Serv ces s the MDM too supp ed for th s job MDS a ows you to define Models, wh ch conta n re ated sets of master data, and can ­accommodate mu t p e vers ons of each mode n the repos tory It a ows you to define Entities, ­represent ng c asses of master data ( ke customers or countr es), the Attributes of those ent t es ­(essent a y the ent ty’s co umns, f you th nk of an ent ty as a tab e), and Attribute Groups, wh ch are subd v s ons of the attr butes used to make the atter eas er to manage and secure Once your Ent t es are defined, you can go about defin ng the r Members, the actua Ent ty data Members can be arranged n Hierarchies, wh ch may be Derived, where a Member of an Ent ty s the parent (v a an ass gned attr bute va ue) of one or more members of another Ent ty (e g , ­spec fic ­product categor es m ght be parents of spec fic products); Explicit, where parents are manua y ­entered because they are not members of a defined ent ty; or Recursive, where one member of an ent ty s the parent of one or more other members of that same ent ty (e g , emp oyees have other emp oyees who are d rect



Chapter 14  Pervas ve ns ght   677

reports, and those emp oyees have d rect reports of the r own) E ­ nt t es may a so be ong to Collections, wh ch are arb trar y named sets of members that may often be ­reported on together In most organ zat ons, there s governance around MDM work; not just anyone can add and mod fy master data As such, MDS offers ts own ro e-based secur ty mechan sm that manages Users, Groups, and Permissions It a so a ows for the defin t on of Business Rules that can be used to va date master data entr es, supp y defau t attr bute va ues, send ema not ficat ons, and more MDS a so has Workflow fac t es to ensure master data tasks are proper y vetted and approved SQL Server MDS came to M crosoft through the acqu s t on of a company ca ed Stratature, wh ch was an ndependent software vendor se ng a stand-a one ASP NET app cat on for MDM Stratature’s app cat on, after some rebrand ng and tweaks, became MDS n SQL Server 2008 R2 and rema ned an ASP NET web app cat on For SQL Server 2012, the app cat on was rewr tten n S ver ght, though t st bears a strong resemb ance to ts ASP NET predecessor It s shown n F gure 14-1

Figure 14-1  Ed t ng members of the Country ent ty n the Master Data Serv ces S ver ght c ent.

A so new n SQL Server 2012 s an Exce add- n for MDS that, wh e not as fu y featured as the stand-a one S ver ght app cat on, prov des a ot of funct ona ty and a ows master data to be ­entered and ed ted r ght n Exce worksheets To be frank, the Exce nterface s much eas er to use than the stand-a one S ver ght app cat on, and we wou d recommend t That sa d, there are certa n tasks that can on y be carr ed out n the S ver ght c ent

678  Part III  Applied SQL

The Exce add- n s shown n F gure 14-2 The add- n’s Master Data r bbon tab s h gh ghted, as are the mode and connect on nformat on, the member data area, the Master Data Exp orer task pane, and the r bbon group for the ntegrated Data Qua ty Serv ces funct ona ty

Note  Data Quality Services, and its integration with MDS, will be discussed next.

Figure 14-2  Ed t ng members of the Country ent ty n the Master Data Serv ces add n for Exce .

Another way to access MDS s through ts Web Serv ces nterface A though not w de y ­mp emented th s way, the Web API a ows nd v dua ne-of-bus ness app cat ons to commun cate w th MDS n order to retr eve, and update, master data The API a so a ows th rd-party app cat ons to present a ternate user nterfaces on top of the MDS eng ne One product that does th s s M DaS from V s onWare (http://www.visionwareplc.com) In add t on to enter ng member data n the stand-a one MDS app cat on and n Exce v a the ­ dd- n, member data can a so be mported nto MDS, and t can be exported as we These are cruc a a funct ons Most master data s not estab shed v a manua entry nto an MDM system, but rather by mport ng that data from ex st ng transact ona systems L kew se, a though the Web Serv ces are ­conven ent, many transact ona systems w most eas y be fed author tat ve master data by ­hav ng that data exported nto the app cat on’s own database tab es wh ch store data for those same ­ent t es



Chapter 14  Pervas ve ns ght   679

MDS has not been w de y adopted by M crosoft customers That may start to change w th SQL Server 2012, g ven the Exce add- n as an a ternate user nterface and the ntegrat on of MDS w th a brand-new M crosoft BI component Data Qua ty Serv ces

Data Quality Services MDM and MDS are a about prevent ng the prob em of ncons stent, naccurate, or erroneous master data But what f you’re n a s tuat on (and many peop e are) where you a ready have a bunch of ­master data and t’s not c ean? Then what do you do? One so ut on s to use a data qua ty too ke SQL Server Data Qua ty Serv ces (DQS), a new member of the M crosoft BI fam y, wh ch a ms to ­manage the process of c ean ng up “d rty data ” There can be many causes for d rty data Imag ne a system where a Customers tab e s ttered w th var ants of the same company name ke “Contoso,” “Contoso Inc ,” Contoso, Inc,” and “Contoso Incorporated ” Then mag ne that you are wr t ng a report that needs to group sa es by customer, and so you end up gett ng four separate groups for Contoso, each w th ts own subtota , rather than one In such a s tuat on, you wou d probab y end up go ng through the data, updat ng the Orders tab e (manua y or w th a SQL query) so that a s ng e Contoso customer record were referenced ­cons stent y, and then e m nate the other Contoso customer records from the Customers tab e Do ng th s wou dn’t be the end of the wor d, but t wou d hard y be a worthy use of your t me Now mag ne that there are other records n that Customers tab e where the company name s b ank, or has random characters n t In a transact ona system, th s sn’t a cr t ca ssue, because those customer entr es wou d e ther be se ected rare y or not at a But for ana yt ca work, you have a prob em When you go to get a count of customers by, say, reg on, or just determ ne an ­average, your numbers w be off So n order to make sure your reports are accurate, you’d probab y go through the Customers tab e and remove these unwanted rows, e ther manua y or through code, f the ­number and type of bad rows warranted Aga n, th s s ted ous work that dra ns product v ty Data Qua ty software s des gned to automate these data c eans ng tasks, and DQS s M crosoft’s new BI component n th s category It de-dupes and c eanses based on a comb nat on of supp ed ­nformat on and art fic a nte gence funct ona ty, and t ets you rev ew the changes t suggests before they are comm tted In DQS, you define Knowledge Bases and then create data qua ty Projects that use them ­ now edge Bases conta n Domains, wh ch n turn have Rules (wh ch spec fy patterns and formats K that the data must adhere to) and Term-Based Relations (wh ch hand e correct ons/rep acements of names or terms, for examp e rep ac ng a occurrences of “Corp ” to “Corporat on”) Doma ns a so have Domain Values (va d doma n data) and have the r own Reference Data wh ch can be sourced from a number of out ets, nc ud ng M crosoft’s Azure DataMarket Know edge Bases a so conta n Matching Policies wh ch n turn cons st of Matching Rules cr ter a for how we var ous doma n va ues shou d match nspected data n order for the atter to be cons dered matches

680  Part III  Applied SQL

Data Qua ty Projects sp t nto two types Cleansing and Matching C eans ng projects eva uate new data based on the og c n a Know edge Base The c eans ng process then judges each row of data and w mark many rows as correct or invalid or new Other rows may be changed and marked as suggested or corrected depend ng on the confidence eve ass gned to the change by DQS M ­ atch ng projects perform de-dup cat on, us ng the match ng po cy of a Know edge Base to dent fy (and e m nate) exact and approx mate matches DQS has both a server component and a (desktop) c ent app cat on The c ent app cat on, shown n F gure 14-3, a ows you to create Know edge Bases, as we as Projects

Figure 14-3  The Data Qua ty Serv ce c ent, dur ng a data d scovery ana ys s operat on n a Know edge Base.

DQS can be used on ts own to perform ts data qua ty dut es as a d screte process But t can a so be ntegrated w th MDS and w th SQL Server Integrat on Serv ces (SSIS—wh ch we w d scuss n the next sect on) In other words, nstead of creat ng d screte C eans ng and Match ng projects n the DQS c ent, DQS can be ca ed upon n an embedded fash on to perform c eans ng and match ng aga nst MDS Ent ty Member data or a dataset passed to t n an SSIS package

Integration Services SSIS s M crosoft’s Extract, Transform, and Load (ETL) too It offers a r ch deve opment ­env ronment for tak ng source datasets and runn ng them through var ous processes of sort ng, fi ter ng, ­reshap ng, and test ng, fo owed by dep oyment of the mod fied data to a dest nat on



Chapter 14  Pervas ve ns ght   681

ETL becomes usefu n a number of scenar os For examp e, automated, unattended process ng of flat fi es nto a database, or per od c rep cat on of data from one transact ona system to another You can a so use SSIS to run adm n strat ve tasks, for examp e database backups, process ng of Ana ys s Serv ces cubes, or even data qua ty checks, as d scussed n the prev ous sect on But n the rea m of BI, the pr mary use for ETL s the transfer of data from a transact ona system nto a data mart or data warehouse Th s requ res not just copy ng the data, but a so chang ng ts structure from a norma zed des gn to one conformed to a star schema, someth ng that we’ d scuss n the next sect on SSIS packages can be thought of as coded programs They cons st of Control Flows wh ch ­perform Tasks, and Data Flows, wh ch app y transforms to data com ng from Sources before they are d ­ spatched to Destinations A Data Flow can be executed as a Task, w th n the Control Flow Packages a so conta n Connection Managers wh ch supp y database connect on nformat on and are referenced by Sources and Destinations Control Flows and Data Flows essent a y const tute ma n programs and subrout nes, and the v­ ar ous transforms ava ab e from the SSIS Too box compr se commands and funct ons n the ­programm ng “ anguage ” There s a r ch array of transforms ava ab e, some of wh ch perform very mechan ca operat ons ( ke a sort), whereas others app y more soph st cated processes (such as fuzzy group ng and ookups) and spec a zed tasks re evant to the SQL Server Ana ys s Serv ces (SSAS) components ( ke manag ng new d mens on data) F gure 14-4 shows the SSIS package des gner wh e ed t ng a data flow wh ch conta ns a DQS- and SSAS-spec fic transforms and dest nat ons The ­transforms are a so v s b e n the SSIS Too box and are h gh ghted

Figure 14-4  The SS S des gner, wh e ed t ng a Data F ow task w th var ous B spec fic transforms and dest nat ons.

682  Part III  Applied SQL

SSIS s a spec a ty n and of tse f It s a r ch too , w th many opt m zat on po nts, and t ­competes d rect y (and favorab y) w th expens ve stand-a one ETL products Each transform offers an array of ­configurab e opt ons, through e aborate tabbed d a ogs As f that weren’t enough, the not on of SSIS as a programm ng anguage can be taken beyond ts metaphor ca sense a Scr pt component (wh ch can serve as a Source, a Dest nat on, or a Transform), a ows you to wr te code n V sua Bas c NET or C# to produce data, consume data, or perform transform tasks, based on advanced computat ons, executed through mperat ve code In the BI workflow, SSIS p ays a cruc a ro e n mov ng data out of transact ona databases and nto ana yt ca ones It serves th s funct on dur ng the n t a bu d-out of the atter, but SSIS s not ­re egated to that one-off ro e Ana yt ca databases need to be kept n sync w th the r transact ona source systems, w th as tt e atency as poss b e, and SSIS can be nd spensab e n keep ng those ­operat ona ETL tasks runn ng smooth y It s a very mportant component n the M crosoft BI stack

SQL Server RDBMS, Fast Track DW, and SQL Server PDW It may str ke you as odd that, as we’re we nto our coverage of the M crosoft BI stack, we ment on the SQL Server Re at ona Database Management System (RDBMS) tse f We do so because wh e the pr mary app cat on of the SQL Server RDBMS s as a transact ona database, t s a so an exce ent data mart and data warehouse so ut on Before we exp a n why, et’s define what these terms actua y mean

Data Marts and Data Warehouses The purpose of both data marts and data warehouses s to prov de conso dat on po nts for ­d sparate systems, reconc e the r data structures, un fy the r data, and a ow quer es across the un verse of transact ons There’s no hard and fast defin t on that d st ngu shes a data mart from a data ­warehouse, other than the re at ve cr ter on that the atter s b gger than the former A good ru e of thumb, however, s that data marts tend to be more departmenta n scope and more ad hoc n construct on and des gn Data warehouses tend to be enterpr se-w de and a -encompass ng and shou d be very carefu y and forma y des gned, n order to be proper y factored and fu y accommodat ng ( nsofar as poss b e) of a data that s enterpr se-w de n re evance Data marts and data warehouses d ffer from transact ona databases not just n the r purpose, but n the r structure The h gh y norma zed schemas that are the bas s of transact ona databases work very we for operat ons or ented around retr ev ng and wr t ng s ng e rows of data, or sma sets of rows The a most obsess ve mperat ve of avo d ng repeated nformat on and factor ng re ated ­nformat on out nto separate tab es makes for effic ency n storage, and t’s both og ca and e egant But when t comes t me to query and aggregate data across numerous re ated tab es, the number of jo ns that may be requ red n a s ng e query qu ck y becomes mpract ca So too does the need to scan through so many rows where the granu ar deta s at the eve of a transact on



Chapter 14  Pervas ve ns ght   683

The Star Schema It s for th s reason that data marts and data warehouses are typ ca y structured us ng a so-ca ed star schema In th s configurat on, des gners determ ne the numer ca co umns that need to be ­ana yzed (known as the measures, e g , sa es), and the categor es these w need to be ana yzed across (known as dimensions, e g , t me, geography, account manager, d str butor, etc ) Des gners a so need to ­determ ne the “gra n” at wh ch measures shou d be stored For examp e, rather than stor ng sa es numbers down to the eve of an nd v dua transact on or ne tem, you may on y care to know the tota sa es per day, z p code, account manager, and d str butor In other words, even before your ­quer es aggregate anyth ng, your mart or warehouse a ready stores the data at a gra n that nvo ves some sma amount of aggregat on Once these des gn dec s ons are made, the mart or warehouse schema s des gned n such a way that a measure data s p aced n a s ng e tab e (ca ed the fact table) and each d mens on s p aced n ts own tab e The fact tab e conta ns fore gn keys to nk to the d mens on tab es at the eve of the gra n, and these serve as the pr mary keys for the d mens on tab es D mens on tab es w conta n a denorma zed v ew of each of the h erarch ca eve s n a d mens on So, for examp e, a geography d mens on tab e’s pr mary key wou d be at the eve of posta code, but each row m ght conta n the correspond ng c ty, state/prov nce, and country Therefore the same c ty, state/prov nce, and country names wou d appear repeated y D mens ons can be defined over a set of norma zed tab es nstead (one tab e per h erarch ca eve ), but typ ca y the s ng e denorma zed tab e approach s used nstead When the norma zed approach s used, the d mens on s sa d to fo ow a snowflake schema, rather than a star schema Th s approach to bu d ng an ana yt ca database us ng re at ona techno ogy s effect ve, c ever, and popu ar It s a so prob emat c n certa n ways When perform ng quer es w th mu t p e ­d mens ons at aggregated eve s (e g , sa es by account manager, country, fisca quarter, and some d ­ str butor group ng), SQL Server must perform a somet mes comp ex star join (so named because t jo ns tab es n a star schema format on), and must use a GROUP BY c ause n order to do t These can be ­expens ve quer es on the r own, but hav ng to perform a number of them terat ve y can rea y ­become tax ng on the RDBMS eng ne

SQL Server Data Warehouse Appliances M crosoft has severa spec a -purpose arch tectures and techno og es, both w th n and beyond the ­re at ona database rea m, to opt m ze d mens ona query work oads One approach prescr bes ­bu d ng out SQL Server nfrastructure us ng opt m zed process ng, network and storage arch tectures, and packag ng up a the hardware and software, pre-tuned and configured, as an app ance Some of M crosoft’s OEMs, ke De and HP, se such turnkey SQL Server-based data warehouse app ances under the Fast Track Data Warehouse mon ker Customers can acqu re these app ances through M crosoft Fast Track partners who may further tune them The arch tectures are pub shed, a ow ng partners and customers to bu d them out on the r own, but the abor and expert se nvo ved n the r configurat on can be s gn ficant, so the app ances may offer compe ng va ue Another app ance-based approach nvo ves a re at ve y new add t on to the SQL Server product fam y SQL Server Para e Data Warehouse (PDW) ed t on SQL PDW s based on a data ­warehous ng 684  Part III  Applied SQL

approach ca ed Mass ve y Para e Process ng (MPP) The MPP approach nvo ves aggregat ng a number of SQL Server nstances together n a c uster and d str but ng the query process ng work oads amongst them When quer es are sent to PDW, t d spatches subquer es to nodes n the c uster w th each subquery scoped to a subset of the overa data to be quer ed The node- eve resu t sets are then merged nto a s ng e resu t set that s returned to the c ent The subquer es are carr ed out n para e , a ow ng the overa query to execute much more qu ck y than f a convent ona SQL Server nstance were to execute t The arch tecture of SQL Server PDW s ustrated n F gure 14-5 Database Servers

Storage Nodes

Control Nodes Active/Passive

Landing Zone

Dual Infiniband

Data Center Monitoring

Management Servers

Dual Fiber Channel

Client Drivers

ETL Load Interface

Corporate Backup Solution Corporate Network

Backup Node

Private Network

Spare Database Server

Figure 14-5  Arch tecture d agram for SQL Server Para e Data Warehouse ed t on.

In add t on, the same sorts of CPU-, network ng-, and storage-opt m zed arch tectures used by Fast Track are emp oyed by PDW and the product s so d as an app ance, assur ng we -executed ­configurat on and nsta at on PDW s not cheap, and t’s not the on y MPP product out there But t s h gh y compet t ve w th the other MPP products on the market, both n performance and pr ce For data warehouses n the hundreds-of-terabytes sca e, PDW offers a huge y powerfu and effect ve SQL Server–based data warehouse so ut on



Chapter 14  Pervas ve ns ght   685

Analysis Services As nnovat ve and we perform ng as the Fast Track and PDW approaches are, they are st ­nonethe ess based on the dea of us ng star schemas to repurpose re at ona databases to perform d mens ona quer es and take on ana yt ca work oads Aggregat ons st need to be performed w th T-SQL’s m ted aggregat on funct ons and GROUP BY construct, perform ng often very comp ex jo ns A though t’s mpress ve that opt m zed CPU/network ng/storage strateg es make such quer es feas b e, the fact rema ns that re at ona databases rea y were not des gned w th these work oads n m nd So what f SQL Server offered another database eng ne that were? Enter OLAP and SSAS

The Multidimensional Engine SSAS actua y has more than one eng ne, but et’s start by d scuss ng the most ongstand ng one ts bas c mu t d mens ona OLAP eng ne OLAP stands for OnL ne Ana yt ca Process ng, wh ch ­d st ngu shes t from the OnL ne Transact ona Process ng (OLTP) work oads wh ch are the sweet spot of re at ona databases OLAP eng nes don’t need to be concerned w th ndexes, key constra nts, or the need to retr eve nd v dua rows For the most part, they are not concerned w th creat ng or updat ng data at a They don’t even organ ze data nto tab es, rows, and co umns! Instead, measures and d mens ons, wh ch are mere y abstract concepts n the context of re at ona data warehouses, are the fundamenta objects n OLAP databases A ke-gra ned group ng of measures and d mens ons coex st n OLAP cubes, and mu t p e cubes can coex st n a s ng e OLAP database The SSAS mu t d mens ona cube des gner d sp ay ng a cube bu t around the Northw nd database s shown n F gure 14-6 The star schema on wh ch the cube s based s shown n the arge center area of the des gner, and the cube’s measures and d mens ons appear on the upper- eft and ower- eft, respect ve y

Figure 14-6  The SSAS mu t d mens ona cube des gner.

686  Part III  Applied SQL

W th such a structure n p ace, the who e not on of jo n ng across tab es and group ng rows s gone, as that work takes p ace on y when the cube s bu t or updated, not when t s quer ed ­Instead, SSAS quer es s mp y spec fy the necessary measures and the d mens ona eve s across wh ch the ­measures shou d be aggregated Quer es can a so be expressed across more than two axes So, ­conceptua y, nstead of return ng data n a structure represent ng an nd v dua spreadsheet, SSAS can return a structure represent ng a workbook (that s, a co ect on of spreadsheets each n ts own tab), a fo der (w th a co ect on of such workbooks), a dr ve (w th a co ect on of such fo ders), and so forth SSAS has ts own query anguage ca ed MDX (wh ch stands for Mu t D mens ona eXpress ons) MDX s opt m zed for d mens ona quer es, and offers an a most d zzy ng array of funct ons that are far super or to the re at ve y m ted set of aggregat ng and ana yt c funct ons n SQL ­Ana ys s Serv ces ­ ggregat ons s opt m zed for aggregat ng dur ng query t me, but t a so can pre-ca cu ate certa n a ­before quer es even take p ace Th s s ach eved through the creat on of aggregat ons n, and ­know edge of the h erarch ca re at onsh ps w th n, the cube For examp e, mag ne a cube had a ­geography d mens on w th posta code as the gra n, but that ■

■

The cube was configured to recogn ze the natura , h erarch ca re at onsh ps between posta code, c ty, state/prov nce, and country The cube a so stored aggregat ons of sa es by state/prov nce

These sett ngs and aggregat ons wou d a ow the cube to return sa es at the state/prov nce eve w thout hav ng to aggregate the sa es for a posta codes n each c ty and a c t es n each state Beyond that, f sa es at the country eve were needed, SSAS wou d know that t cou d der ve these va ues by summ ng the sa es for a states/prov nces n each country and s nce these wou d be ­pre-aggregated, t wou d not need to aggregate anyth ng at the ower eve s of c ty or posta code These sorts of opt m zat ons make ana yt ca quer es run very qu ck y on SSAS Aggregat ons can be bu t a gor thm ca y, or bu t based on the degree to wh ch they wou d benefit ogged quer es Aggregat ons can a so be created manua y, a though that s a feature geared to very advanced SSAS users SSAS cubes must be per od ca y processed, n order to be updated w th any data n the data warehouse that s new or changed s nce the cube was ast processed (or first bu t) Cubes are usua y part t oned so that process ng, wh ch can be t me-consum ng, need be app ed on y to a phys ca y sma part t on of the cube

PowerPivot and SSAS Tabular Mode The re ease of SQL Server 2008 R2 brought w th t a product ca ed PowerP vot, wh ch runs on ­desktop PCs n the form of an Exce add- n, and runs as a shared server w th n SharePo nt Th s may sound mmater a to Ana ys s Serv ces, but that’s actua y not the case at a A though PowerP vot ­executes separate y from standard Ana ys s Serv ces, t s nonethe ess based on SSAS techno ogy, and the SharePo nt vers on of PowerP vot must actua y be nsta ed from the SQL Server nsta at on med a That techno ogy uses a d fferent storage techno ogy than do “convent ona ” SSAS cubes, but

Chapter 14  Pervas ve ns ght   687

w th the re ease of SQL Server 2012, that same storage mode becomes ava ab e (as an opt on) on SSAS tse f The new storage eng ne, ca ed Vert Paq, s part of SQL Server’s fam y of xVe oc ty n-memory techno og es, and s a column store–type database eng ne (Chapter 15 covers xVe oc ty n-memory techno og es n deta ) Co umn stores organ ze the r data such that va ues of a g ven co umn are stored together rather than a the va ues n a row be ng stored together For BI, th s works we , s nce most ana yt ca quer es end up aggregat ng over one or, at most, a few co umns, and add ng the r ­va ues s faster when the va ues for other co umns don’t have to be sk pped over dur ng ­aggregat on In add t on, ­stor ng co umn va ues n prox m ty makes for h gh compress on rates, s nce va ues of a co umn are often c ose, at east n order of magn tude H gh compress on rates, n turn, a ow ­databases of a reasonab e s ze to be p aced n memory, and xVe oc ty n-memory techno og es exp o t that strategy Query ng an n-memory database s extreme y fast, and avo ds the need to create ­aggregat ons, as must be done w th convent ona cubes The PowerP vot Exce add- n supp es ts own c ent w ndow, wh ch s shown n F gure 14-7 Three exp c t measures and one key performance nd cator (KPI) are h gh ghted at the bottom, as are the sma buttons ava ab e to sw tch between the Data V ew mode (p ctured) and the D agram V ew mode (more s m ar to what’s d sp ayed n F gure 14-6)

Figure 14-7  The PowerP vot c ent w ndow, n Data V ew mode.

688  Part III  Applied SQL

In the 2008 R2 t meframe, the convent ona mu t d mens ona (OLAP) mode was the on y one ­ va ab e n the standard SSAS product Databases of the “BI Semant c Mode ” (BISM) type—that s, a Vert Paq/xVe oc ty co umn store–based mode s—were on y ava ab e through PowerP vot In SQL Server 2012, BI Semant c Mode s can be created n SSAS as we When SSAS ma nta ns BI ­Semant c Mode s, t s sa d to be operat ng n “Tabu ar” mode, so-named because the BI Semant c Mode ­exp c t y references ts const tuent re at ona tab es, rather than measures and d mens ons per se F gure 14-8 shows the SSAS Tabu ar mode des gner; note that a though t s hosted n V sua ­Stud o, t bears a strong resemb ance to the PowerP vot c ent The Formu a bar and Data V ew/ D agram V ew contro s (h gh ghted) are essent a y the same n both env ronments A spec a ­extra too bar and menus (a so h gh ghted) rep ace the PowerP vot c ent w ndow’s r bbon Un ke w th ­PowerP vot, SSAS BI Semant c Mode s feature a D rectQuery mode, wh ch a ows the r source ­databases, nstead of the co umn store mode tse f, to be quer ed d rect y Th s mode can be turned on or off ( t s off by defau t) us ng a Propert es w ndow sett ng wh ch s h gh ghted as we

Figure 14-8  The SSAS Tabu ar des gner.

When an SSAS nstance s nsta ed, t must be configured n e ther mu t d mens ona mode or Tabu ar mode If you w sh to have both mu t d mens ona and Tabu ar mode SSAS databases on the same mach ne, you w need to nsta two nstances of SSAS (for examp e, the defau t nstance and a named nstance) w th one nstance configured n mu t d mens ona mode and the other n Tabu ar mode



Chapter 14  Pervas ve ns ght   689

L ke PowerP vot, SSAS Tabu ar mode supports the use of a new express on syntax ca ed DAX (Data Ana ys s eXpress ons), wh ch uses an Exce - ke set of funct ons for ana yt ca query ­PowerP vot and SSAS Tabu ar mode s a so support MDX, a ow ng ex st ng SSAS c ent app cat ons to query ­PowerP vot mode s on SharePo nt and Tabu ar mode s n SSAS (PowerP vot mode s on oca mach nes can on y be quer ed by Exce ) In the Enterpr se ed t on of SQL Server 2012, an mp ementat on of xVe oc ty n-memory t­ echno og es for the relational eng ne a so ex sts A though the fu BISM arch tecture s not nvo ved, ­creat ng a spec a “Co umnstore” ndex on a SQL Server tab e does n fact create an xVe oc ty co umn store n the background Chapter 15 d scusses th s and other aspects of xVe oc ty techno og es n more deta

Data Mining In add t on to ts OLAP/BISM capab t es, SSAS has ts own e aborate data m n ng eng ne Th s eng ne can exam ne a body of data and create stat st ca mode s (us ng var ous a gor thms) that can be used to pred ct a va ue for one co umn g ven known va ues for the others These pred ct ons can be ­de vered w th n resu t sets generated from spec a y formu ated SQL quer es The pred ct ons and mode s are based on near regress on and other ana yt ca methods that genera ze the observed ­corre at ons n the data for wh ch a va ues are known M n ng structures, wh ch n turn conta n one or more m n ng mode s, can be added to SSAS mu t d mens ona mode s Typ ca y each m n ng mode w th n a m n ng structure w use a d fferent a gor thm

Tip  Sometimes it’s advantageous for a single mining structure to contain multiple mining models that use the same algorithm, but with different parameter settings. The m n ng structure des gner s shown n F gure 14-9, w th an ex st ng Dec s on Trees ­ gor thm-based mode d sp ayed The v ewer shows how the var ous data used to tra n the mode a sp ts nto data popu at ons, then sp ts aga n nto subpopu at ons, and so on A so known by the names predictive analytics and machine learning, data m n ng can be used to opt m ze promot ona ma ngs, or as the eng ne for e-commerce upse /cross-se fac t es, and can even be used n med ca app cat ons, based on exam ned data around treatment efficacy, and so forth SSAS data m n ng s very neat techno ogy and s re at ve y easy to use, espec a y w th the Data M n ng add- n for Exce Unfortunate y, SSAS data m n ng has had very tt e nvestment s nce the SQL Server 2005 vers on of the eng ne was re eased A company ca ed Pred x on Software, founded by a former ead ng member of the SSAS Data M n ng team, has products wh ch bu d on top of SSAS data m n ng and extend ts capab t es

690  Part III  Applied SQL

Figure 14-9  The SSAS m n ng structure ed tor, shown v ew ng an ex st ng mode bu t w th the Dec s on Trees

a gor thm.

Power View Just as the server vers on of PowerP vot runs w th n SharePo nt but s actua y sh pped w th SQL Server, so too s a new BI component ca ed Power V ew Power V ew s an ad hoc report ng, ana ys s, and data v sua zat on too for BISM data sources Power V ew uses S ver ght to render compe ng, an mated v sua zat ons, and runs w th n SharePo nt, ut z ng ts document-based secur ty mode Many advanced propert es that can be configured n BISM data sources are d rect y recogn zed by Power V ew A Power V ew report, n fu screen mode, s shown n F gure 14-10 The report conta ns a s ng e page, w th a co umn chart at the top and a bubb e chart at the bottom Not ce the se ect ons n the bubb e chart are fi ter ng the data shown n the co umn chart The fi tered va ues appear n a dark co or and the va ues for the tota s appear n the ghter co or at the top of each co umn



Chapter 14  Pervas ve ns ght   691

Figure 14-10  A Power V ew report w th co umn chart and bubb e chart v sua zat ons. Se ect ons n the atter are fi ter ng va ues n the former.

Reporting Services It may seem counter ntu t ve to ment on SSRS n a BI context, g ven how preva ent use of SSRS s for operat ona report ng aga nst transact ona , re at ona databases But SSRS s a bona fide BI too , too, due to severa esser known features that t supports F rst among those features s the nat ve ab ty n SSRS to ssue MDX quer es aga nst SSAS cubes, br ng ng back the mu t d mens ona resu ts n the form of “flattened” tabu ar resu t sets The MDX q ­ uer es can be supp ed by the report deve oper, but they can a so be defined v sua y Th s means SSRS can be used as an SSAS c ent, and one wh ch requ res no MDX know edge, but st ­a ows that k­ now edge to be app ed, f des red Furthermore, th s funct ona ty works aga nst SSAS ­mu t d mens ona cubes, PowerP vot workbooks ( f they have been up oaded to SharePo nt), and SSAS Tabu ar mode s SSRS a so features some of the best v sua zat on techno ogy n the M crosoft BI stack, nc ud ng numerous types of charts, gauges, maps, data bars, spark nes, and nd cators A though these work just fine aga nst re at ona data, they work espec a y we for SSAS aggregated data, n the context of a dashboard

692  Part III  Applied SQL

Report Parts W th SQL Server 2008 R2, SSRS ga ned the ab ty (us ng the stand-a one Report Bu der 3 0 ­app cat on) to pub sh out the nd v dua tab es, matr ces, and v sua zat ons from any report to a ga ery—where they can be se ected ater, through drag-and-drop, for nc us on n new reports Pub sh ng these so-ca ed Report Parts enab es a se f-serv ce usage scenar o and a ows end-users to construct SSRS reports more eas y, espec a y s nce a under y ng data sources and datasets are mp c t y mported w th the se ected ga ery e ements Report Bu der, w th ts Pub sh Report Parts d a og box d sp ayed, s shown n F­ gure 14-11

Figure 14-11  The Pub sh Report Parts d a og box, w th a tab e and chart se ected for pub cat on.

Alerting New to SSRS n SQL Server 2012 s an A ert ng feature A ert ng automat ca y ema s spec fied ­rec p ents when a va ue n a report exceeds or fa s be ow a defined thresho d, or meets some other formu a c cond t on To some extent, SSRS A ert ng restores funct ona ty that was ost w th the ­deprecat on of SQL Server Not ficat on Serv ces (wh ch was removed n SQL Server 2008) The ­a ert ng adm n strat ve nterface s SharePo nt-based, but on y requ res SharePo nt Foundat on, wh ch s the free e ement of SharePo nt nc uded w th W ndows Server On the SQL Server s de, A ert ng ­funct ona ty s on y ava ab e n the BI and Enterpr se ed t ons of the product



Chapter 14  Pervas ve ns ght   693

Dashboard Components F na y, as we w deta short y, SSRS reports can be nc uded n PerformancePo nt dashboards, ­mak ng SSRS a fu -fledged BI too

Excel and Excel Services Exce may seem we outs de the SQL Server un verse, unt you rea ze that t nc udes a number of features spec fica y geared towards Ana ys s Serv ces Exce can connect to Ana ys s Serv ces ­nat ve y In the Data tab of Exce ’s r bbon, you need on y c ck the Get Externa Data group’s From Other Sources button, and Ana ys s Serv ces w be the second opt on n the menu, r ght after SQL Server ( e , the re at ona eng ne) tse f Once you’ve connected, you can eas y query SSAS databases or PowerP vot mode s us ng ­ votTab es The P votTab e F e d L st w enumerate a measure groups, measures, KPIs, d mens ons, P attr butes, h erarch es, and named sets n your database or mode , and any act ons w be ava ab e n the P votTab e through a r ght-c ck shortcut menu se ect on (Act ons are an advanced SSAS feature creat ng context sens t ve nks to web pages, data, SSRS reports, and other externa content re evant to a part cu ar d mens on, member, or measure ) Exce charts can be eas y created from P votTab es, mak ng Exce a fu -fledged data v sua zat on too for SSAS and PowerP vot Exce a so features severa funct ons n ts formu a anguage spec fica y for query ng cubes, so even regu ar spreadsheet ce s (as opposed to just P votTab e reg ons) can be SSAS- and PowerP vot-connected As w th Report ng Serv ces, Exce (and thus Exce Serv ces, d scussed next) can query mu t d mens ona cubes, PowerP vot workbooks on SharePo nt, and SSAS Tabu ar mode s F gure 14-12 shows an Exce chart bu t aga nst data n a PowerP vot mode

Using Excel Services Exce Serv ces s a component of SharePo nt that perm ts ent re workbooks, nd v dua ­spreadsheets, or spec fic P votTab es, charts, and ce reg ons to be exported to SharePo nt and v ewed ­interactively from a web browser Th s means that workbooks conta n ng SSAS-or PowerP vot-query ng ­P votTab es, charts, and ce ranges can be shared w th anyone w th a web browser and ­perm ss ons to a proper y outfitted SharePo nt server P votTab es can be dr ed down upon, and fi ters and ­parameter va ues can be ­app ed nteract ve y, n the browser, w thout any vers on of Exce requ red on the c ent mach ne F gure 14-13 shows an Exce Serv ces d sp ay of a PowerP vot-connected chart w th a “s cer” contro ­a ow ng the data to be fi tered by product co or

694  Part III  Applied SQL

Figure 14-12  An Exce chart data dr ven from a PowerP vot mode .

Figure 14-13  Exce Serv ces v ew of a worksheet w th PowerP vot dr ven chart and s cer.



Chapter 14  Pervas ve ns ght   695

PerformancePoint Services PerformancePo nt Serv ces (PPS) was once a stand-a one product but was eventua y conso dated nto SharePo nt It a ows for the creat on of BI scorecards (wh ch are sts of KPIs, grouped by ­object ve, show ng the KPI va ues, and the r status assessments), ana yt c gr ds, and ana yt c charts (gr ds and charts that a ow dr -down operat ons, as we as dr ng across d mens ons, r ght from the browser) aga nst a var ety of data sources, w th SSAS foremost among them These features a one make PPS re evant to th s chapter’s d scuss on But what makes t a ock s the ab ty of PPS to create dashboards that ntegrate ts own assets descr bed above w th content bu t w th SSRS and Exce Serv ces Not on y can these d sparate BI dashboard components appear ­s de-by-s de, but fi ters can be added that w a ter the content of a or some of them s mu taneous y In th s way, PPS acts as g ue that bonds together d fferent e ements of the M crosoft BI stack and n turn s mp fies th ngs for end users, who don’t have to use mu t p e too s separate y to benefit from the capab t es of each one F gure 14-14 shows a PPS dashboard conta n ng a PPS scorecard and ana yt c chart, as we as both a map and gauges created w th Report ng Serv ces The PPS and SSRS components coex st n the dashboard rather seam ess y

Figure 14-14  A PerformancePo nt Serv ces (PPS) dashboard w th nat ve PPS and SSRS e ements.

696  Part III  Applied SQL

StreamInsight The ast BI stack component to d scuss s StreamIns ght, M crosoft’s comp ex event process ng (CEP) eng ne that prem ered w th SQL Server 2008 R2 CEP eng nes process data streams where the ­magn tude and frequency of data s extreme y h gh Th nk about sensor data or financ a market data as examp es; n both cases, you can mag ne the mmense vo ume of data and the speed w th wh ch t streams n Ord nar y, keep ng up w th th s k nd of data feed s very d fficu t; CEP seeks to make t manageab e In the case of StreamIns ght, ts code, wr tten n C++, runs very “c ose to the meta ,” a ow ng for ow atency n process ng An adapter arch tecture s used to abstract away nterfaces w th spec fic source and dest nat on data systems Once co ected, the data s he d n abeyance unt process ng code, wh ch can be wr tten n NET and can use LINQ to query StreamIns ght, comes around to d gest the data, or unt an output adapter pushes the data somewhere F gure 14-15 dep cts an arch tectura summary of StreamIns ght

Stream ns ght App cat on Deve opment ven Sources

Devices, Sensors

Source

Stream ns ght Server

Web servers

Source

Query

ven s ores & Da abases

Source

Query

S ock icker, news eeds

M crosoft V sua Stud o

Query

Stream ns ght L brary

M crosoft .NET

ven Targe s

S nk S nk S nk

Pagers & Moni oring devices ven s ores & Da abases KP Dashboards, SharePoin U

Trading s a ions

Figure 14-15  An arch tectura summary of Stream ns ght.

StreamIns ght ne ther uses SQL Server as a repos tory, nor does t ntegrate d rect y w th SSAS or other M crosoft BI stack components But t does sh p w th SQL Server and one can certa n y mag ne a data warehouse and/or cube be ng bu t on top of ts stream ng data, espec a y for operat ona BI app cat ons

SQL Server Editions and SharePoint Version Requirements Pr or to SQL Server 2012, customers who were nterested n the BI capab t es of SQL Server ­rea y had to purchase the Enterpr se ed t on of the product A though SQL Server Standard ed t on ­nc udes SSAS, severa of ts features are unava ab e n that ed t on Th s had meant top-of-the- ne censes were needed for BI work, even for customers who were st gett ng the r toes wet w th t But SQL

Chapter 14  Pervas ve ns ght   697

Server 2012 ntroduces a new BI ed t on, w th pr ces and features that fa between Standard and Enterpr se The BI ed t on does not nc ude advanced data warehouse features n the SQL Server ­relational eng ne, ke part t on ng and xVe oc ty n-memory techno ogy-based co umnstore ndexes, but t does nc ude MDS, DQS, SSIS, SSAS, SSRS, PowerP vot for SharePo nt, Power V ew, and StreamIns ght Th s s gn ficant y ­owers barr ers to entry for soph st cated BI work on the M crosoft p atform and makes the atter compare even more ­favorab y on econom c terms w th ts compet tors Power V ew requ res the SharePo nt Enterpr se C ent Access L cense (eCAL), as do Exce Serv ces, PPS, and the SharePo nt mp ementat on of PowerP vot So an enterpr se cense on the SharePo nt s de s st requ red for the most soph st cated web-based ana ys s work, but the reduct on n cense burden on the SQL Server s de s a we come deve opment Note that wh e Exce 2007 and SharePo nt Server 2007 support some of what has been d scussed n th s chapter, the 2010 vers ons are needed for the fu su te of funct ona ty In part cu ar, ­PowerP vot and Power V ew w not work at a w th the 2007 ed t ons of those products Tab e 14-1 prov des a summary of SQL Server 2012 ed t ons and the SQL Server BI features each nc udes

Note  In Table 14-1, SQL Server Compact is not represented, as it is really a separate ­technology. Fast Track Data Warehouse and Parallel Data Warehouse edition are not ­represented either, as they are distinct relational data warehouse appliances that define their own feature sets. Table 14-1 SQL Server ed t ons and nc uded BI feature sets SSAS Multidimensional, StreamInsight standard SSIS, Standard Report Builder, Report Parts,

SSRS Alerting, SSAS Tabular, MDS, DQS

PowerPivot for SharePoint, Power View

Edition

SSRS

Express

A

Web

•

•

Standard

•

•

B

Bus ness • nte gence

•

•

•

S

•

•

S

Enterpr se, Deve oper

•

Legend: A= Advanced Serv ces (free down oad) requ red; B = bas c feature set for SSAS; S = SharePo nt Enterpr se C ent Access L cense (eCAL) requ red.

698  Part III  Applied SQL

DW features (Columnstore indexes, compression, partitioning), advanced SSIS, StreamInsight Enterprise

•

Summary Many SQL Server pract t oners work exc us ve y w th the product’s re at ona eng ne, and many users n the BI market cont nue to pay s gn ficant sums to cense stand-a one BI products and su tes from other vendors Meanwh e, M crosoft nc udes w th SQL Server one of the most capab e and comp ete BI stacks n the ndustry The 2008 R2 stack was a ready mpress ve, as t nc uded Master Data Management, report ng, ETL, mu t d mens ona OLAP, comp ex event process ng, and n-memory co umn store ana yt cs ­embedded nto Office and SharePo nt But SQL Server 2012 makes the stack s gn ficant y more capab e by ­add ng A ert ng, Data Qua ty, stand-a one co umn store ana yt cs, re at ona Co umnstore ndexes, and ­advanced se f-serv ce ana ys s and data v sua zat on Whether or not you w sh to get heav y nto BI, t’s cruc a to understand what SQL Server’s BI capab t es are, and to know how easy some of them ( ke PowerP vot and Power V ew) are to use Th s chapter has taken you past that hurd e and maybe even p qued your nterest n exp or ng these techno og es further



Chapter 14  Pervas ve ns ght   699

C hapter 1 5

xVelocity In-Memory Technologies —Andrew Brust

W

th the re ease of SQL Server 2008 R2, M crosoft de vered mass ve new Bus ness Inte gence (BI) va ue n the form of PowerP vot, wh ch brought co umn store techno ogy to the M crosoft BI stack Co umn stores organ ze data n a co umn-or ented fash on and use advanced a gor thms to compress the r data The co umn store and compress on techno og es comb ne to a ow even very arge data mode s to fit ent re y n memory, thus a ow ng for ghtn ng-fast response t mes for ­ana yt ca quer es PowerP vot runs on the c ent PC as a M crosoft Exce add-In, and that’s great But t a so runs on M crosoft SharePo nt as a true server, a ow ng for shared ana ys s of data mode s and, by way of a few coo tr cks, a ow ng any Ana ys s Serv ces c ent, nc ud ng Report ng Serv ces, to connect to and query the mode w th MDX So PowerP vot was a great start for co umn store techno ogy n the M crosoft wor d But now there’s more SQL Server 2012 ups the ante w th enhancements to PowerP vot and the ­ntroduct on of co umn store techno og es nto SQL Server These features and capab t es are together branded “xVe oc ty n-memory techno og es ” They nc ude a new “Tabu ar” mode n SQL Server Ana ys s Serv ces (SSAS) wh ch hosts PowerP vot-sty e mode s, add ng enterpr se features ke part t on ng and secur ty, a w thout any dependency on Exce or SharePo nt Such an advance was expected, as PowerP vot s and was a man festat on of SSAS The other way SQL Server 2012 has mproved ts co umn store game s qu te unexpected, however the techno ogy beh nd PowerP vot has now been mp emented n the SQL Server Enterpr se ed t on relational eng ne as a spec a type of ndex ca ed a columnstore ndex W th t, M crosoft has a so dev sed a new query process ng mode ca ed Batch mode

Info  Don’t get hung up on the word “batch” as it relates to the “batch jobs” of old or a Transact SQL (T-SQL) statement batch. SQL Server Batch mode isn’t about punch cards, ­submitting queries to run overnight, or combining several T-SQL queries before a GO ­command. Instead, it refers to processing batches of rows together, rather than one at a time. The ­latter, conventional processing mode is now called Row mode. When SQL Server uses a co umnstore ndex and Batch mode together, t can process quer es w th vast y mproved speed—somet mes an order of magn tude faster than what wou d be ­poss b e

701

w th convent ona ndexes In fact, t makes frequent, terat ve, ana yt ca quer es h gh y feas b e r ght ns de a re at ona data warehouse For some scenar os, t can even e m nate the need to use SSAS ­Arguab y, then, xVe oc ty n-memory techno ogy n the SQL Server re at ona eng ne g ves you BI w thout the cubes But wa t; not so fast As much as xVe oc ty may seem to “ berate” re at ona pract t oners from SSAS, t a so prov des a bunch of ncent ves for them to check t out That’s because the SSAS mode s that use xVe oc ty techno og es a so mp ement someth ng ca ed the Bus ness Inte gence Semant c Mode (BISM), wh ch prov des a re at ona -fr end y fac ty and a new express on anguage ca ed Data Ana ys s eXpress ons (DAX; a so used n PowerP vot) that makes report ng and ana ys s aga nst your data eas er for your users and a so eas er for you Have “xVe oc ty wh p ash” yet? Confused about whether co umnstore ndexes or BISMs are where t’s at? We , don’t be torn because, as t happens, you can use the two together, and then t e n ­v sua zat on too s ke Power V ew to prov de se f-serv ce ana yt cs, h gh-performance query ­capab t es, and re at ona storage together, a n one We’ cover a of that n th s chapter In th s chapter, you w

earn

■

What co umn store databases are and the r ro e n today’s BI marketp ace

■

How xVe oc ty compress on works, n “short strokes ”

■

Wh ch quer es are most su tab e for co umnstore ndex opt m zat on

■

How to create a SQL Server co umnstore ndex and how to make sure t’s be ng used

■

A about the BI Semant c Mode and what t does

■

How to use DAX for measures, Key Performance Ind cators (KPIs), and ca cu ated co umns

■

How to enab e the D rectQuery mode for Tabu ar mode s n SSAS

■

The bas cs of Power V ew, how t ntegrates w th the BI Semant c Mode , and by extens on, re at ona databases

Column Store Databases Co umn store databases have become very hot commod t es n the data warehous ng and BI markets For those used to th nk ng of BI products as “d mens ona ,” the not on of a “co umn” s mp y sounds ke re at ona techno ogy be ng used n p ace of on ne-ana yt ca process ng (OLAP) techno ogy And wh e n a sense that s true, t’s not rea y the po nt Co umn store databases are so-named because they store va ues for a column together rather than segregat ng data by row At first, that may sound absurd But when you ook at the reason ng beh nd a co umn store and the ram ficat ons of us ng one, you see that t great y benefits the ana yt ca query process F gure 15-1 d sp ays the transformat on from a row store structure to a co umn store for a tab e (s mp fied for conceptua c ar ty) 702  Part III  Applied SQL

Row Store EmpID

Age

Income

1

43

90000

2

38

100000

3

35

100000

…

…

…

Column Store EmpID

1

2

3

…

Age

43

38

35

…

Income

90000

100000

100000

…

Figure 15-1  Mapp ng row stores to co umn stores.

Th s tab e conta ns emp oyee data and a more rea st ca y comp ex vers on of t cou d be the bas s for a fact tab e n a data warehouse If you ook at the Income co umn’s data n ts co umn-or ented format, two th ngs become c ear ■

■

Aggregat ng the data over a arge number of rows becomes very easy, s nce th s nvo ves ­scann ng adjacent data Some of the va ues repeat, and a of them are c ose n va ue

What are the ram ficat ons of these two observat ons? F rst, aggregat on quer es can be ­performed very effic ent y Second, the Income co umn (wh ch m ght serve as measure data) s a great ­cand date for h gh-rat o compress on When va ues repeat, they can be represented as one va ue w th a ­number of occurrences, and when data s c ose n va ue, t can be represented by a base va ue that s ­expressed once, a ong w th the de tas from that base va ue for each nd v dua data po nt (where the de tas w requ re ess storage than the fu va ues) Mere y stor ng co umn va ues together a ds the performance of ana yt ca quer es, but c­ ompress on owers storage costs and acce erates performance even more Less I/O s nvo ved n read ng the data, and arge parts, or a , of query work ng sets — even ent re databases — can be oaded nto memory, e m nat ng d sk access and the atency that accompan es t

Column Store Tech in the BI Industry Numerous co umn store products ex st n the market, many of wh ch have been acqu red by the arger IT vendors A number of data warehouse app ance products emp oy co umn store techno ogy, as do the n-bu t BI eng nes n var ous dashboard and data v sua zat on products So M crosoft s not the on y vendor us ng the techno ogy, nor s t the first to do so But w th the re ease of SQL Server 2012 and the advent of xVe oc ty n-memory techno og es, M crosoft has ntegrated co umn store techno ogy n ts techno ogy stack n a w despread way, both n spec a zed products ke PowerP vot and SSAS Tabu ar mode, as we as n the ma nstream SQL Server re at ona database That s un que, unprecedented, and, most mportant, an extreme y powerfu too for SQL Server profess ona s

Chapter 15  xVe oc ty n Memory Techno og es   703

xVelocity in the RDBMS: Columnstore Indexes We be eve BI techno ogy shou d not be v ewed as separate from re at ona techno ogy, but rather as someth ng adjacent to t on the data spectrum The add t on of co umnstore ndexes to the SQL Server re at ona eng ne supports th s v ew of data techno ogy n a very concrete way Many quer es wh ch before wou d have had reasonab e e apsed t m ngs on y n SSAS can now be accommodated ns de re at ona databases, too Co umnstore ndexes are ndexes n every sense of the word They res de n the re at ona database, are backed up and restored a ong w th t, and are based on the co umns you des re The cost-based query opt m zer n SQL Server w do a good job of us ng co umnstore ndexes when they are usefu and w nstead use convent ona row store ndexes at other t mes But co umnstore ndexes can a so be thought of, n effect, as data stores n the r own r ght That’s actua y true of a ndexes, as they conta n the data for the co umns on wh ch they’re bu t But you w typ ca y bu d co umnstore ndexes on all the co umns n the ndexed tab e As such, any quer es where the co umnstore ndex wou d be usefu can get most or a of the r data, rather than just some of t, from the ndex tse f

Building a Columnstore Index Bu d ng a co umnstore ndex requ res use of the standard T-SQL CREATE INDEX syntax, w th the add t on of the COLUMNSTORE keyword For our s mp fied tab e n F gures 15-1 and 15-2, we m ght create a co umnstore ndex on a co umns n the tab e ke th s CREATE NONCLUSTERED COLUMNSTORE INDEX ColStoreEmployee ON Employee (EmpID, Age, Income)

Once a co umnstore ndex s bu t, your aggregat ng quer es stand to be h gh y acce erated We’ve cut to the chase to show you how easy co umnstore ndexes are to create, and how ­comp ete y ntegrated they are nto the norma T-SQL workflow But n rea ty, there are severa th ngs you need to understand before you can product ve y put co umnstore ndexes to work So et’s take a ook at them

What You Can’t Do In order to take fu advantage of co umnstore ndexes, you have to know where they he p and how to ensure they are be ng used But before you earn any of that, t’s mportant to know what the ­ m tat ons are for co umnstore ndexes M crosoft Books On ne prov des a comprehens ve reference on these m tat ons, but we’ st some of the h gh ghts here ■

Co umnstore ndexes (and thus the tab es on wh ch they’re bu t) are read-on y

■

Co umnstore ndexes cannot be c ustered or un que

■

Co umnstore ndexes are not compat b e w th Filestream data

704  Part III  Applied SQL

■

■

■

■

Co umnstore ndexes cannot nc ude sparse co umns or co umns of the fo ow ng data types binary, varbinary, ntext, text, image, varchar(max), nvarchar(max), uniqueidentifier, rowversion, timestamp, sql variant, CLR types ( nc ud ng hierarchyid and spat a types), and xml Co umnstore ndexes cannot accommodate decimal (or numeric) co umns w th prec s on greater than 18 d g ts or datetimeoffset co umns w th sca e greater than 2 Co umnstore ndexes cannot be created on v ews or ndexed v ews, and cannot conta n more than 1,024 co umns Co umnstore ndexes are ncompat b e w th rep cat on, change track ng, and change data capture

At first b ush, th s seems ke a ong st of restr ct ons But read over them carefu y and you w rea ze that very few of the th ngs that a co umnstore ndex can’t do are th ngs you would do n a data warehouse Most data warehouses have fact tab es w th numer c measures and keys, and ­d mens on tab es w th correspond ng keys and str ng/character data The d sa owed b nary, arge text, ­vers on ng, var ant, CLR, and XML data types are not th ngs that you wou d aggregate, or use as key va ues And wh e you m ght use a uniqueidentifier as a key va ue n your transact ona database, a numer c key can be eas y subst tuted for t n a data warehouse The read-on y restr ct on on co umnstore ndexes s perhaps the most m t ng of them a But s nce many data warehouses operate on a n ght y data oad, n many cases t w be acceptab e to drop the co umnstore ndex before the oad and rebu d t as soon as the oad s comp ete Th s assures a m n ma ma ntenance w ndow once per n ght In cases where oads occur more frequent y, new data can be oaded n a stag ng tab e, wh ch can per od ca y be co umnstore- ndexed and then swapped n as a new part t on n the read-on y tab e Does a of th s mean that co umnstore ndexes work on y w th data warehouses? Techn ca y, no, co umnstore ndexes can be used w th any database, nc ud ng transact ona ones Nonethe ess co umnstore ndexes mprove performance the most n ana yt ca quer es typ ca y performed n data warehouse scenar os Spec fica y, co umnstore ndexes work w th quer es that ■

Group, and aggregate arge row sets

■

Use star jo ns (preferab y) and nner jo ns (exc us ve y)

The dea use case for a co umnstore ndex s when t s bu t on a data warehouse fact tab e and ­nc udes a co umns n that tab e That sa d, transact ona tab es that conta n measure data and keys can be treated as f they were fact tab es, and bu d ng co umnstore ndexes on the fact tab e- ke ­co umns can great y a d performance on ana yt ca quer es aga nst those tab es So th nk of ndexes on fact tab es as the dea pedagog ca co umnstore examp e and then rea ze that co umnstore ndexes can he p you n scenar os that are comparab e, even f ess than perfect y comp ant, w th that opt ma case Actua y, the very not on that data warehouse- and d mens ona -sty e BI work can be done on databases that do not feature a forma star schema s exact y the dea that the BI Semant c Mode , espec a y n PowerP vot, s based upon

Chapter 15  xVe oc ty n Memory Techno og es   705

How Columnstore Indexes Work Now you have an overv ew understand ng of creat ng co umnstore ndexes, and a sense of when and where you shou d do so But you are st tak ng on author ty how and why they he p w th certa n types of quer es Not on y can that be nte ectua y d ssat sfy ng, but n pract ca terms t prevents you from troub eshoot ng when the query opt m zer n SQL Server ends up not us ng your co umnstore ndexes, and you th nk t shou d have The d agram n F gure 15-2 represents the page structure of a co umnstore ndex, and po nts out that on y the co umns of nterest are fetched and scanned Column Store Page Structure Only queried columns’ pages are retrieved

Salary Emp ID

Age

1

43

2

38

3

35

…

…

90000 100000 100000 …

Figure 15-2  On y pages from quer ed co umns are retr eved from a co umnstore ndex.

Each co umn’s data s stored n ts own ser es of spec a pages A group of such pages makes up a “segment,” each of wh ch ho ds approx mate y 1 m on rows’ worth of a s ng e co umn’s compressed data Each segment s, n fact, noth ng more than a SQL Server b nary arge object (BLOB) stored n the database a ong w th a convent ona tab e data Co umn data can be compressed n a number of ways Run- ength encod ng can be used to c­ ompress consecut ve appearances of the same va ue as a s ng e express on of that va ue and the number of t mes t occurs If a re at ve y sma set of d st nct va ues ex st n the co umn, then each one can be ass gned a sma key va ue and the keys can be stored nstead of the va ues they ­represent A so, s nce co umnstore ndex segments store metadata dent fy ng the r sma est and argest ­va ues, who e segments can be sk pped f a cond t on n the query ru es out the range of va ues n the ­segment Th s techn que s ca ed “segment e m nat on ” Co umnstore ndexes nvo ve mu t p e ayers of effic ency a ow ng, n many cases, the ent re ­ ork ng set to fit n memory But t gets better the query processor (QP) can process co umn va ues w n batches of approx mate y 1,000 rows, rather than one at a t me So not on y are scans made faster due to remova of extraneous co umns from the data, but who e batches of data can be processed together as the scan takes p ace Th s g ves r se to the new Batch execut on mode n SQL Server, wh ch s where the rea y huge performance ga ns come from In other words, co umnstore ndexes do acce erate certa n quer es, but the r nd rect contr but on—of fac tat ng Batch execut on mode—prov des the rea boost Th s does 706  Part III  Applied SQL

add a tt e comp ex ty though, because t’s poss b e for the QP to use a co umnstore ndex but st work n Row mode And wh e that may st be benefic a , t forfe ts much of the speed-up To make certa n the QP s enter ng Batch mode, you’ need to ook at your query execut on p an And because Batch mode s nvoked on y n para e process ng scenar os, you’ need to make sure that the server’s max degree of para e sm (MDOP) configurat on opt on s set e ther to zero or a ­va ue greater than one You must, of course, a so ensure that SQL Server has mu t p e processor cores at ts d sposa The atter requ rement s espec a y mportant to check f you’re work ng w th a ­v rtua zed nstance of SQL Server s nce, by defau t, Hyper-V a ocates on y a s ng e og ca processor to ts v rtua mach nes (VMs) To use Batch mode, you must make sure to a ocate at east two To do th s w th Hyper-V, you’ need to shut down your VM fu y, as the number of og ca ­ rocessors sett ng s not ed tab e wh e the VM s runn ng or saved Once you’ve shut down the p VM, r ght-c ck t and choose Sett ngs from the context menu In the Sett ngs d a og box, c ck the ­Processor opt on n the eft-hand s debar (fourth from the top), then c ck the drop-down arrow for the Number Of Log ca Processors sett ng and se ect a va ue greater than one Th s task s shown n F gure 15-3

Figure 15-3  Ass gn ng the number of og ca processors to a VM.

Once th s change s made, c ck the OK button (not shown n F gure 15-3) and start the VM aga n Once your mu t core env ronment s ready, go ahead and open SQL Server Management ­Stud o (SSMS), connect to your database, enter a query su ted to co umnstore ndex opt m zat on as ­descr bed

Chapter 15  xVe oc ty n Memory Techno og es   707

above, and c ck the Show Est mated P an too bar button Look n the p an for usage of your co umnstore ndex (for examp e, n a Co umnstore Index Scan), scro ng to the r ght f necessary Po nt to that node n the p an to revea the deta s too t p The process ng mode for the est mated p an w be ­d sp ayed on the th rd ne of the too t p Make certa n that th rd ne says “Batch” for Est mated ­Execut on Mode, and that the fourth ne nd cates “Co umnStore” for Storage, as shown n F gure 15-4

Tip  If the table you are querying has a relatively small number of rows, the query ­ processor may not invoke Batch mode, even if multiple processor cores are available. You can simulate the presence of a greater number of rows using the ROWCOUNT and PAGECOUNT options of the UPDATE STATISTICS T-SQL command, as described in a blog post from the SQL Server query optimization team at http://blogs.msdn.com/b/ queryoptteam/archive/2006/07/21/674350.aspx. The UPDATE STATISTICS command with the ROWCOUNT and PAGECOUNT options should never be used on a production database. Even in a development database, it’s best to run the command on a copy of your table with the relevant columnstore index created. Once you are done profiling the columnstore ­index, you can drop the copy of the table permanently, thus removing any bogus statistics from the database. If you’d like to experiment with a sample database containing sufficient data to invoke Batch mode without the UPDATE STATISTICS trick, have a look at the first two units in Module 3 of Microsoft’s SQL Server 2012 Developer Training Kit at http://social. technet.microsoft.com/wiki/contents/articles/6982.sql-server-2012-developer-training-kitbom-en-us.aspx#Module 3 Exploring and Managing SQL Server 2012 Database Engine Improvements. Beyond co umnstore ndex scans, ook for other shapes where Batch mode may be used (for examp e, n a re ated Hash Match) Before your query, f you’d ke, you can turn on stat st cs for CPU and e apsed t mes Then when your query’s comp ete, c ck the Messages tab to observe the t m ngs for each You cou d eas y contrast these t m ngs w th those for non-co umnstore-ass sted operat ons by add ng the fo ow ng c ause at the end of your query OPTION (IGNORE_NONCLUSTERED_COLUMNSTORE_INDEX)

Th s w force the QP not to use your co umnstore ndex V ew the est mated p an to confirm th s, and then execute the query When the query comp etes, c ck the Messages tab to revea the nonco umnstore-ass sted stat st cs They may be much greater than those from the prev ous query Depend ng on your data, hardware, and other factors, your m eage may vary, and t may take some tweak ng of the query to see the des red resu ts But hopefu y you w be ab e to exper ence first-hand just how revo ut onary co umnstore ndex techno ogy s

Tip  Improved results from columnstore indexes tend to be more prominent when queries are run against data sets of 2 to 3 million rows.

708  Part III  Applied SQL

Columnstore Index Scan (NonClustered) Scan a columnstore index, entirely or only a range. Physical Operation Logical Operation Estimated Execution Mode Storage Estimated I/O Cost Estimated Operator Cost Estimated Subtree Cost Estimated CPU Cost Estimated Number of Executions Estimated Number of Rows Estimated Row Size Ordered Node ID

Columnstore Index Scan Index Scan Batch ColumnStore 0.0090509 0.559059 (12%) 0.559059 0.550008 1 10000000 19 B False 6

Object [AdventureWorksDW2012].[dbo].[FactResellerSalesCS]. [NonClusteredColumnStoreIndex] Output List [AdventureWorksDW2012].[dbo]. [FactResellerSalesCS].ResellerKey, [AdventureWorksDW2012]. [dbo].[FactResellerSalesCS].SalesAmount Figure 15-4  Check ng execut on and storage modes n a query p an.

At th s po nt, you shou d have a good apprec at on for the power of xVe oc ty co umn store and compress on techno ogy, and of how t can be app ed n the re at ona database rea m We w now move to exam n ng ts app cat on n the ana yt ca rea m But have no fear We w come fu c rc e and br ng th s d scuss on back to the re at ona context by the end of the chapter

xVelocity for Analysis: PowerPivot and SSAS Tabular Models In the ast sect on we focused pretty heav y on the nner work ngs of xVe oc ty co umn store and compress on techno ogy In th s sect on, we w move away from that eve of d scuss on, and d scuss the spec a y purposed ana yt ca database techno ogy that xVe oc ty a so supports Rest assured that the same c ever techno ogy s there, under the covers, but the spec fics of t are ess mportant Instead, the extra capab t es and features that resu t are mportant As we ment oned ear er, PowerP vot s based on SSAS But nstead of us ng the convent ona SSAS mu t d mens ona storage and execut on eng ne, PowerP vot uses xVe oc ty n-memory ­techno og es to enab e a new co umn store and n-memory BI (IMBI) execut on eng ne, ca ed Vert Paq The ­n-memory operat ons are, of course, made poss b e by the co umn store and compress on n ­Vert Paq



Chapter 15  xVe oc ty n Memory Techno og es   709

Note  With the introduction of the xVelocity brand, Microsoft has officially renamed VertiPaq to the “xVelocity in-memory analytics engine.” We continue to use the older name here, though we identify VertiPaq as being part of the family of xVelocity in-memory ­technologies.

Clearing Up the Analysis Services Vocabulary W th SQL Server 2012, Ana ys s Serv ces now runs n two d fferent modes—the mode t had worked n exc us ve y n a SQL Server vers ons from 7 through 2008 R2, and a new mode that s based on the same Vert Paq eng ne that s n PowerP vot The vocabu ary used to refer to each mode can be rather ­ncons stent and confus ng Before we go any further, et’s estab sh what’s what, and our own ­convent on for how we w cons stent y refer to each mode The or g na mode n SSAS d dn’t need a name before, because t was the on y mode the ­eng ne worked n It was, and s, based on OLAP techno ogy and offers three storage opt ons MOLAP ­(Mu t d mens ona OLAP), where n t uses ts own storage techno ogy; ROLAP (Re at ona OLAP), where n t uses and stores data n the re at ona database; and HOLAP (Hybr d OLAP), a ­comb nat on of the two techn ques Because MOLAP was the on y fu y nat ve storage mode, th s s the name somet mes ass gned to the ent re or g na non-Vert Paq mode of SSAS You m ght th nk one name wou d be good enough, but n fact a few nterchangeab e terms have ar sen For examp e, s nce the “M” n “MOLAP” stands for mu t d mens ona , that term s somet mes used as the or g na mode’s name As a var ant of the mu t d mens ona mon ker, the acronym UDM, wh ch stands for Un fied D mens ona Mode , and was used heav y when Ana ys s Serv ces 2005 was ntroduced, s somet mes used as we So the or g na Ana ys s Serv ces mode has three names ­MOLAP, mu t d mens ona , and UDM In th s chapter, we w cons stent y use the term ­“mu t d mens ona ” to avo d confus on, but be aware that the other two names are used ­ nterchangeab y e sewhere Now, what about the new mode based on PowerP vot’s techno ogy? We , one th ng t does not get ca ed s “PowerP vot mode,” wh ch s probab y for the best But severa other names, nc ud ng Vert Paq mode, BISM, IMBI, Co umnar, and Tabu ar mode, are band ed about In th s chapter we’ use the ast of these terms (“Tabu ar mode”) to refer to the new mode The name “Tabu ar” s used by M crosoft because mode s that use t exp c t y recogn ze tab es (and co umns), rather than measures and d mens ons, as first-c ass members of the mode And speak ng of the word “mode ,” t’s mportant to note that th s term s typ ca y app ed on y when the Tabu ar mode s n use, whereas the word “cube” s app ed when the mu t d mens ona mode s n force That makes sense, to a po nt, but t a so can get very confus ng, s nce Tabu ar mode does support MDX quer es, and thus MDX c ents, wh ch w v ew a Tabu ar model as f t were a mu t d mens ona cube It’s unfortunate that the vocabu ary has evo ved w thout much r gor, but t’s understandab e, g ven that one mode can emu ate the other By us ng the terms “mu t d mens ona ” and “Tabu ar” n th s chapter, we hope to m t gate the confus on, s nce these terms refer to the un que storage techno ogy used n each mode 710  Part III  Applied SQL

When we speak of the techno ogy that PowerP vot and Ana ys s Serv ces Tabu ar mode have in common, though, we w use the term BISM, s nce both products do n fact mp ement BI ­Semant c Mode s So, BISM w reference the techno ogy, and Tabu ar w reference the Ana ys s Serv ces ­configurat on mode that uses t, and the databases that SSAS manages n that mode

The Lowdown on BISM What BISM enab es s Ana ys s Serv ces– ke funct ona ty ( nc ud ng support for MDX quer es, ­essent a y for backward compat b ty) but w th far s mp er mode ng Instead of requ r ng prec se des gn of measures, d mens ons, attr butes, aggregat ons, and the ke, BISM a ows the mash ng up of tab es from var ous data sources, and then requ res spec fy ng the re at onsh ps between them In the case of PowerP vot, BISM a ows users to bu d the r own quer es, se ect ng numer c fie ds that act as measures, and group ng them by other co umns that w act as d mens ons But users don’t need to be aware of that d st nct on, nor do they need to understand that the r data s n a spec a database that can be quer ed w th the new DAX anguage or the o d MDX anguage Users just th nk they’re work ng w th data Tech-savvy users w recogn ze the env ronment as re at ona ; other users won’t rea y worry about t BISM can treat any tab e as a fact tab e, a d mens on tab e, or both So wh e a database organ zed a ong a star schema w work rea y we , t’s not tru y necessary If th s sounds fam ar, t shou d In the prev ous sect on, we saw how co umnstore ndexes enab e ana yt ca quer es n a re at ona database, and can do so aga nst norma zed transact ona tab es, even f they work best w th star schema fact and d mens on tab es To us, that’s the rea xVe oc ty story t br ngs ana yt ca query power to you, rather than mak ng you trave to where that power ex sts But f you are w ng to trave a b t, xVe oc ty and BISM reward you, espec a y n the new Tabu ar mode of Ana ys s Serv ces, and the correspond ng new vers on of PowerP vot In th s new vers on, a number of c ass c Ana ys s Serv ces features— nc ud ng forma y defined measures, h erarch es, and KPIs—have been added A so, a number of advanced propert es have been added to make the data more d scoverab e n ana ys s c ents ke Power V ew Let’s take a ook at these mode ng features on the PowerP vot and SSAS s de, and then see how they can be comb ned w th the mmed acy of ­co umnstore ndexes on the re at ona s de

Friends, Countrymen, Bring Me Your Data F rst, et’s cover the bas cs of BISM data source compat b ty BISM ets you mport data from a w de array of data sources Here they are, broken down by category ■

C ent/Server SQL Server (and SQL Azure), Orac e, DB2, Sybase, and Inform x

■

Data warehouse app ances SQL Server Para e Data Warehouse ed t on (PDW) and Teradata

■

BI data sources Ana ys s Serv ces

■



Data Feeds/OData Inc ud ng Azure DataMarket, Report ng Serv ces reports, and SharePo nt sts

Chapter 15  xVe oc ty n Memory Techno og es   711

■

Sma databases and flat fi es Access, CSV, and other text fi es

■

Exce ce ranges and tab es

■

Genera OLE DB and ODBC data sources

The reason that so many d sparate data sources can feed a s ng e mode s that the mode s a separate database, so once the data s n the mode t can be mashed up w th data from other sources as ong as re at onsh ps can be set If you’ve worked w th Ana ys s Serv ces n ts mu t d mens ona mode, then you’ recogn ze a s m ar process n creat ng mu t p e data sources and marry ng them n a un fied data source v ew However, w th an Ana ys s Serv ces mu t d mens ona mode database, you wou d then have to des gn a cube W th BISM, you can start query ng the mode r ght away And, n some cases, you can even postpone the spec ficat on of re at onsh ps and a ow them to be automat ca y detected (subject to your rev ew) ater on

Building the BISM Let’s see how a of th s works We’ start from PowerP vot and we’ then move to Ana ys s Serv ces Tabu ar mode That w be easy, s nce ups z ng from one to the other s exp c t y supported, as you w see From here on, we’ cover var ous features and capab t es by bu d ng out a samp e, and ­query ng t n var ous too s We’ document our steps n the form of step-by-step nstruct ons, wh ch w enab e you to fo ow a ong If you prefer just to read the steps and fo ow the screen shots, that’s okay too But cons der the hands-on approach, as t tends to be a great way to earn To start, you’ need to be runn ng Exce 2010 and to down oad and nsta ts PowerP vot ­add- n from www.powerpivot.com. The add- n s free Just make sure to down oad the appropr ate ­vers on (32- or 64-b t) to match the vers on of Exce 2010 that you are runn ng Once the add- n s ­down oaded and nsta ed, start Exce , and c ck on the PowerP vot tab n the r bbon, then c ck on the PowerP vot W ndow button at the far eft, as shown n F gure 15-5 Do ng th s w br ng up the PowerP vot w ndow, where you can mport and mode your data The w ndow has ts own r bbon, whose Home tab’s ent re Get Externa Data group conta ns buttons for mport ng d fferent types of data Th s s shown and h gh ghted n F gure 15-6

Note  The Get External Data group is shown as a drop-down menu in Figure 15-6 due to the resolution of the screen capture. On your PC, the entire group will likely display within the ribbon itself.

712  Part III  Applied SQL

Figure 15-5  The PowerP vot r bbon tab and ts PowerP vot W ndow button.

Figure 15-6  The Get Externa Data r bbon group n the PowerP vot w ndow.



Chapter 15  xVe oc ty n Memory Techno og es   713

A you rea y need, though, s the From Other Sources button toward the bottom-r ght of that ­r bbon group (a so h gh ghted n F gure 15-6) C ck ng t w br ng up the Connect To A Data Source page of the Tab e Import W zard w th a s ng e menu of a poss b e data sources, as shown n F gure 15-7

Figure 15-7  The Connect To A Data Source page of the Tab e mport W zard.

Se ect M crosoft SQL Server, and then c ck Next You’ then be presented w th a data connect on ­d a og F n the deta s necessary to connect to the AdventureWorksDW2012 database and c ck Next

Note  If the AdventureWorksDW2012 database is not installed on your SQL Server instance, download its MDF data file from http://msftdbprodsamples.codeplex.com/downloads/ get/165405 and attach to it in SQL Server Management Studio. Once you have connected to the AdventureWorksDW2012 database, c ck Next In the Se ect Tab es And V ews page, you’ be asked f you want to se ect your data from a st of tab es or supp y your own SQL query; choose the former and c ck Next Once the st of tab es comes up, se ect the f­ o ow ng e ght ■

DimDate

■

DimEmployee

■

DimProduct

■

DimProductCategory

714  Part III  Applied SQL

■

DimProductSubcategory

■

DimPromotion

■

DimSalesTerritory

■

FactResellerSales

Not ce that for each tab e, you can se ect t, then c ck the Prev ew And F ter button to browse the data and, opt ona y, enter fi ter cond t ons Th s s mere y a conven ence feature, as you’ be ab e to fi ter the data after mport ng t as we You can a so set a mode -fr end y a as for each tab e n the Fr end y Name gr d co umn Go ahead and do that now remove the “D m” prefix from the first seven tab e names, set the fr end y name for FactResellerSales to, s mp y, Sales, and add spaces for any tab es that have ntercapped fr end y names (e g , change SalesTerritory to Sales Territory) Now c ck F n sh; once you do, the Import ng page w appear and your data w mport When the mport s comp ete, c ck C ose and your screen shou d ook someth ng ke the one shown n F gure 15-8

Figure 15-8  The PowerP vot w ndow, w th mported data.

Dial M for Modeling As you can see, PowerP vot prov des a spreadsheet- ke ook and fee , w th each tab e hav ng ts own tab (a though, remember, t uses ts own w ndow, separate from the actua Exce c ent) Each co umn can be h dden, unh dden, de eted, or renamed, by r ght-c ck ng the co umn header and se ect ng

Chapter 15  xVe oc ty n Memory Techno og es   715

the appropr ate opt on from the context menu You can c ck on the r bbon’s Des gn tab and use the Add or De ete buttons for perform ng those funct ons on co umns, or doub e-c ck the co umn header to rename the co umn You can a so c ck the co umn header drop-down button and enter fi ter ­cond t ons n the resu t ng pop-up w ndow Tab es can kew se be de eted or renamed by r ght-c ck ng the correspond ng tab and se ect ng the correspond ng funct on from the context menu, or by doub e-c ck ng the tab to ­rename Tab es can a so be moved w th n the tab order, e ­ ther through the tab’s Move context menu opt on and the resu t ng Move Tab e d a og box, or by d ­ ragg ng the tab of the tab e to be moved to the des red ocat on In th s way you can mode your data, and you can do so n a manner that smoothes out the d fferences between nam ng convent ons n d fferent databases

Calculated Columns PowerP vot a so ets you define ca cu ated co umns And t’s pretty easy, too just c ck a ce n a tab e s first empty co umn, then c ck n the formu a bar, enter a formu a there, and press the Enter key You can try th s on the Sales tab e, rather eas y, w th the fo ow ng formu a =[SalesAmount]+[TaxAmt]+[Freight]

After enter ng th s formu a and press ng the Enter key, wa t a moment and you w ­ca cu ated va ue appear n that same co umn, n every row n the tab e

see the

You m ght a so not ce that the name of your new co umn s CalculatedColumn1 Doub e-c ck that name n the co umn header to ed t t, and change t to TotalCharge Note that a though th s co umn s defined by a formu a, the va ues are n fact mater a zed n the mode In other words, the ­ca cu at on s done at co umn defin t on t me; at query-t me, the actua va ues w be ava ab e, just as f they had come from the phys ca data source The formu a we just ooked at s a DAX formu a, a be t a rather s mp e one Let’s ook at one that’s more comp cated Th s t me, for each row n the Sales tab e, wh ch represents the sa e of a p ­ art cu ar product, we want to have a ca cu ated co umn that shows the average sa es amount for a rows ­conta n ng products n the same product subcategory To ca cu ate th s we’ need to re y on the fact that the Sales tab e s jo ned to the Product tab e wh ch, n turn s jo ned to the Product Subcategory tab e Our formu a w have to traverse the “hop” to the Product tab e, get the ProductSubcategoryID va ue correspond ng to the current row’s product, then ook at a rows n the Sales tab e whose products are a so n that product subcategory and get the average of the SalesAmount co umn for that who e set of data To make th ngs a b t eas er on ourse ves, we can create an ntermed ate ca cu ated co umn ca ed ProductSubcat, and define t as =RELATED('Product Subcategory'[EnglishProductSubcategoryName])

Tip  In DAX, table names are always surrounded by single quotes and column names are surrounded by square brackets. There is no space or period between the two. 716  Part III  Applied SQL

Th s DAX formu a te s PowerP vot to use the va ue of the EnglishProductSubcategoryName co umn n the re ated row n the Product Subcategory tab e and make t the va ue of ProductSubcat ­Imp c t y, the formu a must traverse the Product tab e to get to the correspond ng row n the Product ­Subcategory tab e W th ProductSubcat defined, we can get the average sa es number we need w th one more ­ca cu ated co umn, ca ed AvgSalesForSubcat Here s ts formu a =AVERAGEX(FILTER('Sales',[ProductSubcat]=EARLIER([ProductSubcat])),'Sales'[SalesAmount])

Because th s formu a nests three d fferent DAX funct ons, t requ res some exp anat on The FILTER funct on supp es the tab e Sales as ts first parameter, nd cat ng that we want to take a rows n that tab e and supp y a fi ter cond t on to them The fi ter cond t on we want s one that says se ect a rows whose ProductSubCat va ue s equa to that for th s part cu ar row The express on [ProductSubcat]= EARLIER([ProductSubcat]) g ves us just that (th nk of “ear er” as mean ng “th s one”) We then app y the AVERAGEX funct on over a those rows for the SalesAmount co umn, g v ng us the average sa es amount for a products n that subcategory We cou d now create one more ca cu ated co umn, ca ed SalesAboveAverage, us ng th s formu a =[SalesAmount]-[AvgSalesForSubcat]

Th s wou d g ve us the amount above or be ow the product subcategory average that a part cu ar row’s sa e represents Aggregat ng that over var ous categor es can prov de a usefu benchmark of how good sa es are, n re at ve terms Not ce we never had to spec fy the re at onsh ps between Sales, Product, and Product Subcategory, but we rather cou d just assume they were n p ace That’s because the re at onsh ps were bu t for us automat ca y when PowerP vot encountered the correspond ng constra nts n the database Typ ca y you’ on y need to spec fy re at onsh ps between tab es mported from d fferent data sources Let’s take a ook at how th s works by mport ng data from Exce

Analysis in Excel, Using Data from Excel To connect to an Exce workbook, use the From Other Sources button n the PowerP vot w ndow aga n In the Connect To Data Source page of the Tab e Import W zard, se ect Exce F e from the st ( t’s the second-to- ast entry), and then c ck Next In the Connect to M crosoft Exce page, c ck the Browse button, and then nav gate to and se ect the Resellers.xlsx workbook from th s chapter’s samp e code Check the Use F rst Row As Co umn Headers checkbox, and then c ck Next In the Se ect Tab es and V ews page, you shou d see the DimReseller$ sheet sted and se ected; change ts fr end y name from DimReseller to Reseller and c ck F n sh The data w then mport When the mport s done, c ck C ose and the new rese er data w be d sp ayed n ts own new tab

It’s All about Relationships Now et’s take a ook at the re at onsh ps that a ready ex st n the mode C ck the r bbon’s Des gn tab and then the Manage Re at onsh ps button You shou d see a st of re at onsh ps, a nvo v ng tab es from the AdventureWorksDW2012 SQL Server database Note that the re at onsh p from the Sales

Chapter 15  xVe oc ty n Memory Techno og es   717

tab e to the Sales Territory tab e s not act ve; we w need to fix th s ater, but we need to create a re at onsh p between the Sales tab e and new y mported Reseller tab e now C ose out of the ­Ma nta n Re at onsh ps d a og box, and then c ck the r bbon’s Create Re at onsh p button In the Create ­Re at onsh p d a og box, se ect Sales from the Tab e drop-down st and ResellerKey from the Co umn drop-down st The other two drop-down sts n the d a og box shou d now be enab ed, so se ect Reseller from the Re ated Lookup Tab e drop-down st ResellerKey shou d have been automat ca y se ected for you n the Re ated Lookup Co umn drop-down st; f not, then se ect t manua y now Now just c ck OK, and the re at onsh p w be added

Modeling, Part Deux So far, the mode ng techn ques we’ve ooked at were a ava ab e n the first vers on of PowerP vot, wh ch was re eased w th SQL Server 2008 R2 But s nce the second vers on accompan es SQL Server 2012, wh ch a so features BISM-based databases n SSAS Tabu ar mode, a number of new features have been added to both BISM hosts to support common SSAS features Let’s take a ook A though we haven’t yet ooked at Exce quer es aga nst PowerP vot mode s (we w soon), we’ ­exp a n now that mere y us ng a PowerP vot mode s co umn n the Va ues sect on of an Exce ­P votTab e or P votChart makes that co umn nto a measure n the eyes of the Vert Paq eng ne But wh e that’s conven ent for end users who don’t want to spend too much t me w th data mode ng, t wou d be n ce f we cou d create exp c t measures n the mode , and then h de the raw co umns beh nd them It wou d a so be n ce f we cou d construct exp c t h erarch es rather than forc ng users to, for examp e, se ect ­nd v dua fie ds ke EnglishPromotionCategory, EnglishPromotionType, and ­EnglishPromotionName nto a P votTab e and arrange them n nested fash on The good news s that the SQL Server 2012 vers on of PowerP vot ets you construct such h erarch es as we

Creating Measures The eas est way to create an exp c t measure s to se ect one row of a co umn’s data, and c ck on the AutoSum button toward the top-r ght of the r bbon’s Home tab Th s w create a sum-based ­measure on that co umn Try t w th the SalesAmount co umn n the Sales tab e and you w get a measure ca ed Sum Of Sales Amount It w be represented by a s ng e entry beneath the SalesAmount co umn data n the PowerP vot w ndow’s ca cu at on area (wh ch can be h dden or shown by c ck ng the ­Ca cu at on Area button at the far r ght of the r bbon’s Home tab) C ck that entry and you w see that the measure s backed by another DAX formu a, d sp ayed n the formu a bar Ed t the formu a so that the name on the eft s de of the := ass gnment operator s Sales Amount (w th a space), rather than Sum of SalesAmount, and press the Enter key Th s s a shown n F gure 15-9 Now create four more measures n the same manner Base one on the UnitPrice co umn (and name t Unit Price, w th a space), and the others on the TotalCharge, AvgSalesForSubcat, and SalesAboveAverage ca cu ated co umns you created ear er (nam ng them Total Charge, Average Sales For Subcat, and Sales Above Average, respect ve y) Now create one more measure on SalesAmount, but make th s one average-based nstead of sum-based, by c ck ng the drop-down arrow on the AutoSum r bbon button and choos ng Average, as shown n F gure 15-10

718  Part III  Applied SQL

Figure 15-9  A new y created measure.

Figure 15-10  Bu d ng an average based measure.



Chapter 15  xVe oc ty n Memory Techno og es   719

Ed t the formu a of the new measure so that t’s named Average Sales Amount W th th s done, you have a ot of good measures to work w th, and you can h de a the phys ca co umns n the tab e We’ do that a b t ater though

Building KPIs KPIs a ow you to create scorecards As a fina mode ng adjustment before we ook at h erarch es, et’s create a KPI Se ect the sum-based Sales Amount measure and then c ck the Create KPI r bbon ­button Th s w br ng up the Create KPI d a og box Configure t as shown n F gure 15-11 and then c ck OK

Figure 15-11  Bu d ng a Key Performance nd cator.

Th s configurat on w n effect convert the Sales Amount measure to a KPI The KPI’s va ue s the measure’s va ue and the target s based on the hard-coded va ue of $5,000 The status s configured so that a va ue of $4,500 or more s mapped to a green c rc e con for the KPI’s v sua zat on, a va ue between $4,000 and $4,500 s mapped to a ye ow c rc e, and a va ue of ess than $4,000 s mapped to a red c rc e

Hierarchies Come to BISM Now et’s bu d out some h erarch es W th th s and our other work, our mode w , n BI-speak, expose pre-defined d mens ons, measures, and a KPI To bu d h erarch es, we must change from the Exce - ke Gr d v ew to D agram v ew You can do th s by se ect ng the D agram V ew button (th rd from r ght, at h gher reso ut ons) n the r bbon’s Home tab or the D agram button at the far- ower r ght of the data gr d Both are h gh ghted n F gure 15-12 720  Part III  Applied SQL

Figure 15-12  Sw tch ng to D agram v ew.

Once n the D agram v ew, you shou d see a ayout of the tab es n our mode , a ong w th a ­dep ct on of the re at onsh ps between them BI pros w recogn ze th s as a star schema, but that actua y doesn’t matter so much The tab e n the m dd e (essent a y, our fact tab e) has the numbers we’ be query ng and the other tab es (essent a y, our d mens on tab es) have the category data we’ be aggregat ng by If we ook at the Sales Territory tab e, we can see that the co umns for SalesTerritoryCountry and SalesTerritoryRegion cou d form a h erarchy C ck the Sales Territory tab e’s Max m ze button (at the upper-r ght), then rename the aforement oned to co umns to just Country and Region (br ng the co umn name nto ed t mode by doub e-c ck ng the name or r ght-c ck ng t and choos ng ­Rename from the context menu) Now ho d down the Ctrl key and c ck both fie ds, n any order When they are se ected, r ght-c ck one of them and choose Create H erarchy from the context menu The ­H erarchy w be created for you, but w th the name Hierarchy1; rename t to Country – Region nstead F na y, se ect a four rema n ng co umns, then r ght-c ck one of them and choose H de From C ent Too s from the context menu

Tip  We could actually hide the Country and Region columns too, since in Excel they will still be exposed through the hierarchy. However, BISM hierarchies are not visible in Power View, so ­hiding the columns would leave nothing selectable for the Sales Territory table in that tool.



Chapter 15  xVe oc ty n Memory Techno og es   721

Tab e 15-1 sts three other h erarch es that you can bu d In each case, you shou d h de a co umns n the re evant tab es that are not part of h erarch es, and rename the rema n ng co umns to match the r h erarchy nam ng (but rename the FiscalSemester and FiscalQuarter co umns to Fiscal Semester and Fiscal Quarter, rather than Semester and Quarter) Table 15-1 H erarchy and co umn name ass gnments New Hierarchy (and Implied New Column Names)

Table

Columns

Promotion

EnglishPromotionCategory, EnglishPromotionType, EnglishPromotionName

Category

Date

CalendarYear, CalendarSemester, CalendarQuarter, EnglishMonthName

Calendar Year

Date

FiscalYear, FiscalSemester, FiscalQuarter, EnglishMonthName

Fiscal Year

Type

Promotion

Semester

Semester

Quarter

Quarter

Month

Month

Note  For each of the hierarchies in the Date table, the designer will guess wrong about the hierarchy order, and will put Calendar Year and Fiscal Year as the third level in each hierarchy, instead of the first. Fix this by dragging each year-related level to the top of its hierarchy.

Finishing Touches To fin sh up, there’s a tt e more work to do F rst, r ght-c ck on the re at onsh p between the ­Employee and Sales Territory tab es, se ect De ete from the context menu, and confirm the de et on n the resu t ng message box Next, doub e-c ck the re at onsh p between Sales and Sales Territory (represented as a dotted ne) In the resu t ng Ed t Re at onsh p d a og box, check the Act ve checkbox and c ck OK Now max m ze the Sales tab e, se ect a the co umns, but not the five measures or the KPI (the cons at the eft edge nd cate f an tem s a measure or KPI), then r ght-c ck one of the co umns and choose H de From C ent Too s from the context menu When you’re done, c ck the Sales tab e’s Restore button, at the upper-r ght Now, do these fina few c eanup steps ■

■

■

H de a co umns n the Product, Product Subcategory, and Product Category tab es except EnglishProductName, EnglishProductSubcategoryName, and EnglishProductCategoryName, ­respect ve y Rename those three co umns to Product, Product Subcategory, and P ­ roduct Category, respect ve y In the Employee tab e, rename the FirstName and LastName co umns to have spaces before the r ntercapped etters, and h de a other co umns In the Reseller tab e, rename BusinessType, ResellerName, AnnualSales, and AnnualRevenue to have spaces before the r ntercapped etters, and h de a the other co umns

722  Part III  Applied SQL

■

In order to remove error messages on some of the ca cu ated co umns that resu ted from one of our co umn name changes, sw tch back to Data V ew mode and ed t the formu a for the ProductSubcat co umn to reference Product Subcategory rather than EnglishProductSubcategoryName The new formu a shou d appear as fo ows =RELATED('Product Subcategory'[Product Subcategory])

Press Enter once you’re done ed t ng and any exc amat on po nt warn ngs shou d d sappear

Exploring Advanced Mode You’re a most ready to query your mode n Exce now, but before you do, c ck the r bbon’s F e tab and then the Sw tch To Advanced Mode opt on Th s w add the Advanced tab to the r bbon C ck that tab; t shou d appear as shown n F gure 15-13

Figure 15-13  The PowerP vot w ndow r bbon s Advanced tab.

The advanced opt ons a ow you to add more semant c nformat on to your mode That ­nformat on s then exposed to ana ys s and report ng c ents wh ch can take advantage of t At the t me of th s wr t ng, the on y such c ent app cat on s M crosoft’s Power V ew, wh ch we ooked at br efly n the ast chapter and w ook at aga n a b t ater n th s one Here s a summary of the advanced funct ons, nc ud ng exp anat ons of what they do ■

■

■

■

■



The Perspect ves button ets you define m ted v ews of your mode that nc ude on y certa n tab es, or even certa n fie ds w th n them Perspect ves can then be quer ed by c ent too s as f the former were mode s n the r own r ght Th s s great when you have comp ex mode s and groups of users who w on y be nterested n part of what’s n them The Show Imp c t Members button w d sp ay n the Measures gr d any co umns that were added to the Va ues sect on of an Exce P votTab e or P votChart, and wh ch are thus treated by the Vert Paq eng ne as measures The Summar ze By button, wh ch becomes enab ed when a numer c co umn s se ected, ets you define the defau t aggregat on that a c ent app cat on ( nc ud ng Exce and Power V ew) w app y to the co umn when t’s se ected The Defau t F e d Set button nvokes a d a og box of the same name, and ets you nd cate wh ch co umns shou d show by defau t when the who e tab e, rather than spec fic co umns, s se ected The Tab e Behav or button nvokes a ke-named d a og box that ets you define the tab e’s pr mary key and configure upon wh ch co umn or co umns aggregat on group ng shou d be based It a so ets you define the co umn that w supp y a row’s defau t mage and defau t abe , both of wh ch are mportant w th spec fic v sua zat ons, ke the Card, n Power V ew Chapter 15  xVe oc ty n Memory Techno og es   723

■

The Image URL checkbox, wh ch becomes enab ed when a text co umn s se ected, ets you spec fy that the data n that co umn conta ns URLs to mages, rather than text that shou d tse f be treated as data

Querying in Excel That’s about a the mode ng we need to do, so et’s move on to query ng the mode w th Exce Th s s pretty stra ghtforward Just c ck the drop-down arrow on the P votTab e button ( n the m dd e of the r bbon’s Home tab), and then se ect the Chart And Tab e (Hor zonta ) opt on (the th rd one down) The Create P votChart and P votTab e (Hor zonta ) d a og box w pop up w th opt ons to p ace the content n a new worksheet or n the ex st ng one The defau t se ect on s New Worksheet and t’s fine, but we usua y find Ex st ng Worksheet to make more sense If you do se ect the atter, then make sure that the Locat on text box d sp ays the ce ‘Sheet1’!$A$1 Ed t t f necessary, so that t does, and then c ck OK You shou d then see an empty P votTab e and P votChart and your cursor shou d be somewhere ns de the former At the r ght of the screen s the PowerP vot F e d L st task pane, d sp ay ng each tab e from your mode n ts top ha f Dr down on any or a of the tab es, and you shou d see the r measures, KPIs, and h erarch es, p us any unh dden co umns, exact y as you mode ed them Th s s shown n F gure 15-14

Figure 15-14  Empty P votChart and P votTab e nserted from PowerP vot w ndow, and the PowerP vot F e d L st.

724  Part III  Applied SQL

Be ow the st of tab es, you shou d see s x panes, one each for S cers Vert ca , S cers Hor zonta , Report F ter, Co umn Labe s, Row Labe s, and Va ues Drag and drop fie ds and one h erarchy nto certa n sect ons, as spec fied n Tab e 15-2 Table 15-2 P votTab e fie d and h erarchy ass gnments Field List Area

Table

Fields (or Hierarchy)

Va ues

Sales

Sales Amount Value, Sales Amount Status, Average Sales Amount, Sales Above Average

Row Labe s

Date

Calendar Year

S cers Vert ca

Reseller

Reseller Name, Annual Sales

S cers Hor zonta

Sales Territory

Country, Region

Semester

Quarter

Month

Next, c ck somewhere ns de the P votChart The Co umn Labe s and Row Labe s sect ons of the PowerP vot F e d L st shou d change to the Legend F e ds and Ax s F e ds sect ons, respect ve y, and a areas except S cers Vert ca and S cers Hor zonta w once aga n be empty Drag and drop fie ds nto certa n sect ons, as spec fied n Tab e 15-3 Table 15-3 P votChart fie d ass gnments Field List Area

Table

Fields

Va ues

Sales

Total Charge, Sales Amount Value, Unit Price

Ax s F e ds

Promotion

Category

When you’re done, your workbook shou d appear as shown n F gure 15-15

Figure 15-15  P votChart, P votTab e, and S cers n PowerP vot workbook.



Chapter 15  xVe oc ty n Memory Techno og es   725

A few th ngs bear ment on ng before we cont nue, w th regard to the nteract v ty of the ­v sua zat on we have created ■

■

■

Each year n the P votTab e can be dr ed down to revea ts const tuent semesters, quarters, and months Each rectang e n the “S cers” that appear above and to the eft of the P votChart can be c cked upon for nteract ve s mu taneous fi ter ng of the P votChart and the P votTab e If you ho d down the Ctr key, you can se ect mu t p e S cer members You can a so c ck on the C ear F ter g yph at the top-r ght of the S cer to c ear your se ect ons

PowerPivot for SharePoint If you have SharePo nt Enterpr se 2010 at your d sposa , and PowerP vot for SharePo nt has been ­nsta ed on the SharePo nt server farm, you can save your Exce workbook to SharePo nt and then v ew your ana ys s work n the browser To do th s, s mp y c ck on the Exce r bbon’s F e tab, se ect the Save & Send opt on, and then c ck Save To SharePo nt When the Save As d a og box comes up, e ­ nter the URL to your PowerP vot Ga ery c ck Save to nav gate there, and then c ck Save once more to save the workbook Your workbook shou d then appear n the browser and ook s­ m ar to F gure 15-16

Figure 15-16  A PowerP vot for SharePo nt workbook.

Each of the nteract v ty po nts ment oned n the bu eted st above s ava ab e n the browser env ronment as we , mak ng the browser a great p ace to share PowerP vot ana yses bu t n Exce Of course, what you can’t do n the browser s change the makeup ( e , add and/or remove co umns, measures, KPIs, h erarch es, and the ke) of the P votTab e and P votChart, but you can perform dr downs and make se ect ons n the S cers 726  Part III  Applied SQL

Not ce the PowerP vot Ga ery breadcrumb at the top of the page n F gure 15-16 If you c ck t, you’ be ab e to v ew your documents n Ga ery v ew Each workbook n the ga ery w be sted, and you shou d not ce that next to each one, at the far r ght-hand s de, are buttons to open a new Exce workbook that uses the mode n the current workbook as ts data source, and to open a new Power V ew report aga nst the mode We’ cover that opt on soon, but nstead of us ng Power V ew aga nst the workbook, we’ use t aga nst a fu -fledged SSAS Tabu ar mode

Moving to SSAS Tabular How do we construct that mode ? Wh e we cou d start from scratch, a far eas er method s to ­mport the workbook mode nto a new SSAS Tabu ar mode project To do th s, c ose Exce , save your ­changes, and then fire up M crosoft V sua Stud o 2010 and se ect New Project from w th n the menu or on the Start Page In the New Project d a og, se ect the Bus ness Inte gence\Ana ys s Serv ces node from the Insta ed Temp ates tree v ew on the eft, and then se ect Import From PowerP vot from the st n the center, as shown n F gure 15-17

Figure 15-17  Creat ng an SSAS Tabu ar project by mport ng from PowerP vot.

C ck OK A standard fi e open d a og box w appear, n wh ch you shou d nav gate to and s­ e ect the workbook you just created n Exce Doub e-c ck the fi e, or se ect t and c ck Open The ­PowerP vot mode w then be mported nto the SSAS Tabu ar project



Chapter 15  xVe oc ty n Memory Techno og es   727

Tip  Because of security restrictions on the workbook file, you may see a warning message box explaining that the data from your PowerPivot model cannot be imported, but that the ­metadata can. Should this happen, click Yes. Then, once your project has opened, click the Existing Connections button (third from left) on the Analysis Services toolbar, and then select the SQL Server–based connection and click Process. This will restore your SQL Server data (the Excel-based data will not be needed). As we po nted out n Chapter 14, and showed n F gure 14-8, a of the sett ngs and opt ons that were ava ab e n the PowerP vot for Exce w ndow are ava ab e ns de V sua Stud o as we However, s nce V sua Stud o has no r bbon, these opt ons are hosted n a comb nat on of V sua Stud o menus, the Ana ys s Serv ces too bar, and the Propert es w ndow For examp e, the equ va ent of the From Other Sources button n the Get Externa Data group of the r bbon’s Home tab n the PowerP vot w ndow s the Import From Data Source button on the Ana ys s Serv ces too bar or the Mode Import From Data Source menu opt on Many of the features n the PowerP vot w ndow’s Advanced r bbon tab are ava ab e n the Propert es w ndow n V sua Stud o For examp e, se ect one of the tab es n the mode ( e , c ck ts tab) and note that the Propert es w ndow features propert es for Defau t F e d Set and Tab e Behav or that are equ va ent to the r namesake buttons n the PowerP vot w ndow’s Advanced r bbon tab

Role-Based Security SSAS Tabu ar mode offers a few features that PowerP vot does not Secur ty s one examp e, and t’s rather powerfu To add secur ty to your mode , c ck the Ro es button on the Ana ys s Serv ces too bar or choose Mode Ro es from the ma n menu Th s br ngs up the Ro e Manager d a og box, and f you c ck the New button w th n t, you w go nto a mode where you can define a Ro e A Ro e s defined by ts perm ss ons, row fi ters, and members (users) Together, these spec fy wh ch users have access to wh ch data, whether they can read t, process t, do both, or whether they have fu contro Row fi ters can be expressed as per-tab e DAX funct ons

Partitions It’s a so poss b e to part t on your mode Part t ons are created w th n tab es, and each part t on s defined by a subset of that tab e’s data To create part t ons, se ect a tab e by c ck ng on ts tab, then take one of these three act ons c ck the Part t ons button on the Ana ys s Serv ces too bar, choose Tab e Part t ons from the ma n menu, or ed t the Part t ons property n the Propert es w ndow The Part t on Manager d a og box w come up on-screen At the top s a st of part t ons ­In t a y there w be on y one, named the same as ts parent tab e, but you can c ck the New button to ­create ­add t ona part t ons A text box and a drop-down st n the m dd e of the d a og box et you set the name and process ng opt on for the se ected part t on, and you can define the part t on’s data membersh p at the bottom For the ast of these, the Tab e Prev ew mode s d sp ayed by defau t, and a ows you to create fi ter cond t ons on the co umns n the tab e; however, to the upper-r ght of the gr d s a Query Ed tor button that sh fts you to a v ew where you can enter a SQL query nstead 728  Part III  Applied SQL

The dea s that each part t on shou d be defined n such a way that when put together, the ­part t ons return a requ red data from the under y ng tab e If your mode s configured to run n ­D rectQuery mode (wh ch we d scuss next), then keep n m nd on y one part t on n a tab e may be configured for D rectQuery By defau t, the first part t on n the tab e s the D rectQuery part t on, but you can ­transfer that status to any part t on by se ect ng t from the st at the top of the Part t on Manager d a og, and then c ck ng the Set As D rectQuery button (that button w not appear un ess the mode s ­configured for D rectQuery mode) F gure 15-18 shows the Part t on Manager d a og box, w th the Set As ­D rectQuery button, Query Ed tor button, and one fi ter drop-down button h ­ gh ghted

Figure 15-18  The Part t on Manager d a og for a D rectQuery mode .

Moving to DirectQuery D rectQuery s an SSAS Tabu ar feature that causes quer es on your mode to be sent to the SQL Server re at ona database on wh ch t s bu t Enab ng D rectQuery on an SSAS Tabu ar mode configures t to serve as a semant c ayer for report ng and ana ys s, rather than a data store n and of tse f As we a so po nted out n Chapter 14, t’s poss b e to set the DirectQuery Mode property to On ns de the V sua Stud o env ronment Before we can do so, however, we need to ready the mode to work n D rectQuery mode Th s means we must remove a ca cu ated co umns (s nce the r va ues are norma y mater a zed n the mode , they are not compat b e w th D rectQuery), and we must a so remove the Reseller tab e, s nce t was sourced from Exce (D rectQuery works on y for mode s w th a s ng e, SQL Server data source) In add t on to the Reseller tab e tse f, the Exce data source ­po nt ng

Chapter 15  xVe oc ty n Memory Techno og es   729

to Resellers.xlsx must be removed To do th s, de ete the ca cu ated co umns from the Sales tab e, c ck the Ex st ng Connect ons button on the Ana ys s Serv ce too bar (th rd from the eft) and then, n the ­Ex st ng Connect ons d a og box, se ect the Exce c­ onnect on, c ck De ete, and then c ck OK If any s­ ecur ty ro es were created w th row fi ters, those fi ters must be removed (the ro es can stay, ­however) Once the above cr ter a are met, we can sw tch to D rectQuery mode just se ect the Model.bim node n So ut on Exp orer, then set the DirectQuery Mode property to On n the Propert es w ndow If you ook at the status bar you shou d see a message say ng “Sw tch ng the mode to D rectQuery mode…” and then, after a pause, the sett ng w be app ed If you ook carefu y at any of the tab es, you w not ce that the spec a b ank co umn at the far-r ght, wh ch norma y a ows you to create ca cu ated co umns, s gone You can a so ook n the Part t on Manager d a og box and see that the Set As D rectQuery button s v s b e (though d sab ed)

SSAS Deployment At th s po nt, we are a most ready to dep oy our mode to the server Before we do that, however, et’s set a few dep oyment propert es C ck the Project projectname Propert es menu opt on or r ghtc ck the project node n So ut on Exp orer and se ect Propert es from the context menu Th s w br ng up the Property Pages d a og box, w th the Dep oyment propert es sheet se ected ( t’s the on y sheet ­ava ab e, so t’s the on y one that cou d be se ected) It’s p ctured n F gure 15-19, w th a few ­propert es of nterest h gh ghted

Figure 15-19  The Property Pages d a og box.

The Query Mode sett ng w be set to In-Memory by defau t Set t to D rectQuery nstead, as t s shown n F gure 15-19 Here s an exp anat on of that sett ng and the others ■

In-Memory a quer es w

■

D rectQuery a quer es w

730  Part III  Applied SQL

be sent to the Tabu ar mode be sent to the re at ona eng ne/database

■

■

D rectQuery w th In-Memory a quer es w be sent to the re at ona eng ne/database by defau t, but can nstead be sent to the SSAS Tabu ar mode f the connect on str ng n use so spec fies In-Memory w th D rectQuery a quer es w be sent to the SSAS Tabu ar mode by defau t, but can nstead be sent to the re at ona eng ne/database f the connect on str ng n use so ­spec fies

Note  Regardless of the Query Mode setting selected at deployment time, you can change the setting post-deployment in SSMS. Once the Query Mode property s set, there are two other propert es that mer t attent on Make sure the Server sett ng s set correct y and remember to add a backs ash and an nstance name, f necessary, to target an SSAS Tabu ar mode server nstance The Database sett ng, by defau t, s set to be the project name, but you can change t (and thus change the database name) f you’d ke Once you’re done, c ck OK, and then se ect Bu d Dep oy So ut on or Bu d Dep oy projectname from the ma n menu or r ght-c ck the project node n So ut on Exp orer and se ect Dep oy from the context menu The Dep oy d a og w then pop up Th s s the same d a og box PowerP vot d sp ays when data s mported nto the mode However, when you dep oy a D rectQuery mode , on y the metadata s dep oyed and a message to that effect w be d sp ayed when dep oyment s comp ete In e ther case, c ck the C ose button once dep oyment has conc uded

Making the Connection Our next, and a most ast, step s to query th s mode w th Power V ew Un ke w th PowerP vot workbooks, where a Power V ew button s d sp ayed next to the workbook’s thumbna mages n the PowerP vot Ga ery, Power V ew can’t query an Ana ys s Serv ces Tabu ar mode un ess we create a BI Semant c Mode Connect on fi e for t To do th s, po nt your browser to the PowerP vot Ga ery, then c ck on the Documents tab (under L brary Too s) n the r bbon C ck on the drop-down arrow on the r bbon’s New Document button (at the far eft) and se ect BI Semant c Mode Connect on from the drop-down menu, as shown n F gure 15-20

Figure 15-20  Creat ng a B Semant c Mode Connect on fi e n the SharePo nt PowerP vot Ga ery.



Chapter 15  xVe oc ty n Memory Techno og es   731

Note  In order to create BI Semantic Model Connection files, the content type of the same name must be enabled in the Library Settings of your PowerPivot Gallery. If the content type is not enabled, the option for such a file will not appear when you click the New Document drop-down button. The New BI Semant c Mode Connect on page w come up n the browser Enter a name for the fi e ( n the F e Name text box), the name of your server, nc ud ng a backs ash and nstance name f necessary ( n the Workbook URL or Server Name text box), and the name of your database ( n the Database (If Connect ng To A Server) text box) S nce there’s no drop-down st of databases, ­doub e-check to make sure you’ve typed n the database name correct y; better yet, copy the name from the Property Pages d a og box n V sua Stud o and paste t n C ck OK when you’re done

Power View Here We Come Your connect on fi e shou d be created and your browser shou d automat ca y return to the ­PowerP vot Ga ery Now ook for the connect on fi e, scro ng down through the a phabet ca y ­ordered st of documents, as necessary When you get there, you shou d see t has ts own Create Power V ew Report button toward the far-r ght C ck t to enter Power V ew A b ank Power V ew report shou d be d sp ayed w th the tab es from your mode (and the r co umns, measures, and KPIs f you expand the tab e nodes) d sp ayed on the r ght, as shown n F gure 15-21

Figure 15-21  A b ank Power V ew report w th our mode s tab es, co umns, measures, and KP d sp ayed.

732  Part III  Applied SQL

Building the Report You can bu d a n ce ana ys s of the D rectQuery mode n Power V ew by comb n ng a few ­v sua zat ons n one page If you’d ke, you can fo ow the steps be ow to create such a report The resu t ng report shou d ook ke the one shown n F gure 15-22 1st visualization: 1. Check Sa es\Sa es Amount 2. Check Sa es\Un t Pr ce 3. Change v sua zat on type to L ne 4. Add Promot on\Promot on to Ax s sect on 5. S ze and pos t on to upper ha f, eft 2/3 of page

2nd visualization: 1. C ck on a b ank area of the report 2. Check Sa es\Average Sa es Amount 3. Change v sua zat on type to Scatter 4. S ze and pos t on to ent re bottom ha f of page 5. Drag Sa es\Sa es Amount nto the Y Va ue sect on 6. Drag Sa es\Un t Pr ce nto the S ze sect on 7. Drag Sa es Terr tory\Reg on nto Deta s

3rd visualization: 1. C ck on a b ank area of the report 2. Check Date\Year 3. S ze and pos t on to upper r ght ( e , rema n ng area) of page 4. C ck the S cer button n the r bbon

When you’re done, c ck on d fferent years n the s cer to see the ne and scatter charts an mate and update



Chapter 15  xVe oc ty n Memory Techno og es   733

Figure 15-22  A comp eted Power V ew report.

Welcome Back to VertiPaq To br ng th s conversat on fu c rc e, et’s create a co umnstore ndex on the FactResellerSales tab e n AdventureWorksDW2012 (wh ch corresponds to the Sales tab e n our mode ) Here are the necessary steps ■

Connect to your SQL Server re at ona server n SSMS

■

Dr

down on the Datab`ases node

■

Dr

down on the ch d AdventureWorksDW2012 node

■

Dr

down on the ch d Tables node

■

Dr

down on the ch d dbo.FactResellerSales node

■

■

R ght-c ck the ch d Indexes node and se ect New Index Non-C ustered Co umnstore Index from the context menu In the New Index d a og, c ck the Add button

The Se ect Co umns d a og box w then come up C ck the se ect-a checkbox at the far eft of the gray-backgrounded co umn headers bar, and then c ck OK The Se ect Co umns d a og box s shown n F gure 15-23, w th the se ect-a checkbox and OK button h gh ghted

734  Part III  Applied SQL

Figure 15-23  The Se ect Co umns d a og box n SSMS, w th a co umns se ected for a co umnstore ndex.

C ck OK aga n, th s t me n the New Index d a og box Now go back to Power V ew and try some more quer es You may or may not not ce a d fference n speed S nce the FactResellerSales tab e has on y 60,855 rows, t s, frank y, hard to rea ze a benefit from the co umnstore ndex But w th a arge product on data warehouse, you shou d be ab e to ga n rea benefit from a co umnstore ndex, and now you know how to create one You have now seen how the var ous xVe oc ty n-memory techno og es ( e the BISM mode n PowerP vot and SSAS Tabu ar mode and co umnstore ndexes n the SQL Server re at ona eng ne) can work together to a ow fast ana yt ca quer es aga nst a semant c mode ayer that then forwards the quer es to the up-to-date re at ona database, a ow ng for rea -t me BI w th exce ent performance

Summary S nce the first ed t on of th s book, we have argued that BI shou d not be cons dered a n che area, separate from re at ona database work We be eved then, as now, that re at ona techno ogy and BI ex sted a ong a cont nuum, and that t s mportant to know both We’ve proven that po nt n th s chapter by start ng w th the re at ona capab ty of co umnstore ndexes then mov ng to PowerP vot and the Tabu ar mode of Ana ys s Serv ces, wh ch then gave us the capab ty to make the round-tr p back to a re at ona database us ng D rectQuery We then went to Power V ew, a BI too f ever there was one, and used t to report from a BI Semant c Mode act ng as a th n ayer around our re at ona data The pièce de rèsistance was add ng a co umnstore ndex to that re at ona database to a d n the ­performance of the report’s nteract ve capab t es Whether you’re us ng a re at ona database, a mu t d mens ona OLAP cube, or a BI Semant c ­ ode , data s data The quest on s whether you want to m t yourse f to the operat ona co ect on M and ma ntenance of that data, or f you want to branch out and fac tate bus ness users’ ana ys s of that data, to understand the r bus ness better We th nk you shou d branch out n that manner

Chapter 15  xVe oc ty n Memory Techno og es   735

In the prev ous two ed t ons of th s book we had severa BI chapters, nc ud ng three focused most y on bu d ng OLAP cubes In th s ed t on of the book, we needed on y two BI chapters And n th s chapter a one, we’ve shown you how to bu d and report on BI Semant c Mode s that can serve as data sources n the r own r ght or as ana yt ca abstract ons over the ve data n re at ona databases We’ve shown the versat ty of SQL Server, n br ng ng you an ent re data p atform, and not just a re at ona database And n SQL Server 2012, even the BI components of the p atform become ­re at ona database-fr end y We th nk th s chapter has conveyed the va ue of th s d vers ty and ­usab ty, and we th nk that’s a very fitt ng p ace to end th s book

736  Part III  Applied SQL

Index

Symbols $fi ter opt on (WFC Data Serv ces),  518 .DACPAC fi es contents of,  600 .NET Framework d str buted transact ons and,  190 198 evo ut on of,  427 430 Language ntegrated Query (L NQ),  428 System.Enterpr seServ ces namespace,  190 System.Transact ons namespace,  186, 191 .NET funct ons Sq Tr gger attr bute,  149 @SortOrder parameter metadata d scovery, use n,  120 :: syntax,  303 @@TRANCOUNT funct on,  174 176 @ (XPath),  277

A AC D propert es,  170 172 atom c ty, 171 172 cons stency, 171 172 durab ty, 171 172 so at on, 171 172 Act veX Data Objects (ADO),  428 Add F rewa Ru e d a og (W ndows Azure Management Porta ), 645 Add New Doma n Serv ce C ass d a og,  552 556 AddRe atedObject method (DataServ ceContext),  526 AddW thVa ue method (Sq Command object),  441 Adobe PDF fi es Fu Text Search (FTS) and,  365 ADO.NET pass ng TVPs us ng,  52 54



ADO.NET, convent ona ,  436 471 DataSets, 455 473 ORM vs.,  479 raw data objects, us ng,  436 455 ADO.NET data access SQL CLR stored procedures vs.,  132 ADO.NET Ent ty Framework. See Ent ty Framework (EF) ADO.NET transact ons so at on eve s n,  184 186 sett ng so at on eve n exp c t,  185 sett ng so at on eve s n mp c t,  185 186 Sq Transact on object,  185 Transact onScope object (ADO.NET),  185 Advanced Mode (PowerP vot),  723 724 Defau t F e d Set button,  723 Perspect ves button,  723 Show mp c t Members button,  723 Summar ze By button,  723 Tab e Behav or button,  723 AdventureWorks2012 samp e database,  130 aggregates (SQL CLR),  151 155 creat ng, 152 153 Merge method,  155 mu t p e nput parameters and,  155 requ red methods n,  152 Sq UserDefinedAggregate attr bute,  151 152 warn ngs and respons b t es w th,  155 aggregat ons runn ng, 95 96 s d ng, 96 97 A ert ng feature (SSRS) Sharepo nt, requ rement for,  693 ALTER DATABASE AUD T SPEC F CAT ON statement (SQL Server Aud t),  240 ALTER DATABASE command ED T ON parameter,  599 MAXS ZE parameter,  599

737

ALTER PROCEDURE statement ALTER PROCEDURE statement,  220 ALTER SEQUENCE statement,  117 ALTER SERVER AUD T SPEC F CAT ON statement (SQL Server Aud t),  239 ALTER statement CREATE statement vs.,  5 ana yt c funct ons,  98 103 CUME D ST funct on,  98, 100 102 F RST VALUE funct on,  98 99 LAG funct on,  98 100 LAST VALUE funct on,  98 99 LEAD funct on,  98 100 PERCENT LE CONT funct on,  98, 100, 102 PERCENT LE D SC funct on,  98, 100, 102 PERCENT RANK funct on,  98, 100 101 anonymous methods (DataServ ceCo ect on c ass), 664 App c ass (W ndows Phone),  661 V ewMode property,  651, 661 App cat on c ass (W ndows Phone),  661 AsDataV ew method (L NQ to DataSet),  476 AsEnumerab e method (System.Data namespace),  475 query ng typed DataSets w th,  477 ASP.NET, 518 V sua Stud o Deve opment Web Server (Cass n ), 513 WCF serv ces vs.,  348 ASP.NET Deve opment Web Server,  516 assemb es AUTHOR ZAT ON c ause,  139 backup/restore and,  140 CREATE ASSEMBLY command,  139 dependent on other assemb es,  139 dep oy ng, 138 140 dep oy ng w th SSMS,  140 FROM c ause,  139 PERM SS ON SET va ue,  139 W TH PERM SS ON SET c ause,  139 assemb es (SQL CLR) secur ty sett ngs n V sua Stud o,  161 TRUSTWORTHY property,  161 UNSAFE ASSEMBLY perm ss on,  161 asymmetr c key encrypt on,  222 Asynchronous JavaScr pt and XML (AJAX) M crosoft B ng and,  416 AsyncState property (DataServ ceQuery c ass), 665 atom c ty (AC D property),  171 172 AtomPub v ew ng response feed from,  516 517

738  Index

Atom Pub sh ng Protoco (APP) as XSD schema,  260 Atom Pub sh ng Protoco (AtomPub),  509 AttachAsMod fied method (ChangeSet),  560 AttachAsMod fied method (Doma nServ ces c ass), 560 561 Attr bute Groups (MDS),  677 Attr butes (MDS),  677 aud t act ons, database eve ,  240 aud ted events, v ew ng,  242 AUD T GU D opt on (SQL Server Aud t),  237 aud t objects,  235 244 ALTER SERVER AUD T statement,  235 236 AUD T GU D opt on,  237 CREATE SERVER AUD T statement,  235 F LEPATH opt on,  238 MAX F LES opt on,  238 MAX ROLLOVER F LES opt on,  238 MAXS ZE opt on,  238 ON FA LURE opt on,  237 po nt ng to dest nat on,  235 QUEUE DELAY opt on,  236 record ng aud ts to fi e system,  238 239 RESERVE D SK SPACE opt on,  238 246 STATE opt on,  237 238 TO F LE c ause,  238 239 authent cat on W ndows Azure Management Porta and,  625 authent cat on/author zat on, 213 221 estab sh ng a connect on,  213 214 execut on context,  218 221 password po c es,  215 216 user schema seperat on,  216 218 AUTHOR ZAT ON c ause (C ass L brary project),  139 autocomm t transact on mode,  173 auto dep oyment (SQL CLR),  129 Sq Procedure attr bute and,  141 AVERAGEX funct on (DAX),  717 Azure Storage Exp orer up oad ng BACPACs w th,  604

B BACKUP CERT F CATE statement,  233 backups encrypt on of, w th TDE,  231 backwards compat b ty CurveToL neW thTo erance method,  409 410 M nDbCompat b tyLeve method and,  408 STCurveToL ne method and,  409 410

Class Library project

BACPAC fi es contents of,  600 batch execut on env ronment,  176 179 Batch execut on mode,  706 707 forc ng, w th UPDATE STAT ST CS command,  708 MDOP configurat on opt on,  707 processor a ocat on for,  707 v rtua mach nes and,  707 batch scoped transact on mode,  176 179 BEG N D STR BUTED TRANSACT ON statement,  189 190 BEG N D STR BUTED TRANSACT ON statement (T SQL) System.Transact ons namespace vs.,  194 Beg nExecute method (DataServ ceQuery c ass),  665 Beg nTransact on method (ADO.NET) sett ng so at on eve w th,  185 Beg nTransact on method (Sq Connect on object),  450 BEG N TRANSACT ON statement,  173 176 mut p e databases and,  186 B D rect ona sync d rect on (Data Sync),  620, 621 BLOB data (B nary Large Object),  323 325 backup concerns w th,  325 n the database,  324 n the fi e system,  324 325 remote BLOB storage (RBS),  356 varb nary(max) data type,  324 breadth first ndex ng,  314 315 breakpo nts sett ng, 33 BufferW thCurves method (geometry c ass),  406 408 STBuffer method vs.,  406 408 bu kadm n (fixed server ro ),  211 bus ness ent t es represented as objects,  478 bus ness nte gence capab t es (B ),  675 700 ana ys s sercv ces,  686 691 Ana ys s Serv ces,  686 687 data m n ng,  690 691 Data Qua ty Serv ces (DQS),  680 681 Exce Serv ces,  694 695 ntegrat on Serv ces,  681 683 Master Data Serv ces (MDS),  677 680 M crosoft B Stack,  676 PerformancePo nt Serv ces (PPS),  696 Power V ew report,  691 692 Re at ona Database Management System (RDBMS), 683 685 Report ng Serv ces,  692 694 SQL Server/SharePo nt vers on requ rements for, 697 698 Stream ns ght, 697

Bus ness nte gence Semant c Mode (B SM),  702 bu d ng, 712 715 data source compat b ty,  711 712 h erarch es and,  720 722 SSAS and,  711, 712 star schemas and,  711 byte[] Recovery nformat on method (System. Transact ons.Prepar ngEn stment c ass),  198

C C# M crosoft V sua Bas c .NET vs.,  141 cached fi e s ze (F eTab e co umn name),  358 ca cu ated co umns (PowerP vot),  716 717 DAX formu as and,  716 CASE construct,  92 93 CATCH b ock and THROW statements,  110 111 Ce Va ueChanged event,  538 cert ficates,  223 back ng up for TDE,  232 233 DEK and,  230 cert ficates, secur ty,  222 cert ficates, se f s gned,  222 change nterceptor,  548 ChangeObjectState method (DataServ ce c ass),  560 ChangeOperat on enumerat on va ue (DataServ ce c ass), 560 change scr pts mp ementat on of, n SSDT,  20 ChangeSet property (Doma nServ ces c ass),  560 CHANGES keyword consum ng us ng NSERT OVER DML syntax,  80 83 mp emenat on of,  81 82 Chaos (ADO.NET so at on eve ),  184 Char ndex method,  256 CHOOSE funct on,  106 “Choos ng an Encrypt on A gor thm” (TechNet),  225 C RCULARSTR NG keyword,  370 371 C RCULARSTR NG object,  401 402 C ass L brary project,  139 140 AUTHOR ZAT ON c ause,  139 CREATE ASSEMBLY command,  139 FROM c ause,  139 PERM SS ON SET va ue,  139 stored procedures, dep oy ng n,  141 T SQL CREATE FUNCT ON statement,  144 W TH EXECUTE AS CALLER c ause,  144 W TH PERM SS ON SET c ause,  139 Index  739

Class Library projects C ass L brary projects UDFs, dependenc es for,  143 C ent W ns (Data Sync confl ct reso ut on),  624 C ose method (Sq Connect on c ass),  440 C oud project temp ate (V sua Stud o) Enab e W ndows Azure Too s opt on,  642 c oud serv ces nfrastructure as a serv ce ( aaS),  580 T department, a ternat ve to,  579 on prem ses software vs.,  580 C oudSync Comp eted method,  665 CLR assemb y secur ty eve s,  161 CLR ent t es exam n ng/manag ng n SQL Server Object Exp orer, 162 168 exam n ng/manag ng n SSMS Object Exp orer, 162 168 remov ng, 166 CLR funct ons Sq Type as return va ue for,  143 T SQL funct ons and,  145 147 CLR stored procedures,  130 132 ADO.NET data access vs.,  132 AdventureWorks2012 samp e database,  130 dep oy ng, 141 142 ExecuteReader method (Sq Command object),  133 gu de nes for,  136 mak ng ava ab e,  131 M crosoft.Sq Server.Server namespace,  133 .NET CLR and,  132 open ng connect ons to databases from,  133 p p ng data,  134 136 server s de data access,  132 136 Sq DataRecord type,  134 136 Sq MetaData type,  134 136 test ng, 142 168 T SQL tr ggers vs.,  148 c ustered ndex SQL Azure requ rement for,  40 Code first des gn (EDM),  484 COLLATE keyword xm (data type) and,  258, 259 co at ons JO Ns between ncompatab e,  247 co ect ons Enumerab e nterface, use w th,  54 57 pass ng, us ng TVPs,  54 57 co umns renam ng, 31 32

740  Index

co umn store databases,  702, 702 703 aggregat on quer es and,  703 ana yt ca quer es, performance of,  703 B ndustry and,  703 co umnstore ndexes,  704 709 co umnstore ndexes,  704 709 Batch execut on mode,  706 707 bu d ng, 704 data stores, as,  704 F estream data and,  704 m tat ons on,  704 705 MDOP configurat on opt on,  707 page structure of,  706 prec s on of dec ma /numer c co umns,  705 query processor (QP),  706 read on y m tat on, work arounds for,  705 restr cted data types n,  705 s ze m t on,  705 work ng w th,  706 709 COM+, 190 CommandText property (Sq Command c ass),  447 comment (XPath node funct on),  279 Comm t method ( En stmentNot ficat on nterface),  195 COMM T TRANSACT ON statement,  173 176 common tab e express ons (CTEs),  299 Common Tab e Express ons (CTEs),  46 compar son operat ons order of compar son n he rarch ca tab es,  314 Comp ete method (Transact onScope object),  193, 198 199 comp ex event process ng (CEP) eng ne,  697 Component Object Mode (COM),  132 Component Object Mode (COM) p atform,  428 COMPOUNDCURVE keyword (WKT),  370 371 COMPOUNDCURVE object,  403 404 compound curves curve po ygons vs.,  405 storage overhead of, geometry co ect on vs.,,  404 Compute capac ty,  641 Compute resources,  641 COMT (th rd party TCs),  190 CONCAT funct on,  107 conceptua schema (EDM),  482 confl ct reso ut on po cy sett ng, 636 connected data reader. See DataReaders Connect on Managers (SS S packages),  682 connect ons (to SQL Server),  213 221 Connect onStr ng property (Sq Connect on object), 439

cons stency (AC D property),  171 172 conta nment break ng, 245 Contro F ows (SS S packages),  682 convers on funct ons,  103 104 data va dat on us ng,  104 PARSE funct on,  103 104 TRY CONVERT funct on,  103 TRY PARSE funct on,  103 104 CopyToDataTab e method (L NQ to DataSet),  476 CREATE AGGREGATE statement (T SQL),  155 CREATE ASSEMBLY command (C ass L brary project), 139 CREATE CERT F CATE statement,  233 CREATE DATABASE AUD T SPEC F CAT ON statement (SQL Server Aud t),  240 Create Database d a og (Create Server w zard),  589 CREATE DATABASE statement F LEGROUP...CONTA NS F LESTREAM c ause,  329 331 CREATE FUNCT ON statement (T SQL),  144 CREATE SEQUENCE statement,  116 CREATE SERVER AUD T SPEC F CAT ON statement (SQL Server Aud t),  239 Create Server w zard (SQL Azure),  587 588 adm n strat ve user, creat ng,  587 588 Create Database d a og,  589 firewa , configur ng,  587 588 CREATE statement ALTER statement vs.,  5 T SQL object representat on,  5 CreateYourOwnRM so ut on,  195 creat on t me (F eTab e co umn name),  357 Cr stofor, Laurent u,  228 CRUD stored procedures, creat ng,  433 435 cubes (SSAS),  686 687 CUME D ST funct on,  98, 100 102 RANK funct on as bas s for,  101 CURVEPOLYGON keyword (WKT),  370 371 CURVEPOLYGON object,  404 405 curve po ygons compound curves vs.,  405 CurveToL neW thTo erance method (geometry c ass), 409 410

D DACPACs dep oy ng w th SSMS,  600

data definition language (DDL) triggers DACs BACPAC fi es, contents of,  600 .DACPAC fi es, contents of,  600 SQL Azure Management Porta , manag ng w th, 602 SSDT, manag ng w th,  601 data access,  427 508 abstract on s cost to performance,  437 Act veX Data Objects (ADO),  428 ADO.NET raw data,  436 471 creat ng forms for data entry,  438 440 Data access objects (DAO),  428 DataAdapters, 428 429 DataSets (ADO.NET),  455 473 ent ty framework,  482 508 anguage ntegrated query (L NQ),  472 477 .NET, evo ut on of,  427 430 Object Re at ona Mode ng (ORM),  477 507 Open Database Connect v ty (ODBC) AP ,  428 Remote Data Objects (RDO),  428 server s de, w th SQL CLR stored procedures, 132 136 SQL Server Profi er, mon tor ng act v ty w th, 435 436 Data access objects (DAO),  428 DataAdapters,  428 429, 456 457 DataContext vs.,  479 Tab eAdapters vs.,  465 Data Ana ys s eXpress ons (DAX),  702 database snapshots and F LESTREAM,  356 database encrypt on key (DEK),  230 database master key (DMK),  224 encrypted databases, restor ng w th,  233 SMK and,  225 TDE and,  229 database users,  211 212 fixed ro s of,  212 SSMS, creat ng n,  211 212 T SQL, creat ng n,  211 212 data b nd ng,  531 B nd ngSource object,  536 context var ab e,  536 DataBound tem property,  536 Not fyPropertyChanged nterface,  660 remov ng objects from,  537 DataBound tem property,  536 data centers (SQL Azure),  586 587 DataContext object (L NQ to SQL),  479 data defin t on anguage (DDL) tr ggers,  148 Index  741

data encryption. See encryption support data encrypt on. See encrypt on support Data F ows (SS S packages),  682 DataGr d contro (S ver ght) V sua Stud o, add ng to project n,  561 data mart,  683 data m n ng SSAS eng ne for,  690 691 data, p p ng Sq DataRecord and Sq MetaData, w th,  134 136 Data Qua ty Projects, types of,  681 DataReaders c os ng, 445 creat on and use of,  443 DataSets vs., performance,  457 end of stream cond t ons, test ng for,  446 447 terat ng, 443 447 DataServ ceCo ect on c ass anonymous methods,  664 LoadAsync method,  664 665 LoadComp eted event,  664 665 DataServ ceContext (WCF Data Serv ces),  523 AddRe atedObject method,  526 SaveChanges method,  526 527 UpdateObject method,  526 DataServ ceQuery c ass,  665 AsyncState property,  665 Beg nExecute method,  665 EndExecute method,  665 S ng e method,  665 DataServ ce c ass (EDM),  515 Add New Doma n Serv ce C ass d a og,  552 556 AttachAsMod fied method (ChangeSet),  560 561 ChangeObjectState method,  560 ChangeOperat on enumerat on va ue,  560 ChangeSet property,  560 Enab eC entAccess attr bute,  559 GetAssoc atedChanges method,  560 GetChangeOperat on method,  560 GetOr g na method,  560 Hand eExcept on method, overr d ng,  546 OnStartProcess ngRequest method, overr d ng, 546 POCOs, expos ng to WCF R A Serv ces,  559 Subm t method,  560 Transact onScope object,  556 DataSets, 455 473 appropr ate usage of,  456 DataReaders vs., performance,  457 DataTab e objects,  456

742  Index

Ent ty Framework vs.,  456 gener c vs. strong y typed,  458 L NQ to DataSet,  473 477 object or ented program ng and,  477 478 System.Data.DataSetExtens ons.d , 475 DataSets, gener c fi ng/updat ng,  459 462 mp ement ng, 461 462 query ng w th L NQ,  474 476 typed vs.,  458 use of,  458 datasets (SQL Azure Data Sync) defin ng,  636 638 fi ters on,  636 DataSets, strong y typed. See DataSets, typed DataSets, typed benefits of us ng,  470 471 bu d ng, 462 465 bu d ng n V sua Stud o,  462 gener c vs.,  458 mapp ng stored procedures to,  465 471 query ng w th L NQ,  476 477 Tab eAdapter Configurat on W zard,  465 468 V sua Stud o, creat ng projects for,  463 XML Schema Defin t on (XSD) and,  463 data stores co umnstore ndexes and,  704 Data Sync. See SQL Azure Data Sync (Data Sync) data center ocat on and atency ssues,  625 fa ures, poss b e causes of,  640 641 SQL Azure fees and,  626 SQL Server/SQL Azure, ack of sync support for,  626 Data Sync button (W ndows Azure Management Porta ), 631 DataTab e objects (DataSets),  456 Data T er App cat ons (DACs),  599 606 DACPACs and,  600 SSMS, manag ng w th,  600 601 W ndows Azure Management Porta,  601 data/t me funct ons DATEFROMPARTS funct on,  105 DATET ME2FROMPARTS funct on,  105 DATET MEFROMPARTS funct on,  105 DATET MEOFFSETFROMPARTS funct on,  105 T MEFROMPARTS funct on,  105 Data Too s Operat ons w ndow propert es of,  41 Data V ew mode (PowerP vot Exce add n),  688 DataV ew object RowF ter property,  473



DistributedIdentifier property (System.Transactions namespace)

data warehouse(s) co umn store databases,  702 703 data warehouses,  683 Fast Track Data Warehouse,  684 SQL PDW,  684 685 data (XPath node funct on),  279 DATEADD funct on,  62 date data type,  58 DATED FF funct on,  62 DATEFROMPARTS funct on,  94, 105 DATENAME funct on,  62 DATEPART funct on,  62 datet me2 data type,  58 accuracy of,  58 extract ng date/t me us ng CAST/CONVERT,  61 62 range of va ues for,  58 DATET ME2FROMPARTS funct ons,  105 datet me data type,  58 date/t me data types,  58 65 accuracy of,  60 62 date, 58 65 datet me, 58 datet me2, 58 datet meoffset, 58 59 format of,  60 61 funct ons of,  62 65 portab ty of,  58 59 seperat on of,  58 sma datet me, 58 storage space, usage of,  60 t me, 58 t me zone awareness w th,  59 60 date/t me formatt ng codes,  108 DATET MEFROMPARTS funct on,  105 date/t me funct ons DATEFROMPARTS funct on,  94 EOMONTH funct on,  105 106 new funct ona ty,  104 106 SMALLDATET MEFROMPARTS funct on,  105 datet meoffset data type,  58, 59 60 DATET MEOFFSETFROMPARTS funct ons,  105 DAX formu as and ca cu ated co umns,  716 db accessadm n (fixed database ro ),  212 db backupoperator (fixed database ro ),  212 dbcreator (fixed server ro ),  211 db datareader (fixed database ro ),  212 db datawr ter (fixed database ro ),  212 db dd adm n (fixed database ro ),  212



db denydatareader (fixed database ro ),  212 db denydatawr ter (fixed database ro ),  212 db owner (fixed database ro ),  212 db secur tyadm n (fixed database ro ),  212 DDL changes, aud t ng,  241 DDL tr ggers,  150 151 debugger sett ng breakpo nts n,  521 DECRYPT ON BY PASSWORD c ause (BACKUP CERT F CATE statement),  233 DEFAULT constra nt on SEQUENCE objects,  117 deferred execut on (L NQ to Ent t es),  489 DEK restor ng encrypted databases and,  233 De eteCommand property (Sq DataAdapter c ass), 457 458 Den a of Serv ce (DoS) attack UDP as vu nerab t y to,  213 DENSE RANK funct on,  93 dep oyment (SQL CLR),  136 143 assemb y, of,  138 140 preperat on for,  137 138 SQL Server Database Projects,  137 138 stored procedures,  141 142 test ng stored procedures,  142 168 depth first ndex ng,  314 Der ved H erarch es (MDS),  677 D agram V ew mode(PowerP vot Exce add n),  688 D rect on property (Parameter object),  448 D rectQuery feature (SSAS Tabu ar mode),  729 730 D rectQuery mode m tat ons on usage of,  729 D rectQuery Mode property,  729 d rect SQL stored procedures vs.,  493 D rect SQL stored procedures vs.,  442 443 d rty data DQS, c ean ng up w th,  680 d rty reads (transact on),  178, 179 prevent ng w th read uncomm tted so at on eve , 181 ser a zab e so at on eve and,  182 d skadm n (fixed server ro ),  211 D spose method ca ng on Transact onScope nstances,  193 D spose method ( D sposab e nterface),  439 D str buted dent fier property (System.Transact ons namespace), 194

Index  743

distributed transaction coordinator (DTC) d str buted transact on coord nator (DTC),  187 M crosoft D str buted Transact on Coord nator (MS DTC),  187 d str buted transact ons,  186 200 BEG N D STR BUTED TRANSACT ON statement, 189 190 COM+, 190 COMT , 190 en stment, ru es/methods of,  187 189 M crosoft Transact on Server (MTS),  190 .NET Framework and,  190 198 resource manager,  186 resource managers, usage,  198 200 SQL Server and,  189 190 System.Enterpr seServ ces namespace,  190 term no ogy, 186 187 transact on manager/coord nator,  186 187 two phase comm t,  187 DML act on, aud t ng,  241 DML quer es autocomm t transact on mode and,  173 DML tr ggers (SQL CLR),  148 150 dep oy ng automat ca y,  149 Sq Tr gger attr bute (.NET funct ons),  149 Doma n Serv ce C ass temp ate (V sua Stud o),  552 DQS MDS and,  681 SS S and,  681 dr ft detect on SQL Server Object Exp orer and,  7 DROP SEQUENCE statement,  118 durab ty (AC D property),  171 172 durab e en stment,  188

E eager oad ng quer es,  498 EARL ER funct on (DAX),  717 ED T ON parameter (ALTER DATABASE command), 599 EDM des gner/des gn surface (V sua Stud o),  482 488 C# code, v ew ng background,  488 XML, v ew ng background,  487 ELEMENTS keyword (FOR XML syntax),  282 285 e pso da sphere spat a mode . See geodet c spat a mode Enab eC entAccess attr bute (WCF R A),  559 Encrypted F e System (EFS),  229 ENCRYPT ON BY PASSWORD c ause (CREATE CERT F CATE statement),  233

744  Index

encrypt on serv ces restor ng encrypted databases,  233 234 encrypt on support,  222 234 a gor thm ava ab ty for,  225 asymmetr c key encrypt on,  222 b ogs on,  228 bu d ng b ocks for,  224 cert ficates,  222, 223 cert ficates, back ng up,  232 233 data at rest, encrypt ng,  224 228 database master key (DMK),  224 data on the move, encrypt ng,  223 224 encrypted databases, obta n ng nformat on on, 231 Encrypted F e System (EFS),  229 encrypt on keys,  222 encrypt on state method,  232 Force Protoco Encrypt on opt on (SQL Nat ve C ent), 224 ndex ng encrypted data,  228 performance test ng and,  228 se f s gned cert ficate,  222 se f s gned cert ficates,  223 Serv ce Master Key (SMK),  224 SET ENCRYPT ON ON c ause,  231 SQL Server Configurat on Manager too ,  223 SQL Server Nat ve Access C ent AP ,  223 symmetr c key encrypt on,  222 transparent data encrypt on,  229 234 EndExecute method (DataServ ceQuery c ass),  665 endpo nts, 213 en stment, 187 189 durab e, 188 portab e s ng e phase,  188 189 vo at e, 188 En stment var ab e (System.Transact ons namespace), 197 ent t es og ns as,  210 211 updat ng, 495 users/ro es as, n SQL Server,  209 210 Ent t es co ect on State property,  537 Ent t es (MDS),  677 Attr butes, 677 Members, 677 Ent tyC ent property va ue (Textbox),  505 Ent tyC ent (System.Data.Ent ty assemb y),  505 507 Ent tyCommand c ass (Ent tyC ent),  506 Ent tyConnect on c ass (Ent tyC ass),  506

Ent ty Data Mode (EDM),  482 508 add ng to W ndows Azure project,  644 647 best pract ces for,  552 bu d ng, 482 488 conceptua schema,  482 creat ng n V sua Stud o,  483 488 DataServ ce c ass,  515 EDM des gn surface,  482 Ent tyC ent, query ng w th,  505 507 Ent ty Data Mode W zard,  483 Ent ty SQL as nat ve anguage for,  503 expos ng to REST c ents,  514 n t a zeServ ce method, configur ng w th,  515 L NQ to SQL vs.,  480 many to many re at onsh ps,  483 Mapp ng Deta s pane,  500, 501 mapp ng schema,  482 mapp ng stored procedures to,  490 494 MetadataType attr bute,  557 Mode Browser,  486 487 ObjectContext property,  559 one to one correspondence and,  483 storage schema,  482 WCF Data Serv ces, creat ng for,  513 515 Ent ty Data Mode W zard (V sua Stud o),  483 Code first des gn,  484 mode first deve opement,  484 Ent tyDataReader (Ent tyC ent),  506, 507 Ent ty Framework TVP support, ack of,  57 Ent ty Framework (EF),  482 508 Ce Va ueChanged event,  538 c ent s de WCF Data Serv ces app cat ons and,  526 context of ent t es,  537 DataSets vs.,  456 eager oad ng quer es,  498 ensur ng reasonab e quer es from,  490 Ent t es co ect on,  537 Ent tyC ent, 505 507 Ent tyState property,  498, 499 fi and update act ons w th,  499 500 mpedance m smatch, reso v ng,  500 501 L nqToEnt t esDoma nServ ce, 559 L NQ to Ent t es, us ng,  488 490 L nqToSq Doma nServ ce, 559 L NQ to SQL, m grat ng from,  481 L NQ to SQL vs.,  478, 480 482 many to many re at onsh ps,  501 503 n T er and,  499 500 runt me behav or of,  485 486

Extract, Transform, and Load (ETL) tool sav ng ent ty changes,  495 498 WCF Data Serv ces and,  500 WCF Data Serv ces w thout,  511 WCF R A Serv ces and,  500 Ent ty SQL,  503 505 Ent ty Framework, as nat ve anguage for,  488 490, 503 505 anguages capab e of commun cat ng w th,  481 L NQ to SQL vs.,  480 T SQL, resemb ance to,  504 505 Ent tyState property (EF),  498, 499 EOMONTH funct on,  105 106 error hand ng FORMATMESSAGE funct on,  112 THROW statements, us ng,  109 113 EventData property (Sq Tr ggerContext object),  150 Event parameter (CLR tr ggers) FOR UPDATE, NSERT,  149 events hook ng up to hand ers,  535 536 Exce PowerP vot and,  717 query ng n,  724 726 SQL Server databases and,  694 ExecuteAndSend method (Sq P pe object),  133 ExecuteSca ar method (Sq Command c ass),  447 450 execut on context,  218 221 ALTER PROCEDURE statement,  220 W TH EXECUTE AS c auses,  220 expand opt on (WFC Data Serv ces),  518 Exp c t H erarch es (MDS),  677 exp c t transact on mode,  173 176 BEG N TRANSACT ON statement,  173 176 COMM T TRANSACT ON statement,  173 176 nam ng transact ons,  173 nested transact ons,  175 ROLLBACK TRANSACT ON T SQL statement,  174 savepo nts, 175 176 SAVE TRANSACT ON statement,  175 177 @@TRANCOUNT funct on,  174 176 exp c t transact ons data access and,  450 453 Export feature (W ndows Azure Management Porta ), 606 ExportSq Ce ut ty (SQL Server CE),  666 extended stored procedures (XPs),  132 eXtens b e Markup Language (XML),  509. See XML Externa Access (CLR assemb y secur ty eve ),  161 Extract, Transform, and Load (ETL) too SS S as,  681 683 Index  745

Fast Track Data Warehouse

F Fast Track Data Warehouse,  684 Federat on SQL Server Object Exp orer too s for,  611 federat ons D str but on, 609 Federated Tab e,  609 Federat on Key co umn, restr ct ons on,  609 Federat on Member,  609 Federat ons SSMS Object Exp orer as too for,  611 Federat ons (SQL Azure),  607 612 Atom c Un t,  607 centra /reference tab es,  610 and the c oud,  612 creat ng, 608 D str but on, 607 fan out quer es,  611 Federat on Key,  607 Federat on Members,  607 Federat on Members, sp tt ng and dropp ng,  610 Federat on Memebers, us ng,  610 Federat on Root,  607 ex con for,  607 mu t tenancy, 611 support for, n SSMS and SSDT,  611 612 tab es, 609 F dd er add ng support for WCF R A Serv ces nspect ng, 569 mon tor ng network act v ty w th,  510 WCF Data Serv ces, watch ng w th,  523 F e d method (System.Data.Data namespace), 475 query ng typed DataSets and,  477 F LEPATH opt on (SQL Server Aud t),  238 F LESTREAM,  325 335, 355 357 back ng up data and,  326 BULK opt on (OPENROWSET),  334 cons derat ons w th,  355 357 creat ng tab es w th,  330 331 database snapshots and,  356 data, de et ng,  334 335 data, stor ng/retr ev ng,  331 334 enab ng procedure for,  326 366 F LEGROUP...CONTA NS F LESTREAM c ause (CREATE DATABASE statement),  329 F LESTREAM data conta ner,  329 330 F LESTREAM enab ed database, creat ng,  329 331

746  Index

fi e system, restr ct ons on,  356 F eTab e and,  359 F eTab e feature and,  357 365 FTS and,  356 and garbage co ector, tr gger ng,  335 GET F LESTREAM TRANSACT ON CONTEXT funct on, 336 HADR and,  355 NSERT statement and,  331 332 m tat ons on,  355 357 Loca DB (SSDT), ack of support n,  356 og sh pp ng and,  356 m rror ng and,  355 mu t p e fi egroups and,  330 PathName method,  336 rep cat on, restr ct ons on,  355 356 ROWGU DCOL attr bute,  330 331 S NGLE BLOB opt on (OPENROWSET),  334 snapshot so at on eve and,  356 SQL Server Express ed t on, support for,  356 TDE and,  355 W ndows AP and,  360 F estream data co umnstore ndexes and,  704 F LESTREAM data conta ner,  329 330 fast access us ng Sq F eStream,  335 nterna behav or of,  332 333 F LESTREAM, enab ng,  326 329 eve s of access,  327 oca y, 326 328 server nstance,  328 329 sp configure system stored procedure and,  328 SQL Server Configurat on Manager,  326 328 SQL Server Configurat on Manager, VB a ternat ve to, 328 SSDT and,  328 SSMS and,  328 F LESTREAM feature NTFS and databases, coord nat on between,  188 fi e stream (F eTab e co umn name),  357 fi e stream ng,  323 366 BLOB data,  323 325 F LESTREAM,  325 335, 355 357 F eTab e feature,  357 365 Fu Text Search (FTS),  365 Sq F eStream c ass (.NET),  335 Stat st ca Semant c Search,  365 F eTab e feature,  357 365 cata og v ews for,  365 creat ng, 360 361

geometry collection

F LESTREAM and,  359 h erarchy d c ass (data type),  357 man pu at ng, 362 365 requ red CREATE DATABASE statements for,  360 System. O.F eStream and,  359 W ndows Exp orer and,  359 F LETABLEROOTPATH funct on (SQL Server) F eTab e access and,  362 fi e type (F eTab e co umn name),  358 F method (Sq DataAdapter c ass),  457 458, 461 F LTER funct on (DAX),  717 firehose cursor. See DataReaders firewa s C ent Sync Agent and,  624 F RST VALUE funct on,  98 99 fixed server ro s of og ns,  211 flat earth spat a mode . See p anar spat a mode F xPo samp e app cat on,  627 674 Force Protoco Encrypt on opt on (SQL Nat ve C ent), 224 ForceRo Back(Except on) (System.Transact ons. Prepar ngEn stment c ass),  198 ForceRo Back() method (System.Transact ons. Prepar ngEn stment c ass),  198 fore gn key re at onsh p estab sh ng, 26 FORMAT funct on,  107 date/t me formatt ng codes,  108 FORMATMESSAGE funct on and error hand ng,  112 Format parameter (Sq UserDefinedAggregate attr bute), 151 FOR SERVER AUD T c ause (SQL Server Aud t),  239, 241 FOR XML AUTO,  269 271 ELEMENTS keyword,  282 283 FOR XML EXPL C T vs.,  273 jo n c ause and,  269 270 XMLSCHEMA keyword and,  281 FOR XML commands,  268 280 e ement based XML, produc ng w th,  282 283 ELEMENTS keyword,  282 283 FOR XML AUTO,  269 271 FOR XML EXPL C T,  271 276 FOR XML PATH,  277 280 FOR XML RAW,  269 n ne XSD schemas, produc ng w th,  281 282 ROOT e ements and,  280 281 ROOT opt on,  280 281 TYPE opt on,  276 277 XMLSCHEMA keyword and,  281

FOR XML EXPL C T,  271 276 !ELEMENT flag,  273 FOR XML AUTO vs.,  273 FOR XML PATH vs.,  271 FOR XML PATH data (node funct on),  279 FOR XML EXPL C T vs.,  271 XPath express ons and,  277 280 FOR XML RAW,  269 ELEMENTS keyword,  282 285 query resu ts n SSDT and SSMS,  269 XMLSCHEMA keyword and,  281 for (XQuery keyword),  286 FROM c ause (C ass L brary project),  139 FULLGLOBE c ass,  412 413 FULLGLOBE keyword (WKT),  370 371 Fu outer jo n type,  74 Fu Text Search (FTS),  365 and F LESTREAM,  356 L KE operator vs.,  365 M crosoft Word documents and,  365 NEAR keyword,  365 supported fi e types for,  365 funct ons (SQL CLR),  143 147

G garbage co ect on (.NET Framework) D spose method ( D sposab e nterface),  439 us ng statement ( D sposab e objects),  439 Garc a, Rau ,  228 geodet c spat a mode ,  368 369 geography data type and,  374 Geograph c nformat on System (G S),  367 geography data type,  388 400 bu d ng map reg ons w th,  388 391 geodet c spat a mode and,  374 nstance s ze, support for,  411 412 M nDbCompat b tyLeve method,  408 409 STArea method,  391 392 STCurveN method,  405 406 STD stance method,  400 STLength method,  391 392 STNumCurves method,  405 406 vertex order, mportance of,  412 Geography Markup Language (GML),  374 GeomFromGm method, mport ng shapes w th, 374 geometry co ect on storage overyead of, compound curves vs.,  404 Index  747

GEOMETRYCOLLECTION keyword (WKT) GEOMETRYCOLLECT ON keyword (WKT),  371 geometry data type,  375 387 BufferW thCurves method,  406 408 creat ng tab es w th,  375 378 CurveToL neW thTo erance method,  409 410 sVa dDeta ed method,  410 411 MakeVa d method,  410 411 M nDbCompat b tyLeve method,  408 409 over app ng reg ons, man pu at ng,  384 387 p anar spat a mode and,  374 STBuffer method,  378 379 STCentro d method,  380 381 STCurveN method,  405 406 STCurveToL ne method,  409 410 STD fference method,  384 387 STD mens on method,  383 384 STEnve ope method,  380 381 STGeomFromText, mport ng shapes w th,  372 STGeomFromWKB, mport ng shapes w th,  373 ST ntersect on method,  382 383 ST ntersects method,  382 383 ST sVa d method,  410 411 STNumCurves method,  405 406 STSymD fference method,  384 387 STUn on method,  384 387 STxxxFromText, mport ng shapes w th va dat on, 372 373 GeomFromGm method (geometry c ass),  374 geospat a data,  367 424 B ng Maps, ntegrat ng w th,  413 422 c rc es, creat ng w th C RCULARSTR NG,  402 enhancements to, n SQL Server 2012,  400 413 geodet c mode for,  368 369 Geograph c nformat on System (G S),  367 geography data type,  388 400 Geography Markup Language (GML),  374 geometry data type,  375 387 G oba Pos t on ng Sate te (GPS) techno ogy,  367 p anar mode for,  368 spat a equa ty, test ng for,  404 SR D and,  392 standards for,  370 374 STCentro d method,  380 STD mens on method,  383 STEqua s method,  404 We Known B nary (WKB),  373 374 We Known Text (WKT),  370 373 geospat a data (enhancements n SQL Server 2012), 400 413

748  Index

backwards compatab ty for SQL Server and, 408 410 BufferW thCurves method,  406 408 C RCULARSTR NG object,  401 402 COMPOUNDCURVE object,  403 404 CURVEPOLYGON object,  404 405 CurveToL neW thTo erance method,  409 410 FULLGLOBE c ass,  412 413 geography nstance s ze, ncrease n,  411 412 ncreased prec s on,  413 sVa dDeta ed method,  410 411 MakeVa d method,  410 411 M nDbCompat b tyLeve method,  408 409 STCurveN method,  405 406 STCurveToL ne method,  409 410 ST sVa d method,  410 411 STNumCurves method,  405 406 Un t Sphere SR D,  413 GetAncestor method (h erarchy d c ass),  310 313, 357 sDescendentOf vs.,  316 GetAssoc atedChanges method (DataServ ce c ass), 560 GetChangeOperat on method (DataServ ce c ass),  560 GetDataTypeName method (Sq DataReader c ass),  446 GETDATE funct on,  62 GetDescendant method (h erarchy d c ass),  357 GetDescendant Method (h erarchy d c ass),  304 310 GET F LESTREAM TRANSACT ON CONTEXT funct on (F LESTREAM), 336 GetLeve method (h erarchy d c ass),  302 GetOr g na method (Doma nServ ces c ass),  560 GETPATHLOCATOR funct on F eTab e access and,  363 364 GetReparentedVa ue method (h erarchy d c ass),  318 319, 357 GetRoot method (h erarchy d c ass),  303 304 GetSchemaTab e method (Sq DataReader c ass),  446 GETUTCDATE funct on,  62 g oba assemb y cache (GAC),  190 m ss ng dependent assemb es and,  139 G oba Pos t on ng Sate te (GPS) techno ogy,  367 GROUP BY xm (data type) and,  257 GROUP BY c ause,  83 93 GROUP NG SETS operator,  88 W TH CUBE operator,  86 88 W TH ROLLUP operator,  85 86 GROUP NG SETS operator,  88 comb n ng operat ons us ng,  89 90

NULL va ues, hand ng,  90 93 use comb n ng W TH ROLLUP and W TH CUBE,  89 guest user account,  212 213

H hackers, 249 251 adm n strator passwords and,  249 d rect connect ons to nternet and,  249 nte gent observat on and,  250 251 search eng nes and,  250 251 SQL nject on and,  250 SQL Server Browser Serv ce and,  249 250 Hand eExcept on method (DataServ ce c ass) Except on property,  547 Hand eExcept onArgs parameter,  547 overr d ng, 546 UseVerboseErrors property,  547 h erarch ca tab es,  299 322 add ng nodes w th GetDescendant,  304 310 creat ng tab es,  301 302 GetReparentedVa ue method,  318 319 DENT TY va ues and,  302 ndex ng strateg es for,  313 315 sDescendantOf method,  315 317 orphaned nodes and,  319 popu at ng h erarch es,  303 313 pr mary key of,  301 query ng, 315 317 reorder ng nodes w th n,  317 321 subtrees, transp ant ng,  319 321 us ng transact ons to prevent co s ons,  311 h erarch ca tab es, creat ng,  301 302 GetAncestor method,  310 313 GetDescendant method,  304 305 GetLeve method,  302 ToStr ng method,  305 H erarch es (MDS),  677 Der ved, 677 Exp c t, 677 Recurs ve, 677 h erarchy d c ass (data type),  357 GetAncestor method,  357 GetDescendant method,  357 GetReparentedVa ue method,  357 sDescendantOf method,  357 h erarchy d data type,  300 301 GetAncestor method,  310 313 GetDescendant method,  304 310 GetLeve method,  302

Indexes And Keys link (SQL Azure Management Portal) GetReparentedVa ue method,  318 319 GetRoot method,  303 304 namespace ocat on of,  301 Parse method,  321 322 Read method,  321 322 :: syntax for,  303 ToStr ng method,  305 T SQL extens ons for,  300 Wr te method,  321 322 XML vs.,  299 H gh Ava ab ty D saster Recovery (HADR) F LESTREAM attr bute and,  355 HttpContext.Current.User property,  548 Hub Database (Data Sync term),  622 Hub W ns (Data Sync confl ct reso ut on),  624

I DataServ ceStreamProv der nterface (WCF Data Serv ces), 512 DbCommand nterface (System.Data namespace),  436 DbConnect on nterface (System.Data namespace), 436 DbDataReader nterface (System.Data namespace), 436 DbParameter nterface (System.Data namespace),  436 DbTransact on nterface (System.Data namespace),  436 DENT TY attr bute and SEQUENCE objects,  115 DENT TY va ues ass gn ng key va ues w th,  302 D sposab e nterface D spose method,  439 us ng statement and,  439 En stmentNot ficat on nterface,  195 Enumerab e nterface,  54 57 Enumerab e methods use n L NQ to Ent t es quer es,  490 Enumerab e (.NET nterface) arrays vs., as return va ue for TVFs,  145 147 F funct on,  106 mpedance m smatch, reso v ng,  500 501 MPERSONATE perm ss on ( og n),  221 mp c t transact ons data access, runt me behav or for,  454 data access w th,  453 455 nc ude attr bute (metadata c asses),  558 ndexes And Keys nk (SQL Azure Management Porta ), 592 Index  749

indexing strategies, hierarchical tables ndex ng strateg es, h erarch ca tab es,  313 315 breadth first,  314 315 depth first,  314 nDoubt method ( En stmentNot ficat on nterface), 195 nfrastructure as a serv ce ( aaS),  580 n t a zeServ ce method (EDM),  515 nject on attacks avo d ng, us ng parameters,  442 san t z ng nputs to avo d,  440 n memory B ( MB ). See Vert Paq nner jo n type,  74 Not fyPropertyChanged nterface (MVVM),  660 nsertCommand property (Sq DataAdapter c ass), 457 458 NSERT OVER DML syntax,  76 83 CHANGES, consum ng,  80 83 manag ng s ze of og fi es us ng,  79 OUTPUT... NTO, as fi terab e a ternat ve to,  77 80 nserts bu k, us ng TVPs,  49 51 ntegrat on Serv ces (SS S) SQL Server Data Too s (SSDT) support for,  4 nterceptor(s) change, 548 HttpContext.Current.User property,  548 query, 547 wr t ng, 547 548 nter eaved transact ons,  176 179 nternet Exp orer AtomPub respons feeds, configur ng for,  516 517 RSS feeds n 64 b t vers on,  516 WCF Data Serv ces, test ng n,  515 518 s arch ve (F eTab e co umn name),  357 sDescendantOf method (h erarchy d c ass),  315 317, 357 GetAncestor vs.,  316 317 return va ue of,  315 s d rectory (F eTab e co umn name),  357 va ues for,  358 s h dden (F eTab e co umn name),  357 s offl ne (F eTab e co umn name),  357 so at on (AC D property),  171 172 so at onLeve property (Transact onOpt ons object), 185 so at on eve s SET TRANSACT ON SOLAT ON LEVEL statement, 179 so at on eve s (of transact ons),  179 186 ADO.NET transact ons and,  184 186

750  Index

read comm tted,  181 read comm tted snapshot,  183 184 read uncomm tted,  179 181 repeatab e read,  182 ser a zab e, 182 snapshot, 182 183 s readon y (F eTab e co umn name),  357 s system (F eTab e co umn name),  357 s temporary (F eTab e co umn name),  357 sVa dDeta ed method (geometry c ass),  410 411

J JavaScr pt Object Notat on (JSON),  509 jo n methods chos ng, 74 75 types of,  67

K keyboard shortcuts execute a scr pt,  12 execute scr pt w th debugger,  12 stepp ng through code n debugger,  34 key encrypt on (asymmetr c),  222 key encrypt on (symmetr c),  222 key performance nd cator (KP ),  688 Know edge Bases, n DQS,  680 681 known performance nd cators (KP s) bu d ng, 720

L LAG funct on,  98 100 anguage ntegrated query (L NQ),  472 477 L NQ to DataSet,  473 477 L NQ to Objects,  472 T SQL vs.,  472 473 ast access t me (F eTab e co umn name),  357 LAST VALUE funct on,  98 99 ast wr te t me (F eTab e co umn name),  357 ax va dat on (XSDs),  263 264 anyAttr bute dec arat ons and,  263 processContents and,  263 azy oad ng quer es,  481 Ent ty Framework and,  498 499 LEAD funct on,  98 100 Left outer jo n type,  74 et (XQuery keyword),  286

Master Data Services (MDS)

L ghtwe ght Transact on Manager (LTM),  187 MS DTC, promot ng to,  188 189 vo at e en stment and,  188 L KE operator (T SQL) FTS eng ne vs.,  365 L NESTR NG keyword (WKT),  370 convert ng to POLYGON w th STBuffer,  378 379 L NQ Pocket Reference (A bahar and A bahar ),  472 L NQ to DataSet,  473 477 AsDataV ew method,  476 CopyToDataTab e method,  476 gener c DataSets, query ng,  474 476 Enumerab e objects as return va ue, 476 System.Data.DataSetExtens ons.d , 475 typed DataSet, query ng,  476 477 L NQ to Ent t es,  480, 488 490 change track ng,  498 499 deferred execut on,  489 Enumerab e methods, use n quer es,  490 object context azy oad ng,  498 499 runt me (m s)behav or of quer es,  492 L nqToEnt t esDoma nServ ce (EF),  559 L NQ to Ent t es Query (D rect SQL) property va ue (textbox), 488 L NQ to Ent t es Query (Stored Procedure) property va ue (textbox),  492 L NQ to Gener c DataSet property (Textbox),  474 L NQ to Objects,  472 L NQ to REST,  518 runt me dynam c behav or of,  523 L NQ to SQL DataContext object,  479 Ent ty Framework vs.,  478, 480 482 Ent ty SQL vs.,  480 anguages capab e of commun cat ng w th,  481 m grat ng to Ent ty Framework,  481 SQL CE and,  667 SQL CE databases, quer es aga nst,  671 unsuppored features n W ndows Phone,  667 WCF R A Serv ces, expos ng to,  559 W ndows Phone 7 and,  482, 667 L nqToSq Doma nServ ce (EF),  559 L NQ to Strong y Typed DataSet property va ue (textbox), 476 LoadAsync method (DataServ ceCo ect on c ass), 664 665 runt me behav or of,  664 LoadComp eted event (DataServ ceCo ect on c ass), 664 665

Loca DB dep oy ng to,  30 31 SQL Server Data Too s, use n,  27 31 LOG funct on,  109 og ca funct ons,  106 107 CHOOSE funct on,  106 CONCAT funct on,  107 FORMAT funct on,  107 F funct on,  106 LOG NPROPERTY funct on,  216 og ns, 210 213 aud t ng, 239 240 check ng credent a s dur ng authent cat on,  214 database users,  211 212 fixed server ro s of,  211 guest user account,  212 213 MPERSONATE perm ss on,  221 LOG NPROPERTY funct on and,  216 sys.sq og ns system v ew and,  216 og sh pp ng F LESTREAM and,  356

M MakeVa d method (geometry c ass),  410 411 Management Porta . See SQL Azure Management Porta Many To Many Re at onsh ps property va ue (textbox), 501 Mapp ng Deta s pane (EDM),  500, 501 mapp ng schema (EDM),  482 mash up defined,  413 Mass ve y Para e Process ng (MPP),  685 Master Data Management (MDM),  677 Master Data Serv ces (MDS),  677 680 Attr bute Groups,  677 Attr butes (of Ent t es),  677 Bus ness Ru es, defin ng,  678 DQS and,  681 Ent t es, defin ng n,  677 Exce add n,  678 679 export ng member data,  679 H erarch es of Members,  677 Master Data Management (MDM),  677 Mode s, 677 secur ty contro s n,  678 Web Serv ces nterface,  679 Workflow fac t es n,  678 Index  751

mathematical functions mathemat ca funct ons,  109 123 LOG funct on,  109 max degree of para e sm (MDOP) configurat on opt on, 707 MAX F LES opt on (SQL Server Aud t),  238 MAX ROLLOVER F LES opt on (SQL Server Aud t),  238 MAXS ZE opt on (SQL Server Aud t),  238 MAXS ZE parameter (ALTER DATABASE command),  599 measures, 718 724 creat ng, 718 720 Med aE ement contro (WPF c ent app cat ons),  354 Member Database (Data Sync term),  622 Members (MDS) Exce add n, enter ng w th,  678 679 export ng data,  679 Web Serv ces nterface, enter ng data w th,  679 memory mapped fi es support for, n F eTab e,  364 Merge method (SQL CLR aggregates),  155 MERGE statement,  65 76 and cons stant tr gger behav or,  75 mp ementat on of,  75 76 jo n method, chos ng,  74 75 jo n methods, types of,  67 output of,  73 74 source requ rements for,  67 source/target of, defin ng,  67 tab e rep cat on us ng,  70 target requ rements for,  67 WHEN MATCHED c ause,  68 69 WHEN NOT MATCHED BY SOURCE c ause,  71 73 WHEN NOT MATCHED BY TARGET c ause,  69 Message Transm ss on Opt m zat on Mechan sm (MTOM) BLOBs and,  348 metadata c asses,  557 559 Compos t on attr bute,  558 nc ude attr bute,  558 MetadataType attr bute,  557 nest ng, 557 Roundtr pOr g na attr bute,  558 metadata d scovery,  118 122 parameter zed quer es for,  120 121 @SortOrder parameter,  120 sys.dm exec descr be first resu t set for object procedure, 118 sys.dm exec descr be first resu t set funct on, 118 119 sys.sp descr be first resu t set procedure,  118 sys.sp descr be undec ared parameters procedure,  118, 122

752  Index

MetadataType attr bute (EDM),  557 M crosoft ADO.NET Data Serv ces,  527 M crosoft B ng Maps AJAX and,  416 417 geospat a data n SQL Server and,  413 422 M crosoft B Stack,  676 M crosoft Books On ne co umnstore ndexes n,  704 M crosoft D str buted Transact on Coord nator (MS DTC),  187 b ock ng promot on to, n nked databases,  190 LTM, promot ng from,  188 189 performance ssues w th,  187 M crosoft Exce spreadsheets Fu Text Search (FTS) and,  365 M crosoft PowerPo nt decks Fu Text Search (FTS) and,  365 M crosoft SQL Server Compact ed t on (SQL CE),  666 672 creat ng, 667 671 ExportSq Ce ut ty,  666 L NQ to SQL and,  667 L NQ to SQL quer es aga nst,  671 SQL Server Compact Too box,  666 WCF Data Serv ces, runn ng aga nst oca ,  671 M crosoft.Sq Server.Server namespace c asses for oca server/remote c ent data access n, 132 .NET code attr butes for SQL CLR projects n,  129 Send method, parameters for,  133 Sq Context object,  133 Sq P pe object,  133 System.Data.d .NET Framework and,  132 M crosoft s SQL Server 2012 Deve oper Tra n ng K t, 708 M crosoft Sync Framework Too k t,  627 M crosoft Sync Serv ce Framework M crosoft Sync Framework Too k t,  627 SQL Azure Data Sync and,  621 SQL Azure Data Sync vs.,  627 M crosoft Transact on Server (MTS),  190 M crosoft V sua Bas c .NET C# vs.,  141 M nDbCompat b tyLeve method (geometry c ass), 408 409 Mode Browser stored procedures, mapp ng n,  490 494 Mode Browser (V sua Stud o),  486, 486 487 mode ng,  715 718, 718 724 Mode s (MDS),  677

Mode V ew V ewMode (MVVM) pattern,  651 661 Not fyPropertyChanged nterface,  660 LoadData method,  651 WCF Data Serv ces, ca ng,  662 674 MSDN SQL CLR attr bute coverage n,  129 MSSQLSERVER Propert es d a og box (SQL Server Network Configurat on),  223 Mu t D mens ona eXpress ons (MDX),  687 MULT L NESTR NG keyword (WKT),  371 mu t p e act ve resu t sets (MARS),  176 179 savepo nts and,  178 185 transact ons and,  177 MULT PO NT keyword (WKT),  371 MULT POLYGON keyword (WKT),  371 mu t tenancy Federat ons, support n,  611

N name (F eTab e co umn name),  357 NEAR keyword (Fu Text Search),  365 nested transact ons behav or of ROLLBACK statements n,  175 network atency SQL Azure data centers, choos ng to reduce,  630 New Tab e nk (SQL Azure Management Porta ),  590 nH bernate brary,  478 node (XPath node funct on),  279 nonrepeatab e reads read uncomm tted so at on eve and,  181 repeatab e read so at on eve and,  182 ser a zab e so at on eve and,  182 Nonrepeatab e read (transact on),  178 NT LE funct on,  93 NULL va ues as hand ed by GROUP NG SETS,  90

O object or ented program ng DataSets and,  477 478 Object Re at ona Mode ng (ORM),  477 507 ADO.NET vs.,  479 defin ng,  478 Ent ty Data Mode (EDM),  482 508 ent ty framework,  482 508 L NQ to SQL,  479 482 nH bernate brary,  478

OVER clause occas ona y connected systems b d rect ona synchron zat on,  620 character st cs of,  620 621 creat ng, 626 629 data management for,  620 621 OData W ndows Phone 7, consum ng on,  662 674 OData UR spec ficat ons, ocat on of,  518 OFFSET/FETCH NEXT syntax ROW NUMBER funct on vs.,  115 usage, 114 115 ON FA LURE opt on (SQL Server Aud t),  237 OnL ne Ana yt ca Process ng (OLAP),  686 687 on ne ana yt ca process ng (OLAP) quer es W TH ROLLUP and W TH CUBE c auses vs.,  83 OnL ne Transact ona Process ng (OLTP) OLAP vs.,  686 on ne transact on process ng (OLTP) system,  255 256 OnStartProcess ngRequest method (DataServ ce c ass), overr d ng,  546 sBatchRequest flag,  546 ProcessRequestArgs parameter,  546 Open Database Connect v ty (ODBC) AP ,  428 Open Data Protoco (OData) WCF Data Serv ces and,  511 Open Geospat a Consort um (OGC) WKT, WKB, and GML and,  370 Open method (Sq Connect on object),  439 OPENROWSET BULK opt on,  334 S NGLE BLOB opt on,  334 OpenSq F estream funct on (SQL Server),  335 Open statement (Sq Connect on nstance),  193 OPENXML system funct on,  284 285 sp xm preparedocument stored procedure and, 284 285 Orac e database(s) access ng through SQL statements,  53 54 Orac eDataReader object,  53 54 ORDER BY xm (data type) and,  257 outbound data, defined (SQL Azure),  583 OUTPUT... NTO h stor ca ogs, use n creat ng,  76 NSERT OVER DML as fi terab e a ternat ve to, 77 80 Output Parameter property (Textbox),  448 OVER c ause,  93 97 aggregate funct ons, trad t ona ,  94 aggregat ons, runn ng,  95 96 Index  753

Parameterized Direct SQL (text box property) OVER c ause, Continued aggregat ons, s d ng,  96 97 DENSE RANK funct on,  93 NT LE funct on,  93 PART T ON BY c ause,  94 RANGE vs. ROWS, us ng,  97 RANK funct on,  93 ROW NUMBER funct on,  93

P Parameter zed D rect SQL (text box property),  441 Parameters co ect on configur ng,  449 Parameters co ect on (Sq Command object),  441 parent path ocator (F eTab e co umn name),  358 PARSE funct on,  103, 104 Parse method (h erarchy d c ass),  321 322 ToStr ng and,  321 part a y conta ned databases,  244 249 co at ons, 247 249 conta ned user, creat ng,  245 246 creat ng, 245 features of,  246 249 n t a Cata og c ause and,  246 sys.dm db unconta ned ent t es DMV and,  246 tempdb, 247 249 unconta ned ent t es v ew,  246 247 PART T ON BY c ause,  94 password po c es,  215 216 sett ng w th Secur ty Po cy app et,  215 passwords aud t ng, 239 path ocator (F eTab e co umn name),  357 stream d vs., as permenant reference,  358 PathName method (varb nary(max)),  336 return va ue of,  336 Pat ndex method,  256 PERCENT LE CONT funct on,  98, 100, 102 PERCENT LE D SC funct on,  98, 100, 102 PERCENT RANK funct on,  98, 100 101 performance ssues DTC and,  187 PerformancePo nt dashboards SSRS reports and,  694 PerformancePo nt Serv ces (PPS),  696 PERM SS ON SET va ue,  139 PERM SS ON SET va ue (C ass L brary project),  139 perm ss ons for users execut on context to set,  220

754  Index

phantom reads (transact on),  178 read uncomm tted so at on eve and,  181 repeatab e read so at on eve and,  182 ser a zab e so at on eve and,  182 P ng Sync Serv ce button (C ent Sync Agent),  634 P pe object (Sq Context) Send method, over oad ng,  134 P pe property (Sq Context object),  133 p p ng data Sq DataRecord and Sq MetaData, w th,  134 136 p a n o d CLR object (POCO),  478, 481 p anar spat a mode ,  368 geometry data type and,  374 p atform as a serv ce (PaaS),  580 POCOs expos ng to WCF R A Serv ces w th Doma nServ ce c ass,  559 PO NT keyword (WKT),  370 371 Po cy Based Management Framework (PBM),  209 POLYGON keyword (WKT),  370 371 portab ty of databases w th part a conta nment,  244 PowerP vot Advanced Mode,  723 724 B SM and,  711 712 ca cu ated co umns,  716 717 Connect To A Data Source page,  714 Create H erarchy opt on,  721 Create P votChart d a og box,  724 Create Re at onsh p button,  718 D agram v ew,  720 down oad source for,  712 Exce and,  717 Exce , query ng n,  724 726 From Other Sources button,  714 h erarch es and,  720 722 KP s, bu d ng,  720 Manage Re at onsh ps button,  717 measures, 718 724 mode ng w th,  715 718 P votTab e button,  724 P votTab e (Hor zonta ) d a og box,  724 re at onsh ps, manag ng,  717 718 row/co umn man pu at on n,  715 716 SharePo nt, for,  726 727 SSAS Tabu ar, mport ng nto,  727 728 Tab e mport W zard,  714 PowerP vot (Exce add n),  687 690 key performance nd cator (KP ),  688 SharePo nt as requ red by,  687 SSAS B Semant c Mode vs.,  689

PowerP vot Ga ery Documents tab,  731 Power V ew Reports, creat ng n,  732 Power V ew B SM data sources and,  691 Power V ew Report,  732 734 Pred x on Software,  690 Prepared method (System.Transact ons. Prepar ngEn stment c ass),  198 Prepare method ( En stmentNot ficat on nterface), 195 processadm n (fixed server ro ),  211 processContents, 263 process ng nstruct on (XPath node funct on),  279 Pro Cyc ng Tour (Ph ade ph a, PA),  388 Programm ng Ent ty Framework, Second Ed t on (Lerman), 487 promotab e s ng e phase en stment (PSPE),  188 189 pub c (fixed server ro ),  211 pub c ro (database ro ) secur ty r sks w th,  212 Pub sh funct on (SSDT),  137 138

Q Query nterceptors,  547 query performance depth vs. breadth ndex ng and,  314 Query Performance page (SQL Azure Management Porta ), 596 query processor (QP),  706 QUEUE DELAY opt on (SQL Server Aud t),  236 237

R RA SERROR statement requ rements for,  112 THROW vs.,  110, 111 113 RANGE c ause ROWS c ause vs.,  97 RANK funct on,  93 CUME D ST funct on and,  101 PERCENT RANK funct on and,  101 raw data objects (ADO.NET),  436 455 connect ons/commands, creat ng,  437 440 data readers, terat ng,  443 447 parameters, us ng,  440 442 sca ar va ues, return ng,  447 455 stored procedures, ca ng,  442 508 updates/transact ons, batch ng,  450 455

resource manager(s) (RM) ReadComm tted (ADO.NET so at on eve ),  185 read comm tted so at on eve ,  181 read comm tted snapshot so at on eve ,  183 184 USE master statement,  184 Read method (h erarchy d c ass),  321 322 ReadUncomm tted (ADO.NET so at on eve ),  184 read uncomm tted so at on eve ,  179 181 d rty reads and,  179 Rea y S mp e Synd cat on (RSS) as XSD schema,  260 Rea y S mp y Synd cat on [RSS],  509 RecordSet object vs. DataSets,  455 recurs ve common tab e express ons autocomm t transact on mode and,  173 Recurs ve H erarch es (MDS),  677 Reflect on prov der (WCF Data Serv ces),  512 remote BLOB storage (RBS),  356 Remote Data Objects (RDO),  428 Repeatab eRead (ADO.NET so at on eve ),  185 rep cat on F LESTREAM, m tat ons w th,  355 356 Report Bu der SAR reports, dep oy ng,  616 Report Bu der 3.0 (SSRS),  693 report ng author ng, 614 615 prov s on ng for,  613 614 Report ng Serv ces project too ng (V sua Stud o) SQL Azure Report ng (SAR) and,  615 Report ng Serv ces (SSRS) SQL Server Data Too s (SSDT) support for,  4 report ng (SQL Azure),  612 617 reports dep oy ng, 615 617 Report W zard SAR reports and,  614 Representat ona State Transfer (REST) protoco ,  509 expos ng EDMs to,  514 WCF Data Serv ces and,  511 RESERVE D SK SPACE opt on (SQL Server Aud t), 238 239 resource manager Sq Connect on as,  200 resource manager (RM) En stmentNot ficat on nterface, requ rement for, 195 resource manager(s) best pract ces for,  199 resource manager(s) (RM),  186, 198 200 CreateYourOwnRM so ut on,  195 Index  755

Results As Grid (SSDT) resource manager(s) (RM), Continued creat ng, 194 198 re at ons between two,  199 200 ro back ca s, use n,  198 199 ro back, ssu ng from,  199 Transact onScope and,  198 Resu ts As Gr d (SSDT),  134 Resu ts As Text (SSDT),  134 Resu ts To Gr d (SSMS),  134 Resu ts To Text opt on (SSMS),  134 R ght outer jo n type,  74 ro e (Compute resource),  641 Ro back method ( En stmentNot ficat on nterface), 195 ROLLBACK TRANSACT ON T SQL statement,  174 behav or of, n nested transact ons,  175 ROOT e ement requ rement for n XML documents,  280 Roundtr pO g na attr bute (metadata c asses),  558 RowF ter property (DataV ew object),  473 ROWGU DCOL attr bute (F LESTREAM),  330 331 ROW NUMBER funct on,  93, 113 114 OFFSET/FETCH NEXT syntax vs.,  115 WHERE c ause and,  113 ROWS c ause RANGE c ause vs.,  97

S Safe (assemby secur ty eve ),  161 Samp eR aContext c ass (WCF R A Serv ces),  566 SARASPNETTest samp e app cat on,  597 SaveChanges method (DataServ ceContext),  526 savepo nts exp c t transact ons and,  175 176 MARS and,  178 185 SAVE TRANSACT ON statement,  175 177 schemas change prop gat on, mp ementat on of,  6 7 compar ng, 35 38 .NET namespaces vs.,  24 scope of,  24 use n transfer ng ownersh p of database objects, 216 218 search eng nes hackers and,  250 251 secur ty, 207 252 asymmetr c key encrypt on,  222 authent cat on/author zat on, 213 221 b ogs on,  228

756  Index

cert ficates,  222, 223 cert ficates, se f s gned,  222 databases, h d ng beh nd firewa s/SQL Azure databases, 626 database users, fixed ro s of,  212 Data Sync and,  625 dem tar zed zone (DMZ), sett ng up,  626 Encrypted F e System (EFS),  229 encrypt on support,  222 234 fixed server ro s of og ns,  211 guest user account and,  212 213 hackers and,  249 251 og ns, 210 211 MDS, contro s n,  678 part a y conta ned databases,  244 249 Po cy Based Management Framework (PBM),  209 pr nc pa s/ent t es, 209 210 pub c ro (database ro ). secur ty r sks w th,  212 reduc ng the surface area for attack,  209 Secure by Defau t,  208 Secure by Dep oyment,  208 Secure by Des gn,  208 Secure Commun cat ons,  208 252 Secur ty Framework,  208 209 se f s gned cert ficates,  223 SET ENCRYPT ON ON c ause,  231 sp configure stored procedure,  209 SQL Azure connect on sett ngs,  597 SQL CLR, spec fy ng eve of,  161 162 SQL Server Aud t,  234 244 SQL Server Configurat on Manager too ,  223 SQL Server Nat ve Access C ent AP ,  223 SQL Server Surface Area Configurat on Too ,  209 SSAS Tabu ar mode,  728 symmetr c key encrypt on,  222 transparent data encrypt on,  229 234 Trustworthy Comput ng n t at ve (M crosoft),  207 W ndows og n support n SQL Server,  210 W ndows Phone and, oca data on,  667 secur tyadm n (fixed server ro ),  211 secur ty cert ficates,  222 Secur ty Framework,  208 209 Se ect a Database nk (SQL Azure Management Porta ), 594 Se ectCommand property (Sq DataAdapter c ass), 457 458 Se ect Target Schema d a og,  36 se f s gned cert ficates,  222, 223 semant c zoon,  594 Send method (Sq P pe object),  133

SendResu tsEnd method (Sq P pe object),  133 SendResu tsRow method (Sq P pe object),  133 SendResu tsStart method (Sq P pe object),  133 SEQUENCE object,  115 118 ALTER SEQUENCE statement,  117 CREATE SEQUENCE statement,  116 DEFAULT constra nt on,  117 DROP SEQUENCE statement,  118 sequences m tat ons on,  117 123 SEQUENCE object,  115 118 Ser a zab e (ADO.NET so at on eve ),  185 Ser a zab e attr bute UDTs and,  158 serveradm n (fixed server ro ),  211 server ro es og ns and,  211 server s de pag ng,  113 115 OFFSET/FETCH NEXT syntax,  114 115 ROW NUMBER funct on,  113 114 Serv ce Master Key (SMK),  224 serv ces Atom Pub sh ng Protoco (AtomPub),  509 defin ng,  509 510 Rea y S mp y Synd cat on [RSS],  509 Representat ona State Transfer (REST) protoco ,  509 S mp e Object Access Protoco (SOAP),  509 V rtua Pr vate Network [VPN],  509 SET ENCRYPT ON ON c ause,  231 SET MPL C T TRANSACT ONS T SQL statement,  176 SET TRANSACT ON SOLAT ON LEVEL statement, 179 setupadm n (fixed server ro ),  211 shard ng scheme,  607 SharePo nt PPS and,  696 SharePo nt Enterpr se 2010,  726 727 PowerP vot and,  726 727 SharePo nt vers ons requ rements for,  697 S ver ght, 518 DataGr d contro , add ng to project n V sua Stud o, 561 L NQ to REST,  518 S ver ght App cat on temp ate,  549 551 System.Runt me.Ser a zat on (WCF assemb y),  550 System.Serv ceMode assemb y,  550 System.Serv ceMode .Doma nServ ces.C ent assemb y, 550



sp configure stored procedure System.Serv ceMode .Doma nServ ces.C ent.Web assemb y, 550 System.Serv ceMode .eb.Extens ons assemb y,  550 WCF R A Serv ces and,  549 WCF R A Serv ces c ent, bu d ng for,  561 568 S ver ght Deve oper Runt me,  549 S ver ght For W ndows Phone temp ate (V sua Stud o), 647 S verL ght Too k t ocat on of and nsta ng,  648 S mp e Object Access Protoco (SOAP),  509 S ng e method (DataServ ceQuery c ass),  665 sma datet me data type,  58 SMALLDATET MEFROMPARTS funct on,  105 Smart Bus ness nte gence So ut ons w th M crosoft® SQL Server® 2008 (Lang t, Goff, et a ),Z  676 SMK AP s used to encrypt,  225 back ng up/restor ng,  225 DMK and,  225 snapshot sav ng, 25 Snapshot (ADO.NET so at on eve ),  185 snapshot so at on eve ,  182 183 enab ng, 183 F LESTREAM and,  356 SOAP as XSD schema,  260 software as a serv ce (SaaS),  611 So ut on Exp orer add ng WCF Data Serv ces to project n,  519 ASP.NET Deve opment Web Server,  516 dep oy ng W ndows Phone app cat ons to W ndows Azure w th,  672 674 W ndows Forms App cat on temp ate,  519 source code contro (SCC) n SSDT,  23 spat a data types,  374 400 fu st ng of a methods, ocat on of,  370 geography data type,  388 400 geometry data type,  375 387 spat a nstance va dat on,  410 411 ST sVa d, sVa dDeta ed, and MakeVa d methods, 410 411 Spat a Reference Ds (SR D) compar ng, requ rements for,  392 geospat a data and,  392 Un t Sphere SR D,  413 sp configure stored procedure,  209

Index  757

sp xml preparedocument (system stored procedure) sp xm preparedocument (system stored procedure), 284 285 SQL Azure,  579 618 accounts requ red for,  584 adm n strat ve user, configur ng,  587 588 check ng database for compat b ty w th,  40 c ents, connect ng to,  596 599 c ustered ndex, requ rements for,  40 Create Server w zard,  587 588 database, prov s on ng,  589 data centers, ocat ons of,  586 587 data, enter ng,  590 592 dep oy ng to,  602 604 Dep oy to SQL Azure feature (SSMS),  603 Export Data T er App cat on funct on (SSMS),  603 Federat ons, 607 612 firewa , configur ng,  587 588 firewa , sett ng up w th T SQL,  588 free tr a membersh p for,  583 584 h story of,  581 hostname for database,  588 ndex des gn,  592 593 m tat ons on databases n,  581 582 management/v sua zat ons n,  593 618 manag ng databases,  589 599 m grat ng to/synch ng w th,  599 606 outbound data, charges for,  583 p atform as a serv ce (PaaS),  580 pr c ng, 583 584 pub sh ng to, us ng SQL Server Object Exp orer, 39 42 query ng n a browser,  592 restr ct on st, source for,  582 SARASPNETTest samp e app cat on,  597 sca ab ty requ rements for,  582 servers, prov s on ng,  586 588 sett ng up for,  584 589 s ze m t on,  581 SQL Azure C ent Sync Agent,  624 SQL Azure Data Sync (Data Sync),  621 626 SQL Azure Management Porta ,  589 599 SQL Azure Report ng (SAR),  612 617 SQL Server Data T er App cat ons (DACs),  599 606 SQL Server, m grat on from,  605 606 SQL Server, m grat on to,  606 SQL Server vs.,  582, 599 SSDS as predecessor for,  581 subscr pt on rates for,  583 tab es, creat ng,  590 592 Tabu ar Data Stream (TDS) protoco ,  581

758  Index

tr a account, sett ng up,  585 updates, dep oy ng to,  604 updates, dep oy ng w th SSDT,  604 updates, dep oy ng w th SSMS,  604 W ndows Azure Management Porta ,  586 588 W ndows L ve D, requ rement for,  584 XML support n,  581 SQL Azure C ent Sync Agent (Data Sync),  622, 624 configur ng,  633 634 nsta ng, 632 633 P ng Sync Serv ce button,  634 Subm t Agent Key button,  634 Test Connect on button,  634 SQL Azure, connect ng to,  596 599 encrypt on sett ngs,  597 firewa , c ent P sett ngs n,  597 from SSMS,  597 599 host name sett ngs,  596 SQL Server Authent cat on sett ngs,  597 SSDT, 597 599 SSDT SQL Server Object Exp orer and,  599 SSMS Object Exp orer and,  599 TCP sett ngs,  597 SQL Azure Data Sync (Data Sync),  621 626 capab es/features of,  621 622 C ent Sync Agent,  624 configur ng,  630 641 confl ct reso ut on po c es,  624 datasets, defin ng,  636 hub database, sett ng up,  635 m tat ons on wh e n prev ew re ease,  621 manua synchron zat on,  623 M crosoft Sync Serv ce Framework vs.,  627 M crosoft Sync Serv ces Framework and,  621 performance/costs w th,  625 626 prov s on ng the server,  630 requ rements for,  629 secur ty, 625 sync d rect ons supported by,  621 622 Sync Groups,  623 624 Sync Groups, creat ng,  631 641 Sync Now button (W ndows Azure Management Porta ), 639 term no ogy for,  622 W ndows Azure Management Porta ,  631 SQL Azure Management Porta ,  589, 589 599 DACs and,  602 dashboard, v ew ng,  593 596 Data screen,  591 dep oy ng databases w th,  604

Des gn tab,  590 Federat on Members, sp tt ng/dropp ng,  610 Federat ons, creat ng n,  608 ndexes And Keys nk,  592 New Tab e nk,  590 query execut on p ans, v sua z ng n,  594 618 Query Performance page,  596 Se ect a Database nk,  594 tab es, creat ng,  590 592 T SQL query too ,  592 Upgrade feature, to dep oy updates,  604 SQL Azure Report ng (SAR),  612 617 East US data center and,  587 m tat ons/restr ct ons on,  613 report author ng for,  614 615 Report Bu der to dep oy reports,  616 Report ng Serv ces project too ng, dep oy ng w th, 615 reports, dep oy ng,  615 617 Report W zard, us ng,  614 SAR credent a s,  616 SSRS vs.,  617 W ndows Azure Management Porta too s for, 613 614 SQL Azure, sett ng up,  584 589 accounts for,  584 589 adm n strat ve user, creat ng,  587 588 Azure account, sett ng up,  585 Create Database d a og,  589 Create Server w zard,  587 588 firewa , configur ng,  587 588 server, prov s on ng,  586 W ndows Azure Management Porta ,  586 588 Sq C ent (.NET Framework),  429 SQL CLR,  125 168 AdventureWorks2012 samp e database,  130 aggregates, 151 155 best pract ces for,  168 code attr butes,  129 dep oyment, 136 143 d str buted transact ons and,  202 204 ent t es, exam n ng/manag ng,  162 168 funct ons, 143 147 ntegrat on, enab ng,  126 .NET deve opers and,  130 samp e code,  130 secur ty, 161 162 SQLCLRDemo samp e project,  144 stored procedures,  130 132



SqlFacet attribute (SQL CLR) System.Transact ons.Transact onScope, us ng w th, 203 204 transact ons and,  201 202 tr ggers, 148 151 T SQL funct ons and,  145 147 TVFs n,  145 147 types, 156 160 V sua Stud o/SQL Server ntegrat on,  126 130 SQLCLRDemo samp e project,  144 sq :co umn (SQL Server XQuery extens on),  293 Sq Command c ass (System.Data.Sq C ent namespace), 436 AddW thVa ue method,  441 CommandText property,  447 ExecuteSca ar method,  447 450 Parameters co ect on,  441 stored procedures, ca ng w th,  442 443 Transact on property,  450 Sq Connect on RM, as,  200 Sq Connect on c ass (System.Data.Sq C ent namespace), 436 Beg nTransact on method,  450 C ose method,  440 Connect onStr ng property,  439 Open method,  439 Sq Connect on nstance Open statement,  193 Transact onScope and,  193 Sq Context object (M crosoft.Sq Server.Server namespace), 133 Sq DataReader Sq P pe objects and,  133 Sq DataReader c ass (System.Data.Sq C ent namespace),  437, 457 behav or of,  444 co umn va ue extract on w th ndexed notat on,  445 De eteCommand property,  457 458 Ent tyDataReader vs.,  507 F method,  457 458, 461 GetDataTypeName method,  446 GetSchemaTab e method,  446 nsertCommand property,  457 458 Se ectCommand property,  457 458 UpdateCommand property,  457 458 Update method,  457 458 Sq DbType enumerat on (System.Data.Sq Types namespace), 135 Sq Facet attr bute (SQL CLR),  129

Index  759

SqlFileStream class (.NET) Sq F eStream c ass (.NET),  335 355 and fast BLOB transact ons,  335 data access,  338 347 forms c ent, bu d ng,  337 338 GET F LESTREAM TRANSACT ON CONTEXT funct on, 336 nterna behav or of,  335 336 PathName method and,  336 secur ty cons derat ons w th,  356 stream ng HTTP serv ce, creat ng w th,  348 352 System.Data.Sq Types namespace,  341 System.Data.Sq Types namespace and,  341 WPF c ent, bu d ng w th,  352 355 Sq Funct on attr bute (SQL CLR) paramaters of,  129 Sq H erarchy d type unsuppored h erarchy d funct ona ty n,  322 Sq Method attr bute (SQL CLR),  129 Sq Parameter c ass (System.Data.Sq C ent namespace), 437 Sq P pe object (M crosoft.Sq Server.Server namespace), 133 ExecuteAndSend method,  133 Send method,  133 SendResu tsEnd method,  133 SendResu tsRow method,  133 SendResu tsStart method,  133 Sq DataReader objects and,  133 Sq Procedure attr bute (StoredProcedures c ass),  141 SQL Server xm data type and,  288 297 XQuery and,  288 296 SQL Server Ana ys s Serv ces (SSAS) B Semant c Mode (B SM) type,  689 B SM and,  711, 712 data m n ng,  690 691 data m n ng eng ne for,  690 dep oyment propert es, sett ng,  730 731 Mu t D mens ona eXpress ons (MDX),  687 OLAP and,  686 687 OLAP cubes,  686 687 PowerP vot, 687 690 Pred x on Software data m n ng software for,  690 SQL Server Data Too s (SSDT) support for,  4 SSAS Tabu ar Mode,  687 690 SS S and,  682 SSRS and,  692 storage opt ons n,  710 Tabu ar mode,  689 690 Tabu ar Mode,  687 690

760  Index

Vert Paq storage mode ,  688 xVe oc ty and, vocabu ary for,  710 711 SQL Server Aud t,  234 244 ALTER DATABASE AUD T SPEC F CAT ON statement, 240 ALTER SERVER AUD T SPEC F CAT ON statement, 239 ALTER SERVER AUD T statement,  235 236 aud t act ons, database eve ,  240 AUD T GU D opt on,  237 ava ab ty of,  234 cata og v ews, query ng,  244 CREATE DATABASE AUD T SPEC F CAT ON statement, 240 CREATE SERVER AUD T SPEC F CAT ON statement, 239 CREATE SERVER AUD T statement,  235 creat ng an aud t object,  235 236 database events, aud t ng,  240 241 events, v ew ng,  242 244 Event V ewer (Adm n strat ve Too s),  242 F LEPATH opt on,  238 FOR SERVER AUD T c ause,  239, 241 ogs, v ew ng n SSMS,  242 MAX F LES opt on,  238 MAX ROLLOVER F LES opt on,  238 MAXS ZE opt on,  238 ON FA LURE opt on,  237 QUEUE DELAY opt on,  236 237 record ng to fi e system,  238 239 record ng to W ndows event og,  239 RESERVE D SK SPACE opt on,  238 239 server events, aud t ng,  239 240 SQL Server Aud t Act on Groups and Act ons post, 240 STATE opt on,  237 238 sys.fn get aud t fi e funct on, v ew ng ogs w th, 243 244 TO F LE c ause,  238 SQL Server Authent cat on SQL Azure, connect ng to,  597 SQL Server Books On ne,  173 Access Contro top cs,  213 CLR aggregates n,  151 SQL CLR attr bute coverage n,  129 SQL Server Browser serv ce UDP, rep aced by,  213 SQL Server Common Language Runt me ntegrat on (SQL CLR) T SQL vs.,  45

SQL Server Compact Too box,  666 SQL Server Configurat on Manager F LESTREAM, enab ng,  326 328 SQL Server Configurat on Manager too ,  223 SQL Server Database Project bu d errors,  137 bu d ng, w th SQL CLR,  137 138 dep oy ng, 138 140 mport ng databases nto,  137 138 SQL Server Database Project C# temp ate,  143 SQL Server Database Projects (SSDT) DACPACs and,  600 SQL Server Database Project type,  7 SQL Server Data Qua ty Serv ces (DQS),  680 681 SQL Server Data Serv ces (SSDS),  581 SQL Server Data Too s (SSDT),  3 44 adopt ng, 42 43 Bus ness te gence Deve oper Stud o (B DS) and,  4 change scr pts generated by,  20 change scr pts, mp ementat on of,  20 connected deve opment n,  6 7 CREATE statements,  5 creat ng d sconnected database project w th,  7 8 creat ng new connected database project w th, 10 16 data or ented funct ona ty, ack of,  42 des gn ng tab e changes us ng,  16 17 ntegrat on Serv ces (SS S) support n,  4 Loca DB as test env ronment for,  27 31 manag ng deve opment/des gn cha enges w th,  4 5 mode based deve opment n,  5 6 offl ne tab e des gner,  25 27 pub sh ng to SQL Azure,  39 42 refactor ng, 31 33 Report ng Serv ces (SSRS) support n,  4 runn ng on y se ected statements n,  14 schemas, compar ng,  35 38 serv ces/too ng n,  4 5 snapshots, tak ng,  25 source code contro (SCC) and,  23 source for,  9 spat a v ewer, ack of,  378 SQL Server Ana ys s Serv ces (SSAS) support n,  4 SQL Server Database Project type n,  7 8 SQL Server Management Stud o (SSMS) vs.,  3 SQL Server Object Exp orer and,  6 SQL Server Object Exp orer, connect ng w th,  10 16 Tab e Des gner (connected case), us ng,  17 22 target ng p atforms,  9 test ng/debugg ng n,  33 35

SQL Server Profiler T SQL object representat on,  5 vers on ng/snapshots, 8 9 V sua Stud o and,  3 V sua Stud o Database Profess ona (DbPro) vs.,  3 work ng w th offl ne databases,  22 24 SQL Server Deve oper Too s (SSDT) xm (data type) and,  258 SQL Server Express ed t on F LESTREAM support n,  356 SQL Server ntegrat on Serv ces (SS S),  681 683 Connect on Managers,  682 Contro F ows,  682 Data F ows,  682 DQS and,  681 SSAS and,  682 Transforms, 682 SQL Server Management Stud o (SSMS) aud t ogs, v ew ng n,  242 database users, creat ng n,  211 212 F LESTREAM, enab ng w th,  328 sett ng defau t database w th,  12 spat a v ewer n,  378 SQL Server Data Too s (SSDT) vs.,  3 xm (data type) and,  258 SQL Server Nat ve Access C ent AP ,  223 SQL Server Object Exp orer app y ng schema changes n,  6 7 convert ng databases from V sua Stud o Database Profess ona ed t on (DbPro),  7 creat ng a connected database project w th,  10 16 dr ft detect on n,  7 Federat on too s n,  611 mport ng data us ng copy/paste,  28 mport ng data us ng scr pts,  38 nte Sense behav or n,  14 master vs. app cat on database n,  11 push ng changes from Loca DB to ve database, 36 38 Se ect Target Schema d a og,  36 sett ng defau t database,  12 SQL Server Database Project type n,  7 8 SSMS s Object Exp orer vs.,  6 Stored Procedures node, ocat on of,  33 tab e des gner, use n connected database,  17 22 TVP types, d sp ay ocat on n,  47 SQL Server Para e Data Warehouse ed t on (PDW), 684 685 MPP and,  685 SQL Server Profi er aunch ng, 435 Index  761

SQL Server Relational Database Management System (RDBMS) SQL Server Profi er Continued mon tor ng database act v ty w th,  435 436 temp ates, sav ng sett ngs n,  436 SQL Server Re at ona Database Management System (RDBMS), 683 685 data marts/warehouses,  683 data warehouse app ances and,  684 685 Fast Track Data Warehouse,  684 SQL PDW,  684 685 star schema and,  684 SQL Server Report ng Serv ces (SSRS),  692 694 A ert ng feature,  693 PerformancePo nt dashboards and,  694 Report Bu der 3.0,  693 SSAS and,  692 SQL Server Stored Procedure temp ate,  135 SQL Server Surface Area Configurat on Too ,  209 Sq Transact on c ass (System.Data.Sq C ent namespace), 437 data access, use n,  450 Sq Tr gger attr bute (.NET funct ons),  149 Sq Tr ggerContext object EventData property,  150 Sq Type (System.Data.Sq Types namespace),  143 CLR funct ons, return va ues n,  143 Sq UserDefinedAggregate attr bute,  151 152 Format parameter,  151 Sq UserDefinedType attr bute UDTs and,  158 sq :var ab e funct on (SQL Server extens on of XQuery), 295 SSAS B Semant c Mode ,  689 SSAS Tabu ar mode B Semant c Mode Connect on fi e,  731 dep oyment, 730 731 D rectQuery feature,  729 730 D rectQuery mode and part t ons,  729 Part t on Manager d a og,  728 729 part t ons and,  728 729 Power V ew reports, creat ng,  732 734 ro e based secur ty n,  728 V sua Stud o and,  727 732 SSDT Connect on Propert es d a og box,  139 DACPACs, dep oy ng from,  601 DACs and,  601 F LESTREAM, ack of support n Loca DB,  356 Pub sh Database d a og box,  138 139 Pub sh funct on,  137 138 Pub sh profi es, sav ng,  139

762  Index

Resu ts As Gr d,  134 Resu ts As Text,  134 SQL Azure, connect ng to,  597 599 SQL Azure, dep oy ng to,  602 603 SQL Azure, m grat on to,  605 SQL CLR, enab ng n,  126 T SQL vs.,  126 127 updates to SQL Azure, dep oy ng,  604 SSDT SQL Server Database Project type (V sua Bas c), 127 130 automated dep oyment,  129 ent t es, add ng to,  128 SQL CLR code attr butes,  129 SS S Too box,  682 Transforms, 682 SSMS Assemb y Propert es d a og box,  162 CLR ent t es, exam n ng/manag ng n Object Exp orer, 162 168 DACPACs and,  600 601 dep oy ng assemb es w th,  140 Dep oy to SQL Azure feature,  603 Execute Procedure d a og box,  166 Export Data T er App cat on funct on,  603 New Assemb y d a og box,  140, 162 Resu ts To Gr d,  134 Resu ts To Text opt on,  134 SQL Azure, connect ng to,  597 599 SQL Azure, dep oy ng to,  603 SQL Azure, m grat ng to,  605 606 SQL Azure updates, dep oy ng w th,  604 SQL CLR, enab ng n,  126 test ng stored procedures n,  142 SSMS Object exp orer Scr pt object type As opt on,  166 SSMS Object Exp orer Federat on support n,  611 SSMS s Object Exp orer SQL Server Object Exp orer vs.,  6 SSRS SQL Azure Report ng (SAR) vs.,  617 STArea method (geography c ass),  391 392 star schema,  684 STATE opt on (SQL Server Aud t),  237 238 State property (Ent t es),  537 Stat st ca Semant c Search,  365 Semant c Language Stat st cs Database for,  365 STBuffer method (geometry c ass),  378 379 BufferW thCurves method vs.,  406 408 output of,  380



sys.server audit specification details (audit catalog view)

STCentro d method (geometry c ass),  380 381 STCurveN method (geometry c ass),  405 406 STCurveToL ne method,  409 410 STD fference method (geometry c ass),  384 387 STD mens on method (geometry c ass),  383 384 STD stance method (geography c ass),  400 STEnve ope method (geometry c ass),  380 381 STGeomFromText method (geometry c ass),  372 373 STGeomFromWKB method (geometry c ass),  373 374 ST ntersect on method (geometry c ass),  382 383 ST ntersects method (geometry c ass),  382 383 ST sVa d method (geometry c ass),  410 411 STLength method (geography c ass),  391 392 STNumCurves method (geometry c ass),  405 406 storage requ rements, find ng DATALENGTH funct on,  404 storage schema (EDM),  482 stored procedures ca ng w th Sq Command objects,  442 443 conceptua schema, mapp ng to,  490 creat ng, 32 33 CRUD operat ons wrapped by,  433 435 d rect SQL vs.,  493 D rect SQL vs.,  442 443 execut on context and,  220 221 mapp ng to typed DataSets,  465 474 Mode Browser, mapp ng n,  490 494 pr mary key va ue, m ts on return of,  495 propert es of mapped, sett ng,  467 runt me behav or of ORM mapped,  497 498 StoredProcedures c ass Sq Procedure attr bute,  141 Stored Procedures node ocat on of n SQL Server Object Exp orer,  33 Stored Procedure w th Reader property (Text boxes),  446 stream d (F eTab e co umn name),  357 path ocator vs., as permenant reference,  358 stream ng HTTP serv ce access to storage areas w th,  351 creat ng pages for,  349 creat ng w th Sq F eStream,  348 352 Stream ng prov der DataServ ceStreamProv der nterface,  512 Stream ng prov der (WCF Data Serv ces),  512 str ng funct ons,  107 109 Strong y Typed DataSet property (Textbox),  468 STSymD fference method (geometry c ass),  384 387 STUn on method (geometry c ass),  384 387 STxxxFromText method (geometry c ass),  372 373 STxxxFromWKB method (geometry c ass),  373 374

Subm t Agent Key button (C ent Sync Agent),  634 Subm t method (DataServ ce c ass) overr d ng, 560 SW TCHOFFSET funct on,  63 symmetr c key encrypt on,  222 Sync From The Hub sync d rect on (Data Sync),  621 Sync Group (Data Sync),  622, 623 624 arch tecture of,  623 configur ng schedu es/confl ct reso ut on,  636 confl ct reso ut on po cy,  624 creat ng, 631 641 dataset, defin ng,  636 638 dep oy ng, 638 641 nfin te oops, avo d ng,  626 m t ng datasets for, to ncrease performance,  625 manua synchron zat on,  623 nam ng, 632 on prem se database, add ng,  632 635 SQL Azure hub database, sett ng up,  635 synchron zat on w th,  623 624 Sync Loop,  626 Synchron zat on Confl ct (Data Sync term),  622 Sync Job sett ng frequency of,  636 Sync Job (SQL Azure) check ng status of,  639 Sync Loop (Data Sync term),  622, 626 Sync Now button (W ndows Azure Management Porta ), 639 Sync Schedu e (Data Sync term),  622 nterva s, m ts on,  623 Sync To The Hub sync d rect on (Data Sync),  621 sysadm n (fixed server ro ),  211 sys.database aud t spec ficat on deta s (aud t cata og v ew),  244 sys.database aud t spec ficat ons (aud t cata og v ew), 244 SYSDATET ME funct on,  62 SYSDATET MEOFFSET funct on,  62 sys.dm aud t act ons (aud t cata og v ew),  244 sys.dm aud t c ass type map (aud t cata og v ew),  244 sys.dm db unconta ned ent t es DMV and,  246 sys.dm exec descr be first resu t set for object funct on,  118, 122 sys.dm exec descr be first resu t set funct on,  118 119 sys.dm server aud t status (aud t cata og v ew),  244 sys.fn get aud t fi e funct on (SQL Server Aud t),  243 sys.server aud t spec ficat on deta s (aud t cata og v ew), 244 Index  763

sys.server audit specifications (audit catalog views) sys.server aud t spec ficat ons (aud t cata og v ews), 244 sys.server fi e aud ts (aud t cata og v ews),  244 sys.sp descr be first resu t set procedure,  118 sys.sp descr be undec ared parameters procedure,  118, 122 System.Data.DataSetExtens ons.d , 475 AsEnumerab e method,  475 F e d method,  475 System.Data.Ent ty assemb y,  506 System.Data. so at onLeve enumerat on (ADO.NET 2.0), 184 185 Chaos, 184 ReadComm tted, 185 ReadUncomm tted, 184 Repeatab eRead, 185 Ser a zab e, 185 Snapshot, 185 Unspec fied,  185 System.Data namespace,  436 System.Data.DataSetExtens ons.d and,  475 System.Data.Sq C ent namespace,  133, 436 d rect ng comp er to,  438 System.Data.Sq Types namespace Sq DbType enumerat on,  135 Sq Type, 143 System.Enterpr seServ ces namespace (.NET Framework), 190 Transact onOpt on attr bute,  190 System. O.F eStream F eTab e and,  359 System.Runt me.Ser a zat on (WCF assemb y),  550 System.Serv ceMode assemb y (WCF),  550 System.Serv ceMode .Doma nServ ces.C ent assemb y (WCF),  550 System.Serv ceMode .Doma nServ ces.C ent.Web assemb y (WCF),  550 System.Serv ceMode .Web.Extens ons assemb y (WCF), 550 System.Transact ons assemb y,  556 System.Transact ons namespace BEG N D STR BUTED TRANSACT ON statement (T SQL) vs.,  194 D str buted dent fier property,  194 En stment var ab e,  197 System.Transact ons namespace (.NET Framework), 191 198 Transact on Management AP ,  186, 191 192 System.Transact ons namespace (Transact onScope c ass), 341

764  Index

System.Transact ons.Prepar ngEn stment c ass,  198 byte[] Recovery nformat on method,  198 ForceRo Back(Except on) method,  198 ForceRo Back() method,  198 Prepared method,  198 System.Transact ons.Transact on.Current vs. Sq Transact on, 203 SYSUTCDATET ME funct on,  62

T Tab eAdapter Configurat on W zard,  465 468 Tab eAdapterManager ser a z ng updates w th,  471 Tab eAdapters (V sua Stud o),  463 465 DataAdapters vs.,  465 Tab eAdapter Configurat on W zard,  465 468 tab e des gner add ng co umns to tab es n,  25 connected database, usage,  17 22 fore gn key re at onsh p, estab sh ng,  26 offl ne database, usage,  25 27 renam ng co umns,  31 32 tab es mod fy ng n Object Expo orer,  17 22 tab e va ued funct ons (TVFs),  145 147, 256 xm .va ue method, bu d ng w th,  293 Tab e Va ued Parameters (TVPs),  46 57 bu k nserts/updates us ng,  49 51 m tat ons on,  57 pass ng co ect ons us ng,  54 57 pass ng us ng ADO.NET,  52 54 s mp fy ng data nsert on w th,  47 49 and tempdb,  46 Tabu ar mode (SSAS),  710 734 TCP c ents User Datagram Protoco (UDP),  213 Test Connect on button (C ent Sync Agent),  634 test ng/debugg ng Loca s w ndow, use n,  34 sett ng breakpo nts,  33 text box contro s mod fy ng propert es of,  441 textbox(es) Ent tyC ent property va ue,  505 L NQ to Ent t es Query (D rect SQL) property va ue, 488 L NQ to Ent t es Query (Stored Procedure) property va ue,  492

L NQ to Gener c DataSet property va ue,  474 L NQ to Strong y Typed DataSet property,  476 L NQ to Strong y Typed DataSet property va ue, 476 Many To Many Re at onsh ps property va ue,  501 Output Parameter property va ue,  448 Stored Procedure w th Reader property va ue,  446 Strong y Typed DataSet property va ue,  468 Update w th Exp c t Transact on property va ue, 451 Update w th mp c t Transact on property va ue, 453 text (XPath node funct on),  279 THROW statement,  109 113 CATCH b ock, use n,  110 111 FORMATMESSAGE funct on and,  112 RA SERROR vs.,  110, 111 113 TRY b ocks and,  110 113 usage, 110 111 t me data type,  58 T MEFROMPARTS funct on,  105 t me zone awareness SW TCHOFFSET funct on,  63 TODATET MEOFFSET funct on,  63 us ng datet meoffset data type,  59 60 T ps, Tr cks, and Adv ce from the SQL Server Query Opt m zat on Team (b og),  708 TLA (Three Letter Acronym),  548 TODATET MEOFFSET funct on,  63 TO F LE c ause (SQL Server Aud t),  238 239 ToStr ng method (h erarchy d c ass),  305 310 Parse and,  321 return va ue of,  306 TRANCOUNT funct on. See @@TRANCOUNT funct on transact on coord nator (TC),  187 COMT , 190 En stmentNot ficat on nterface and,  195 L ghtwe ght Transact on Manager (LTM),  187 Transact on Management AP (.NET Framework System.Transact ons namespace),  186 Transact on Management AP (System.Transact ons namespace), 191 transact on manager (TM),  186 187 Transact onOpt on attr bute (System. Enterpr seServ ces namespace),  190 D sab ed, 190 NotSupported, 190 Requ red, 190 Requ resNew, 190 Supported, 190

Transact SQL (T-SQL) Transact onOpt ons object (ADO.NET) so at onLeve property, sett ng,  185 Transact on property (Sq Command object),  450 455 transact ons, 169 206 AC D propert es,  170 172 autocomm t mode,  173 batch execut on env ronment,  176 179 batch ng updates w th,  450 455 batch scoped mode,  176 179 defined,  170 D rty read,  178 d str buted, 186 200 exp c t mode,  173 176 mp c t mode,  176 nter eaved, 176 179 so at on eve s,  179 186 oca support for, n SQL Server,  172 178 mark ng, 173 nam ng, 173 nonrepeatab e read,  178 phantom read,  178 SET MPL C T TRANSACT ONS T SQL statement, 176 SQL CLR and,  201 204 term no ogy for,  178 trac ng w th SQL Profi er,  435 Transact onScope Comp ete() statement,  198 199 RMs and,  198 Sq Connect on nstances and,  193 Transact onScope c ass as rep acement for Sq Transact on object,  450 nest ng b ocks,  454 455 System.Transact ons namespace,  341 Transact onScope object,  185 Transact onScope object (ADO.NET),  185 Comp ete method,  193 Transact onScope object (System.Transact ons assemb y), 556 Transact SQL (T SQL),  45 124 ana yt c funct ons,  98 103 bu k nserts/updates us ng TVPs,  49 51 convers on funct ons,  103 104 date/t me data types,  58 65 date/t me funct ons,  104 106 GROUP NG SETS operator,  83 93 NSERT OVER DML syntax,  76 83 m tat ons on TVPs n,  57 og ca funt ons,  106 107 mathemat ca funct ons,  109 123 Index  765

Transact-SQL (T-SQL) Transact SQL (T SQL) Continued MERGE statement,  65 76 metadata d scovery,  118 122 Orac e databases, access us ng,  53 54 OVER c ause,  93 97 pass ng co ect ons us ng TVPs,  54 57 pass ng TVPs us ng ADO.NET,  52 54 SEQUENCE object,  115 118 server s de pag ng,  113 115 SQL Server Common Language Runt me ntegrat on (SQL CLR),  45 str ng funct ons,  107 109 tab e rep cat on us ng MERGE statement,  70 Tab e Va ued Parameters (TVPs),  46 57 THROW statement,  109 113 WHEN MATCHED c ause,  68 69 WHEN NOT MATCHED BY SOURCE c ause,  71 73 WHEN NOT MATCHED BY TARGET c ause,  69 Transact SQL (T SQL) database users, creat ng n,  211 212 exp c t transact on mode and,  173 Transforms (SS S),  682 transparent data encrypt on (TDE),  229 234 cert ficates, back ng up,  232 233 creat ng keys/cert ficates for,  229 230 DMKs and,  229 enab ng, 231 232 encrypt on state method,  232 restor ng encrypted databases,  233 234 Transparent Data Encrypt on (TDE) F LESTREAM and,  355 Tr ggerAct on property (Tr ggerContext object),  149, 150 Tr ggerContext object Tr ggerAct on property,  149, 150 tr ggers us ng, vs MERGED output,  73 tr ggers (SQL CLR),  148 151 dep oy ng automat ca y,  149 DML tr ggers,  148 150 Trustworthy Comput ng n t at ve (M crosoft),  207 TRUSTWORTHY property (CLR assemb es),  161 TRY b ock and THROW statements,  110 113 TRY/CATCH construct and THROW statements,  109 110 TRY CONVERT funct on,  103 TRY PARSE funct on,  103 104 T SQL CREATE AGGREGATE statement,  155

766  Index

CREATE FUNCT ON statement,  144 Federated Tab es, creat ng n,  609 Federat on Members, sp tt ng and dropp ng,  610 Federat ons, creat ng n,  608 firewa ru es for SQL Azure, sett ng w th,  588 funct ons, poss b e return va ues for,  145 L NQ vs.,  472 473 sett ng secur ty eve n,  162 SQL Azure Management Porta , query ng n,  592 SSDT vs.,  126 127 W TH EXECUTE AS CALLER c ause,  144 T SQL code wrapp ng n transact ons,  201 204 T SQL tr ggers CLR stored procedures vs.,  148 TVFs return ng data n a part cu ar order,  147 TVPs ALTER TABLE...AS TYPE, ack of,  57 Common Tab e Express ons (CTEs),  46 Ent ty Framework, ack of support n,  57 type cast ng xm .va ue method and,  292 typed DataSet bu d ng, 462 465 TYPE opt on (FOR XML commands),  276 277 SELECT statements and,  276

U UDFs CREATE FUNCT ON statement (C ass L brary projects), 144 W TH EXECUTE AS CALLER c ause (C ass L brary projects), 144 UDTs byte s ze m ts on,  158 conceptua z ng, 160 dep oy ng, 160 Ser a zab e attr bute,  158 Sq UserDefinedType attr bute,  158 syntax for, n T SQL,  156 Us ng CLR ntegrat on n SQL Server 2005 (Rathakr shnan), 156 UN QUE constra nt xm (data type) and,  258 Un t Sphere SR D,  413 UNSAFE ASSEMBLY perm ss on (CLR assemb es),  161 Unsafe (CLR assemb y secur ty eve ),  161 Unspec fied (ADO.NET so at on eve ),  185

WCF custom services

UpdateCommand property (Sq DataAdapter c ass), 457 458 Update method (Sq DataAdapter c ass),  457 458 UpdateObject method (DataServ ceContext),  526 updates bu k, us ng TVPs,  49 51 UPDATE STAT ST CS (T SQL command),  708 UpdateText funct on use for updat ng XML,  257 Update w th Exp c t Transact on property (Textbox), 451 Update w th mp c t Transact on property (Textbox), 453 USE master statement,  184 User defined aggregates. See aggregates (SQL CLR) user defined funct ons (UDFs),  143 147 SQL Server Database Project C# temp ate,  143 user defined types (UDTs),  156 160 users benefits of seperat ng from schemas,  218 mod fy ng w th schemas.,  216 218 Us ng CLR ntegrat on n SQL Server 2005 (Rathakr shnan), 156 us ng statement ( D sposab e objects),  439

V varb nary(max) PathName method,  336 varb nary(max) data type,  324 empty str ng vs. nu va ue n,  343 F LESTREAM attr bute,  325 335 Vert Paq, 709 734 Vert Paq storage mode (SSAS),  688 V ewMode property (App c ass),  651, 661 v ews xm .va ue, bu d ng w th,  293 V rtua Mach ne (Compute resource),  641 v rtua mach nes a ocat ng add t ona processors for,  707 Batch execut on mode,  707 V rtua Pr vate Network [VPN],  509 V sua C# node WCF Serv ce Web Ro e,  644 V sua Stud o Add New Doma n Serv ce C ass d a og,  552 556 ASP.NET Empty Web App cat on temp ate,  512 Azure deve opement and,  642 643 Bus ness nte gence\Ana ys s Serv ces node,  727 C ass L brary project,  139 140

c ent prox es, automat c generat on of for WCF R A Serv ces,  565 C oud project temp ate,  642 CLR ent t es, exam n ng/manag ng n SQL Server Object Exp orer,  162 168 D rectQuery Mode property,  729 Doma n Serv ce C ass temp ate,  552 EDM des gn surface,  482 Ent ty Data Mode W zard,  483 events, hook ng up to hand ers,  535 536 Generated Code fo der, v ew ng,  565 generat ng XML n, for ORM,  479 Mapp ng Deta s pane,  500 Mode Browser,  486 487 Perm ss on Leve combo box,  161 Query Mode sett ng (Property Pages d a og),  730 Report ng Serv ces project too ng,  615 S ver ght App cat on temp ate,  549 551 SQL Server, ntegrat on w th,  126 130 SSDT SQL Server Database Project type,  127 130 Tab eAdapter Configurat on W zard,  465 468 Tab eAdapters, 463 465 text box contro s, mod fy ng propert es of,  441 typed DataSets, bu d ng n,  462 465 V sua Stud o Deve opment Web Server (Cass n ), 513 WCF Data Serv ce, bu d ng,  512 515 WCF Serv ce Web Ro e (V sua C# node),  644 Web temp ate (C# node),  646 W ndows Azure Project type,  643 644 W ndows Azure Too s for V sua Stud o,  642 643 V sua Stud o Database Profess ona ed t on (DbPro) convert ng databases to SQL Server Object Exp orer, 7 SQL Server Data Too s (SSDT) vs.,  3 V sua Stud o Deve opment Web Server (Cass n ),  513 V sua Stud o SQL Server Database Projects UDFs, dependenc es for,  143

W WCF configurat ons sett ng up,  509 510 WCF custom serv ces,  544 546 creat ng, 544 raw cod ng for,  510 regenerat ng prox es after mod fy ng,  545 WCF Data Serv ces c ent proxy generator too and, 545 Index  767

WCF Data Access WCF Data Access,  509 578 opt ons, 510 WCF Data Serv ces,  511 548 WCF R A Serv ces,  548 577 WCF Data Serv ces $fi ter opt on,  518 asynchronous vs. synchronous ca s n,  525 authent cat on support n,  548 batch updates, runt me behav or of,  539 543 BLOBs and,  348 bu d ng, 512 513 c ent app cat ons, bu d ng,  518 543 c ent s de app cat ons and Ent ty Framework,  526 Content D header, batch requests and,  541 custom serv ce operat ons, creat ng,  544 546 data b nd ng and,  536 data entry c ent, bu d ng,  530 543 DataServ ceContext, 523 DataServ ce c ass (EDM),  515 data type, convey ng n AtomPub,  518 Ent ty Data Mode , creat ng for,  513 515 Ent ty Framework and,  500, 511 512 Ent ty Framework, w thout,  512 expand opt on,  518 extend ng, 544 548 h erarch ca updates n,  538 host ng n W ndows Azure,  641 662 n t a zeServ ce method (EDM),  515 nsert ng new ent t es n,  528 530 nterceptors, wr t ng,  547 548 ayers of abstract on n,  524 L NQ to REST,  518 M crosoft ADO.NET Data Serv ces,  527 mu t user confl cts, check ng for,  528 OData, 511 overr d ng serv ce methods,  546 547 Reflect on prov der,  512 REST and,  511 SQL CE databases, runn ng aga nst oca ,  671 Stream ng prov der,  512 test ng w th nternet Exp orer,  515 518 T SQL quer es and,  525 WCF R A serv ces vs.,  548 WCF R A Serv ces vs.,  577 578 W ndows Phone 7.1 SKD, c ent brary n,  652 WCF Data Serv ces c ent proxy generator too custom serv ces and,  545 WCF R A Serv ces,  548 577 asynchronous ca s, work ng around for W ndows Phone 7 and S ver ght,  566 568

768  Index

automat c va dat on of data n,  569 c ent doma n serv ces synchron zat on w th,  550 Compos t on attr bute (metadata c asses),  558 doma n serv ce, bu d ng,  552 556 doma n serv ce c ass,  559 560 EDM, bu d ng so ut on to recogn ze,  552 Enab eC entAccess attr bute,  559 Ent ty Data Mode , creat ng,  551 552 Ent ty Framework and,  500 estab sh ng nks w th,  549 551 expos ng POCOs w th Doma nServ ce c ass,  559 nc ude attr bute (metadata c asses),  558 nsta ng on V sua Stud o,  549 L nqToEnt t esDoma nServ ce, 559 L nqToSq Doma nServ ce, 559 L NQ to SQL, expos ng to,  559 metadata c asses,  557 559 metadata c asses, bu d ng,  552 556 MetadataType attr bute,  557 .NET Fram ng protoco , nspect ng w th F dd er,  569 ObjectContext property (EDM),  559 proxy code, exam nat on of,  566 Roundtr pO g na attr bute (metadata c asses), 558 Samp eR aContext c ass,  566 S ver ght and,  549 S ver ght App cat on temp ate,  549 551 S ver ght c ent, bu d ng,  561 568 System.Runt me.Ser a zat on assemb y,  550 System.Serv ceMode assemb y,  550 System.Serv ceMode .Doma nServ ces.C ent assemb y, 550 System.Serv ceMode .Doma nServ ces.C ent.Web assemb y, 550 System.Serv ceMode .eb.Extens ons assemb y,  550 test ng comp ete so ut ons,  569 577 WCF Data Serv ces vs.,  548, 577 578 WCF Serv ce Web Ro e (C# node, V sua Stud o),  644 Web (Compute resource),  641 Web temp ate (C# node, V sua Stud o),  646 We Known B nary (WKB),  373 374 STGeomFromWKB, mport ng shapes w th,  373 374 STxxxFromWKB, mport ng shapes w th va dat on, 373 374 We Known Text (WKT) vs.,  373 We Known Text (WKT),  370 373 C RCULARSTR NG keyword,  370 COMPOUNDCURVE keyword,  370 371 CURVEPOLYGON keyword,  370 371 FULLGLOBE keyword,  370 371

GEOMETRYCOLLECT ON keyword,  371 L NESTR NG keyword,  370 MULT L NESTR NG keyword,  371 MULT PO NT keyword,  371 MULT POLYGON keyword,  371 PO NT keyword,  370 371 POLYGON keyword,  370 371 STGeomFromText, mport ng shapes w th,  372 373 STxxxFromText, mport ng shapes w th va dat on, 372 373 We Known B nary (WKB) vs.,  373 WHEN MATCHED c ause,  68 69 m ts on,  68 69 WHEN NOT MATCHED BY SOURCE c ause and MERGE statement,  71 73 m ts on,  71 WHEN NOT MATCHED BY TARGET c ause,  69 m ts on,  69 WHERE c ause ROW NUMBER funct on and,  113 xm .ex st method and,  291 where (XQuery keyword),  286 w ndow ng. See OVER c ause W ndows Azure best pract ces for,  642 c ents, creat ng for W ndows Phone 7,  647 666 Compute capac ty,  641 Compute resources,  641 data serv ce, add ng,  643 644 dep oy ng, 672 674 En ty Data Mode , add ng,  644 647 so ut ons, creat ng n,  642 643 WCF Data Serv ces, host ng n,  641 662 WCF Serv ce Web Ro e (V sua C# node),  644 W ndows Azure Project type,  643 644 W ndows Azure Too s for V sua Stud o,  642 643 W ndows Azure Management Porta ,  586 588 Add F rewa Ru e d a og,  645 authent cat on and,  625 DACs and,  601 Export feature,  606 manua synchron zat on of Sync Groups n,  623 m grat ng/synch ng w th,  601 602 S ver ght and,  586 Sync Between On Prem se And SQL Azure Databases opt on,  631 Sync Between SQL Azure Databases opt on,  631 Sync Now button,  639 WCF Data Serv ces, dep oy ng W ndows Phone app cat on w th,  673

WITH CUBE operator W ndows Azure Project type (V sua Stud o),  643 644 W ndows Azure Too s for V sua Stud o,  642 643 W ndows Commun cat ons Foundat on (WCF) serv ce ASP.NET vs.,  348 Message Transm ss on Opt m zat on Mechan sm (MTOM), 348 WCF Data Serv ces,  348 W ndows Exp orer and F eTab e,  359 W ndows Forms App cat on temp ate (So ut on Exp orer ),  519 W ndows Forms user nterface,  530 W ndows L ve D SQL Azure requ rement for,  584 W ndows Phone 7,  518, 619 674 App c ass, mod fy ng,  661 662 App cat onBar contro , mod fy ng,  649 App cat on c ass,  661 C oudSync Comp eted method,  665 creat ng app cat ons,  647 666 databases, creat ng for OCS,  628 629 L NQ to REST,  518 L NQ to SQL and,  482, 667 Mode V ew V ewMode (MVVM) pattern,  651 661 occas ona y connected systems and,  620 OData, consum ng,  662 674 secur ng oca data on,  667 SQL Azure Data Sync (Data Sync),  621 626 SQL CE database, creat ng,  667 671 SQL CE databases, embed ng as resource n,  666 SQL server on,  666 672 storage capac ty on,  666 synchronous quer es and,  663 v ew, creat ng,  647 651 WCF Data Serv ces and,  662 674 W ndows Phone Emu ator,  671 W ndows Phone 7.1 SDK ocat on of and nsta ng,  647 S ver ght For W ndows Phone temp ate (V sua Stud o), 647 S verL ght Too k t,  648 WCF Data Serv ces c ent brary,  652 W ndows Phone Databound App cat on project (V sua Stud o),  647 W ndows Phone Databound App cat on project (V sua Stud o),  647 W ndows Phone Emu ator,  671 W ndows Presentat on Foundat on (WPF),  518 W TH CUBE operator,  86 88 Index  769

WITH EXECUTE AS CALLER clause (Class Library project) W TH EXECUTE AS CALLER c ause (C ass L brary project), 144 W TH EXECUTE AS CALLER (W TH EXECUTE AS c ause), 221 W TH EXECUTE AS c auses,  220 W TH EXECUTE AS OWNER (W TH EXECUTE AS c ause), 221 W TH EXECUTE AS SELF (W TH EXECUTE AS c ause), 221 W TH EXECUTE AS (W TH EXECUTE AS c ause), 221 W TH PERM SS ON SET c ause (C ass L brary project), 139 W TH ROLLUP operator,  85 86 Worker (Compute resource),  641 WPF c ent app cat ons Med aE ement contro ,  354 port numbers, sett ng for,  352 Wr te method (h erarchy d c ass),  321 322

X XML, 255 298 character data as,  256 257 des gn cons derat ons for, n re at ona database, 255 256 FOR XML commands,  268 280 h erarchy d data type vs.,  299 ndexes, 266 268 schema defin t ons (XSDs),  259 266 shredd ng, us ng OPENXML,  284 285 stored procedures and,  256 UpdateText funct on to update,  257 xm (data type),  257 268 XML DML,  296 297 XQuery and,  285 297 xm (data type),  257 268 COLLATE keyword and,  258, 259 GROUP BY and,  257 ORDER BY and,  257 SQL Server Deve oper Too s (SSDT) and,  258 SQL Server Management Stud o (SSMS) and,  258 UN QUE constra nt and,  258 work ng w th, as var ab e,  257 258 work ng w th, n tab es,  258 259 XML schema defin t ons (XSDs),  259 266 XML DML,  296 297 xm .mod fy(de ete) method,  297 xm .mod fy( nsert) method,  297 xm .mod fy(rep ace), 297

770  Index

xm .ex st method,  291 292 WHERE c ause and,  291 XML ndexes,  266 creat ng, 266 PROPERTY keyword of,  267 restr ct ons on,  268 US NG XML NDEX syntax for,  268 xm .mod fy(de ete) method,  297 xm .mod fy( nsert) method,  297 xm .mod fy(rep ace) method,  297 xm .query method,  293 296 sq :co umn and,  293 sq :var ab e funct on,  295 XML schema defin t ons (XSDs),  259 266 CHECK constra nt and,  261 data ntegr ty, ensur ng w th,  262 data typ ng data objects w th,  260 263 n ne, produc ng w th FOR XML commands,  281 282 ax va dat on and,  263 264 st types and,  262 spec ficat ons, source for,  260 SQL Server schema co ect ons and,  260 263 xsd:un on, 264 266 XML Schema Defin t on (XSD),  463 XML Schema Defin t on (XSD) fi es v ew ng, 468 XMLSCHEMA keyword (FOR XML syntax),  281 xm .va ue method,  292 293 TVFs, bu d ng w th,  293 XPath comment (node funct on),  279 data (node funct on),  279 @ (flag),  277 FOR XML PATH and,  277 280 ocat on of standards for,  286 node (node funct on),  279 process ng nstruct on (node funct on),  279 text (node funct on),  279 XQuery vs.,  286 XQuery, 285 297 express ons, 285 288 FLWOR express ons,  286 288 for (keyword),  286 et (keyword),  286 ocat on of spec ficat on,  285 SQL Server and,  288 296 where (keyword),  286 XML DML,  296 297 xm .ex st method,  291 292

Zhou, Joe

xm .query method,  293 296 xm .va ue method,  292 293 XPath express ons,  286 XPath vs.,  286 xVe oc ty n memory ana yt cs eng ne. See Vert Paq vocabu ary for,  710 716 xVe oc ty techno og es,  701 736 Bus ness nte gence Semant c Mode (B SM),  702



co umnstore ndexes,  704 709 Data Ana ys s eXpress ons (DAX),  702 PowerP vot, 709 734 SASS and,  709 734

Z Zhou, Joe,  569

Index  771

About the Authors Leonard Lobel s a M crosoft MVP n SQL Server and a Pr nc pa Consu tant at Ta an, Inc , a M crosoft Nat ona Systems Integrator and Go d ­Competency Partner W th over 30 years of exper ence, Lenn s one of the ndustry’s ­ead ng NET and SQL Server experts, hav ng consu ted for Ta an’s c ents n a var ety of doma ns, nc ud ng pub sh ng, financ a serv ces, reta , hea th care, and e-commerce Lenn has served as ch ef arch tect and ead deve oper on arge sca e projects, as we as adv sor to many h gh-profi e c ents

About Tallan Ta an (http://www.tallan.com) s a, nat ona techno ogy consu t ng firm that prov des web deve opment, bus ness nte gence, customer ­re at onsh p management, custom deve opment, and ntegrat on ­serv ces to customers n the financ a serv ces, hea th care, government, reta , educat on, and manufactur ng ndustr es Ta an s one of 40 M crosoft Nat ona Systems Integrators (NSI) n the Un ted States, and a member of M crosoft’s Bus ness Inte gence Partner Adv sory Counc For more than 25 years, Ta an’s hands-on, ­co aborat ve approach has enab ed ts c ents to obta n rea cost and t me sav ngs, ncrease revenues, and generate compet t ve advantage Lenn s a so ch ef techno ogy officer (CTO) and cofounder of S eek ­ echno og es, Inc , a New York-based deve opment shop w th an ear y adopter T ph osophy toward new techno og es He s a sought after and h gh y rated speaker at ndustry ­conferences such as V sua Stud o L ve!, SQL PASS, SQL B ts, and oca techno ogy user group ­meet ngs He s a so ead author of th s book’s prev ous ed t on, Programming ­Microsoft SQL Server 2008 Lenn can be reached at [email protected] or [email protected] Andrew J. Brust s Founder and CEO of B ue Badge Ins ghts (http://www.bluebadgeinsights.com), an ana ys s, strategy and adv sory firm serv ng M crosoft customers and partners Brust pens ZDNet’s “B g on Data” b og (http://bit.ly/bigondata); s a M crosoft Reg ona D rector and MVP; an adv sor to the New York Techno ogy Counc ; Co-Cha r of V sua Stud o L ve!;

a frequent speaker at ndustry events and a co umn st for V sua Stud o Magaz ne He has been a part c pant n the M crosoft ecosystem for 20 years; worked c ose y w th both M crosoft s Redmond-based corporate team and ts fie d organ zat on for the ast 10; has served on M crosoft’s Bus ness Inte gence Partner Adv sory Counc ; and s a member of severa M crosoft " ns ders" groups that supp y h m w th ns ght around mportant techno og es out of Redmond

Paul Delcogliano s a techno ogy d rector at Broadr dge F nanc a Serv ces, Inc Pau has been work ng w th the M crosoft NET Framework s nce ts first pub c ­ntroduct on and has been deve op ng M crosoft SQL Server app cat ons even onger He bu ds systems for a d verse range of p atforms nc ud ng M crosoft W ndows, the Internet, and mob e dev ces Pau has authored many art c es and co umns for var ous trade pub cat ons on a var ety of top cs He can be reached by ema at [email protected] Pau wou d ke to thank h s fam y for the r pat ence and understand ng wh e he was frant ca y try ng to meet h s dead nes He wou d a so ke to thank Lenn for offer ng h m another opportun ty to contr bute to the book The second t me around was better than the first