Enterprise SOA. Service-Oriented Architecture Best Practices [9 ed.] 9780131465756, 0131465759

Citation preview

Enterprise SOA: Service-Oriented Architecture Best Practices By Dirk Krafzig, Karl Banke, Dirk Slama Publisher: Prentice Hall PTR Pub Date: November 09, 2004 ISBN: 0-13-146575-9 Pages: 408 Table of • Contents • Index

Ent erprise SOA present s a com plet e roadm ap for leveraging t he principles of Service- Orient ed Archit ect ures t o reduce cost and risk, im prove efficiency and agilit y, and liberat e your organizat ion from t he vagaries of changing t echnology. Benefit from t he lessons of four ent erprise- level SOA case st udies from Credit Suisse, Halifax Bank of Scot land, and ot her world- class ent erprises Make your business t echnology independent and m anage infrast ruct ure het erogeneit y by focusing on archit ect ure, not specific im plem ent at ion t echniques Recognize t he t echnical and nont echnical success fact ors for SOA in t he ent erprise Define and com m unicat e t he econom ic value proposit ion of an SOA Apply pragm at ic design principles t o solve t he problem s of dat a and process int egrit y in an SOA environm ent Whet her you're a m anager, archit ect , analyst , or developer, if you m ust drive great er value from I T services, Ent erprise SOA will show you howfrom st art t o finish.

Enterprise SOA: Service-Oriented Architecture Best Practices By Dirk Krafzig, Karl Banke, Dirk Slama Publisher: Prentice Hall PTR Pub Date: November 09, 2004 ISBN: 0-13-146575-9 Pages: 408 Table of • Contents • Index

Copyright Praise for Enterprise SOA The Coad Series Acknowledgments About the Authors Dirk Krafzig Karl Banke Dirk Slama Foreword Reader's Guide Who Should Read This Book A Roadmap for This Book Chapter 1. An Enterprise IT Renovation Roadmap Section 1.1. Agony Versus Agility Section 1.2. Enterprise Software Is a Different Animal Section 1.3. The Importance of Enterprise Software Architectures Section 1.4. The Requirements for an Enterprise Software Architecture Section 1.5. The Relation of Enterprise Architecture and Enterprise Standards Section 1.6. Organizational Aspects Section 1.7. Lifelong Learning Section 1.8. The Enterprise IT Renovation Roadmap Chapter 2. Evolution of the Service Concept Section 2.1. Milestones of Enterprise Computing Section 2.2. Programming Paradigms Section 2.3. Distributed Computing Section 2.4. Business Computing Section 2.5. Conclusion Chapter 3. Inventory of Distributed Computing Concepts Section 3.1. Heterogeneity of Communication Mechanisms Section 3.2. Communication Middleware Section 3.3. Synchrony Section 3.4. Interface Versus Payload Semantics Section 3.5. Tight Versus Loose Coupling Section 3.6. Conclusion Part I. Architectural Roadmap

Chapter 4. Service-Oriented Architectures Section 4.1. What Is a Software Architecture? Section 4.2. What Is a Service-Oriented Architecture? Section 4.3. Elements of a Service-Oriented Architecture Section 4.4. Conclusion Chapter 5. Services as Building Blocks Section 5.1. Service Types Section 5.2. Layers on the Enterprise Level Section 5.3. Conclusion Chapter 6. The Architectural Roadmap Section 6.1. The Architectural Roadmap Section 6.2. Fundamental SOA Section 6.3. Networked SOA Section 6.4. Process-Enabled SOA Section 6.5. Conclusion Chapter 7. SOA and Business Process Management Section 7.1. Introduction to BPM Section 7.2. BPM and the Process-Enabled SOA Section 7.3. Conclusion Chapter 8. Managing Process Integrity Section 8.1. Data Versus Process Integrity Section 8.2. Technical Concepts and Solutions Section 8.3. Recommendations for SOA Architects Section 8.4. Conclusion Chapter 9. Infrastructure of the Service Bus Section 9.1. Software Buses and the Service Bus Section 9.2. Logging and Auditing Section 9.3. Availability and Scalability Section 9.4. Securing SOAs Section 9.5. Conclusion Chapter 10. SOA in Action Section 10.1. Building Web Applications Section 10.2. Enterprise Application Integration Section 10.3. Business-to-Business Section 10.4. Fat Clients Section 10.5. Designing for Small Devices Section 10.6. Multi-Channel Applications Section 10.7. Conclusion Part II. Organizational Roadmap Chapter 11. Motivation and Benefits Section 11.1. The Enterprise Perspective Section 11.2. The Personal Perspective Section 11.3. Conclusion Chapter 12. The Organizational SOA Roadmap Section 12.1. Stakeholders and Potential Conflicts of Interest Section 12.2. The Organizational SOA Roadmap Section 12.3. Four Pillars for Success Section 12.4. An Ideal World Section 12.5. The Real WorldOrganization-Wide Standards Section 12.6. Recommendations for the SOA Protagonist

Section 12.7. Conclusion Chapter 13. SOA-Driven Project Management Section 13.1. Established Project Management Methodologies Section 13.2. SOA-Driven Project Management Section 13.3. Configuration Management Section 13.4. Testing Section 13.5. Conclusion Part III. Real-World Experience Chapter 14. Deutsche Post AG Case Study Section 14.1. Project Scope Section 14.2. Implementation Section 14.3. Technology Section 14.4. Lessons Learned, Benefits, and Perspectives Chapter 15. Winterthur Case Study Section 15.1. Project Scope Section 15.2. Implementation Section 15.3. Technology Section 15.4. Lessons Learned, Benefits, and Perspectives Chapter 16. Credit Suisse Case Study Section 16.1. Project Scope Section 16.2. Implementation Section 16.3. Technology Section 16.4. Lessons Learned, Benefits, and Perspectives Chapter 17. Halifax Bank Of Scotland: IF.com Section 17.1. Project Scope Section 17.2. Implementation Section 17.3. Technology Section 17.4. Lessons Learned, Benefits, and Perspectives Index

Copyright The aut hors and publisher have t aken care in t he preparat ion of t his book, but m ake no expressed or im plied warrant y of any kind and assum e no responsibilit y for errors or om issions. No liabilit y is assum ed for incident al or consequent ial dam ages in connect ion wit h or arising out of t he use of t he inform at ion or program s cont ained herein. Publisher: John Wait Edit or in Chief: Don O'Hagan Execut ive Edit or: Paul Pet ralia Edit orial Assist ant : Michelle Vincent i Market ing Manager: Chris Guzikowski Cover Designer: Jerry Vot t a Managing Edit or: Gina Kanouse Proj ect Edit or: Christ y Hackerd Copy Edit or: Benj am in Lawson I ndexer: Lisa St um pf Com posit or: Mary Sudul Manufact uring Buyer: Dan Uhrig The publisher offers excellent discount s on t his book when ordered in quant it y for bulk purchases or special sales, which m ay include elect ronic versions and/ or cust om covers and cont ent part icular t o your business, t raining goals, m arket ing focus, and branding int erest s. For m ore inform at ion, please cont act : U. S. Corporat e and Governm ent Sales ( 800) 382- 3419 corpsales@pearsont echgroup.com For sales out side t he U. S., please cont act : I nt ernat ional Sales int ernat [email protected] Visit us on t he Web: www.phpt r.com Library of Congress Cat aloging- in- Publicat ion Dat a: 2004110321 Copyright © 2005 Pearson Educat ion, I nc.

All right s reserved. Print ed in t he Unit ed St at es of Am erica. This publicat ion is prot ect ed by copyright , and perm ission m ust be obt ained from t he publisher prior t o any prohibit ed reproduct ion, st orage in a ret rieval syst em , or t ransm ission in any form or by any m eans, elect ronic, m echanical, phot ocopying, recording, or likewise. For inform at ion regarding perm issions, writ e t o: Pearson Educat ion, I nc. Right s and Cont ract s Depart m ent One Lake St reet Upper Saddle River, NJ 07458 Text print ed in t he Unit ed St at es on recycled paper at Phoenix BookTech in Hagerst own, Maryland First print ing, Novem ber, 2004

Dedication I dedicat e t his book t o m y fam ilym y wife Pet ra, m y kids David and Johanna, and m y parent s. Wit hout t hem , t his book would not have been possible. Dirk Krafzig I dedicat e t his book t o all who helped m e along t he road wit h t heir pat ience, dedicat ion, fait h, and inspirat ion: m y t eachers, m y colleagues, m y cust om ers, m y part ners at it ernum , m y parent s, and m y beloved Sim a. Karl Banke For Olli, who decided t o writ e screenplays inst ead. Dirk Slam a

Praise for Enterprise SOA " As we st art ed wit h t he developm ent of t he e- Plat form at t he Wint ert hur, we knew t hat t here were st ill m any quest ions t o be answered. Today, we can look back at a process, which has creat ed t he corresponding archit ect ural guidelines, processes, and infrast ruct ure com ponent s. I n t he m eant im e, we are reaping t he benefit s of our st rat egy and are t ransferring, st ep by st ep, our t radit ional applicat ion landscape int o a loosely coupled SOA. This form s, as well, t he basis for our next st ep in t he direct ion of Business Process Managem ent . This book clearly describes t he m any concept s t hat we painst akingly developed at t hat t im e and answers t he m ost im port ant quest ions t hat are encount ered on t he way t o an adapt able applicat ion landscape for large- scale ent erprises. From m y point of view, t his is a book t hat should be read by all t hose who are considering rem odeling t heir applicat ion landscape." Daniele Liset t o, Head Technical and Applicat ion Plat form s, Wint ert hur Group " Ent erprise SOA provides st rat egies t hat help large ent erprises t o increase t he agilit y of t heir I T syst em sone of t he m ost pressing issues of cont em porary I T. Covering bot h a business and archit ect ural view, t hese st rat egies aim t o prom ot e t he im plem ent at ion of an I T infrast ruct ure t hat can serve as a base for t he developm ent of t ruly flexible business processes. This book covers it s subj ect wit h great profoundness based on real- world evidence. I t is in t he int erest of everybody involved wit h soft ware archit ect urepart icularly for anybody who int ends t o est ablish a Service- Orient ed Archit ect uret o read t his book." Dr. Helge Heß, Direct or Business Process Managem ent , I DS Scheer AG " The SOA principles described in t his book are t he foundat ion on which ent erprises can build an I T archit ect ure t hat will sat isfy t oday's m ost im port ant I T requirem ent sagilit y and flexibilit yat affordable cost s." Mart in Frick, Head of I T, Wint ert hur Group " By delivering SAP's next - generat ion applicat ions based on a Service- Orient ed Archit ect ure, SAP is at t he forefront of m aking Web services work for t he ent erprise. The Ent erprise Services Archit ect ure enables unprecedent ed flexibilit y in business process deploym ent , allowing com panies t o execut e and innovat e end- t o- end processes across depart m ent s and com panies, wit h m inim um disrupt ion t o ot her syst em s and exist ing I T invest m ent s. This st rat egy com es t o life wit h SAP Net Weaver, which is t he t echnological foundat ion of t he Ent erprise Services Archit ect ure. I t provides easy int egrat ion of people, inform at ion, and syst em s in het erogeneous I T environm ent s and provides a fut ure proof applicat ion plat form . Ent erprise SOA provides readers wit h t he archit ect ural blueprint s and SOA- driven proj ect m anagem ent st rat egies t hat are required t o successfully adopt SOA on an ent erprise level." Dr. Pet er Graf, SVP Product Market ing, SAP The SOA principles out lined in t his book enable ent erprises t o leverage robust and proven m iddleware plat form s, including CORBA, t o build flexible and business- orient ed service archit ect ures. The aut hors also clearly describe t he right st rat egies for using Model Driven Archit ect ure ( MDA) t o m anage SOA Service Reposit ories in a plat form - independent way, enabling ent erprises t o bet t er address t he problem of het erogeneit y at m any levels. The Obj ect Managem ent Group was creat ed j ust t o address t his cent ral problem of int egrat ion in

t he face of const ant ly changing het erogeneit y and plat form churn, so I st rongly recom m end t his book for t he bookshelf of every ent erprise archit ect and developer. Richard Mark Soley, Ph.D. Chairm an and Chief Execut ive Officer, Obj ect Managem ent Group, I nc.

The Coad Series Pet er Coad, Series Edit or David J. Anderson Agile Managem ent for Soft ware Engineering: Applying t he Theory of Const raint s for Business Result s David Ast els Test Driven Developm ent : A Pract ical Guide David Ast els, Granville Miller, Miroslav Novak A Pract ical Guide t o eXt rem e Program m ing Andy Carm ichael, Dan Haywood Bet t er Soft ware Fast er Donald Kranz, Ronald J. Norm an A Pract ical Guide t o Agile Unified Process Jam es McGovern, Scot t W. Am bler, Michael E. St evens, Jam es Linn, Vikas Sharan, Elias Jo A Pract ical Guide t o Ent erprise Archit ect ure Jill Nicola, Mark Mayfield, Michael Abney St ream lined Obj ect Modeling: Pat t erns, Rules, and I m plem ent at ion St ephen R. Palm er, John M. Felsing A Pract ical Guide t o Feat ure- Driven Developm ent

Acknowledgments Many people have cont ribut ed t o t his book in m any ways, bot h direct ly and indirect ly over a period of t wo years. I t is t herefore im possible t o list everybody who influenced, support ed, or aided us on our j ourney from t he init ial idea t o t he final m anuscript . Nevert heless, we owe our grat it ude t o at least t he following people: We would like t o t hank our publisher, Pearson Educat ion, for t he support t hat led t o t his book being com plet ed. I n part icular, we would like t o t hank Paul Pet ralia, our edit or, and Christ y Hackerd. The case st udies in t his book would not have been possible wit hout ext ensive cooperat ion wit h SOA prot agonist s from different organizat ions. We would like t o t hank Uwe Bat h ( Deut sche Post ) , Fiorenzo Malet t a ( Wint ert hur) , Claus Hagen ( Credit Suisse) , and Willie Nisbet and Allan Kelly ( Halifax Bank of Scot land) . This book would also not have been possible wit hout count less discussions and crit ical feedback of colleagues, friends, and reviewers. We ext end our grat it ude t o Beat Aeschlim ann, Arne Allee, Michael Aschenbrenner, Arnaud Blandin, Ndom e Cacut alua, Frank Eversz, Diet m ar Grubert , St efan Havenst ein, Georgia and Paul Hickey, St efan Krahm , Jam es McGovern, Dirk Marwinski, Manfred Mayer, St eve Morris, I ngrid Müller, Joanis Papanagnu, Arnold Pot t , Kai Rannenberg, Kirst en Scharf, Uli St eck, Paul St em m et , Michael St evens, Harald St örrle, Pet er St racke, Josef Wagenhuber, and Frank Werres. Sebast ian- Svant e Wellershoff cont ribut ed t o t he design of our art work. Birgit Anders support ed t he aut hor- t eam by proofreading, form at t ing, drawing figures, and m any ot her act ivit ies t hat were necessary t o produce a m anuscript for t his book. We all appreciat ed t he accurat e and swift m anner in which she worked. Mark Roant ree was our proofreader for linguist ic and som e of t he t echnical aspect s of t his book. Roland Trit sch and Joachim Quant z have bot h closely worked wit h t he aut hor t eam and developed m any of t he ideas t hat were present ed in t his book. They were like a virt ual ext ension of t he aut hor t eam . Last but not least , we owe our special grat it ude t o our part ners, fam ilies, and friends for t heir pat ience, t im e, support , and encouragem ent . We know t hat we cannot m ake it up t o you.

About the Authors Dirk Krafzig Karl Banke Dirk Slam a

Dirk Krafzig Dirk has been dealing wit h t he challenges of ent erprise I T and dist ribut ed soft ware archit ect ures t hroughout his ent ire working life. He devot ed him self t o SOA in 2001 when he j oined Shinka Technologies, a st art - up com pany and plat form vendor in t he early days of XML- based Web services. Since t hen, Dirk has acquired a rich set of real world experience wit h t his upcom ing new paradigm bot h from t he view point of a plat form vendor and from t he perspect ive of soft ware proj ect s in different indust ry vert icals. Writ ing t his book was an issue of personal concern t o him as it provided t he opport unit y t o share his experiences and m any insight s int o t he nat ure of ent erprise I T wit h his readers. Today, Dirk is designing ent erprise applicat ions and m anaging proj ect s, applying t he guiding principles out lined in t his book. Dirk has a Ph.D. in Nat ural Science and an MSc in Com put er Science. He lives in Düsseldorf, Germ any, and is 39 years old, m arried, and t he fat her of t wo children.

Karl Banke Soft ware archit ect ure has been wit h Karl since he program m ed his first TRON- like gam e on t he t hen st at e- of- t he art ZX81 in t he early 1980s. Aft er graduat ing as a Mast er of Physics, he gained his com m ercial experience in various consult ing assignm ent s, m ost ly in t he financial and t elecom m unicat ions sect or. He m oved t hrough st ages of consult ant , t echnical lead, soft ware archit ect , and proj ect m anager using a variet y of obj ect - orient ed t echnologies, program m ing languages, and dist ribut ed com put ing environm ent s. Soon realizing t hat he was t oo const rained as an em ployee in doing what he t hought necessary in soft ware developm ent , he co- founded t he com pany it ernum in 2000, where he current ly act s as a principal consult ant and general m anager. Karl perm anent ly lives in Mainz, Germ any when not t em porarily relocat ed by a current proj ect .

Dirk Slama Having spent t he last t en years at t he forefront of dist ribut ed com put ing t echnology, Dirk has developed an in- dept h underst anding of ent erprise soft ware archit ect ures and t heir applicat ion in a variet y of indust ry vert icals. Dirk was a senior consult ant wit h I ONA Technologies, working wit h Fort une 500 cust om ers in Europe, Am erica, and Asia on large- scale soft ware int egrat ion proj ect s. Aft er t his, Dirk set up his own com pany, Shinka Technologies, which successfully developed one of t he first XML- based Web services m iddleware product s, st art ing as early as 1999. Dirk holds an MSc in com put er sciences from TU- Berlin and an MBA from I MD in Lausanne. He is a co- aut hor of Ent erprise CORBA ( Prent ice Hall, 1999) , t he leading book on CORBA- based syst em archit ect ures. Dirk is current ly working as a solut ion archit ect for Com put er Sciences Corporat ion in Zurich, Swit zerland. Cont act : aut hors@ent erprise- soa.com

Foreword At t he t urn of t he ninet eent h cent ury, a wave of new t echnologies such as t he st eam engine, elect ricit y, t he loom , t he railway, and t he t elephone em erged. Urbanizat ion and t he m ass product ion of goods in large fact ories fundam ent ally changed how m ankind lived and worked t oget her. One hundred years lat er, t he indust rial revolut ion had not slowed down: At t he t urn of t he t went iet h cent ury, aut om at ion, specializat ion, and a never- ending spiral of efficiency im provem ent have result ed in m odern econom ies wit h unheard- of indust rial product ivit y. Aft er a phase of consolidat ion during t he t ransit ion from t he t went iet h t o t he t went y- first cent ury, globalizat ion and virt ualizat ion have now becom e t he key drivers of our econom ic lives. Wit hout a doubt , t hey will yet again change how we live and work t oget her. I f we t ake a closer look at t he past 20 years, we can observe t hat est ablished business rules have been const ant ly redefined. New business m odels em erged; sm all com panies quickly grew int o billion- dollar m ult inat ionals, aggressively at t acking ot her est ablished com panies. A wave of m ergers, acquisit ions, and buyout s changed t he overall indust rial landscape. I T has played a m aj or role in all of t his, be it t hrough cont rolling product ion processes and supply chains or by creat ing real- t im e links bet ween financial m arket s, t hus virt ually elim inat ing arbit rage opport unit ies by closing t he t im e gaps of t rading around t he globe. The I nt ernet boom and t he " virt ual ent erprise" are cornerst ones of t his ongoing developm ent . Ent irely new product s and services have been creat ed, which would have been unt hinkable wit hout t he support of m odern I T. Wit hout a doubt , t oday's m odern ent erprises are com plet ely dependent on t heir I T. Consequent ly, t oday's I T is driven by t he sam e dynam ics as t he ent erprise it self. Today, we expect an ext rem ely high level of flexibilit y and agilit y from our ent erprise I T. During t he post I nt ernet - boom years, cost efficiency quickly becam e anot her key requirem ent , if not t he m ost im port ant one. Ent erprise I T has changed as a result of t he const ant ly increasing pressure. I n t he early days of ent erprise com put ing, I T was m erely responsible for providing st orage and processing capacit y, wit h m ore and m ore business logic being added t hroughout t he decades. During t he different boom phases in t he 1980s and 1990s, a plet hora of new applicat ions em erged, oft en side by side wit h t he inform at ion silos t hat had been developed in t he previous 20 years. Today, t he increasing cost pressure is forcing us t o efficient ly reuse exist ing syst em s while also developing new funct ionalit y and const ant ly adapt ing t o changing business requirem ent s. The t erm " legacy syst em " is now oft en replaced wit h " herit age syst em " in order t o em phasize t he value t hat lies in t he exist ing syst em s. The increases in reuse and harm onizat ion requirem ent s have been fueled by t he urgency of int egrat ing t he hist orically grown I T landscapes in order t o im prove I T efficiency and agilit y. As a result , we could observe at a t echnical level t he em ergence of m iddleware t ools and Ent erprise Applicat ion I nt egrat ion ( EAI ) plat form s in what can be seen as a post - RDBMS phase. While a lot of t rial- and- error proj ect s were execut ed in t he 1990s, wit h m ore or less high levels of success, t he developm ent of EAI and m iddleware concept s has now been culm inat ed in t he principles of Service- Orient ed Archit ect ure ( SOA) , which can be seen as an im port ant evolut ionary

point in t he developm ent of int egrat ion t echnologies. What is im port ant about SOA is t hat it has t aken away t he focus from fine- grained, t echnologyorient ed ent it ies such as dat abase rows or Java obj ect s, focusing inst ead on business- cent ric services wit h business- level t ransact ion granularit y. Furt herm ore, SOA is not an ent erprise t echnology st andard, m eaning it is not dependent on a single t echnical prot ocol such as I I OP or SOAP. I nst ead, it represent s an archit ect ural blueprint , which can incorporat e m any different t echnologies and does not require specific prot ocols or bridging t echnologies. The focus is on defining cleanly cut service cont ract s wit h a clear business orient at ion. At t he Wint ert hur, as in any ot her large com pany, we have been facing all of t he preceding issues of hist orically grown syst em s and inform at ion silos. We had t o find a solut ion t o increase our I T efficiency and agilit y. The Wint ert hur, wit h approxim at ely 20,000 em ployees worldwide and over 130 billion Swiss franks of asset s being m anaged ( as of Decem ber 31, 2003) , is a leading Swiss insurance com pany. As is t he case wit h any well- organized com pany, we rely on our I T infrast ruct ure t o m anage asset s, product s, processes, cust om ers, part ners, em ployees, and any ot her aspect of business life. Our core business syst em s are based on highly reliable m ainfram e com put ers t hat we invest ed in over t he past decades. However, like m ost ot her ent erprises relying on m ainfram es for t heir back- end syst em s, we saw t he increasing need over t he years t o open up t hese back- end syst em s. The m ain reason for t his was t o enable reuse of t he core business logic and dat a on t hese syst em s for new I nt ernet and int ranet front - end syst em s on nonm ainfram e plat form s such as UNI X and Windows. To facilit at e t his developm ent , we built up an applicat ion and int egrat ion plat form , which laid t he t echnical basis for Wint ert hur's SOA. While t he init ial developm ent st art ed off at our core Swiss m arket unit , t he plat form is nowadays reused abroad, because of it s success and t he prevailing analogous t echnical requirem ent s of ot her m arket unit s. Thus, we creat e t he basis t o realize synergies and enhance our int ernat ional init iat ives. Building on our t echnical plat form , com bined wit h our in- house experience in t he area of SOA and wit h t he experience t hat our holding com pany Credit Suisse Group has gat hered in sim ilar rearchit ect ural effort s, we have been ext rem ely successful. The Wint ert hur SOA has achieved t he goal of opening up our back- end syst em s in new applicat ion developm ent areas on ot her plat form s. A solid SOA- based archit ect ural approach is at t he heart of our I T st rat egy. This book is im port ant because it provides ent erprise archit ect s wit h a roadm ap for t he successful est ablishm ent of SOA at t he ent erprise level. While a lot of t he underlying principles of t he original Wint ert hur SOA have had t o be derived from past experience and int uit ion due t o lack of SOA lit erat ure at t he t im e, t his book provides a concret e guide, blueprint s, and best pract ices for SOA archit ect s. I n addit ion t o t he Wint ert hur case st udy in chapt er 15, you will find m any m ore concret e exam ples of how large corporat ions have st art ed t o adopt t he principles of SOA in t heir I T archit ect ures. I t is also very im port ant t hat t his book not only focuses on t he t echnical aspect s of SOA, but also places st rong em phasis on t he delicat e issues of est ablishing SOA at t he ent erprise level, t ruly deserving t he t it le Ent erprise SOA. The SOA principles described in t his book are t he foundat ion on which ent erprises can build an I T archit ect ure t hat will sat isfy t oday's m ost im port ant I T requirem ent sagilit y and flexibilit yat affordable cost s. Mart in Frick, Head of I T at t he Wint ert hur Group

Reader's Guide The reader's guide provides an indicat ion as t o who should read t his book and t he benefit s t o be gained. A sum m ary of each chapt er provides an overview of t he st ep- by- st ep approach required for t he successful int roduct ion of Service- Orient ed Archit ect ures ( SOA) .

Who Should Read This Book This book is aim ed at t he various st akeholders of ent erprise soft ware archit ect ures, including soft ware archit ect s and evangelist s, designers, analyst s, developers, m em bers of I T st rat egy depart m ent s, proj ect m anagers, represent at ives of product vendors, and t hose int erest ed in soft ware archit ect ure and it s relat ion t o st ruct ures and processes wit hin large- scale organizat ions. Furt herm ore, t his book is an excellent int roduct ion t o t he real world of com m ercial com put ing for st udent s in a variet y of disciplines. I f you are a soft w a r e a r ch it e ct , t his book provides you wit h hands- on guidelines for t he design of SOAs. You will find t he definit ion of an SOA t oget her wit h it s key t erm s as we dist inguish t he SOA from approaches such as com ponent archit ect ures and soft ware buses. Furt herm ore, t his book provides concret e guidance for t he m ost im port ant design decisions one will encount er in pract ice. These guidelines com prise ident ifying services, assigning t he appropriat e service t ype and allocat ing t he ownership of dat a t o services. You will also discover how t o ut ilize expansion st ages in order t o enable st epwise SOA int roduct ion. This book also provides valuable advice on t he design of a funct ional infrast ruct ure for business processes and on how t o achieve process int egrit y, approach het erogeneit y, and init iat e t he t echnical infrast ruct ure. We discuss t hese guidelines wit h respect t o different applicat ion t ypes, including Web applicat ions, fat client s, m obile applicat ions, EAI , and m ult i- channel applicat ions. For t he purpose of soft ware archit ect s, Chapt ers 4 t o 10 are m ost valuable. I n addit ion, Chapt er 13, which covers SOA proj ect m anagem ent , will be helpful in ensuring an efficient collaborat ion wit hin an SOA proj ect . Finally, t he case st udies in Part I I I give you pract ical exam ples of how archit ect s in ot her organizat ions int roduced an SOA. Do you see yourself in t he role of an SOA e va n ge list ? I f you int end t o im plem ent an SOA wit hin your own organizat ion, you m ust successfully prom ot e your ideas. Most im port ant ly, you m ust be able t o com m unicat e t he benefit s of t he SOA t o all st akeholders of t he applicat ion landscape wit hin your organizat ion. Chapt er 11 will be of special int erest t o you because it present s t he key benefit s of SOA for t he organizat ion and each individual st akeholder. I n addit ion, Chapt er 12 provides an in- dept h descript ion of t he st eps required t o set up an SOA, wit h considerable pract ice- orient ed advice as t o t he int roduct ion of appropriat e processes and boards. Aft er reading t his book, you should have a deeper underst anding of SOAs, enabling you t o effect ively argue t he benefit s t o different st akeholders and t o est ablish t he necessary processes and boards t o m ake your SOA endeavor a success! I f you are a soft w a r e de sign e r, a n a lyst , or de ve lope r working in an SOA proj ect , alt hough you are likely t o work in a specific part of your applicat ion landscape, t his book will help you obt ain a bet t er underst anding of t he ent ire process. Furt herm ore, t here are key challenges such as process int egrit y t hat direct ly im pact your work. This bookin part icular Chapt ers 7 t o 10 helps t o address t hese challenges in a coordinat ed m anner wit hin your SOA proj ect . I f you work in t he I T st r a t e gy depart m ent of an large organizat ion, you should read t his book in order t o find out how SOAs can add t o your I T st rat egy. Your work is likely t o be driven by t he dem and for agilit y and cost effect iveness. Many ent erprises have experienced proj ect s t hat failed t o deliver t he required funct ionalit y and t herefore lost business opport unit ies. Furt herm ore, m any applicat ion landscapes suffer from high m aint enance cost s for t heir inherit ed asset s and t he int egrat ion of new applicat ions. I n Part I I (Chapt ers 11 13 ) you will read about t he various possibilit ies for overcom ing t hese issues wit h an SOA. Finally, several st rat egies for int roducing t he SOA wit hin t he organizat ion are present ed. Part I I I (Chapt ers 14 t o 17 ) cont ains several case st udies wit h real- world evidence t hat validat es t he SOA approach. Those success st ories provide

" living proof" of SOA success and offer an im pression of t he different ways an SOA can be est ablished. I f you are an experienced pr oj e ct m a n a ge r , you should read t his book in order t o underst and t he specific benefit s of SOAs for proj ect m anagem ent . The SOA approach im plies a m aj or sim plificat ion of t he overall soft ware developm ent process, and t his book m akes t hese benefit s accessible. However, SOAs will challenge you, and as a result , t his book present s solut ions t o t he m ost im port ant problem s one encount ers in an SOA proj ect , bot h from t he t echnical and proj ect m anagem ent viewpoint s. You will find Chapt er 13, which focuses on proj ect m anagem ent , and Chapt ers 11 and 12 , which depict t he polit ical environm ent , t o be m ost beneficial. I t should be not ed t hat t his book does not int roduce a new soft ware developm ent m et hodology. You will require a sound knowledge of your organizat ion's favorit e m et hodology, accom panied wit h endurance, social com pet ence, polit ical cleverness, and m anagem ent skills. This book will com plem ent t hese skills so t hat t hey can be successfully applied in an SOA proj ect . For a ve n dor of st a n da r d soft w a r e pa ck a ge s, t his book present s valuable guidance for product m anagem ent and sales. SOAs will soon gain t rem endous im port ance in t he ent erprise soft ware m arket . As a sa le spe r son or a pr odu ct m a n a ge r , you need t o underst and t he requirem ent s of your ent erprise cust om ers in order t o be able t o offer solut ions t hat fit your cust om er's needs. I n part icular, Chapt er 11 will be very beneficial because it depict s t he benefit s of service- orient ed soft ware from t he viewpoint of t he various st akeholders. Being able t o offer service- orient ed soft ware im plies a significant com pet it ive advant age. The inherent st rengt h of SOAs will becom e t he st rengt h of your product . I t enables you t o sell sophist icat ed vert ical solut ions, generat ing product revenues for your com pany wit hout t he burden of high int egrat ion cost s t hat inhibit t he sales process.

A Roadmap for This Book The successful adopt ion of an Ent erprise SOA is based on t hree fundam ent al fact ors: archit ect ure, organizat ion, and lessons drawn from real- world experience. The I T archit ect ure is t he t echnical enabler for an SOA. A successful SOA adopt ion accelerat es an ent erprise by reducing t he gap bet ween st rat egy and process changes on one hand and support ing I T syst em s on t he ot her. The I T archit ect ure and t he business organizat ion are m ut ually dependent , alt hough t hey bot h drive each ot her. Finally, real- world experience, in part icular previous long- t erm I T infrast ruct ure init iat ives ( bot h successful and unsuccessful) influence and validat e m any of t he core concept s of SOA. Not surprisingly, t his book is st ruct ured around t hese t hree fact ors. Aft er we int roduce t he subj ect area in Chapt ers 1 t o 3, Part I , Chapt ers 4 t o 10 , focuses on t he a r ch it e ct u r e . Part I I , Chapt ers 11 t o 13 , discusses t he challenges of int roducing an SOA at t he level of t he or ga niza t ion, depict ing it s benefit s, processes, and proj ect m anagem ent . Part I I I , Chapt ers 14 to 17 , provides r e a l- life e x a m ple s of successful SOA int roduct ions. Ch a pt e r 1 , " An Ent erprise I T Renovat ion Roadm ap," ident ifies t he need for agilit y and cost savings as t he m ain drivers for t he int roduct ion of SOAs. Ch a pt e r 2 , "The Evolut ion of t he Service Concept ," describes how com m ercial inform at ion t echnology has m oved t oward t he service concept over t he last 40 years. Today's SOA is t he prelim inary endpoint of m any years of painful " t est ing." Knowing and underst anding previous pit falls and m ist akes help t o avoid t hem in new proj ect s. Ch a pt e r 3 , " I nvent ory of Dist ribut ed Com put ing Concept s," int roduces t he fundam ent al concept s of dist ribut ed com put ing t hat are required for subsequent discussions in Part I (Chapt ers 410 ) . Part icular t opics will be com m unicat ion infrast ruct ures, synchronous versus asynchronous com m unicat ion, payload sem ant ics, granularit y, and loose versus t ight coupling.

Part I: Architectural Roadmap

Ch a pt e r 4 , " Service- Orient ed Archit ect ures," describes t he part icular requirem ent s of large organizat ions for building an archit ect ure and defines t he t erm " Service- Orient ed Archit ect ure" as it is used t hroughout t his book. Ch a pt e r 5 , " Services as Building Blocks," is a direct cont inuat ion of Chapt er 4. I t int roduces different service t ypesnam ely basic, int erm ediary, process- cent ric, and ext ernal servicesand gives an in- dept h discussion of t heir key charact erist ics. Ch a pt e r 6 , " The Archit ect ural Roadm ap," com plet es t he discussion st art ed in Chapt er 5. Using t he concept of building blocks, t he high- level st ruct ure of SOAs is depict ed. Chapt er 6 int roduces t wo key concept s: SOA layers and expansion st ages. SOA layers aim t o organize t he

aforem ent ioned services at t he ent erprise level. Expansion st ages are well- defined levels of m at urit y of an SOA t hat enable a st epwise im plem ent at ion. I n t his book, t hree expansion st ages are dist inguished: fundam ent al SOA, net worked SOA, and process- enabled SOA. Ch a pt e r 7 , " SOA and Business Process Managem ent ," shows how SOAs and BPM can com plem ent each ot her in pract ice. This chapt er draws a dem arcat ion line bet ween t he responsibilit ies of a BPM infrast ruct ure and t he funct ional infrast ruct ure provided by t he SOA. Ch a pt e r 8 , "Process I nt egrit y ," delves int o t he challenges of dist ribut ed archit ect ures wit h respect t o consist ency and how SOAs approach t his m aj or issue. This chapt er provides num erous helpful, hands- on guidelines t ackling real- world const raint s such as het erogeneit y, changing requirem ent s, or budget . Ch a pt e r 9 , "I nfrast ruct ure of a Service Bus." By t his point , t he reader will know a lot about service t ypes, t he handling of business processes, and SOA layers. This chapt er will address t he issue of t he t ype of runt im e infrast ruct ure t hat is required in order t o put an SOA in placean infrast ruct ure t hat is com m only known as t he " service bus." Chapt er 9 highlight s t he fact t hat t he service bus is oft en het erogeneous and provides t echnical services such as dat a t ransport , logging, and securit y. Ch a pt e r 1 0 , " SOA in Act ion," discusses how SOAs apply t o specific applicat ion t ypes such as Web applicat ions, EAI , fat client s, m obile devices, and m ult i- channel applicat ions.

Part II: Organizational Roadmap

Ch a pt e r 1 1 , " Mot ivat ion and Benefit s," provides a num ber of im port ant reasons as t o why an organizat ion should im plem ent an SOA. I t depict s t he benefit s for t he organizat ion as well as for t he individual st akeholders. Ch a pt e r 1 2 , " The Organizat ional SOA Roadm ap," nam es four pillars for t he success of an SOA int roduct ion at t he ent erprise levelnam ely, budget , init ial proj ect , t eam , and buddies. This chapt er deals wit h challenges such as conflict s of int erest s of different st akeholders or financing t he overheads of t he SOA infrast ruct ure and gives pract ical advice on how t o overcom e t hese obst acles. Ch a pt e r 1 3 , "Proj ect Managem ent," provides best pract ices of SOA proj ect m anagem ent . Most im port ant ly, t his chapt er depict s how service cont ract s can drive t he ent ire developm ent effort . I t shows how different t asks can be decoupled and synchronized at t he sam e t im e and how com plexit y and risk can be reduced. Furt herm ore, t his chapt er describes t est ing, configurat ion m anagem ent , risk assessm ent , and est im at ing cost s and delivery dat es.

Part III: Real-World Experience

Ch a pt e r 1 5 , "Case St udy: Deut sche Post AG." The Deut sche Post World Net is a m ult inat ional group com prising t hree m ain brands and m ore t han 275,000 em ployees. The SOA was set up for t he Mail Corporat e division at Deut sche Post , a part ner t o t hree m illion business cust om ers, providing services t o 39 m illion households t hrough 81,000 delivery st aff, 13,000 ret ail out let s, 3,500 delivery bases, and 140,000 let t erboxes. The SOA at Deut sche Post AG covers a m ainly Java- based environm ent . This fact indicat es t hat a SOA can also be beneficial in hom ogeneous environm ent s. Ch a pt e r 1 5 , "Case St udy: Wint ert hur ." Wint ert hur Group, a leading Swiss insurance com pany, has approxim at ely 23,000 em ployees worldwide achieving a prem ium volum e of 33.5 billion Swiss Francs in 2003. I n 1998, Wint ert hur's Market Unit Swit zerland developed a concept for an Applicat ion Service Plat form . Since t hen, t his int egrat ion plat form , called " e- Plat form ," has been im plem ent ed and used as t he t echnological basis for t he realizat ion of an SOA. Today, t he SOA includes m ost of t he m ission- crit ical business applicat ions. I t s t echnical focus is on m ainfram ebased CORBA services. Well- organized processes and a service reposit ory have been recognized as key success fact ors at Wint ert hur. Ch a pt e r 1 6 , "Case St udy: Credit Suisse." Credit Suisse Group is a leading global financial services com pany operat ing in m ore t han 50 count ries wit h about 60,000 st aff. Credit Suisse report ed asset s under m anagem ent of 1,199 billion Swiss Francs in Decem ber 2003. The SOA was init ially im plem ent ed in order t o creat e m ult i- channel banking applicat ions and online t rading port als. I n addit ion, t he SOA was ut ilized t o consolidat e t he core business applicat ion port folio. Credit Suisse has im plem ent ed t hree different service buses in order t o approach t he different requirem ent s of synchronous com m unicat ion, asynchronous com m unicat ion, and bulk dat a t ransfer. Ch a pt e r 1 7 , "Case St udy: I nt elligent Finance." Halifax Bank of Scot land ( HBoS) is a UK Financial Services provider wit h divisions in Ret ail Banking, I nsurance & I nvest m ent , Business Banking, Corporat e Banking, and Treasury. HBoS is t he UK's largest m ort gage and savings provider wit h a cust om er base of about 22 m illion. I nt elligent Finance was launched as a division of Halifax plc wit h t he aim of at t ract ing new cust om ers from out side Halifax and specifically t o t arget t he UK clearing banks. I nt elligent Finance was launched as Proj ect Greenfield in 2000, st art ing an ent ire new banking operat ion from scrat ch, based on an SOA. Three years lat er, by t he end of 2003, I nt elligent Finance had 820,000 cust om er account s, represent ing asset s of £15.5 billion. The I nt elligent Finance syst em was probably one of t he largest and m ost advanced early SOA deploym ent s in t he financial services indust ry in Europe.

Chapter 1. An Enterprise IT Renovation Roadmap This book m akes a big prom ise: I t is offering an I T renovat ion roadm ap, which will leverage t he concept s of Service- Orient ed Archit ect ures ( SOA) on bot h t he t echnical and organizat ional levels in order t o creat e sust ainable im provem ent s in I T efficiency and agilit y. The aim of t his roadm ap is t o st rike a good balance bet ween im m ediat e gains on one hand and long- last ing im provem ent s t o t he ent erprise I T landscape on t he ot her. An SOA should increase t he capabilit y of an ent erprise t o address new business requirem ent s on t he short t erm by reusing exist ing business logic and dat a m odels, t hus incurring only m inim al cost , resource, and t im e overheads, while m inim izing risks, especially when com pared t o rewrit ing ent ire applicat ion syst em s. I n addit ion, an SOA should provide endurable benefit s in t erm s of agilit y because it provides a long- t erm st rat egy for t he increase of t he flexibilit y of an I T infrast ruct ure. This chapt er closely looks at t he problem s faced by ent erprise soft ware t oday, t he result ing requirem ent s for an ent erprise I T archit ect ure such as an SOA, and how such an archit ect ure can be est ablished on t he organizat ional level.

1.1. Agony Versus Agility I n 2003, Nicolas G. Carr published t he heat edly debat ed art icle " I T doesn't m at t er" in t he Harvard Business Review, claim ing t hat " ... like elect rical grids or railroads, I T would becom e a ubiquit ous com m odit y." Regardless of your posit ion on t his issuewhet her or not you consider ent erprise I T a com m odit yent erprises heavily depend on t he I T backbone, which is responsible for running alm ost all processes of m odern ent erprises, be t hey relat ed t o m anufact uring, dist ribut ion, sales, cust om er m anagem ent , account ing, or any ot her t ype of business process. Because of t oday's highly com pet it ive global econom y, t hese business processes underlie const ant change: Ent erprises m ust const ant ly sense changes in m arket condit ions and swift ly adapt t heir st rat egies t o reflect t hese changes. Therefore, it is a key requirem ent for m odern ent erprise I T t hat changes in com pany st rat egy be reflect ed quickly and efficient ly in t he com pany's I T syst em s, which are t he backbone for execut ing t he st rat egy. This is exact ly where t he ent erprise soft ware dilem m a st art s: Today's ent erprise soft ware developm ent alm ost always suffers from lack of agilit y and from inefficiency. This m eans t hat ent erprises are not able t o m at ch business requirem ent s ont o underlying I T infrast ruct ure fast enough, effect ively lim it ing t he capabilit y of t he ent erprise t o react appropriat ely t o m arket dem ands. I n addit ion, t he inefficiency of ent erprise soft ware developm ent m eans t hat t he developm ent t hat is act ually done cost s t oo m uch when com pared t o t he act ual out put . I f we look at a t ypical ent erprise soft ware syst em , we can norm ally observe an init ial phase of high product ivit y and agilit y, as shown in Figure 1- 1 . During t his Green field phase, t he syst em is built wit h m uch new funct ionalit y, and init ial change request s can be im plem ent ed relat ively quickly and efficient ly. However, aft er t he init ial syst em im plem ent at ion has been put in place and t he first couple of change request s have been execut ed, t he abilit y t o m ake m ore changes t o t he syst em det eriorat es dram at ically, and m aint enance over t im e becom es harder and harder.

Figu r e 1 - 1 . Ch a n ge r e qu e st s r e du ce t h e a gilit y of a syst e m ove r t im e .

This st agnat ion phase, which alm ost any ent erprise soft ware syst em experiences over t im e, cannot be explained by a single reasona num ber of fact ors cont ribut e t o t his phenom enon. Som e of t hese reasons are relat ed t o soft ware t echnology, such as t he difficult y of m aking st ruct ural changes t o an exist ing code base. However, m ost of t he reasons are not of a t echnical nat ure but rat her are relat ed t o reasons on t he organizat ional level. For exam ple, aft er t he init ial launch phase of a syst em , t he proj ect m anagem ent and key dom ain expert s are likely t o m ove on t o t he next proj ect , oft en leaving t he m aint enance of t he syst em t o less skilled m aint enance workers, m any t im es even wit hout doing a proper hand- over. I n addit ion, aft er t he init ial phase of euphoria about t he new syst em , it m ight lose it s int ernal lobby over t im e and t hus t he necessary polit ical support wit hin t he organizat ion. Tight proj ect budget s oft en m ean t hat fixes and changes cannot be done properly and are only done on an ad- hoc basis, wit hout t aking t he exist ing order of t he syst em int o considerat ion. Typically, t here is no t im e and budget for doing a proper refact oring of t he syst em t o ensure it s long- t erm m aint ainabilit y. Finally, crit ical st ruct ural decisions are oft en m ade on t he side, wit hout execut ing proper cont rolfor exam ple, an engineer m ight quickly writ e a sm all bat ch program t o synchronize dat a bet ween t wo syst em s, and 10 years lat er, a sm all arm y of developers is needed t o deal wit h t he consequences of t his ad- hoc decision. When looking at ent erprise soft ware, we are usually not looking at isolat ed syst em s, but at large num bers of syst em s wit h com plex cross- dependencies t hat have grown over m any years and wit h a high level of het erogeneit y and redundancies. We rarely have sufficient in- house knowledge about t he different syst em s, and we oft en need t o cope wit h very different design and program m ing st yles. Finally, people are get t ing used t o t his sit uat ion and are st art ing t o t hink in t erm s of workarounds, not in t erm s of t he " right " st ruct ures. Som e organizat ions m ight have found bet t er ways of coping wit h t hese problem s t han ot hers, but it 's hard t o find any organizat ion t oday t hat can claim t hat it has com plet ely sort ed out t hese issues. For t his reason, m any organizat ions require an ent erprise I T renovat ion roadm ap t o help providing a sust ainable t ransform at ion int o a m ore agile I T organizat ion t hat is able t o quickly and efficient ly adapt t o changing business requirem ent s. This chapt er lays out t he cornerst ones of t his roadm ap as it is present ed in t his book.

1.2. Enterprise Software Is a Different Animal I n order t o bet t er underst and t he problem s of ent erprise soft ware, we need t o look at t he specific charact erist ics of it , which are different from t hose of ot her t ypes of soft ware, such as syst em soft ware, deskt op applicat ions, em bedded syst em s, scient ific soft ware, or video gam es. As t he nam e indicat es, ent erprise soft ware is t ight ly coupled wit h t he int ernal organizat ion, processes, and business m odel of t he ent erprise. Ent erprise soft ware underlies bot h crossdepart m ent al dependencies and ext ernal business relat ionships. Consequent ly, an archit ect ure for ent erprise soft ware m ust deal wit h large num bers of different requirem ent s. Many of t hese requirem ent s are conflict ing, while ot hers are unclear. I n alm ost every case, t he requirem ent s are a m oving t arget due t o t he perm anent change of m arket s, t he organizat ion of t he ent erprise, and it s business obj ect ives. I t is t his involvem ent in all aspect s of t he ent erprise and t he business t hat m akes ent erprise soft ware highly com plex. Ent erprise applicat ions rarely cont ain a large am ount of com plicat ed algorit hm s. The code t hat describes a piece of business logic is usually very sim ple. The st ruct ure of a COBOL- based billing syst em is m uch sim pler t han, for exam ple, an em bedded syst em for a Mars robot wit h com plex real- t im e and m ult i- t hreading requirem ent s. I n ent erprise applicat ions, one usually finds very sim ple dat a st ruct ures, which again are different from ot her syst em s such as geographic inform at ion syst em s ( GI S) . Let 's consider an exam ple in order t o illust rat e t he difference bet ween ent erprise applicat ions and ot her soft ware: An ent erprise applicat ion such as a Cust om er Relat ionship Managem ent Syst em ( CRM) , a billing syst em , a shipping syst em , or an insurance claim s processing syst em . The st akeholders in t hese syst em s include different business unit s and pot ent ially even t he CEO, as well as different I T proj ect s, I T m aint enance, and operat ions. I n t hese scenarios, we will be facing highly het erogeneous t eam s and oft en very polit ical environm ent s. The t echnology landscape will be highly het erogeneous as well, including m any different applicat ion and m iddleware plat form s. The business dat a and cont ent will have a very long lifet im e, especially when com pared wit h t he m uch short er cycles of t echnology innovat ion. We need t o deal wit h const ant ly changing funct ional requirem ent s t hat are usually not well- defined. I n addit ion, we will be facing m any cross- dependencies bet ween funct ional requirem ent s, as well as het erogeneous t echnology plat form s. The num ber of end users will be pot ent ially very large, and t he applicat ions will have t o be rolled out t o large num bers of PCs, oft en m ore t han 10,000. Take, on t he ot her hand, a deskt op applicat ion, such as a word processor or spreadsheet applicat ion. A sm aller, m ore hom ogeneous t echnical t eam will develop t his applicat ion. I t will be used by office workers as well, but t he problem space is m ore well- defined. The applicat ion logic is self- cont ained, wit h very few cross- dependencies. Finally, t here is no roll- out problem because t he end user is t ypically responsible for t he inst allat ion him self. As we can see from t hese exam ples, ent erprise soft ware is unique in m any respect s, and t herefore, it requires unique m easures t o ensure t he efficiency of it s developm ent and m aint enance.

1.3. The Importance of Enterprise Software Architectures According t o t he second law of t herm odynam ics, any closed syst em cannot increase it s int ernal order by it self. I n fact , any act ivit y t hat is geared t oward ordering t he syst em will increase it s overall disorder ( called ent ropy) . I n m any respect s, t his law is also applicable t o ent erprise soft ware, which oft en has very sim ilar charact erist ics. Consequent ly, out side int ervent ion is cont inually required t o help creat e a higher order and t o ensure t hat developm ent effort s are not lost . I n ent erprise soft ware, t he archit ect t akes on t he role as an out side influencer and cont roller. I t is his responsibilit y t o oversee individual soft ware proj ect s from t he st rat egic point of view of t he overall organizat ion, as well as from t he t act ical, goal- orient ed viewpoint of t he individual proj ect . He has t o balance different requirem ent s while at t em pt ing t o creat e an enduring order wit hin t he ent erprise soft ware landscape. The ent erprise soft ware archit ect ure is t he archit ect 's m ost im port ant t ool at hand. Soft ware archit ect s are const ant ly confront ed wit h changes t o and expansion of funct ionalit y t hat increase syst em com plexit y and reduce efficiency. By refact oring current solut ions, archit ect s const ant ly st rive t o reduce com plexit y and t hereby increase t he agilit y of t he syst em ( see Figure 1- 2 ) .

Figu r e 1 - 2 . Soft w a r e a r ch it e ct s u se r e fa ct or in g t o figh t t h e con st a n t in cr e a se in syst e m com ple x it y. [View full size image]

Apart from t he event s t hat increase com plexit y during norm al usage of t he archit ect ure, single event s can also have an im port ant effect on ent erprise I T. They m ight occur in m aj or changes t o exist ing j urisdict ion, t he end- of- life of a support ed product , or t he int roduct ion of large chunks of com put ing infrast ruct ure, such as in t he course of a m erger or acquisit ion. Such event s require a m aj or effort at very short not ice t o keep t he archit ect ure in a sim ple and m aint ainable st at e. Devast at ing consequences have been observed as a result of m ergers and acquisit ions: concise financial report ing being lost aft er a m erger and raw syst em capacit y being exhaust ed aft er an acquisit ion. Because it is unknown a priori when such effect s will occur, it is vit al t o keep t he ent erprise archit ect ure in a m aint ainable and changeable st at e all t he t im e. As we will see in t he rem ainder of t his book, Service- Orient ed Archit ect ures are part icular well suit ed t o cope wit h t he needs of such an ongoing increm ent al process of opt im izat ion.

1.4. The Requirements for an Enterprise Software Architecture As a result of t he aforem ent ioned t ight coupling wit h t he int ernal organizat ion, processes, and business m odel of t he ent erprise, an ent erprise soft ware archit ect ure m ust fulfill very different requirem ent s t han, for exam ple, a soft ware archit ect ure for a syst em t hat is cont rolled by a sm all num ber of highly qualified dom ain expert s, such as t he Mars robot or a video gam e engine. I n order t o im prove agilit y and efficiency, an ent erprise soft ware archit ect ure m ust provide part icular charact erist ics: Sim plicit y. The ent erprise archit ect ure m ust be sim ple in order t o allow efficient com m unicat ion bet ween key personnel. As previously discussed, m any people are involved in t he specificat ion and const ruct ion of ent erprise soft ware. All t hese people have different roles and consequent ly different viewpoint s wit h regard t o t he soft ware. I t is also likely t hat several different skill set s exist am ong personnel. These m ight range from I T coordinat ors of funct ional depart m ent s t o t echnical archit ect s. Many I T coordinat ors will have det ailed business dom ain knowledge but no t echnical expert ise. On t he ot her hand, t echnical archit ect s will probably have an excellent t echnical educat ion but have lit t le underst anding of t he vert ical business. Nevert heless, all t he people involved m ust be able t o underst and and m anage t he archit ect ure at t heir respect ive levels ( e.g., specifying new funct ionalit y at t he business level and im plem ent ing and m aint aining it ) . Fle x ibilit y a n d m a in t a in a bilit y. Every ent erprise syst em is subj ect t o ongoing change. I t m ust cont inuously be adapt ed t o new requirem ent s due t o t he need of evolving m arket s, legal changes, or business reorganizat ions. Therefore, t he archit ect ure m ust lead t o a highly flexible and m aint ainable syst em . The archit ect ure m ust define dist inct com ponent s t hat can be rearranged and reconfigured in a flexible m anner. Local changes cannot be perm it t ed t o have an im pact on t he global syst em . Providing t hat t he ext ernal API of a com ponent rem ains st able, an int ernal change should not affect operat ions out side t he com ponent . I n t his cont ext , one needs t o underst and t hat ext ernal int erfaces of com ponent s m ust be designed very carefully. To a great ext ent , int erfaces m ust be generic and not specific t o a single usage scenario. However, defining generic int erfaces requires excellent dom ain knowledge, experience, and t o som e ext ent , luck. Finally, t he int ernal im plem ent at ion of a com ponent m ust allow efficient m aint enance, m aking it is easy t o add or m odify funct ionalit y. Re u sa bilit y. Reusabilit y has been a m aj or obj ect ive of soft ware engineering for decades, wit h varying degrees of success. I t is in t he int erest of an ent erprise t o gain as m uch benefit from it s soft ware asset s as possible. This can be achieved by creat ing an invent ory of useful building blocks and cont inually reusing t hem . One obvious reason for reuse is reduced developm ent and m aint enance cost , which can be accom plished by sharing com m on funct ionalit y in code libraries t hat are used across different proj ect s. However, perhaps a m ore im port ant aspect of reusabilit y is t he abilit y t o share dat a across applicat ions in realt im e, t hus reducing cont ent redundancies. Having t o m aint ain t he sam e dat aset in m ult iple dat abases becom es a night m are in t he long t erm . Unfort unat ely, it is not easy t o achieve t he goals of reuse. Large organizat ions have learned t hat reuse is not always efficient because it is part icularly cost ly t o adm inist er, find, and underst and t he com ponent s t hat should be reused, and som et im es t his cost out weighs t he benefit s. D e cou plin g of fu n ct ion a lit y a n d t e ch n ology. The archit ect ure m ust m ake an ent erprise

organizat ion independent of t he t echnology. I t m ust decouple t he long lifecycle of t he business applicat ion landscape from t he short er innovat ion cycles of t he underlying t echnology. Moreover, an archit ect ure t hat is designed t o last longer t han one or t wo of t hese t echnology innovat ion cycles m ust cope not only wit h changing t echnologies but also wit h t he act ual lifecycles of inst alled t echnologies, which can be m uch longer. I t is t herefore a m aj or requirem ent t hat t he archit ect ure t olerat e bot h het erogeneit y and change t o it s t echnical infrast ruct ure. Furt herm ore, t he developm ent of business funct ionalit y m ust be decoupled from t he underlying t echnology. I n part icular, t he archit ect ure m ust avoid dependencies on specific product s and vendors. This book illust rat es how a Service- Orient ed Archit ect ure can help achieve t he design goals for ent erprise soft ware syst em s as described previously.

1.5. The Relation of Enterprise Architecture and Enterprise Standards For m any decades, ent erprise I T organizat ions have at t em pt ed t o im prove agilit y and efficiency by hom ogenizing t heir syst em s t hrough t he int roduct ion of ent erprise- wide I T st andards, but m ost ly wit h very lim it ed success. Therefore, it is im port ant t o underst and t hat an ent erprise archit ect ure is not equal t o an ent erprise st andard, as we discuss in t his sect ion. I n t he 1980s, wit h relat ional dat abase syst em s becom ing m ainst ream , we saw a wave of so- called Ent erprise Dat a Model ( EDM) proj ect s. The idea of t hese st andardizat ion proj ect s was t o define one global dat a m odel for all t he business ent it ies in an ent erprise, which was t o be shared am ong all t he different organizat ions and syst em s in a com pany. Alm ost all of t hese EDM proj ect s failed, and t oday, t here are usually as m any different dat abase schem as out t here as t here are dat abases in an ent erprise. There are a variet y of different reasons for t he failure of t hese EDM proj ect s, including polit ical t urf wars bet ween different depart m ent s, conflict ing int erest s bet ween t he different st akeholders ranging from business represent at ives over applicat ion specialist s t o DBMS adm inist rat ors, t he sheer t echnical com plexit y of t he undert aking, and t he fact t hat due t o t he dynam ics and com plexit y of m odern ent erprises, it is usually im possible t o capt ure a snapshot of t he com plet e st at e of t he ent erprise at a given point in t im e. I n t he 1990s, we saw t he next at t em pt t o hom ogenize t he ent erprise applicat ion landscape, t his t im e t hrough ent erprise- wide m iddleware st andards. The concept of t he Ent erprise Soft ware Bus becam e popular. The idea was t hat by agreeing on a ubiquit ous, t echnology- independent , ent erprise- wide st andard for com m unicat ion bet ween soft ware m odules, t he problem of applicat ion int egrat ion would be solved once and for all. However, t he realit y in alm ost all ent erprises t oday is t hat in addit ion t o applicat ion het erogeneit y, we now face t he problem of m iddleware het erogeneit y as well. I n m any cases, m iddleware such as CORBA was only used t o solve point - t o- point int egrat ion problem s on a per- proj ect basis, inst ead of being est ablished as a global soft ware bus; as a result , m any ent erprises now have nearly as m any incom pat ible m iddleware syst em s as t hey have applicat ions. I n general, it seem s fair t o say t hat ent erprise st andardizat ion effort s in I T have failed t o deliver on t heir prom ise of hom ogenizat ion and easy applicat ion int egrat ion. Too m any generat ions of m iddlewareranging from DCE over CORBA t o SOAP and WSDLhave been t out ed as silver bullet s but have failed t o becom e est ablished as t he ubiquit ous Ent erprise Soft ware Bus, leaving behind a high level of cynicism am ong t he people involved. As a reader of a book on Service- Orient ed Archit ect ures, you m ight now be asking yourself, " So what is different t his t im e?" I s SOA not yet anot her ent erprise- wide st andardizat ion effort , t his t im e under t he label of t he Ent erprise Soft ware Bus? How are SOAP and WSDLwhile m aybe t echnically superior and m ore flexiblegoing t o address t he organizat ional challenges of global st andards t hat m ade t he Ent erprise Dat a Model, t he Ent erprise Soft ware Bus, and m any ot her ent erprise st andardizat ion effort s fail t o a large ext ent ? This book t akes t he posit ion t hat SOA is neit her a t echnology nor a t echnology st andard, but inst ead it represent s a t echnology- independent , high- level concept t hat provides archit ect ural blueprint s, such as t he ones out lined in t he first part of t his book. These archit ect ural blueprint s are focusing on t he slicing, dicing, and com posit ion of t he ent erprise applicat ion layer in a way t hat t he com ponent s t hat are creat ed and exposed as services in t he SOA are not only t echnically independent but also have a direct relat ionship t o business funct ionalit y. They enable t he st ruct uring of applicat ion com ponent s on t he local level while also cat ering for global int egrat ion of

t hese com ponent s. As we will show in t his book, an SOA does not rely on t he support of part icular runt im e prot ocols, such as SOAP or I I OP. Therefore, an SOA does not im pose adherence t o t echnical st andards on t he global level and is not based on st rict norm s and specificat ions ( see Figure 1- 3 ) .

Figu r e 1 - 3 . En t e r pr ise D a t a M ode ls a n d Soft w a r e Bu se s w e r e popu la r a ppr oa ch e s t o t h e ch a lle n ge s of e n t e r pr ise com pu t in g in t h e 1 9 8 0 s a n d 1 9 9 0 s. [View full size image]

Applicat ions t hat are st ruct ured according t o t he guiding SOA principles laid out in t his book will be able t o fit int o any int egrat ion scenario, regardless of t he runt im e prot ocols required. Having said t his, work will st ill be required t o bridge t echnology gaps, such as different com m unicat ion prot ocols, but t he effort s required t o bridge t hese gaps will be m arginal when com pared t o t he com plexit y of int egrat ing applicat ions on t he st ruct ural level.

1.6. Organizational Aspects When t alking about ent erprise I T, it is im port ant t o realize t hat m anyif not m ost of t he problem s associat ed wit h it are not of a t echnical nat ure but can be found on t he organizat ional level inst ead. Quit e nat urally, we have already im plicit ly t ouched on m any of t hese organizat ional aspect s in our discussion so far ( for exam ple, when discussing t he reasons for t he failure of ent erprise st andards such as t he Ent erprise Dat a Model or t he Ent erprise Soft ware Bus, which largely result ed from problem s on t he organizat ional and not t he t echnical level) . The I T organizat ion and t he way proj ect s are m anaged in a large ent erprise are again very different from what one would find, for exam ple, in a com pany t hat produced em bedded syst em s or gam es. First and forem ost , it is im port ant t o realize t hat m ost likely in no ot her part of t he soft ware indust ry will we find a developm ent and m aint enance process t hat is so closely aligned wit h t he end cust om er. I f an ent erprise is developing a new financial report ing syst em , it will have t o be done hand- in- hand wit h t he finance depart m ent and any ot her st akeholders of t he financial report ing syst em , possibly up t o t he CEO. A soft ware t eam t hat is developing em bedded cont rol soft ware for a dishwasher is unlikely t o have daily m eet ings wit h a housewife about t he exact funct ionalit y of t he soft ware. An im port ant consequence is t hat we are dealing wit h a m uch m ore com plex and m ore am biguously defined decision- m aking process, which is driven m ore oft en by business st rat egy and polit ical agendas t han by t echnical argum ent s. The organizat ional environm ent we are dealing wit h is ext rem ely het erogeneous, and m any different opinions will have t o be incorporat ed int o any decision t hat is m ade, be it a decision about budget s, funct ional requirem ent s, proj ect priorit ies, or t he int erest ing quest ion of what act ually defines t he success of an I T proj ect . For all t hese reasons, it is vit al t hat our ent erprise I T renovat ion roadm ap provides not only a t echnical roadm ap but also an organizat ional roadm ap, which out lines how t he t echnical archit ect ure is t o be est ablished on t he ent erprise level from t he polit ical and organizat ional point of view. The second part of t his book provides an overview of t his organizat ional roadm ap.

1.7. Lifelong Learning Ent erprise soft ware has always suffered from t he m ism at ch bet ween t echnical and businessrelat ed concept s and t he different languages spoken by t he people on bot h sides of t he fence. As a result , we have not only faced inefficiencies, but we also have oft en lost im port ant knowledge and consequent ly had t o reinvent m any solut ions. Many at t em pt s have been m ade in t he past t o find a com m on denom inat or bet ween business and t echnical concept s. For exam ple, SQL was invent ed in t he 1970s wit h t he vision t hat it would give non- t echnical business analyst s a t ool t o access, analyze, and m anipulat e business dat a direct ly. Today, SQL is largely seen as a t ool for t echnical expert s, and it has t urned out t hat m ost of t he ent it ies found in relat ional dat abases are t oo fine- grained and closely int ert wined wit h t echnical concept s t o have a m eaning on t he business level. I t is a key goal of an SOA t o provide services t hat have a concret e m eaning on t he business level. Because of t his one- t o- one m apping bet ween business and t echnology ent it ies, SOA provides a unique chance for t he first t im e in I T hist ory t o creat e art ifact s t hat have an enduring value for bot h t he business as well as t he t echnology side. SOA provides a chance t o m ake t hings t hat have been learned t he hard way usable for t he organizat ion in t he long run. Sim ilarly t o hum an beings, organizat ions will never be able t o st op learning if t hey want t o be successful for long. SOA provides an excellent plat form for t his lifelong learning on t he organizat ional level because an SOA enables us t o const ant ly com pare t he nom inal and t he act ual and t o react accordingly t o fill t he gaps or adapt t he archit ect ure t o reflect changes in business st rat egy. Consequent ly, t he t hird part of t his book provides a num ber of real- world case st udies, which can provide a good st art ing point for learning t he lessons result ing from ot her organizat ions' adopt ion of SOAs and t he im pact t hey had.

1.8. The Enterprise IT Renovation Roadmap As we have out lined in t his int roduct ion, we need st rong ent erprise archit ect ure concept s t o address t he st ruct ural problem s of t he ent erprise I T landscape, accom panied by a st rat egy for how t o est ablish t he archit ect ure on t he organizat ional level. SOA provides t hese concept s, but we have t o be aware t hat im plem ent ing an archit ect ure like an SOA is an ongoing process, requiring const ant guidance and overseeing. The SOA archit ect needs t o bridge m any conflict ing requirem ent s, result ing from frequent changes of business requirem ent s, t he evolut ion of applicat ion and infrast ruct ure t echnology, and last but not least , changes t o t he archit ect ure it self. We need ways t o int roduce st ep- wise im provem ent s, which will bring us slowly but st eadily closer t o our goal. We will have t o accept t hat t he pat h we are about t o ent er will not be a st raight pat h, and t here will be m any influences out side of our cont rol. Nevert heless, t he aut hors believe t hat t he int roduct ion of an SOA will bring m any long- t erm benefit s t o an ent erprise. Figure 1- 4 depict s t he overall vision for how our roadm ap can bring an ent erprise t hat is suffering from developm ent inefficiencies back t o a level of high efficiency and agilit y.

Figu r e 1 - 4 . Se r vice - Or ie n t e d Ar ch it e ct u r e is a k e y e le m e n t of a n e n t e r pr ise I T r e n ova t ion r oa dm a p. [View full size image]

This book aim s t o flesh out an ent erprise I T renovat ion roadm ap. This roadm ap is not only relat ed t o t echnology, but it also equally addresses organizat ional challenges. The roadm ap ant icipat es const ant changes in st rat egic direct ions and assum es t hat t he underlying archit ect ure will have t o be const ant ly adapt ed t o include t he result s of lessons learned as we go along. Consequent ly, t he t hree part s of t his book are st ruct ured t o capt ure t hese different requirem ent s.

Figu r e 1 - 5 . An En t e r pr ise I T r e n ova t ion r oa dm a p n e e ds t o a ddr e ss t h r e e dim e n sion s: a r ch it e ct u r e , or ga n iza t ion , a n d r e a l- w or ld

e x pe r ie n ce .

Part I of t his book provides t he archit ect ural roadm ap, m apping out t he different expansion st ages of an SOA, which will gradually help I T organizat ions t o get back t o a high level of agilit y and efficiency on t he t echnical level. Part I I of t his book looks at t he organizat ional roadm ap, providing an in- dept h discussion of how t he archit ect ure can be est ablished on t he organizat ional level, t he different st akeholders and how t hey m ust be involved, and how an SOA can be leveraged t o drive proj ect m anagem ent m ore efficient ly. Finally, Part I I I of t his book provides a num ber of case st udies, which provide real- world experiences from large corporat ions t hat have already st art ed t heir j ourney on t he SOA- based ent erprise renovat ion roadm ap.

Chapter 2. Evolution of the Service Concept Before looking at t he road ahead, we want t o t ake a st ep back and look at t he evolut ion of t he service concept by exam ining t he m ilest ones of ent erprise com put ing and how t hey have shaped t he concept of " services." We will look at t hree core developm ent direct ions: program m ing languages, dist ribut ion t echnology, and business com put ing. Each has undergone a m aj or evolut ion in t he past 40 years, leading t o a level of abst ract ion t hat support ed t he em ergence of Service- Orient ed Archit ect ures.

2.1. Milestones of Enterprise Computing The t erm " service" has been present in com m ercial com put ing for a long t im e and has been used in m any different ways. Today, for exam ple, we find large com panies, such as I BM, prom ot ing t he concept of " services on dem and." At t he beginning of t he new cent ury, t he t erm " Web services" becam e ext rem ely popular, alt hough it has oft en been used t o refer t o very different com put ing concept s. Som e people use it t o refer t o applicat ion services delivered t o hum an users over t he Web, like in t he popular salesforce.com applicat ion. Ot her people use t he t erm " Web services" t o refer t o applicat ion m odules m ade accessible t o ot her applicat ions over t he I nt ernet t hrough XMLbased prot ocols. Because of t he m any different ways in which t he t erm " service" has been used over t he years in t he I T indust ry, it is necessary t o define m ore precisely how we use it in t his book. However, before looking at a m ore form al, t echnology- orient ed definit ion in Chapt er 4, " Service- Orient ed Archit ect ures," we will look at a m ore generic definit ion t hat bet t er suit s t he purpose of t his chapt er, which is exam ining t he root s of " our" underst anding of services. The Merriam Webst er's Dict ionary gives various definit ions for t he t erm " service," including " useful labor t hat does not produce a t angible com m odit y" and " a facilit y supplying som e public dem and." [ 1] I n t his book, t he t erm " service" t akes it s m eaning from t hese definit ions. I t denot es som e m eaningful act ivit y t hat a com put er program perform s on request for anot her com put er program . Or, in m ore t echnical t erm s, a service is a rem ot ely accessible, self- cont ained applicat ion m odule. Applicat ion front ends are m aking t hese services accessible t o hum an users ( see Figure 2- 1 ) . Oft en, t he t erm s " client " and " server" are used synonym ously for " service consum er" and " service provider," respect ively. [1]

http://www.m-w.com.

Figu r e 2 - 1 . Ou r u n de r st a n din g of t h e t e r m se r vice : A se r vice pr ovide r ( com m on ly a r e m ot e se r ve r ) pe r for m s som e t a sk a t t h e r e qu e st of a se r vice con su m e r ( t h e clie n t ) .

The services covered in t his book provide abst ract ion from a lot of t heir t echnical det ails, including locat ion and discovery. Typically, our services provide business funct ionalit y, as opposed t o t echnical funct ionalit y. Consist ent wit h t he definit ion from Merriam Webst er's Dict ionary, our services are not designed for one specific cust om er, but inst ead t hey are " a facilit y supplying

som e public dem and"t hey provide a funct ionalit y t hat is reusable in different applicat ions. The cost effect iveness will depend st rongly on t he num ber of different cust om ers a service has, t hat is, t he level of reuse t hat can be achieved. A concret e im plem ent at ion of an SOA provides service consum ers wit h seam less access t o t he different services available in it . Thus, aft er a service consum er is " wired" int o t he inst ance of t he SOAaft er t he " ring t one" of t he SOA is availableusage of t he services is seam less and t ransparent . However, Chapt er 1 said t hat an SOA is an archit ect ure per se and not a service bus or any ot her specific m iddleware, and t herefore, it describes st ruct ure, not concret e t echnology. Consequent ly, inst ances of SOAs m ight t ake very different t echnical shapes and form s in different ent erprises. A crucial fact or in t he developm ent of services as we underst and t hem in t he cont ext of t his book is t he quest for t he right degree of abst ract ion. Ult im at ely, a service encapsulat es som e act ivit y of a cert ain com plexit y. Using a service m akes t he world m ore convenient for t he service consum er. Consequent ly, an appropriat e int eract ion pat t ern m ust exist bet ween t he service provider and service consum er. Given t he analogy of t he t elephone net work as t he service infrast ruct ure, services can be anyt hing from hect ic chat t er t o concise and focused conversat ion. They can also include som e form of t elephone conference, answering m achine, or call redirect ion. I n t he rem ainder of t his book, you will not ice surprising sim ilarit ies bet ween t he service concept and t he t elephone analogy. I t is int erest ing t o not ice t hat t he service m odel, as it is defined in t his book, has been preceded by m any t echnologies and t echnical concept s in t he last 30 years t hat shared m any of t he sam e underlying ideas and concept s. For exam ple, look at t he creat ion of reusable business funct ions on m ainfram e com put ers. Sm all com put er net works creat ed a fert ile ground for service- relat ed innovat ions, m ost not ably t he creat ion of rem ot e procedure calls. Com ponent - based developm ent and obj ect orient at ion prom ot ed int erface- driven design paradigm s. Com binat ions of t hese concept s spawned plat form s for dist ribut ed com put ing. At t he sam e t im e, relat ional dat abase developm ent progressed rapidly, creat ing reliable dat a st ores for ent erprises. Business funct ionalit y left t he m ainfram es and m oved t o one of several dist ribut ed paradigm s. I n t he 1980s, packaged business solut ions becam e available t o a wide range of funct ional areas and indust ries. By 1995, t he World Wide Web offered an ext rem ely sim ple yet ext rem ely successful service: t he provision of cont ent over a global net work. The sam e net work infrast ruct ure is used t oday t o enable int eract ion bet ween rem ot e com put er syst em s t hrough I nt ernet prot ocols such as ebXML and SOAP. The root s of service- orient at ion can be found in t hree different areas: program m ing paradigm s, dist ribut ion t echnology, and business com put ing. The developm ent of different program m ing language paradigm s has not only cont ribut ed t o t he im plem ent at ion plat form for t he different elem ent s of an SOA but also has influenced t he int erfacing t echniques used in an SOA, as well as t he int eract ion pat t erns t hat are em ployed bet ween service providers and service consum ers. Many of t he concept s originally found in program m ing languages have also m ade t heir way int o t he dist ribut ion t echnology t hat is present ly used t o offer rem ot e access t o services provided by different applicat ions on different t echnical plat form s. Finally, and m aybe m ost im port ant ly, t he evolut ion of business com put ing has result ed in a large num ber of propriet ary as well as packaged applicat ions ( see Figure 2- 2 ) such as Ent erprise Resource Planning ( ERP) , Cust om er Relat ionship Managem ent ( CRM) , and Supply Chain Managem ent ( SCM) , which t oday are providing t he cont ent t he dat a and t he business logict hat brings an ent erprise SOA t o life. Because of t he closeness of services t o concret e business funct ionalit y, service- orient at ion has t he pot ent ial t o becom e t he first paradigm t hat t ruly brings t echnology and business t oget her on a level where people from bot h sides can equally underst and and t alk about t he underlying concept s.

Figu r e 2 - 2 . Th e lon g h ist or y of pr ogr a m m in g la n gu a ge s, dist r ibu t ion t e ch n ology, a n d bu sin e ss com pu t in g h a s in flu e n ce d t h e de ve lopm e n t

of a n e w pa r a digm , ca lle d se r vice - or ie n t a t ion . [View full size image]

2.2. Programming Paradigms The first program m ing paradigm t hat enabled abst ract ion from t he det ails of com put er program s was funct ional decom posit ion and t he relat ed t echnology of funct ional analysis. Funct ional decom posit ion pioneered t he form al int roduct ion of flow chart s, showing dat a t hat flows t hrough a num ber of processes. One of t he first people t o form alize it was Myers in 1976 [ Myers76] . However, t he first language t hat was suit ed t o funct ional decom posit ion was t he COBOL ( Com m on Business Orient ed Language) program m ing language t hat was creat ed as early as 1959 by a body called CODASYL ( Conference on Dat a Syst em s Languages) and t hat was lat er adopt ed as an ANSI st andard. Having undergone m any im provem ent s and addit ions since t hen, it is st ill a prevailing program m ing language in m any m aj or organizat ions. Apparent ly, m ore code is writ t en in COBOL t han any ot her language. I n 1970, while working at t he Polyt echnic Universit y of Zurich, Niklaus Wirt h invent ed Pascal, a language t hat explicit ly encouraged funct ional decom posit ion and t hat rem ains one of t he m ost popular t eaching languages for com put er science. Funct ional program m ing concept s rem ain popular because t hey are easy t o underst and by st udent s, program m ers, and cust om ers. They provide a powerful t ool t o creat e reusable blocks of codefunct ionst hat can even be sold in t he form of soft ware libraries. Funct ional program m ing cont ribut ed t o t he service concept because funct ions essent ially provide som e form of abst ract ion. However, t he am ount of abst ract ion t hey can provide is lim it ed. I t soon becam e apparent t hat t he funct ional paradigm had it s lim it s. Mult ipurpose reusable funct ions are hard t o creat e. Oft en, t he caller m ust provide m any param et ers, and a lot of dat a m ust be passed int o m ult iple funct ions in order t o obt ain t he required result . The concept s of soft ware m odules and soft ware com ponent s were creat ed t o cope wit h t his growing com plexit y. These concept s cam e in m any different flavors. The first t im e t hese concept s appeared was in t he original im plem ent at ion of t he ADA program m ing language, m odules in t he Modula2 com put ing environm ent ( also creat ed by Niklaus Wirt h in 1979) , and t he hugely com m ercially successful MS Visual Basic's VBX com ponent s. Alt hough very different , t hey share t he com m on abst ract ion of soft ware com ponent s as a cont ainer for bot h dat a and t he funct ions t hat operat e on t hat dat a. Even before t he advent of soft ware com ponent s, it was considered good program m ing pract ice t o shield a funct ion's user from it s int ernal det ails. At t his point , t he concept was int roduced as a language elem ent known as encapsulat ion. The significant increase of abst ract ion and encapsulat ion t hat com ponent s provide are an im port ant st ep t owards service orient at ion. However, t heir m ain purpose was in- sit u developm ent reuse, while service orient at ion focused on dist ribut ion and runt im e reuse. By t he early 1980s, m odularizat ion and com ponent program m ing were widely recognized as t he next big t rend in soft ware developm ent , and t he MODULA language provided a st able and m at ure program m ing plat form for t his t rend. However, t he Japanese and U.S. governm ent s poured m assive am ount s of m oney int o t he developm ent and m arket ing of t heir own program m ing environm ent s, PROLOG and ADA. The uncert aint y t hat em erged from t his proliferat ion of plat form s delayed t he adopt ion of com ponent - orient ed program m ing long enough for it t o be out run in popularit y by obj ect - orient ed program m ing, [ 2] which int roduced t he obj ect as a program m ing and runt im e concept . Originally, obj ect - orient ed program m ing was developed for sim ulat ion purposes. The first obj ect - orient ed language, SI MULA, was developed as early as 1967 at t he Norwegian Com put ing Cent er in Oslo by Ole- Johan Dahl and Krist en Nygaard, and it has since been developed furt her [ Kirk89] . Obj ect orient at ion ent ered m ainst ream program m ing paradigm s in t he m id 1980s wit h t he creat ion of

Sm allt alk [ Gold83] and C+ + [ St ro85] . New versions of m ost ot her obj ect - orient ed languages, such as Java, were invent ed, while ot hers, such as Pascal or even COBOL, were ext ended t o em brace obj ect orient at ion in one way or anot her. [2]

It took approximately 15 years until the concepts of component-orientation reemerged in the late 1990s, this time supported by concrete component platform implementations such as Enterprise Java Beans.

Obj ect s are m uch like com ponent s in t hat t hey support encapsulat ion and t he bundling of dat a and funct ions and add t he concept of individual ent it ies ( t he obj ect s) as inst ances of classes. The obj ect s com m unicat e t hrough m essage exchange, but m ore im port ant ly, obj ect orient at ion adds t he concept of inherit ance, where t ypes can be derived from ot her t ypes. The derived t ype inherit s all t he int ernals and behavior of it s ancest or. This spawned new concept s such as t he program m ing by int erface paradigm . A com m on problem of obj ect - orient ed program m ing is t hat t he level of abst ract ion and granularit y t hat is exposed t o t he client s of a com ponent is t oo fine t o enable efficient reuse, let alone dist ribut ion ( see Figure 2- 3 ) . Therefore, obj ect - orient at ionwhile being great for large, usually relat ively isolat ed and m onolit hic applicat ionsproved t o be a dead- end from t he point of view of dist ribut ed com put ing. Service- orient at ion aim s t o overcom e a lot of t he problem s of dist ribut ed obj ect com put ing, especially wit h respect t o t he right level of granularit y and access pat t erns for rem ot e services. Service- orient at ion also oft en goes a st ep backward when it com es t o t he quest ion of how t ight ly dat a and funct ionalit y should be coupledwhile OO im poses encapsulat ion of dat a and funct ionalit y, service- orient at ion oft en assum es t hat dat a and funct ionalit y are separat ed.

Figu r e 2 - 3 . Th e de ve lopm e n t of pr ogr a m m in g la n gu a ge s h a d a st r on g im pa ct on t h e in t e r fa cin g t e ch n iqu e s for dist r ibu t e d com pon e n t s a n d t h e im plie d a cce ss pa t t e r n t o t h e se com pon e n t s. [View full size image]

As we can see, t he evolut ion of t he different program m ing paradigm s also had a huge im pact on t he concept of service- orient at ion. Modern program m ing languages and developm ent

environm ent s are providing t he t echnical foundat ion for ent erprise- level services. However, not all program m ing concept s are direct ly applicable on t he ent erprise service level. Wit h t he ever t ight er coupling of rem ot ing and program m ing language t echnology, it becom es even m ore im port ant t o be aware of t hese m ism at ches bet ween program m ing languages and ent erprise services and t o avoid falling int o t he t rap of exposing t oo finely grained t echnical concept s t o t he ent erprise level. [ 3] [3]

See Sections 9.1.2.2 and 9.1.2.3 for detailed insight into this specific discussion.

2.3. Distributed Computing Where t he t erm service is used in t his book, we usually assum e t hat t he service does not necessarily reside on t he sam e physical m achine as t he calling applicat ion. The capabilit y t o call a rem ot e com put er program from anot her com put er program in a seam less and cont rolled way is at t he heart of our underst anding of services. An im port ant challenge of dist ribut ed com put ing is t o find abst ract ions for bot h rem ot eness and t he act ual service t ask at t he sam e t im e. Alt hough rem ot eness can be hidden from t he service consum er, t he service provider m ust st ill choose it s invocat ion scenarios wit h t he right granularit y. The dist ribut ed com put ing infrast ruct ure was developed in t he last 30 years. Business com put ing originally m eant m ainfram e com put ing and involved large com put ers wit h cost s in t he m ult im illions of dollars, perform ing t asks m ost ly on t heir own. At som e point , t hey m orphed int o m ore int eract ive m ult i- user syst em s. Rat her t han dist ribut ing t he com put ing power, only dat a capt ure and display was dist ribut ed using t erm inal devices such as t he DEC VT100 or t he I BM3270. Som e of t he first t hings such syst em s had t o share am ong t hem selves were dat a and out put devices such as t ape recorders or print ing syst em s. I n t he early 1970s, com put ers becam e sm aller and cheaper. The price/ perform ance rat io m ade com put er t echnology suit able for a broad range of applicat ions. Research inst it ut ions quickly realized t hat t hey could operat e bot h m ore econom ically and m ore independent ly when t hey were able t o use various sm all com put ers rat her t han one m ainfram e syst em . At t he universit ies of St anford and Berkeley, t wo research program s event ually led t o t he creat ion of t he Unix operat ing syst em . The St anford Universit y Net work spun off t he com pany Sun Microsyst em s in 1982, which is t oday one of t he largest vendors of Unix com put ers. Unix is different from it s predecessorsand m any of it s successorsin t hat it s design quickly adopt ed t he net work as an essent ial part of t he syst em environm ent . Two ideas fost ered t his design perspect ive: facilit at ing rem ot e cont rol of com put ers and program s and providing services t o ot her com put ers in t he net work. The first t rain of t hought creat ed t ools such as t elnet and t he Berkeley r- t ools suit e. The second one feat ured rem ot e print ing and t he t ransparent provision of st orage spacet he NFS file syst em released by Sun Microsyst em s in 1984. I n fact , t he lat t er was t he original raison d'et re for t he SUN- RPC st andard, t he first Rem ot e Procedure Call syst em . Even t hough dist ribut ed com put ing was available as early as 1980, it was m ainly confined t o t he academ ic world unt il well int o t he 1990s. Unix com put ers m ainly act ed as so- called workst at ionspowerful com put at ional and visualizing enginesin research facilit ies and universit ies. I n t he business environm ent , a hub- and- spoke dist ribut ion m odel prevailed unt il well int o t he 1990s. A num ber of deskt op com put er syst em s would t ypically access a cent ral syst em for st orage and print ing. Oft en, t hese file servers used an ent irely different operat ing plat form from it s client s. As st ruct ured and relat ional dat abases becam e m ore m at ure, businesses adopt ed a client / server approach. A large chunk of t he applicat ion resided wit h t he client t hat rem ot ely accessed a dat abase server. Execut ion logic was split bet ween client and server as dat abases, m ost not ably Sybase, int roduced t he concept of funct ions t hat were execut ed wit hin t he dat abase and t hat did not need t o be shipped wit h t he client applicat ionso- called st ored procedures. Anot her rem arkable innovat ion was Novell's Net Ware Loadable Modules ( NLM) , which were program s t hat ran on t he server. The next logical st ep was t o blur t he dist inct ion bet ween t he client and t he server. Com bining concept s from dist ribut ed com put ing plat form s such as t he Dist ribut ed Com put ing Environm ent ( DCE) wit h t he newly em erging paradigm of obj ect - orient ed developm ent , CORBA ( Com m on

Obj ect Request Broker Archit ect ure) was creat ed. I nst ead of providing servers, which expose large num bers of rem ot ely accessible funct ions, t he funct ionalit y was now broken down int o uniquely ident ifiable, rem ot ely accessible obj ect s t hat were able t o m anage t heir own st at e. Different obj ect s could com m unicat e wit h each ot her by m eans of an Obj ect Request Broker ( ORB) . To connect t o t hese obj ect s, no knowledge of where t hey act ually reside is necessary. I nst ead, an ORB provides abst ract ion m echanism s, such as nam ing services, which t ake care of t he runt im e discovery of t he obj ect s. Sim ilarly t o obj ect - orient ed program m ing, CORBA em braced t he concept of program m ing by int erfacein fact , all CORBA obj ect s can be im plem ent ed in various program m ing languages, while t heir int erfaces are described using a com m on I nt erface Definit ion Language ( I DL) . Technically very elegant and sophist icat ed, CORBA prom ot ed t he act ual reuse of live obj ect s but lent it self t o a design wit h rat her sm all ent it ies. CORBA's vision of dist ribut ed business obj ect s never fully m at erialized because it s fine- grained m odel oft en proved t o be t oo com plex t o be suit able for t he purposes of ent erprise- level soft ware reuse. I n addit ion, CORBA program m ing is rat her dem anding for inexperienced developers because t hey m ust cope wit h m any com plex t echnical concept s, such as t he non- t rivial language m appings of t he CORBA I DL. However, CORBA is st ill a widely used dist ribut ion t echnology, especially in t elecom m unicat ions and financial services, alt hough we rarely see t he full feat ure- set of CORBA used in t hese applicat ions ( see Chapt er 9, "I nfrast ruct ure of a Service Bus," for m ore det ails) . As a result , t he evolut ion of dist ribut ed infrast ruct ures changed it s direct ion in t he m id 1990s, t aking t he lim it at ions of t he early dist ribut ed obj ect archit ect ures int o considerat ion. I n a new at t em pt , t he idea of clust ering a set of obj ect s int o a single server was developed t o provide a higher level of abst ract ion and t o increase t he richness of t he service t hat such a server could offer. Driven by t he high dem and for m ore sophist icat ed plat form s for I nt ernet applicat ions, Sun Microsyst em s int roduced Ent erprise Java Beans ( EJB) in 1997. Sim ilar t o CORBA, EJB also relies on a dist ribut ed obj ect m odel. However, t he EJB m odel is based on a cont rolledand t herefore usually lim it ednum ber of servers t hat host t he act ual obj ect s. EJB was inspired by CORBA but also by older applicat ion cont ainers and t ransact ion m onit ors, such as CI CS or Tuxedo. EJB cat ers for different t ypes of obj ect s, including dat a- cent ric ent it y beans, as well as session- orient ed obj ect s. For exam ple, st at eless session beans do not need t he concept of inst ance ident it y. Anot her st rengt h of EJB is t he cont ainer concept , which is responsible for t he m anagem ent of resources ( obj ect s, connect ions, t ransact ions, et c.) in an EJB server. Alt hough not a new concept ( t he core resource m anagem ent concept s found in EJB can be t raced back t o CI CS and ot her m ainfram e t ransact ion m onit ors) , EJB put a lot of effort int o m aking resource m anagem ent as t ransparent t o t he developer as possible. Finally, sim ilar t o ot her rem ot e com put ing plat form s, such as DCE and CORBA, EJB includes higher- level t echnical services, such as t ransact ion m anagem ent , nam ing services, and securit y. I n addit ion t o core- applicat ion rem ot ing t echnologies, such as RPC, CORBA, DCOM, and EJB, t he 1990s saw t he em ergence of a large num ber of addit ional dist ribut ed com put ing m iddleware solut ions, addressing dist ribut ed t ransact ion m anagem ent ( e.g., t he X/ open- based CORBA Obj ect Transact ion Service, t he Java Transact ion Service, and t he Microsoft Transact ion Server) , m essaging ( e.g., CORBA Not ificat ion and Java Messaging Service) , EAI ( Ent erprise Applicat ion I nt egrat ion, oft en including m essage rout ing and t ransform at ion, as well as applicat ion adapt ers) , securit y ( m ost not ably t he Secure Socket Layer) , and m any ot her problem areas. Alt hough t hey provided a great infrast ruct ure for t he developm ent of individual syst em s, t he sheer num ber of different dist ribut ed com put ing concept s, st andards, and product s also caused a problem : m iddleware het erogeneit y . Given t hat m ost of t he m iddleware was init ially developed t o address t he problem of applicat ion het erogeneit y , t his is an ironic developm ent , caused by t he fact t hat it proved t o be lit erally im possible t o im pose ent erprise- wide st andards for m iddleware in large ent erprises ( refer t o Sect ion 1.6 in Chapt er 1, " An Ent erprise I T Renovat ion Roadm ap" ) . As a result of t his developm ent , XML becam e popular in t he m id 1990s as a m iddlewareindependent form at for t he exchange of dat a and docum ent s bet ween different applicat ions. XML

is basically t he sm allest com m on denom inat or upon which t he I T indust ry could agree. Unlike CORBA I DL, Microsoft I DL, or Java int erfaces, XML is not bound t o a part icular t echnology or m iddleware st andard and is oft en used t oday as an ad- hoc form at for processing dat a across different , largely incom pat ible m iddleware plat form s. XML does not require a heavy- weight infrast ruct ure, such as an ORB, and it com es wit h a large num ber of t ools on m any different plat form s, which enables t he processing and m anagem ent of XML dat a, including different opensource parsing API s, such as SAX and DOM. XML's inherent flexibilit y posit ions it as t he m ost suit able st andard for solving t he applicat ion het erogeneit y problem , as well as t he m iddleware het erogeneit y problem . However, XML's great flexibilit y is pot ent ially also it s biggest problem because efficient applicat ion int egrat ion and dat a m anagem ent requires higher- level dat a st ruct ures and m essaging form at s. As a result , a plet hora of higher- level st andards t hat at t em pt t o address t hese issues has em erged in XML space. I t t ook a long t im e for people t o agree upon even t he core st andards for t he specificat ion of XML- based com plex dat a t ypes, such as XML DTDs and Schem as. Recognizing t he need for higher- level XML m essaging st andards on one hand, and at t em pt ing t o leverage t he ubiquit y of t he I nt ernet on t he ot her, engineers at Microsoft invent ed XML- based Web services wit h t he creat ion of SOAP ( Sim ple Obj ect Access Prot ocol) in 1998. The init ial SOAP version was specifically designed t o work on t he widely est ablished HTTP I nt ernet prot ocol, enabling server- t o- server com m unicat ion over t he infrast ruct ure t hat was already est ablished for browser- t o- server com m unicat ion. Given t hat t he est ablished I nt ernet infrast ruct ure already solved m any pressing problem s, such as securit y ( SSL, firewalls, access cont rol, et c.) , load balancing, failover, and applicat ion m anagem ent , t his seem ed like a logical next st ep. Using st andard HTTP POST and GET request , SOAP client s were able t o call funct ions using t he est ablished I nt ernet infrast ruct ure. Microsoft lat er also developed an int erface definit ion language for SOAP services called WSDL ( Web Service Definit ion Language) . WSDL describes t he service int erface, m uch as I DL describes t he obj ect int erface when using CORBA. Wit h t he problem of m iddleware het erogeneit y in m ind, SOAP and WSDL are designed t o enable t he definit ion of various bindings t o different lower- level com m unicat ion prot ocols, for exam ple, t o enable SOAP com m unicat ion over an exist ing m essaging m iddleware. SOAP was carried forward by I BM and Microsoft and becam e a W3C recom m endat ion in 2001. As we see lat er in t his book, XML- based Web services ( e.g., based on SOAP and WSDL) can be a great plat form for a Service- Orient ed Archit ect ure. However, it is im port ant t o keep in m ind t hat Web services are not t he only viable t echnology plat form for an SOA because t he SOA archit ect ural concept s are not dependent on a single t echnology plat form . The developm ent of dist ribut ed com put ing archit ect ures, such as DCE, CORBA, DCOM, EJB, and XML Web servicesand t he real- world experience won wit h it in large- scale applicat ion developm ent shas provided t he basis for t he concept s of Service- Orient ed Archit ect ures. Sim ilar t o obj ect orient at ion, which t oday seem s t o present t he endpoint in t he developm ent of program m ing concept s, service orient at ion is t he result of a long evolut ionary process and has t he pot ent ial t o finally provide som e st abilit y in an environm ent t hat has been const ant ly evolving over t he past 30 years.

2.4. Business Computing Alt hough t he evolut ion of program m ing languages and dist ribut ed com put ing event ually provided t he t echnical concept s t hat t oday are t he first cornerst one of Service- Orient ed Archit ect ures, it is equally im port ant t o look at t he developm ent s in business com put ing t hat provided t he cont ent t hat represent s t he second cornerst one of Service- Orient ed Archit ect ures: business dat a and business logic. The hist ory of com put ing was always closely relat ed t o solving business problem s, st art ing as early as 1940 wit h com put ers being used as large- scale calculat ors and as a replacem ent for large filing cabinet s. Funct ions t hat nowadays are considered " t echnical" provided im m ediat e business value in t he advent of com put ing. The following decades creat ed furt her levels of abst ract ion, m aking it easier t o t hink of a com put er in t erm s of a provider for business services. However, business com put ing m aint ained it s focus on m ainfram e com put ers. Most soft ware was cust om - built , originally writ t en in m achine language and lat er in funct ional languages such as COBOL or FORTRAN. Yet business com put ing proved a crucial success fact or for ent erprises. For exam ple, logist ics com panies used com put ers t o com put e rout es for shipm ent s t hrough t heir vast int ernat ional t ransport net works. Ret ail giant Wal- Mart was am ong t he first t o creat e cust om - m ade supply- chain m anagem ent syst em s t o opt im ize t he purchase and dist ribut ion of goods. I n t he 1970s, t he original hom egrown filing cabinet applicat ions were gradually replaced using fully fledged dat abase syst em s, including relat ional dat abases, which encapsulat e t he st orage of com plex int errelat ed dat a. Alt hough t oday we regard t he st orage m echanism it self as a t echnical concept , it seem ed fairly business- orient ed when it was creat ed. I n fact , SQL was developed as a language t hat was int ended t o be used m ainly by business analyst s, not dat abase program m ers. I n 1972, four form er I BM em ployees founded SAP in Germ any, an event t hat m arked a m ilest one for business com put ing. SAP int roduced R/ 2 in 1981, t he first business- com put ing plat form t hat enabled ent erprise- wide real t im e processing of financial dat a and resource planning inform at ion. However, by t he m id 1980s, corporat e soft ware developm ent seem ingly reached sat urat ion level. Most com panies were convinced t hat t hey had achieved m ost of what was possible wit h com put ing in t heir environm ent . College st udent s were discouraged from m aj oring in soft ware developm ent because people assum ed t hat only m aint enance would be needed in t he fut ure, which probably would be perform ed in som e rem ot e offshore locat ion. Then com put ers began t o reach em ployee deskt ops. Syst em s such as t he Com m odore PET and ot hers int roduced a new concept t o com put ing. Dat a was obt ained from rem ot e st orage, while com put at ion and visualizat ion were perform ed locally. This paradigm was boost ed by t he huge success of t he I BM PC, which was launched in 1984. Through several st ages of hardware and soft ware developm ent at bot h t he client and server sides, t he client / server paradigm held st eady. The focus for advancem ent shift ed back and fort h bet ween t he t wo. On one hand, client com put ers becam e m ore powerful, sport ing graphical user int erfaces and raw com put at ional power, and net works becam e fast er. On t he ot her hand, dat abase vendors worked hard t o add value t o t heir syst em s by providing fault t olerance, scalabilit y, and load balancing using clust er t echniques and procedures t hat were execut ed wit hin t he dat abase. Driven by an increasing econom ical globalizat ion and new m anufact uring m odels such as Just - inTim e product ion, t he supply and dist ribut ion chains of com panies becam e increasingly sophist icat ed, relying m ore on com plex I T syst em s t o m anage t hese processes. The soft ware

m arket react ed t o t his newly awakened dem and in ent erprise com put ing by developing com plex ent erprise applicat ions, such as Ent erprise Resources Planning ( ERP) and Supply Chain Managem ent ( SCM) . Over t wo decades, t he m arket for ent erprise solut ions has becom e increasingly com plex, offering applicat ions ranging from Cust om er Relat ionship Managem ent and Product Lifecycle Managem ent t o highly specialized applicat ions, such as applicat ions t hat m anage com plex t ransport at ion net works or billing syst em s for t elecom m unicat ions com panies. As a result , a plet hora of new ent erprise soft ware com panies em erged, and old ones grew even bigger, including SAP, Siebel, Oracle, PeopleSoft , J.D. Edwards, Baan, Manugist ics, and ot hers. Wit h t he em ergence of t he I nt ernet , m any of t hese com panies t ransform ed t heir ent erprise applicat ions t o m ake use of t he new t echnology ( e.g., m ySAP) and t o cat er for new business m odels in t he area of eCom m erce and B2B. New com panies, such as salesforce.com , em erged even aft er t he heydays of t he I nt ernet were over, t aking on est ablished com panies such as Siebel by delivering it s applicat ions ent irely over t he I nt ernet . However, t he availabilit y of new ent erprise soft ware oft en caused as m any problem s as it aim ed t o solve. Oft en, ERP, CRM, and Ent erprise Port al solut ions com pet ed wit hin a single organizat ion t o becom e t he cent ral reposit ories of business dat a and processes. Keeping in m ind t hat as lat e as 1990, m ost ent erprise I T syst em s served no m ore t han a single depart m ent , it com es as no surprise t hat m any challenges em erged on t he organizat ional level along wit h a huge dem and for Ent erprise Applicat ion I nt egrat ion ( EAI ) . However, suffering from problem s sim ilar t o t he Ent erprise Soft ware Bus m odel, t he EAI - t ypical hub- and- spoke m odel was oft en lim it ed t o solve t he problem s of individual applicat ion int egrat ion proj ect s, failing t o deliver a m ore holist ic view t o t he problem s of having t o int egrat e across organizat ional boundaries. An ent erprise SOA aim s t o leverage t he business logic and dat a t hat resides in t he m any applicat ions, dat abases, and legacy syst em s of t oday's ent erprises, offering t hem a sust ainable st rat egy for flexibly adapt ing t heir I T syst em s t o changes in funct ional requirem ent s and business st rat egy wit hout im posing ont o t hem t he use of a single m iddleware or EAI plat form . The services exposed in an SOA are int ended t o be m apping direct ly t o business ent it ies, t hus enabling ent erprise int egrat ion on t he business level, not t he t echnical level.

2.5. Conclusion This chapt er looked at t he hist orical developm ent s of program m ing languages, dist ribut ion t echnology, and business com put ing and how each of t hese areas cont ribut ed t o t he developm ent of service orient at ion. The evolut ion of program m ing languages has not only provided us wit h m ore product ive developm ent plat form s but has also significant ly cont ribut ed t o t he underst anding of int erfacing t echniques and access pat t erns for services in an SOA. A key lesson learned is t hat not all program m ing language concept s are suit able in dist ribut ed com put ing and t hat service orient at ion is a deliberat e st ep back from obj ect orient at ion, aim ing t o provide m ore coarse- grained com ponent s wit h sim pler access pat t erns. The evolut ion of dist ribut ion t echnology has given us a variet y of rem ot e access t echnologies from which we can choose t oday, t oget her wit h an infrast ruct ure for t ransact ion m anagem ent , securit y, load- balancing, failover, and ot her crit ical feat ures. Finally, t he evolut ion of business com put ing has lead t o t he developm ent of advanced ent erprise applicat ions, such as ERP and CRM, which are t oday providing t he cont ent t hat represent s t he second cornerst one of an SOAt he dat a and business logic t hat brings our services t o life.

References [ Myers76] Myers, Glenford J. Com posit e/ St ruct ured Design. Van Nost rand Reinhold Co. 1976. [ Gold83] Goldberg, Adele and David Robson . Sm allt alk- 80: The Language and it s I m plem ent at ion. Addison- Wesley. 1983. [ Kirk89] Kirkerud, Bj orn . Obj ect - Orient ed Program m ing wit h Sim ula. Addison- Wesley. 1989. [ St ro85] St roust rup, Bj arne . The C+ + Program m ing Language. Addison- Wesley. 1991.

URLs ht t p: / / www.m - w.com

Chapter 3. Inventory of Distributed Computing Concepts Before exam ining SOA elem ent s in det ail in t he following chapt ers, we will review exist ing concept s of dist ribut ed com put ing. This is im port ant because we are not planning t o develop an SOA from scrat ch. I nst ead, an SOA will have t o incorporat e exist ing m iddleware t echnologies and dist ribut ed com put ing concept s. This is part icularly im port ant because earlier at t em pt s t o replace exist ing m iddleware wit h a new, ubiquit ous soft ware bus ( e.g., CORBA) have failed, and a successful SOA will have t o em brace exist ing and upcom ing t echnologies inst ead of replacing or precluding t hem . Many aut hors cover t he int rinsic det ails of com m unicat ion net works and m iddleware, such as Tanenbaum [ Tan2002, Tan2003] and Coulouris [ Cou2001] . Aim ing our discussion at t he applicat ion archit ect ure level, we will provide only a brief overview of t he m ost fundam ent al com m unicat ion m iddleware concept s here ( including Rem ot e Procedure Calls, Dist ribut ed Obj ect s, and Message- Orient ed Middleware) , followed by a m ore det ailed discussion on t he im pact t hat different t ypes of com m unicat ion m iddleware have on t he applicat ion level ( including synchrony, invocat ion sem ant ics, and applicat ion coupling) .

3.1. Heterogeneity of Communication Mechanisms Techniques for t he dist ribut ion of ent erprise soft ware com ponent s are m anifold. As will be seen in t he rem ainder of t his book, t his het erogeneit y is inevit able due t o t he various com m unicat ion requirem ent s of ent erprises. The sit uat ion is com parable t o com m unicat ion in real lifem any form s of com m unicat ion exist ( verbal, non- verbal, writ t en, et c.) , and every form has it s own purpose. I t is not possible t o replace one form wit h anot her wit hout reducing expressiveness. Figure 3- 1 depict s t hree possible levels of het erogeneit y of dist ribut ion t echniques: Com m unicat ion m ode Product s Addit ional runt im e feat ures

Figu r e 3 - 1 . D ist r ibu t ion t e ch n iqu e s for e n t e r pr ise a pplica t ion s a r e ch a r a ct e r ize d by m a n ifold r e qu ir e m e n t s a n d con se qu e n t ly by va r iou s dim e n sion s of h e t e r oge n e it y. [View full size image]

Com m unicat ion m odes are basically dist inguished bet ween synchronous and asynchronous m echanism s. Evident ly bot h are required in real- world proj ect s. However, in pract ice, t here are usually num erous variant s of t hese basic m odes of com m unicat ion. Obviously, one can encount er num erous product s t hat provide dist ribut ion m echanism s. I n addit ion, a concept t hat is supposed t o cover all t he dist ribut ion issues of an ent erprise m ust also provide a set of addit ional runt im e feat ures such as securit y support , fault t olerance, load balancing, t ransact ion handling, logging, usage m et ering, and audit ing. I t should be not ed t hat our classificat ion schem e is arbit rary. I t is possible t o define ot her classificat ions or t o find addit ional levels of het erogeneit y. However, independent of t he classificat ion schem e, it is t rue t hat ent erprise dist ribut ion t echniques t end t o creat e het erogeneit y at different levels. From a t echnical point of view, t his scenario leads t o t hree different layers, as shown in Figure 32. The first layer cont ains t he core asset s of t he ent erprise applicat ion landscape, including all business logic. The second layer provides t echnology- dependent adapt ers t hat connect t he core asset s t o various soft ware busses. Finally, t he t hird layer represent s t he sum of t he ent erprise's com m unicat ion facilit ies.

Figu r e 3 - 2 . Te ch n ology- de pe n de n t a da pt e r s con n e ct pa r t icipa n t s of a n e n t e r pr ise a pplica t ion la n dsca pe w it h it s com m u n ica t ion in fr a st r u ct u r e .

The rem ainder of t his chapt er focuses on t he second layer. Chapt ers 4 t o 7 provide an in- dept h discussion of t he first layer, while Chapt er 9 discusses t he t hird layer.

3.2. Communication Middleware A com m unicat ion m iddleware fram ework provides an environm ent t hat enables t wo applicat ions t o set up a conversat ion and exchange dat a. Typically, t his exchange of dat a will involve t he t riggering of one or m ore t ransact ions along t he way. Figure 3- 3 shows how t his m iddleware fram ework act s as an int erm ediary bet ween t he applicat ion and t he net work prot ocol.

Figu r e 3 - 3 . A com m u n ica t ion m iddle w a r e fr a m e w or k isola t e s t h e a pplica t ion de ve lope r s fr om t h e de t a ils of t h e n e t w or k pr ot ocol.

I n t he very early days of dist ribut ed com put ing, t he com m unicat ion bet ween t wo dist ribut ed program s was direct ly im plem ent ed based on t he raw physical net work prot ocol. Program m ers were involved wit h acut e det ails of t he physical net work. They had t o creat e net work packet s, send and receive t hem , acknowledge t ransm issions, and handle errors. Therefore, a lot of effort was spent on t hese t echnical issues, and applicat ions were dependent on a specific t ype of net work. Higher- level prot ocols such as SNA, TCP/ I P, and I PX provided API s t hat helped reduce t he im plem ent at ion effort s and t echnology dependencies. They also provided abst ract ion and a m ore com fort able applicat ion developm ent approach. These prot ocols enabled program m ers t o t hink less in t erm s of fram es at OSI layer 2 or packet s at layer 3 and m ore in t erm s of com m unicat ion sessions or dat a st ream s. Alt hough t his was a significant sim plificat ion of t he developm ent of dist ribut ed applicat ions, it was st ill a cum bersom e and error- prone process. Program m ing at t he prot ocol layer was st ill t oo low- level. As t he next evolut ionary st ep, com m unicat ion infrast ruct ures encapsulat ed t he t echnical com plexit y of such low- level com m unicat ion m echanism s by insulat ing t he applicat ion developer from t he det ails of t he t echnical base of t he com m unicat ion. A com m unicat ion m iddleware fram ework enables you t o access a rem ot e applicat ion wit hout knowledge of t echnical det ails such as operat ing syst em s, lower- level inform at ion of t he net work prot ocol, and t he physical net work address. A good m iddleware fram ework increases t he flexibilit y, int eroperabilit y, port abilit y, and m aint ainabilit y of dist ribut ed applicat ions. However, it is t he experience of t he recent t wo decades t hat t he developer's awareness of t he dist ribut ion is st ill crucial for t he efficient im plem ent at ion of a dist ribut ed soft ware archit ect ure. I n t he rem ainder of t his chapt er, we will briefly exam ine t he m ost im port ant com m unicat ion m iddleware fram eworks.

3.2.1. RPC Rem ot e Procedure Calls ( RPCs) apply t he concept of t he local procedure call t o dist ribut ed applicat ions. A local funct ion or procedure encapsulat es a m ore or less com plex piece of code and m akes it reusable by enabling applicat ion developers t o call it from ot her places in t he code. Sim ilarly, as shown in Figure 3- 4 , a rem ot e procedure can be called like a norm al procedure, wit h t he except ion t hat t he call is rout ed t hrough t he net work t o anot her applicat ion, where it is execut ed, and t he result is t hen ret urned t o t he caller. The synt ax and sem ant ics of a rem ot e call rem ain t he sam e whet her or not t he client and server are locat ed on t he sam e syst em . Most RPC im plem ent at ions are based on a synchronous, request - reply prot ocol, which involves blocking t he client unt il t he server replies t o a request .

Figu r e 3 - 4 . RPC st u bs a n d libr a r ie s e n a ble loca t ion t r a n spa r e n cy, e n ca psu la t e t h e fu n ct ion a l code for t h e RPC com m u n ica t ion in fr a st r u ct u r e , a n d pr ovide a pr oce du r e ca ll in t e r fa ce .

The developm ent of t he RPC concept was driven by Sun Microsyst em s in t he m id 1980s and is specified as RFC prot ocols 1050, 1057, and 1831. A com m unicat ion infrast ruct ure wit h t hese charact erist ics is called RPC- st yle, even if it s im plem ent at ion is not based on t he appropriat e RFCs. I t is int erest ing t o not e t hat t he need t o provide plat form - independent services was one of t he m ain drivers in t he developm ent of RPC- st yle prot ocols. Part icularly, t he widely used SUN RPC prot ocol ( RFC 1057) and it s language bindings were developed t o enable t ransparent access t o rem ot e file syst em s. NFS ( RFC 1094) was im plem ent ed on t op of SUN RPC and is one of t he m ost popular ways t o enable net worked file syst em access in Unix- like environm ent s. At t he end of t he 1980s, DCE ( Dist ribut ed Com put ing Environm ent ) em erged as an init iat ive t o st andardize t he various com pet ing rem ot e procedure call t echnologies. DCE also adds som e higher- level services such as securit y and nam ing services. However, for reasons t hat were m ainly polit ical, DCE failed t o win widespread indust ry support .

3.2.2. DISTRIBUTED OBJECTS

I n t he early 1990s, obj ect - orient ed program m ing em erged as a replacem ent for t he t radit ional m odular program m ing st yles based on procedure or funct ion calls. Consequent ly, t he concept of Dist ribut ed Obj ect s was invent ed t o m ake t his new program m ing paradigm available t o developers of dist ribut ed applicat ions. Typically, Dist ribut ed Obj ect s are support ed by an Obj ect Request Broker ( ORB) , which m anages t he com m unicat ion and dat a exchange wit h ( pot ent ially) rem ot e obj ect s. ORBs are based on t he concept of I nt eroperable Obj ect References, which facilit at e t he rem ot e creat ion, locat ion, invocat ion, and delet ion of obj ect s ( see Figure 3- 5 ) oft en involving obj ect fact ories and ot her helper obj ect s. By doing so, ORB t echnology provides an obj ect - orient ed dist ribut ion plat form t hat prom ot es obj ect com m unicat ion across m achine, soft ware, and vendor boundaries. ORBs provide locat ion t ransparency and enable obj ect s t o hide t heir im plem ent at ion det ails from client s.

Figu r e 3 - 5 . ORBs e n a ble clie n t a pplica t ion s t o r e m ot e ly cr e a t e , loca t e , a n d de le t e se r ve r obj e ct s ( e .g., t h r ou gh fa ct or y obj e ct s) a n d com m u n ica t e w it h t h e m t h r ou gh r e m ot e m e t h od in voca t ion s.

The m ost com m on ORB im plem ent at ions are CORBA, COM/ DCOM, and RMI . While RMI is lim it ed t o Java and COM/ DCOM is rest rict ed t o Microsoft plat form s, CORBA spans m ult iple plat form s and program m ing languages. Today, m ost ent erprise archit ect ures em body obj ect - orient ed com ponent s ( such as for t he im plem ent at ion of graphical user int erfaces) , but t here is rarely an ent erprise archit ect ure t hat is built purely on obj ect t echnology. I n m ost cases, legacy applicat ions based on program m ing languages such as COBOL or C are crit ical part s of an ent erprise applicat ion landscape ( som e people prefer t he t erm soft ware asset s over legacy soft ware) . I t is t herefore vit al t hat a com ponent t hat should be reused on an ent erprise- wide level provides an int erface t hat is suit able bot h for obj ect - orient ed and t radit ional client s. This is part icularly t rue for t he service of an SOA.

I n t his cont ext , it is im port ant t o underst and t hat obj ect - orient ed applicat ions t ypically com e wit h a fine- grained int eract ion pat t ern. Consequent ly, applying t he obj ect - orient ed approach t o building dist ribut ed syst em s result s in m any rem ot e calls wit h lit t le payload and oft en very com plex int eract ion pat t erns. As we will see lat er, service- orient ed syst em s are m ore dat acent ric: They produce fewer rem ot e calls wit h a heavier payload and m ore sim ple int eract ion pat t erns. Nevert heless, it is ent irely possible t o use ORB t echnology t o im plem ent a dat a- orient ed, coarsegrained SOA int erface. This leads t o a very rest rict ed applicat ion of t he ORB t echnology and t ypically t o an RPC- st yle usage of t he ORB ( see Figure 3- 6 ) .

Figu r e 3 - 6 . An ORB ca n be u se d a s a com m u n ica t ion in fr a st r u ct u r e for t h e im ple m e n t a t ion of a n SOA. I n t h is ca se , t h e a dva n ce d ca pa bilit ie s of t h e ORB t o cope w it h m u lt iple in st a n ce s of r e m ot e obj e ct s a r e n ot u se d.

3.2.3. MOM Wit h t he advent of I BM's MQSeries ( now I BM WebSphere MQ) and Tibco Soft ware's Rendezvous in t he m iddle of t he 1990s, Message- Orient ed Middleware ( MOM) t echnology becam e popular, and it has since becom e an int egral part of t he com m unicat ion infrast ruct ure landscape of large ent erprises. Alt hough t here are alt ernat ive im plem ent at ion approaches ( e.g., UDP m ult icast - based syst em s) , t he m ost com m on MOM im plem ent at ions are based on t he concept of m essage queuing. The t wo key com ponent s of a m essage queuing syst em are m essage and queue. Typically, a m essage consist s of a header and a payload. The st ruct ure of t he header field is

usually predefined by t he syst em and cont ains net work rout ing inform at ion. The payload is applicat ion- specific and cont ains business dat a, possibly in XML form at . Messages t ypically relat e t o a specific t ransact ion t hat should be execut ed upon receiving t he m essage. Depending on t he queuing syst em , t he nam e of t he t ransact ion is eit her part of t he header or t he applicat ion payload. The queue is a cont ainer t hat can hold and dist ribut e m essages. Messages are kept by t he queue unt il one or m ore recipient s have collect ed t hem . The queue act s as a physical int erm ediary, which effect ively decouples t he m essage senders and receivers. Message queues help t o ensure t hat m essages are not lost , even if t he receivers are m om ent arily unavailable ( e.g., due t o a net work disconnect ion) . Em ail is a good exam ple of t he applicat ion of m essaging concept s. The em ail server decouples sender and receiver, creat ing durable st orage of em ail m essages unt il t he receiver is able t o collect t hem . Em ail m essages cont ain a header wit h inform at ion t hat enables t he em ail t o be rout ed from t he sender's em ail server t o t he receiver's em ail server. I n addit ion, em ail m essages can be sent from a single sender t o a single receiver ( or t o m ult iple recipient s, t hrough m ailing list s for exam ple) , and one can receive em ail from m ult iple senders. Message queuing syst em s provide sim ilar concept s of connect ing senders and receivers in different ways ( one- t o- one, one- t o- m any, m any- t o- m any, et c.) ( see Figure 3- 7 ) . The underlying t echnical concept s are t ypically referred t o as point - t o- point and publish- subscribe m odels. Point t o- point represent s t he m ost basic m essaging m odel: One sender is connect ed t o one receiver t hrough a single queue. The publish- subscribe m odel offers m ore com plex int eract ions, such as one- t o- m any or m any- t o- m any. Publish- subscribe int roduces t he concept of t opics as an abst ract ion, t o enable t hese different t ypes of int eract ions. Sim ilar t o point - t o- point , a sender can publish m essages wit h a t opic wit hout knowing anyt hing about who is on t he receiving side. Cont rary t o point - t o- point com m unicat ions, in t he publish- subscribe m odel, t he m essage is dist ribut ed not t o a single receiver, but t o all receivers who have previously indicat ed an int erest in t he t opic by regist ering as subscribers.

Figu r e 3 - 7 . M OM de cou ple s t h e cr e a t or s a n d con su m e r s of m e ssa ge s pr ovidin g con ce pt s su ch a s poin t - t o- poin t m e ssa gin g a n d pu blish su bscr ibe .

Alt hough t he basic concept s of m essage queuing syst em s ( m essage, queue, and t opic) are relat ively sim ple, a great deal of com plexit y lies in t he m any different ways t hat such a syst em can be configured. For exam ple, m ost m essage queuing syst em s enable t he int erconnect ion of m ult iple physical queues int o one logical queue, wit h one queue m anager on t he sender side and anot her on t he receiver side, providing bet t er decoupling bet ween sender and receiver. This is sim ilar t o em ail, where senders t ransm it em ail t o t heir own m ail server, which in t urn rout es t he m ail t o t he receiver's m ail server, a process t hat is t ransparent t o t he users of em ail. Furt herm ore, queues can be int erconnect ed t o form net works of queues, som et im es wit h int elligent rout ing engines sit t ing bet ween t he different queues, creat ing event - driven applicat ions t hat em ploy logic sim ilar t o t hat of Pet ri net s [ Rei1992] . Message queuing syst em s t ypically also provide a num ber of different service levels ( QoSqualit y of service) , eit her associat ed wit h specific m essages or specific queues. These service levels det erm ine, for exam ple, t he t ransact ional capabilit y of t he m essage, t he send/ receive acknowledge m odes, t he num ber of allowable recipient s, t he lengt h of t im e a m essage is valid, t he t im e at which t he m essage was sent / received, t he num ber of t im es t o at t em pt redelivery, and t he priorit y of t he m essage, relat ive t o ot her m essages. Generally, MOM encourages loose coupling bet ween m essage consum ers and m essage producers, enabling dynam ic, reliable, flexible, high- perform ance syst em s t o be built . However, one should not underest im at e t he underlying com plexit y of ensuring t hat MOM- based syst em s work efficient ly, a feat ure t hat is not oft en visible at t he out set .

3.2.4. TRANSACTION MONITORS Wit h t he rising dem and for user- friendly online applicat ions in t he 1980s, t ransact ion m onit ors[ 1] becam e popular. They provide facilit ies t o run applicat ions t hat service t housands of users. I t is

t he responsibilit y of a t ransact ion m onit or t o efficient ly m ult iplex t he requirem ent s for com put ing resources of m any concurrent client s t o resource pools. Most im port ant ly, t hey m anage CPU bandwidt h, dat abase t ransact ions, sessions, files, and st orage. Today, t ransact ion m onit ors also provide t he capabilit y t o efficient ly and reliably run dist ribut ed applicat ions. Client s are t ypically bound, serviced, and released using st at eless servers t hat m inim ize overhead by em ploying a non- conversat ional com m unicat ion m odel. Furt herm ore, up- t o- dat e t ransact ion m onit ors include services for dat a m anagem ent , net work access, aut horizat ion, and securit y. [1]

Often referred to as TP monitors, TPMs, or TX monitors.

Popular exam ples of t ransact ion m onit ors are CI CS ( Cust om er I nform at ion Cont rol Syst em ) , I MS ( I nform at ion Managem ent Syst em ) , Encina, or Tuxedo, which all provide facilit ies for rem ot e access. A variet y of different dist ribut ion concept s can be found t o support t he part icular st rengt hs of respect ive t ransact ion m onit ors. Alt hough it is not t he int ent ion of t his book t o discuss current t echnology and product s in det ail, a short glance at I BM's CI CS and I MS can provide useful insight s. CI CS is a t im e- sharing syst em . Alt hough m ore t han 20 years old, it is st ill a key elem ent of I BM's ent erprise product st rat egy. I t is probably t oday's m ost im port ant runt im e environm ent for m ission- crit ical ent erprise applicat ions. Even t oday, t here are new applicat ions developed for CI CS. Nat ive prot ocols such as SNA or TCP/ I P and various com m unicat ion infrast ruct ures such as obj ect brokers and m essaging m iddleware can be used t o int egrat e CI CS applicat ions wit h non- CI CS applicat ions [ Bras2002] . One of t he m ost com m on ways t o connect t o a CI CS applicat ion is CI CS's Ext ernal Call I nt erface ( ECI ) . The ECI basically provides an RPC- like library t hat enables rem ot e applicat ions t o invoke CI CS t ransact ion program s. Based on t he ECI , t he CI CS Transact ion Gat eway ( CTG) provides an obj ect - orient ed int erface for Java. Cont rary t o t he CI CS t im e- sharing concept , it s predecessor I MS was based on processing queues. Alt hough I MS appears t o be archaic, it is st ill im port ant in pract ice due t o it s base of inst alled t ransact ion program s. There are also m any different ways t o rem ot ely invoke an I MS t ransact ion program [ Bras2002] . The m ost popular ways are I MS Connect and MQSeries OTMA- Bridge [ Lon1999] . While I MS Connect im poses an RPC- like access t o I MS t ransact ion program s, t he MQSeries OTMA- Bridge is based on t he MOM concept .

3.2.5. APPLICATION SERVERS Wit h t he boom ing dem and for Web applicat ions in t he dot - com era of t he lat e 1990s, t he applicat ion server becam e ext rem ely popular. An applicat ion server m ediat es bet ween a Web server and backend syst em s, such as dat abases or exist ing applicat ions. Request s from a client 's Web browser are passed from t he Web server t o t he applicat ion server. The applicat ion server execut es code t hat gat hers t he necessary inform at ion from ot her syst em s and com poses an HTML reply, which is ret urned t o t he client 's browser. An applicat ion server can be sim ple, such as Microsoft ASP ( Act ive Server Pages) , which com es wit h I I S ( I nt ernet I nform at ion Server) , or t hey can be com plex and expensive syst em s t hat im plem ent load balancing, dat a m anagem ent , caching, t ransact ion m anagem ent , and securit y. The basic funct ions of an applicat ion server can be described as host ing com ponent s, m anaging connect ivit y t o dat a sources, and support ing different t ypes of user int erfaces, such as t hin Web int erfaces or fat client applicat ions. Taking a closer look at t he basic m echanism s of an applicat ion server, one can som et im es get t he im pression t hat not m uch has changed from t he days of I BM CI CS and VT3270 t erm inals, only t hat t hese days, t he user int erfaces are m ore colorful. When looking at t he high end of applicat ion servers, not ably Microsoft .NET Server, BEA WebLogic, and I BM WebSphere, it can som et im es be difficult t o find out exact ly what is st ill part of t he core applicat ion server funct ionalit y because t hese com panies have st art ed t o use t heir respect ive brands t o describe a very broad array of product s. For exam ple, J2EE, t he applicat ion

server fram ework from t he non- Microsoft cam p, st art ed wit h core applicat ion server funct ionalit y including JSP ( Java Server Pages) for t he dynam ic generat ion of HTML and EJB ( Ent erprise Java Beans) for m anaging and host ing m ore com plex soft ware com ponent s. Wit hin a couple of years, J2EE has becom e a com plex and sophist icat ed fram ework ( see Figure 3- 8 ) .

Figu r e 3 - 8 . J2 EE com pr ise s a va r ie t y of st a n da r ds.

3.3. Synchrony Synchronous and asynchronous com m unicat ions are t wo different form s of int eract ion t hat bot h require t he support of a generic t echnology for dist ribut ed syst em s. Synchronous com m unicat ion is charact erized by t he im m ediat e responses of t he com m unicat ion part ners. The com m unicat ion follows a request / reply pat t ern t hat enables t he free flow of conversat ion, oft en based on t he use of busy wait s. Applicat ions wit h user int eract ion specifically require t his conversat ional m ode of int eract ion. Synchronous com m unicat ion requires t hat t he client and server are always available and funct ioning. Asynchronous com m unicat ion is less st ringent . Bot h com m unicat ion part ners are largely decoupled from each ot her, wit h no st rict request / reply pat t ern. Typically, one part y creat es a m essage t hat is delivered t o t he recipient by som e m ediat or, and no im m ediat e response is needed. The sender can st ore cont ext inform at ion and ret rieve it when t he recipient ret urns t he call, but t here is not necessarily a response. I n cont rast t o a synchronous request - reply m echanism , asynchronous com m unicat ion does not require t he server t o be always available, so t his t ype can be used t o facilit at e high- perform ance m essage- based syst em s. Typically, synchronous com m unicat ion is im plem ent ed by RPC- st yle com m unicat ion infrast ruct ures, while asynchronous m echanism s are im plem ent ed by MOM. However, it is ent irely possible t o im plem ent synchronous com m unicat ion based on MOM, and it is also possible t o build MOM- st yle int eract ion over RPC. Nevert heless, RPC is m ore suit able if im m ediat e responses are required, and MOM is t he t echnology of choice for decoupled, asynchronous com m unicat ion. Due t o t he m anifold requirem ent s of m ost real- world scenarios, t ypical ent erprise syst em s em body bot h synchronous and asynchronous com m unicat ion. For t his purpose, a variet y of different com m unicat ion infrast ruct ures is used, ranging from sim ple FTP ( File Transfer Prot ocol) t o m ore advanced m iddleware plat form s, such as RPC and MOM. I n addit ion, t here are also com m unicat ion infrast ruct ures t hat support bot h com m unicat ion m odesfor exam ple, pipelining RPC, which support s asynchronous com m unicat ion in addit ion t o t he st andard synchronous RPC com m unicat ion. To conclude t his discussion on synchrony, we will provide an overview of t he m ost com m on ways of im plem ent ing synchronous and asynchronous com m unicat ion wit h bot h RPC/ ORB and MOM t echnology. We will look at t he following exam ples: Sim ulat ed synchronous services wit h queues Asynchronous one- way: fire- and- forget RPC Callbacks and polling services The first exam ple, sim ulat ed synchronous com m unicat ion, can oft en be found in m ainfram e environm ent s, where a m essage queuing syst em has been chosen as t he st andard m eans for rem ot e int eract ion wit h t he host , such as OS/ 390 wit h MQSeries access. This is a com m on scenario in m any large ent erprises. Oft en, t hese com panies have gone one st ep furt her, developing fram eworks on t op of t his com binat ion of OS/ 390 and MQSeries t hat enable servicelike int erfaces t o t he m ost widely used t ransact ions on t he m ainfram e. This is done by im plem ent ing client - service wrappers t hat shield t he underlying MQ infrast ruct ure from t he client developer. These service wrappers oft en sim ulat e synchronous int eract ions wit h t he m ainfram e

by com bining t wo underlying queues, one wit h request sem ant ics and t he ot her wit h reply sem ant ics, using correlat ion I Ds t o pair m essages int o request / reply t uples. Effect ively, t his relegat es t he m essage queuing syst em t o playing a low- level t ransport funct ion only, not generally leveraging any of t he advanced feat ures of t he m essaging syst em . Figure 3- 9 provides an overview of t his approach.

Figu r e 3 - 9 . Sim u la t e d syn ch r on ou s se r vice s w it h qu e u e s. A cor r e la t ion I D m a ps a r e ply m e ssa ge t o t h e cor r e spon din g r e qu e st . On t h e clie n t side , t h is is h idde n by a se r vice w r a ppe r , w h ich give s t h e ca lle r t h e im pr e ssion of syn ch r on y. [View full size image]

The second exam ple, fire- and- forget RPC, assum es an RPC or ORB im plem ent at ion wit h asynchronous one- way sem ant ics: The client fires off a request t o t he server wit hout expect ing an answer. This can be achieved eit her by defining an operat ion signat ure t hat does not include any ret urn values or by using specific feat ures of t he m iddleware, such as a CORBA I DL operat ion using t he keyword oneway. Figure 3- 10 provides an overview of t his approach.

Figu r e 3 - 1 0 . A syn ch r on ou s on e - w a y ca ll im plie s fir e - a n d- for ge t se m a n t ics. Th e r e qu e st is fir e d off by t h e clie n t w it h ou t a r e ply fr om t h e se r ve r .

There are t wo key issues wit h t his approach: The first is t hat t he client has no guarant ee t hat t he server will receive and process t he request appropriat ely. This problem reduces t he applicabilit y of t his m et hod significant ly. The second problem is t hat m ost RPCs/ ORBs t ypically use a reliable com m unicat ion prot ocol such as TCP. Sending a one- way request t hrough TCP generally m eans t hat t he client is blocked unt il delivery t o t he server on t he TCP level has been com plet ed. I f a server is get t ing swam ped wit h request s, it m ight becom e unable t o process all incom ing one- way request s on t he TCP layer. Effect ively, t his m eans t hat t he client is blocked unt il t he server is at least able t o read t he request from t he net work. Therefore, it is not advisable t o use t his approach t o im plem ent large- scale event not ificat ion. I nst ead, an appropriat e MOM should be

chosen. The t hird exam ple, callbacks and polling services, is t he m ost com m on way of decoupling client s and server in RPC- st yle applicat ions, wit hout having t o m ove t o a fully fledged MOM solut ion. The basic idea is sim ilar t o t he convent ional callback, as it is realized in funct ional program m ing languages wit h funct ion point ers, or in OO languages using obj ect references: A client sends a request t o t he server, and t he server st ores t he request and ret urns cont rol back t o t he client ( possibly sending an acknowledgm ent t hat it received t he request ) . Aft er having processed t he request , t he server ( now act ing as a client ) sends t he result back t o t he client ( now act ing as a server) , using t he st andard RPC/ ORB invocat ion m echanism ( see Figure 3- 11) . Som et im es, it is not possible for t he client t o act as a server ( e.g., due t o firewall rest rict ions) . I n t hese cases, t he client can periodically poll t he server for t he availabilit y of t he result .

Figu r e 3 - 1 1 . Ca llba ck s a n d pollin g se r vice s: A clie n t se n ds a r e qu e st t o a se r ve r ( " t r igge r " ) . Th e se r ve r st or e s t h e r e qu e st e d a ct ivit y in a da t a ba se be for e r e plyin g w it h a n a ck n ow le dgm e n t t o t h e clie n t . Th e se r ve r h a s a t h r e a d t h a t t a k e s pe n din g r e qu e st s fr om t h e da t a ba se , pr oce sse s t h e m , a n d se n ds ba ck t h e r e su lt t o t h e or igin a t in g clie n t u sin g ca llba ck .

This approach can som et im es provide a good com prom ise for sat isfying t he need t o decouple client s and servers wit hout having t o add t echnology such as a MOM. However, oft en t he im plem ent at ion of such a rem ot e callback can be m ore difficult t han it originally appears. This is especially t rue if t he applicat ion requires a high degree of reliabilit y. I n t hese cases, it is necessary t o int roduce som e kind of m echanism for ensuring reliabilit y, such as t hrough com bining a serverside dat abase wit h som e kind of cust om - built acknowledgm ent prot ocol. Also, t he server- side logic can becom e quit e com plex: To ensure t hat all request s are event ually processed, t he dat abase m ust be const ant ly polled for pending request s, pot ent ially adding a huge burden on dat abase perform ance. For t his reason, one should carefully weigh t he use of dat abase t riggers. Here, it is im port ant t o ensure t hat t he execut ion of t he t rigger is not part of t he sam e t ransact ion t hat put s t he new request in t he dat abase. I n t his case, you could encount er a sit uat ion where t he client is blocked because it has t o wait not only unt il t he server has st ored t he request in t he dat abase before ret urning an acknowledgm ent t o t he client , but also unt il t he dat abase t rigger has been execut ed. This will effect ively elim inat e t he decoupling effect of t he callback im plem ent at ion. As shown in Figure 3- 12, t he server- side im plem ent at ion can alt ernat ively use int ernal m essage queues t o ensure an efficient m eans of st oring incom ing request s in a reliable and efficient m anner, t hus avoiding m any of t he issues wit h t he pure- dat abase approach described previously.

Figu r e 3 - 1 2 . Ca llba ck s a n d qu e u e s. Sim ila r t o t h e pr e viou s e x a m ple , e x ce pt t h a t qu e u e s a r e in t r odu ce d on t h e se r ve r side t o e n su r e be t t e r de cou plin g on t h e se r ve r side .

3.4. Interface Versus Payload Semantics Typically, an int eract ion bet ween a client and a server ( or a sender and a receiver) result s in t he execut ion of a t ransact ion ( or som e ot her act ivit y) on t he receiving end. I n order t o det erm ine t he t ype of t ransact ion or act ivit y t hat was request ed by t he caller ( or sender) , it is necessary t o specify t he operat ion. This is norm ally perform ed in one of t wo ways: The request ed t ransact ion/ act ivit y can be encoded in t he operat ion signat ure of t he server com ponent 's int erface, or it can be em bedded in t he m essage it self. I n t he first case, t he request ed t ransact ion ( or ot her kind of act ivit y) is defined by using selfdescript ive funct ion nam es such as saveCustomer() , retrieveCustomer() , or transferMoney() . RPC- st yle int erfaces provide t his t ype of sem ant ically rich int erface, which we refer t o as int erface sem ant ics ( see Figure 3- 13 ) .

Figu r e 3 - 1 3 . RPC- st yle in t e r a ct ion is t ypica lly ba se d on in t e r fa ce se m a n t ics. Eve r y pr oce du r e ca ll h a s a m e a n in gfu l n a m e t h a t in dica t e s it s pu r pose .

I n t he second case, t he request ed t ransact ion is em bedded direct ly int o t he m essage ( see Figure 3-14 ) . This can be done as part of t he m essage header ( if t he MOM provides such a field as part of t he m essage header dat a st ruct ure) , or as part of t he applicat ion specific payload. We refer t o t his as payload sem ant ics .

Figu r e 3 - 1 4 . Th e n a m e of a r e m ot e ca ll w it h pa yloa d se m a n t ics h a s n o fu n ct ion a l m e a n in g in it s ow n r igh t . Th e r e m ot e fu n ct ion a lit y r e qu ir e d is e n code d in t h e m e ssa ge t h a t is se n t . Th e r e ce ive r t ypica lly h a s t o de t e r m in e t h e fu n ct ion n a m e a n d t h e dispa t ch m e ssa ge t o t h e a ssocia t e d bu sin e ss fu n ct ion .

Payload sem ant ics is widely used in t he cont ext of MOMs t hat provide API s wit h funct ions such as MQGET()/MQPUT() or sendMessage()/onMessage()/receiveMessage() for t he client s and servers t o com m unicat e wit h each ot her. The sem ant ics of t hese funct ions is purely t echnical ( see Figure 3- 15 ) .

Figu r e 3 - 1 5 . M OM is ge n e r a lly ba se d on pa yloa d se m a n t ics. Fu n ct ion s su ch a s sendMessage() a n d processMessage() a r e pu r e ly t e ch n ica l, w it h ou t a n y bu sin e ss se m a n t ics.

I nt erface sem ant ics provide users wit h well- defined int erfaces t hat are int uit ive and easy t o underst and. Changes t o t hese int erfaces require m odificat ions t o all applicat ions t hat depend on t he part icular int erface, even if t hey do not depend on t he operat ion or argum ent t hat was added or changed. Payload sem ant ics, on t he ot her hand, result in syst em s where changes t o m essage form at s can have a pot ent ially lesser im pact on t he different com ponent s of t he syst em . New funct ionalit y can easily be added t o a syst em by creat ing new m essages t ypes. Consum ers t hat

are not dependent on t he new m essages t ypes rem ain unalt ered. Thus, payload sem ant ics result s in a weaker coupling at t he t ype level. The choice of int erface sem ant ics versus payload sem ant ics is not an obvious one, as each approach has it s pros and cons. St rongly t yped languages, such as Java, lim it t he flexibilit y of t he program m er by applying st rict t ype checking at com pile t im e. Alm ost all dependencies caused by t he change of a t ype in t he syst em can be discovered at com pile t im e, t hus significant ly reducing t he num ber of runt im e errors. Weakly t yped languages, such as TCL, offer m uch m ore flexible dat a m anipulat ion, oft en based on st ring m anipulat ion. These t ypes of languages are generally used for script ing, especially in Web environm ent s, where fast result s are required. However, t he applicat ion of int erface sem ant ics and payload sem ant ics cannot be viewed in blackand- whit e t erm s. While RPCs are not rest rict ed t o pure funct ional call sem ant ics, neit her are MOMs lim it ed t o payload sem ant ics. This can be illust rat ed wit h one insight ful exam ple. Consider an RPC call such as transferMoney() . The t ransm it t ed dat a can significant ly cont ribut e t o t he det erm inat ion of t he code t hat is execut ed:

String transferMoney (amount: decimal; cur, accFrom, accTo: String); { switch (cur) case 'EUR': handleEurTransfer (amount, accFrom, accTo); case 'GBP': handleGbpTransfer (amount, accFrom, accTo); case 'USD': handleUsdTransfer (amount, accFrom, accTo); . . . }

Going one st ep furt her, it is possible t o rem ove all int erface sem ant ics from an RPC- st yle int erface. I n t he following exam ple, a service support s exact ly one funct ion, executeService() . This funct ion has only one param et er, a plain st ring. This st ring encodes all funct ional param et ers and t he request ed funct ionalit y:

String executeService (message: String); { int i = determineFunctionNumber (message); switch (i) case 1: handleCase1 (message); case 2: handleCase2 (message); case 3: handleCase3 (message); . . . }

3.4.1. DOCUMENT-CENTRIC MESSAGES

Wit h t he em ergence of self- descript ive dat a st ruct ures such as XML, an approach t o handling m essage t ypes referred t o as docum ent - cent ric has becom e popular. Docum ent - cent ric m essages are sem ant ically rich m essages where t he operat ion nam e, it s param et ers, and t he ret urn t ype are self- descript ive. However, t he passed param et ers and ret urned obj ect can be ext rem ely flexible and can include any num ber of opt ional param et ers. SOAP ( Sim ple Obj ect Access Prot ocol) is a t echnology t hat is part icularly suit able for t his t ype of solut ion. As long as t he underlying XML Schem as im pose only loose const raint s on t he docum ent st ruct ure, t he param et ers can be ext ended in any m anner required wit hout breaking com pat ibilit y wit h previous soft ware versions. Consider t he following exam ple t hat describes t he booking of a flight . I t is st raight forward t o enhance t he prot ocol wit h addit ional param et ers, such as t he t im e of day of t he flight , t he dat e and t im e of arrival, or t he verbose nam es of t he airport s. As long as t hese fields are not required, t he previous version of t he prot ocol, in addit ion t o all soft ware t hat relies on t hat version, rem ains t ot ally valid. Because t his t ype of com m unicat ion includes a st ruct ured docum ent in bot h reply and response, it is called docum ent - driven com m unicat ion: [View full width]



LH400 2003-11-08 false

LH401 2003-11-17 false

Karl Banke 1970-08-05



3.5. Tight Versus Loose Coupling Recent ly, a lot of at t ent ion has focused on com parisons bet ween loose coupling and t ight coupling approaches t o applicat ion int eract ions. On t he t echnology side, t his has m ainly been driven by t he pot ent ial of Web services t o dynam ically discover and bind t o ot her services, such as t hrough UDDI ( Universal Descript ion, Discovery and I nt egrat ion) . On t he business side, t his has been driven by t he growing need of ent erprises t o increase flexibilit y wit h respect t o changes in t heir own business processes and t he ways in which t hey int eract wit h part ner com panies. Tradit ionally, business processes have been designed wit hin t he boundaries of an ent erprise, or even wit hin t he different business unit s of t he ent erprise. These act ivit ies were m anaged wit h t he help of det ailed, real- t im e inform at ion. Processes t hat span m ult iple business unit s or ent erprises t ypically have t o deal wit h a very different set of requirem ent s, needing a higher degree of flexibilit y. I n t hese kinds of scenarios, one sees a m uch higher degree of uncert aint y, a m uch m ore frequent change in t erm s of part icipant s and t heir roles, and a const ant evolut ion of t he t ypes of int eract ions required. There appears t o be a consensus t hat for t hese t ypes of " in- flux" sit uat ions t o operat e, a loosely coupled archit ect ure is required because loose coupling is seen as helping t o reduce t he overall com plexit y and dependencies. Using loose coupling m akes t he applicat ion landscape m ore agile, enables quicker change, and reduces risk. I n addit ion, syst em m aint enance becom es m uch easier. Loose coupling becom es part icularly im port ant in t he B2B world, where business ent it ies m ust be able t o int eract independent ly. The relat ionships bet ween business part ners oft en change rapidlyalliances are set t led and cancelled, and business processes bet ween t rading part ners are adopt ed t o new m arket requirem ent s. Two com panies t hat are part ners in one m arket m ight be com pet it ors in anot her m arket . Therefore, it is essent ial t hat t he underlying I T infrast ruct ure reflect t his need for flexibilit y and independence. I deally, no business relat ionship should im pact anot hernew business relat ionships should be able t o be est ablished wit hout any effect on exist ing ones. Funct ionalit y t hat is offered t o one business part ner m ight not necessarily be available t o ot hers. A change t hat is relevant for one business part ner should have no im pact on ot her part ners. One t rading part ner m ay not cause anot her t o block while wait ing for a synchronous response, nor m ay one I T syst em depend on t he t echnical availabilit y of t he I T syst em of a business part ner. The t erm coupling refers t o t he act of j oining t hings t oget her, such as t he links of a chain. I n t he soft ware world, coupling t ypically refers t o t he degree t o which soft ware com ponent s depend upon each ot her. However, t he rem aining quest ion is: "What are t hese dependencies, and t o what degree can one apply t he propert ies of t ight and loose?" Soft ware coupling can happen on m any different levels. One of t he first issues is t o different iat e bet ween build t im e ( com pile t im e) dependencies and runt im e dependencies. However, t his is t ypically only sufficient when looking at m onolit hic applicat ions. I n a dist ribut ed environm ent , we believe t hat in order t o det erm ine t he degree of coupling in a syst em , one needs t o look at different levels. Table 3- 1 provides an overview of t hese levels and shows how t hey relat e t o t he t ight versus loose coupling debat e.

Ta ble 3 - 1 . Tigh t Ve r su s Loose Cou plin g Le v e l

Tigh t Cou plin g

Loose Cou plin g

Le v e l

Tigh t Cou plin g

Loose Cou plin g

Physical coupling

Direct physical link required

Physical int erm ediary

Com m unicat ion st yle

Synchronous

Asynchronous

Type syst em

St rong t ype syst em ( e.g., int erface sem ant ics)

Weak t ype syst em ( e.g., payload sem ant ics)

I nt eract ion pat t ern

OO- st yle navigat ion of com plex obj ect t rees

Dat a- cent ric, self- cont ained m essages

Cont rol of process logic

Cent ral cont rol of process logic

Dist ribut ed logic com ponent s

Service discovery and binding

St at ically bound services

Dynam ically bound services

Plat form dependencies

St rong OS and program m ing language dependencies

OS- and program m ing language independent

I n t he following, we will exam ine t he it em s in Table 3- 1 in det ail. For dist ribut ed syst em s, t he way t hat rem ot e com ponent s are connect ed is possibly t he m ost obvious t echnical fact or when looking at t he problem of " coupling." A physical int erm ediary enables loose coupling on t his level. Therefore, MOM syst em s are loosely coupled on t he physical level, wit h m essage queues act ing as an int erm ediary, decoupling senders and receivers of m essages. RPC- st yle applicat ions are t ight ly coupled on t his level because client s and servers int eract direct ly wit h each ot herclient s require servers t o be alive and accessible in order t o int eract wit h t hem . The im pact of synchronous versus asynchronous com m unicat ion on t he level of coupling is oft en closely relat ed t o t he physical linking of t he dist ribut ed com ponent s, as described previously. Asynchronous com m unicat ion is generally associat ed wit h loose coupling. However, t his assum es t hat t he underlying m iddleware is capable of support ing t he asynchronous com m unicat ion in a loosely coupled m anner. Assum e a one- way RPC call: There is st ill a not ion of t ight coupling here, even if t he client does not wait for t he reply of t he servert he client will only be able t o send t he one- way request t o t he server if it is direct ly connect ed and if t he server is up and running. This is a good exam ple for t he varying degrees of " coupledness" asynchronous com m unicat ion t hrough a proper MOM is m ore loosely coupled t han asynchronous one- way RPC calls. Looking at t he t ype syst em of a dist ribut ed applicat ion as t he next level of " coupledness," we find t hat t he st ronger t he t ype syst em , t he st ronger t he dependencies bet ween t he different com ponent s of t he syst em . This is t rue not only during t he applicat ion developm ent phase, but also ( and perhaps m ore im port ant ly) when changing or reconfiguring t he running syst em . Earlier, we different iat ed bet ween int erface sem ant ics and payload sem ant ics. I nt erface sem ant ics provide an explicit int erface and operat ion nam es and also st rongly t yped argum ent s. Effect ively, t his m eans com ponent s are t ight ly coupled t oget her on t his level because every change of an int erface ripples t hrough t he ent ire applicat ion, as far as dependent com ponent s are concerned. The benefit is t hat we discover t he affect ed part s of t he applicat ion t hat need t o be adapt ed t o reflect t hese changes at com pile t im e, t hus avoiding runt im e except ions due t o incom pat ible m essage form at s. Payload sem ant ics, on t he ot her hand, enable a looser coupling of com ponent s because m essage form at s are generally m ore flexible. I n som e cases, m essage form at validat ion m ight be applied, such as t hrough XML Schem a validat ion. However, t his requires efficient m anagem ent of t he up- t o- dat e schem a definit ions bet ween part icipant s. Not ice t hat t he problem s wit h changes t o m essage form at s is not elim inat ed by em ploying payload sem ant ics: One m ust st ill know t hose part s of t he syst em t hat are affect ed by changes in order t o ensure t hat t hey can act appropriat ely on t he new form at . I n m any cases, t his m eans t hat t he problem has sim ply

m oved from build t im e t o runt im e. Anot her im port ant fact or t o exam ine is t he int eract ion pat t erns of t he dist ribut ed com ponent s. For exam ple, an ORB- based syst em will t ypically im pose an OO- st yle navigat ion of com plex obj ect t rees. The client has t o underst and not only t he logic of each individual obj ect , but also t he way t o navigat e across obj ect s, again result ing in a fairly t ight coupling. Given t hat RPC- st yle int erfaces do not enable such com plex navigat ion, t he degree of coupling is lower when com pared t o a dist ribut ed obj ect syst em . MOM- based syst em s t ypically im pose a m uch sim pler int eract ion m odel, where oft en a single queue is sufficient as an ent ry point for client s, and all input for server- side t ransact ions is provided in a single m essage. Relat ed t o t his discussion is t he quest ion of whet her we generally assum e t hat t he syst em is st ruct ured around RPC- st yle services or around queues and t opics. Generally, t opics and queues provide m ore flexibilit y for changing t he syst em at runt im e by rearranging t he configurat ion of queues and how t hey are relat ed t o each ot her. The powerful configurat ion m anagem ent of m ost MOM syst em s great ly increase t he " looseness" of t he coupling bet ween syst em com ponent s. Anot her im port ant fact or is t he ownership or cont rol of process logic. I f processes are m anaged cent rally, t his result s in t ight coupling bet ween t he different sub- processes and t ransact ions. For exam ple, dat abase m echanism s m ight be used for ensuring referent ial int egrit y and general consist ency of t he dat a owned by t he different sub- processes. This is oft en t he case, for exam ple, wit h large, m onolit hic ERP ( Ent erprise Resource Planning) syst em s. I f business processes are highly dist ribut ed, as in a B2B environm ent , t he different sub- processes and t ransact ions are generally m ore independent of each ot her, or m ore loosely coupled, in t he cont ext of our current discussion. Oft en, t his m eans t hat one m ust accept t he fact t hat t here is no globally defined consist ent process st at e. Sim ilarly, t he dat a owned by t he different part icipant s m ight not always be consist ent one syst em m ight have already cancelled an order for which anot her syst em st ill owns an invoice. Finally, t he way in which part icipant s in t he syst em locat e each ot her has a great im pact on t he level of coupling in t he syst em . St at ically bound services yield very t ight coupling, whereas dynam ically bound services yield loose coupling. Looking up services in a nam ing or direct ory server reduces t he t ight ness wit h which com ponent s are t ied t oget her, alt hough it st ill requires t he client t o know t he exact nam e of t he service t o which it want s t o bind. Services such as UDDI enable a m ore flexible locat ion of services, using const raint s such as " Find m e t he next print er on t he second floor." Not ice t hat dynam ic service discovery as provided by UDDI for Web Services is not a new concept ; it has previously been provided by ot her st andards such as t he CORBA Nam ing Service. Not ice also t hat past experience has shown t hat t he num ber of applicat ions requiring com plet ely dynam ic service discovery has been fairly lim it ed. When m aking archit ect ural decisions, one m ust carefully analyze t he advant ages and disadvant ages of t he level of coupling. Generally speaking, OLTP- st yle ( online t ransact ion processing) applicat ions, as t hey are found t hroughout large ent erprises, do not norm ally require a high degree of loose couplingt hese applicat ions are t ight ly coupled by t heir nat ure. When leaving t he scope of a single ent erprise or single business unit , especially in B2B environm ent s, loose coupling is oft en t he only solut ion. However, in m ost cases, t he increased flexibilit y achieved t hrough loose coupling com es at a price, due t o t he increased com plexit y of t he syst em . Addit ional effort s for developm ent and higher skills are required t o apply t he m ore sophist icat ed concept s of loosely coupled syst em s. Furt herm ore, cost ly product s such as queuing syst em s are required. However, loose coupling will pay off in t he long t erm if t he coupled syst em s m ust be rearranged quit e frequent ly.

3.6. Conclusion Today's ent erprise applicat ion landscapes are charact erized by a variet y of different t echnologies and concept s for dist ribut ion. On one hand, t his variet y arises wit hin t he ent erprise organizat ion it self for hist orical reasons, personal preferences of different people, and t he dynam ics of acquisit ions and m ergers. As a m at t er of fact , m any redundant concept s exist wit hin t he sam e organizat ional unit . On t he ot her hand, com plem ent ary concept s and t echnologies also exist . Due t o t he requirem ent s of different t ypes of dist ribut ion problem s t hat coexist in one corporat ion, different solut ions arise as well. A m odern archit ect ure m ust be able t o em brace all t hese t echnologies and concept s. Het erogeneit yincluding het erogeneit y of m iddlewarem ust be underst ood as a fundam ent al fact t hat cannot be fought but inst ead m ust be m anaged. Furt herm ore, an archit ect ure m ust accom m odat e frequent changes of t he underlying dist ribut ion infrast ruct ure. As a m at t er of fact , t he lifecycles of t oday's infrast ruct ure product s are largely incom pat ible wit h t he lifecycles of ent erprise applicat ions. Thus, you m ust prot ect t he asset s of an exist ing applicat ion landscape and sim ult aneously t ake advant age of t he lat est infrast ruct ure product s. I n t his chapt er, we have discussed t he necessit y of carefully choosing t he right approach t o int egrat ing t wo dist ribut ed soft ware com ponent s. Am ong ot her issues, you m ust decide on t he appropriat e com m unicat ion infrast ruct ure, synchrony, call sem ant ics, usage of an int erm ediary, and obj ect - orient ed versus dat a- cent ric int erfaces. All t hese decisions im pact t he coupling of t he t wo syst em s.

References [ Bras2002] Braswell, Byron, George Forshay, and Juan Manuel Mart inez . I BM Web- t o- Host I nt egrat ion Solut ions, 4t h ed. I BM Redbook SG24- 5237- 03, 2002. [ Lon1999] Long, Rick, Jouko Jänt t i, Robert Hain, Niel Kenyon, Mart in Owens, and André Schoem an . I MS e- business Connect Using t he I MS Connect ors. I BM Redbook SG24- 5427- 00, 1999. [ Tan2003] Tanenbaum , Andrew S. Com put er Net works, 4t h ed. Prent ice- Hall, 2003. [ Tan2002] Tanenbaum , Andrew S. and Maart en van St een . Dist ribut ed Syst em s: Principles and Paradigm s. Prent ice- Hall, 2002. [ Cou2001] Coulouris, George, J. Dollim ore, and T. Kindberg . Dist ribut ed Syst em s Concept s and Design, 3rd ed. Addison- Wesley, 2001. [ Rei1992] Reisig, Wolfgang . A Prim er in Pet ri Net Design. New York: Springer Com pass I nt ernat ional, 1992.

URLs ht t p: / / www.rfc- edit or.org ht t p: / / www.om g.org ht t p: / / www.m icrosoft .com / com ht t p: / / j ava.sun.com / j 2ee ht t p: / / www.bea.com

Part I: Architectural Roadmap The first part of t his book describes t he archit ect ural roadm ap t o t he service- enabled ent erprise. We define t he t erm Service- Orient ed Archit ect ure ( SOA) , provide a classificat ion of service t ypes, describe t he different expansion st ages of a SOA, show how t o address key issues such as business processes and t ransact ions, out line a service bus infrast ruct ure, and describe how t o use t he different SOA concept s in real- world applicat ions.

Chapter 4. Service-Oriented Architectures This chapt er provides a definit ion of Service- Orient ed Archit ect ure and int roduces it s key concept snam ely, applicat ion front end, service, service reposit ory, and service bus. I t lays t he concept ual foundat ion for a m ore in- dept h discussion about t he ways in which SOAs help address t he specific issues of ent erprise applicat ions t hat are covered in Chapt er 5, " Services as Building Blocks," and Chapt er 6, " The Archit ect ural Roadm ap." Sect ion 4.1 gives a general definit ion of t he t erm archit ect ure as we use it t hroughout t his book. Sect ion 4.2 defines t he t erm Service- Orient ed Archit ect ure. Sect ion 4.3 describes elem ent s of a Service- Orient ed Archit ect ure, such as applicat ion front ends, services, t he service reposit ory, and t he service bus in det ail.

4.1. What Is a Software Architecture? The lit erat ure in our field provides m any different definit ions of soft ware archit ect ure. Booch, Rum baugh, and Jacobson [ BRJ99] claim t hat "An archit ect ure is t he set of significant decisions about t he organizat ion of a soft ware syst em . . ." Brass, Clem ent s, and Kazm an define soft ware archit ect ure in [ BCK03 ] : "The soft ware archit ect ure of a program or com put ing syst em is t he st ruct ure or st ruct ures of t he syst em , which com prise soft ware elem ent s, t he ext ernally visible propert ies of t hose elem ent s, and t he relat ionships am ong t hem ." The I EEE St andard 610.121990 claim s t hat " Archit ect ure is t he organizat ional st ruct ure of a syst em ." Fowler charact erizes archit ect ure in [ Fow02 ] : " 'Archit ect ure' is a t erm t hat lot s of people t ry t o define, wit h lit t le agreem ent . There are t wo com m on elem ent s: One is t he highest - level breakdown of a syst em int o it s part s; t he ot her, decisions t hat are hard t o change." You can find even m ore definit ions at ht t p: / / www.sei.cm u.edu/ archit ect ure/ definit ions.ht m l. For our purposes, we define soft ware archit ect ure in t he sidebar, "Definit ion of Soft ware Archit ect ure."

Definition of Software Architecture A soft ware archit ect ure is a set of st at em ent s t hat describe soft ware com ponent s and assigns t he funct ionalit y of t he syst em t o t hese com ponent s. I t describes t he t echnical st ruct ure, const raint s, and charact erist ics of t he com ponent s and t he int erfaces bet ween t hem . The archit ect ure is t he blueprint for t he syst em and t herefore t he im plicit high- level plan for it s const ruct ion.

I n t his book, we will also use t he t erm s applicat ion and applicat ion landscape. An applicat ion is a set of soft ware com ponent s t hat serves a dist inct ive purpose, and an applicat ion landscape is t he sum of all applicat ions of an organizat ion. I deally, all applicat ions of an applicat ion landscape com ply wit h a single archit ect ural blueprint . However, in pract ice, t hey usually don't . We also casually use one part icular phrase: " Soft ware com ponent X belongs t o archit ect ure Y." More precisely, t his phrase m eans: " Soft ware com ponent X belongs t o an applicat ion landscape which is designed according t o an archit ect ure Y."

4.2. What Is a Service-Oriented Architecture? Now we int roduce t he basic concept s of SOAs as we use t hem in t he rem ainder of t his book. As we previously em phasized, t his book focuses on ent erprise archit ect ures and t heir specific charact erist ics. Consequent ly, we will also discuss t he specific charact erist ics of SOAs. As we m ent ioned earlier, an SOA is based on four key abst ract ions: applicat ion front end, service, service reposit ory, and service bus ( see Figure 4- 1 ) . Alt hough t he applicat ion front end is t he owner of t he business process, services provide business funct ionalit y t hat t he applicat ion front ends and ot her services can use. A service consist s of an im plem ent at ion t hat provides business logic and dat a, a service cont ract t hat specifies t he funct ionalit y, usage, and const raint s for a client [ 1] of t he service, and a service int erface t hat physically exposes t he funct ionalit y. The service reposit ory st ores t he service cont ract s of t he individual services of an SOA, and t he service bus int erconnect s t he applicat ion front ends and services. [1]

A client can either be an application frontend or another service.

Figu r e 4 - 1 . Se r vice s a n d a pplica t ion fr on t e n ds a r e t h e m a j or a r t ifa ct s of a n SOA. I n a ddit ion , w e a lso h a ve a se r vice r e posit or y a n d se r vice bu s.

Definition of Service-Oriented Architecture A Service- Orient ed Archit ect ure ( SOA) is a soft ware archit ect ure t hat is based on t he key concept s of an applicat ion front end, service, service reposit ory, and service bus. A service consist s of a cont ract , one or m ore int erfaces, and an im plem ent at ion.

The whole concept of an SOA focuses on t he definit ion of a business infrast ruct ure. When we use t he t erm " service," we have in m ind a business service such as m aking airline reservat ions or get t ing access t o a com pany's cust om er dat abase. These services provide business operat ions such as get reservat ion, cancel booking, or get cust om er profile. Unlike business services, t echnical infrast ruct ure services, such as a persist ency service or a t ransact ion service, provide operat ions such as begin t ransact ion, updat e dat a, or open cursor. Alt hough t his kind of t echnical funct ionalit y is very useful when it com es t o im plem ent ing a business operat ion, it has lit t le st rat egic relevance from t he SOA point of view. More generally, t echnology m ust not have any im pact on t he high- level st ruct ure of t he applicat ion landscape or cause dependencies bet ween com ponent s. Act ually, t he SOA m ust decouple business applicat ions from t echnical services and m ake t he ent erprise independent of a specific t echnical im plem ent at ion or infrast ruct ure. The applicat ion front ends are t he act ive elem ent s of t he SOA, delivering t he value of t he SOA t o t he end users. Nevert heless, you m ust always t ake int o account t hat t he services provide st ruct ure t o t he SOA. Alt hough t he services can oft en rem ain unalt ered, t he applicat ion front ends are subj ect t o change, as are t he business processes of t he ent erprises. Consequent ly, t he lifecycle of applicat ion front ends is m uch short er t han t he lifecycle of t he underlying services. This is why we regard services as t he prim ary ent it ies of st rat egic im port ance in an SOA ( see Figure 42) .

Figu r e 4 - 2 . Th e e st im a t e d life cycle s of da t a , se r vice s, a pplica t ion fr on t e n ds, a n d t e ch n ologie s a r e diffe r e n t . [View full size image]

4.3. Elements of a Service-Oriented Architecture I n t his sect ion, we t ake a closer look at t he key elem ent s of t he SOA, including t he applicat ion front end, services, t he service reposit ory, and t he service bus.

4.3.1. APPLICATION FRONTENDS Applicat ion front ends are t he act ive players of an SOA. They init iat e and cont rol all act ivit y of t he ent erprise syst em s. There are different t ypes of applicat ion front ends. An applicat ion front end wit h a graphical user int erface, such as a Web applicat ion or a rich client t hat int eract s direct ly wit h end users, is t he m ost obvious exam ple. However, applicat ion front - ends do not necessarily have t o int eract direct ly wit h end users. Bat ch program s or long- living processes t hat invoke funct ionalit y periodically or as a result of specific event s are also valid exam ples of applicat ion front ends. Nevert heless, it is ent irely possible t hat an applicat ion front end delegat es m uch of it s responsibilit y for a business process t o one or m ore services. Ult im at ely, however, it is always an applicat ion front end t hat init iat es a business process and receives t he result s. Applicat ion front ends are sim ilar t o t he upper layers of t radit ional m ult ilayer applicat ions. Alt hough you m ight expect t hat services m ore closely resem ble t he lower layers, t his is not t he case. The following chapt ers dem onst rat e t hat services have a different st ruct ure, which is charact erized by vert ical slicing.

4.3.2. SERVICES A service is a soft ware com ponent of dist inct ive funct ional m eaning t hat t ypically encapsulat es a high- level business concept . I t consist s of several part s ( see Figure 4- 3 ) . Con t r a ct . The service cont ract provides an inform al specificat ion of t he purpose, funct ionalit y, const raint s, and usage of t he service. The form of t his specificat ion can vary, depending on t he t ype of service. One non- m andat ory elem ent of t he service cont ract is a form al int erface definit ion based on languages such as I DL or WSDL. Alt hough it is not m andat ory, a form al service int erface definit ion adds a significant benefit : I t provides furt her abst ract ion and independence of t echnology, including program m ing language, m iddleware, net work prot ocol, and runt im e environm ent . However, it is im port ant t o underst and t hat t he service cont ract provides m ore inform at ion t han a form al specificat ion. The cont ract can im pose det ailed sem ant ics on t he funct ionalit y and param et ers t hat is not subj ect t o I DL or WSDL specificat ions. I n realit y, m any proj ect s m ust cope wit h services t hat cannot provide form al service int erface descript ions. [ 2] I n t hese cases, t he service can deliver access libraries or a det ailed t echnical descript ion at t he net work prot ocol level. However, it is im port ant t o underst and t hat every service requires a service cont ract part icularly if no form al descript ion based on a st andard such as WSDL or I DL is available. [2]

Notice that the key task of a project aiming to introduce SOAs at the enterprise level is often not to implement new business functionality, but rather to identify suitable existing application modules and components and wrap them with service interfaces with the appropriate level of functionality and granularity, thus making them available as services in an easier-to-use and better documented manner.

I nt erfa ce. The funct ionalit y of t he service is exposed by t he service int erface t o client s t hat are connect ed t o t he service using a net work. Alt hough t he descript ion of t he int erface is part of t he service cont ract , t he physical im plem ent at ion of t he int erface consist s of service st ubs, which are incorporat ed int o t he client s [ 3] of a service and dispat cher. [3]

Application frontends or other services.

I m ple m e n t a t ion . The service im plem ent at ion physically provides t he required business logic and appropriat e dat a. I t is t he t echnical realizat ion t hat fulfills t he service cont ract . The service im plem ent at ion consist s of one or m ore art ifact s such as program s, configurat ion dat a, and dat abases. Bu sin e ss logic. The business logic t hat is encapsulat ed by a service is part of it s im plem ent at ion. I t is m ade available t hrough service int erfaces. However, program m ing against int erfaces is desirable, whet her or not one applies a service- orient ed approach. D a t a . A service can also include dat a. I n part icular, it is t he purpose of a dat a- cent ric service ( see Chapt er 5) .

Figu r e 4 - 3 . A se r vice con sist s of bot h da t a a n d bu sin e ss logic a lon g w it h in t e r fa ce s a n d t h e ir de scr ipt ion s.

As previously discussed, services are not j ust t he encapsulat ion of som e code of t he form er lower layers of applicat ions. Every service is an ent it y of dist inct ive funct ional m eaning t hat t ypically encapsulat es a high- level business ent it y. Services im pose a st rong vert ical slicing of t he applicat ion t hat defines t he coarse- grained st ruct ure of t he whole syst em , sim ilarly t o com ponent orient ed soft ware design. Therefore, from t he client perspect ive, a service is a black box ent it y.

4.3.3. SERVICE REPOSITORY A service reposit ory provides facilit ies t o discover services and acquire all inform at ion t o use t he

services, part icularly if t hese services m ust be discovered out side t he funct ional and t em poral scope of t he proj ect t hat creat ed t hem . Alt hough m uch of t he required inform at ion is already part of t he service cont ract , t he service reposit ory can provide addit ional inform at ion, such as physical locat ion, inform at ion about t he provider, cont act persons, usage fees, t echnical const raint s, securit y issues, and available service levels. I t should be not ed t hat we focus on service reposit ories t hat are m ainly used for purposes wit hin t he boundaries of a single ent erprise. Reposit ories t hat are used for cross- ent erprise service int egrat ion t ypically have different requirem ent sin part icular, t hose reposit ories t hat are m ade public t hrough t he I nt ernet . These requirem ent s can com prise legal issues ( t erm s and condit ions of usage) , st yle of present at ion, securit y, user regist rat ion, service subscript ion, billing, and versioning. Obviously, a service reposit ory is a very useful elem ent of an SOA. Alt hough you can build an SOA and achieve m any of it s benefit s wit hout est ablishing a service reposit ory, a reposit ory is indispensable in t he long t erm . An archit ect ure can cope wit hout a reposit ory if t he scope of a service is j ust one proj ect , if it has very few services, or if all proj ect s are st affed wit h t he sam e t eam m em bers. I n realit y, t hough, m ost ent erprise scenarios are charact erized by m any concurrent proj ect s, changing t eam s, and a variet y of services. A service reposit ory can be arbit rarily sim ple; at one ext rem e, no t echnology m ight be required. A bat ch of print ed service cont ract s locat ed in an office and accessible by all proj ect s is already a valid service reposit ory. However, bet t er ways exist t o provide t his inform at ion while ret aining t he sim plicit y of t he reposit ory. Oft en, you'll find a t ype of propriet ary dat abase t hat cont ains som e form alized adm inist rat ive dat a and a m ore or less form al service cont ract for every version of a service. I n som e cases, com panies have developed t heir own t ools t hat aut om at ically generat e t he service descript ion from t he form al service definit ions ( e.g., an HTML generat or t hat t akes WSDL as input , sim ilar t o a JavaDoc generat or) . This is part icularly useful if t he form al service definit ion is annot at ed wit h addit ional inform at ion about t he service. Not ice t hat t his inform at ion is t ypically very different from t he m et a- inform at ion provided for low- level API s, such as Java classes. This is due t o t he different roles t hat service definit ions play in an SOA. Services t ypically are m ore coarse- grained, self- cont ained, and capable of support ing different usage pat t erns. I n part icular, services are t ypically not linked as in code libraries but are bound t o at runt im e. All t he preceding result s in different docum ent at ion requirem ent s. The following are exam ples of inform at ion t hat should be cont ained in an ent erprise- wide service reposit ory: Service, operat ion, and argum ent s signat ures, such as in t he form of WSDL and XML Schem a definit ions. Service owner. I n an Ent erprise SOA, owners can operat e at t he business level ( responsible for quest ions and change request s on t he funct ional level) , developm ent level ( responsible for t echnical quest ions and change request s) , and operat ions level ( responsible for quest ions regarding t he best ways t o link t o a service, or operat ional problem s) . Access right s, such as inform at ion about access cont rol list s and t he underlying securit y m echanism , or a descript ion of t he process t hat m ust be followed wit hin t he ent erprise so t hat a new syst em can ut ilize a part icular service. I nform at ion about t he int ended perform ance and scalabilit y of t he service, including average response t im es, and pot ent ial t hroughput lim it at ions. This can be sum m arized as part of a generic SLA ( Service Level Agreem ent ) t em plat e. Transact ional propert ies of t he service and it s individual operat ions. This includes inform at ion on t he read/ writ e/ updat e charact erist ics, whet her t he operat ion is idem pot ent ,

and associat ed com pensat ion logic ( see Chapt er 8, "Process I nt egrit y ," for m ore det ails) .

Manage Your Service Repository Centrally To provide a service reposit ory wit h high qualit y services, consider set t ing up an archit ect ure board. The archit ect ure board's responsibilit y is t o perform const ant m aint enance, m onit oring, and coordinat ion from a cent ral locat ion. I t m ust m anage t he reposit ory and carefully review t he service ent ries it cont ains. This includes t he fundam ent al design of t he service it self, as well as it s descript ion in t he service reposit ory. Consequent ly, t he archit ect ure board m ust be involved from t he out set of t he developm ent of new services in order t o coordinat e t he service specificat ion across different proj ect s and business unit s, and it m ust ensure t hat a good com prom ise bet ween ease of im plem ent at ion, usabilit y, and reusabilit y can be achieved.

I t is im port ant t o dist inguish bet ween developm ent t im e and runt im e binding of services. Binding refers t o t he way in which service definit ions and service inst ances are locat ed, incorporat ed int o t he client applicat ion, and finally bound t o at t he net work level.

4.3.3.1 Development-Time Binding I f services are discovered and bound t o at developm ent t im e, t he signat ures of t he service operat ions are known in advance, as well as t he service prot ocol and t he physical locat ion of t he service ( or at least t he exact nam e of t he service in a direct ory service) . Figure 4- 4 describes a process in which services are bound t o at developm ent t im e.

Figu r e 4 - 4 . D e ve lopm e n t t im e discove r y im pose s a fa ir ly sim ple m ode l. Th e de ve lope r is r e spon sible for loca t in g a ll r e qu ir e d in for m a t ion fr om t h e se r vice r e posit or y in or de r t o cr e a t e a clie n t t h a t in t e r a ct s cor r e ct ly w it h t h e se r vice in st a n ce . [View full size image]

Alt hough developm ent t im e binding is quit e a sim ple m odel, it is sufficient for m ost purposes. I t enables proj ect s t o ident ify funct ionalit y t hat has been creat ed by form er proj ect s and t o reuse t hese services.

4.3.3.2 Runtime Binding Runt im e binding is far m ore com plex t han developm ent t im e binding. One can different iat e bet ween different levels of runt im e binding: Ru n t im e se r vice look u p by n a m e . This is t he m ost st raight forward case, and it is also t he m ost com m only used m eans of dynam ically binding t o services: The service definit ion is known at developm ent t im e, and t he client logic is developed accordingly. The client is enabled t o dynam ically bind t o different service inst ances by looking up services wit h specific nam es in a direct ory. For exam ple, a client applicat ion looks up print ing services wit h different nam es, depending on t he print er nam e select ed by t he user. Ru n t im e se r vice look u p by pr ope r t ie s. This is sim ilar t o t he preceding, except t hat services are discovered by propert ies, not by nam e. For exam ple, a print ing service can search a service reposit ory for t hree different predefined print ing service int erfaces it underst ands, t oget her wit h ot her propert ies such as locat ion of t he print er ( "FLOOR == 2") and docum ent form at s t hat t he print er is able t o print ("DOCTYPE == PostScript") . Ru n t im e se r vice discove r y ba se d on r e fle ct ion . I n t he final case, t he act ual specificat ion of t he service definit ion is not known at developm ent t im e. Assum e t hat a client discovers a service wit h t he right propert ies ( "FLOOR == 2 AND DOCTYPE == PostScript") but wit h an unknown print ing service int erface. I n t his case, som e kind of reflect ion m echanism [ 4] m ust be im plem ent ed at t he client side, which enables t he client t o dynam ically discover t he sem ant ics of t he service and t he form at of valid request s. This t ype of service discovery is t he m ost com plex and least widely used because it requires very com plex logic t o dynam ically int erpret t he sem ant ics of unknown service int erfaces. [ 5] [4]

Similar to mechanisms that are used in many object-oriented programming languages, such as Java's reflection mechanism. [5]

Notice that there is a case in between, where systems have to cope with many similar yet different message types, such as 12 different formats of a customer data record. A number of existing EAI tools (e.g., in Microsoft Biztalk Server) enable the user to graphically match fields in different data structures, providing runtime translation between

these incompatible data formats. However, these mappings between data structures are provided at development time and are not dynamically determined by the system at runtime.

Runt im e service binding t hat goes beyond t he com plexit y of dynam ic service lookup by propert ies wit h predefined service int erfaces is very rare and is lim it ed t o very few applicat ion dom ains. A rare exam ple for a problem dom ain t hat really requires highly dynam ic service binding is in t he wireless world, such as a Bluet oot h applicat ion: Bluet oot h client s dynam ically discover services based on locat ion and ot her propert ies. But again, even in t his scenario, Bluet oot h client s t ypically support a lim it ed set of predefined services.

Make Service Binding as Simple as Possible Always aim t o m ake service binding as sim ple as possible because t he level of com plexit y and risk increases exponent ially wit h t he level of dynam ics in t he service binding process. Service lookup by nam e wit h predefined service int erfaces represent s t he best t rade- off bet ween flexibilit y and im plem ent at ion com plexit y in t he m aj orit y of cases.

4.3.4. SERVICE BUS A service bus connect s all part icipant s of an SOAservices and applicat ion front endswit h each ot her. I f t wo part icipant s need t o com m unicat efor exam ple, if an applicat ion front end needs t o invoke som e funct ionalit y of a basic servicet he service bus m akes it happen. I n t his respect , t he service bus is sim ilar t o t he concept of a soft ware bus as it is defined in t he cont ext of CORBA. However, significant differences exist bet ween t hese concept s. Most im port ant ly, t he service bus is not necessarily com posed of a single t echnology, but rat her com prises a variet y of product s and concept s. An in- dept h discussion of t he service bus is found in Chapt er 9, "I nfrast ruct ure of a Service Bus." For t he t im e being, it will suffice t o highlight t he following charact erist ics of a service bus: Con n e ct iv it y . The prim ary purpose of t he service bus is t o int erconnect t he part icipant s of an SOA. I t provides facilit ies t hat enable t he part icipant s of an SOAapplicat ion front ends and servicest o invoke t he funct ionalit y of services. H e t e r oge n e it y of t e ch n ology. The service bus m ust em brace a variet y of different t echnologies. The realit y of ent erprises is charact erized by het erogeneous t echnologies. Consequent ly, t he service bus m ust be able t o connect part icipant s t hat are based on different program m ing languages, operat ing syst em s, or runt im e environm ent s. Furt herm ore, you will usually find a m ult it ude of m iddleware product s and com m unicat ion prot ocols in t he ent erprise, and all t his m ust be support ed by t he service bus. H e t e r oge n e it y of com m u n ica t ion con ce pt s. Sim ilar t o t he het erogeneit y of t echnologies, t he service bus m ust also em brace a variet y of different com m unicat ion concept s. Due t o t he divergent requirem ent s of different applicat ions, t he service bus m ust enable different com m unicat ion m odes. Obviously, you m ust at least have facilit ies for synchronous and asynchronous com m unicat ion. Te ch n ica l" se r vice s." Alt hough t he purpose of t he service bus is prim arily com m unicat ion, it m ust also provide t echnical services such as logging, audit ing, securit y, m essage t ransform at ion, or t ransact ions.

4.4. Conclusion I n t his chapt er, we int roduced t he key concept s of Service- Orient ed Archit ect ure. We began our discussion wit h a general definit ion of soft ware archit ect ure: " . . . set of st at em ent s t hat describe soft ware com ponent s and assigns t he funct ionalit y of t he syst em t o t hese com ponent s. I t describes t he t echnical st ruct ure, const raint s, and charact erist ics of t he com ponent s and t he int erfaces bet ween t hem . . ." The definit ion of an SOA is based on t his definit ion. I t st at es t hat " A Service- Orient ed Archit ect ure ( SOA) is a soft ware archit ect ure t hat is based on t he key concept s applicat ion front end, service, service reposit ory, and service bus."

References [ BCK03] Bass, Len, Paul Clem ent s, and Rick Kazm an . Soft ware Archit ect ure in Pract ice. AddisionWesley, 2003. [ BRJ99] Booch, Grady, Jam es Rum baugh, and I var Jacobson . Unified Modeling Language User Guide. Addision- Wesley, 1999. [ Fow02] Fowler, Mart in . Pat t erns of Ent erprise Applicat ion Archit ect ure. Addision- Wesley, 2002.

URLs ht t p: / / www.sei.cm u.edu/ archit ect ure/ definit ions.ht m l

Chapter 5. Services as Building Blocks The focus of a Service- Orient ed Archit ect ure is on t he funct ional infrast ruct ure and it s business services, not t he t echnical infrast ruct ure and it s t echnical services. A business- orient ed service in an SOA is t ypically concerned wit h one m aj or aspect of t he business, be it a business ent it y, a business funct ion, or a business process. This chapt er provides an in- dept h discussion of t he different t ypes of services. I n Sect ion 5.1, we est ablish a classificat ion t hat dist inguishes bet ween basic, process- cent ric, int erm ediary, and public ent erprise services, and we discuss t he charact erist ics of t hese different classes of services and what t his m eans from a design, im plem ent at ion, and proj ect m anagem ent point of view. These service t ypes have significant ly different charact erist ics wit h respect t o reusabilit y, m aint ainabilit y, scalabilit y, and perform ance; t herefore, it 's crucial t o underst and t hem in order t o im plem ent t hem efficient ly. Sect ion 5.2 int roduces SOA layers for design at t he applicat ion landscape level. As we will dem onst rat e, SOA layers are largely independent of t he syst em archit ect ure's t iers, which is a m aj or benefit of t he SOA approach.

5.1. Service Types An im port ant feat ure of a soft ware archit ect ure is t hat it breaks down t he overall st ruct ure of a soft ware syst em int o sm aller com ponent s. These com ponent s are int ended t o be flexible building blocks.

5.1.1. MOTIVATION Being able t o classify service t ypes is a precondit ion for t he effect ive design of SOAs. Com m on la n gu a ge . Being able t o t alk about t he specific nat ure of different services at an abst ract level will enable t he different st akeholders in an SOA proj ect business analyst s, archit ect s, designers, m anagers, and program m erst o com m unicat e t heir ideas and concerns m ore effect ively. Ve r t ica l slicin g. Classifying services according t o t heir specific nat ure is a prerequisit e t o breaking down a com plex applicat ion landscape int o m anageable part s. I n an SOA, t his will nat urally lead t o a " vert ical slicing," which is an im port ant part of SOA- cent ric proj ect m anagem ent ( see Chapt er 13, "SOA Proj ect Managem ent "). Effe ct ive e st im a t in g. The classificat ion of services is ext rem ely helpful when it com es t o m aking proper est im at es on t heir im plem ent at ion and m aint enance cost . These cost s depend on t he com plexit y of t he im plem ent at ion, t he level of design for reuse, and t he frequency of change. These fact ors will vary by service t ype. Se pa r a t ion of code se gm e n t s w it h diffe r e n t r e u se ch a r a ct e r ist ics. I t is good pract ice t o separat e code t hat is supposed t o be reused from ot her code t hat is unique t o a single proj ect . This separat ion im proves t he reusabilit y of t he " purified" code because it elim inat es any proj ect - specific ballast t hat would com plicat e reuse. I t also helps you avoid fruit less effort s t o m ake proj ect - specific code fit for a reuse t hat will never happen. Being able t o classify your services according t o our classificat ion m at rix will enable a cleaner separat ion bet ween reusable and once- off code. Ch oosin g t h e r igh t im ple m e n t a t ion st r a t e gy. Different t ypes of services require different im plem ent at ion st rat egies. Choosing an unnecessarily com plex im plem ent at ion st rat egy for a sim ple service will nat urally lead t o inefficiencies. For exam ple, services t hat m aint ain conversat ional st at e can be very helpful in order t o sim plify client s. The client im plem ent at ion can be " t hin," and t he service can provide all t he necessary business logic t o support even com plex business processes. However, st at eful services oft en have a negat ive im pact on t he scalabilit y of a dist ribut ed syst em . I t is t herefore advisable t o ident ify services t hat inevit ably require conversat ional st at e and separat e t hem from ot her services. M a n a gin g ch a n ge . Finally, it is im port ant t o separat e business logic t hat is exposed t o a high frequency of change from business logic t hat is m ore st able. Doing so can significant ly reduce t he cost s and risks of m aint enance. Once again, our classificat ion m at rix will be helpful in ident ifying services wit h different change charact erist ics. I t should be not ed t hat t his is good pract ice in any developm ent sit uat ion, not j ust for SOAs. However, SOAs are part icularly well suit ed t o enable t his kind of code separat ion.

5.1.2. CLASSIFICATION We different iat e bet ween four classes of services: basic services, int erm ediary services, processcent ric services, and public ent erprise services. Figure 5- 1 int roduces a basic not ion we will use t hroughout t his book.

Figu r e 5 - 1 . W e dist in gu ish ba sic, pr oce ss- ce n t r ic, in t e r m e dia r y, a n d pu blic e n t e r pr ise se r vice s.

I n t he rem ainder of t his book, we will also use t he t erm s part icipant or SOA part icipant . These t erm s com prise bot h applicat ion front ends and services. Table 5- 1 provides an overview of t he different service t ypes and t heir key charact erist ics.

Ta ble 5 - 1 . Se r vice t ype s

Ba sic Se r vice s

I n t e r m e dia r y Se r vice s

Pr oce ssCe n t r ic Se r vice s

Pu blic En t e r pr ise Se r vice s

Sim ple dat acent ric or logic- cent ric services

Technology gat eways, adapt ers, façades, and funct ionalit yadding services

Encapsulat e process logic

Service shared wit h ot her ent erprises or part ner organizat ions

I m ple m e n t a t ion Low t o Com ple x it y m oderat e

Moderat e t o high

High

Service specific

St a t e M a n a ge m e n t

St at eless

St at eless

St at eful

Service specific

Re u sa bilit y

High

Low

Low

High

Fr e qu e n cy of Ch a n ge

Low

Moderat e t o high

High

Low

M a n da t or y Ele m e n t of SOA

Yes

No

No

No

D e scr ipt ion

We now exam ine t he elem ent s of Table 5- 1.

5.1.3. BASIC SERVICES Basic services are t he foundat ion of t he SOA. They are pure servers in t he SOA and m aint ain no conversat ional session st at e. Basic services cut int o dat a- cent ric and logic- cent ric services. However, in pract ice, t here is oft en a sm oot h t ransit ion from a dat a- cent ric service t o a logiccent ric service. As a m at t er of fact , m any services deal wit h bot h dat a and behavior. Thus, we cannot classify t hem as eit her purely dat a- cent ric or logic- cent ric. Fort unat ely, t he m ix of dat a and business logic does not render t he SOA unsound. Services t hat provide bot h dat a and business logic can be as agile and reusable as " pure" services. Let 's consider, for exam ple, a cont ract adm inist rat ion service. This service t ypically st ores dat a set s t hat represent cont ract s. I n t his respect , t his service is dat a- cent ric. But t his service also needs t o provide plausibilit y checks t hat decide whet her any dat a set represent valid cont ract s. I n t he second respect , t his service is logic- cent ric.

5.1.3.1 Data-Centric Services I t is t he purpose of a dat a- cent ric service t o handle persist ent dat a. This includes t he st orage and ret rieval of dat a, locking m echanism s, and t ransact ion m anagem ent . A dat a- cent ric service also handles ( and ut ilizes) a physical dat a st orage facilit y such as a relat ional dat abase, file syst em , or t ape library. I n t his respect , a dat a- cent ric service behaves sim ilarly t o t he dat a access layer of a t radit ional applicat ion. The m aj or difference is t he vert ical layering of dat a- cent ric services. Whereas a t radit ional dat a access layer m anages dat a for t he ent ire applicat ion, a dat a- cent ric service deals wit h one m aj or business ent it y only. Furt herm ore, a dat a- cent ric service st rict ly encapsulat es it s dat a ent it ies. Any ot her service t hat requires access t o t his dat a needs t o use t he service int erface of t he corresponding dat a- cent ric service ( see Figure 5- 2 ) . Consequent ly, an applicat ion requires several coordinat ed dat a- cent ric services.

Figu r e 5 - 2 . Th e SOA st r ict ly de fin e s t h e ow n e r sh ip of da t a .

One of t he m ost im port ant t asks of t he SOA archit ect is t o ident ify t he relevant business ent it ies t hat are represent ed by dat a- cent ric services. This t ask is sim ilar t o t radit ional analysis of t he business dom ain. Met hods t hat are based on ent it y relat ionship m odels ( ER) or obj ect orient eddesign provide a sound base for t he design of services. Not ice t hat t hese t echniques are prim arily used for designing t he dat a obj ect s t hat are eit her m anaged by services or t hat serve as input and out put dat a st ruct ures. I t is im port ant t o underst and t hat cross- service relat ionships are not allowed, and as such, no cross- service navigat ion exist s as in dist ribut ed obj ect t echnology. This m eans t hat t he com plex value obj ect s m anaged by t he services m ust be sufficient ly selfcont ained or m ust cont ain unique ident ifiers t hat enable t hem t o relat e one com plex value obj ect t o anot her. Sim ilar t o t he design of obj ect s or abst ract dat a t ypes, a dat a- cent ric service can also encapsulat e dat a behavior. I n t his respect , SOAs provide m any of t he benefit s of obj ect orient at ion. However, SOAs do not require obj ect - orient ed program m ing languages for t heir im plem ent at ion. The independence of program m ing languages and t he underlying t echnology is one of t he great est st rengt hs of t he SOA approach. I n pract ice, you will oft en find t hat program m ing languages such as COBOL, C, or PL/ I and t radit ional t ransact ion processing m onit ors are used for t he im plem ent at ion of m ission- crit ical services. However, t he usage of SOAs and dat a- cent ric services raises cert ain issues. Alt hough t he vert ical layering t hat is leveraged by SOAs is wort hwhile, especially for big applicat ions, benefit s such as flexibilit y and reusabilit y are not wit hout cost . Tradit ional applicat ions t ypically access one m onolit hic dat a st ore t hat has no vert ical subst ruct ure even if t he applicat ion is horizont ally layered. The funct ionalit y of t he underlying dat abase or t ransact ion m onit or is fully leveraged. Physical t ransact ions oft en span all part s of t he dat a m odel and, from t he developers' perspect ive, t his is very convenient . There is no need for explicit considerat ions of dat a int egrit y. This is achieved in a t ransparent fashion by t he dat abase or t ransact ion m onit or. The downside is a result ing m onolit hic st ruct ure wit h m any im plicit and explicit dependencies. The issue of dat a ownership gains t rem endous im port ance wit h t he vert ical layering of applicat ions. Alt hough dat a ownership has been on t he agenda since t he very first days of dat a m odeling, it was not really relevant for t he im plem ent at ion of t radit ional applicat ions due t o t he m onolit hic st ruct ure of t he dat a access layer. So, t his aspect of dat a m odeling was m ainly of academ ic concern and was reflect ed in nam ing convent ions for dat abase t ables and access m odules. Wit h t he vert ical slicing of SOAs, t he assignm ent of ent it ies t o dat a- cent ric services becom es a design decision wit h a m aj or im pact on m any charact erist ics of t he result ing applicat ions. I n such a case, explicit effort s are required t o overcom e t hese dependencies ( see Chapt er 8, "Process I nt egrit y ").

5.1.3.2 Logic-Centric Services Logic- cent ric services encapsulat e algorit hm s for com plex calculat ions or business rules. I n t radit ional applicat ions, t his t ype of funct ionalit y is oft en encapsulat ed in libraries and business fram eworks. A very inst ruct ive exam ple for a logic- cent ric service is an insurance product engine. This is a service t hat encapsulat es t he knowledge, t erm s, and condit ions of t he product s of an insurance com pany. I t is capable of com put ing fees, paym ent s, or refunds and can validat e cust om er applicat ions, suggest new offers, or sim ulat e fee adj ust m ent s. This t ype of funct ionalit y is generally part of t he back office syst em for insurance cont ract / policy m anagem ent . Consequent ly, only a back office clerk can provide legally binding inform at ion. This is very unfort unat e for up- t odat e business processes of sales and claim s. Tradit ionally, people have t aken a num ber of approaches t o resolve t his problem . The first deals wit h approxim at ions. I n t his approach, t he front office applicat ion provides an approxim at ion of t he real value t o be validat ed by t he back office using a m anual process. Alt hough t he front office processes of insurance com panies have had t o cope wit h approxim at ions in recent decades, cust om ers nowadays expect accurat e dat a and rapid processing. Part icularly for t he sales of new cont ract s, legally binding dat a t hat is inst ant ly available is increasingly m andat ory. The second approach is based on duplicat ion of t he relevant business logic. On one hand, t his approach sat isfies t he needs of t he sales and claim s depart m ent s, while on t he ot her hand, it can raise m any t echnical issues in t radit ional environm ent s. Most im port ant ly, you m ust cope wit h t he deploym ent of t he appropriat e funct ionalit y and regular updat es t o pot ent ially t housands of PCs and lapt ops. You will also encount er het erogeneous t echnology in front and back office im plem ent at ions. While back office environm ent s are m ost ly based on cent ralized m ainfram es, t he front office syst em s are usually based on PC t echno- logy. This m eans t hat every new version of t he calculat ion rules also requires a port ing effort before deploym ent , a process t hat is bot h cost ly and risky. I n t he t hird approach, t he users of t he front office applicat ions are provided wit h access t o t he back office applicat ions. Alt hough t his m ight look like a solut ion of st riking sim plicit y at first glance, it is not an opt ion for m ost insurance com panies. Many insurance com panies have no desire t o grant access right s t o t heir back office applicat ions t o front office personalpart icularly if independent agencies are involved. A logic- cent ric service providing t he funct ionalit y of t he product engine would overcom e m any of t he aforem ent ioned challenges. The insurance com pany could operat e t his service cent rally ( see Figure 5- 3 ) . I t s funct ionalit y could also be int egrat ed bot h wit h t he int ernal I T syst em s of t he different depart m ent s and wit h business part ners. Furt herm ore, façades ( discussed lat er in t his chapt er) can be ut ilized t o provide different " views" for different t ypes of users ( e.g., independent agencies versus claim s depart m ent ) .

Figu r e 5 - 3 . An in su r a n ce pr odu ct e n gin e e n ca psu la t e s t h e k n ow le dge of t h e in su r a n ce com pa n y's pr odu ct s.

5.1.4. INTERMEDIARY SERVICES I nt erm ediary services can be classified as st at eless services t hat bridge t echnical inconsist encies or design gaps in an archit ect ure. They are bot h client s and servers in an SOA. They cut int o t echnology gat eways, adapt ers, façades, and funct ionalit y- adding services and act as st at eless m ediat ors when bridging t echnological or concept ual gaps. Alt hough m any int erm ediary services seem t o be rat her t echnical, we can dist inguish t hem from t echnical infrast ruct ure services because t hey provide a business- orient ed API , whereas purely t echnical services provide a t echnical API . Many int erm ediary services are highly proj ect - specific.

5.1.4.1 Technology Gateways Technology gat eways bridge t echnological gaps ( see Figure 5- 4 ) . They t herefore incorporat e t wo or m ore t echnologies for com m unicat ion or dat a encoding. Technology gat eways act as proxies for t heir business services and represent t he funct ionalit y of t he underlying services in an environm ent t hat is t echnologically different from t he original business service's runt im e environm ent .

Figu r e 5 - 4 . A t e ch n ology ga t e w a y br idge s t h e ga p be t w e e n diffe r e n t t e ch n ologie s.

They are part icularly useful for legacy int egrat ion proj ect s where one oft en encount ers t he conflict ing dem ands of a need t o ext end exist ing legacy applicat ion wit h a requirem ent not t o " cont am inat e" new com ponent s wit h legacy t echnology. There are good reasons for bot h requirem ent s, and a t echnology gat eway can be very useful in such cases. A t ypical scenario provides a good illust rat ion of t he problem . Assum e t hat a t erm inal- based legacy applicat ion cont ains valuable business logic. Due t o it s out dat ed user int erface, a reengineering proj ect is planned for t he near fut ure. However, a m ore urgent proj ect requires t he business logic of t his applicat ion before t he reengineering proj ect is com plet ed. The new proj ect is im plem ent ed based on a Service- Orient ed Archit ect ure and up- t o- dat e t echnology for dist ribut ed com put ing. I n such a case, a t echnology gat eway can act as a m ediat or bet ween t he legacy applicat ion and t he com ponent s of t he new proj ect . The new proj ect accesses t he t echnology gat eway using a m odern t echnology such as a Web service. The t echnology gat eway t ranslat es t he Web service request s t o t erm inal dat a st ream s in order t o com m unicat e wit h t he legacy applicat ion. Alt hough it is a m onolit hic applicat ion t hat serves all request s, it is advisable t o dist inguish bet ween different services according t o service- orient ed design principles. I t is an irony of ent erprise I T t hat you never know whet her t he reengineering proj ect of t he legacy applicat ion will ever t ake place. However, even if t his is t he case, t he design of t he exist ing services will be very useful. For t hese services, you only m ust replace t he legacy im plem ent at ion wit h t he reengineered basic services.

5.1.4.2 Adapters An adapt er is a special t ype of int erm ediary service t hat m aps t he signat ures and m essage form at s of one service t o t he requirem ent s of a client .

We can illust rat e t he concept wit h an airline booking exam ple. Assum e t hat airline A m erges wit h airline B and t hat bot h airlines already have an SOA in place. Apart from ot her services, bot h airlines have an est ablished cust om er service. More t han likely, one of t he first business requirem ent s aft er t he m erger will be t o enable access t o cust om er dat a across t he borders of t he t wo form er corporat ions. For t he sake of cust om er service, t his feat ure should be put in place as soon as possible. I t is also a very t ypical requirem ent t hat t he exist ing applicat ions of bot h airlines should rem ain largely unalt ered. A sim ple solut ion t o t his problem is based on t wo adapt ers. One m aps request s of applicat ion 1 t o service 2, and t he ot her m aps request s of applicat ion 2 t o service 1.

5.1.4.3 Façades The purpose of a façade is t o provide a different view ( probably aggregat ed) of one or m ore exist ing services ( see Figure 5- 5 ) . As such, façades oft en act also as t echnology gat eways and/ or adapt ers.

Figu r e 5 - 5 . A fa ça de se r vice pr ovide s a h igh - le ve l vie w of on e or m or e ba sic se r vice s.

An in- dept h discussion of façades is found in Gam m a, et . al. [ GHJV95] . Alt hough Gam m a describes t he encapsulat ion of obj ect - orient ed subsyst em s, t he ent ire concept is widely t ransferable t o SOAs. Gam m a defines a façade as " . . . a unified int erface t o a set of int erfaces . . . t hat m akes t he subsyst em easier t o use." Façades can be used t o provide a specific view of a set of underlying services. As such, a façade can act as a sim plifying access layer for a proj ect . I t can hide funct ions t hat are not needed by t he proj ect , aggregat e services, or add st andard param et ers. Furt herm ore, a façade can hide t echnical com plexit y and het erogeneit y. Façades can also be very useful in a rat her unexpect ed wayt hey can serve as a single point of coordinat ion bet ween a proj ect 's developm ent t eam and a m aint enance t eam t hat is responsible for t he SOA. Cont rary t o t he proj ect - specific view, a com m on service access layer provides uniform access t o

t he com plet e funct ional infrast ruct ure of t he organizat ion. Creat ing one is a very t em pt ing idea because such a layer could provide easy access t o t he ent ire funct ional infrast ruct ure for all proj ect s. You could also add specific value t o t his layer, such as support for dist ribut ed t ransact ions and t he capabilit y t o hide t heir com plexit y from proj ect s. I t also would m ake t he developm ent and deploym ent of services easier because t here would be exact ly one unified and m andat ory process t o m ake t he services available. I n part icular, som e current J2EE- based ent erprise archit ect ures t end t o em ploy t his t ype of access layer ( see Figure 5- 6 ) .

Figu r e 5 - 6 . Fr om t h e pr oj e ct 's poin t of vie w , a se r vice a cce ss la ye r is a con ve n ie n t w a y t o a cce ss t h e fu n ct ion a l in fr a st r u ct u r e of t h e or ga n iza t ion . H ow e ve r , a t t h e e n t e r pr ise le ve l, t h e se la ye r s h a ve som e se ve r e disa dva n t a ge s. D u e t o t h e ir m on olit h ic n a t u r e , t h e y t e n d t o spoil m a n y of t h e be n e fit s of a n SOA.

I n spit e of t hese benefit s, t he idea of a com m on service access layer has cert ain disadvant ages. Many sim ilarit ies exist bet ween service access layers and t radit ional dat a access layers. As we have already discussed, a dat a access layer spans t he ent ire dat a m odel of an applicat ion. I t is very convenient for t he developm ent of t his part icular applicat ion, but it result s in a m onolit hic st ruct ure t hat m akes reuse or st andalone usage of vert ical subcom ponent s difficult . The t echnology of t he dat a access layer is also a very st rong const raint for t he developm ent and for service access layers. Though a com m on service access layer is very well suit ed t o t he developm ent of single proj ect s, it also has m any unwant ed charact erist ics at t he ent erprise level. I nt roducing a binding design rule t hat leverages a com m on service access layer will probably lead t o except ions, which will out weigh m any of t he benefit s of t he access layer. Event ually, t hese except ions will est ablish an uncont rollable shadow archit ect ure t hat reflect s t he necessit ies of real- world het erogeneit ies. Because t he shadow archit ect ure is " officially forbidden," t he archit ect ure t eam cannot openly discuss m any im port ant design decisions. This ult im at ely leads t o solut ions t hat are driven by short - t erm polit ical feasibilit y rat her t han by long- t erm design goals. A com m on service access layer im pact s bot h backend and front end int egrat ion. The possible consequences are illust rat ed in t he following exam ple. Assum e a large organizat ion has a t radit ional applicat ion landscape based on m ainfram es, t ransact ion m onit ors, and COBOL. New

applicat ions are required as part of a new J2EE- based st rat egy. The COBOL applicat ion should be reengineered and reused as funct ional backend services. I n t his scenario, one generally needs t o design vert ical slices of previously m onolit hic m ainfram e subsyst em s t o creat e services of reasonable granularit y. I n pract ice, t he int egrat ion of new services becom es a m aj or challenge for t he new SOA. As a result of t he " com m on service access layer" design st rat egy, all aggregat ions m ust be perform ed in t he J2EE cont ainer at a layer above t he service access layer. Alt hough t his m ight look reasonable at first glance, t he int egrat ion wit h t he Java layer is not accept able in m any cases due t o com plicat ed t echnical designs, m aint ainabilit y requirem ent s, t ransact ion securit y issues, developm ent cost s, and perform ance issues. More oft en, a m ainfram e- based int erm ediary service is m uch m ore efficient in all t hese respect s. One will also encount er difficult ies at t he front end due t o different t echnologies. The ent erprise benefit s great ly when it can choose t he best of breed product s from a funct ional perspect ive rat her t han from a t echnical perspect ive. For applicat ions based on nonJava- t echnology such as .NET, t he Java layer provides no value. At t he sam e t im e, it int roduces a new elem ent of com plexit y. While all business logic is im plem ent ed in C# and COBOL, t he Java layer only passes t he calls and param et ers from t he front end t o t he backendt he Java layer adds no value.

Keep Access Layers Project-Specific The requirem ent s of proj ect s wit h regard t o access t o an underlying funct ional infrast ruct ure are generally very different . I nt roducing generic designs leads t o a significant overhead in im plem ent at ion and unwant ed dependencies at a t echnological level. I t is t herefore highly advisable t o design and use access layers for t he purpose of a single proj ect only.

5.1.4.4 Functionality-Adding Services I n m any cases, you want t o add funct ionalit y t o a service wit hout changing t he service it self. I n t his case, you would est ablish a funct ionalit y- adding service t hat provides t he funct ionalit y of t he original service and adds t he required new charact erist ics. There could be several reasons for such a design. I f t he original service is a t hird- part y product for which t here is no source code available, a funct ionalit y- adding service can be ext rem ely helpful. The funct ionalit y- adding service can also serve as an evolut ionary st ep. I f t he original service is current ly under const ruct ion by a different developm ent t eam , one can decouple bot h developm ent s wit h t he funct ionalit y- adding service. Consequent ly, a second st ep is required t o reint egrat e bot h developm ent effort s int o one service in order t o achieve a clean design. The funct ionalit y- adding service could also be t he first st ep for reim plem ent ing t he original service. This is a part icularly good st rat egy if t he original service is of poor qualit y, if you need t he addit ional funct ionalit y as soon as possible, or if you int end t o im prove t he qualit y of t he original service. I n t his case, you would creat e a pure façade in t he first st ep. I n t he second st ep, you would add t he new funct ionalit y. Finally, in t he t hird st ep, you would reengineer all t he funct ionalit y of t he original service and m igrat e it in sm all port ions t o t he funct ionalit y- adding service unt il it im plem ent s t he ent ire funct ionalit y, and t hen t he original service could be decom m issioned.

5.1.5. PROCESS-CENTRIC SERVICES Process- cent ric services can encapsulat e t he knowledge of t he organizat ion's business processes.

They cont rol and m aint ain t heir st at e. From a t echnical viewpoint , process- cent ric services are t he m ost sophist icat ed class of services. They require careful design and deliberat e effort s t o achieve an efficient im plem ent at ion. Sim ilar t o int erm ediary services, a process- cent ric service act s as a client and server sim ult aneously. A m aj or difference from int erm ediary services is t he fact t hat process- cent ric services are st at eful because t hey m ust m aint ain t he st at e of a process for t heir client s. As wit h every addit ional elem ent of an archit ect ure, process- cent ric services int roduce som e com plexit y. However, t here are cert ain benefit s: En ca psu la t e pr oce ss com ple x it y. Process- cent ric services can facilit at e applicat ion front ends. They can com plet ely hide t he com plexit y of process cont rol from t he applicat ion. The process- cent ric service enables different t eam s t o work concurrent ly on t he im plem ent at ion of present at ion and processes. The service int erface ult im at ely leads t o a clear encapsulat ion and facilit at es t est ing and int egrat ion. The rich business API s t hat process- cent ric services t ypically provide facilit at es t he developm ent of very lean applicat ions. En a ble loa d ba la n cin g. Process- cent ric services enable load balancing nat urally. While t he applicat ion front end focuses on t he present at ion, a service can execut e t he underlying processes on anot her m achine. This division of labor can be part icularly helpful in Web applicat ions where user int erface responsiveness is a high priorit y. Le ve r a ge m u lt i- ch a n n e l a pplica t ion s. Mult i- channel applicat ions can require process logic t hat is shared by m ult iple channels. Feat ures such as channel swit ching or co- browsing ult im at ely need an inst ance t hat cont rols t he process and t hat is independent of t he channel. Se pa r a t e pr oce ss logic. Process logic should be carefully separat ed from core business logic and dialog cont rol logic. Process- cent ric services provide appropriat e m easures t o do j ust t hat . When process- cent ric services are in place, you can assign core business logic t o basic services, and you can im plem ent dialog cont rol in t he applicat ion front end. The separat ion of process logic is t he precondit ion for efficient business process m anagem ent ( see Chapt er 7, " SOA and Business Process Managem ent " ) . I t is not ewort hy t hat properly designing process- cent ric services and achieving t he aforem ent ioned benefit s is not an easy t ask. Many t radit ional designs suffer from an inaccurat e encapsulat ion of t he process logic. You'll rarely find applicat ions wit h a clear dist inct ion bet ween business logic and process cont rol, and you'll seldom find a clear dist inct ion bet ween process cont rol and dialog cont rol. I ndividuals m ust have m uch experience t o design t hese layers properly. The sam e holds for process- cent ric services. Chapt er 7 gives an in- dept h discussion on t his issue and provides valuable design guidelines. Not ice t hat process- cent ric services are m ost ly proj ect - specific. I n t he airline booking exam ple, t he required business logic and dat a is largely independent of concret e usage. A t ravel agent requires far m ore processes t han t he cust om er, but bot h are concerned wit h t he sam e business ent it ies. Therefore, and despit e all obvious benefit s, one m ust bear in m ind t hat process- cent ric services do not cont ribut e t o t he funct ional infrast ruct ure of t he SOA due t o t heir m arginal reusabilit y. Process- cent ric services are not m andat ory for an SOA. Because it is highly advisable t o keep an archit ect ure as sim ple as possible, you m ust carefully consider t he t rade offs when considering im plem ent ing process- cent ric services. Figures 5- 7 and 5- 8 illust rat e t wo different approaches. I n Figure 5- 7 , t he applicat ion front end m akes use of a process- cent ric service. As an alt ernat ive t o process- cent ric service, t he applicat ion front end can encapsulat e t he processes in it s process cont rol layer ( see Figure 5- 8 ) . Process- specific obj ect s or subrout ines represent t he process definit ion and cont rol t he ent ire workflow t hat is init iat ed by t he applicat ion. Alt hough t his approach appears t o be very sim ple, it can be appropriat e for m any real world cases.

Figu r e 5 - 7 . Th e a pplica t ion fr on t e n d ca n de le ga t e t h e e n t ir e pr oce ss con t r ol t o a pr oce ss- ce n t r ic se r vice t h a t e x e cu t e s t h e pr oce ss on be h a lf of t h e a pplica t ion fr on t e n d.

Figu r e 5 - 8 . I n m a n y ca se s, t h e a pplica t ion fr on t e n d con t r ols t h e e n t ir e pr oce ss. Pr oce ss obj e ct s or su br ou t in e s in t h e pr oce ss con t r ol la ye r e n ca psu la t e t h e pr oce ss de fin it ion a n d in vok e t h e a ct or s of t h e pr oce ss.

5.1.6. PUBLIC ENTERPRISE SERVICES Whereas m ost of t he service t ypes described previously are only for use wit hin t he boundaries of

a part icular ent erprise, public ent erprise services are services t hat an ent erprise offers t o part ners and cust om ers. For exam ple, a shipping com pany can offer services t hat enable large cust om ers t o t rack t heir shipm ent s as part of t heir own j ust - in- t im e m anagem ent syst em s. I n anot her exam ple, a t elecom carrier m ight offer an SMS ( Short Messages Service) service, enabling cust om ers t o easily add SMS funct ionalit y t o t heir syst em s. Because t he consum ers of public ent erprise services are usually not known in advance and t he relat ionship bet ween consum er and provider is m uch looser, t hese services have very specific requirem ent s: I n t e r fa ce a t t h e bu sin e ss docu m e n t le ve l. I nt erfaces at t he ent erprise level have t he granularit y of business docum ent s. They have a st andalone business m eaning and include t he com plet e cont ext t hat is necessary t o be legally unam biguous. Therefore, ent erprise services are coarse- grained. D e cou plin g. Ent erprise services need t o support decoupling of t he business part ners. This generally im plies asynchronous com m unicat ion and payload sem ant ics. Se cu r it y. Crossing ent erprise borders raises securit y issues. Today, m any t ypical SOAs focus on int ra- ent erprise services, where it is m uch easier t o define a pragm at ic securit y policy. Crossing t he organizat ion's borders im plies t he need for a m uch higher st andard of securit y m echanism s such as aut hent icat ion, encrypt ion, and access cont rol. Accou n t in g/ billin g. While int erdepart m ent al account ing assesses t he capabilit y of different depart m ent s wit hout t he need for cash flow, t he billing of cross- ent erprise services im plies a real cash flow. I t is t herefore necessary t o put m ore reliable m echanism s in place. SLA. The operat ions of a public ent erprise service will probably be regulat ed by SLAs ( Service Level Agreem ent s) , which will lead t o a different form of t reat m ent for t hese services. A norm al precondit ion of effect ive SLA cont rol is service m et ering.

5.2. Layers on the Enterprise Level I n t his chapt er, we have already discussed different service t ypes. Now we t ake a first look at t he overall st ruct ure of an applicat ion landscape and how t he services relat e t o each ot her. Tradit ionally, soft ware layers provide im port ant levels of abst ract ion. You can assum e t hat layers are sequent ially ordered, where layer N is above layer N+ 1. Code segm ent s wit hin one layer N can use ot her code segm ent s wit h t he sam e layer in addit ion t o code segm ent s of t he layers N+ 1. I n a dist ribut ed environm ent , t he concept of t iers exist s, where a t ier is a set of cont iguous layers t hat can be deployed separat ely. Alt hough layers and t iers are ext rem ely useful abst ract ions for t he const ruct ion of single applicat ions ( or services) , neit her is suit ed as an abst ract ion at t he ent erprise level. Service- Orient ed Archit ect ures provide applicat ion front ends and services t hat are m uch m ore suit ed t o t his purpose. SOA layers, which you m ust not confuse wit h t hese t radit ional soft ware layers, and t iers provide a concept ual st ruct ure at t he ent erprise level t hat organizes t he applicat ion front ends and services ( see Figure 5- 9 ) . Each layer cont ains dist inct t ypes of services and applicat ion front ends: En t e r pr ise la ye r . The t op layer of SOAs cont ains applicat ion front ends and public ent erprise services, which are t he end- point s t hat provide access t o t he SOA. These endpoint s facilit at e bot h t he com m unicat ion bet ween end users and t he SOA ( applicat ion front ends) and enable cross- ent erprise ( or cross- business unit ) int egrat ion ( public ent erprise services) . Pr oce ss la ye r . The process layer cont ains process- cent ric servicest he m ost advanced service t ype. I n t e r m e dia r y la ye r . The t hird layer cont ains int erm ediary services. These services act as façades, t echnology gat eways, and adapt ers. You can also use an int erm ediary service in order t o add funct ionalit y t o an exist ing service. Ba sic la ye r . The bot t om layer cont ains t he basic services of t he SOA. Basic services represent t he foundat ion of t he SOA by providing t he business logic and dat a. The basic layer also cont ains proxies for ot her com panies' public ent erprise services.

Figu r e 5 - 9 . N o 1 :1 r e la t ion sh ip e x ist s be t w e e n t r a dit ion a l t ie r s a n d SOA la ye r s. Th e se con ce pt s a ct u a lly a r e la r ge ly in de pe n de n t .

As we will see in Chapt er 6, " The Archit ect ural Roadm ap," m any problem s can be solved wit h t wo or t hree SOA layers. I n t hese cases, t here is no benefit t o art ificially int roducing addit ional layers sim ply t o have a " com plet e" SOA. Recall t hat an SOA is about sim plificat ion. As long as a problem can be solved wit h sim ple m easures, it is best t o do so. Alt hough we will not discuss t his m at t er in great det ail, we m ust briefly consider t he deploym ent of SOAs and t he result ing syst em archit ect ure in order t o prevent a com m on m isunderst anding: SOA layers do not correspond t o physical t iers. I t is not necessary for services, which originat e from different SOA layers, t o be deployed at different t iers. Nor m ust all services of one SOA layer be deployed at t he sam e locat ion. The syst em archit ect ure is driven by m at t ers such as available hardware and syst em soft ware, syst em m anagem ent requirem ent s, and com pat ibilit y. These issues are largely independent of requirem ent s such as m aint ainabilit y or sim plicit y t hat drive t he design of t he services. Act ually, t he design of t he SOA and t he syst em archit ect ure are largely independent aspect s of t he applicat ion landscape, which is t he rem arkable st rengt h of t he SOA paradigm .

Decouple System and Software Architecture A m aj or benefit of t he SOA paradigm is t o be able t o design t he syst em and t he soft ware archit ect ure largely independent ly of each ot her. This result s in a high level of flexibilit y regarding t he deploym ent of t he SOA. Do not int roduce unnecessary t echnical dependencies and rules for short - t erm benefit s! SOAs enable a largely independent design of t he syst em and t he soft ware archit ect ure. This result s in a high level of flexibilit y regarding t he deploym ent of t he SOA. Do not spoil t hese benefit s by int roducing t echnical " short - cut s" and design rules for short - t erm benefit s!

Figure 5- 10 shows an exam ple of how a syst em archit ect ure and SOA layers can relat e. I t depict s t hree t iers at t he syst em archit ect ure levelt he Web server, applicat ion server, and host . You can see t hat t he SOA layers do not m ap direct ly t o t hese t iers.

Figu r e 5 - 1 0 . Th e de ploym e n t of a n SOA is la r ge ly in de pe n de n t of t h e SOA la ye r s. [View full size image]

5.3. Conclusion I n t his chapt er, we have int roduced four classes of services: basic, process- cent ric, int erm ediary, and public ent erprise services. These different services are predom inant ly used by t he applicat ion front ends, which are t he act ive elem ent s of t he SOA but are not services t hem selves. I t is im port ant for a designer of a Service- Orient ed Archit ect ure t o be aware of t he dist inct ive charact erist ics of t he different service t ypes because t hey all have different requirem ent s wit h respect t o design, im plem ent at ion, operat ion, and m aint enance. I n addit ion, t his service classificat ion is ideally suit ed t o provide an overall st ruct ure wit hin t he SOA t hat is expressed by t he different SOA layers, including ent erprise layer, process layer, int erm ediary layer, and basic layer.

References [ GHJV95] Gam m a, Erich, Richard Helm , Ralph Johnson, and John Vlissides . Design Pat t erns. Addison- Wesley, 1995. [ Her2000] Herzum , Pet er and Oliver Sim s . Business Com ponent Fact ory: A Com prehensive Overview of Com ponent - Based Developm ent for t he Ent erprise. OMG Press, 2000. [ Rei1992] Reisig, Wolfgang . A Prim er in Pet ri Net Design. New York: Springer Com pass I nt ernat ional 1992.

Chapter 6. The Architectural Roadmap I m plem ent ing an SOA at t he ent erprise level is a significant endeavor t hat is likely t o t ake m any years. During t his t im e, we probably will encount er m any obst acles and will have t o cope wit h frequent ly changing requirem ent s. I t is t herefore essent ial t hat we look at a realist ic roadm ap for rolling out our Service- Orient ed Archit ect ure. I t is im port ant t hat our archit ect ure and our roadm ap is designed t o cope wit h t he com plexit y and dynam ics of an ent erprise environm ent . Chapt er 1 int roduced an SOA- based ent erprise I T renovat ion roadm ap. I n t his chapt er, we will focus on t he archit ect ural aspect s of t his roadm ap. I n Chapt er 12 of t his book, we will t hen focus on t he polit ical and organizat ional aspect s.

6.1. The Architectural Roadmap For t he archit ect ural roadm ap, we have ident ified t hree expansion st ages t hat signify different levels of m at urit y of an SOA. The expansion st ages indicat e t he allocat ion of responsibilit y bet ween t he applicat ion front ends and t he services ( see Figure 6- 1 ) . The first st age is called fundam ent al SOA. I n t his expansion st age, m uch of t he com plexit y and responsibilit y is st ill allocat ed at t he applicat ion front end. Alt hough a fundam ent al SOA does not provide all feat ures of a fully leveraged SOA, it is already a useful plat form for an ent erprise applicat ion landscape. The next expansion st age is called net worked SOA. At t his st age, int erm ediary services aggregat e low- level basic services t o m ore sophist icat ed services. Finally, we have process- enabled SOAs. At t his st age, t he applicat ion front ends delegat e process cont rol t o t he SOA.

Figu r e 6 - 1 . Th e e x pa n sion st a ge s fu n da m e n t a l SOA, n e t w or k e d SOA, a n d pr oce ss- e n a ble d SOA a r e m ile st on e s of t h e a r ch it e ct u r a l r oa dm a p t o t h e se r vice - e n a ble d e n t e r pr ise .

I n Figure 6- 2 , we depict t he t ypical developm ent of an applicat ion. I t shows how perm anent change request s and a lack of cont inuously applied archit ect ural concept dim inish m aint ainabilit y and how an SOA- driven refact oring can reest ablish agilit y. I n Part I I I of t his book, we present several case st udies t hat show how different ent erprises experienced difficult ies when ext ending t heir hist orically grown applicat ion landscapes and how a t ransit ion t o a Service- Orient ed Archit ect ure helped t o increm ent ally overcom e t heir difficult ies.

Figu r e 6 - 2 . Agon y ve r su s a gilit y.

[View full size image]

I t should be not ed t hat t he concept of expansion st ages cannot be applied in a black- and- whit e fashion. As soon as t he SOA is est ablished, you will probably find areas of differing m at urit y t hat are at a different expansion st age wit hin it fort unat ely, wit hout any undesired im pact on t he SOA it self. This reveals a m aj or benefit of Service- Orient ed Archit ect urest hey enable ent erprises t o st art sm all and evolve in sm all st eps as required in fut ure proj ect s. Act ually, t he SOA enables bot h evolut ionary developm ent of t echnology and funct ionalit y. Furt herm ore, t he SOA support s different developm ent s t hat run in parallel or in sequence. The SOA brings all t hese developm ent s t oget her in a sm oot h fashion, wit hout harm ing a single proj ect or t he overall SOA endeavor. However, t he expansion st ages differ in regard of t he dist ribut ion of responsibilit ies bet ween applicat ion front ends and services. Wit h an increasing m at urat ion of t he SOA, t he services gain m ore and m ore responsibilit ies.

Allow Different Expansion Stages Typically, different expansion st ages coexist in t he sam e SOA. Do not fight t his het erogeneit y! Develop different areas of t he SOA in parallel t o t he requirem ent s of business- driven proj ect s.

The definit ion of your SOA st rat egy and t he required level of m at urit y st rongly depend on t he scope of t he business int egrat ion you are planning t o reach. Obviously, im plem ent ing an SOA st rat egy is always a quest ion of budget , and t he furt her you are planning t o advance your SOA, t he longer you will have t o invest ( see also Chapt er 12, " The Organizat ional SOA Roadm ap," for a discussion on t he budget requirem ent s of an SOA) . I dent ifying t he required scope of t he business int egrat ion is usually a good first st ep on t he way t oward t he definit ion of t he overall SOA st rat egy because t here is a st rong correlat ion bet ween t he int egrat ion level one is aim ing for on t he one hand and t he required m at urit y level of t he SOA on t he ot her. Figure 6- 3 depict s t his dependency.

Figu r e 6 - 3 . Th e m a t u r a t ion of t h e SOA w it h r e spe ct t o t h e e x pa n sion st a ge s oft e n cor r e la t e s t o a n e n la r ge m e n t of t h e scope of bu sin e ss

in t e gr a t ion .

For exam ple, if t he required scope of business int egrat ion is only t he int ra- depart m ent al level, a fundam ent al SOA will already help achieving good levels of m aint ainabilit y of t he syst em at hand. Obviously, t he lim it ed int egrat ion scope effect ively lim it s t he am ount of flexibilit y or agilit y t hat can be achieved because only a lim it ed view t o t he overall business processes is st udied. I n addit ion, invest ing in a very advanced SOA ( such as a process- enabled SOA) is likely t o be cost ly because t he required I T invest m ent s would not be j ust ified by t he result ing business benefit s. On t he cont rary, in a scenario where t he scope of business int egrat ion t o be achieved is broadly defined, an approach based on a net worked or process- enabled SOA is m uch m ore efficient . However, an ext rem ely com plex scenario, including cross- ent erprise- int egrat ion and com plex processes, m ight not even be feasible on t he basis of a fundam ent al SOA. When deciding on your SOA st rat egy, you should always ask yourself how m uch agilit y do you really need? Also consider t he learning curve in your organizat ion. Are you fit for process- enabled services? Choose an SOA st rat egy t hat best m at ches your agilit y requirem ent s, your int egrat ion scenario, your archit ect ural and developm ent skills, and ( obviously) your budget const raint s. Your SOA st rat egy should be based on an evolut ionary approach, subsequent ly developing your SOA expansion st ages, t aking all of t he above m ent ioned fact ors int o considerat ion. This approach also allows you t o gradually broaden your int egrat ion horizon, m oving from int radepart m ent al int egrat ion t oward cross- depart m ent al or even cross- ent erprise int egrat ion.

6.2. Fundamental SOA A fundam ent al SOA consist s of t wo layers: t he basic layer and t he ent erprise layer. Dist inguishing t hese t wo layers helps single applicat ions t o define a proper high- level st ruct ure. I t also enables t wo or m ore applicat ions t o share business- logic and live dat a. Alt hough a fundam ent al SOA represent s a rat her sim ple approach, it provides a st rong plat form for large ent erprise applicat ion landscapes. I n fact , it is a m aj or im provem ent com pared t o m any of t oday's real- world scenarios. I t is also an excellent st art ing point for t he int roduct ion of an ent erprise- wide SOA because it enables ent erprise organizat ions t o st art sm all on t he t echnical side and focus on ot her crit ical success fact ors ( see Chapt er 12, " The Organizat ional SOA Roadm ap" ) . A sim ple exam ple ( see Figure 6- 4 ) shows how one applicat ion can be divided int o m eaningful com ponent s using an SOA. The Airline Web sit e ut ilizes four services t hat encapsulat e t he m aj or business ent it ies and t heir behaviors t hat are relevant t o t he business processes t hat are exposed t o t he cust om ers.

Figu r e 6 - 4 . A fu n da m e n t a l SOA con sist s of t w o la ye r s. Applica t ion fr on t e n ds a n d ba sic se r vice s pr ovide a fu lly fu n ct ion a l ba se for e n t e r pr ise a pplica t ion s. [View full size image]

We can now expand t he original exam ple by adding anot her applicat ion. A key charact erist ic of a fundam ent al SOA is t hat it enables t wo or m ore applicat ions t o share live dat a and business logic. I n our exam ple, we consider a t radit ional billing applicat ion t hat is required t o keep t rack of t erm s of paym ent and t he handling of dem and not es ( see Figure 6- 5 ) . This t ype of applicat ion is t radit ionally provided wit h cust om er and billing dat a t hrough night ly bat ch j obs. I nt egrat ing billing and invoicing int o a fundam ent al SOA enables t he booking and t he billing applicat ions t o share live dat a. I n pract ice, [ 1] t his m eans t hat t he billing applicat ion get s access t o t he cust om er service and billing services t hat , in t urn, m ake bat ch processes t hat t ransfer dat a bet ween t he applicat ions obsolet e. [ 2] I n addit ion, t here are clear benefit s for t he booking syst em . As a result of up- t o- dat e billing, t he booking syst em also has access t o precise, up- t o- dat e dat a at any t im e. This increases t he capabilit y of t he booking syst em t o t reat cust om er request s in a m ore appropriat e m anner. For exam ple, a change in t he credibilit y st at us of a cust om er, which is det ect ed by t he billing syst em , is inst ant ly available for t he booking syst em . As previously m ent ioned, in m any t radit ional environm ent s, t his inform at ion is only available t o t he dat abases

of t he booking applicat ion aft er night ly bat ch updat es. [1] It should be noted that a real-world scenario would be much more complex, as depicted in Figure 6-5. It would involve a customer care application, a sort of workbasket, and additional business services. However, the aforementioned benefits still apply. [2]

Note that an update in one application is instantly visible in the other application. In a traditional EAI scenario, you should consider decoupling the original data change from notifying the second application due to throughput and performance considerations. In the SOA world, these notifications are obsolete.

Figu r e 6 - 5 . En t e r pr ise a pplica t ion in t e gr a t ion be com e s obsole t e if se ve r a l a pplica t ion s sh a r e live da t a .

[View full size image]

The int roduct ion of a fundam ent al SOA is t he first im port ant st ep t oward a t ruly SOA- enabled ent erprise. The following sum m arizes t he m ain charact erist ics and scope of a fundam ent al SOA: A fundam ent al SOA is an appropriat e base for an ent erprise applicat ion landscape. Due t o it s sim plicit y, it is t echnically easy t o im plem ent . I t is a good st art ing point for an SOA t hat enables t he int roduct ion of m ore advanced expansion st ages in t he fut ure. The applicat ion front ends are st ill com plex. They m ust cope wit h t he cont rol of business processes and t he full int egrat ion of t he backend. A fundam ent al SOA increases t he m aint ainabilit y of an ent erprise applicat ion landscape. Shared services can m ake dat a replicat ion ( ent erprise applicat ion int egrat ion) largely obsolet e.

6.3. Networked SOA The next expansion st age is called net worked SOA, and it deals wit h backend com plexit y in addit ion t o t echnical and concept ual int egrat ion. I t includes a layer of int erm ediary services t hat can include façades, t echnology gat eways, adapt ers, and funct ionalit y adding services. We st art our discussion wit h façades. As we discussed in Chapt er 5, " Services as Building Blocks," façades can serve various needs. Most im port ant ly, t hey encapsulat e som e of t he com plexit y of underlying basic services by providing an aggregat ed API t hat enables client s t o m ore easily ut ilize t he funct ionalit y of t he basic layer. I n Chapt er 8, "Process I nt egrit y ," we will discuss one aspect of t his com plexit yt he challenges of achieving process consist ency. I nt roducing a façade is one possible solut ion t o t his issue. Figure 6- 6 provides an exam ple of t he booking and t he billing services t hat m ust updat e t heir dat abases in a coordinat ed m anner. Depending on t he det ailed requirem ent s and t he concret e t echnology upon which t he booking and t he billing services are built , it is not always sim ple t o guarant ee consist ency. Act ually, you will want t o shield client developers from t his kind of com plexit y. I t is oft en necessary t o apply part icular skills in serverside developm ent t o m ast er t he challenges of t he design and im plem ent at ion t asks. Moreover, if m ult iple client s require sim ilar funct ionalit y, t here should be no duplicat ion of workload. Thus, in our exam ple, t his com plexit y is encapsulat ed by t he int erm ediary service " BookAndBill."

Figu r e 6 - 6 . Th e in t e r m e dia r y se r vice Book An dBill e n ca psu la t e s t h e h a n dlin g of dist r ibu t e d t r a n sa ct ion s t h a t spa n t h e se r vice s Book in g a n d Billin g.

[View full size image]

Ut ilizing t echnology gat eways is a handy t echnique t o enable one service t o be used in different t echnical environm ent s. [ 3] I n Figure 6- 7 , we describe a scenario in which t he flight service t hat is im plem ent ed on CI CS is exposed t o EJB, .NET, and MQSeries environm ent s by t echnology gat eways. This enables developers t o specify, design, develop, t est , operat e, and m aint ain exact ly

one inst ance of t he flight service, including live dat a, and reuse it in m any het erogeneous environm ent s at t he sam e t im e. [3]

Note that from a purely technical point of view, there is no difference between a technology gateway and an additional interface to the underlying basic service. Thus, it is possible to add some Java classes to an existing PL/I based service in order to provide an additional interfacing technology. The communication between the Java code and the PL/I code would become an internal matter of the service. The main difference between these two approaches is at the project management and team organization level. Most often, it is advisable to separate these technically different artifacts. In practice, you will find different subteams working on the different areas anyway. Separating their work by means of a service contract, configuration management, and distinct entries in the service repository is generally preferable. The same discussion applies to the adapters discussed in the following paragraphs.

Figu r e 6 - 7 . Te ch n ology ga t e w a ys e x pose t h e fu n ct ion a lit y of se r vice s in t e ch n ologica lly diffe r e n t e n vir on m e n t s. [View full size image]

The perfect world does not need t echnology gat eways. I n t he first place, you would not encount er het erogeneous t echnologies. Secondly, given t hat t he client s are het erogeneous in t he real world, you would choose a single uniform bridging t echnology t o int egrat e all client s. I n our exam ple, you m ight want t o im plem ent a t ype of XML RPC int erface direct ly as part of t he CI CS- based service and use t his int erface in all client environm ent s. Unfort unat ely, t he ongoing evolut ion of t echnology can raise m any unforeseen issues and new requirem ent s. You m ight want t o adopt current im provem ent s wit hout reim plem ent ing exist ing services. Furt herm ore, you cannot predict which t echnologies you will need t o int egrat e in t he fut ure. I t is also unclear which polit ical or com m ercial const raint s will have an im pact on a t echnology decision at a specific point in t im e. As a m at t er of fact , t he SOA paradigm enables t he creat ion of clear designs t hat cope wit h t his kind of het erogeneit y. Alt hough it is not desirable t o have m ult iple t echnology gat eways t hat provide access t o t he sam e service, t hese t echnology gat eways do no real harm t o t he archit ect ure. Figure 6- 8 depict s a t ypical chronology. I t shows t he m aj or m ilest ones of a check- in applicat ion aft er it s launch in 1986. The first m ilest one was t he int egrat ion wit h part ner airlines. Besides ot her requirem ent s, t he part ner airlines needed access t o flight dat a. This int egrat ion was accom plished by ext ending t he dialog cont rol layer t hat previously support ed only t he 3270 present at ion. Aft er int egrat ion wit h t he part ner airline, t he dialog cont rol layer also had t o com m unicat e wit h m essage queues t hat int egrat e t he part ner airlines. The int egrat ion code bet ween t he dialog cont rol and t he m essage queues sim ulat ed t erm inal sessions. This result ed in

t he fact t hat every change in t he cont rol flow of t he applicat ionsuch as new m asksrequired a change in t he int egrat ion code. Alt hough t his dependency reduced agilit y, it was st ill accept able. The next m ilest one was t he launch of an online port al. This was im plem ent ed in Java and also required access t o flight dat a. The int egrat ion wit h t he exist ing m essage queues seem ed t o be t he easiest and cheapest solut ion. However, t his decision was rat her short - sight ed because any change in one syst em had a pot ent ial im pact on ot her syst em s, and m aint enance becam e a night m are. I t was at t his point t hat a new requirem ent was specified. A graphical user int erface was needed t o replace t he 3270 represent at ion. Because t his GUI required a com plet ely new dialog cont rol, t he new proj ect t urned out t o be infeasible. Driven by t he requirem ent t o im plem ent a m odern user int erface, an SOA was put in place. Based on t he SOA and t echnology gat eways, t he GUI becam e feasible and t he m aint ainence cost s could be significant ly reduced. Finally, it provided t he base for fut ure developm ent s such as m obile int egrat ion.

Figu r e 6 - 8 . Th e t ypica l life cycle of a n e n t e r pr ise a pplica t ion t r a ve r se s diffe r e n t de ve lopm e n t st a ge s. [View full size image]

Adapt ers are useful in int egrat ion scenarios. They bridge concept ual gaps bet ween a service and it s client s. I n t he sim plest case, an adapt er m aps signat ures and t ransform s param et ers. You m ust not underest im at e t he im port ance of such adopt ions. As a m at t er of fact , m uch of t he com plexit y of ent erprise archit ect ures result s from sm all differences in how sim ilar ent it ies are handled in different part s of t he archit ect ure. Alt hough m any of t hese differences do not result from logical considerat ions, t hey are const raint s t hat you m ust accept in real- world proj ect s. For exam ple, t he booking applicat ion can regard any person t hat has regist ered at t he Web sit e as a cust om er, whereas CRM m ight require t hat a person have purchased at least one t icket in t he last t hree years in order t o be considered a cust om er. Figure 6- 9 depict s t hree different scenarios in which you can operat e a booking and CRM applicat ion in parallel. The first scenario represent s t he t radit ional archit ect ure. Two dist inct applicat ions are int egrat ed t hrough night ly bat ch runs. As a result , t wo separat e dat abases cont ain redundant dat a t hat is synchronized overnight . The second scenario is service- enabled.

Alt hough t his m ight already have several benefit s, t wo separat e dat abases are st ill involved, along wit h t he associat ed issues. The final scenario abolishes one redundant dat abase. The CRM applicat ion ut ilizes t he Cust om er service of t he Booking applicat ion using an adapt er. This is t he scenario of choice because it provides t he sim plest st ruct ure, abolishes bat ches, enables live dat a sharing, reduces m aint enance cost s, and guarant ees dat a consist ency.

Figu r e 6 - 9 . Ada pt e r s br idge con ce pt u a l ga ps be t w e e n se r vice s a n d t h e ir clie n t s, a n d t h e y m a p sign a t u r e s a n d a dopt se m a n t ics. Ada pt e r s a r e a ve r y pow e r fu l t ool for e n a blin g a pplica t ion in t e gr a t ion a ccor din g t o t h e SOA pa r a digm .

[View full size image]

There are various sit uat ions in which funct ionalit y- adding services can be very useful. Let 's consider t hree different exam ples ( see Figure 6- 10) . I n t he first exam ple, a new client uses an exist ing legacy applicat ion. I t is a com m on requirem ent t hat t he legacy applicat ion rem ain unt ouched while t he funct ionalit y is expanded. I n our exam ple, an addit ional service m aint ains add- it ional at t ribut es for dat a ent it ies st ored by t he legacy applicat ion. Sim ilar t o dat abase views, t he dat a of bot h sources is t ransparent ly j oined by a funct ionalit y- adding service in t he int erm ediary layer. From t he client 's perspect ive, a single service delivers all t he dat a. The second exam ple involves a packaged applicat ion. Because you t ypically have no source code available, any changes or enhancem ent s t o exist ing funct ionalit y m ust be done by t he vendor of t his package or by an int erm ediary service. The t hird exam ple uses a t radit ional client / server applicat ion. This applicat ion has a client t hat requires a specific com m unicat ion prot ocol t o be funct ional, which suspends t he possibilit y of sim ply changing t he exist ing server applicat ion. I f a new client requires addit ional funct ionalit y, you eit her m ust m odify t he exist ing client or add t his funct ionalit y wit h an appropriat e service in t he int erm ediary layer.

Figu r e 6 - 1 0 . Th r e e diffe r e n t e x a m ple s for t h e u sa ge of fu n ct ion a lit ya ddin g se r vice s in t h e in t e r m e dia r y la ye r . [View full size image]

The int roduct ion of a net worked SOA represent s t he second st ep t oward t he evolut ion of an SOAenabled ent erprise. The following sum m arizes t he m ain charact erist ics and scope of a net worked SOA: Applicat ion front ends can be m ore light weight when com pared t o a fundam ent al SOA. However, t hey rem ain com plex because t hey m ust cope wit h t he business processes. I nt erm ediary services bridge t echnical and concept ual gaps. The applicat ion front ends are shielded from t he com plexit y of backend syst em s. A net worked SOA enables t he ent erprise t o flexibly int egrat e it s soft ware asset s independent ly of underlying t echnology.

6.4. Process-Enabled SOA The t hird expansion st age is t he fully leveraged SOA. The key feat ure of t he process- enabled SOA is t he m aint enance of process st at e in process- cent ric services. Sim ilar t o int erm ediary services, a process- cent ric service is bot h client and server in an SOA. The m aj or difference bet ween t hese service t ypes is t he fact t hat process- cent ric services are st at eful. This is a crucial difference because handling st at e is a crit ical issue for server- side soft ware. There are several possible reasons for int roducing a process- cent ric service: Encapsulat ing t he com plexit y of processes Sharing st at e bet ween m ult iple client s Handling long- living processes Encapsulat ing process st at e in a process- cent ric service enables applicat ion front ends t o be sim ple and light weight and at t he sam e t im e very user- friendly wit h an elaborat e handling of t he user session. Figure 6- 11 ext ends t he booking exam ple first int roduced in Figure 6- 5 . A new service " Booking process" encapsulat es t he business process " Booking." The Booking process ut ilizes t he BookAndBill service and t he Cust om er service. Alt hough m ost of t he work is carried out by t he Booking service, t he applicat ion front end has access t o all layers of t he SOA.

Figu r e 6 - 1 1 . A fu lly de ve lope d pr oce ss- e n a ble d SOA h a s fou r la ye r s. [View full size image]

The scenario in Figure 6- 11 is t he st raight forward evolut ionary st ep arising from t he scenario in Figure 6- 5 . However, a green- field design would be different . I n our exam ple, t he BookAndBill façade could becom e part of t he process- cent ric service. Om it t ing t he BookAndBill service ( see Figure 6- 12) reduces our scenario t o t hree layers. Because one of t he prim ary goals of SOAs is sim plicit y, t his is a desirable st ep. Furt herm ore, reducing t he num ber of t iers bet ween t he applicat ion front end and basic layer reduces t he syst em 's lat ency and t he num ber of elem ent s t hat can pot ent ially fail.

Figu r e 6 - 1 2 . Th e Book in g pr oce ss e n ca psu la t e s a ll fu n ct ion a lit y n e ce ssa r y t o book fligh t s. I t a lso m a in t a in s t h e se ssion st a t e .

[View full size image]

But t here are also possible scenarios where our BookAndBill service can be beneficial. Assum e t hat several client s, such as a Web sit e, a B2B gat eway, and a m obile applicat ion, are running sim ult aneously ( see Figure 6- 13) . Typically, t hese applicat ions require a dist inct process- cent ric service. Fact oring out t he BookAndBill funct ionalit y becom es a reasonable design decision in t his case.

Figu r e 6 - 1 3 . Se ve r a l pr oce sse s ca n u t ilize t h e Book An dBill se r vice a t t h e sa m e t im e . [View full size image]

However, as depict ed in Figure 6- 14, an alt ernat ive design exist s t hat fact ors out com m on booking process funct ionalit y t o a process- cent ric service t hat can incorporat e t he BookAndBill funct ionalit y. The shared process- cent ric service bot h m aint ains a channel- independent process st at e and shields t he channel- specific process- cent ric services from backend com plexit y.

Figu r e 6 - 1 4 . I n ge n e r a l, e ve r y ch a n n e l of a m u lt ich a n n e l a r ch it e ct u r e r e qu ir e s ch a n n e l- spe cific pr oce ss logic. N e ve r t h e le ss, diffe r e n t ch a n n e ls a lso sh a r e be h a vior .

[View full size image]

Finally, process- cent ric services can handle long- living processesin part icular if user int eract ion or

asynchronous backend services are involved. Obviously, one requires a locat ion t hat m aint ains t he st at e of a process for t he durat ion of it s execut ion. Assum e t hat t he booking process support s wait ing list s. This m eans t hat a cust om er can regist er a seat in a fully booked flight . I f cancellat ions occur, t he cust om er is not ified using em ail or SMS ( m obile phone t ext m essages) , or he can ret rieve t he st at us of his regist rat ion in a lat er session at t he Web sit e. This kind of funct ionalit y im plies t hat t he booking process is long- living. Ext ernal event s ( here, cancellat ions of ot her cust om ers) change t he st at e of t he process. A possible im plem ent at ion scenario would assum e t hat t he cancellat ion process is anot her client of t he booking process, as depict ed in Figure 6- 15. I f a cancellat ion is m ade for a flight for which t he wait ing list is not em pt y, t he cancellat ion process st art s t he appropriat e booking process. The booking process not ifies t he wait ing cust om er and convert s t he ent ry in t he wait ing list int o a proper booking.

Figu r e 6 - 1 5 . Th e book in g pr oce ss is a syn ch r on ou sly cou ple d w it h t h e w a it in g list se r vice .

A process- enabled SOA represent s t he endpoint of t he evolut ion we have described in t his chapt er. A process- enabled SOA Enables light weight applicat ion front ends t hat " only" need t o worry about user int eract ion Encapsulat es com plexit y of business processes and handles t heir st at e Encapsulat es com plexit y of backend syst em s in int erm ediary services and process- cent ric services Enables t he process logic locat ed in t he process layer t o be clearly separat ed from ot her t ypes of code including dialog cont rol t hat is locat ed in t he applicat ion front ends and t he basic services' core business logic

Represent s t he m ost sophist icat ed expansion st age and is t herefore m ore difficult t o im plem ent t han ot her expansion st ages I s required for int egrat ion of highly independent organizat ions and t he im plem ent at ion of com plex processes. However, it is not as cost effect ive in well- cont rolled environm ent s such as int ra- depart m ent al int egrat ion scenarios.

6.5. Conclusion I n t he previous chapt er, we int roduced four SOA layers t hat serve as a concept ual const ruct t hat enables t he efficient organizat ion of ent erprise applicat ion landscapes. Based on t he concept of SOA layers, we dist inguished t hree expansion st ages t hat define t he level of sophist icat ion an SOA has achieved. The first expansion st agefundam ent al SOAdelivers a m aint ainable business infrast ruct ure t hat provides t he applicat ion landscape wit h flexible building blocks cont aining t he business logic and dat a of t he ent erprise. While t he int egrat ion of t hese building blocks is st ill allocat ed at t he applicat ion front end in t he first expansion st age, t he second st agenet worked SOAencapsulat es t his kind of com plexit y in an int erm ediary layer. Finally, t he process- enabled SOA depict s t he t ruly SOA- enabled ent erprise, leveraging support for t he business processes of t he ent erprise in process- cent ric services.

Chapter 7. SOA and Business Process Management I n previous chapt ers, we discussed t he general " renovat ion roadm ap" t hat describes t he evolut ion of an ent erprise I T infrast ruct ure t oward a m ore agile Service- Orient ed Archit ect ure. We exam ined fundam ent al and net worked SOAs as t he init ial st ages in great det ail. According t o our roadm ap, t he final st age in t his evolut ion are process- enabled SOAs. Alt hough process- cent ric services can be realized in m any different ways ( and can t hus t ake m any different shapes and form s) , business process m anagem ent ( BPM) represent s possibly t he m ost consequent approach t o process- enabling an SOA. Consequent ly, we provide a general int roduct ion t o BPM in t his chapt er, followed by a discussion on how BPM fit s int o t he SOA landscape. I n t his chapt er, we focus m ore on t he t echnical ( com put at ional and archit ect ural) aspect s, while Chapt ers 12 , " The Organizat ional SOA Roadm ap," and Chapt er 13, " SOA- Driven Proj ect Managem ent ," concent rat e on t he I T st rat egy and proj ect - m anagem ent aspect s.

7.1. Introduction to BPM Business process m anagem ent has a num ber of predecessors. Following t he Tot al Qualit y Managem ent wave of t he lat e 1980s, a new paradigm em erged in t he early 1990s: Business Process Reengineering ( BPR) . I n 1993, Michael Ham m er and Jam es Cham py published t heir New York Tim es best seller Reengineering t he Corporat ion [ HC93] , which was followed by a plet hora of ot her m anagem ent books on t he t opic of process- orient at ion and reengineering. The prom ise of reengineering was t o deliver dram at ic business perform ance im provem ent s in a relat ively short period of t im e by com plet ely reinvent ing exist ing business processes, t hat is, st art ing from scrat ch wit h new, opt im ized ways of doing business, t hrowing out t he old, encrust ed, inefficient procedures of t he past . However, aft er t he init ial BPR boom ( and m illions of dollars spent on m anagem ent consult ancies t o lead BPR proj ect s in Fort une 500 com panies) , t he process m ovem ent becam e idle. Many proj ect s result ed in com plet e failure, wit h expert s claim ing t hat bet ween 60% and 70% of reengineering effort s failing t o achieve expect ed result s. Ham m er and ot hers at t ribut e t hese failure rat es t o resist ance t o change, lack of underst anding of t he business m odels and underlying processes, and failure of nerve on t he part of t he client com panies. Alm ost t en years aft er BPR, you can observe a revival of t he process m ovem ent under t he um brella t erm business process m anagem ent ( BPM) , as advocat ed by Howard Sm it h and Pet er Fingar in BPM: The Third Wave [ SF03] . When com paring BPR and BPM, it appears as if one t hing has fundam ent ally changed: While in t he 1990s reengineering m eant " st art ing from scrat ch," process m anagem ent builds on and t ransform s t hat which already exist sit recom m ends increm ent al change and evolut ionary opt im izat ion. However, BPM is st ill about processes, which are t he m ain com pet it ive different iat or in all business act ivit y. BPM is a general m anagem ent t opic t hat focuses on t he st rat egic and operat ional aspect s of process orient at ion in a given business area. Mapping a BPM m odel ont o an ent erprise I T landscape is a challenging t ask, which has som e int erest ing t echnical as well as I T m anagem ent relat ed aspect s.

7.1.1. BPM VERSUS BPMS When discussing BPM, it is im port ant t o different iat e bet ween t he business and I T sides. When looking at t he business side of BPM, you will oft en find relat ed keywords such as I SO 9000 and Six Sigm a. The I T side of BPM is oft en accom panied by keywords such as process m odeling and workflow m anagem ent ( see Figure 7- 1 ) .

Figu r e 7 - 1 . I T a n d bu sin e ss pe ople h a ve diffe r e n t vie w s of t h e pr oce sse s of a n or ga n iza t ion . [View full size image]

A BPMS ( Business Process Managem ent Syst em ) provides t he t echnical plat form for realizing BPM m anagem ent init iat ives. I t com prises several part s including a BPM engine, facilit ies for business process m onit oring, design t ools and facilit ies for sim ulat ion. A BPMS inst allat ion can include several product s or cust om m ade soft ware com ponent s. Closing t he gap bet ween t he business and I T sides of BPM ( or ot her process- orient ed approaches) has been som et hing of a holy grail in I T for t wo decades. Current ly, it seem s t hat t he solut ion m ight be a conversion bet ween m ore soft ware engineering approaches such as CASE ( Com put er Aided Soft ware Design) and MDA ( Model Driven Archit ect ures) on one hand, and workflow m anagem ent and BPM approaches on t he ot her ( see [ Fra03] ). BPM int roduces t he concept of " process processing" and st resses t hat t his concept is not lim it ed t o t he aut om at ic execut ion of digit al process m odels, but " encom passes t he discovery, design, and deploym ent of business processes, as well as t he execut ive, adm inist rat ive, and supervisory cont rol over t hem t o ensure t hat t hey rem ain com pliant wit h business obj ect ives" [ SF03] . This describes at a high level t he feat ures t hat are t ypically included in BPMS, a new soft ware cat egory t hat support s t he ent ire lifecycle of m odeling, execut ing, and m onit oring business processes.

7.1.2. WHEN TO CHOOSE A BPMS The cost and com plexit y of int roducing a BPM engine or plat form should not be underest im at ed. Most BPM product s are fairly com plex and require a m ixt ure of highly skilled developers and adm inist rat ors for inst allat ion, im plem ent at ion, and m aint enance. Thus, t he decision for a BPM solut ion is a crit ical one. When does it act ually m ake sense t o consider a t echnical BPM solut ion, and what are t he decision crit eria? I T a n d bu sin e ss m u st w or k h a n d- in - h a n d. You m ust underst and t hat t he int roduct ion of a BPM soft ware solut ion will not enable t he process- orient ed ent erprise on it s own. I f t he ent erprise is not prepared for a process- orient ed operat ion at t he business level, a BPM plat form will always be lim it ed t o covering very sm all sect ions of t he ent erprise. On t he ot her hand, if an ent erprise has gone t hrough t he process of defining and docum ent ing key business processes, for exam ple as part of an I SO 9000 cert ificat ion or a Six Sigm a qualit y m anagem ent proj ect , it is likely t hat t he int roduct ion of a BPM engine will be widely accept ed at bot h t he t echnological and business levels. See Chapt er 12 for a m ore det ailed discussion.

Ut ilize pr oce ss t e m pla t e s. I t is int erest ing t hat som e BPM vendors are st art ing t o bundle t heir BPM plat form s wit h process t em plat es for several different vert ical indust ries, such as banking, insurance, and m anufact uring. Alt hough a process definit ion, such as claim s processing in insurance, is likely t o be different for every single com pany, t he provision of a t em plat e for specific processes as a st art ing point can be ext rem ely valuable. This can be part icularly int erest ing if you consider t hat t he BPM concept s are not about st art ing from scrat ch ( like in business process reengineering) but rat her are about increm ent al changes: st art ing a proj ect for defining and im plem ent ing process definit ions from scrat ch im poses significant ly higher risks t han a proj ect t hat can st art from a working process t em plat e, even if t his t em plat e m ust be adapt ed t o fit t he individual needs of t he ent erprise. M a t ch t h e r igh t t e ch n ology t o you r pr oble m . To det erm ine whet her it m akes sense t o choose a BPM engine t o support a part icular business process, you should underst and t he nat ure of t he business process it selfdifferent t ypes of processes are best addressed using different t ypes of t echnologies. Two key charact erist ics of a business process are it s com plexit y and dynam ism ( frequency of change) on one hand, and t he degree of coordinat ion t he part icular process requires on t he ot her ( see Figure 7- 2 ) .

Figu r e 7 - 2 . D iffe r e n t t ype s of pr oce sse s a r e be st a ddr e sse d u sin g diffe r e n t t ype s of t e ch n ologie s, su ch a s EAI , a pplica t ion se r ve r s, or BPM / W or k flow .

Adopt t h e de ve lopm e n t m ode l. A BPM plat form can also provide significant benefit s at t he level of soft ware developm ent processes: I t provides a com plet e developm ent m odel t hat enforces a clean separat ion bet ween business logic and low- level t echnical code. This can be a m aj or benefit , especially if a t eam has highly het erogeneous skills. BPMs are also

part icularly useful in sit uat ions where people have m ade m any ad- hoc changes t o t he business m odel and a t ransparent runt im e m anagem ent of process inst ances is required. However, given t he high resource requirem ent s of a BPM int roduct ion, you m ust carefully evaluat e t he real need for BPM. Generally, you should avoid using BPM if t he workflows or business processes are of a sim ple t o m oderat e com plexit y because a t eam wit h knowledge of an exist ing developm ent plat form would be significant ly fast er in developing developing t hese workflows or business processes using an ordinary program m ing language.

7.1.3. OVERVIEW OF A BPM SYSTEM A BPM soft ware product should enable business analyst s, soft ware developers, and syst em adm inist rat ors t o m odel and deploy business processes ( at developm ent t im e) and t o int eract wit h, m onit or, and analyze process inst ances ( at run- t im e) . The following looks at act ual m odeling and execut ion of business processes in a BPMS.

7.1.3.1 Modeling Languages A num ber of com pet ing m odeling languages have been proposed by a variet y of different indust ry consort ia, alt hough t he final shoot - out has yet t o happen. However, alm ost all t hese process m odeling languages are based on or at least influenced by t he t heoret ical concept s of Pet ri [ Rei 1992] and or t he m ore recent work of Milner [ Mil80 ] . Two of t he m ost popular approaches are Business Process Execut ion Language for Web Services ( BPEL4WS) and Business Process Modeling Language ( BPML) . BPEL4WS is based on I BM's Web Service Flow Language ( WSFL) and Microsoft 's XLANG. BPML is developed by t he Business Process Managem ent I nit iat ive ( see BPMI .org) , which is support ed by vendors such as I nt alio, SAP, SeeBeyond, Sun, and ot hers. While process definit ion languages such as BPEL4WS and BPML are designed t o enable t he exchange of process definit ions bet ween process engines from different vendors, graphical m odeling languages exist t hat enable hum an users t o underst and, creat e, and m odify process definit ions m ore easily. The BPMN ( Business Process Modeling Not at ion) is a language t hat has been defined by t he BPMI in order t o support a st andardized, graphical represent at ion of business process diagram s. I n a way, t he BPMN approach is sim ilar t o t he UML approach, in t hat it provides a graphical represent at ion of obj ect m odels t hat is independent of t he im plem ent at ion language. I n fact , m any sim ilarit ies exist bet ween UML act ivit y diagram s and BPMN. However, BPMN is specifically designed wit h BPM engines in m ind. Consequent ly, it includes a m apping of it s graphical obj ect s t o BPML. We use BPMN at various point s in t his book t o graphically illust rat e business processes. Figure 7- 3 shows how BPMN is posit ioned at t he int erface bet ween business and I T: While UML is widely used wit hin t he I T organizat ion as a m eans t o com m unicat e abst ract concept s and m odels, BPMN aim s t o becom e t he de fact o st andard used bet ween I T and business t o discuss t he scope and funct ionalit y of processes and applicat ions.

Figu r e 7 - 3 . BPM N is posit ion e d a t t h e in t e r fa ce be t w e e n bu sin e ss a n d I T.

Anot her feat ure of m any m odern BPM syst em s is t he capabilit y t o provide different views of a process definit ion, including a m ore abst ract view designed for business analyst s and a m uch m ore det ailed view for t echnical st aff.

7.1.3.2 Architecture of a BPM System Depending on t he BPM product at hand, processes are usually m odeled graphically ( e.g., based on a graphical not at ion such as BPMN) , st ored in a block- st ruct ured m odel ( e.g., in BPEL4WS or BPML) , and execut ed by a process engine. What is com m on t o m ost pure- play BPM engines is t heir foundat ion on a m ixt ure of Pi- Calculus [ Mil80 ] and Pet ri Net m odels [ Rei1992] . BPM t ools vary in t heir support for different m odeling languages, applicat ion int egrat ion, process m onit oring, and so fort h. However, we provide an overview of a generic BPM syst em archit ect ure in Figure 74. At t he heart of a BPM is t he process engine, which creat es and int erpret s runt im e inst ances of form al process definit ions. Process definit ions ( developm ent t im e) and process inst ances ( runt im e) are st ored in reposit ories, and t he syst em provides appropriat e int erfaces t o design, deploy, and configure process definit ions and t o m onit or and m anage process inst ances.

Figu r e 7 - 4 . Ove r vie w of a ge n e r ic BPM syst e m a r ch it e ct u r e .

7.1.4. VISION AND CAVEAT The BPM vision is a st rong oneinst ead of hard- coding inform at ion and rules regarding im port ant business processes direct ly int o t he applicat ion code, we t ake t his inform at ion out of t he applicat ion syst em s and put it under t he cont rol of a BPM syst em . The BPM facilit at es t he m odificat ion, reconfigurat ion, and opt im izat ion of process definit ions wit h graphical t ools t hat can be used by less t echnology- orient ed business analyst s. As depict ed in Figure 7- 5 , t his BPM vision and our SOA vision are a very good fit . The SOA provides t he backend funct ionalit y t hat is required by a BPMS in order t o im plem ent it s process funct ionalit y. All t he concept s discussed wit h respect t o SOAincluding t he evolut ionary developm ent of basic, int erm ediary, and process layersm ake sense in light of t his pict ure. SOA becom es t he enabling infrast ruct ure for t he process- orient ed ent erprise.

Figu r e 7 - 5 . Rou n dt r ip e n gin e e r in g a t t h e bu sin e ss le ve l: Th e SOA pr ovide s t h e e n a blin g bu sin e ss in fr a st r u ct u r e .

However, t here is one caveat t o all t his: Rem em ber t he hub- and- spoke m odel as it was proposed by lat e- ninet een- ninet iet h EAI - product s? I n t he hub- and- spoke m odel, a cent ralized hub connect s a variet y of different applicat ions, enabling t he seam less int erplay of different t ransact ions in t hese sub- syst em s. A pict ure sim ilar t o t he cent ral hub is t he fam ous " soft ware bus" prom ot ed by t he CORBA com m unit y. This never worked out in your ent erprise? There was never one cent ralized soft ware bus or hub, but rat her a m ult it ude of com pet ing int egrat ion and EAI product s and proj ect s? Does t his BPM vision not resem ble t he cent ral hub/ bus vision and cause a m aj or

concern? Would t his BPM vision not require t he cent ralizat ion of all business process definit ions, as depict ed in Figure 7- 6 ?

Figu r e 7 - 6 . Th e t opologie s of BPM a n d h u b- a n d- spok e syst e m s look sim ila r .

I n a way, you are right if you experience déj à vu. The chances t hat your ent erprise will st andardize on j ust one BPM plat form and m igrat e all it s core business processes ont o t his cent ralized plat form are indeed fairly slim . However, we believe t hat t here is st ill light on t he horizon and t hat BPM could play an im port ant part in your ent erprise's t ransit ion t oward a m ore agile and flexible I T infrast ruct ure wit h an underlying SOA. I n Figure 7- 7 , we show a m ore realist ic scenario: I ndividual depart m ent s or organizat ions will adopt SOA and BPM. The int egrat ion across t hese organizat ional boundaries will not be based on cent ralized hub, bus, or BPM archit ect ures for a variet y of reasons. The cloud in t he figure represent s t he void bet ween t he different organizat ions, a void t hat will not be filled by a cent ralized t echnology st andard cont rolled by a single polit ical unit . I nst ead, t he connect ions bet ween t he different organizat ions will cont inue t o be ad- hoc and oft en short - lived. A num ber of com m on ( som et im es conflict ing) st andards, such as SOAP and CORBA I I OP, enable basic com m unicat ion bet ween t he different organizat ions. I n som e cases, higher- level prot ocols such as ebXML or Roset t aNet m ight play a role. However, organizat ions will have t o cope wit h t he fact t hat t he int egrat ion wit h ext ernal organizat ions ( or ot her business unit s, subsidiaries, et c.) will st ill need prot ocols and logic t hat require m ore flexibilit y t han wit hin a well cont rolled, int ra- organizat ional environm ent . I f t he organizat ion hooks int o t his " cloud" using a BPM, t he organizat ion receives a very powerful t ool for react ing t o frequent ly changing requirem ent s for int egrat ion wit h t hird part ies.

Figu r e 7 - 7 . Th e scope of a BPM S is ge n e r a lly lim it e d t o a sin gle bu sin e ss u n it . Cr ossin g t h e bor de r s of t h e or ga n iza t ion r e qu ir e s dist r ibu t e d pr oce ss con t r ol a n d la r ge ly h e t e r oge n e ou s st a n da r ds.

Alt hough t his vision does not prom ise t o unravel t he ent erprise syst em int egrat ion " spaghet t i," it provides a chance for individual organizat ions t o react t o t he realit ies of int ra- and ext ra- com pany syst em int egrat ion in t he m ost flexible m anner, shielding t he syst em s under t heir own cont rol from out side dynam ics and at least providing a cleaner archit ect ure t hat is ext ensible and adapt able.

7.2. BPM and the Process-Enabled SOA Having int roduced t he foundat ions of BPM and it s long- t erm vision, we now t ake a closer look at what t his m eans for t he SOA archit ect .

7.2.1. THE PAST: DATA AND FUNCTIONS VERSUS OBJECTS VERSUS SERVICES This chapt er focused on BPM and process- orient at ion, which com es at t he very end of our " ent erprise renovat ion roadm ap." We should now st ep back and look at t he origins of SOAs and what can be learned from t heir evolut ion. I n t he early days of funct ional program m ing, dat a and funct ionalit y were st rict ly separat ed. Wit h t he em ergence of obj ect orient at ion, people began t o m erge dat a and funct ionalit y int o encapsulat ed, reusable obj ect im plem ent at ions. This worked part icularly well for large, m onolit hic applicat ions, such as com plex graphical user int erfaces. I n t he m iddle of t he 1990s, people st art ed t o apply t he concept s of obj ect orient at ion t o dist ribut ed syst em s. CORBA and a num ber of ot her st andards for dist ribut ed obj ect com put ing em erged. However, when applying dist ribut ed obj ect t echnology in large- scale proj ect s, it event ually becam e clear t hat t his approach had som e severe lim it at ions. As a result , Service- Orient ed Archit ect ures em erged, wit h support ing t echnology plat form s such as XML Web services. So what problem s wit h dist ribut ed obj ect s led t o t he em ergence of SOAs? The first problem t hat we usually cit e is t he fine level of granularit y of dist ribut ed obj ect s, which oft en leads t o perform ance problem s due t o lat ency and ot her issues. An SOA addresses t hese perform ance issues by adopt ing pat t erns of m ore coarse- grained obj ect s, which require less frequent int eract ion bet ween client s and servers, t hus reducing t he num ber of net work roundt rips, m arshalling overhead, and so fort h. However, a second and pot ent ially m ore crit ical problem exist s: Due t o t he com plicat ed int eract ion pat t erns t hat result from t he fine- grained nat ure of dist ribut ed obj ect s, t he reuse of dist ribut ed obj ect s becam e increasingly com plex. The com plex dependencies bet ween different obj ect s prevent ed efficient reuse and m ade t hese syst em s very inflexible and hard t o m aint ain because few people really underst ood t hese dependencies and t he im pact t hat changes t o individual obj ect s and t heir int erfaces would have on overall syst em s. Wit h SOAs, we t ake a deliberat e st ep back from t he highly com plex, fine- grained, highly dependent dist ribut ed obj ect m odels t oward less com plex, relat ively coarse- grained, loosely coupled ( i.e., less dependent ) com ponent int erfaces.

7.2.2. THE FUTURE: CORE BUSINESS LOGIC VERSUS PROCESS CONTROL LOGIC Rat her t han revert t o a paradigm t hat separat es dat a and funct ionalit y, t he SOA should develop an archit ect ure t hat different iat es bet ween core business logic and process cont rol logic. Bot h of t hese concept s com prise dat a and funct ionalit y, alt hough in very different ways. Unfort unat ely, t hese concept s are not as clean as t he concept s of dat a and funct ionalit y, but we believe t hat t hey

are st ill essent ial for t he successful im plem ent at ion of an SOA- based " ent erprise I T renovat ion roadm ap." Let 's t ake a look at each concept using concret e exam ples. Core business logic com prises basic dat a access services, com plex calculat ions, and com plex business rules. Dat a access services represent core business logic because t hey m ake core business ent it ies available t o different syst em s. An exam ple is a service t hat enables different applicat ions t o read and updat e shared cust om er dat a. An exam ple of a com plex calculat ion would be t he calculat ion of an insurance prem ium based on st at ist ical dat a t hat is encapsulat ed by t he service. Different processes such as sales, prem ium adj ust m ent , and risk assessm ent require t his core business logic. I t is not always clear whet her business rules represent core business logic ( residing in a basic service) or whet her t hey should be a direct part of t he process logic ( e.g., residing in a BPMS or process- cent ric service) . Oft en, t his will depend on t he level of com plexit y of t he business rule. The sim ple rule t hat " all orders over USD 100,000 m ust be m anually validat ed" m ight well be part of t he process cont rol logic. However, a com plex claim s validat ion engine t hat incorporat es legislat ive dat a t o check t he validit y of insurance claim s would clearly fall int o t he cat egory of core business logic. Relat ed core business logic usually resides in a single service. As we st at ed before, t hese services are relat ively coarse- grained and loosely coupled ( or independent ) and do not have com plex int erfaces ( even t hough t hey m ight encapsulat e a very com plex piece of logic such as a claim s validat ion engine) . I n part icular, services t hat represent core business logic cont rol t heir own t ransact ions; t hey do not part icipat e in ext ernally cont rolled, pot ent ially long- lived t ransact ions. The int eract ion wit h services represent ing core business logic is usually OLTP- st yle, based on underlying short - lived t ransact ions. However, services represent ing core business logic m ight part icipat e in com plex processes " orchest rat ed" by a BPM or a process- cent ric service. Process cont rol logic, on t he ot her hand, has very different charact erist ics. As discussed at t he beginning of t his chapt er, m any processes are dynam ic in t hat t hey are prone t o frequent change and oft en require com plex coordinat ion wit h t he process part icipant s. Oft en, t hese processes are relat ed t o non- t angible asset s, in service indust ries for exam ple. Ot her exam ples include cont ract m anagem ent , supply- chain m anagem ent , sales of com plex t ailored product s, or soft ware out sourcing processes. Services m anaging process cont rol logic, such as process- cent ric services, do not usually have applicat ion st at e such as currency exchange rat es, cust om er dat a, or airline seat availabilit y. However, t hey do have st at e relat ed t o t he process it self. This process st at e includes inform at ion regarding process part icipant s ( people and services) , input from part icipant s, t he act ual posit ion wit hin t he process flow, and basic rules. Process- cent ric services are highly dependent on ot her services, in part icular t hose services t hat represent t he core business logic. Such services provide t he glue t o bind t he core business services. I n part icular, process- cent ric services m ust coordinat e com plex act ivit ies t hat can span m ore t han one person, several m aj or business ent it ies, m ult iple locat ions, or long periods of t im e. The benefit s of cleanly separat ing SOA services ( see Figure 7- 8 ) int o services cont aining core business logic ( wit h a pot ent ially long lifet im e) and processes cont aining cont rol logic ( wit h a usually m uch short er lifet im e) are m anifold. I n part icular, t his approach should benefit t he overall goal of our " ent erprise renovat ion roadm ap," which is increased agilit y. I f t he separat ion is t horough, changes t o exist ing processes and t he int roduct ion of new processes should happen relat ively sm oot hly because changes will be lim it ed t o services t hat represent process cont rol. Furt herm ore, t he approach support s t he encapsulat ion of crit ical st at eful codein part icular, t his m eans t hat changing one process does not affect anot her process. Finally, business logic will be im plem ent ed only once, t hus helping t o reduce redundancies and inconsist encies.

Figu r e 7 - 8 . A k e y r e qu ir e m e n t t ow a r d a n a gile e n t e r pr ise a r ch it e ct u r e

is t h a t it cle a r ly dist in gu ish e s be t w e e n pr oce ss con t r ol logic a n d cor e bu sin e ss logic.

7.2.3. DESIGN IMPLICATIONS FOR SOA ARCHITECTS Decom posing an SOA int o services t hat represent core business logic and t hose t hat represent process cont rol does not sim ply m ean m oving back from a dist ribut ed obj ect paradigm t oward a funct ional paradigm . Services represent ing core business logic go far beyond sim ple dat a access services. Sim ilarly, process cont rol logic is not lim it ed t o pure funct ionalit y. Alt hough process cont rol logic does not usually own large am ount s of core business dat a, such as t he cust om er dat abase, it st ill can m anage a lot of int ernal dat a, such as user input , input from basic services, and so fort h. The challenge for t he SOA archit ect is t o ident ify and cat egorize services accordingly, t aking m any aspect s int o considerat ion, including business and process requirem ent s, t he exist ing applicat ion landscape, m ake- versus- buy decisions, resource availabilit y ( e.g., developm ent skills) , SOA design considerat ions, and budget const raint s. However, if an ent erprise is serious about t he long- t erm renovat ion of it s I T landscape, t his decom posit ion process will represent an im port ant cornerst one in t he overall process.

7.3. Conclusion As we saw in t his chapt er, an SOA represent s a good foundat ion for adopt ing a process- orient ed approach in t he long t erm . You can int roduce process orient at ion at different levels in an SOA, wit h t he m ost powerful level being represent ed by a Business Process Managem ent Syst em ( BPMS) . However, m igrat ing t o a process- enabled SOA is a long process t hat can easily run for a num ber of years. I n t he m eant im e, ent erprises m ust find ways t o deal wit h processes and, in part icular, process consist ency in t he short t erm . This issue is t he focus for t he next chapt er.

References [ Fra03] Frankel, David S. BPM and MDA: The Rise of Model- Driven Ent erprise Syst em s. Business Process Trends, ht t p: / / www.businessprocesst rends.com / , June 2003. [ HC93] Ham m er, Michael and Jam es Cham py . Re- engineering t he Corporat ion. London: Nicholas Brealey, 1993. [ Mil80] Milner, Robin . " A Calculus of Com m unicat ion Syst em s." Lect ure Not es in Com put er Science, volum e 92, 1980. [ Rei1992] Reisig, Wolfgang . A Prim er in Pet ri Net Design. New York: Springer Com pass I nt ernat ional, 1992. [ SF03] Sm it h, Howard and Pet er Fingar . BPM: The Third Wave. Tam pa: Meghan- Kiffer Pr., 2003.

URLs ht t p: / / www.bpm i.org ht t p: / / www- 306.ibm .com / soft ware/ solut ions/ webservices/ pdf/ WSFL.pdf ht t p: / / www- 106.ibm .com / developerworks/ library/ ws- bpel/ ht t p: / / www.got dot net .com / t eam / xm l_wsspecs/ xlang- c/ default .ht m ht t p: / / www.businessprocesst rends.com /

Chapter 8. Managing Process Integrity Achieving consist ency in t he execut ion of com plex business processes t hat span m ult iple subsyst em s is one of t he m ost challenging problem s in I T. The first part of t his chapt er provides an overview of t he problem scope, com m on solut ions for achieving process int egrit y, and how t hey fit int o an SOA. The second part of t he chapt er provides a set of concret e recom m endat ions for SOA archit ect s, based on an ext ension t o our airline exam ple.

8.1. Data Versus Process Integrity Process int egrit y is not necessarily a well- est ablished or well- defined concept . The core building blocks of process int egrit y are based on t he widely est ablished concept s of dat a int egrit y. However, as we will see in t he following, dat a int egrit y is insufficient for addressing all int egrit y requirem ent s of com plex business processes spanning m ult iple I T syst em s, which is why it is necessary t o int roduce process int egrit y as a concept t hat helps t o address t hese m ore com plex and dem anding requirem ent s.

8.1.1. DATA INTEGRITY Dat a int egrit y is an um brella t erm t hat refers t o t he consist ency, accuracy, and correct ness of dat a. The classical m echanism s for ensuring dat a int egrit y are oft en closely t ied t o t he concept s of relat ional dat abases. The prim ary t ypes of dat a int egrit y include ent it y, dom ain, and referent ial int egrit y . Ent it y int egrit y requires t hat each row in t he t able be uniquely ident ified. Dom ain int egrit y requires t hat a set of dat a values fall wit hin a specific range ( dom ain) for exam ple, a birt h dat e should not be in t he fut ure. Referent ial int egrit y refers t o t he validit y of relat ionships bet ween different dat a t uples. Finally, user- defined dat a int egrit y refers t o t ypes of dat a int egrit y t hat usually cannot be enforced by com m ercial dat abase t ools. User- defined dat a int egrit y is t ypically enforced using a dat a access layer, t riggers, and st ored procedures. These are all fairly t echnical concept s, and t ypical business requirem ent s for dat a int egrit y go far beyond t echnical concept s. These are all fairly t echnical concept s, and t ypical business requirem ent s for dat a int egrit y go far beyond t echnical concept s. These concept s are t ypically lim it ed t o a single dat abase. As you will see in t he following, m ore flexible concept s for ensuring process int egrit y are required if you are leaving t he dom ain of a single dat abase or applicat ion syst em .

8.1.2. PROCESS INTEGRITY The problem wit h com plex business processes t hat span m ult iple I T syst em s goes beyond t he issues of t radit ional dat a consist ency. I n t hese kinds of sit uat ions, we are not dealing wit h short lived updat es of dat a cont ained in a cent ral reposit ory, but inst ead wit h long- lived processes t hat cross m ult iple syst em s. These processes do not oft en have a well- defined st at e because it is not possible t o obt ain access t o all t he part icipant s all t he t im e, a requirem ent necessary t o det erm ine t he process st at e. This is part icularly t rue for processes t hat span t he boundaries of business unit s or ent erprises. Take t he exam ple of a m anufact urer who receives product part s from a supplier. I f t he m anufact urer receives a last - m inut e cancellat ion for an order, t here will be t im e int ervals where t he int ernal syst em s reflect t his cancellat ion while t he order for relat ed part s is st ill wit h t he supplier. This is what we refer t o as a process inconsist ency .

8.1.3. TECHNICAL FAILURES VERSUS BUSINESS EXCEPTIONS The key t o m aint aining process int egrit y is t o ensure t hat failures wit hin or bet ween t he execut ion of t he different st eps t hat com prise a com plex business process are capt ured and t he necessary st eps t aken t o resolve t he problem . I t is necessary t o different iat e bet ween t echnical failures on one hand and except ions and special business cases on t he ot her:

Te ch n ica l fa ilu r e s. Technical failures include dat abase crashes, net work problem s, and program logic violat ions, am ong m any ot hers. Oft en, t hese problem s can be addressed in t heir respect ive cont ext , for exam ple t hrough backup and recovery m echanism s ( provided by t he DBMS; e.g., t ransact ions) , ret ries ( in t he case of net work problem s) , except ion handlers ( e.g., a Java catch clause) , or process rest art s ( e.g., aft er a process t erm inat ed because of a C+ + NULL point er except ion) . However, in m any cases, t echnical failures m ust be addressed using m ore com plex, oft en cust om - built solut ions. For exam ple, syst em s m ust cope wit h net work problem s by t em porarily st oring a process st at e locally unt il t he subsyst em t o which it at t em pt ed t o connect can be reached again. Bu sin e ss e x ce pt ion s. Business except ions can range from very sim ple except ions t o arbit rarily com plex ones. An exam ple of a sim ple except ion is an at t em pt by a cust om er t o book a flight on a dat e t hat lies in t he past . Such a sim ple dom ain inconsist ency ( see t he previous discussion on dat a inconsist encies) can be addressed at t he dat abase level. For t he sake of usabilit y, it can be handled direct ly in t he user int erface ( e.g., Java script in t he browser) . However, sim ple dom ain inconsist encies have local im pact only. An exam ple of a m ore com plex business except ionwit h a m ore proliferat ing im pact is an out of st ock except ion, for exam ple in an online Web shop. A st raight forward solut ion is t o t ell t he cust om er t hat t he request ed it em is not available. However, in t he real world, t his is unaccept able. A bet t er opt ion is t o t rigger a process such as reorder it em t o ensure t hat t he it em is available t he next t im e a cust om er want s t o buy it . However, t his m ight also be unaccept able because t he cust om er is st ill lost . The best solut ion m ight be t o const ant ly m onit or invent ory and reorder in advance. This avoids or at least m inim izes out of st ock sit uat ions and leads t o t he discussion on special cases. Spe cia l ca se s. I n alm ost all com plex business processes, t he com plexit y lies not in t he happy pat h case but in special cases t hat are dependent on t he cont ext of t he process. For exam ple, a t rading syst em m ight choose com plet ely different execut ion pat hs, depending on t he t ype and size of t rade and t he cust om er's risk profile. The business process, under considerat ion, could com prise a credit check of t he cust om er. A negat ive result of t he credit check could be eit her an except ion ( result ing in a refusal of t he t rade) or a special case requiring a different approach ( e.g., result ing in anot her subprocess asking t he cust om er t o provide addit ional collat eral) . I n m any sit uat ions, t he ent ire business process is a " special case" from a process int egrit y point of view. A good exam ple is t hat of airline seat reservat ions: in order t o ensure m axim um ut ilizat ion of airplanes, m any airlines deliberat ely overbook flight s, m aking t he assum pt ion t hat a cert ain percent age of bookings will be canceled or t hat som e passengers will not show up. Such syst em s are const ant ly opt im ized t o find t he opt im um level of overbooking, based on recent flight st at ist ics. Alt hough at first sight , it appears inconsist ent t o overbook a flight , achieving process consist ency in t his case requires finding t he " right " level of overbooking. The boundaries bet ween business except ions, special cases, and com plex processes such as a flight booking are not black and whit e. Oft en, each problem scenario requires it s own specific solut ion. I n t his chapt er, we concent rat e m ainly on problem s t hat are on t he except ion or failure side of t he equat ion and not so m uch on special cases. However, in m any cases, problem s ( except ions or failures) at t he t echnical or business level lead t o process inconsist encies t hat cannot be im m ediat ely addressed and t hat m ust be t reat ed as special cases.

8.1.4. WHO OWNS THE PROCESS LOGIC? Process logic is rarely cent ralized but inst ead is spread across different syst em s, which m akes it hard t o devise generic st rat egies for ensuring process int egrit y. Alt hough in t he ideal world, all key process definit ions would be m anaged by a cent ral BPM ( Business Process Managem ent ) or Workflow Managem ent Syst em ( WMS) , t his is rarely t he case. Alt hough cent ralized m anagem ent is reasonable in t he B2B world because each com pany want s t o ret ain cont rol over it s own

processes, even a single com pany is unlikely t o have cent ralized processes ( or at least process im plem ent at ions capt ured in a cent ral syst em ) . Take t he exam ple of a m anufact uring com pany t hat has t wo syst em s, one for order processing and one for billing. This is quit e a com m on scenario in m any com panies. The t wo syst em s are synchronized t hrough night ly bat ch updat es, where t he order processing m akes all t he changes and addit ions t hat cam e in during t he day available t o t he billing syst em using FTP ( File Transfer Prot ocol) . Assum e t hat a cust om er m akes an order one day and cancels t he order t he next day. The inform at ion about t he order has been passed from t he order processing syst em t o t he billing syst em during t he night ly bat ch. The next day, t he cust om er cancels t he order, and t he order processing syst em is able t o st op t he delivery of t he order t o t he cust om er. However, t he billing syst em cannot be not ified unt il t he following night , when t he next bat ch run is execut ed. I n t he m eant im e, t he billing syst em m ight already have debit ed t he cust om er's account . Upon receiving t he cancellat ion t he next day, t he billing syst em now m ust provide t he cust om er wit h a credit not e or t ake alt ernat ive st eps t o undo t he debit . I n t he real world, t he special cases caused by t he decoupling of such syst em s are m uch m ore com plex and have oft en led t o a sit uat ion where individual syst em s have grown t o a t rem endously large size wit h a huge int ernal com plexit y. Of course, we now have a wide range of m iddleware t echnologies t o enable real- t im e dat a exchange bet ween syst em s where process logic is split bet ween different subsyst em s, but t his t ype of bat ch scenario is st ill a realit y for m ost com panies. By exam ining t he m any ent erprise I T archit ect ures t hat have grown over decades ( t he fam ous Gart ner I nt egrat ion Spaghet t i com es t o m ind) , we will see t hat t he m aj orit y of ent erprises have no cent ralized, ent erprise- wide workflow syst em s but rat her t hat workflows are deployed m ore im plicit ly. I nst ead of a clean separat ion of applicat ion logic and business rules, you will oft en find t hat t he logic com prising a part icular logical workflow or process is scat t ered across a m ult it ude of syst em s and subsyst em s, buried in new ( e.g., Java- based) and old ( e.g., COBOL- based) applicat ions, t ied t oget her using point - t o- point int egrat ion or m iddleware hubs. Even wit hin a single applicat ion syst em , business logic is likely t o be spread across t he present at ion t ier ( fat client s or present at ion servers such as ASP or JSP) , t he m iddle t ier ( EJBs, Web services) , and t he dat abase t ier ( e.g., st ored procedures) . All t his m akes it very hard t o realize consist ent processes. Regardless of how im plicit or explicit processes are realized in a dist ribut ed syst em ( and how deliberat e t he decision for using or not using a dedicat ed BPM or workflow product ) , process int egrit y is one of t he m ost difficult problem s t o solve.

8.2. Technical Concepts and Solutions You can choose from a wide range of solut ions t o im plem ent process int egrit y. These solut ions range from sim ple t echnical solut ions such as dist ribut ed logging and t racing t o advanced t ransact ion concept s. BPM syst em s provide t he facilit y t o address process int egrit y on a less t echnical and m ore business- orient ed level. BPMs are used t o m odel and execut e business processes, as we discussed in Chapt er 7, " SOA and Business Process Managem ent ." They enable us not only t o explicit ly m odel special cases but also t o provide definit ions for t he appropriat e count erm easures in case of except ions. We look at each of t hese solut ions in t urn, followed by recom m endat ions for t heir applicat ion in an ent erprise SOA.

8.2.1. LOGGING AND T RACING Logging and t racingat different levels of sophist icat ionis probably st ill t he m ost com m only used approach for providing at least rudim ent ary levels of process int egrit y on an ad- hoc basis. Log t races are com m only used for debugging but are also used in product ion syst em s in order t o ident ify and solve problem s in day- t o- day operat ions. Part icularly in t he case of com plex syst em s t hat int egrat e large num bers of non- t ransact ional legacy syst em s, logging is oft en t he only viable approach t o providing a m inim um level of process int egrit y, especially if processes or workflows are im plem ent ed im plicit ly ( i.e., t here is no dedicat ed BPM syst em ) . Oft en, t he operat ors of t hese t ypes of syst em s em ploy adm inist rat ors t hat m anually fix problem s based on t he analysis of log files. I f t he log file provides a com plet e t race of t he st eps execut ed in a part icular process unt il t he failure occurred, t he adm inist rat or has som e chance of fixing t he problem , even if t his oft en requires going direct ly t o t he dat abase level t o undo previous updat es or fix som e problem t o enable t he process t o cont inue. A key problem wit h logging- based problem analysis and repair is t he lack of correlat ion bet ween t he different log ent ries t hat relat e t o a part icular logical process inst ance, especially for processes t hat are im plem ent ed im plicit ly. I f a process fails due t o a t echnical fault , som eone m ust ident ify what has happened so far in order t o com plet e or undo t he process. To do t his, you m ust find t he log ent ries t hat relat e t o t he process, possibly across different log files from different syst em s. I deally, som e kind of correlat ion I D ( relat ed t o process inst ances) should exist for each log ent ry because t his helps wit h t he log consolidat ion ( as depict ed in Figure 8- 1 ) . However, oft en t he only way t o correlat e event s is by com paring t im est am ps, which is a difficult t ask, especially wit h syst em s t hat use dist ribut ed log files. For exam ple, how is it possible t o relat e a JDBC except ion writ t en int o a local log by an EJB applicat ion server t o a dat abase deadlock event t hat was writ t en t o a dat abase log ( bot h log ent ries are pot ent ially relevant for ident ifying and fixing t he problem in t he applicat ion) ? I n t his case, a significant chance exist s t hat bot h except ions occurred at t he sam e t im e. The problem becom es even m ore difficult if processes are long- lived and you are t rying t o find out which cust om er account was m odified earlier by a process t hat failed lat er, such as when updat ing a shipping order.

Figu r e 8 - 1 . Con solida t e d logs ca n h e lp syst e m a dm in ist r a t or s de a l w it h e r r or sit u a t ion s in dist r ibu t e d, lon g- live d pr oce sse s. Log con solida t ion ca n be pe r for m e d a cr oss syst e m s ( e .g., pr ovidin g t h e a bilit y t o r e la t e u pda t e s in on e da t a ba se t o u pda t e s in a n ot h e r ) , a s w e ll a s w it h in

syst e m s ( e .g., r e la t in g da t a ba se logs t o a pplica t ion se r ve r logs) .

Nevert heless, logging and t racing is st ill im port ant in m any operat ional syst em s and is oft en t he only possible way t o achieve at least m inim um process int egrit y. Many proj ect s have t herefore invest ed heavily in building a sophist icat ed infrast ruct ure t hat helps wit h dist ribut ed logging and log analysis. Oft en, t his infrast ruct ure is built on or t ight ly int egrat ed wit h syst em m anagem ent plat form s such as I BM Tivoli or HP Openview. Chapt er 9, "I nfrast ruct ure of a Service Bus," provides a det ailed overview of how t his can be achieved in an SOA environm ent . Not ice t hat dist ribut ed log consolidat ion can only part ly address t echnical failures and does not address process inconsist encies t hat relat e t o business- level problem s at all.

8.2.2. ACID TRANSACTIONS Online Transact ion Processing ( OLTP) has been a key elem ent of com m ercial com put ing for several decades. OLTP syst em s enable large num ber of users t o m anipulat e shared dat a concurrent ly. For exam ple, in an online flight reservat ion syst em , sales agent s around t he world share access t o flight booking inform at ion.

I n order t o support shared m anipulat ion of dat a, OLTP syst em s are based on t he concept of t ransact ions. Tradit ionally, t he t erm t ransact ion has been used t o describe a unit of work in an OLTP syst em ( or dat abase) t hat t ransform s dat a from one st at e t o anot her, such as booking a seat on a part icular flight . The t erm ACI D ( at om icit y, consist ency, isolat ion, durabilit y) has been coined t o describe t he ideal charact erist ics of concurrent ly execut ed and possibly dist ribut ed t ransact ions, which ensure t he highest level of dat a int egrit y. I t is described in I SO / I EC 100261: 1992 sect ion 4. We will use t he sim plified exam ple of a m oney t ransfer wit h a debit and a credit updat e t o illust rat e t he propert ies of ACI D t ransact ions: At om icit y: ACI D t ransact ions are at om ic " all or not hing" unit s of work. I f any one part of a t ransact ion fails, t he ent ire t ransact ion is rolled back. I f t he debit updat e works but t he credit updat e fails, t he original debit updat e m ust be rolled back. Con sist e n cy: ACI D t ransact ions t ransform dat a from one consist ent st at e t o anot her. I n our exam ple, t he sum of all account balances m ust be t he sam e as before t he t ransact ion. Of part icular im port ance is t he st ipulat ion t hat a t ransact ion ensures referent ial int egrit y: if a cust om er account is delet ed, orphan address records originally relat ed t o t he cust om er m ust also be rem oved ( see t he discussion on dat a int egrit y in t he previous sect ion) . I sola t ion : The int ernal st at e of a running t ransact ion is never visible t o any ot her t ransact ion. This is usually achieved t hrough locking. I n our exam ple, t here is a window of t im e bet ween t he debit and t he credit , during which t he sum of all account s will not add up. However, nobody out side of t he t ransact ion can see t his inconsist ency. D u r a bilit y: Com m it t ed updat es of a t ransact ion are perm anent . This ensures t hat t he consist ent , up- t o- dat e st at e of t he syst em can be recovered aft er a syst em failure during t he execut ion of t he t ransact ion. I f we have a crash aft er t he debit but before t he credit , we can st ill recover t he st at e before t he st art of t he t ransact ion. Alm ost all com m ercial DBMS product s support t he concept of t ransact ions enabling concurrent access t o a dat abase, albeit wit h varying degrees of " ACI Dit y." The DBMS m ust provide an appropriat e concurrency cont rol m echanism in order t o deal wit h t he concurrent execut ion of t ransact ions. The t ransact ion perform ance and behavior in case of access conflict s depends st rongly on t he choice of opt im ist ic versus pessim ist ic concurrency cont rol st rat egies, t he choice of locking ( exclusive or shared locks) versus t im est am ps versus versioning, and t he locking/ versioning granularit y ( t ables, pages, t uples, obj ect s) . Most DBMSs can be program m ed or configured t o use different concurrency cont rol policies ( isolat ion level) in order t o enable different t ypes of applicat ions or t ransact ions t o chose t he appropriat e m ix of perform ance versus t ransact ional int egrit y.

8.2.3. TRANSACTION MONITORS AND DISTRIBUTED 2PC Building applicat ions wit h ACI D propert ies becom es m ore difficult if you operat e out side t he dom ain of a single syst em . Transact ion Processing Monit ors ( TPMs) can be used t o ensure t he ACI D propert ies of a t ransact ion t hat spans m ult iple dat abases or ot her t ransact ional resources ( as depict ed in Figure 8- 2 ) . A resource m anager t ypically m anages t hese resources, which include DBMS and t ransact ional queue m anagers.

Figu r e 8 - 2 . Ex a m ple for 2 PC: A clie n t be gin s a n e w t r a n sa ct ion ( 1 ) , e x e cu t e s t w o u pda t e s ( 2 & 3 ) , a n d a t t e m pt s t o com m it t h e ch a n ge s ( 4 ) . Th e t r a n sa ct ion coor din a t or se n ds pr e pa r e r e qu e st s t o bot h

r e sou r ce m a n a ge r s ( 5 ) . Th e fin a l st e p of t h e t r a n sa ct ion ( 6 ) e it h e r com m it s or a bor t s a ll u pda t e s. I f bot h r e sou r ce m a n a ge r s a gr e e t o com m it t h e t r a n sa ct ion , t h e t r a n sa ct ion coor din a t or se n ds a com m it t o a ll pa r t icipa n t s. I f on e pa r t icipa n t w a n t s t o a bor t , t h e t r a n sa ct ion coor din a t or a sk s a ll pa r t icipa n t s t o a bor t .

Most com m only, t he so- called Two- Phase Com m it Prot ocol ( 2PC) is used t o ensure ACI D propert ies for t ransact ions t hat span m ore t han a single resource m anager. Transact ions are coordinat ed am ong different resource m anagers t hrough a t ransact ion coordinat or , which is part of t he t ransact ion m onit or. At t he end of each dist ribut ed t ransact ion, t he t ransact ion coordinat or will coordinat e t he com m it m ent of t he t ransact ion across t he part icipat ing resource m anagers: I n t he first phase ( prepare) , all part icipat ing resource m anagers m ust ensure t hat all relevant locks have been acquired and t hat t he " before" and " aft er" st at e of dat a t hat has been m odified in t he cont ext of t he t ransact ion has been persist ent ly capt ured. Depending on t he out com e of t he first phase ( " vot ing" ) , t he t ransact ion coordinat or inform s all part icipat ing resource m anagers whet her t o com m it or rollback t he changes. A single " abort " vot e will cause t he ent ire t ransact ion t o be rolled back, helping t o ensure t he at om icit y propert y of t he t ransact ion. Only if all part icipant s vot e t o " com m it " will t hey also be asked t o m ake t he changes perm anent and visible. The 2PC prot ocol assum es a very t ight coupling bet ween all t he com ponent s involved in t he execut ion of t he t ransact ion. All part icipant s m ust " play by t he rules" in order t o ensure t hat t he out com e of each t ransact ion is consist ent . I f a part icipant does not abide by t he rules, t he result will be a so- called " heurist ic" out com e, t hat is, a t ransact ion whose final st at e is undefined. Furt herm ore, deadlocks becom e an issue t hat m ust be handled wit h great care. The m ost im port ant st andard in t he area of t ransact ion m onit ors and 2PC is t he X/ Open st andard for Dist ribut ed Transact ion Processing ( X/ Open DTP) . Am ong ot her prot ocols and API s, t he X/ Open st andard defines t he so- called XA int erface, which t ransact ion m onit or use t o int eract wit h a resource m anager, for exam ple t o execut e t he " prepare" and " com m it " calls. Most com m ercial RDBMS and queue m anagers provide support for t he XA int erface, enabling t hem t o part icipat e in dist ribut ed t ransact ions. Well- est ablished t ransact ion m onit ors include CI CS, I MS, Encina ( I BM) , and Tuxedo ( BEA) . I n t he Java world, JTS ( Java Transact ion Service) is widely est ablished, and it s count erpart in t he Microsoft world is MTS ( Microsoft Transact ion Server) .

8.2.4. PROBLEMS WITH 2PC AND TIGHTLY COUPLED ACID TRANSACTIONS Alt hough ACI D t ransact ions are a good t heoret ical concept for ensuring t he int egrit y of a single dat abase or even a dist ribut ed syst em , in m any cases, t hey are im pract ical in real- world applicat ions. We will now cover t he key lim it at ions of t ight ly coupled ACI D t ransact ions in m ore det ail.

8.2.4.1 Performance Even in a non- dist ribut ed syst em , ensuring t he isolat ion propert y of a concurrent ly execut ed t ransact ion can be difficult . The problem is t hat t he higher t he isolat ion level, t he poorer t he perform ance. Therefore, m ost com m ercial DBMSs offer different isolat ion levels, such as cursor st abilit y, repeat able read, read st abilit y, and uncom m it t ed read. I n dist ribut ed syst em s, t he execut ion of a dist ribut ed 2PC can have an even m ore negat ive im pact on t he perform ance of t he syst em , in som e cases due t o t he overhead of t he required out - of- band coordinat ion going on behind t he scenes. Finding t he right t radeoff bet ween perform ance and a high degree of concurrency, consist ency, and robust ness is a delicat e process.

8.2.4.2 Lack of Support for Long-Lived Transactions Anot her big problem wit h syst em s based on t ransact ion m onit ors and ACI D propert ies is t hat m ost dat abase m anagem ent syst em s are designed for t he execut ion of short - lived t ransact ions, while m any real- world processes t end t o be long- lived, part icularly if t hey involve int eract ions wit h end users. Most OLTP syst em s ( and t ransact ion m onit ors and underlying dat abases) use pessim ist ic locking t o ensure isolat ion for concurrent t ransact ions. I f a lock is held for long, ot her users cannot access t he resource in quest ion during t hat t im e. Because som e RDBMS- based applicat ions st ill prefer page- level locking t o row- level locking, a lock can block resources t hat reside wit hin t he sam e page but t hat are not direct ly involved in t he t ransact ion. The problem is m ade worse if t he lock is held for a long t im e. A t ypical solut ion t o t he problem wit h pessim ist ic locking is t he applicat ion of an opt im ist ic, t im est am p- based approach. However, t his m ust oft en be perform ed at t he applicat ion level ( e.g., by adding a t im est am p colum n t o t he relevant t ables) due t o a lack of out - of- t he- box support from m any DBMSs for t im est am p- based versioning. I f t he applicat ion is responsible for ensuring consist ency, we now have a problem wit h our t ransact ion m onit or: t he t ransact ion m onit or assum es t hat all t hese issues are dealt wit h direct ly bet ween t he t ransact ion m onit or and t he DBMS during t he 2PC. The XA- prepare and - com m it calls im plem ent ed by t he resource m anager are expect ed t o m anage t he t ransit ion from t he " before" t o t he " aft er" st at e and locking/ unlocking of t he involved resources. Effect ively, t his m eans t hat we m ust include cust om ized logic int o t he 2PC t hat deals wit h t hese issues at t he applicat ion level by checking for t im est am p conflict s before applying changes. Even if som e t ransact ion m onit ors support applicat ion- level locking t hrough callbacks, which can be insert ed int o t he t wo- phase com m it , t his approach severely lim it s an " off- t he- shelf" approach because work t hat should originally have been split bet ween t he t ransact ion m onit or and t he DBMS m ust now be perform ed at t he applicat ion level ( at least part ially) .

8.2.4.3 Problems with the Integration of Legacy Systems and Packaged Applications

Perhaps t he biggest problem wit h t ransact ion m onit ors and 2PC is t he lack of support for t wophase com m it using an XA int erface from legacy syst em s and packaged applicat ions, such as an ERP or CRM syst em . Even if an ERP such as SAP uses an XA- capable dat abase such as Oracle or DB2 int ernally, t his does not m ean t hat SAP can part icipat e in a 2PC: all access t o t he SAP m odules m ust go t hrough an SAP API , such as BAPI , which is a non- t ransact ional API . This lack of support for 2PC in m any legacy syst em s and packaged applicat ions severely lim it s t he applicat ion scope of dist ribut ed t ransact ions and TP m onit ors because we norm ally need t o int egrat e applicat ions inst ead of dat abases int o com plex workflows.

8.2.4.4 Organizational Challenges I f used at all, t he adopt ion of 2PC has t radit ionally been lim it ed t o t ight ly coupled, well- cont rolled int ra- ent erprise environm ent s or perhaps single applicat ion syst em s wit h short - lived t ransact ions. Transact ion coordinat ors are t he cont rol cent er for t he orchest rat ion of t he 2PC am ongst resource m anagers. Not only is t ight coupling required at t he prot ocol level, but also successful orchest rat ion am ongst part icipant s requires t hat everybody " plays by t he rules" in order t o avoid frequent heurist ic out com es t hat leave t ransact ions in an ill- defined st at e. Such t ight cont rol over dat abases, applicat ions, t ransact ion fram eworks, and net work availabilit y is usually lim it ed t o int ra- ent erprise environm ent s, if it can be achieved at all. Conduct ing int er- organizat ional t ransact ions bet ween aut onom ous business part ners is a com plet ely different sit uat ion. I n an int er- organizat ional environm ent , we cannot assum e t ot al cont rol over all aspect s of t ransact ion processing. Relat ionships bet ween t rading part ners are t ypically loosely coupled, and t he nat ure of int ra- business t ransact ions reflect s t his loose coupling. The t echnical im plem ent at ions of t hese t ransact ions m ust deal wit h t his loose coupling and also wit h t he fact t hat t he level of t rust bet ween t he t ransact ion part icipant s is different from t hat of a t ight ly coupled, well- cont rolled environm ent . For exam ple, securit y and invent ory cont rol issues prevent hard locking of local dat abases: I m agine a denial of service at t ack from a rogue part ner t hat result s in a sit uat ion where locks are t aken out on all available it em s in your invent ory dat abase, prevent ing you from doing business while t he locks rem ain in place. [ 1] [1]

Even with optimistic concurrency control, this would be a risk because a participant could block access during the normally short time period of the transaction resolution (prepare/commit), during which all participants must lock the updated data, even when applying optimistic concurrency control policies for transaction execution.

8.2.4.5 2PC Is Not Suited for Discontinuous Networks When int egrat ing B2B syst em s over t he I nt ernet , t he discont inuous nat ure of t he I nt ernet wit h it s lack of QoS ( qualit y of service) propert ies m ust be t aken int o account . This applies t o t he execut ion of t he act ual business logic of a t ransact ion across t he I nt ernet in addit ion t o any out of- band int eract ions bet ween t ransact ion part icipant s t hat are part of t he t ransact ion coordinat ion because t hey are also execut ed over t he I nt ernet . Relying on a pot ent ially discont inuous m edium such as t he I nt ernet for execut ion of t he t wo- phase com m it could lead t o sit uat ions t hat increase t he t im e bet ween t he prepare and com m it calls in an unaccept able way, wit h t he pot ent ial for m any heurist ic out com es.

8.2.5. NESTED AND MULTILEVEL TRANSACTIONS The com plexit y of m any business t ransact ions and t he fact t hat business t ransact ions are pot ent ially long running has led t o t he developm ent of advanced t ransact ion concept s such as m ult ilevel and nest ed t ransact ions.

A m ult ilevel t ransact ion T is represent ed by a set of sub- t ransact ions T = { t 1 , t 2 , t 3 , ..., t n } . A sub- t ransact ion t i in T can abort wit hout forcing t he ent ire t ransact ion T t o abort . T can, for exam ple, choose t o rerun t i , or it can at t em pt t o find an alt ernat ive m eans of com plet ion. I f t i com m it s, t he changes should only be visible t o t he t op- level t ransact ion T. I f T is abort ed, t hen so is t i . Mult ilevel and nest ed t ransact ions are slight ly different in t he way in which t hey deal wit h releasing locks on t he com plet ion of a sub- t ransact ion. Alt hough som e com m ercial t ransact ion m onit or im plem ent at ions support t he concept of nest ed or m ult ilevel t ransact ions, t he problem wit h t heir adopt ion lies in t he lack of support from resource m anagers ( dat abase and queue m anagers) . Very few com m ercial resource m anagers provide sufficient support for nest ed t ransact ions. [ 2] Unfort unat ely, t his lack of support from t he m aj or com m ercial resource m anagers m akes t hese good t heoret ical concept s som ewhat unusable in t he real world. [2]

More commonly, RDBMS offer support for Savepoints or similar concepts, which define a point in time to which a partial rollback can occur.

8.2.6. PERSISTENT QUEUES AND TRANSACTIONAL STEPS Persist ent queues can increase t he reliabilit y of com plex processes. They can even creat e t ransact ional st eps in a m ore com plex process: when com bined wit h t ransact ions, persist ent queues can guarant ee consist ency for t he individual st eps of a process or workflow. An applicat ion can de- queue m essages in t he cont ext of a t ransact ion. I f t he t ransact ion abort s, t he de- queue is undone, and t he m essage ret urned t o t he queue. People usually use abort count lim it s and error queues t o deal wit h m essages t hat repeat edly lead t o an abort ed t ransact ion. Figure 8- 3 shows an exam ple of a t ransact ional process st ep wit h persist ent queues. Not ice t hat if t he queue m anager is not part of t he dat abase syst em ( i.e., is an independent resource m anager) , t he t ransact ion effect ively becom es a dist ribut ed t ransact ion and requires a t ransact ion m onit or t hat can coordinat e t he t wo- phase com m it bet ween t he dat abase and queue m anagers.

Figu r e 8 - 3 . A t r a n sa ct ion a l pr oce ss st e p w it h pe r sist e n t qu e u e s.

Leverage the Concept of Transactional Steps A t ransact ional st ep is a set of closely relat ed act ivit ies, execut ed in t he cont ext of a single t ransact ion. At t he beginning of t he t ransact ion, an input t oken or m essage is read from an input queue. The result of t he relat ed act ivit ies is writ t en int o an out put queue. Transact ional st eps are a key concept for ensuring process int egrit y because t hey facilit at e t he decom posit ion of com plex, long- lived business processes int o individual st eps wit h short execut ion t im es and high t ransact ional int egrit y. Transact ional st eps dram at ically increase t he robust ness and flexibilit y of dist ribut ed syst em s.

8.2.7. T RANSACTION CHAINS AND COMPENSATION The concept of t ransact ional st eps provides a great way for ensuring t he int egrit y of individual process st eps. You can creat e com plex workflows or processes by chaining or linking t oget her individual st eps ( " t ransact ion chains" ) . However, t he rem aining issue is t o ensure t he int egrit y of t he overall process or workflow. Alt hough abort count lim it s and error queues provide a good m eans of ensuring no loss of inform at ion upon failure of an individual st ep, it is st ill necessary t o fix t hese failures aft er det ect ing t hem . One possible way t o deal wit h failures in individual process st eps is t o ident ify com pensat ing t ransact ions t hat logically undo a previous t ransact ion. For exam ple, t he com pensat ion for a debit t ransact ion would be a t ransact ion t hat credit s t he sam e am ount . An im plem ent at ion of a dist ribut ed m oney t ransfer bet ween t wo banks could be split int o different st eps t hat debit an account A at bank X and pass a m essage t o t he receiving bank Y using a t ransact ional queue. I f a problem occurs at bank Y ( e.g., t he account does not exist ) , bank X would have t o execut e a com pensat ing t ransact ion, such as credit ing t he appropriat e account A ( adm it t edly, t his is a sim plified exam ple, ignoring t he com plexit y of int erm ediary clearinghouses, end- of- day set t lem ent s, et c.) . Not ice t hat , unlike nest ed or m ult ilevel t ransact ions, chained t ransact ions effect ively relax t he isolat ion propert ies of ACI D t ransact ions because t he result s of each link in a chain of t ransact ions is m ade visible t o t he out side world. For inst ance, assum e t hat we have credit ed an account A in t he first st ep of a t ransact ion chain. When at t em pt ing t o execut e t he corresponding debit on anot her account B in t he next st ep, we discover t hat t he t arget account B does not exist . We could now launch a com pensat ing t ransact ion t o debit account A and undo t he previous changes. However, a t im e int erval now exist s bet ween t he credit and subsequent debit of account A, during which t he funds result ing from t he credit t ransfer could have been wit hdrawn by anot her t ransact ion, leading t o t he failure of t he com pensat ing t ransact ion because t he funds were no longer available. [ 3] [3]

Of course, in this example, the problem could be easily solved by reversing the order of the debit and the credit (starting with the debit first). However, the example still illustrates the problem.

I n order t o apply com pensat ing t ransact ions in a workflow, we need t o log t he input and/ or out put of t he individual st eps of a workflow because we m ight need t his dat a as input for com pensat ing t ransact ions.

Combine Transaction Chains with Compensating Transactions ACI D propert ies are usually t oo st rong for com plex workflows. I nst ead, chained t ransact ions wit h com pensat ing t ransact ions offer a viable m eans of dealing wit h process int egrit y. Transact ion chains com bine individual t ransact ional st eps int o m ore com plex workflows. Com pensat ing t ransact ions undo previously execut ed st eps if a problem is encount ered during t he execut ion of a part icular st ep in a t ransact ion chain.

8.2.8. SAGAS SAGAs are form al workflow m odels t hat build on t he concept of chained t ransact ions ( st eps) . A SAGA describes a workflow wherein each st ep is associat ed wit h a com pensat ing t ransact ion. I f a workflow st ops m aking progress, we can run com pensat ing t ransact ions for all previously com m it t ed st eps in reverse order. Alt hough form al SAGAs are st ill at t he research st age, t he concept of using a com pensat ing t ransact ion for dealing wit h specific failure sit uat ions in a com plex workflow is valid. A num ber of problem s exist wit h SAGAs and com pensat ions in com plex workflows, part icularly wit h t he com plexit y of workflow graphs and t he corresponding com pensat ion graphs. Typically, t he com plexit y of com pensat ion graphs increases exponent ially wit h t he com plexit y of t he act ual workflow graph. Thus, it is generally im possible t o define com plet e com pensat ion graphs. I n addit ion, t he need t o deal wit h failures during t he execut ion of com pensat ing t ransact ion chains adds even m ore com plexit y. Even if form al SAGAs and com pensat ions for each possible com binat ion of failures are unavailable, it oft en m akes sense t o apply t he concept of com pensat ing t ransact ions for individual failure sit uat ions t hat have been specifically ident ified t o fit t his approach, such as failures t hat we expect t o happen frequent ly, perhaps because t hey are caused by business condit ions such as " out of funds."

8.2.9. BPM AND PROCESS INTEGRITY As int roduced in Chapt er 7, Business Process Managem ent plat form s provide feat ures t o enable business analyst s and developers t o design, execut e, and m anage inst ances of com plex business processes. Many BPM plat form s are in fact based on or at least incorporat e som e of t he feat ures described previously, such as chains of t ransact ional st eps. I n t he following discussion, we exam ine how t he form al int roduct ion of a BPM will help t o increase t he process int egrit y aspect s of an ent erprise applicat ion. First ly, explicit ly m odeling workflows and processes and separat ing t hem from low- level t echnical code will have a posit ive im pact on what process int egrit y act ually m eans wit h respect t o a part icular process because t he BPM approach provides a com prehensible separat ion bet ween " t echnical int egrit y" and " process int egrit y." Secondly, if t he BPM engine provides a m echanism for m onit oring and m anaging process inst ances at runt im e, we receive a powerful m echanism for cont rolling t he int egrit y aspect s of individual process inst ances at no ext ra cost .

Finally, som e BPM engines support t he concept of com pensat ing t ransact ions, which is a key concept for m anaging t he rollback of part ially execut ed processes aft er a failure sit uat ion. A key quest ion is whet her com pensat ing t ransact ions are sufficient t o ensure process int egrit y, given t hat t he com pensat ion- based m odel is less st rict t han, for exam ple, t he ACI D propert ies of a dist ribut ed t ransact ion. I n part icular, it is useful t o exam ine how t o handle failures of com pensat ing t ransact ions. Do we act ually int roduce com pensat ions for failed com pensat ions? Alt hough in som e rare cases t his m ight be required ( som e t ransact ion processing syst em s in financial inst it ut ions have sophist icat ed m et a- error handling facilit ies t hat cope wit h failures in failure handling layers) , t his will not be t he case in m ost syst em s. Thus, t he support for com pensat ing t ransact ions as offered by som e BPMs oft en present s a sound and flexible alt ernat ive t o plat form s t hat require very t ight coupling wit h applicat ions and resource m anagers, such as t ransact ion m onit ors.

8.2.10. RELATED WEB SERVICE STANDARDS This book t akes on t he posit ion t hat Web services are only one possible t echnical plat form for SOAs ( see Chapt er 9, "I nfrast ruct ure of a Service Bus" ) . However, we want t o t ake a quick look at st andards t hat relat e t o Web services and process int egrit y, because t his subj ect is likely t o becom e very im port ant in t he fut ure, as Web services becom e m ore pervasive. Unfort unat ely, t he area of st andards for Web service- based business t ransact ions ( see Figure 8- 4 ) .

Figu r e 8 - 4 . A n u m be r of diffe r e n t in du st r y a llia n ce s a n d st a n da r diza t ion bodie s a r e cu r r e n t ly w or k in g on diffe r e n t W e b se r vice s- ba se d t r a n sa ct ion pr ot ocols. H ow e ve r , m u ch of t h is w or k is st ill in t h e e a r ly st a ge s, a n d n o dom in a n t st a n da r d h a s e m e r ge d ye t .

I t is im port ant t o realize t hat m ost of t his is st ill at an early st age. Sim ply defining a new t ransact ion st andard is usually insufficient for solving process int egrit y problem s. Even if a st andard em erges t hat is support ed by a num ber of com m ercial t ransact ion m anagers or coordinat ion engines, it is insufficient : The real problem is not t o im plem ent a t ransact ion or coordinat ion engine t hat is com pliant t o a specific prot ocol, but rat her t o find widespread support for such a new coordinat ion prot ocol from resource m anagers. Wit hout t he support of com m ercial dat abases, queue m anagers, and off- t he- shelf ent erprise applicat ion packages, a new t ransact ion st andard is not wort h m uch t o m ost people. Recall t hat it t ook t he X/ Open st andard for Dist ribut ed Transact ion Processing alm ost t en years from it s init ial specificat ion t o it s adopt ion by t he m ost widely used com m ercial dat a- base product s such as Oracle, I BM DB2, and SQL Server. Even t oday, t he support found in t hese product s for t he crit ical XA int erface ( which is t he DB side of t he X/ Open st andard) is oft en weak. Som e of t he m ore recent ( pre- Web services, post X/ Open) t ransact ion st andards, such as t he CORBA Obj ect Transact ion Service ( OTS) and Java Transact ion Service ( JTS) , were designed t o fit int o t he widely adopt ed X/ Open fram ework, which also applies t o som e of t he previous Web services- based t ransact ion st andards. However, as we discussed in t he first part of t his chapt er, X/ Open- based 2PC t ransact ions are not suit ed t o t he long- lived business t ransact ions t hat you are likely t o encount er in m ost Web service cases, which is why t he need exist s for a new t ransact ion or coordinat ion prot ocol. I t rem ains t o be seen which of t he cont enders will event ually becom e t he dom inant st andard for loosely coupled t ransact ion coordinat ion and at what point in t im e com m ercial resource m anagers will support t his st andard. Unt il t hen, we m ust cope wit h ad- hoc solut ions.

8.3. Recommendations for SOA Architects Having discussed t he concept s of dat a and process int egrit y on a m ore concept ual level, it is now t im e t o exam ine concret e recom m endat ions for SOA archit ect s. I n order t o m ake t his discussion as hands- on as possible, we will first int roduce an ext ension t o our airline exam ple, which will serve as a basis for t he discussion t hat follows.

8.3.1. EXAMPLE SCENARIO: TRAVEL ITINERARY MANAGEMENT The m anagem ent t eam of our airline has decided t o expand t he current online offering by providing airline cust om ers wit h t he abilit y t o not only book individual flight s but also t o creat e com plet e it ineraries for t heir t rips, including m ult iple flight , hot el, and car reservat ions. The new it inerary m anagem ent syst em will reuse and build upon several of t he airline's exist ing I T syst em s, in part icular t he cust om er m anagem ent and billing syst em s. I n addit ion, t he cust om er dat abase will be expanded t o support t he m anagem ent of com plex it ineraries. Managem ent decided t hat t wo different front ends are required for t he new syst em : a Web- based online front end and a call cent er for t elephone support of cust om ers. A furt her decision was t o use different t echnologies in each of t he t wo front ends: t he Web- based front end will use t hin HTML client s, whereas t he front end for t he call cent er agent s needs m ore com plex funct ionalit y and will t hus be im plem ent ed as a VB GUI ( fat client ) . The high- level archit ect ure of t he syst em is based on t hree backend services: cust om er m anagem ent ( including it ineraries) , billing, and com plex processes ( in order t o m anage cust om er com plaint s regarding it ineraries or invoices) . I n addit ion, a num ber of part ner syst em s will have t o be int egrat ed, including part ner airlines' flight reservat ion syst em s and hot el and car reservat ion syst em s ( alt hough t his is not a key aspect of t his discussion) . Figure 8- 5 provides an overview of t he syst em archit ect ure.

Figu r e 8 - 5 . Th e n e w t r a ve l it in e r a r y m a n a ge m e n t syst e m pr ovide s se r vice s t o m a n a ge cu st om e r s, it in e r a r ie s, in voicin g, a n d com ple x incide nt s. [View full size image]

We will look at t wo key t ransact ions t hroughout t he rest of t his chapt er: confirm it inerary and creat e invoice. Confirm it inerary is an im port ant t ransact ion of t he cust om er m anagem ent syst em , which is responsible for confirm ing t he individual flight , hot el, and car reservat ions on an it inerary, involving pot ent ially com plex int eract ions wit h part ner syst em s. This t ransact ion is pot ent ially irreversible, in t hat t he syst em m ight require a cancellat ion fee when at t em pt ing t o cancel a previously confirm ed booking ( assum ing t hat a cancellat ion is possible) . Creat e invoice is a t ransact ion of t he billing syst em . Assum ing t hat a cust om er has proven credit wort hiness, t he syst em creat es an invoice, calculat es t he t ot al am ount and t axes for each individual it em on t he it inerary, and sends a let t er wit h a print ed version of t he invoice t o t he cust om er by m ail. These t wo t ransact ions are int erest ing for several reasons. First , t hey are closely relat ed at t he business level because t he confirm at ion of an invoice inevit ably causes cost s and t herefore m ust be accom panied by a valid invoice under all circum st ances. Second, t hese t ransact ions cross several organizat ional boundaries because t hey use t wo services provided by different depart m ent s ( cust om er m anagem ent is a m arket ing funct ion, and invoicing is a back- office funct ion) and even involve several different com panies ( ot her airlines, hot els, and car rent al com panies) . Finally, t hese t ransact ions also cross several t echnical boundaries. Not ice in part icular t hat in our airline exam ple, t hese t wo t ransact ions are relat ed t o t wo independent dat abases.

8.3.2. OPTIMISTIC CONCURRENCY CONTROL SHOULD BE THE DEFAULT

To begin wit h, it is necessary t o explain how t o deal wit h concurrent access t o shared dat a in an SOA. Two widely est ablished m odels for m anaging concurrent access t o shared dat a by m ult iple users exist : opt im ist ic and pessim ist ic concurrency cont rol. Bot h com e in m ult iple flavors, but t he general charact erist ics of each m odel can be sum m arized as follows: Pe ssim ist ic con cu r r e n cy con t r ol. This m odel gives users exclusive access right s t o a set of dat a t hat t hey int end t o m odify, usually t hrough t he acquisit ion of a lock, which is associat ed wit h t he dat a in quest ion. Ot her users are locked out t hey cannot perform act ions t hat would conflict wit h t he lock unt il t he lock owner releases it . This m odel is predom inant ly used in sit uat ions where heavy cont ent ion for dat a exist s. The t wo key problem s wit h t his m odel are lockout s and deadlocks. For t hese reasons, pessim ist ic concurrency cont rol assum es t hat lock t im es are short , which is norm ally only t he case in aut om at ic processing of dat a records. Opt im ist ic con cu r r e n cy con t r ol. I n t his m odel, no locks are acquired during t he t ransact ion execut ion. Opt im ist ic concurrency cont rol perm it s different t ransact ions t o read t he sam e st at e concurrent ly and checks for pot ent ial writ e conflict s and dat a inconsist encies only at t he end of t he t ransact ion, when changes are act ually writ t en t o t he dat abase. Because t his im plies t hat t he danger of losing one's changes at t he end of t he t ransact ion exist s, t his m odel is m ost effect ive in environm ent s wit h low cont ent ion for dat a. Each approach has it s own advant ages and disadvant ages, depending on t he specific problem cont ext . As we will show, opt im ist ic concurrency cont rol is t he preferable m odel in an SOA.

Apply Optimistic Concurrency Control Opt im ist ic concurrency cont rol is t he m odel of choice for long- running t ransact ions, in part icular t hose requiring int eract ions wit h hum an users or ot her sub- syst em s. I n addit ion, opt im ist ic concurrency cont rol support s a m ore loosely coupled approach because resources are less dependent on client s, for exam ple wit h respect t o lock durat ion. For t hese reasons, opt im ist ic concurrency cont rol is t he m odel of choice in an SOA because it significant ly reduces dependencies bet ween different service com ponent s. Of course, t his is assum ing t hat you are not lim it ed by exist ing concurrency cont rol policies, as would be t he case if a legacy syst em had t o be incorporat ed int o an SOA. I n t hat case, a flexible way t o incorporat e t he exist ing policies int o t he overall concurrency policy of t he SOA is required, and you m ight have t o look at int roducing int erm ediary services for bridging incom pat ible concurrency cont rol policies.

8.3.2.1 Implementing Optimistic Concurrency Control Given t he im port ance of opt im ist ic concurrency cont rol in an SOA, it is useful t o exam ine specific im plem ent at ion m odels. I n order t o det erm ine writ e conflict s, t he opt im ist ic concurrency m odel m ust , at t he end of each t ransact ion, det erm ine whet her a com pet ing t ransact ion has changed t he dat a aft er it was init ially checked out . There are different ways t o achieve t his: using t im est am ps, version count s, or st at e com parison ( direct ly or using check sum s) . When using t im est am ps or version count s wit h relat ional dat abases, t he m ost popular approach is t o add a colum n for t he t im est am p or version num ber t o each t able t hat m ust be cont rolled

( alt ernat ively, you can add only a colum n t o t he t op- level dat a st ruct ure) . I n order t o det erm ine writ e conflict s, you m ust add t he t im est am p or version num ber t o t he prim ary key used in t he WHERE clause of t he UPDATE st at em ent . This m inim izes t he int eract ions required wit h t he DBMS. I f our t im est am p or version num ber does not m at ch t he num ber correlat ing t o t he prim ary key, t his m eans t hat som ebody has changed t he dat a aft er our init ial read. I n t his case, no rows will be updat ed, and t he user m ust reread t he dat a and reapply t he changes. The benefit of st at e com parison is t hat you don't need t o alt er t he st ruct ure of t he dat abase. Microsoft .NET provides an elegant im plem ent at ion of t hese concept s in it s ADO.NET fram ework. ADO Dat aSet s cont ain com plex dat a obj ect s t hat you can easily read and writ e t o and from dat abases, t ransform int o XML, expose t hrough Web services, and version t hrough ADO DiffGram s. However, in environm ent s t hat do not provide direct support for st at e com parison, t he updat e logic is m ore com plex, and we m ust m aint ain t wo versions of dat a t hroughout t he t ransact ion, which com plicat es t he dat a st ruct ure used in our applicat ion. I n t hese cases, t im est am p or version num ber- based concurrency cont rol is preferable. When choosing t im est am p or version num ber- based concurrency cont rol, it is necessary t o design service int erfaces accordingly. This is best achieved by ext ending t he root level elem ent s of t he dat a st ruct ures used in t he service definit ions t o include t he t im est am p or version num ber. Not ice t hat t hese dat a st ruct ures are t ypically fairly coarse- grained or even docum ent - orient ed. The different elem ent s in such dat a st ruct ures are usually st ored in different t ables in a dat abase. The assem bly and disassem bly of t hese com plex dat a st ruct ures int o different rows in a dat abase is hidden from t he user. I n an SOA, we can deal wit h t hese dat a st ruct ures in an elegant m anner t hat does not require operat ing at t he dat abase level. Specifically, it is possible t o em ploy opt im ist ic concurrency cont rol wit h a level of granularit y t hat best fit s individual services. Concept ually, you can view t his as a check- in/ check- out m echanism for relat ed dat a. I t is only necessary t o focus on version cont rol at t he level of root elem ent s of t hese coarse- grained dat a obj ect s, not at t he level of individual rows. Not ice t hat when em bedding version inform at ion in dat a st ruct ures t hat are passed bet ween services in an SOA, you assum e t hat you can t rust client s not t o at t em pt t o m odify t he version inform at ion. I f you cannot t rust your client s, it is necessary t o revert t o t he st at e com parison approach.

8.3.2.2 Use of Optimistic Concurrency Control in the Example A good exam ple of an ent it y t hat is accessed concurrent ly is t he cust om er profile. Assum e t hat a cust om er is exam ining his profile online while sim ult aneously t alking t o a call cent er agent by t elephone, discussing a quest ion relat ed t o his profile. While wait ing for t he agent 's answer, t he cust om er changes t he m eal preference t o veget arian. At t he sam e t im e, t he agent updat es t he cust om er's address det ails as inst ruct ed by t he cust om er during t heir t elephone conversat ion. Bot h read t he profile at t he sam e t im e, but t he cust om er subm it s his changes j ust before t he agent hit s t he Save but t on. Assum e t hat t he cust om er profile is prot ect ed by an opt im ist ic concurrency cont rol m echanism , based on t im est am ps or version num bers, for exam ple. The agent now loses his changes because t he syst em det ect s a writ e/ writ e conflict based on t he version num ber or t im est am p of t he agent 's copy of t he cust om er profile and t hus refuses t o apply t he agent 's changes. The agent m ust now reread t he profile before reapplying t he changesan annoying sit uat ion, from t he agent 's point of view. So far, we have m ade t he im plicit assum pt ion t hat t he cust om er profile is prot ect ed by an opt im ist ic concurrency cont rol m echanism . Does t his st ill m ake sense in t he light of t he previous conflict sit uat ion, which result ed in t he loss of t he agent 's changes t o t he cust om er profile? That depends on t he concret e usage pat t erns of t he syst em . Recall t hat opt im ist ic concurrency cont rol is predom inant ly used in sit uat ions wit h low cont ent ion for dat a. I f one assum es t hat t he chances of t wo users ( such as cust om er and agent ) accessing t he sam e profile at t he sam e t im e are

ext rem ely low ( which seem s likely in our overarching exam ple) , t he opt im ist ic approach m ight st ill be an accept able solut ion. This decision will t ypically be m ade on a case- by- case basis, looking at different ent it ies in a syst em individually. St ill, it is usually safe t o assum e t hat for all ent it ies in an SOA, opt im ist ic concurrency cont rol is t he default im plem ent at ion st rat egy, unless we know from t he out set t hat we are looking at an ent it y wit h very high cont ent ion. However, it is usually hard t o predict realist ic cont ent ion levels, and t herefore t he opt im ist ic approach should be used as t he default t o st art wit h. Over t im e, you will learn m ore about t he crit ical ent it ies of t he deployed syst em , and you will react accordingly, by m igrat ing t he access t o t hese highly cont ent ious ent it ies t o m ore suit able concurrency cont rol st rat egies. This evolut ionary approach helps t o dram at ically reduce im plem ent at ion com plexit y, enabling you t o focus on t hose few ent it ies t hat act ually require a m ore sophist icat ed concurrency cont rol m echanism . Assum e in our cust om er profile exam ple we discover over t im e t hat conflict ing writ e access is m ore com m on t hat init ially ant icipat ed. I n t his case, we can offer a num ber of different solut ions. First , we could im plem ent a " m erge" rout ine, which would enable an agent t o m erge his changes wit h t hose of t he cust om er in t he case of a conflict ( see Figure 8- 6 ) . This would work as long as bot h are updat ing different part s of t he cust om er profile, as in t he previous exam ple, where one changed t he address and t he ot her changed t he m eal preferences. This approach would require a m ove from a version or t im est am p- based approach t o a st at e com parison- based approach.

Figu r e 8 - 6 . N or m a l ve r sion con flict de t e ct ion ca n be com bin e d w it h " m e r ge " r ou t in e s t o e n a ble m or e e ffe ct ive h a n dlin g of ve r sion con flict sit u a t ion s.

Second, we could split t he cust om er profile int o finer- grained ent it ies. For exam ple, inst ead of

having one dat a st ruct ure cont aining all cust om er profile dat a, we could have one for general dat a ( such as nam e and dat e of birt h) , one for address inform at ion, one for cust om er preferences, and so on. This should also help reduce t he pot ent ial for writ e/ writ e conflict s. Finally, pessim ist ic concurrency cont rol enables us t o avoid ent irely conflict sit uat ions t hat result in t he loss of updat es by det erm ining pot ent ial read- for- updat e conflict s when reading t he dat a, far before we apply our changes ( at t he cost of locking out one of t he users wit h writ e int ent ions) . However, not ice t hat such pessim ist ic st rat egies in an SOA usually com e at m uch higher im plem ent at ion cost s and t hus should be kept t o a m inim um . The next sect ion provides an appropriat e exam ple.

8.3.2.3 Use of Pessimistic Concurrency Control in an Example Given t he generally higher im plem ent at ion com plexit y, pessim ist ic concurrency cont rol in an SOA is usually lim it ed t o sit uat ions where a writ e/ writ e conflict wit h result ing loss of changes is ext rem ely crit ical. This sit uat ion can arise, for exam ple, when updat es are perform ed m anually and require a considerable am ount of t im e but a m erger wit h anot her user's updat es is alm ost im possible. Recall t hat we are discussing pessim ist ic concurrency cont rol at t he applicat ion level, not at t he dat abase leveldat abase locking m echanism s are generally not designed for long- lived t ransact ions! This m eans t hat t he locking m echanism t hat is required for a pessim ist ic concurrency cont rol st rat egy m ust be im plem ent ed at t he applicat ion levelt hat is, all services in our SOA t hat m anage or m anipulat e dat a ( which would be m ainly basic and int erm ediary services, according t o our service classificat ion in Chapt er 5, " Services as Building Blocks" ) m ust be im plem ent ed in a way t hat support s t he pessim ist ic concurrency cont rol st rat egy. As a result , we would significant ly increase t he im plem ent at ion com plexit y ( and hence t he cost ) of our SOA. I n addit ion, applicat ion- level lock m anagem ent usually result s in fairly com plex workflows because it requires m uch int eract ion wit h hum an users. For exam ple, we need t o provide a m anagem ent infrast ruct ure t hat deals wit h sit uat ions where a user t akes out a lock on a crit ical ent it y but t hen fails t o release it , for exam ple because t he user is out sick for a couple of days during which t he ent it y rem ains locked. Exam ples of syst em s providing t his kind of sophist icat ed infrast ruct ure include docum ent m anagem ent syst em s and insurance claim s processing syst em s ( oft en com bined wit h t he concept of work basket s, which assign pending claim s t o individual clerks) . I n our it inerary m anagem ent exam ple, we int roduced t he concept of an " incident ," such as a com plaint about an it inerary or an invoice. I n our exam ple, call cent er agent s process t hese incident s. An incident can be a very com plex dat a st ruct ure, including inform at ion about t he cust om er, t he it inerary, t he invoice, a change hist ory, a cont act hist ory ( including copies of em ails and faxes) , a working st at us, and so on. I ncident s are allocat ed on a per- agent basis, m eaning t hat a single agent is responsible for t he resolut ion of an incident . Thus, t he cust om er only has t o deal wit h one person, who has full knowledge of t he incident . I n order t o achieve t his exclusive associat ion bet ween incident s and agent s, we need t o im plem ent a pessim ist ic concurrency cont rol st rat egy based on locks, which prevent s an agent from updat ing an incident owned by anot her agent . The im plem ent at ion of t he basic locking st rat egy is relat ively st raight forward. For exam ple, t he incident t able in t he dat abase can be expanded t o include a " LOCKED BY" colum n. I f a row cont ains no " LOCKED BY" ent ry, it is not locked. Ot herwise, t he " LOCKED BY" field cont ains t he I D of t he agent claim ing t he lock. Not ice t hat all applicat ion m odules accessing t he incident m ust play by t he rules, checking t he " LOCKED BY" field. I n our exam ple, t he incident t able is encapsulat ed by an incident service int erface, which t akes care of ensuring t hat only request s from client s wit h t he right credent ials are allowed t o access a part icular incident . Alt hough all t his sounds relat ively st raight forward, it is st ill considerably m ore com plex t han a

sim ple opt im ist ic concurrency cont rol st rat egy. I n addit ion, we now need a m anagem ent infrast ruct ure t hat enables us t o deal wit h t he allocat ion of locks on incident s. The allocat ion of locks will m ost likely be em bedded in som e kind of higher- level work- allocat ion syst em , based on t he availabilit y of agent s wit h t he right skills, com bined wit h a work load dist ribut ion algorit hm . Each agent m ight own a personal work basket cont aining all incident s allocat ed t o him or her. Furt herm ore, we need a m anagem ent funct ion t hat enables m anagers t o m anually reallocat e incident s t o ot her agent s, in case a cust om er calls wit h an urgent request when t he original agent is unavailable, for exam ple. Figure 8- 7 provides an exam ple design for t he logic t hat would be required.

Figu r e 8 - 7 . Pe ssim ist ic con cu r r e n cy con t r ol w it h de dica t e d lock s. Re a d- for - u pda t e con flict s a r e de t e ct e d e a r lie r , a n d on ly on e u se r ca n a cce ss t h e in cide n t obj e ct a t t h e t im e . [View full size image]

All t his significant ly increases t he cost and com plexit y of t he solut ion. What was init ially a relat ively sim ple concurrency cont rol problem has suddenly grown int o a com plex workflow and t ask m anagem ent syst em .

8.3.3. MAKE UPDATE OPERATIONS IDEMPOTENT Having discussed st rat egies t hat enable us t o handle updat e conflict s in an SOA using concurrency cont rol, t he next requirem ent is t o look at problem s arising from failures during updat e operat ions. These occur because client s rem ot ely invoke updat e operat ions on t he server housing t he service for t he t ransact ion. Handling failures during rem ot e updat e operat ions in a dist ribut ed environm ent is a challenging t ask because it is oft en im possible for a client t o det erm ine whet her t he failure on t he rem ot e server occurred before or aft er t he server execut ed t he dat abase

updat e. Therefore, in t he ideal world, service operat ions t hat change t he dat abase ( updat e t ransact ions) should be idem pot ent t hat is, if t hey are invoked repeat edly, t he corresponding funct ion should only execut e once. For exam ple, suppose a service im plem ent at ion encapsulat es a dat abase updat e. A client invokes t he service, t riggering t he execut ion of t he local t ransact ion. I f t he service im plem ent at ion crashes before it has sent t he reply back t o t he client , t he client can't t ell whet her t he server has com m it t ed t he t ransact ion. Thus, t he client does not know whet her it is safe t o resubm it t he request ( or t o call an appropriat e com pensat ion operat ion) . Figure 8- 8 shows t wo scenarios for a failure during t he rem ot e execut ion of an ItineraryManager::add_booking() operat ion. I n t he first version, t he updat e is successfully execut ed, but t he server fails before ret urning t he reply t o t he client . I n t he second version, t he server fails before act ually com m it t ing t he changes t o t he dat abase, which will event ually lead t o a rollback of t he changes. I n bot h cases, t he client sees only t hat a problem occurred on t he server side; he has no easy way of finding out whet her t he addit ional booking was added t o t he it inerary. This represent s a problem for our client im plem ent at ion because we cannot safely reinvoke t he operat ion wit hout t he risk of adding t he sam e booking t wice ( our im plem ent at ion is not idem pot ent ) .

Figu r e 8 - 8 . Fa t a l fa ilu r e s a t t h e se r ve r side m a k e it h a r d for clie n t s t o de t e ct w h e t h e r a n u pda t e w a s e x e cu t e dt h a t is, w h e t h e r t h e fa t a l fa ilu r e occu r r e d be for e or a ft e r t h e da t a ba se u pda t e w a s e x e cu t e d succe ssfully.

[View full size image]

Not ice t hat t his t ype of problem applies only t o cert ain t ypes of updat e t ransact ions: we need t o different iat e bet ween updat e operat ions wit h " set " sem ant ics on one hand and " creat e/ add/ increm ent " sem ant ics on t he ot her. Typically, it is safe t o reinvoke an operat ion wit h " set " sem ant ics because in t he worst - case scenario, we sim ply override t he first updat e and don't act ually change t he overall out com e. " Creat e/ add/ increm ent " sem ant ics are a bigger problem . Thus, t here are t wo possible solut ions for t hese kinds of problem s: The first approach is t o change t he sem ant ics of an operat ion from " creat e/ add/ increm ent " t o " set ," which usually will not be

possible for " creat e" sem ant ics, but in m any cases will be possible for " add/ increm ent " sem ant ics. The second approach is t o m ake t he non- idem pot ent t ransact ions idem pot ent by slight ly rest ruct uring t he t ransact ion. We can achieve t his by adding unique sequence num bers, as we will now discuss.

8.3.3.1 Use Sequence Numbers to Create Idempotent Update Operations I n t he previous sect ion, we discussed different failure scenarios for t he Itinerary::add_booking() updat e operat ion ( refer t o Figure 8- 8 ) , an operat ion wit h " add/ increm ent " sem ant ics. Changing t he sem ant ics of t his operat ion t o " set " is not easy because it would require reading t he ent ire it inerary, adding t he booking on t he client side, and sending back t he ent ire it inerary t o t he serverresult ing in an undesirably coarse level of granularit y. I nst ead, we can use unique sequence num bers t o m ake t he add_booking() operat ion idem pot ent . Sequence num bers can be passed eit her im plicit ly ( as part of t he m essage header or som e ot her place for st oring request cont ext inform at ion) or explicit ly ( as a norm al request argum ent ) . This is a design choice, and it depends on t he flexibilit y of t he SOA infrast ruct ure in t he ent erprise. The following exam ple shows a possible solut ion for t he add_booking() problem wit h explicit sequence num ber passing:

interface Itinerary { SQN getSequenceNumber(); void add_booking (in SQN s, in Booking b); }

I f perform ance or lat ency is an issue ( we are pot ent ially doubling t he num ber of rem ot e int eract ions, at least for all updat e operat ions) , we can assign sequence num bers in bulkt hat is, a client can ask for a set of sequence num bers in a single call. Assum ing t he num ber space we use is fairly large, t his is not a problem if client s do not use all t he sequence num bers, t hey can sim ply discard t he ones t hey don't need. Sequence num bers should generally be m anaged at t he server side and assigned t o client s upon request . You could t hink of ways in which client s could m anage t heir own sequence num bers, by com bining unique client I Ds wit h sequence num bers for exam ple. However, t his increases t he com plexit y of t he problem , and rat her t han providing a server- side solut ion, it places t he burden on pot ent ially m ult iple client s.

8.3.3.2 Idempotent Operations Simplify Error Handling A key benefit of using only idem pot ent operat ions in an SOA is t hat we can handle errors in a m ore easy and elegant fashion. First ly, we can reinvoke operat ions a num ber of t im es, pot ent ially m inim izing t he num ber of problem s relat ed t o once- off error sit uat ions ( e.g., a server crash due t o a m em ory corrupt ion) . I n addit ion, we can group relat ed rem ot e calls int o a single block for error handling purposes. This block can be execut ed repeat edly, regardless of where t he failure occurred in t he block, because it is safe t o reinvoke previously execut ed idem pot ent operat ions. The following pseudo- code shows an exam ple for t he execut ion of t wo idem pot ent operat ions, confirm_itinerary() and create_invoice(), in a single block:

while (retry limit not reached) { try { itineraryManager.confirm_itinerary(); invoiceManager.create_invoice(); } catch (FatalError e) { // manage retry limit counter } } if (not successful) { // we now have a number of possible error scenarios // which we must address }

Of course, we are not saying t hat you should not cat ch all possible error sit uat ions. I n part icular, a client im plem ent at ion should handle user- defined except ions individually on a per- call basis. However, t his approach m akes sense for handling fat al failures in m ore com plex processes t hat are m anaged t hrough recovery fram eworks based on dist ribut ed log consolidat ion ( see t he previous sect ion) . I n t his case, t he fram ework is responsible for det erm ining, for exam ple, if an it inerary has been confirm ed but no corresponding invoice creat ed. A syst em adm inist rat or could det ect t his problem descript ion and m anually fix t he problem , by using an SQL console, for exam ple.

8.3.4. AVOID DISTRIBUTED 2PC I n m any cases, t he adopt ion of log- based or ot her sim ple solut ion ( as described in Sect ion 8.2.1) is insufficient , such as when t wo operat ions m ust be execut ed t oget her wit h " all or not hing" sem ant ics ( at om icit y) , and a failure of one operat ion would lead t o a process inconsist ency t hat cannot be handled by sim ple recovery rout ines. Exam ine t he confirm_itinerary() and create_invoice() operat ions described previously. Because it is absolut ely crit ical for our airline not t o confirm an it inerary wit hout creat ing a corresponding invoice, t he logging- based m anual recovery fram ework described in t he previous discussion on idem pot ent operat ions m ight not be accept able. The airline m ight fear t hat t he ant icipat ed volum e of problem s could be t oo large for a syst em s adm inist rat or t o resolve m anually in a t im ely m anner, or t he airline sim ply m ight not want t o rely on a syst em s adm inist rat or t o deal wit h problem s t hat are direct ly relat ed t o a pot ent ial loss of incom e. An int uit ive solut ion is t o use a t ransact ion m onit or and dist ribut ed 2PC t o ensure t he at om icit y of our t wo updat e operat ions, confirm_itinerary() and create_invoice(). However, as we have discussed in Sect ion 8.2.4, t here are m any problem s wit h t he dist ribut ed t wo- phase com m it . I n general, you should t ry t o avoid using 2PC on t he SOA level ( i.e., across m ult iple service inst ances) . I n t he following, we will present a num ber of pot ent ial solut ions based on our it inerary m anagem ent scenario and discuss t heir respect ive benefit s and drawbacks.

8.3.4.1 First Iteration: Client Controlled Transactions A possible alt ernat ive t o t he sim ple error handling m echanism would be t he int roduct ion of a t ransact ion m anager, which enables t he grouping of bot h crit ical operat ions int o one dist ribut ed t ransact ion, as depict ed in Figure 8- 9 .

Figu r e 8 - 9 . W it h clie n t - con t r olle d t r a n sa ct ion s, t h e t r a n sa ct ion bou n da r y spa n s t h e e n t ir e syst e m , in clu din g t r a n sa ct ion a l clie n t s, a ll se r vice im ple m e n t a t ion s, a n d r e sou r ce m a n a ge r s. [View full size image]

Technically, t his approach would solve our problem because we could now ensure t hat no it inerary is finalized wit hout creat ing a corresponding invoice. However, t here are severe problem s wit h t his approach: The approach requires t hat we enable our client s t o deal wit h dist ribut ed t ransact ions. This is generally a bad idea, especially for light weight user int erfaces and Web servers, because it dram at ically increases t he com plexit y of service usage. Ot her issues t hat were ident ified during t he discussion on dist ribut ed 2PC and t ight ly coupled ACI D t ransact ions at t he beginning of t his chapt er can arise, including t he very t ight coupling at t he t echnology and prot ocol layers, low perform ance, lack of support for longlived t ransact ions, and problem s wit h int egrat ing legacy applicat ions and applicat ion packages. Essent ially, we are in danger of creat ing a solut ion t hat is com plex t o im plem ent and adm inist er and t hat severely lim it s t he reuse pot ent ial of our backend services because now only t ransact ional client s can use t hem . For t hese reasons, we need t o look at alt ernat ive solut ions.

Avoid Exposing Transaction Logic to Service Clients I n 99% of cases, exposing t ransact ion cont rol t o service client s is a bad idea. Dist ribut ed 2PC relies on ext rem ely t ight coupling of client s and server- side services on m any levels, which is fundam ent ally against t he design principles of ServiceOrient ed Archit ect ures, which are about independent , loosely coupled services.

8.3.4.2 Second Iteration: Server Controlled Transactions Rat her t han expose t ransact ion logic t o service client s, you should consider m oving t ransact ion cont rol t o t he server side. Transact ions t hat involve m ult iple updat es ( or even t hat span m ult iple resource m anagers) should be encapsulat ed inside a single service operat ion, where possible. I n our exam ple, we could consider com bining our confirm_itinerary() and create_invoice() operat ions int o a single confirm_itinerary_and_create_invoice() operat ion. This would elim inat e t he need t o expose t ransact ion logic t o our service client because we have now m oved t he m anagem ent of t he dist ribut ed t ransact ion t o t he server side, as depict ed in Figure 8- 10.

Figu r e 8 - 1 0 . Tr a n sa ct ion s spa n n in g m u lt iple r e sou r ce m a n a ge r s ( da t a ba se s a n d qu e u e s) ca n be e n ca psu la t e d in side a sin gle se r vice ope r a t ion . H ow e ve r , t h e da n ge r of cr e a t in g m on olit h ic se r vice s t h a t a r e n ot r e u sa ble e x ist s. [View full size image]

Alt hough in t his approach we can now hide t he com plexit y for dist ribut ed t ransact ion processing from our service client s, t here are st ill som e issues wit h t his design: Possibly t he biggest problem wit h t his approach is t hat our it inerary m anager has now becom e a m onolit hic service t hat assum es cont rol over anot her service's dat abase and t hat is hardly reusable. By let t ing one service im plem ent at ion access anot her service's dat abase direct ly, we have failed t o achieve t he m ost im port ant design goal of an SOA, nam ely, t he creat ion of loosely coupled, independent services. This approach requires t hat bot h dat abases part icipat e in a 2PC. I gnoring t he previously described design issues, t his m ight also represent a t echnical problem : recall t hat t he billing syst em t hat is responsible for invoice m anagem ent is a legacy syst em . This syst em m ight not be designed t o part icipat e in a 2PC, eit her because t he underlying dat abase is not 2PC enabled ( i.e., XA- com pliant ) , or because t he billing applicat ion relies on t ot al cont rol over t he dat abase ( a com m on scenario) and cannot handle t ransact ions t hat bypass it . Finally, t his second it erat ion of our design st ill requires an infrast ruct ure for handling a t ransact ion t hat spans t wo dat abases. This m eans t hat we st ill incur t he pot ent ially high license cost for a t ransact ion m onit or, plus t he addit ional overhead for im plem ent at ion and m aint enance, which should not be underest im at ed. Not only is our im plem ent at ion com plex, but it also requires higher- skilled developers and syst em s adm inist rat ors.

8.3.4.3 Third Iteration: Implicit Application Level Protocol Given t he problem s wit h t he first t wo it erat ions of our design, t he t hird design it erat ion could consider m oving t he process int egrit y issues t hat we have wit h our confirm_itinerary() and create_invoice() operat ions t o t he applicat ion level. We could agree upon an im plicit applicat ion

level prot ocol as follows: We split t he creat ion and act ual sending of t he invoice int o t wo operat ions ( so far, we have assum ed t hat create_invoice() would creat e and send t he invoice) . The invoice is creat ed before t he it inerary is finalized. This helps t o ensure t hat no it inerary lacks a corresponding invoice. I f we can creat e t he invoice successfully, we can next confirm t he it inerary and call send_itinerary(). Because send_invoice() can st ill fail, we creat e a background t ask in t he process engine t hat checks every night for inconsist encies bet ween it ineraries and invoices. I f t his process det ect s a confirm ed it inerary for which t he invoice has not been sent , t he process ensures t hat t his t akes place. Figure 8- 11 provides an overview of t his approach. Alt hough it solves all t he issues relat ed t o dist ribut ed t ransact ion processing, it is som ewhat lim it ed. I n part icular, it requires t hat client s now adhere t o a com plex yet im plicit prot ocol at t he applicat ion level. For exam ple, no client can confirm an it inerary wit hout previously creat ing an invoice. We are not only relying on our client s t o play by t he rules, but we are also dram at ically increasing t he com plexit y for our client s by forcing t hem t o im plem ent com plex business logic. Chapt er 7 provides a discussion of t he disadvant ages of put t ing com plex business logic int o applicat ion front ends.

Figu r e 8 - 1 1 . An im plicit a pplica t ion - le ve l pr ot ocol cou ld be a gr e e d u pon in or de r t o e n su r e con sist e n cy be t w e e n in voice s a n d fin a lize d it in e r a r ie s.

8.3.5. BUILD TRANSACTIONAL STEPS Having discussed t he issues wit h dist ribut ed 2PC and im plicit applicat ion- level prot ocols in t he previous sect ion, we have st ill not arrived at a com plet ely sat isfact ory solut ion for our confirm_itinerary_and_create_invoice() problem . I n t his sect ion, we look at how t he

concept of t ransact ional st eps m ight provide a bet t er solut ion t han t he previous design it erat ions. Recall t hat a t ransact ional st ep is a set of act ivit ies t hat are closely relat ed t o one anot her, execut ed in t he cont ext of a single t ransact ion, wit h a queue as an input feed for t hese act ivit ies and anot her queue as a st ore for t he out put of t hat st ep. We now look at how t his concept can be applied t o our problem .

8.3.5.1 Fourth Iteration: Fully Transactional Step I f we apply t he t ransact ional st ep concept t o our confirm_itinerary_and_create_invoice() problem , a possible solut ion m ight look as follows: The confirm_itinerary() operat ion sim ply st ores a m essage in a " pending confirm at ions" queue. This queue serves as t he input for a background t hread, which represent s a t ransact ional st ep. This t hread uses t he " pending confirm at ions" queue as an input queue. For each pending confirm at ion, t he appropriat e st eps ( confirm at ion and invoice creat ion) are execut ed in t he cont ext of a t ransact ion, and t he result is st ored in an out put queue of " confirm ed it ineraries." The syst em not ifies t he cust om er as soon as t he it inerary has been successfully finalized. Alt ernat ively, in t he case of a problem , a dialogue wit h t he cust om er begins, aim ed at solving t he problem s wit h t he it inerary. Figure 8- 12 shows a possible im plem ent at ion archit ect ure.

Figu r e 8 - 1 2 . Th e in t r odu ct ion of a t r a n sa ct ion a l st e p on t h e se r ve r side e n a ble s u s t o de cou ple t h e clie n t 's confirm_itinerary() ca ll fr om t h e a ct u a l se r ve r - side pr oce ssin g of t h e con fir m a t ion r e qu e st . [View full size image]

This approach solves a num ber of t he problem s wit h t he first couple of design it erat ions and also provides addit ional benefit s: We rem ove com plexit y from t he client s and inst ead place it int o a service under our own cont rol. We no longer risk losing incom e due t o t he syst em confirm ing it ineraries wit hout sending out a corresponding invoice because t he int eract ions wit h t he input and out put queues are t ransact ionally secured. However, t here are also som e significant drawbacks: This approach is not m uch bet t er t han t he second design it erat ion wit h it s confirm_itinerary_and_create_invoice() approach because t his design also requires t hat t he I t ineraryManager accesses t he billing syst em 's dat abase direct ly, t hus bypassing t he service layer, which was designed t o provide service- orient ed abst ract ions from t his dat abase in t he first place. Finally, we have now reint roduced t he need for a t ransact ion m onit or, which coordinat es t he 2PC bet ween different queues and dat abases, wit h all associat ed cost s and com plexit ies.

8.3.5.2 Fifth Iteration: Semi-Transactional Step Rat her t han adding com plex logic t o t he it inerary m anager service and accessing t he billing syst em 's dat abase direct ly, we now creat e a new Confirm at ionManager service. This is a processcent ric service ( see Chapt er 6, " The Archit ect ural Roadm ap" ) t hat im plem ent s a " less" t ransact ional st ep t o encapsulat e t he required funct ionalit y. Figure 8- 13 provides an overview of t he im plem ent at ion archit ect ure. The confirm _it inerary( ) operat ion st ores only a request in t he " pending confirm at ions" queue, which serves as an input queue for background t hreads. This t hread processes pending confirm at ions in a t ransact ion. The t ransact ion calls t he basic services I t ineraryManager and I nvoiceManager. Not ice t hat t hese calls are part of t he t ransact iont hat is, we do not assum e t hat a t ransact ion cont ext is propagat ed t o t hese servicest hese services execut e t heir own t ransact ions t o updat e t heir dat abases but are not part of t he Confirm at ionManager's t ransact ion.

Figu r e 8 - 1 3 . I n t r odu cin g a de dica t e d Con fir m a t ion M a n a ge r in com bin a t ion w it h se m i- t r a n sa ct ion a l st e ps e n a ble s u s t o lim it t h e t r a n sa ct ion bou n da r y t o t h e confirm_itinerary() im ple m e n t a t ion . W e a r e t h u s n o lon ge r for ce d t o a cce ss a n ot h e r se r vice 's da t a ba se dir e ct ly, a s t h is is a fu n da m e n t a l viola t ion of SOA de sign pr in ciple s. [View full size image]

This final design it erat ion based on a " less" t ransact ional st ep solves our m ost pressing problem s: We are now back t o a clean separat ion bet ween " basic" and " process- orient ed" services ( see Chapt er 5 on service t ypes) , significant ly enhancing t he reusabilit y and m aint ainabilit y of t he syst em .

The basic services ( cust om er and invoice m anager) now fully encapsulat e t heir dat abases, and we do not bypass t hese services as part of t he it inerary confirm at ion. I nst ead, t hey ret ain t heir reusabilit y pot ent ial. The new Confirm at ionManager service now cont ains t he specialized ( and probably less reusable) business logic, which is required t o finalize it ineraries. We do not require a 2PC across m ult iple resource m anagers ( we rely on t he queue m anager's built - in t ransact ion m echanism ) , and t hus we do not need a t ransact ion m onit or, which reduces com plexit y and cost s. However, t here is one out st anding issue wit h t his design: The t ransact ional bracket s around t he in- queue and out - queue only offer a lim it ed guarant ee of non- approval of it ineraries wit hout creat ing a corresponding invoice. These bracket s provide only a guarant ee t hat we do not lose any it inerary confirm at ion request s due t o t he t ransact ional nat ure of t he queues. However, we st ill need t o deal wit h t he scenario in which t he t ransact ion logic fails because it is possible t o confirm t he it inerary even when t he subsequent creat ion of an invoice fails. Because t he request s t o t he it inerary and invoice m anager are not t ransact ional, abort ing t he confirm at ion m anager's t ransact ion will not undo t he changes already applied t o cust om er and invoice dat abases. I nst ead of abort ing t he st ep's t ransact ion in case of a problem , we should creat e an error t oken, place t he t oken int o an error queue, and com m it t he t ransact ionwe are st ill guarant eed not t o lose any inform at ion because t he error queue is part of t he t ransact ion. However, we m ust deal wit h t his problem , and we will look int o t hese issues in t he next t wo sect ions.

8.3.5.3 Sixth Iteration: Choosing the Right Level of Granularity for Individual Steps The basic idea of t ransact ional st eps is t hat we can com bine t hem t o creat e chains or even graphs of relat ively independent yet logically relat ed st eps. Choosing t he right granularit y for individual st eps is difficult . On one hand, we do not want t o clut t er t he syst em wit h m eaningless m icro- st eps. On t he ot her hand, t he m ore fine- grained t he individual st eps, t he m ore flexibly t hey can be rearranged t o address changes in t he business logic or t o enable bet t er error handling. I n t he case of t he it inerary m anagem ent exam ple, we need t o ant icipat e t wo basic problem s: a problem wit h t he it inerary it self ( e.g., a confirm ed seat on a flight is no longer available due t o cancellat ion of t he flight ) or a problem wit h t he creat ion of t he invoice. Each case m ust be addressed individually by invoking t he m at ching com pensat ion logic. We could also have achieved t his by adding appropriat e error handling logic. For exam ple, we could have individual t ry/ cat ch blocks for each rem ot e invocat ion, as follows:

BEGIN { pendingConfirmationQ.get(); try { itineraryMgr.confirm_itinerary(); } catch (e1){ errorQ.put(e1) COMMIT; return; } try {

invoiceMgr.create_invoice(r1); } catch (e2) { errorQ.put(e2) COMMIT; return; } confirmedItineraryQ.put(); } COMMIT;

However, t his approach lim it s t he flexibilit y of rearranging t he execut ing order of each individual st ep because code m ust be m odified t o change t he order of execut ion. Anot her approach is t o change t he design by split t ing t his st ep int o t wo independent st eps, which are t hen linked t oget her. Each st ep is now responsible for handling only a single part of t he overall t ransact ion. I n t he case of a problem wit h each of t hese st eps, we can put t he result ing error t oken int o a separat e error queue. The first error queue is responsible for handling problem s wit h it ineraries, while t he second is responsible for handling problem s wit h invoices. Figure 8- 14 provides an overview of how t he init ially large t ransact ional st ep can be broken down int o t wo m ore fine- grained st eps.

Figu r e 8 - 1 4 . I n t r odu cin g fin e r - gr a in e d t r a n sa ct ion a l st e ps in cr e a se s t h e fle x ibilit y of t h e syst e m , e spe cia lly w it h r e spe ct t o e r r or h a n dlin g. [View full size image]

Assum ing a sufficient ly generic syst em design ( in part icular t he m essage form at s passed bet ween t he different queues) , t he use of m ore fine- grained t ransact ional st eps facilit at es t he reconfigurat ion of t he execut ion order by changing t he configurat ion of our input and out put queues. This perm it s us t o change t he syst em on t he fly t o im prove t he way in which we handle error sit uat ions. For exam ple, we m ight want t o change t he order of st eps t o creat e t he invoice before confirm ing t he it inerary. Not ice t hat even if t his reordering does not happen com plet ely " on- t he- fly" ( which is t he likely caseoft en t he " on- t he- fly" reconfigurat ion fails due t o som e m inor dat a conversion or configurat ion issues) , we will be pret t y close t o a com plet ely flexible solut ion. I n part icular, t his approach elim inat es t he need t o set up a large proj ect t o im plem ent t he required changes because such a reconfigurat ion represent s m uch lower risks com pared t o a m aj or change in t he applicat ion code. Anot her benefit of t his approach is t hat we lim it t he num ber of error scenarios in individual st eps, and we can associat e m ore m eaningful error queues wit h each st epeach problem t ype is report ed int o a separat e error queue.

8.3.6. USE SIMPLE YET FLEXIBLE COMPENSATING LOGIC I n t he t heory of SAGAs and chained t ransact ional st eps, each st ep in a chain is associat ed wit h a com pensat ing t ransact ion. I n case of failure during t he processing of a t ransact ion chain, we sim ply call t he com pensat ing t ransact ions for each of t he st eps execut ed so far. I n our it inerary m anagem ent exam ple, confirm_itinerary() and create_invoice() could be associat ed wit h com pensat ing t ransact ions, which would aim t o undo t he previous changes, as shown in Figure 815 .

Figu r e 8 - 1 5 . I n a n ide a l w or ld, e a ch st e p in a t r a n sa ct ion ch a in is a ssocia t e d w it h a com pe n sa t in g t r a n sa ct ion . I n ca se of fa ilu r e , t h e com pe n sa t in g t r a n sa ct ion s for a ll su cce ssfu lly e x e cu t e d st e ps a r e ca lle d t o u n do a ll pr e viou s ch a n ge s.

This works in t heory, but in pract ice, t here are several lim it at ions t o t his approach. First , in m any cases, a specific operat ion will have no sim ple com pensat ing t ransact ion. Take our confirm_itinerary() operat ion: I f an it inerary has been confirm ed once, real cost s have been creat ed because we will have m ade reservat ions for flight s, cars, and hot els, which cannot sim ply

be canceledin m any cases, a fee will be associat ed wit h a cancellat ion. Second, m any business processes are not lineart hat is, t hey are not sim ple chains of t ransact ions but are based on com plex, cont ext - sensit ive decision graphswhere cont ext refers t o t echnical as well as business- relat ed inform at ion and const raint s. Alt hough in t heory t he concept of com pensat ion should not only apply t o linear chains but also t o com plex graphs, t his dram at ically increases t he com plexit y of com pensat ions, especially due t o t he cont ext sensit ivit y of m any decision t rees, which oft en will have an im pact on different ways t o com pensat e part icular nodes. Finally, we are likely t o encount er problem s not only in t he t ransact ional st eps but also in t he com pensat ing t ransact ions. How do we deal wit h a problem in a com pensat ing t ransact ion? I s t here a com pensat ion for a failed com pensat ion? I n m any cases, all of t his m eans t hat a 1: 1 m apping bet ween t ransact ional st eps and com pensat ing t ransact ions is not feasible. Recall our discussion on except ions versus special cases at t he beginning of t his chapt er. For exam ple, is an " out of st ock" sit uat ion an except ion or sim ply a special case t hat m ust be dealt wit h at t he business level? I n our it inerary m anagem ent exam ple, it seem s very likely t hat an overly sim plist ic com pensat ion approach is not going t o work. For exam ple, we will m ost likely have t o t ake int o account cancellat ion fees result ing from a com plet e or part ial cancellat ion of an it inerary. Figure 8- 16 shows how a revised, m ore realist ic workflow for it inerary finalizat ion and invoice creat ion m ight look.

Figu r e 8 - 1 6 . Th is e x a m ple sh ow s h ow t h e h a n dlin g of e x ce pt ion s in t h e it in e r a r y fin a liza t ion be com e s pa r t of t h e w or k flow . Th u s, w e a r e n o lon ge r de a lin g w it h e x ce pt ion s or com pe n sa t in g t r a n sa ct ion s. I n st e a d, w e t r e a t t h e se sit u a t ion s a s " spe cia l ca se s" w it h in t h e n or m a l w or k flow .

Consider BPM If It Is Getting More Complex I f t he com plexit y of your process definit ions, including special cases and com pensat ion logic, is get t ing out of hand, you m ight want t o consider using a BPM plat form . Make sure t hat t his plat form provides you not only wit h sufficient ly rich graphical t ools for m odeling your st andard and special case processes but also t he abilit y t o m odel com pensat ing t ransact ions and a fram ework for aut om at ically m apping t hese associat ed com pensat ing t ransact ions t o a t ransact ional execut ion environm ent .

8.3.7. COMBINE SOA, MOA, AND BPM TO INCREASE FLEXIBILITY When exam ining proj ect s wit h very com plex process logic t hat spans t he boundaries of ent erprises or ot her organizat ional barriers, it m akes sense t o look at t he com binat ion of SOA, MOA, and BPM t o increase t he flexibilit y wit h which processes can adapt t o changes on eit her side of t he organizat ion's boundary, as depict ed in Figure 8- 17.

Figu r e 8 - 1 7 . Com bin in g SOA, M OA, a n d BPM pr ovide s a fle x ible m e a n s

of in t e gr a t in g com ple x pr oce sse s t h a t cr oss e n t e r pr ise bou n da r ie s.

[View full size image]

I n such a case, t he SOA would provide basic services and process- orient ed services wit hin each individual ent erprise, as we discussed in Chapt er 4. The basic services provide t he core business logic, while t he process- cent ric services ( e.g., im plem ent ed by a BPM engine) provide t he act ual business process logic. Using an MOA rat her t han an SOA t o provide int egrat ion across ent erprise boundaries m akes sense in m any cases. First ly, an MOA can provide great er flexibilit y wit h respect t o m essage form at s and different t ypes of m essaging m iddleware. I n addit ion, inherent ly asynchronous MOAs are bet t er suit ed t o prot ect an ent erprise against t im e delays on t he part ner side, which are out side t he cont rol of t he issuing side. I n addit ion, t he SOA usually provides st ore and forward funct ionalit y ( " fire and forget " ) , which elim inat es som e of t he issues we discussed earlier on wit h respect t o m aking operat ions idem pot ent . Finally, an MOA in com binat ion wit h a BPM is well suit ed t o represent Pet ri- Net - like com m unicat ion t rees wit h m ult iple branches, which can oft en be required in t hese t ypes of int egrat ion scenarios.

8.4. Conclusion Ensuring process int egrit y is oft en m ore a proj ect m anagem ent issue t han a t echnical issuet he available t echnical solut ions are relat ively well known, but how and when t o select each solut ion is oft en a key problem . Finding t he right t radeoff bet ween int egrit y requirem ent s from a business perspect ive and sophist icat ion of t he t echnical solut ion is a t ask t hat requires careful analysis from responsible archit ect s and proj ect m anagers. I t is im port ant for proj ect m anagers t o underst and t hat a very clear t radeoff exist s bet ween t he level of process int egrit y and im plem ent at ion cost s. For exam ple, while t he t wo- phase com m it prot ocol pot ent ially provides a very high level of int egrit y ( at least in hom ogeneous, t ight ly coupled environm ent s) , it com es wit h very high im plem ent at ion cost s because it requires highly skilled archit ect s and developers and very expensive soft ware. However, t ransact ional st eps incur lower cost s and offer reasonable process int egrit y propert ies. They oft en provide a very good cost / int egrit y t radeoff ( Figure 8- 18 cat egorizes several approaches t o process int egrit y in regard t o t heir cost int egrit y rat io) .

Figu r e 8 - 1 8 . I t is k e y for pr oj e ct m a n a ge r s t o r e a lize t h e t r a de off be t w e e n t h e le ve l of pr oce ss in t e gr it y pr ovide d by a pa r t icu la r t e ch n ica l solu t ion a n d it s im ple m e n t a t ion cost s. [View full size image]

I f t echnical proj ect m anagers can com m unicat e t hese t radeoffs clearly t o t he decision m akers at t he business level, t hey enable t hem t o m ake j udgm ent s regarding t he level of consist ency

requirem ent s t hey have and t he m oney t hey are prepared t o spend on t hem . Chapt er 13, "SOA Proj ect Managem ent," adds t o t his discussion by providing concret e t ools for proj ect m anagers t o enable t hem t o find t he right t radeoff bet ween im plem ent at ion cost s and int egrit y requirem ent s.

References [ Co70] Codd, E. F . A Relat ional Model of Dat a for Large Shared Dat a Banks. ACM Com m unicat ions, Volum e 13, No 6, June 1970.

Chapter 9. Infrastructure of the Service Bus I n t his chapt er, we describe t he different elem ent s t hat const it ut e an SOA- enabling infrast ruct ure, referred t o in t he following as a service bus. One of t he fundam ent al principles t hat t his chapt er is based upon is t he assum pt ion t hat it is im possible for large organizat ions t o enforce st andardizat ion on a single t echnical archit ect ure. We believe t hat t his is especially t rue wit h respect t o com m unicat ion m iddleware, applicat ion plat form s, or int erface t echnology. Consequent ly, t his chapt er is not about describing specific t echnical st andards for a service bus. I nst ead, we propose t o look at a service bus as a kind of m et a bus, which is com prised of t he different exist ing soft ware buses and m iddleware plat form s t hat you will find in your organizat ion. The j ob of t he service bus is not only t o enable basic int eract ion wit h different service com ponent s across t he different plat form s, but also t o t ie t oget her t he different higher- level infrast ruct ure funct ions of t hese plat form s. Aft er a general overview of t he service bus concept in Sect ion 9.1, Sect ion 9.2 looks at logging and audit ing, followed by scalabilit y and availabilit y in Sect ion 9.3, and securit y in Sect ion 9.4.

9.1. Software Buses and the Service Bus People oft en use t he t erm soft ware bus t o refer t o t he t echnical infrast ruct ure of t he dist ribut ed environm ent . We consider a soft ware bus t o be analogous t o t he well- known concept of a hardware bus. Much as a hardware bus enables t he int egrat ion of hardware part s from different vendors, for exam ple when assem bling a deskt op com put er, a soft ware bus is t he st andardized way of hooking t oget her any soft ware com ponent s.

9.1.1. BASIC CONCEPTS OF A REAL-WORLD SERVICE BUS The m ost widely known soft ware bus is OMG's CORBA, essent ially a com m unicat ion infrast ruct ure for individual obj ect inst ances. The CORBA infrast ruct ure enables an obj ect t o locat e any ot her obj ect on t he bus and invoke any of t hat obj ect 's operat ions. The CORBA m odel does not m ake a st rict dist inct ion bet ween client s and servers and is essent ially a sym m et rical bus. CORBA is a very m at ure t echnology, but unfort unat ely, it s underlying concept leans it self t o a very finegrained com m unicat ion infrast ruct ure t hat creat ed a hist ory of m aint enance and perform ance problem s in m any proj ect s. Whereas CORBA is very generic bus t echnology wit h a focus on obj ect orient at ion, anot her concept of a soft ware bus recent ly em erged called t he Ent erprise Service Bus [ DC2004] . Alt hough as of t his writ ing, it cannot be considered anywhere near a st andard, it s m ain elem ent s are a coarse- grained XML com m unicat ion prot ocol t oget her wit h a m essage- orient ed m iddleware core t o perform t he act ual m essage delivery. A num ber of ot her soft ware buses are on t he m arket , am ong t hem t he Ent erprise Java Beans as part of t he J2EE specificat ion, Microsoft 's .NET, and various m essaging product s, including I BM MQSeries and Tibco Rendezvous. All t hese buses require st andardizat ion on a single int eract ion and com m unicat ion m odel, as shown in Figure 9- 1 . For exam ple, CORBA and EJB prom ot e synchronous obj ect - orient ed com m unicat ion, whereas m essaging product s such as MQSeries support asynchronous docum ent - orient ed com m unicat ion.

Figu r e 9 - 1 . I de a l soft w a r e bu s t h a t su ppor t s a sin gle com m u n ica t ion m ode l. All a pplica t ion s m u st con for m t o t h e sa m e st a n da r d, su ch a s CORBA or M QSe r ie s.

I n a real ent erprise applicat ion landscape, a single com m unicat ion m odel will hardly suffice. I nst ead, you will need various com m unicat ion m odels based on t he requirem ent s of t he individual applicat ion. Vendors have long underst ood t his point , and t hey provide environm ent s t hat support m ult iple com m unicat ion m odels at t he sam e t im e. The J2EE environm ent , for exam ple, support s various com m unicat ion t ypes, as shown in Figure 9- 2 . EJBs provide synchronous obj ect - orient ed com m unicat ion, t he Java Message Service ( JMS) provides m essaging, t he syst em support s com m unicat ion using em ail and SOAP, and t he Servlet specificat ion provides general support for HTTP applicat ions.

Figu r e 9 - 2 . A soft w a r e bu s t h a t su ppor t s va r iou s com m u n ica t ion m ode ls a t t h e sa m e t im e , su ch a s syn ch r on ou s, a syn ch r on ou s, or file ba se d com m u n ica t ion .

I n t he real world, t he sit uat ion is usually even m ore com plicat ed because product s from different vendors t hat support sim ilar com m unicat ion m odels are oft en in use at t he sam e t im e. This sit uat ion can arise when different depart m ent s of t he com pany int roduce com pet ing t echnology or when a new t echnology ent ers t he environm ent as t he result of a m erger or acquisit ion. A t ypical ent erprise " soft ware bus" will usually look like t hat depict ed in Figure 9- 3 .

Figu r e 9 - 3 . Th e in fr a st r u ct u r e of a r e a l- w or ld e n t e r pr ise w ill n or m a lly con sist of va r iou s pr odu ct s t h a t su ppor t sim ila r com m u n ica t ion m ode ls.

Of course, single applicat ions m ight use various com m unicat ion m odels when com m unicat ing wit h each ot her. For exam ple, an applicat ion m ight call anot her one using a synchronous t echnology if an im m ediat e answer is required and m ight use an asynchronous com m unicat ion m odel if it requires guarant eed and reliable delivery. However, no m at t er what t echnology you use, an ent erprise SOA infrast ruct ure m ust conform t o cert ain st andards across t he board in order t o achieve cert ain goals, such as securit y and audit ing t hat is independent of t he product or t he net work prot ocol. I n t his respect , a service bus is not like a general soft ware bus. An ent erprise m ust creat e a higher- level ent it y, som e kind of Über bus or Met a bus t hat endorses all t he various product s and t echnologies of t he ent erprise.

Create a Meta Bus Do not enforce t he usage of a single product , even one t hat support s m any com m unicat ion m odels, as t he sole service bus of t he organizat ion. Plan for a service bus on a m et a level t hat support s t echnical services at a higher level and adds flexibilit y t hat can be crit ical for t he business during acquisit ions or m aj or rest ruct uring.

This is why m ost of t he product s available on t he m arket cannot be considered service buses in t heir own right . I nst ead, t hey fall int o oneor bot hof t wo cat egories: com m unicat ion fram eworks and execut ion cont ainers. Com m unicat ion fram eworks are essent ially infrast ruct ures t hat only facilit at e com m unicat ion wit hout m any addit ional infrast ruct ure services. Pract ically all com m unicat ion fram eworks are built around t he concept of a st ub and a dispat cher. Typical com m unicat ion fram eworks are MOM product s and plain rem ot e m et hod invocat ion fram eworks. Execut ion cont ainers provide m uch m ore sophist icat ed infrast ruct ure support , including support for t ransact ions and securit y. Typical execut ion cont ainers are CORBA and t he EJB cont ainer. However, do not develop t his m et a bus in isolat ion from concret e applicat ion proj ect s. Avoid keeping t he concept s t oo abst ract ; do not build t oo m any t echnical feat ures int o it init ially. Chapt er 12, " The Organizat ional SOA Roadm ap," describes how and why you should develop an SOA infrast ruct ure gradually, hand- in- hand wit h concret e business applicat ion proj ect s. Chapt er 16 , " Credit Suisse Case St udy," discusses a case st udy, out lining how a com pany int roduced a synchronous I nform at ion Bus ( CSI B) , an asynchronous Event Bus I nfrast ruct ure ( EBI ) , and a file t ransfer- based Bulk I nt egrat ion I nfrast ruct ure ( BI I ) , all driven by t he dem and from applicat ion proj ect s, which in t urn were driven by concret e business dem ands.

9.1.2. SERVICE STUB AND DISPATCHER A service st ub is a piece of soft ware t hat is locat ed at t he client side of a service ( see Figure 9- 4 ) . I t provides a local API t hat present s a convenient access m et hod for t he rem ot e service. The service st ub encapsulat es t echnical funct ionalit y such as handling of t he net work and applicat ion prot ocol, m arshalling and unm arshalling of dat a, and st andard securit y m echanism s.

Figu r e 9 - 4 . A se r vice st u b r e pr e se n t s t h e se r vice for a clie n t . I t pr ovide s a con ve n ie n t API t h a t t h e clie n t de ve lope r ca n e a sily u se . Th e se r vice st u b e n ca psu la t e s t h e com ple x it y of t h e t e ch n ica l a cce ss of t h e se r vice su ch a s n e t w or k pr ot ocol, m a r sh a llin g a n d u n m a r sh a llin g of da t a , st a n da r d se cu r it y m e ch a n ism s, e t c.

The service dispat cher is t he count erpart of t he service st ub. I t receives incom ing net work request s t hat are generat ed by t he service st ub. The service dispat cher analyzes t hese request s t echnically and invokes t he request ed operat ion wit h t he appropriat e dat a.

9.1.2.1 Code Generation Code generat ion is a powerful t echnique t hat can be applied t o SOA proj ect s in an efficient m anner. I n such a scenario, t he code generat or encapsulat es all t echnical knowledge regarding dist ribut ion and service invocat ion. The applicat ion developers can focus on t he business aspect s of int erface definit ion, while t he code generat or covers t echnical issues. Code generat ion decouples t he funct ional code from t he t echnical code. I t provides support for different program m ing languages, various net work and applicat ion prot ocols, and different operat ing syst em s wit h a single code base. Code generat ion can be used t o generat e t est client s and t est serversa handy feat ure t hat can prove beneficial for t he overall developm ent process. Last but not least , code generat ion t ypically increases t he qualit y of t he t echnical code. Cont rary t o handcraft ed code, t he generat ed code will be uniform . I m provem ent s in generat ed code ( e.g., bet t er m arshalling or connect ion m anagem ent ) can be m ade wit hout causing an im pact on current applicat ions if t he changes are rest rict ed t o t he inside of t he API of t he generat ed code. Proj ect s t hat do not use code generat ion oft en suffer from a cut - and- past e syndrom e. I n t his scenario, developers of business funct ionalit y int egrat e one funct ioning version of t he t echnical code int o t heir im plem ent at ion of t he business logic. This code will be repeat edly reused, while slight changes t o t hat code can lead t o m any different im plem ent at ions of t he sam e t echnical funct ionalit y.

Successful Use of Code Generation Use code generat ion t o aut om at e repet it ive t asks t hat cannot be resolved using code libraries, such as generat ing t ype- safe service int erface st ubs, which provide highperform ance m arshalling code. Avoid changing generat ed code m anually at all cost s! You cannot regenerat e code wit hout losing your changes, which will be a night m are if you use generat ed code across m any different int erfaces. I nst ead, m odify t he code generat or if possible. Alt ernat ively, use sm art script s t hat can different iat e bet ween handcraft ed and generat ed code wit hin a single file.

Basically, you can use code generat ion in a t op- down or bot t om - up fashion. Whereas t op- down code generat ion is based on form al int erface definit ions such as I DL or WSDL, t he bot t om - up approach analyzes t he source code of t he service im plem ent at ion. A t ypical candidat e for t opdown code generat ion is CORBA, where st ubs and skelet ons are generat ed from I DL int erface definit ions. Many m odern Web service fram eworks, such as .NET or J2EE applicat ion servers, support bot t om - up m apping of exist ing API s ( e.g., Java classes) t o Web service int erfaces ( WSDL) , oft en using a com binat ion of int ernal code generat ion and reflect ion.

9.1.2.2 Top-Down Approach The precondit ion for t op- down code generat ion is t he usage of a form al int erface definit ion language such as I DL or WSDL. Using a form al int erface definit ion language has a great im pact on t he developm ent process. I t decouples t he form al descript ion of int erfaces from t he act ual coding. As illust rat ed in Figure 9- 5 , t his decoupling enables developm ent t eam s t o sim plify t he coordinat ion of t he service program m ers and t heir client s.

Figu r e 9 - 5 . Code ge n e r a t ion is a pow e r fu l m e ch a n ism t o in cr e a se de ve lopm e n t pr odu ct ivit y a n d qu a lit y. Th e code ge n e r a t or e n ca psu la t e s t h e t e ch n ica l com ple x it y of t h e com m u n ica t ion be t w e e n clie n t a n d se r vice . Th is e n a ble s t h e de ve lope r s of bot h clie n t a n d se r vice t o focu s on t h e fu n ct ion a l code .

9.1.2.3 Bottom-Up Approach Code generat ion can also be perform ed using low- level im plem ent at ion API s ( e.g., Java classes or CI CS t ransact ion program s) rat her t han from abst ract , im plem ent at ion- independent int erface definit ions ( see Figure 9- 6 ) .

Figu r e 9 - 6 . Bot t om - u p code ge n e r a t ion is ba se d on t h e se r vice im ple m e n t a t ion . I t is t h e r e for e de pe n de n t on a spe cific pr ogr a m m in g la n gu a ge a n d t h e de t a ils of it s r u n t im e e n vir on m e n t a n d im ple m e n t a t ion t e ch n iqu e . Th e code ge n e r a t or ca n u se a se r vice de scr ipt ion la n gu a ge su ch a s W SD L a s a n in t e r m e dia t e r e pr e se n t a t ion .

This t echnique is part icularly useful when t ransform ing legacy applicat ions int o services. I f exist ing code m ust be exposed as a service, t he generat ion of t he service int erface can save a lot of developm ent t im e. The caveat is t hat we get service int erfaces t hat are program m ing language- specific, t echnologyfocused, and very fine- grained. This is part icularly t rue for t he t ransform at ion of t he t radit ional applicat ion wit h t erm inal- based user int erfaces such as VT3270. The user screens, which are t he nat ural st ruct ural elem ent s of t hese applicat ions, m ight be of an inappropriat e granularit y for a service int erface. The bot t om - up approach can also be applied t o t he developm ent of new applicat ions. Alt hough t here is no explicit represent at ion of t he int erface in a form al docum ent , t he careful design of t he int erface is st ill pivot al and cannot be om it t ed. Wit h t he bot t om - up approach com es t he danger of ad- hoc design, alt hough t here are also benefit s t o t his approach. For exam ple, no addit ional developm ent t ools are required for t he design of t he service. The developer of t he service can use t he preferred developm ent environm ent and m odeling t ools t hat are used for t he developm ent and design of t he original code. I f t he code generat or is int egrat ed int o t his environm ent , you can achieve very short t urn- around t im es. The bot t om - up approach leverages a developm ent m odel t hat is driven by t he im plem ent at ion of t he services. Very efficient developm ent environm ent s exist t hat support t he bot t om - up approach such as Microsoft 's Visual St udio .NET. The downside of such efficiency is a high degree of

t echnical dependencies t hat could result in a t echnology lock- in t hat j eopardizes all t he flexibilit y t hat should have been leveraged using t he SOA approach. Therefore, you should t ake care when deciding whet her t o em ploy a bot t om - up or t op- down st rat egy.

Top-Down Service Design Is the Preferred Approach in an SOA Service definit ions are probably t he single m ost im port ant art ifact s in an ent erprise SOA. Therefore, it is im port ant t hat you put a lot of t hought int o t he specificat ion of each individual service. Ensure t hat all service definit ions Meet business requirem ent s Are designed at t he right level of granularit y Provide pot ent ial for lat er reuse Can be im plem ent ed in a way t hat ensures scalabilit y and int egrit y Are independent of any underlying im plem ent at ion Provide appropriat e service level specificat ions Service definit ions should be designed explicit ly; t hey should not be generat ed from lower- level API s aut om at ically.

9.1.2.4 Code Generation With MDA Model Driven Archit ect ure ( MDA) is a cont em porary approach t o m anaging t echnologyindependent service specificat ions, and im plem ent ing and m anaging " SOA m et a- bus" archit ect ures, as described in Sect ion 9.1.1. MDA is t he um brella- t erm for a num ber of specificat ions t hat are current ly st andardized by t he Obj ect Managem ent Group ( OMG) , including UML, XMI , and MOF. MDA leverages UML t o specify Plat form I ndependent Models ( PI Ms) and Plat form Specific Models ( PSMs) . I n MDA t erm s, a PI M is t he form al specificat ion of t he t echnology- neut ral st ruct ure and funct ion of a syst em , while a PSM adds t he t echnical det ails which are needed for t he concret e im plem ent at ion of a soft ware com ponent . The Met a- Obj ect Facilit y ( MOF) is at t he heart of t he MDA concept . MOF- based m et am odels allow t he im plem ent at ion of m odel reposit ories, t he exchange of m odels ( via XMI ) , and t he t ransform at ion bet ween different m odels, e.g. from a PI M t o a PSM. While not lim it ed t o code generat ion t echniques, MDA is well suit ed t o using t hese t echniques by generat ing t he m apping from an abst ract m odel t o a concret e m odel im plem ent at ion. MOF is part icularly well suit ed t o define and im plem ent m odel t ransform at ions. A num ber of t ools and st andards support t he t ransform at ion of MOF m et a- dat a, e.g. t ransform at ion of UML m odels int o XML, CORBA I DL, Java, and lat ely int o WSDL. This allows for t he im plem ent at ion of very sophist icat ed code generat ors, which can generat e highly t arget ed code, as shown in Figure 9- 7 .

Figu r e 9 - 7 . M D A ca n be a good ba sis for t h e " SOA m e t a - bu s," pr ovidin g pla t for m - in de pe n de n t se r vice de fin it ion a n d m ode l t r a n sfor m a t ion s t h a t su ppor t a w ide r a n ge of m iddle w a r e a n d SOA in fr a st r u ct u r e pla t for m s. [View full size image]

I n t he light of our discussion on " t op- down" versus " bot t om - up" code generat ion ( refer t o t he previous sect ion) , t echnology- independent service definit ions would be usually defined in a t opdown fashion as a PI M, represent ing only business funct ionalit y. MDA t ools can t hen be used t o generat e plat form specific service int erfaces, e.g. in CORBA I DL or WSDL. I n m any cases, MDA t ools will be used in com binat ion wit h exist ing code generat ors, e.g. CORBA I DL com pilers or WSDL com pilers. However, m ost MDA t ools go beyond t he generat ion of service st ubs by support ing t he generat ion of prot ot ypical service im plem ent at ions, GUI descript ors, SQL code, et c. While MDA is a very powerful concept , it is clearly not a " silver bullet " : Like any ot her t echnology, MDA depends on a specific set of st andards and t ools. However, wit h it s focus on m odel t ransform at ion, MDA in com binat ion wit h SOA could help especially t hose ent erprises which are suffering part icularly badly from applicat ion and m iddleware het erogeneit y.

9.1.3. EXECUTION CONTAINERS The execut ion cont ainer is t he runt im e environm ent for t he service im plem ent at ion. Most runt im e environm ent s for ent erprise soft ware also provide appropriat e cont ainers for services, m ainly because t hey can provide guidanceand som et im es solut ionswhen cat ering for t he t echnical challenges t hat we describe in t his chapt er. Alt hough t he form al definit ion of cont ainer was coined wit h t he em ergence of t he Ent erprise Java Beans ( EJB) st andard, m any older applicat ion plat form s provide sim ilar feat ures for efficient , secure, and t ransact ional execut ion of service request s. These include t ransact ion m onit ors such as CI CS, dist ribut ed obj ect syst em s such as CORBA, Web applicat ion servers ( Servlet s) , and

dat abase m anagem ent syst em s ( st ored procedures) . The following sum m arizes t he generic feat ure set of an execut ion cont ainer: D ispa t ch in g a n d se r vicin g. There are different possibilit ies for rout ing an incom ing service call t o t he im plem ent at ion of t he corresponding service operat ion. The necessary dispat cher could eit her be part of t he service im plem ent at ion or an int egrat ed part of t he cont ainer. Tr a n sa ct ion m a n a ge m e n t . An execut ion cont ainer should include built - in facilit ies for t he m anagem ent of t ransact ions. For exam ple, dat a- cent ric services needing access t o dat abases and files under t ransact ion cont rol require t hese facilit ies. Process- cent ric services t hat invoke a variet y of ot her services while execut ing one inst ance of a business process also need t ransact ion m anagem ent . Se cu r it y. The securit y requirem ent s for execut ion cont ainers are sim ilar t o t hose for m any ot her server environm ent s. An execut ion cont ainer m ust provide facilit ies for aut hent icat ion, aut horizat ion, and t ransport encrypt ion. Loggin g. Cont rary t o m any m onolit hic archit ect ures wit h applicat ions t hat run com plet ely under t he cont rol of one single t ransact ion m onit or and use one dat abase, an SOA m ust cope wit h various issues of vert ical and horizont al dist ribut ion and decoupling. Logging is a part icular m easure t o address m any of t hese issues. The execut ion cont ainer m ust provide facilit ies t o keep t rack of all service invocat ions. For t he sake of perform ance or due t o legal considerat ions, one m ust be able t o configure t he scope of t he logging. At a m inim um , it m ust be possible t o swit ch on and off t he logging of a service invocat ion's payload. Furt herm ore, t he execut ion cont ainer m ust be able t o provide facilit ies for t he ret rieval and consolidat ion of log ent ries. Billin g. The developm ent , m aint enance, and operat ion of a service are not free. Therefore, it is a valid approach t o charge for t he usage of a service. This can be perform ed bot h wit hin an ent erprise ( cross- depart m ent al billing) and bet ween different organizat ions. I n order t o do billing, t he execut ion cont ainer m ust provide usage m et rics and it m ust m eet Service Level Agreem ent s ( SLAs) . Furt herm ore, you will need user account s t hat keep t rack of accum ulat ed cost s and t hat allow service usage only under defined com m ercial condit ions. Syst e m s m a n a ge m e n t fu n ct ion a lit y. I t is oft en necessary t o run a service in a highly aut om at ed operat ion environm ent . I n such an environm ent , t he service m ust provide facilit ies t o cooperat e wit h syst em s m anagem ent t ools. At a m inim um , it m ust provide funct ionalit y for st art ing and st opping services. Furt herm ore, it is beneficial t o have st at ist ical funct ions t hat connect t o t hese syst em s m anagem ent t ools and report errors, average ( or m inim um or m axim um ) processing t im es, resource consum pt ion, current ly act ive operat ions, upt im e, and m any m ore det ails. I n t his cont ext , it is wort h m ent ioning t hat m any t radit ional runt im e environm ent s such as t ransact ion m onit ors running on a m ainfram e provide a cert ain subset of t his funct ionalit y out - of- t he- box. More sophist icat ed syst em s m anagem ent funct ionalit y could be easily added t o t he base operat ions. Bot h developm ent t eam s and operat ions are aware of syst em s m anagem ent necessit ies and are used t o t he associat ed issues. As a result , t radit ional environm ent s are st ill a reasonable choice for t oday's m ission- crit ical applicat ions. M e ssa ge t r a n sfor m a t ion . Message t ransform at ion is a feat ure t hat you t ypically find in an EAI or B2B scenario. Here, you m ight have a het erogeneous set of client s t hat pot ent ially use different form at s t o represent t he payload of a service invocat ion, such as different flavors of SOAP m essages. I n such a case, it is very convenient if t he execut ion cont ainer provides preprocessing facilit ies t hat t ransform all incom ing calls t o a single unified form at . I n Sect ions 9.2 t o 9.4, we discuss som e of t he aforem ent ioned aspect s in m ore det ail.

9.1.3.1 Cross-Container Integration I ndividual execut ion cont ainers oft en provide a rich out - of- t he- box funct ionalit y t hat m akes t he developm ent , deploym ent , and m anagem ent of individual services reasonably st raight forward. However, alm ost all ent erprises suffer from " m iddleware overload." They m ust deal not only wit h a m ult it ude of incom pat ible applicat ions but also wit h t he exist ence of m any different applicat ion plat form s and com m unicat ion m iddleware syst em s. I n m ost ent erprises, m any different t ypes of execut ion cont ainers can be found, ranging from m odern .NET, Java Servlet s, and EJB cont ainers t o m ainfram e t ransact ion m anagers such as CI CS and I MS. The key challenge of an ent erprise SOA is t o define an archit ect ure t hat enables applicat ions t o use different services independent ly of t heir cont ainer. Alt hough sim ple service int eroperabilit y can be achieved relat ively easily t hrough int eroperable m essaging prot ocols such as I I OP or SOAP, t he challenge is t o connect services t hat reside in different cont ainers beyond sim ple request / response int eroperabilit y, including securit y and t ransact ionalit y. Figure 9- 8 provides an exam ple of a cust om er and flight booking syst em deployed across m ult iple service plat form s.

Figu r e 9 - 8 . I n a syst e m w h e r e se r vice s a r e im ple m e n t e d on in com pa t ible e x e cu t ion con t a in e r s, on e m u st in t r odu ce a h or izon t a l in fr a st r u ct u r e la ye r t h a t m a n a ge s t e ch n ica l cr oss- con t a in e r in t e gr a t ion of se r vice s. I f t h is ca n n ot be de ploye d a s a t e ch n ology la ye r , it is n e ce ssa r y t o in cor por a t e a ppr opr ia t e de sign pr in ciple s a n d pr oce du r e s t o e a ch con t a in e r in or de r t o a ddr e ss t h e r e la t e d pr oble m s. [View full size image]

I n som e cases, t he int roduct ion of an ext ernal horizont al infrast ruct ure layer can aut om at e som e of t he t asks of int egrat ion services across cont ainer boundaries at t he t echnical level. However, t his is oft en im possible, and m ore flexible solut ions m ust be incorporat ed int o t he syst em . I n part icular, t he problem of dat a and process int egrit y across cont ainer boundaries is a difficult one. Alt hough t echnologies such as X/ Open- based t ransact ion m onit ors can enable t ransact ion processing across m ult iple dist ribut ed and het erogeneous syst em s, t hese t echnologies are oft en im pract icable due t o t he m any t echnical and organizat ional lim it at ions ( see Chapt er 8, " Managing Process I nt egrit y" ) .

9.2. Logging and Auditing I n t his book, we provide t echnical and organizat ional t ools t o build a syst em t hat is not prone t o failures. Yet it is of course virt ually im possible t o rule out all failures. A lot of t he m easures described elsewhere in t his book are pot ent ially cost ly, and for t his reason, t hey are oft en om it t ed in pract ice. For exam ple, in real- world sit uat ions, t he budget s for infrast ruct ure invest m ent s t o set up a failsafe dat abase clust er or a redundant net work set up sim ply m ight not be approved. Hence, you som et im es m ust rely on m easures such as logging. Chapt er 8, " Managing Process I nt egrit y," has already given an in- dept h discussion on process int egrit y for which logging is a crucial building block. Anot her reason for operat ion disrupt ion m ight be t hat one of t he services t hat t he applicat ion depends on m ust undergo unplanned m aint enance. Consider an airline booking applicat ion t hat am ong ot hersrelies on a pricing service t o det erm ine t he price for specific flight s on a given dat e. Consider a serious bug wit hin t he m odule t hat perform s flight price calculat ion. No com pany want s som et hing like t his t o happenespecially if t he calculat ed prices are far t oo lowand t he corresponding service will oft en be shut down im m ediat ely t o prevent furt her dam age. The bot t om line is t hat even if you have planned for t he syst em t o handle a lot of error condit ions aut om at ically, unplanned and unrecoverable errors can st ill happen. We will call such errors failures in t he rem ainder of t he chapt er. As depict ed in Figure 9- 9 , failures require act ivit ies at different levels, including: user int eract ion, logging, and syst em s m anagem ent .

Figu r e 9 - 9 . An e r r or m u st be r e por t e d t o t h e u se r , t o a log file or da t a ba se , a n d t o a syst e m s m a n a ge m e n t syst e m .

When coping wit h failures, t he concept s and t echniques discussed in t his chapt er are not only relevant t o SOAs. Most of t hem are sim ply good coding pract ices or design pat t erns. However, in a dist ribut ed and loosely coupled archit ect ure, t hey are of m uch great er im port ance t han in a st andalone m onolit hic applicat ion. The dist ribut ed nat ure of t he archit ect ure and t he fact t hat t he source code or core dum ps of t he act ual services are not available for debugging m ake explicit failure- handling m easures essent ial. I n case of such failures, it is usually necessary t o perform cert ain m anual act ivit ies t o ret urn t hings t o norm al. I n lucky circum st ances, t his m ight be as easy as reset t ing a power swit ch. On t he ot her hand, it m ight result in a num ber of em ployees browsing m illions of rows in m ult iple dat abase t ables. I t is not hard t o see t hat resolving problem s is a lot easier if we know where and when t he error occurred and which users were involved in it . I t is, t herefore, m andat ory t hat every SOA has a reliable logging and audit ing infrast ruct ure. Alt hough logging and audit ing are quit e sim ilar from a t echnical point of view, t hey differ considerably at t he requirem ent s level.

Usually, runt im e out put from a syst em is m apped t o different log levels. These levels aream ong ot her t hingsused t o dist inguish audit ing, logging, t racing, and debugging out put . They are usually ident ified by a num ber of a t ext warnings, such as " DEBUG," " TRACE," " I NFO," " WARN," " ERROR," " AUDI T," et c. Audit ing is norm ally put int o place t o sat isfy som e legal requirem ent s, such as docum ent ing t hat a credit card was act ually charged because t he client ordered flight t icket s, or t o docum ent t hat t he t icket act ually did get print ed and t hat it was sent t o t he address given by t he client . Audit ing creat es a new subsyst em t hat by it self im pact s syst em operat ion. When norm al logging fails, for exam ple, if t he log file or disk part it ion is full, you can usually carry on m erely by halt ing t he logging process. However, if audit ing it self fails, it m ust be considered a failure, and t he syst em m ust st op it s operat ion. Aft er all, cont inuing t o run wit hout audit ing in place m ight violat e som e legal obligat ion, and no com pany want s t hat t o happen. Tracing is usually disabled while a syst em is in product ion and is only enabled in case of a m aj or problem because it is oft en so det ailed t hat it significant ly degrades syst em perform ance. Tracing is usually swit ched on explicit ly t o t rack down a part icular error. I n case of int erm it t ent errors, t his will of course result in a degradat ion of syst em com ponent s for a pot ent ially lengt hy int erval unt il t he error can be ident ified. Finally, debugging consist s of st at em ent s t hat are only significant t o applicat ion developers. Therefore, debugging code is oft en excluded from t he product ion code alt oget her. I n t he rest of t his chapt er, we will focus on logging and audit ing. Bot h are t reat ed in a largely sim ilar fashion.

9.2.1. ERROR REPORTING One of t he m ost com m on issues wit h error report ing is t hat failures can go unnot iced. Unfort unat ely, it is all t oo easy t o build a syst em where failures are not reliably det ect ed. A com m on m ist ake is when a program cat ches all except ions during developm ent and discards t hem silent ly. As t he proj ect m oves onusually t oward an overly opt im ist ic deadlinedevelopers m ove on t o t heir next t asks, and t he RAS ( Reliabilit y/ Availabilit y/ Serviceabilit y) feat ures are never com plet ed. Of course, cases like t his should be avoided in any kind of soft ware developm ent by em ploying proper coding st andards. I n an SOA, t hey becom e an even great er problem because of t he loosely coupled nat ure of t he applicat ion. Sim ilarly, an error t hat is logged but not report ed t o t he cust om er can cause a lot of confusion. For exam ple, if t he airline t icket print ing service is t em porarily unavailable, an error m ight not be report ed t o t he cust om er. I nst ead, it m ight be discovered and fixed during a rout ine log screening at t he end of t he week. By t hat point , t he cust om er is likely t o have called in and opened a cust om er service case, causing expenses t hat m ight ot herwise have been avoided. I t is crucial for t he business owners t o clearly define bot h t o developers and soft ware designers what t heir requirem ent s are for logging, audit ing, and report ing. Likewise, it is m andat ory t o report an error each and every t im e it occurs. When using dist ribut ed services, t his can be achieved using t he t echnologies of t he underlying plat form . For exam ple, when using SOAP, you can ut ilize t he SOAP error m echanism . Sim ilarly, if you are using a dist ribut ed obj ect t echnology such as EJB, you can use rem ot e except ions t o report an error.

9.2.2. DISTRIBUTED LOGGING Using a fram ework of pot ent ially dist ribut ed services does lit t le t o m ake logging easier. I n fact , using dist ribut ed services for logging is rarely appropriat e. Usually, t he requirem ent s for logging

are t hat it m ust be bot h reliable and light weight . Addit ionally, it should be easy t o use in order t o encourage program m ers t o log whenever t hey t hink t hat t here is som et hing wort h logging. Given t hese requirem ent s, it is ast onishing how oft en one com es across a cent ral logging service. Grant ed, t he idea t o set up logging as a service in a dist ribut ed environm ent is very t em pt ing, but it is easy t o see t hat such an approach is not light weight by t hinking in t erm s of t he net work load and lat ency involved. I f logging is im plem ent ed using obj ect - orient ed t echnologies, t he cost of m arshalling and unm arshalling log record obj ect s adds t o t his overhead. I t is also not reliable because m any t hings can go wrong when st oring t he individual log records. This st art s wit h net work failure and ends wit h st oring t he act ual log records in a file or dat abase t able. Finally, it is out of t he quest ion t o use dist ribut ed t ransact ions t o m ake ent ries in a log facilit y because t his is probably as com plex a process as one could encount er.

Log Locally but View Globally Local logging is essent ial due t o t he need for a logging facilit y t o be light weight and reliable. Global viewing of logs is required for t he analysis of errors in dist ribut ed processes.

To ensure t hat logging is bot h reliable and light weight , t he best approach is t o log locally and consolidat e t he logs as illust rat ed in Figure 9- 10. Whet her log dat a is writ t en t o a file or dat abase does not really m at t er, nor does t he form at of t he log records it self. What m at t ers, however, is t hat each and every log ent ry carries som e com m on inform at ion t o enable log consolidat ion. This inform at ion includes t he t im e- st am p, t he origin of t he log ( m et hod or procedure) , and t he nam e of t he user who m ade t he call ( if legally possible) . Addit ionally, each service call should include a unique t oken t hat can be used during log file consolidat ion t o build a dist ribut ed st ack t race.

Figu r e 9 - 1 0 . St r u ct u r e of a dist r ibu t e d loggin g e n vir on m e n t . Th e in dividu a l se r vice s w r it e t o de dica t e d logs of a r bit r a r y for m a t t h a t a r e con solida t e d t o for m a com m on st or a ge . Th e log se r vice s t h a t a r e a va ila ble a t t h e va r iou s loca t ion s sh ou ld ide a lly h a ve ide n t ica l in t e r fa ce s. An a u dit t r a il se r vice ca n be u se d t o qu e r y t h e con solida t e d st or a ge a n d r e qu e st con solida t ion fr om t h e va r iou s log se r vice s. [View full size image]

Session and Transaction Tokens A good service- orient ed logging facilit y needs t o ensure t here are t okens such as session- t okens and t ransact ion- t okens, which can be used t o consolidat e t he log files as t hey are const ruct ed and for searching aft er an except ional event .

Consider t he exam ple of purchasing an airline t icket , shown in Figure 9- 11. The airline t icket service it self logs t o an XML file. When called, it generat es a unique request I D t hat is logged wit h all ent ries t o t hat file and passed t o ot her necessary services. The billing service logs t o an RDBMS, while t he flight reservat ion uses a record- based log file. The t hree log sources are t hen all accessible using a log service for t hat part icular locat ion. Ticket ing t akes place at a different locat ion. I t logs using a line- orient ed file, again using t he unique request I D. The cont ent s of t he logs for t hat locat ion are also m ade available using a log service. I f engineered properly, t he int erfaces of t he t wo log services should be ident ical. The log services st ore t he consolidat ed log inform at ion in a cent ralized dat a st ore. This operat ion does not need t o be overly reliable because it is int rinsically idem pot ent . The com m on dat a st ore can t hen be queried using an audit t rail service.

Figu r e 9 - 1 1 . Th e a ir lin e t ick e t se r vice u se s ba sic se r vice s dist r ibu t e d ove r t w o loca t ion s. Loggin g is pe r for m e d by t h e in dividu a l se r vice s u sin g RD BM S, XM L- , Re cor d- Ba se d- , a n d lin e - or ie n t e d file s. Th e loca l log se r vice s con solida t e t h e loca l da t a on r e qu e st or pe r iodica lly in t o a com m on st or a ge . A com m on a u dit t r a il se r vice pr e se n t s a n ove r a ll pict u r e of t h e a pplica t ion .

[View full size image]

9.2.3. LOGGING AND TRANSACTION BOUNDARIES As we m ent ioned previously, logging is a light weight act ivit y, and as such, log dat a should not be writ t en t o t he sam e dat abase as t he t ransact ion being logged. The reason is obvious: I n case of an error, not only will t ransact ions get rolled back, but t he logs will also be rolled back. I t can be very hard t o det erm ine t he cause and precise circum st ances of t he failure, especially when using an environm ent t hat has sophist icat ed cont ainer- provided t ransact ion m anagem ent . For exam ple, when using an EJB cont ainer, t he user m ust be aware of t he rest rict ions im posed. I f, for som e reason, it is infeasible t o use t he logging facilit ies provided, logging t o a file usually does t he t rick. I f you need t o log t o a resource, such as a t ransact ion m essage queue or an RDBMS, it is necessary t o m anage all resources m anually.

Never Log Under the Control of a Transaction Monitor Never log under t he cont rol of a t ransact ion m onit or. Ensure t hat each com plet ed call t o a logging syst em yields an ent ry in t he appropriat e log dat a st ore. Prevent rollback m echanism s of a t ransact ion m onit or from ext inguishing log ent ries.

Logging t o a file can also show som e unwant ed behaviour. File syst em s such as Linux ext 3 or NTFS use a writ e buffer. This m eans t hat dat a is not writ t en persist ent ly unt il t he operat ing syst em synchronizes t he file syst em . This m ight require t hat t he file syst em be synchronized m anually aft er each call. I n t he case where t here are m ult iple explicit updat es using a t ransact ional resource, logging should be perform ed before and aft er each of t hese act ivit ies, as shown in Figure 9- 12 when m aking a flight reservat ion wit h t wo flight legs. This sit uat ion occurs com m only when EJB Ent it y Beans are used as t he persist ence m echanism of t he service.

Figu r e 9 - 1 2 . Tr a n sa ct ion log.

[View full size image]

Nat urally, one of t he occasions where logging is m andat ory is before calling a rem ot e service and aft er t hat call ret urns. This m akes sense not only because m aking a rem ot e call is m ore errorprone, but also because an error can oft en be fixed by running t he rem aining services in t he call chain again. Consider t he exam ple in Figure 9- 13, where t he billing and flight reservat ion succeeds but t he t icket ing call never ret urns. Aft er we det erm ine t hat no t icket was print ed, all we m ust do is rerun t he t icket ing service.

Figu r e 9 - 1 3 . Ca ll ch a in .

9.2.4. LOGGING FRAMEWORKS AND CONFIGURATION

A com m on act ivit y is t o build t he consolidat ion part of t he syst em and provide services for rem ot e ret rieval of log ent ries. However, a norm al proj ect should not be concerned wit h building t he part of t he syst em t hat writ es t he act ual log records in t he first place. Norm ally, t he execut ion and developm ent environm ent will provide som e form of logging services. Many cont ainers provide logging in t heir own right ; EJB or CORBA cont ainers oft en include logging consolidat ion facilit ies. A developm ent environm ent m ight be able t o weave logging int o t he source code based on welldefined rules. Furt herm ore, a runt im e environm ent m ight provide a high degree of logging in any case, based on aut om at ic int ercept ion of cert ain act ivit ies, such as m aking a HTTP request or obt aining a dat abase connect ion from a pool. Finally, t here are soft ware product s and libraries available for t he sole purpose of logging. The Apache log4j fram ework is probably t he m ost prom inent exam ple in t he Java program m ing language. Since t he arrival of Java 1.4, t he JDK it self cont ains sim ilar libraries for logging [ GS2003] . That being said, t here is no way t o avoid t he fact t hat logging is ult im at ely t he program m er's responsibilit y. Too m uch business logic is coded by hand inst ead of being aut om at ically generat ed. The input and out put for t his business logic usually provides t he m ost valuable insight int o t he cause of an error. Wit h t he advent of Model Driven Archit ect ure ( MDA) and highly sophist icat ed code generat ion, t his m ight be aut om at ed in t he fut ure. Of course, it is very unlikely t hat even generat ed code is com plet ely bug- free. Thus, an im port ant part of code generat ion is sensibly placed log and audit st at em ent s. Tradit ional m idrange and m ainfram e applicat ions usually provide a robust built - in logging infrast ruct ure. For exam ple, I BM's CI CS offers logging using t he CI CS log m anager. This infrast ruct ure is used for all logging purposes, from basic user- level logs up t o CI CS t ransact ion logging and recovery. This logging facilit y connect s t o t he syst em - level MVS log st ream s. One of it s m ost powerful feat ures besides it s robust ness is it s abilit y t o consolidat e log m essages from different physical and logical m achines t hat use I BM's Sysplex t echnology. Furt herm ore, logs can be aut om at ically placed int o different archive st ages so t hat accessing and searching t he m ost recent logs becom es easy, while at t he sam e t im e, t he full hist orical inform at ion is preserved. Logging should ideally be configurable in a fine- grained m anner. For exam ple, fram eworks such as log4j enable users t o define how t o handle a part icular log event based on it s origin and severit y. Usually, t his is used t o log int o different t arget files based on t he origin, t o t urn logging off for low severit y, or t o t urn logging off com plet ely for a part icular origin. This is sensible from a perform ance perspect ive, but it can have t he opposit e effect in cert ain scenarios. Assum e t hat t he flight reservat ion service has t he nast y behavior t hat j ust som et im esflight legs are not reserved properly. Fort unat ely, t he program m ers have included a high degree of logging in t heir code. Given t he dat a, it is quit e easy t o det erm ine why t his happens. The flight reservat ion service is current ly configured t o log only event s of severit y " error" and above, but inform at ion of t he level " t race" is needed t o really figure out what is happening. Unfort unat ely, t he syst em does not allow for runt im e configurat ion. This m eans t hat t he ent ire flight reservat ion service m ust be t aken offline, t he configurat ion m ust be changed, and t hen t he syst em m ust be brought back online. This can cause an unwant ed disrupt ion of service because ( a) it 's t he holiday season and ( b) it 's 5 P.M., and cust om ersa lot of cust om ersj ust want t o use t his syst em .

Runtime Configuration I f t here is a requirem ent for fine- grained log configurat ion, t ake care t o ensure t hat t he set t ings can be changed at runt im e. Ot herwise, everyt hing should be logged.

9.3. Availability and Scalability I t is m andat ory for any ent erprise archit ect ure t o provide funct ionalit y in t he way it has been com m issioned t o do so. I n t his cont ext , we consider a syst em as providing availabilit y if it is operat ional for 100% of planned upt im e. I n ot her words, we consider a syst em t o be 100% available if it prevent s any unplanned downt im e. Not e t hat we do not include planned downt im e in our concept of availabilit y. For exam ple, if an applicat ion becom es unavailable because a dat abase is t aken offline for backup every night , it does not reduce t he availabilit y of t he applicat ion. I nst ead, it lim it s t he service level t hat t he applicat ion support s. Scalabilit y in t he cont ext of t his book m eans t hat an ent erprise archit ect ure provides som e way t o increase it s capacit y beyond init ial planning. Capacit y designat es t he work a syst em can perform in a given am ount of t im e, com m only m easured in t ransact ions per second ( TPS) . Ot her m easures of capacit y are t he num ber of concurrent users a syst em can support and t he am ount of st orage space it provides. I deally, scalabilit y should provide a linear solut ion, m eaning t hat if t he capacit y of t he syst em is doubled, t he resources availablem em ory, CPUs, net work and m anagem ent overheadshould at m ost need t o be doubled. I n pract ice, syst em s can scale linearly only t o a given point . For m ost pract ical purposes, scalabilit y m eans t hat a clean and defined pat h exist s t o increase t he capacit y of a syst em by a request ed am ount . I n general, scalabilit y is only considered up t o a cert ain boundary. For exam ple, a requirem ent m ight be t hat an applicat ion m ust be scalable from an init ial load of 100 t o 10,000 users. One of t he m ost com m on confusions t hat arise surrounding issues of scalabilit y and availabilit y is t hat t hey are oft en not rigorously defined or t hat t hey are defined based on insufficient inform at ion. The so- called Service Level Agreem ent ( SLA) lies at t he heart of t his m at t er. An SLA t ypically defines a set of perform ance figures t hat are an int egral part of t he cont ract bet ween one organizat ion t hat provides an I T service and anot her t hat consum es t hat service. The m ost com m on perform ance figure is t he guarant eed operat ion t im e. The operat ion t im e ( com m only referred t o as upt im e) st at es t he t im e t hat t he syst em m ust be available, along wit h an accept able am ount of unplanned downt im e. The SLA also st at es t he capacit y of t he syst em . This includes st orage capacit y, num ber of concurrent users, and TPS. Oft en, an SLA also st at es t he response t im es t hat are accept able for a cert ain percent age of request s. Care m ust be t aken when t he requirem ent s for any I T syst em are defined. Furt her care should be t aken for any syst em t hat relies on ot her syst em s t o funct ion properly. Far t oo m any so- called " SLAs" in t he indust ry only st at e t he wishful t hinking of t he people who originally " defined" t hem during a sales pit ch. As a consequence, t hey also st at e requirem ent s t hat far exceed what t he syst em s act ually needs t o deliver in t he worst - case scenario. For exam ple, it is unaccept able t o require an airline booking syst em t o support a num ber of concurrent request s t hat is equal t o all airline seat s available for t he ent ire year. Even if t he booking syst em could fulfill t his requirem ent , t here would be no benefit for t he business. Therefore, such requirem ent s m ust be based on known values, such as t he current num ber of t ransact ions and t he act ual num ber of users t hat are concurrent ly connect ed t o t he syst em , where possible. I f such num bers are not available, ext rapolat ion from current ly known values is required. I t is bot h valid and com m onplace t o add a safet y m argin, but you m ust t ake int o considerat ion t hat t he requirem ent s for t he syst em 's availabilit y ult im at ely im pact s engineering effort s. At t he sam e t im e, it is valid t o quest ion whet her any business process needs t o operat e unaffect ed in t he cases of nuclear war or an alien invasion. Finally, you should be wary of t he concept of guarant eed response t im es for as yet unspecified and unim plem ent ed business funct ionalit y. I t is fairly easy t o plan for capacit y and availabilit y; it is alm ost im possible t o plan for an unknown com put at ional algorit hm .

By now, it should be clear t hat availabilit y and scalabilit y com e at a price. A num ber of applicat ions exist in which you should be prepared t o spend a large am ount of m oney t o guarant ee availabilit y. Consider air t raffic cont rol syst em s or a cont rol syst em for a nuclear power plant . I n t hese cases, not only t he funct ioning of t he econom y but also hum an lives depend on t hese syst em s rem aining perm anent ly operat ional. I n ot her cases, t he syst em s t hat are affect ed by your I T syst em m ight be very expensive or very hard t o replace, such as a m ult i- billion dollar spacecraft whose I T syst em s ent er unplanned downt im e j ust as it ent ers t he at m osphere of one of t he m oons of Jupit er. However, for a large num ber of syst em s, som e am ount of unplanned downt im e is inconvenient but accept able. One exam ple is an online banking syst em where m ost users m ight not even not ice a downt im e of a couple of hours every m ont h. A second exam ple is an airline check- in syst em . Alt hough downt im e is very inconvenient and large queues form in front of t he check- in desk, fallback procedures exist t hat allow for a m anual check- in process. The st ronger t he requirem ent s, t he higher t he price t ag. Building an ext rem ely fail- proof syst em t hat is scalable from init ial hardware set up t o soft ware developm ent t o operat ing procedures is very expensive. On a different not e, you m ust never forget t hat t he overall perform ance of t he syst em is ult im at ely lim it ed by t he weakest link in t he t echnology chain. This has an im port ant effect on any int egrat ion or service- enabling effort because t he im pact t hat legacy syst em s and dat abases have on overall syst em perform ance is regularly underest im at ed. I n principle, t he service layer scales linearly t o support unlim it ed capacit y, regardless of t he underlying t echnology. However, t he support ing syst em s st op scaling at som e point , such as when t he num ber of connect ions a dat abase server can support is exceeded, when t he st orage area is full, or when t ransact ions occurring at a cert ain rat e creat e concurrency problem s at t he dat abase level. Oft en, t his is not a t echnical but an adm inist rat ive problem : hardware m ust be bought , dat abase configurat ions m ust be changed, host com put ing power m ust be ordered, and so on. Because t hese t hings t ypically t ake t im e t o progress t o t he proper channels in an ent erprise's I T operat ion, it is im port ant t o st ress t est t he ent ire applicat ion st ack as soon as possible in t he proj ect t o allow enough t im e for uncovering any backend scalabilit y issues.

Stress Test Early Oft en, sim ple m easures on I T syst em level will im prove perform ance and scalabilit y radically. However, t im e is required t o im plem ent t hese changes. To buy yourself t his crucial t im e, do not be afraid of using st ress t est ing early in t he developm ent process.

Finally, not e t hat session st at e should be placed in t he applicat ion front end. I f t he applicat ion front end is not suit able, you should creat e lean process- cent ric services, which should be carefully separat ed from non- conversat ional services.

9.3.1. SCALABILITY AND AVAILABILITY USING WEB SERVICES Web services are generally built on one of t he widely available t echnologies t o deliver dynam ic Web pages, m ost not ably Microsoft .NET and J2EE. I n principle, t hese t echnologies provide easy scalabilit y. You can use a load balancer t o forward t he request s t o any num ber of fram ework inst ances or cont ainers t hat are configured ident ically. I f needed, you can add addit ional hardware t o host m ore inst ances of t he cont ainer in order t o increase syst em capacit y unt il t he net work is sat urat ed. Most off- t he- shelf load balancers provide t he concept of st icky sessions, where a

request t hat originat es from a cert ain I P address is always forwarded t o t he sam e cont ainer. This will enable t he cont ainer t o preserve conversat ional st at e wit h t he client . As we discussed in Chapt er 8, lim it ed resources such as open dat abase connect ions or t ransact ion cont ext s should never be part of t his conversat ional st at e. Newer load balancers even analyze t he request for inform at ion about t he server on which t he session relies inst ead of using only t he I P address. I n general, failure of t he cont ainer will result in t he loss of t he conversat ional st at e. St rict ly speaking, from t he preceding definit ion, t his does not lim it availabilit yt he load balancer will not ice t hat a syst em has com e down and will forward t he request t o anot her m achine. I t m ight nevert heless prevent t he unint errupt ed operat ion of t he syst em . Again, t his present s addit ional m ot ivat ion t o m ake service int erfaces st at eless and idem pot ent wherever possible. Most fram eworks support som e not ion of preserving session st at e. The act ual im plem ent at ions vary widely: st at e m ight be st ored in a dat abase or in a shared file syst em , or it m ight be replicat ed t o all m em bers of a clust er using m ult icast or t o a single dedicat ed server. Sim ilarly, session st at e replicat ion m ight or m ight not be t ransact ional. I n any case, preserving session st at e creat es a not iceable overhead. For exam ple, st at e t hat is convenient t o st ore but t hat is easily re- creat ed should not be replicat ed. You m ight even consider t he com plet e abandonm ent of conversat ional session st at e, even if t his requires it s re- creat ion wit h every service invocat ion. The sim plicit y and scalabilit y obt ained can easily out weigh t he ext ra cost s for hardware. However, t he best solut ion is t o st ore t he session st at e in t he applicat ion front end. This m akes m ost of t he challenges of m aint aining server- side st at e irrelevant . As an exam ple, consider checking in for a flight using a self- service t erm inal. Consider t he st age of t he check- in service where t he seat is assigned ( see Figure 9- 14) . The call signat ure can easily be craft ed t o be bot h idem pot ent and st at eless. I n t his case, no session replicat ion is required. I f t he first t ry t o assign a seat fails because t he syst em has gone down, t he t erm inal ret ries t he call. The load balancer has not iced t hat it has lost connect ion t o t he first inst ance of t he service and uses anot her inst ance t o ret ry t he call. This call subsequent ly ret urns t he required inform at ion t o print t he boarding card.

Figu r e 9 - 1 4 . Fa ilove r u sin g a h a r dw a r e loa d ba la n ce r a n d a se r vice t h a t u se s n o con ve r sa t ion a l st a t e .

Interoperability with Off-the-Shelf Load Balancers Always aim for a coarse- grained service int erface. Use idem pot ent m et hods wherever possible. These m easures will provide opt im al int eroperabilit y wit h off- t he- shelf loadbalancing rout ers.

9.3.2. SCALABILITY AND AVAILABILITY USING EJBS Creat ing scalable applicat ions using EJBs is fairly sim ple because m ost EJB servers support clust ering. This enables an adm inist rat or t o add addit ional server inst ances and hardware unt il t he capacit y requirem ent is reached. Usually, t he client s' EJB hom e int erface im plem ent at ion m akes a decision based on cert ain algorit hm s as t o where t o creat e t he required EJB. I n case of a st at eless session bean, t he rem ot e st ub it self m ight load balance on a call- by- call basis. Typical algorit hm s include random , load- based, and round robin. Som e EJB cont ainers also facilit at e t he provision of cust om algorit hm s. Most EJB cont ainers will bind a specified rem ot e client t o a single server inst ance t o lim it t he am ount of t ransact ion coordinat ion needed if m ult iple EJBs t ake part in t he sam e t ransact ion. I f t hey do not , you can easily em ulat e t his behavior by using a façade pat t ern t o push down t he t ransact ion boundary. Regarding availabilit y, t he sam e concept s discussed in t he previous sect ion on Web services hold t rue. St at eless session bean st ubs will det ect t he failure of a server and t ry t o connect t o a different server. St at eless session beans should be used wherever possible t o avoid conversat ional st at e. I f conversat ional st at e is a firm requirem ent , som e EJB cont ainers provide dat a replicat ion and failover for st at eful session beans. Likewise, ent it y beans t hat are not part of a t ransact ion can be reloaded from a different server. Because t he lat t er is quit e specialized behavior, and because it is also highly vendor- specific, it is again best t o t ry t o m aint ain as lit t le conversat ional st at e as possible. Som e EJB cont ainers also support t he idea of an idem pot ent m et hod in a bean. This is a powerful concept because it enables t he st ub t o t ransparent ly recreat e a call if t he original call fails in flight . These feat ures of EJB cont ainers oft en lead t o perform ance problem s in a t ight ly coupled environm ent wit h a fine- grained obj ect m odel. However, t hey are very useful in t he face of t he scalabilit y and availabilit y issues of an SOA.

Avoid Stateful Beans' Fine-Grained Interaction Patterns Using st at eless session beans wit h coarse- grained int erfaces will enable you t o m ake effect ive use of your applicat ion server clust er. Avoid fine- grained obj ect int eract ion pat t erns. The st riking sim plicit y of t his concept will im prove not only t he availabilit y and scalabilit y of t he syst em but also it s robust ness and m aint enance- friendliness.

As an exam ple, consider booking an airline t icket , as illust rat ed in Figure 9- 15. An exam inat ion of t hat part of t he booking process cont ains t he following t hree st eps:

1 . A reservat ion is ent ered int o t he syst em . 2 . The cust om er's credit card is charged. 3 . The reservat ion is m arked paid and closed.

Figu r e 9 - 1 5 . A st a t e fu l book in g se r vice t h a t ca lls t h e r e se r va t ion a n d t h e pa y- m e n t se r vice . Th e r e se r va t ion n u m be r , cr e dit ca r d da t a , a n d st a t e of pa ym e n t a r e r e plica t e d in or de r t o su ppor t fa ilove r .

I f t he client not ices t hat t he call fails, it can ret ry t he call using a replicat ed inst ance of t he booking service. The key dat a t o be replicat ed is t he prim ary key of t he reservat ion record and t he credit card dat a, along wit h t he st at e of t he paym ent . I f a reservat ion key exist s and t he st at e of paym ent is " charged," t hen t he im plem ent at ion of t he booking service can cont inue wit h closing t he reservat ion. I f t here is no prim ary key for t he reservat ion, it can ret ry t he whole process. I f t he st at e of paym ent is " not at t em pt ed" and a reservat ion prim ary key exist s, it cont inues charging t he credit card. Not e t hat due t o t he t ransact ion boundaries, a st at e m ight arise t hat cannot be handled aut om at icallyif t he paym ent fails in flight and t he paym ent st at e in t he replicat ed dat a is at t em pt ed. I n t his case, t he im plem ent at ion can run checks using service m et hods of t he paym ent service t o det erm ine t he out com e of t he at t em pt ed call.

9.3.3. SCALABILITY AND AVAILABILITY USING CORBA Wit h CORBA, m ost of t he concept s discussed in t he previous sect ion hold t rue. The CORBA specificat ion defines requirem ent s for " fault - t olerant CORBA." Am ong ot her t hings, t his specificat ion is concerned wit h det ect ing com m unicat ion fault s, providing t ransparent failover, and replicat ing t he obj ect st at e of dist ribut ed obj ect inst ances. Load balancing wit hin CORBA is usually perform ed by t he individual vendor im plem ent at ions using low- level funct ions of t he CORBA GI OP ( General I nt er- ORB Prot ocol) . Because CORBA fault t olerance is a relat ively new specificat ion, various vendor- specific im plem ent at ions for availabilit y and obj ect st at e replicat ion exist .

9.3.4. SCALABILITY AND AVAILABILITY USING CICS Even t hough it m ight seem old- fashioned, I BM's Cust om er I nform at ion Cont rol Syst em ( CI CS) rem ains widespread for m ission- crit ical services. Basically, CI CS is a t ransact ion server t hat provides very efficient facilit ies for loading, init ializing, and execut ing program s. While CI CS servers usually run on m ainfram e or m idrange com put ers, a variet y of t ools is available t o connect t o a CI CS server. For exam ple, t he CI CS Transact ion Gat eway ( CTG) enables Java program s t o act as a CI CS client using t he Java Connect or Archit ect ure ( JCA) . Ot her program m ing languages, such as COBOL, C+ + , and Visual Basic, can use t he CI CS universal client t o access t he CI CS server. Connect ivit y t o I BM's m essaging product , MQSeries, is also available. I n t his way, any client of an MQSeries server can m ake use of exist ing CI CS program s. Finally, CI CS program s can be direct ly exposed as SOAP Web services using SOAP for CI CS. To ensure a seam less operat ion wit h exist ing CI CS program s, any CI CS- based service should ensure t hat no conversat ional st at e is held. One t ransact ion should m ap t o exact ly one service call. CI CS program s can run for an arbit rarily long t im e int erval. I f t he service im plem ent at ion breaks while t he CI CS call is in flight , it can be very hard t o det erm ine t he result of t he t ransact ion. Therefore, ext ra care m ust be t aken t o ensure t hat CI CS program s used in services are sm all and t hat t hey execut e quickly. Scalabilit y and availabilit y of CI CS it self is provided by I BM's CI CSPlex and SYSPlex t echnologies. CI CS can even part icipat e in dist ribut ed t ransact ions, for exam ple using JCA. However, as discussed in Chapt er 8, dist ribut ed t ransact ions m ust be handled wit h care. Because CI CS program s usually execut e on syst em s wit h very high load and t ransact ion densit y, using dist ribut ed t ransact ions wit h CI CS is oft en inefficient and m ight have a significant im pact on ot her CI CS j obs accessing shared resources.

9.3.5. SCALABILITY AND AVAILABILITY OF WRAPPED LEGACY APPLICATIONS I t is oft en desirable t o use an exist ing t erm inal- based legacy applicat ion wit hin a service. Just as oft en, it is not possible t o m ake changes t o t he exist ing applicat ion. I n t hese cases, one of t he m ost popular and effect ive ways of using legacy applicat ions in newly developed applicat ions is screen scraping. The charact er st ream t hat is used t o build t he t erm inal user screen, such as on a VT100 t erm inal, is analyzed. Effect ively, t he applicat ion m im ics a VT100 t erm inal client . This approach has several benefit s. I t is a fairly cheap and st raight forward way t o include som e of t he exist ing funct ionalit y int o a service. Specifically, it is a low- im pact solut ion because no

changes t o t he original applicat ion are required. There are also downsides t o t his approach, m ainly in relat ion t o scalabilit y and availabilit y. First , a lot of exist ing VT100 syst em s will have daily m aint enance windowsusually at night when t he applicat ion and t hus t he service are not available. This m ust be incorporat ed int o t he relevant SLAs for t he service. Second, any changes t o t he applicat ion t erm inal screen require m aint enance work in t he service applicat ion. Furt herm ore, granularit y of a t ypical st at eless service call can easily span m ult iple t erm inal screens. This will lead t o rat her high lat ency when invoking such a service. On t he ot her hand, a design t hat perform s m ult iple fine- grained service calls requires a st at eful service, which is generally not advisable, part icularly because such syst em s norm ally work on a pooled resource, in t his case, t erm inal connect ions. Furt herm ore, t he service can only scale t o t he am ount of t ransact ions and sessions t hat are support ed by t he legacy applicat ion. A business t hat aim s for t his t ype of reuse m ust be prepared t o face t he consequences of t his decision. I t will usually save a large am ount of m oney, but t his com es at t he cost of som ewhat lim it ed scalabilit y and robust ness wit h increased m aint enance [ TDM2000] .

9.3.6. SCALABILITY AND AVAILABILITY IN A HETEROGENEOUS SOA Ult im at ely, services in m ost SOAs t hat run on different plat form s are likely t o be int egrat ed. For exam ple, t he flight booking EJB m ight call a SOAP Web service t o charge t he cust om er's credit card. Because m any SOA init iat ives st art out providing a whole new ent erprise I T infrast ruct ure, it is easy t o lose sight of t his fact . However, if properly deployed, an SOA support s t he concept of het erogeneous service plat form s working t oget her. The individual services are only loosely coupled and share no com m on t ransact ion cont ext s. Therefore, if all individual services are designed t o be available and scalable, t he overall syst em will be available and scalable in principle. However, t he sit uat ion is not quit e so sim ple. For exam ple, increasing scalabilit y on t he t op of t he service st ack m ight not increase overall scalabilit y of t he syst em because services down t he applicat ion st ack m ight be overloaded. Thus, changing an SLA for a front end service usually requires changing SLAs for backend services as well. As an analogy, im agine a call cent er t hat relies on a single dat abase t o operat e at it s m axim um user load. Sim ply adding m ore agent seat s and phone lines t o t he call cent er does not increase it s capacit y unless t he dat abase syst em is also upgraded. The upt im e of services t hat run wit hin a het erogeneous infrast ruct ure does not am ount t o t he lowest upt im e of t he service chain. I f t hree services provide an upt im e of 98% , t he result ing upt im e is t he product of t he individual upt im es: UT = 98% x 98% x 98% = 94.1% However, t he weakest link in t he service chain st ill has t he highest im pact on availabilit y in addit ion t o scalabilit y. Therefore, proj ect m anagem ent m ust first focus on bringing t he weakest t echnology in line wit h t he SLA's requirem ent s.

9.4. Securing SOAs I n SOAs, securit y is cert ainly of prim ary concern and should have solid foundat ions in t he infrast ruct ure. Before you t ake any t echnical st eps t o im plem ent a specific securit y st ruct ure, you should capt ure t he securit y requirem ent s for all relevant pieces of soft ware during a risk analysis. Based on t his analysis, you build t he soft ware archit ect ure t o incorporat e t he defined securit y requirem ent s. Perform ing a risk analysis is beyond t he scope of t his book, but det ails on t he subj ect can be found in [ PT2001] . I nst ead, based on a num ber of im plem ent at ion proj ect s carried out by t he aut hors, we discuss here som e t echnical issues t hat you m ust address when designing a securit y infrast ruct ure in a dist ribut ed soft ware archit ect ure. The m ain issues of a basic securit y infrast ruct ure are aut hent icat ion, aut horizat ion, and confident ialit y. The focus broadens where t he infrast ruct ure is exposed on t he out side of som e well- cont rolled environm ent , such as a specific depart m ent or geographic locat ion. I n t hese sit uat ions, concerns such as non- repudiat ion, m essage int egrit y, and legally binding ident it y assert ion becom e increasingly im port ant [ GS1997] . I n t his sect ion, we discuss t hese principles and provide exam ples of how t hey relat e t o plat form s t hat are com m only used t o creat e SOAs. The guiding principle is t hat we want t o use securit y as a t ool t o fost er and encourage t he use of our ent erprise archit ect ure, rat her t han securit y sim ply being a m eans t o prevent incident s.

9.4.1. AUTHENTICATION Aut hent icat ion m eans t hat a service caller is using a m echanism t o prove it s ident it y t o t he called resource. The caller m ust provide credent ials such as a usernam e and password or a digit al cert ificat e. Oft en, securit y requirem ent s for aut hent icat ion include an im plicit aut horizat ion requirem ent . The t ypical scenario is t hat callers m ust aut hent icat e t hem selves before being allowed t o use t he environm ent . Three levels of aut hent icat ion can be readily dist inguished. First , it is possible for t he caller t o aut hent icat e against t he applicat ion front end, second, aut hent icat ion against t he SOA fram ework is possible, and finally, aut hent icat ion against a single service is also a possible opt ion. Figure 916 illust rat es aut hent icat ion at different levels of t he applicat ion.

Figu r e 9 - 1 6 . Au t h e n t ica t ion a ga in st diffe r e n t le ve ls of t h e in fr a st r u ct u r e . I n t h is ca se , t h e W e b sit e su ppor t s a u t h e n t ica t ion . Au t h e n t ica t ion a ga in st t h e in fr a st r u ct u r e is a lso su ppor t e d. I n dividu a l se r vice s, su ch a s t h e cu st om e r se r vice , m igh t a lso r e qu ir e or su ppor t a u t h e n t ica t ion .

[View full size image]

Aut hent icat ion against t he applicat ion front end is generally st raight forward and quickly deployed. Oft en, it will be a vit al business requirem ent t o prot ect access t o t he applicat ion it self. Aut hent icat ion against individual services is oft en desirable in order t o lim it access t o t hese services and t o creat e a clean audit pat h wit hin service invocat ions. However, when aut hent icat ion becom es part of each individual service, a significant am ount of code clut t ering result s. I f t he aut hent icat ion m echanism and st orage change, a significant am ount of reengineering m ight be necessary t o propagat e t hese changes int o t he various services. I n addit ion, different services probably rely on different t ypes of st orage and aut hent icat ion m echanism s in t he first place. Therefore, it is generally bet t er t o insist t hat t he user be aut hent icat ed against t he infrast ruct ure rat her t han against individual services. I t is a prerequisit e for SOA- wide aut horizat ion and ident it y assert ion, as we discussed in t he following. However, som e very im port ant advant ages are available. For exam ple, having a com m on user for subsequent operat ions in an SOA m akes m onit oring t he environm ent a lot easier. Com m on t ypes of at t ackssuch as password guessing or denial of service at t ackscan be prevent ed or at least t reat ed in a st andard way t hroughout t he SOA. I n addit ion, m aint enance benefit s great ly because changes in t he underlying aut hent icat ion m echanism can be perform ed across t he board rat her t han against individual applicat ion front ends or services.

Authenticate Against the SOA Aut hent icat ion against t he SOA facilit at es cleaner and easier- t o- m aint ain code wit h a com parably sm all overhead.

Aut hent icat ion against t he SOA is usually based upon t he fram ework t hat is used t o build t he SOA infrast ruct ure or a com m on fram ework on t op of t he basic infrast ruct urefor exam ple, a J2EEcom pliant server t hat can be ext ended using a pluggable aut hent icat ion provider. An aut hent icat ion fram ework usually consist s of at least an aut hent icat or and a cont ext . Technically, a cont ext is oft en associat ed wit h conversat ional st at e in a session ( see Chapt er 8) . The aut hent icat or is a soft ware com ponent t hat checks a user's credent ials, such as a usernam e and password or a digit al cert ificat e. The infrast ruct ure t hen creat es a cont ext t hat is accessible

from t he business dom ain in order t o obt ain t he aut hent icat ed user, com m only known as t he principal. The predom inant m iddleware fram eworks all support aut hent icat ion m echanism s t hat can be leveraged in an SOA and t hat can be readily ext ended t o sat isfy individual ent erprise needs. For exam ple, a com pany m ight creat e an aut hent icat or wit hin a newly creat ed SOA t hat uses usernam es and passwords from an exist ing m ainfram e applicat ion. The J2EE environm ent provides JAAS ( Java aut hent icat ion and aut horizat ion service) , a m echanism t hat describes t he process of aut hent icat ion and aut horizat ion of client int eract ions. One drawback of such specificat ions is t hat t hey t end t o be at least part ly propriet ary. Cont ext s t hat originat e from different fram eworks are usually incom pat ible. Even when t hey support a com m on client API such as JAAS, t heir im plem ent at ions can differ significant ly, and t hey m ight support different and incom pat ible ext ension m echanism s. Such syst em s share t he fact t hat t he act ual aut hent icat ion process is fully t ransparent t o t he individual service. A service m ight decide t o check t he current principal and t hus rej ect any operat ion, but it will not perform t he act ual login process. This isolat es t he service im plem ent at ion from changes in t he aut hent icat ion m echanism . For exam ple, it is possible t o change aut hent icat ion credent ials from usernam e/ password t o digit al cert ificat es wit hout t he need t o change t he service im plem ent at ion. I n short , t he service needs only t o know t he cont ext , not t he aut hent icat or. I f aut hent icat ion against t he SOA it self is not pract icalperhaps because individual services are engineered in a way t hat requires individual aut hent icat ion, possibly using different credent ialssingle sign- on fram eworks can be used t o relax t he problem s of int egrat ion. Typical single sign- on fram eworkssuch as Net egrit y Sit em inder, Oblix Net Point , and Cit rix Met afram eplug int o t he aut hent icat ion hooks of t he various soft ware com ponent s used, including m iddleware, Web servers, and m ainfram e applicat ions. They provide a consist ent view of an aut hent icat ed principal wit hin all t hese environm ent s. The individual syst em s m ight on t heir ownrely on different aut hent icat ion m echanism s t hat have t heir own st orage m echanism s and different credent ials. A single sign- on fram ework can provide credent ial m apping, providing t he correct logon credent ials for each invoked service or even backend applicat ions. Single sign- on is generally st raight forward and t ransparent t o im plem ent for program m ers but needs a significant effort t o int roduce and m aint ain, especially when credent ial m apping is involved. The general principle of a single sign- on fram ework is illust rat ed in Figure 9- 17. Not e t hat m ost single sign- on product s are not lim it ed t o sim ple aut hent icat ion and credent ial m apping; t hey usually offer m echanism s and st orage for aut horizat ion, t oo. Som e fram eworks even provide hooks t o include biom et ric equipm ent such as ret ina scanning or face recognit ion.

Figu r e 9 - 1 7 . Pr in cipa l of a sin gle sign - on in fr a st r u ct u r e . Th e u se r is or igin a lly a u t h e n t ica t e d a ga in st a sin gle a pplica t ion t h a t in t e r fa ce s t o t h e sin gle sign - on fr a m e w or k . Th e fr a m e w or k pr ovide s login con t e x t s a n d t ok e n s for t h e ot h e r a pplica t ion s a n d t h e a pplica t ion fr on t e n d.

[View full size image]

9.4.1.1 Authentication and Middleware Aut hent icat ion is fairly st raight forward t o im plem ent when st andard m iddleware such as J2EE, CORBA, or .NET is used as t he basic runt im e foundat ion. Most of t hese fram eworks support som e not ion of aut hent icat ion and caller ident it y. Of course, t he process of passing credent ials m ight not be very secure. For exam ple, it is com m on pract ice t o use unencrypt ed passwords when logging in t o a rem ot e EJB cont ainer. Therefore, m ost dist ribut ed environm ent s use t he not ion of a login session. Once logged in, a session I D is passed bet ween t he part icipant s. A frequent ly used t echnique for session creat ion is t he serverside session m anagem ent in Web applicat ions where HTTP cookies or URL rewrit ing is used t o m aint ain a session. Sim ilar t echniques apply t o CORBA and t o aut hent icat ion when obt aining a J2EE nam ing cont ext . There is also t he not ion of a login cont ext in t he Java Aut hent icat ion and Aut horizat ion Service ( JAAS) , now a st andard in t he J2EE applicat ion environm ent . This login cont ext provides callbacks t hat enable t he called runt im e environm ent t o query t he caller ident it y using t he callback:

... import javax.security.auth.login.LoginContext; ... LoginContext loginContext = null; try { // Create LoginContext with username, password and // login url loginContext = new LoginContext("AirBookers", new AirBookersCallbackHandler(username, password, url)); loginContext().login(); } catch (Exception exc) { }

The callers t hen pass t heir ident it y on each subsequent invocat ion using an im plem ent at ion of t he PrivilegedAction int erface, in t his case FlightBookingAction:

Subject subject = loginContext.getSubject(); FlightBookingAction flightAction = new FlightBooking-Action(url); Security.runAs(subject, flightAction); System.exit(0);

Aft er t he user has com plet ed t he int eract ion, t he login cont ext can be discarded by calling loginContext.logout(). Not e t hat t he client int erface for JAAS aut hent icat ion is rat her com plicat ed t o handle. However, t he act ual service im plem ent at ion is usually not concerned wit h t he det ails of t he aut hent icat ion m echanism . The im plem ent at ion does not direct ly call any of t he available callbacks. I nst ead, t he fram ework provides t he current caller wit hin a cont ext obj ect , for exam ple a SessionContext obj ect in case of st at eless and st at eful session EJBs. The caller principal can t hen be obt ained from t he session obj ect :

... Principal principal = ctx.getCallerPrincipal(); System.out.println("Called by "+principal.getName()); ...

More sophist icat ed fram eworks prot ect t he login session by varying t he session I D during t he session by issuing new session I Ds or t ransact ion num bers for every service invocat ion. Creat ing aut hent icat ion as part of an SOA is easy because of it s lim it ed im pact on t he underlying applicat ions. Rem em ber t hat services are oft en creat ed from exist ing applicat ions. Service enablem ent of such applicat ions includes obt aining t he caller principal from t he service infrast ruct ure. Even if t he applicat ion does not support anyt hing sim ilar t o a pluggable aut hent icat ion m odule ( PAM) , it is fairly easy t o creat e som et hing sim ilar in alm ost every applicat ion. For exam ple, if t he legacy applicat ion has an exist ing dat a st ore of users t hat it has a st rict requirem ent t o use, you can always creat e shadow users for every principal in t he infrast ruct ure securit y st ore and m ap t hem t o t he exist ing users of t he legacy applicat ion. Som et im es, you m ight see t he concept of aut hent icat ion as a service. Several fram eworks are available t hat provide ident it y services t hat require a cert ain am ount of infrast ruct ure t o be available. Alt hough t hey are very powerful, t here is also som e resist ance t o adopt ion of t hese fram eworks. Deploying such syst em s in any large organizat ion is generally a cost ly exercise t hat m ust be based on a st rat egic decision m ade at t he ent erprise level. Even t hen, individual proj ect t eam s m ight show a cert ain resist ance t o using t hem because t hey can creat e addit ional risks in proj ect s in t erm s of soft ware developm ent skills or adherence t o t heir SLAs. I n t he end, using t hese fram eworks can help an SOA effort as well as kill it . Wit hin t he ent erprise, t he best opt ion is t o st andardize on a single st orage facilit y t o access ent erprise- wide user dat a. Direct ory services such as LDAP servers provide a very efficient m eans of st oring and looking up such dat a. When designing an aut hent icat ion archit ect ure from scrat ch, you should t ake care t o m ake it " pluggable." This m akes it possible t o change t he im plem ent at ion as well as t he underlying concept of an aut hent icat ion m echanism , such as changing user st ores or swit ching from usernam e/ password aut hent icat ion t o cert ificat e- based aut hent icat ion.

9.4.1.2 Authentication and SOAP Web services using SOAP are one of t he m ost popular ways of building a Service- Orient ed Archit ect ure. Alt hough various t echniques exist t o secure a Web service environm ent [ H2003] ,

SOAP provides only lim it ed st andardized support t o sat isfy t he m ost basic aut hent icat ion needs. SOAP 1.2 does not include a st andard m eans of passing credent ials, nor does it support t he concept of a caller cont ext . Alt hough bot h can be built easily using sim ple SOAP calls and t he SOAP header, t his am ount s t o a propriet ary approach. Any bindings int o t he applicat ion t o access t he cont ext and caller principal m ust be handcraft ed. There are specificat ions t hat offer solut ions, m ost not ably t he WS- I Basic Securit y Profile and SAML, a fram ework for exchanging aut horizat ion and aut hent icat ion inform at ion. Bot h define a m eans for aut hent icat ion at t he prot ocol level, but unfort unat ely, t hat is all t hey do. No st able st andard bindings are available int o com m on program m ing languages such as Java or C+ + t hat can be leveraged in a proj ect . Hand coding support t hat st rict ly adheres t o t he specificat ions is by no m eans t rivial. I t is not so m uch a problem for service im plem ent at ion but for t he client applicat ions. For applicat ion developers, an SOA using SOAP t hat does not adhere t o som e st andard is likely t o discredit t he SOA init iat ive in t he first place. For t he m om ent , t he best com prom ise m ight well be t o use only a sm all and clearly const rained subset of t he specificat ions t hat eit her are available or t hat can be easily provided for t he client environm ent s of t he SOA. Unfort unat ely, at t he t im e of writ ing, t his process can result in a som ewhat weak securit y m odel. The WS- I Securit y Profile of t he Web Services I nt eroperabilit y Organizat ion is likely t o provide a m ore consist ent and t horough t oolset for securing Web services in t he fut ure. I t willam ong ot her t hingssupport user aut hent icat ion using various t echnologies. The Securit y Assert ion Markup Language ( SAML) developed by OASI S can be used t o include aut hent icat ion and aut horizat ion- relat ed st at em ent s in addit ion t o signed m essages int o a SOAP docum ent . I n SAML, securit y st at em ent s are packaged int o assert ions t hat are issued by assert ion providers. There are t hree basic t ypes of assert ions: aut hent icat ion, at t ribut es, and aut horizat ion. Sim ilarly, t here are t hree m at ching assert ion providers, alt hough a single ent it y m ight well act as assert ion provider for all t hree t ypes. An applicat ion can request a securit y assert ion from an assert ion provider, such as using usernam e and password. The assert ion provider will ret urn an ( opt ionally signed) assert ion t hat always cont ains a t im est am p, an assert ion I D, and t he subj ect of t he assert ion, t ypically t he applicat ion user. I t can cont ain condit ional inform at ion; for exam ple, about t he lengt h of t im e t he assert ion rem ains valid. The assert ion or t he assert ion I D can be included wit h ot her SOAP m essages and can be aut om at ically verified by t he called applicat ion. Single sign- on fram eworks are obviously prim e candidat es t o act as assert ion providers. The following is an exam ple of an aut hent icat ion assert ion:



"www.dolphinair.com" "cn=Dirk,co=travel,ou=check-in"



9.4.2. AUTHORIZATION Aut horizat ion is t he m echanism used t o grant a caller access t o a specific resource. I n t he sim plest case, t he caller m ight have t he right t o generally use a cert ain service, but m ore oft en, t he aut horizat ion decision is m uch m ore dynam ic. For exam ple, a user m ay alt er only cert ain dat a set s t hat belong t o t hat part icular user. Consider a flight planning exam ple, where cockpit crews m ay change t heir own flight plans but not t he flight plans of ot her flight s. I n aut horizat ion, no com m on abst ract ion principle lies at t he core of t he aut horizat ion process, ot her t han allowing or denying cert ain act ions. As wit h aut hent icat ion, aut horizat ion can be perform ed at several levels, such as t he applicat ion front ends, t he individual services, and t he SOA infrast ruct ure. Aut horizat ion wit hin t he applicat ion front ends is one possibilit y. However, if used on it s own, t his creat es a severe securit y flaw because t he underlying services would not offer any aut horizat ion prot ect ion at all. The ease of im plem ent at ion com es at t he high price of reduced securit y, as well as inconsist ent aut horizat ion policies for t he sam e business act ivit y due t o t he im plem ent ing applicat ion. Mirroring aut horizat ion decisions at t he applicat ion level m ight appear sensible in order t o reduce round t rips t o a rem ot e server, easing t he load on t he rem ot e server and t he net work while at sam e t im e providing a bet t er user experience. Considerat ion for t he m aint enance of t he syst em is an issue during t he design phase. As wit h aut hent icat ion, t he level for aut horizat ion m ight not necessarily be t he SOA infrast ruct ure level. I nst ead, bot h t he st orage of t he aut horizat ion dat a and t he level of dynam ism in det erm ining caller perm issions det erm ine t he preferred locat ion for an aut horizat ion decision in an SOA. The param et ers used t o det erm ine t he access decision can be com plicat ed, and several approaches can be ident ified. They m ainly differ in t he way in which t he aut horizat ion relevant inform at ion is st ored and t he way t he aut horizat ion decision is det erm ined: St or e a cce ss da t a w it h t h e a u t h or ize d r e sou r ce . This is a m odular approach where t he act ual dat a t hat det erm ines t he aut horizat ion decision is st ored wit h t he resource it self. As a com m on exam ple, consider t he file access right s in t he Unix file syst em . No cent ral regist ry st ores t he read, writ e, and execut e right s in addit ion t o t he owner and group inform at ion for a file. This dat a is st ored wit h t he file it self. The benefit of such an organizat ion is t hat t he aut horizat ion decision is very effect ive from a perform ance viewpoint . An obvious disadvant age is t hat t he int roduct ion of a new aut horizat ion policy can be a rat her com plicat ed exercise. St or e a cce ss da t a ce n t r a lly. This approach is for t he exam ple used in t he access cont rol syst em of t he Windows NT operat ing syst em fam ily. Oft en, aut horizat ion inform at ion is st ored and accessed by m eans of access cont rol list s ( ACLs) , whereby a resource is ident ified by a cert ain pat h. Each resource feat ures cert ain capabilit ies, such as read, writ e, delet e, or execut e. Access t o t hese resources can be eit her grant ed or denied. Bot h of t hese m echanism s basically define a st at ic binding bet ween a resource and eit her a user or a m apping of users int o a group or role. I n a real- world scenario, t his is rarely sufficient . The following t ypes of access decisions can be readily dist inguished: St a t ic a u t h or iza t ion de cision ba se d on t h e cu r r e n t u se r or a st a t ic gr ou p m e m be r sh ip. This is easily det erm ined by a resource ext ernal t o t he act ual business process providing access t o t he aut horizat ion dat a is guarant eed.

D yn a m ic a u t h or iza t ion de cision ba se d on a dyn a m ic gr ou p m e m be r sh ip ( r ole ) . This m ight be based on any fact or ext ernal t o t he applicat ion but accessible for t he aut horizat ion fram ework. Popular exam ples are decisions t hat grant access during a cert ain period of t he day or based on t he value of session variables t hat are only t em porarily available. D yn a m ic a u t h or iza t ion ba se d on a n a t t r ibu t e of t h e a u t h or ize d r e sou r ce . This is t he t ype of decision t hat is oft en at t he core of business processes. As an exam ple, consider t he aut horizat ion of a credit card t hat is based on t he card num ber, expirat ion dat e, and cust om er address. Anot her exam ple is t he aut horizat ion of a banking t ransact ion t hat is based on t he user ent ering a valid t ransact ion num ber from a list . Oft en, business requires t he audit ing of aut horizat ion and aut hent icat ion processes. This part icularly includes bot h form s of dynam ic aut horizat ion decisions. A fair am ount of aut horizat ion can be perform ed using st andard t echnologies, such as t he aut horizat ion m echanism in fram eworks such as .NET or t he Java Aut horizat ion and Aut hent icat ion Fram ework ( JAAS) . Language bindings are also available t hat enable aut horizat ion at t he m essage level of SOAP, even t hough t hese are, at t he t im e of writ ing, m ost ly propriet ary solut ions. CORBA enables declarat ive aut horizat ion using t he CORBA securit y service. Aut horizat ion can in principle be perform ed by an ext ernal service. Technologies such as SAML and t he upcom ing WS- I securit y support it out of t he box. Alt hough fram eworks and API s are available t o support aut horizat ion using declarat ive m echanism s or even rule- based decision fram eworks, you m ust bear in m ind t hat SOA is fundam ent ally about t he int egrat ion of exist ing applicat ions. Applicat ions t hat are t o be int egrat ed m ight very well use aut horizat ions m echanism s t hat cannot be m apped easily ont o a com m on syst em . This is because at least som e part of t he aut horizat ion procedure will generally be part of t he service im plem ent at ion, inst ead of residing on t he service infrast ruct ure level. For exam ple, t he aut horizat ion schem e wit hin a cert ain service im plem ent at ion m ight not easily m ap t o t he one in anot her service im plem ent at ion. Oft en, t he aut horizat ion decision is buried deep wit hin t he service im plem ent at ion. At t he sam e t im e, it is oft en at t he heart of t he business process. Requiring ext ernalizat ion of t he aut horizat ion procedure in t he business process also creat es a risk for service enablem ent in int egrat ion proj ect s and poses an ent ry barrier for applicat ions t o j oin an int egrat ion init iat ive. All t his m akes it rat her unusual t o perform aut horizat ion com plet ely wit hin t he service fram ework it self because it is genuinely hard t o m ove aut horizat ion out of t echnology. However, a refact oring of code during service enablem ent t o m ake specific part s of aut horizat ion pluggable can provide a good com prom ise wit h lim it ed im pact and high flexibilit y. This can include decisions t hat depend on t he principal's st at ic or dynam ic group m em bership. I t can also include rule- based decisions, where t he param et ers for t hese decisions are available at infrast ruct ure level. Lat er in t he service lifecycle, securit y fram eworks can provide or replace t he aut horizat ion im plem ent at ion well wit hin t he lifecycle of t he applicat ion.

9.4.3. ENCRYPTION AND T RANSPORT SECURITY When t alking t o people who cover securit y for business client s, one is oft en confront ed wit h t he requirem ent of encrypt ion. The general im pression is t hat if som et hing is " encrypt ed," it is safe, as long as decrypt ion cannot be perform ed easily and swift ly. To underst and t he pit falls of t his perspect ive, it is necessary t o dist inguish bet ween t wo different form s of encrypt ion: encrypt ion at t he m essage level and encrypt ion at t he t ransport level. The form er can be em ployed t o creat e real end- t o- end securit y and is generally t he preferable m et hod. Furt herm ore, because encrypt ion can be perform ed at t he m essage level, it is possible t o encrypt only t he payload or payload

segm ent s of t he m essage. I f such a m essage uses t he SOAP prot ocol and t he W3C's recom m endat ion for XML encrypt ion synt ax and processing, a sect ion of t he m essage using encrypt ion m ight look like t his:

John Smith ae12345-fght

A23B45C56



The benefit of t his approach is t hat it not only provides superior securit y but t hat cert ain rules can st ill be applied t o t he ot herwise encrypt ed m essages. This m akes it possible t o use m essagebased rout ing, t o perform advanced audit checks on t he unencrypt ed part s of t he m essages, and t o perform analysis t o det ect denial of service at t acks, at t em pt s for m essage replay, or sim ilar at t acks t hat m ight ot herwise go unnot iced. Of course, t his com es at a price in t hat t he necessary infrast ruct ure t o support m essage- level encrypt ion m ust be in place. I n t he preceding exam ple, t he paym ent inform at ion t hat m ight be sent as part of a t icket purchase includes t he nam e of t he cust om er as well as t he it inerary I D in clear t ext . Only t he act ual paym ent inform at ion is encrypt ed. This has t he sam e basic challenges one faces when creat ing a t horough aut horizat ion infrast ruct ure, bot h organizat ional and t echnical. Because of t his overhead, it is oft en a sensible alt ernat ive t o apply securit y at t he t ransport level. The preferred way t o accom plish t his is t o use t he Secure Socket Layer ( SSL) prot ocol. The benefit s of t his approach are obvious because SSL works alm ost everywhere, from m obile phones t o m ainfram e com put ers. I t requires lit t le overhead t o init ialize, and resource usage is lim it ed. Where resources are at a prem ium , it can be m ade fast er using special crypt ographic devices. When t wo- way SSL is em ployed, it can even be used t o pass user credent ials and t o perform t ransparent user logon. The m ain advant age, however, is t hat it is widely underst ood. This is a very im port ant issue because m any failures in securit y infrast ruct ure are due t o accident al m isconfigurat ion t hat in t urn occurs because t he im plicat ions of t he infrast ruct ure param et ers are not fully underst ood. However, t here are also downsides t o t his approach. I nt rusion det ect ion becom es pract ically im possible because t here is no way t o " list en in" t o t he occurring t raffic. When no end- t o- end net work connect ion can be em ployed, t his set up is a pot ent ial t arget for a m an- in- t he m iddle at t ack because m essages m ust be decrypt ed som ewhere along t he t ransport chain. This can be a sim ple net work " hop," or m essages need t o be decrypt ed by som e part of t he SOA archit ect ure, such as a MOM server t hat encrypt s t hem again when forwarding t o t he recipient . Furt herm ore, t wo- way SSL for aut hent icat ion is no longer usable because t he original cert ificat e does not ret ain a value beyond t he first net work hop. I n sum m ary, encrypt ion at t he m essage level is definit ely m ore desirable t han encrypt ion at t he t ransport level. Yet m ore oft en t han not , t he lat t er will be t he only feasible solut ion due t o t he lack of st andards for creat ing m essage- level securit y and fundam ent al t echnical considerat ions; for exam ple, wit h syst em s where perform ance is at a prem ium , you oft en cannot afford t he com put at ional and m em ory overhead of m essage encrypt ion and decrypt ion.

9.4.4. T RUST DOMAINS A layered archit ect ure such as an SOA oft en em ploys t he concept of t rust ed dom ains. Aut hent icat ion and aut horizat ion is locat ed in t he applicat ion front end and in t he high- level process- cent ric services. Ot her backend services and applicat ions m ight require no securit y at all or m ight support a very lim it ed set of securit y feat ures. For exam ple, no aut horizat ion or userlevel securit y m ight be required by t he backend services. Oft en, m inim al securit y is im posed using a firewall set up, where connect ions t o t he insecure backend services are allowed only from specific host s. This is som ewhat analogous t o com m on set ups using a Web server or an applicat ion server and a backend dat abase. I n t his sit uat ion, not every user is aut hent icat ed against t he dat abase; inst ead, t he server uses a pool of connect ions t o t he dat abase t hat are all associat ed wit h a single dat abase user. I t is evident t hat such a set up is applicable only wit hin a closed and secured com put ing net work. For a t ypical ent erprise com put ing set up, t his m aps t o cert ain net work segm ent s in a closed dat a cent er environm ent . One benefit of t his approach is t he ease of set up and developm ent of t he backend services. The developm ent effort associat ed wit h aut hent icat ion is sim ply om it t ed. I n addit ion, t he backend services can be independent of any ent erprise single sign- on infrast ruct ure, m aking t hem sim pler and ult im at ely m ore robust . The great est benefit s of t he approach are also it s great est weaknesses. For exam ple, all audit ing wit hin t he backend services m ust be perform ed m anually. I t is t hus m ore error- prone and harder t o m aint ain. This can lead t o enorm ous effort s if legal regulat ions require in- dept h changes t o audit ing. I t is oft en desirable t o m ake a backend service publicly available. I n t he case where t he required securit y infrast ruct ure is m issing in t hese services, t hey usually m ust be proxied by anot her service t hat adds no business value at all. This adds an addit ional level and int roduces a m aint enance effort if t he original service changes. For exam ple, an airline m ight decide t o expose t heir booking process t o a part ner airline, as in Figure 9- 18.

Figu r e 9 - 1 8 . A t r u st e d dom a in ca n be con ve n ie n t bu t ca n in t r odu ce ove r h e a d a n d e n t r opy if a se r vice in t h e t r u st e d dom a in m u st be pr om ot e d for t h ir d- pa r t y u se . I n t h is e x a m ple , a n a ir lin e is offe r in g it s book in g se r vice t o a pa r t n e r a ir lin e .

[View full size image]

Trust Domains Trust dom ains require a cont rolled, closed environm ent . I ndividual securing of even basic services is likely t o save m oney in t he long run and reduce m aint enance effort s.

9.4.5. SECURITY AND HETEROGENEITY Typical I T landscapes in t he ent erprise are very het erogeneous. For securit y considerat ions, t he sam e rule applied t o SOAs holds t rue: Do not fight het erogeneit y but rat her em brace it . More oft en t han not , it is im pract ical t o bind a backend service, let alone an ent ire legacy im plem ent at ion, com plet ely int o t he global ent erprise securit y fram ework. On t op of t hat , het erogeneit y also exist s am ong different client devices t o your SOA, as well as t he difference bet ween t he infrast ruct ure inside and out side t he ent erprise or t he depart m ent running t he SOA. For t he out side world, it is im port ant t hat t he SOA appeals t o as m any pot ent ial client s as possible. The general rule is t o secure t he services exposed t o t he out side world as m uch as possible. I t m ight be an opt ion t o provide cert ain services in m ult iple versions for different environm ent s and t o ensure t hat access t akes place t hrough t he correct device. Wit hin t he SOA, it is a good idea t o provide a unified securit y fram ework. However, no possible part icipant of an SOA should be alienat ed by t he required part icipat ion in a securit y fram ework t hat one does not want or cannot support . Thus, it should be an opt ion rat her t han a requirem ent . I n addit ion t o providing an SOA securit y fram ework, t he infrast ruct ure should be well prot ect ed using firewalls and t rip wiring bet ween t he different layers of t he applicat ion, as illust rat ed in

Figure 9- 19.

Figu r e 9 - 1 9 . I llu st r a t ion of se cu r it y in fr a st r u ct u r e t h a t is ca t e r in g for h e t e r o- ge n e ou s fr on t e n ds a n d ba ck e n ds a lik e . I n t h is e x a m ple , a n u m be r of se r vice im ple m e n t a t ion s pa r t icipa t e in a com m on se cu r it y in fr a st r u ct u r e , w h ich u se s a com m on se cu r it y st or e . Th e cu st om e r im ple m e n t a t ion is pa r t of a t r u st dom a in w it h r e spe ct t o t h e SOA in fr a st r u ct u r e . Acce ss t o t h is dom a in sh ou ld be police d in a cle a r ly de fin e d w a y.

[View full size image]

Secure the OutsidePolice the Inside Use off- t he- shelf t echnology, such as firewalls, t o lock out int rusion at t he net work perim et er. Use m onit oring and t rip wiring at syst em boundaries inside t he ent erprise t o det ect and report any suspicious act ivit y.

Let us st udy t he pract ical im plicat ions of such a set up wit h t he cust om er booking process. The booking process originat es on t he airline Web sit e. At t his level, cust om ers will usually aut hent icat e t hem selves wit h t he syst em . The abilit y of t he cust om er t o book a flight is decided wit hin t he Web layer. The booking process t hen calls int o t he cust om er booking service using a single privileged user. A firewall ensures t hat no call wit h t hat user ident it y is m ade from anywhere but t he Web applicat ion. The process- cent ric service checks aut horizat ion of t he caller and calls out in t urn t o t he different backend services passing t he user. They in t urn call out t o t he act ual legacy im plem ent at ions. Alt hough t wo of t hese share t he sam e user st ore wit h t he applicat ion, credent ial m apping is perform ed in order t o access t he cust om er inform at ion syst em on t he m ainfram e. None of t he business services perform any aut horizat ion because t hese decisions are m ade wit hin t he legacy applicat ions t hem selves. I f t he process originat es from a t ravel agent applicat ion front end, aut hent icat ion is perform ed direct ly wit hin t he applicat ion. The applicat ion is also bound int o t he general aut horizat ion schem e, present ing t he t ravel agent wit h only t he aut horized opt ions. Each t ravel agent is aut horized as a separat e user.

9.4.6. OTHER SECURITY TOPICS When a service is offered t o t hird part ies out side t he ent erprise, m at t ers of non- repudiat ion and m essage int egrit y are vit ally im port ant . Bot h t echnologies are direct ly relat ed t o concept s of asym m et ric encrypt ion, oft en using a public key infrast ruct ure [ GS1997] . For m essage int egrit y, it is sufficient t o digit ally sign t he m essage using a sym m et ric or asym m et ric key, provided t he key is known only t o t rust ed part ies. For m ost pract ical purposes, t he m essage will be signed using t he privat e key of a m essage sender, and t he receiver will check t he signat ure using t he public key of t he sender ( see Figure 9- 10) . Not e t hat t he sender and receiver need not be t he original sender and final receiver of t he m essage but t hat t he signat ure can be added t o a sent m essage only for cert ain part s of t he rout e, for exam ple bet ween t wo t rust ed dom ains. Furt herm ore, t he keys need not be m aint ained by an independent t hird part y. I t is perfect ly feasible and oft en easier t o use a PKI t hat is host ed wit hin t he service provider for t his purpose, especially where t he service provider engages in business only wit h ent it ies t hat are well known t o t he original service provider. Non- repudiat ion m eans t hat t he service provider and service user assum e legal responsibilit y for t he act ual m essage cont ent s ( see Figure 9- 10) . For exam ple, if t he airline orders a cert ain num ber of m enus from t he cat erer, t he cat erer m ight want t o ensure t hat t he airline has act ually placed t his exact order and not som et hing different . Likewise, t he airline can use t he signed order confirm at ion of t he cat erer in case t here is a disput e bet ween t he airline and t he cat erer. For m essages t o be non- refut able, it is usually required t hat a t rust ed ent it y m anages t he public keys of t he part ies engaged in t he act ual business t ransact ion. Moreover, t hese ent it ies will usually be licensed by som e governm ent agency t o conform t o cert ain st andards, processes, and service levels. Message signat ures can t hen be considered suit able for usage in a legal disput e or m ult im illion dollar cont ract s, for exam ple t he gross purchase of fuel or t he ordering of an aircraft . Technically, m essage signing can be easily accom plished if t he m essage is in plain t ext form at and can easily be m anipulat ed. The best exam ple is signing XML snippet s t hat can be placed inside t he body of a SOAP m essage.

Figu r e 9 - 2 0 . Com m u n ica t ion sce n a r io for m e ssa ge in t e gr it y ( t op) a n d n on - r e pu dia t ion ( bot t om ) . For m e ssa ge in t e gr it y, t h e r ou t e r s sign t h e

m e ssa ge a n d ch e ck t h e sign a t u r e t o e n su r e m e ssa ge in t e gr it y a ft e r le a vin g a n d be for e e n t e r in g t h e t r u st e d dom a in s. For n on - r e pu dia t ion , a t r u st e d t h ir d pa r t y is n e e de d.

[View full size image]

9.5. Conclusion I n t his chapt er, we looked at t he different elem ent s t hat const it ut e an SOA- enabling infrast ruct ure, which we refer t o as a service bus. Based on our convict ion t hat m iddleware het erogeneit y is an ent erprise realit y t hat can't be solved by int roducing yet anot her set of int erfaces and com m unicat ion prot ocols, we described an approach which is based on loose coupling on t he infrast ruct ure level, effect ively allowing t he co- exist ence of m ult iple, het erogeneous m iddle ware plat form s wit hin a higher- ranking m et a bus which is designed t o t ie t oget her t hese different plat form s. We also st ressed t he fact t hat t his m et a bus should not be developed in isolat ion from concret e applicat ion proj ect s in order t o avoid working on an overly abst ract level. Pract ical experience has shown t hat one can work for m any years on t he perfect ent erprise infrast ruct ure, wit hout ever reaching a t ruly st able result . Consequent ly, a pragm at ic, proj ect driven approach should be t aken. The different applicat ion execut ion cont ainers t hat we find in an ent erprise represent t he basis for building services in an SOA. We discussed t he general nat ure of t hese cont ainers, specific im plem ent at ions, and how t o int egrat e across m ult iple cont ainers. To m axim ize t he robust ness of our SOA, we discussed dealing wit h syst em failures wit h t he m ain obj ect ive of gat hering as m uch knowledge as possible about crit ical part s of t he service. This enables recovery from a service failure t o be as seam less as possible, bot h aut om at ically and m anually. To m inim ize such failures in t he first place, we discussed concept s for scalabilit y and availabilit y. Bot h can be addressed using st andard product s and procedures and som et im es hardware com ponent s such as balancers are a good opt ion. On t he ot her hand, m ost fram eworks for dist ribut ed obj ect s such as EJB and CORBA support t he basic building blocks needed. Proper planning is essent ial for providing an available and scalable syst em . This holds t rue for t he init ial definit ion of t he SLAs up t o any subsequent exercise t o scale a syst em by adding addit ional hardware and soft ware. I t is fairly easy t o accom plish availabilit y at t he syst em level and bet ween individual request s. Guarant eeing in- request availabilit y is considerably harder. I t is not required nor appropriat e for a vast num ber of applicat ion scenarios, and it int roduces a significant overhead int o syst em perform ance, very m uch com parable t o t he problem s of dist ribut ed t ransact ions. For pract ical reasons, t he easiest way t o accom plish availabilit y is t o m aint ain request cycles and t ransact ions t hat are as short as possible. I f anyt hing goes wrong at t hese point s, t hen using t he st rat egies from t he previous sect ioncoping wit h failureswill usually be sufficient for cont rolled recovery. Always consider t he weak point s in your applicat ion first . Keep in m ind t hat t he applicat ion is only as st rong as it s weakest link, bot h for availabilit y and scalabilit y. Finally, not e how m uch easier it becom es t o cope not only wit h availabilit y and scalabilit y but also wit h service failures if your business logic becom es st at eless or even idem pot ent . The st rat egies t hat deal wit h st at eful syst em s are m ore com plex and t end t o im pact on t he overall syst em perform ance and ult im at ely on scalabilit y and availabilit y. On t op of providing reliabilit y and availabilit y, any I T infrast ruct ure m ust support cert ain securit y feat ures. Alt hough t echnologies and t ools are available t hat can be used t o provide t ransparent , ent erprise- wide aut hent icat ion and aut horizat ion for an SOA, it is very likely t hat a first st ep will provide only aut hent icat ion as part of t he SOA fram ework it self. The reasons are m ainly root ed in

t he fact t hat aut hent icat ion can be int roduced in m ost legacy syst em s wit h very lit t le effort , while refact oring t he aut horizat ion aspect s of an applicat ion requires m ore effort . Also, it will be far easier t o im plem ent from an organizat ional perspect ive. Legacy applicat ions are likely t o be placed wit hin t rust ed dom ains t o allow for int egrat ion int o t he SOA wit hout dist urbing t he ongoing operat ion of t he applicat ion. SOAs are oft en int roduced t oget her wit h a single sign- on fram ework. Shift ing m ore and m ore aspect s of applicat ion securit y, such as aut horizat ion, encrypt ion, and PKI , t o use t he fram ework over t im e, an SOA can em ploy and fost er t he im plem ent at ion of a t rue ent erprise end- t o- end securit y infrast ruct ure.

References [ DC2004] Chappell, Dave . Ent erprise Service Bus. O'Reilly, 2004. [ GS1996] Garfinkel, Sim son , et al. Pract ical UNI X and I nt ernet Securit y . O'Reilly, 1996. [ GS1997] Garfinkel, Sim son , et al. Web Securit y and Com m erce. O'Reilly, 1997. [ GS2003] Gupt a, Sam udra . Logging in Java wit h t he JDK 1.4 Logging API and Apache log4j . Apress, 2003. [ H2003] Hart m an, Bret , et al. Mast ering Web Services Securit y . Wiley, 2003. [ PT2001] Pelt ier, Thom as R. I nform at ion Securit y Risk Analysis. Auerbach Pub, 2001. [ PH2000] Piedad, Floyd and Michael Hawkins . High Availabilit y: Design, Techniques and Processes. Prent ice Hall, 2000. [ TDM2000] Tardugno, Ant hony , Thom as DiPasquale, and Robert Mat t hews . I T Services Cost s, Met rics, Benchm arking and Market ing. Prent ice Hall, 2000. [ SK2003] Schm eh, Klaus . Crypt ography and Public Key I nfrast ruct ure on t he I nt ernet . Wiley & Sons, 2003.

URLs ht t p: / / www.redbooks.ibm .com / redbooks/ SG242234.ht m l ht t p: / / www.oasis- open.org/ com m it t ees/ t c_hom e.php?wg_abbrev= securit y ht t p: / / www.w3.org/ Encrypt ion/ 2001/ ht t p: / / www.om g.org/ t echnology/ docum ent s/ form al/ corba_2.ht m

Chapter 10. SOA in Action This chapt er shows how a service can be im plem ent ed in t he real world t o serve t he needs of different usage m odels. You can provide t he sam e service using different prot ocols and different granularit y according t o t he cont ext in which you use it . As is t he case wit h t radit ional soft ware, it is im possible t o design a service based only on abst ract principles. However, you can em ploy careful planning and refact oring of a service t o m ake an im plem ent at ion suit able for m any different usage scenarios. To illust rat e t he different design considerat ions applicable t o different usage m odels, we em ploy a passenger check- in scenario. A passenger m ight check in wit h a m obile phone or PDA, using an elect ronic ( or physical) t icket at an airline boot h. Alt ernat ively, t he passenger m ight be checked in wit h one airline on behalf of a second airline, perhaps if t he second flight leg is operat ed by a different carrier. I n som e scenarios, t he sam e service m ight be accessed using m ult iple channels at t he sam e t im e. I n it s m ost generic form , m ult iple services are at work. At t he heart of everyt hing is t he check- in service, assigning seat s t o passengers on airplanes and keeping t rack of t he seat occupat ion in individual planes. As input , it t akes any num ber of passenger t icket coupons and ret urns t he appropriat e num ber of boarding passes. Usually, ot her services will also be involved. Prior t o perform ing t he check- in, t he t icket service can be used t o det erm ine which t icket s are available for a passenger and t o validat e t he t icket s before check- in, in addit ion t o m arking t he t icket coupons aft er check- in has been com plet ed. The cust om er inform at ion service m ight be cont act ed t o read dat a regarding t he preferences or frequent flyer account of t he cust om er. Preferences for seat ing can be used in t he booking it self. A m eal preference can be used wit h services available from t he airline's cat erer t o prepare only t he required am ount of special food, t hus decreasing t he airline's overhead. Personal dat a regarding t he passenger m ight be forwarded t o t he cust om s or im m igrat ion office at t he passenger's dest inat ion. For t he t im e being, t his discussion will be lim it ed t o t he services m ent ioned previously, alt hough m ore services m ight be involved for issues such as baggage handling and airport services. As a prerequisit e, it is wort hwhile t o det erm ine if t he provided services are aggregat ion- friendly. The t icket service's sole purpose is looking up t icket s and checking t heir validit y. These calls can be considered idem pot ent at t he sem ant ic level. I f t hey fail, t he caller can reat t em pt t he call and expect t he sam e result as t he previous call. I nvalidat ing t he coupon is an operat ion t hat will change som e persist ent ly st ored dat a. The call t o t his operat ion is logically t ied wit h t he assign seat s call of t he check- in service it self. The lat t er is likely t o live in a local t ransact ion and can be rolled back upon failure. I n t his event , t here will not be any at t em pt t o change t he st at e of t he t icket voucher, even t hough such changes can be easily reset in a com pensat ing call. Finally, set t ing t he m eal preference is an operat ion t hat m ight also be considered idem pot ent . However, in an operat ional syst em , t his t ype of process is m ore likely t o be im plem ent ed in an asynchronous m anner. I n general, it is necessary only t o guarant ee t hat t he cat erer get s t he inform at ion about m eal preferences well before t akeoff rat her t han at som e specific point in t im e. Throughout t his chapt er, we will use t he sam e exam ple in a set of different scenarios. I n Sect ion 10.1 , we discuss a Web applicat ion. I n Sect ion 10.2, our exam ple t akes t he form of an EAI int egrat ion scenario. We em ploy a B2B m odel in Sect ion 10.3. I n Sect ion 10.4, we discuss a fat client scenario, and in Sect ion 10.5, we deploy it by using a sm all m obile device such as a cellular t elephone. Finally, in Sect ion 10.6, we discuss t he m ult i- channel scenario.

10.1. Building Web Applications As previously discussed, Web applicat ions are part icularly suit ed as client s for a Service- Orient ed Archit ect ure because t hey offer a nat ural m eans of st oring cont ext or st at e inform at ion when m aking calls t o a m ainly st at eless service archit ect ure. I n addit ion, t hey offer a rich user int erface, and users of Web applicat ions are accust om ed t o a high degree of int eract ivit y. Alt hough t he int eract ion m odel in Figure 10- 1 rem ains a possibilit y for providing check- in funct ionalit y using a Web int erface, it is likely t hat an airline will provide t he user wit h a richer set of choices during t he booking process.

Figu r e 1 0 - 1 . Ge n e r a l se r vice in t e r a ct ion for pa sse n ge r ch e ck - in . Th e ch e ck - in se r vice t a k e s t ick e t s a s in pu t pa r a m e t e r s a n d cr e a t e s boa r din g pa sse s a s ou t pu t pa r a m e t e r s, in va lida t in g t h e r e le va n t cou pon of t h e t ick e t . N ot e t h a t a ll se r vice s a r e ba sica lly st a t e le ss.

[View full size image]

These choices will at least include t he select ion of seat s and m eals available on t he flight . On t he ot her hand, som e of t he opt ions t hat were originally present m ight not be applicable. For exam ple, checking in online is oft en only possible if a regist ered user has purchased t he t icket because logging in t o t he airline's Web sit e is required in order t o perform t he check- in operat ion. On t op of t hat , t wo people t raveling t oget her m ust check in separat ely if t hey purchased t heir t icket s separat ely. I n a Web applicat ion, users will aut hent icat e t hem selves against t he Web t ier, usually t hrough a Web page form wit h usernam e and password or client - side cert ificat es. Users can t hen be st ored wit hin t he Web t ier of t he applicat ion. For subsequent calls t o services, t here are t wo possible m odels: principal propagat ion or t rust . Principal propagat ion is t rivial and uses fram eworks t hat are built t o support t his feat ure. For exam ple, wit hin t he J2EE fram ework, t he sam e principal obj ect used in t he Web t ier can direct ly be used t o call ot her J2EE services such as Ent erprise Java Beans. For ot her service t ypes, such as CORBA or SOAP, t he process is not as st raight forward. I t m ight include t he need for m apping credent ials from t he Web t ier t o t he service layer in a specific way. As m ent ioned in Chapt er 9, " I nfrast ruct ure of t he Service Bus," SOAP does not support a st andardized m eans of passing credent ials. Because Web sit es can operat e wit hin a cont rolled corporat e environm ent , t rust bet ween t he Web t ier and t he service layer is a com m on scenario. All calls t o t he service layer are t hen carried out using a com m on

ident it yor no ident it y at alland t he act ual caller principal is j ust passed as a param et er using call param et ers. Of course, t his requires t hat t he Web applicat ion is not exposed direct ly t o a sensit ive net work segm ent . The int eract ion diagram in Figure 10- 2 illust rat es t he need t o expose t he " assign seat s" funct ionalit y t o t he out side client . I n addit ion, displaying t he available seat s t o t he user for a given plane's seat ing configurat ion is also necessary. Alt hough it is t em pt ing t o provide t his t ype of layout in an ad- hoc m anner wit hin t he Web applicat ion, perhaps as a num ber of configurat ion files, t his im plem ent at ion would soon becom e unm anageable. Airlines oft en change t heir seat ing configurat ions and add new planes, and it is im port ant t o provide t his inform at ion t o cust om ers. I n addit ion, t he seat configurat ion of t he airplane and t he list ing of available seat s can be t ransferred wit hin a single call, providing a good exam ple of a coarse- grained dat a st ruct ure.

Figu r e 1 0 - 2 . I n t e r a ct ion t h a t sh ow s t h e ch e ck - in pr oce ss for a W e b a pplica t ion . Th e st a t e of t h e a pplica t ion is m a in t a in e d in t h e W e b a pplica t ion .

[View full size image]

Figure 10- 2 shows t he full int eract ion diagram for t he Web applicat ion invocat ion. Not e t hat we m ade use of t he result s from Chapt er 8, "Process I nt egrit y ," by pushing st at e as far up t he chain as possible. I n fact , all st at e is st ored in t he Web applicat ion it self. This enables a rich int eract ion bet ween cust om er and applicat ion wit hout t he need for any st at eful services. Not e t hat t he t ransact ion boundaries of t he original exam ple have not changed. There is no need t o expose t ransact ions direct ly t o t he client . A reasonable am ount of int eract ion t akes place bet ween t he client and t he Web applicat ion direct ly wit hout any need t o involve t he service layer. This is t he result of a well- defined locat ion for st oring conversat ional st at e along wit h an opt im ist ic concurrency m odel ( see Chapt er 8) . Aft er t he available t icket s have been ret rieved from t he Ticket Service, users m ay choose any select ion of flight coupons for which t hey want t o check in. Any logical checks, such as whet her a check- in

for t he coupon can be perform ed at t he current dat e, can be easily carried out at t he Web t ier. This is achievable because one person generally handles a single coupon at a t im e. Of course, race condit ions can st ill occur, such as if som eone t ries t o check in using t he t elephone and a Web applicat ion sim ult aneously. The sam e holds t rue for seat select ion. While m aking t he seat ing select ion, one user m ight select som e seat s t hat are already reserved by anot her user. However, given t he t im e available for check- in and t he t ot al num ber of passengers on a plane, t his is rat her unlikely. I n t he event of an error when assigning seat s, t he user can be prom pt ed wit h an updat ed seat m ap t o perform a reselect . Alt hough t he service calls for reserving seat s and invalidat ing a t icket are t echnically idem pot ent , t his fact cannot be exposed t o t he cust om er. One reason is t hat you shouldn't expose a recoverable error t o t he cust om er in t he first place. The ot her is t hat alt hough changing t he st at e of a t icket is idem pot ent , t he creat ion of boarding passes m ight well not be. I n t hat case, it is usually necessary t o prevent t he cust om er from m aking t he sam e call m ore t han once. I n Web applicat ions, t his is a com m on scenario t hat can happen easily if t he cust om er im pat ient ly hit s t he reload or subm it but t on repeat edly. I f not handled properly, such behavior can at worst st all t he applicat ion. The solut ion is t o apply a one- t im e t ransact ion pat t ern ( OTT) . This involves st oring a t ransact ion I D wit h associat ed st at es such as running, finished, and failed. I f a cust om er reat t em pt s an operat ion, t he syst em ret rieves t he st at e of t he one- t im e t ransact ion I D and forwards t he cust om er t o an appropriat e page. Figure 10- 3 illust rat es t his behavior.

Figu r e 1 0 - 3 . On e - t im e t r a n sa ct ion pa t t e r n . Th e W e b a pplica t ion m a n a ge s a n u m be r of t r a n sa ct ion t ok e n s a n d se r ve s r e qu e st s ba se d on t h e st a t e of t h e t ok e n s. For e x a m ple , a t ok e n ca n st a r t pr e cise ly on e ope r a t ion . W h ile t h e t ok e n is in t h e r u n n in g st a t e , som e m e a n in gfu l in for m a t ion is r e t u r n e d t o t h e clie n t .

As we m ent ioned before, service calls can be eit her synchronous or asynchronous and can be long running. The one- t im e t ransact ion pat t ern is a suit able t echnology for handling such an invocat ion scenario. Here, t he invocat ion ret urns a st at us page im m ediat ely aft er dispat ching t he asynchronous call. The st at us page checks periodically for available result s using t he OTT t oken.

This is a com m on t echnique for com plicat ed and long- running processes such as purchasing airline t icket s. I t is wort h m ent ioning t hat t he creat ion of a Web applicat ion can j ust ify im plem ent ing an SOA. I n m any Web applicat ions, it is necessary t o int erface wit h an organizat ion's operat ional core syst em s, which can be anyt hing from m ainfram e syst em s or relat ional dat abases t o CORBA or J2EE servers. Thus, t he creat ion of Web applicat ions oft en involves t he int egrat ion of m ult iple t ypes of ent erprise syst em s. Usually, t his is achieved by using various vendor- specific API s t o connect t o t he different syst em s and exposing som e consolidat ed API t o t he applicat ion. Because t his int egrat ion effort m ust be perform ed one way or anot her, t he im plem ent at ion of t hese int egrat ions as services present s relat ively sm all overhead and m anifest s it self m ainly during design t im e in t he way t ransact ion boundaries and principal propagat ion are addressed in t he applicat ion. This will at least lead t o an int ernal API t hat is " service- ready" and t hat can be exposed as a service using appropriat e t ools. Web applicat ions can be creat ed using a num ber of t ools, including Perl, PHP, Pyt hon, Java Server Pages ( JSP) , and Microsoft 's ASP.NET. When creat ing a Web applicat ion t hat is sim ply a client t o exist ing services, any environm ent t hat fost ers access t o t he t echnology used t o im plem ent t hese services can be a reasonable choice. For exam ple, if your SOA is built using Ent erprise Java Beans, t hen using Java Server Pages and Servlet s for t he Web applicat ion is t he nat ural choice. Likewise, if t he SOA is im plem ent ed using XML over HTTP, perhaps wit h SOAP, you can use any plat form wit h support for t he lat est Web service t echnologies for t he Web applicat ion. For effect ively em ploying an OTT pat t ern, t he plat form should also facilit at e t he easy creat ion of request int ercept ors. J2EE Servlet filt ers provide one such m echanism . Many Web applicat ions involve not only t he use of exist ing services but also t he direct int egrat ion of exist ing syst em s. I t is usually a good idea t o focus on a com m on skill set for t he creat ion of such an applicat ion. The J2EE plat form and Microsoft .NET are suit able t o t hat end, not least because com ponent s creat ed on t hese plat form s can easily be t urned int o Web services.

10.2. Enterprise Application Integration At first glance, Ent erprise Applicat ion I nt egrat ion ( EAI ) appears t o be t he perfect environm ent t o em ploy an SOA: I n fact , plent y of reasons t o use an SOA as a driver for EAI exist , and vice versa. However, EAI s pose a cert ain set of requirem ent s in addit ion t o profound non- t echnical issues t hat you m ust consider. Today, m ost large organizat ions have a highly fragm ent ed applicat ion infrast ruct ure, in which a vast num ber of client applicat ions have been creat ed using m ult iple program m ing plat form s and com m unicat ion infrast ruct ures from m ult iple vendors. A recent exam ple of t he endeavor t o solve all applicat ion int egrat ion problem s is t he int roduct ion of a corporat e " st andard" ERP ( Ent erprise Resource Planning) suit e. This process failed in m ost organizat ions due t o t he relat ively low flexibilit y such a solut ion provides and t he fact t hat businesses have a need for t act ical I T solut ions, as opposed t o st rat egically planned applicat ions. This exam ple of failure provides a reason t o be wary of addressing EAI problem s wit h an SOA. Nevert heless, if an SOA is properly deployed, it will address m any t echnical and core concerns relat ed t o EAI . There are t wo ways of looking at EAI in a service- orient ed world: as a service provider and as a service consum er. As a service consum er, you t ypically want a service t hat is easy t o access, highly available, funct ional com plet e, and st able over an indefinit e period of t im e. As a service provider, you m ight want a service t hat is easy t o deploy and t o upgrade; in an EAI environm ent , qualit y of service, single sign- on, and ot her at t ribut es becom e param ount . At t he out set , it is wort h not ing t hat so- called EAI product suit es are generally unsuit ed t o t ackle t he aforem ent ioned EAI challenges. Most provide a clear int egrat ion pat h t hat is usually support ed by accom panying t ools, som e of which provide for qualit y of service and ship pluggable securit y m odules, but t hey usually fail t o provide a st able view of t he result of t he int egrat ion process. Furt herm ore, because t hey only part ly adhere t o open or indust ry st andards, t hey are very unlikely t o be st able for a reasonable period of t im e. This is one of t he prim e reasons t hat organizat ions t hat t ake SOA seriously alm ost always choose t o build t heir core service archit ect ure t hem selves while em ploying st andards wherever possible ( see t he case st udies in Chapt ers 14 t hrough 17 ) .

10.2.1. SERVICE ENABLEMENT I n order t o provide EAI funct ionalit y in a service- orient ed environm ent , t he m ost com m on t ask is service enablem ent . Service enablem ent is t he process t hat creat es a service t o encapsulat e t he funct ionalit y provided by an exist ing applicat ion. The t ype of applicat ion encapsulat ed can be anyt hing from a m onolit hic m ainfram e applicat ion t o a dist ribut ed applicat ion built on t op of CORBA or DCOM. Of course, service enablem ent is m ore t han j ust wrapping and exposing an exist ing program m ing int erface. As an exam ple, consider t he m ovem ent of an exist ing check- in applicat ion in a serviceorient ed dom ain. Assum e t hat as shown in Figure 10- 4t he original applicat ion is a t ypical client / server applicat ion. As a first st ep for service enablem ent , t he applicat ion is separat ed int o a visual and a non- visual part . Com m unicat ion bet ween bot h part s is grouped based on t heir business dom ain. Access t o t he non- visual layer is defined by one or m ore int erfaces. I f possible, t he im plem ent at ion of t he int erfaces and t he persist ent dat a upon which t hey act can also be separat ed. Finally, t he applicat ion is m oved t o t he service infrast ruct ure.

Figu r e 1 0 - 4 . Tr a n sfor m a t ion of a m on olit h ic a pplica t ion in t o a se r vice or ie n t e d a pplica t ion . [View full size image]

Depending on t he applicat ion, one or m ore services m ight em erge from t his analysis. For exam ple, when analyzing a real- world check- in applicat ion, it is quit e likely t hat services such as t icket ing, baggage handling, and act ual check- in will surface during such an analysis. Aft er t his analysis is com plet e, consolidat ed int erface descript ions for t he com m unicat ion can be derived. This should include provisions for undoable act ions and idem pot ency wherever possible. The im plem ent at ion and possibly t he int erfaces need t o be changed t o include infrast ruct ure services such as user m anagem ent and securit y as well as int ernal changes for server- side resource handling. Now t he visual layer and t he service layer can act ually be separat ed. I t is evident from t he check- in exam ple t hat t he exam inat ion of a single applicat ion is unlikely t o yield fully reusable services. Alt hough it is reasonable t o assum e t hat one can obt ain a fairly com plet e descript ion of t he check- in service it self, it is rat her unlikely t hat sufficient inform at ion about t he baggage handling or t he t icket ing service can be obt ained. I f m ore t han one applicat ion exist s t hat relies on com m on underlying principles, im m ediat e reuse benefit s can be obt ained from t he service enablem ent . For exam ple, m any applicat ions share t he concept and even t he persist ence m echanism for ent it ies, such as cust om er or cont ract in a business scenario. Fact oring t hese out int o services not only provides inst ant benefit s for reuse but also act s as a successful dem onst rat ion of t he suit abilit y of t he Service- Orient ed Archit ect ure t hat is likely t o em power ot her depart m ent s t o use t he newly creat ed services and cont ribut e new

services. When service- enabling applicat ions for EAI in t he way we have described, it is essent ial t hat a service regist ry be available. This can be used t o ident ify t hose rem aining services t hat are already available in t he applicat ion and t hat can be fact ored out of t he exist ing applicat ion. Anot her com m on scenario is t o provide services for an already dist ribut ed archit ect ure. Recall t hat a dist ribut ed archit ect ure alone does not necessarily consolidat e an SOA. Oft en, t he int eract ion pat t erns bet ween t he applicat ion front end and t he dist ribut ed com ponent s are t oo fine- grained. A sensible approach is t o define façade services t hat sit bet ween t he applicat ion and t he dist ribut ed obj ect s. The applicat ion is st ripped of all knowledge regarding it em s such as dist ribut ed obj ect s and uses t he service as it s sole com m unicat ion channel t o t he backend logic ( see Figure 10- 5) . The service t hen aggregat es t he funct ionalit y available in t he dist ribut ed com put ing environm ent . At t he sam e t im e, coupling in t he dist ribut ed obj ect layer can be replaced by explicit ly coupling services. I t will also use a com m on service infrast ruct ure, as you have seen previously. I t is oft en desirable t o replace t he dist ribut ed obj ect im plem ent at ion wit h a m ore localized im plem ent at ion t o reduce net work load and lat ency. Because t he im plem ent at ion has been encapsulat ed in t he service layer, it can easily be replaced wit hout having an effect on t he applicat ions t hat use t he service. The service façades t o t he dist ribut ed com put ing environm ent should be cut in a way t hat enables t hem t o act as t he sole owner of t he dat a upon which t hey operat e.

Figu r e 1 0 - 5 . Cr e a t in g a se r vice la ye r t h a t r e pla ce s dir e ct in t e r a ct ion w it h dist r ibu t e d obj e ct s.

[View full size image]

There are, of course, ot her scenarios t hat can be used t o enable t ransform at ion t oward an SOA. For exam ple, rich applicat ions t hat rely on backend services ( see Sect ion 10.4) great ly benefit

from t he creat ion of a service layer.

EAI Can Drive an SOA EAI is an excellent driver for service- enabling exist ing applicat ions. From a business perspect ive, EAI is int roduced t o sim plify t he applicat ion infrast ruct ure and t o fost er reuseproviding good m ot ivat ion for creat ing an SOA.

10.2.2. STABILITY AND UPGRADE ABILITY Service st abilit y and t he abilit y t o upgrade are t wo of t he m ost desirable feat ures in an EAI environm ent . The reason is t hat t he service consum er m ight reside in a different depart m ent or even in a different count ry from t he service provider. The service provider m ust be able t o upgrade a service wit hout having an im pact on current applicat ions t hat int egrat e t his service. You can use various m et hods t o solve t his problem , wit h t he t wo m ain st yles being backward com pat ibilit y and t he provision of different versions. Backward com pat ibilit y is illust rat ed in Figure 10-6 . Effect ively, t his m eans t hat service int erfaces can only be ext ended wit hout violat ing exist ing int erface cont ract s. This is sim ilar t o t he discussion on payload sem ant ics in Chapt er 3, " I nvent ory of Dist ribut ed Com put ing Concept s." Unfort unat ely, t his approach t ends t o weaken t he int erface cont ract s over t im e, which in t urn m akes it difficult t o enforce a specific usage st yle.

Figu r e 1 0 - 6 . A se r vice t h a t su ppor t s ve r sion in g by e x t e n sion of t h e r e qu e st docu m e n t for m a t . Alt h ou gh t h is is e a sy t o a ch ie ve w it h pu r e pa yloa d se m a n t ics, it ca n be difficu lt in ot h e r e n vir on m e n t s.

Alt hough it is fairly st raight forward t o achieve t his using XML com m unicat ion such as SOAP, it can be arbit rarily hard t o do so using CORBA or DCOM wit hout revert ing t o pure payload sem ant ics. On t he ot her hand, you can easily provide different product ion versions of a service by deploying t he service in different locat ions. These locat ions can be looked up in a regist ry t hat also cont ains inform at ion regarding t he m ost up- t o- dat e service available. Alt hough t his creat es significant overhead in m anaging t he SOA, it com plet ely decouples service upgrades from exist ing applicat ions. I t is also st raight forward t o provide t his using any available com m unicat ion m odel. The out - of- dat e services are cont inuously m onit ored and can be shut down when no m ore applicat ions are accessing t hem . Moreover, it m ight be desirable t o reim plem ent t he old services in t erm s of t he new services. By keeping t he cont ract unchanged, t he old service client s can com m unicat e t ransparent ly wit h t he new im plem ent at ions. Figure 10- 7 illust rat es t wo versions of t he check- in service working wit h an online regist ry. Using t he online regist ry for binding t o t he act ual service im plem ent at ion at runt im e m akes it possible t o m igrat e older services t o different , less capable hardware.

Figu r e 1 0 - 7 . An SOA t h a t e n a ble s diffe r e n t ve r sion s of a sin gle se r vice t o be pr odu ct ive a t t h e sa m e t im e .

[View full size image]

EAI Requires Repositories EAI scenarios should use service reposit ories because t his is t he best way t o ensure st abilit y and upgrade abilit y. I t is crucial t o give your int ernal client s peace of m ind when using t he SOA.

10.3. Business-to-Business I n a business- t o- business ( B2B) environm ent , a corporat ion offers som e service t o anot her com pany over public net works such as t he I nt ernet . I n a B2B scenario, bot h t he service consum er and t he service provider benefit great ly from st andard com m unicat ion prot ocols. The service consum er profit s from t he increased flexibilit y gained when changing t o anot her provider or when using several providers at t he sam e t im e. The service provider can also profit for exam ple, depending on t he t ype of com m unicat ion, it m ight be possible t o elim inat e t he necessit y t o ship soft ware t o t he client . The sam e goes for t he securit y infrast ruct ure. Because securit y infrast ruct ures can get very com plicat ed, t he client and server m ust be cert ain t o use a m echanism t hat can be t rust ed, where bot h ends of t he com m unicat ion chain can ident ify a securit y violat ion using t heir st andard syst em m anagem ent t ools. St andards are also desirable because m any int eract ions creat e a legal obligat ion bet ween t he part ies. I n addit ion, it is at t ract ive t o define a st andard form at for t he act ual dat a being t ransferred. This saves bot h sides t im e and m oney in service developm ent and int egrat ion. UN/ CEFACT ( ebXML) and Roset t aNet provide exam ples of init iat ives t o est ablish dat a st andards. [ 1] Alt hough t he result ing st andards are not yet m at ure and t hey are not appropriat e for all B2B problem s, it is wort hwhile t o see if t hey are applicable in specific cases. [1]

See the URLs list at the end of this chapter.

Most init iat ives t hat define st andard prot ocols result in t he creat ion of rat her large docum ent s t hat are exchanged in a business t ransact ion. This is basically analogous t o a service- orient ed approach wit h very coarse granularit y. Because of t he lat ency t hat can be experienced on a public net work connect ion, it is far m ore efficient t o send a single large docum ent inst ead of m aking short procedural int eract ions. Alt hough t he business process it self can be st at eful, it pays t o creat e st at eless service invocat ions. I n a B2B scenario, t he m ost com m on way t o achieve t his is t o pass a t oken along wit h every service invocat ion. These t okens can be used once or on a cyclical basis if t he num ber of t okens is large com pared t o t he frequency of int eract ions.

Go Stateless for B2B St at eless sem ant ics are especially im port ant in B2B scenarios. They enable int eract ion wit h rem ot e cust om ers over connect ions wit h high net work lat ency. They also creat e m uch fewer const raint s for t he calling applicat ions.

Alt hough t he aut hors do not believe t hat a runt im e- nam ing service or reposit ory is m andat ory in an SOA environm ent , it is of great use in a B2B scenario. The pressure for a service t o provide locat ion t ransparency is far higher t han in a cont rolled ent erprise environm ent . Aft er a business part ner uses a service, it m ight require significant effort t o swit ch t o a different version. Even if it can be achieved by configurat ion only, it st ill creat es an overhead t hat m ight result in unwant ed

downt im e. Bot h service user and provider can be separat ed by large dist ancesas m uch as t housands of m iles. This m akes locat ion t ransparency a necessit y in order t o provide disast er recovery capabilit ies. I f a cust om er uses a service of a com pany whose m ain com put ing cent er goes down in a fire, t he cust om er expect s t o be t ransferred t o t he secondary sit e aut om at ically. Of course, t his also includes t he service reposit ory it self being failsafe.

Location Transparency for Stable B2B Services B2B scenarios require real locat ion t ransparency using service reposit ories. This enables cust om ers t o securely est ablish long- t erm relat ionships regardless of changes t o t he supplier's infrast ruct ure.

B2B scenarios can include online billing m echanism s, alt hough t hey are m ore com m on in a business- t o- consum er ( B2C) scenario. A B2B scenario will generally log only access inform at ion on t he side of t he service provider. This inform at ion can lat er be used t o creat e a m ont hly invoice. I n a B2B scenario, t he check- in service can be used by part ner airlines t o check in cust om ers on a flight leg t hat is operat ed by t he service provider. I n cont rast t o t he exam ples considered so far, t he m ain difference is t hat t he service provider cannot validat e t he act ual request against it s own t icket dat abase. There are several opt ions for handling t his sit uat ion. The first opt ion is t o t rust t he rem ot e syst em because it is well est ablished t hat t he business part ner is t rust wort hy and provides only valid request s. Here, t he rem ot e check- in service does not have any inform at ion about t he t icket s t hat are used on individual flight legs, nor does t he service have any inform at ion regarding t he respect ive passenger. Dat a t hat direct ly relat es t o cust om er service qualit ysuch as t he cust om er's m eal preferencem ust t herefore be passed in t he check- in call it self. This is shown in Figure 10- 8.

Figu r e 1 0 - 8 . On e - w a y com m u n ica t ion sce n a r io. Th e t ick e t in g com pa n y pe r for m s t h e ch e ck - in u sin g t h e ope r a t or 's se r vice , pa ssin g a ll t h e r e le va n t da t a du r in g t h e ca ll it se lf. [View full size image]

The second opt ion is t o synchronize all relevant t icket and cust om er dat a bet ween t he part ner com panies on a regular basis. The t icket ing com pany forwards t he t icket dat a t o t he operat ing airlines, and t he operat ing airlines in t urn forward t he flight inform at ion back t o t he t icket ing com panies. Alt hough t he check- in process is sim ilar t o t he ones discussed in earlier chapt ers, service access wit hin a single corporat ion is all t hat is needed during t he act ual check- in operat ion. Finally, t he t hird opt ion in t he B2B scenario is t hat t he service provider becom es a service consum er at t he ot her airline's t icket inform at ion service. I n t his sit uat ion, bot h businesses creat e an effect ive t wo- way com m unicat ion scenario. Because t he client s t rust each ot her, t he operat ing airline can validat e t he t icket s of t he part ner's cust om ers and ret rieve som e part of t he cust om er preferences. This scenario is shown in Figure 10- 9.

Figu r e 1 0 - 9 . Tw o- w a y com m u n ica t ion sce n a r io du r in g t h e ch e ck - in pr oce ss. Th e fligh t 's ope r a t or u se s t h e se r vice s of t h e t ick e t in g com pa n y t o va lida t e fligh t cou pon s a n d r e t r ie ve cu st om e r pr e fe r e n ce s. [View full size image]

When using ext ernal services, care m ust be t aken t o cope wit h t he breakdown of t he ext ernal service. Even if SLAs exist s t hat guarant ee a cert ain availabilit y of t he ext ernal service, net work failures or failures of equipm ent on t he client side should be ant icipat ed. The error should be handled gracefully using t he sam e general principles out lined in Chapt er 9. I n general, t he best t ype of service for t his scenario depends on bot h t he level of cust om er service one want s t o provide and t he t echnical capabilit ies and geographical locat ions of t he airlines. I f t he part ners have weak connect ivit y, it m ight be best t o use t he first opt ion and provide t he necessary account ing inform at ion separat ely from t he act ual process. A com m on reason for choosing t his t ype of solut ion is when lat ency is so high t hat any addit ional server- side processing int roduces unbearable risks on syst em perform ance. Anot her st rong reason for t his scenario is when only cert ain part ners are t echnically capable of operat ing as service providers. I f t he part ners are separat ed by a large dist anceconsider a European and a Sout h Am erican carrieran infrequent synchronizat ion m echanism m ight offer t he best solut ion. I f t he part ners are locat ed close t oget her and can use a st able connect ion wit h sufficient bandwidt h, t hey can use t he t hird scenario. This offers cust om ers t he best service, enabling t hem t o buy a t icket from one airline and check in wit h anot her airline easily. I t also result s in t he best dat a qualit y.

10.4. Fat Clients The t erm " fat client " is a com m on synonym for an applicat ion t ype t hat provides a lot of it s core processes on t he client m achine and usually relies on a single backend syst em , which can be a relat ional dat abase or a host syst em t hat provides som e procedural capabilit ies. Fat client s have a bad reput at ion in I T depart m ent s m ainly because t hey creat ed real challenges for deploym ent and m aint enance. Because t hey aggregat e a lot of com put at ional logic, t hey require frequent rollout s of t he whole applicat ion package t hroughout t he ent erprise. On t he ot her hand, fat client applicat ions have enj oyed a good reput at ion am ong m any end users. They provide swift operat ions com pared t o m any poorly built Web applicat ions t hat have been creat ed t o replace t hem . They also provide com plex int eract ion m odels and advanced graphical capabilit ies t hat are very hard t o re- creat e using Web int erfaces. Therefore, in m any usage scenarios, fat client s are regarded as m ore user- friendly. Alt hough fat client applicat ions need a considerable am ount of deploym ent and m aint enance effort , t hey also help t o save bandwidt h and increase response t im es com pared t o t ypical Web applicat ions. Anot her st rong advant age of fat client s is t hat t hey offer easy access t o local hardware devices such as print ers and sm art card readers, which m akes t hem part icularly well suit ed for use in a cont rolled environm ent wit h m edium t o long release cycles. You can use services t oget her wit h fat client s t o t ransform fat client s t o rich client s, in which t he client applicat ion direct ly accesses different services and keeps t rack of any necessary st at e inform at ion. This works in m uch t he sam e way as for t he Web applicat ion layer in Sect ion 10.1. The core advant age is t hat t he rich client applicat ion can be com plet ely decoupled from t he backend im plem ent at ion. I t t hus rem ains robust in t he face of alt erat ion in t he dat a m odel and ult im at ely against a possible change in t he overall backend infrast ruct ure, such as t he gradual replacem ent of host syst em s wit h int egrat ed services.

Build Rich Clients, Not Fat Clients Usage of t hin client s does not necessarily m ean reduced funct ionalit y. Services can enable you t o build rich client s rat her t han fat client s.

Fat client s can differ in t heir aut hent icat ion schem es. Whereas a Web applicat ion user needs t o aut hent icat e it self t o t he servere.g. using usernam e and passwordfat client s m ight be t rust ed client s or m ight be aut hent icat ed using a cert ificat e. Where backend syst em s do not support t his t ype of aut hent icat ion, t he service m ight present som e default usernam e and password as a proxy user. When using a proxy user, you m ust t ake care t o m inim ize t he result ant securit y risk ( see Chapt er 9) . I n part icular, t he proxy user credent ial should not be st ored in clear t ext ; t hey m ust be configurable, and t he allowed origins of proxy user request should be rest rict ed t o a welldefined set of net work addresses.

Proxy Users Protect Backend Systems Proxy users can isolat e backend syst em s from physical users. Because t his also involves a securit y risk, you m ust t ake ext ra care t o prevent t he m isuse of proxy users.

I n t he check- in exam ple, rich client s include check- in kiosks and workst at ions of check- in clerks. These rich client s access print ers for boarding passes and baggage t ags and are usually equipped wit h card readers for elect ronic t icket s, credit cards, or frequent t raveler cards. Advanced checkin kiosks m ight also connect t o baggage scales and conveyors. They provide int erfaces t hat m ake it as easy as possible t o navigat e across t he seat m ap of t he airplane and t o choose seat s for check- in. Yet t hey do not perform t he act ual check- in process and do not direct ly m anipulat e dat a. I nst ead, t hey use a service layer t hat is usually ident ical t o t he one used by Web applicat ions. Figure 10- 10 shows a t ypical rich client set up.

Figu r e 1 0 - 1 0 . Ex a m ple of a fa t clie n t con t a in in g a sm a r t ca r d r e a de r , lu gga ge sca le , lu gga ge t a g pr in t e r , a n d boa r din g pa ss pr in t e r . Th e fa t clie n t u se s st a n da r d se r vice s t h a t in t u r n r e ly on r e la t ion a l da t a ba se s a n d h ost syst e m s.

10.5. Designing for Small Devices For t he foreseeable fut ure, t he design of services for sm all devices effect ively m eans designing for a specific device. ( For t he purpose of t his chapt er, a sm all device is one wit h a sm all screen of about 150 x 100 pixels, lim it ed processing power, m em ory oft en far less t han 256kB, and lim it ed dat a ent ry support alt hough wit h a capabilit y t o creat e net work connect ions t o facilit at e part icipat ion in an SOA.) One of t he m ost im port ant point s is t hat t he int eract ion pat t ern m ust m at ch t he capabilit ies of t he device. Rich int eract ion pat t erns for different classes of applicat ions oft en fail wit h sm all devices because of t he physical charact erist ics of t he device and also because of addit ional cost and longer int eract ion t im e for t he user im posed by t hese pat t erns. Because net work connect ivit y m ight be at a prem ium , t he service im plem ent at ion it self will usually be as lean as possible t o m inim ize lat ency. I t is likely t hat securit y will be lim it ed t o m ost ly t ransport securit y because processing power and available m em ory m ight not allow for t he encrypt ion and signat ure of m essages as defined in Chapt er 9. Alt hough J2ME MI DP 2.0 support s t ransport securit y using HTTPS, t here is no support in version 1.0. This m eans t hat you m ust use som e ot her m eans t o prevent passing ext ensive credent ials bet ween server and client . Aut hent icat ions can be perform ed in ways such as using t he t elephone num ber, using usernam e/ password, or using cert ificat es st ored in t he client . Alt hough using t he t elephone num ber m ight seem like t he best scenario, t here are a num ber of reasons t o avoid it . For securit y reasons, runt im e of environm ent s such as J2ME MI DP do not enable access t o t elephony funct ionalit y, including t he phone num ber. Furt herm ore, unless t he delivery pat h is cont rolled, t he pot ent ial for abuse of t he t elephone num ber for aut hent icat ion is very high, bot h because it is easy t o t am per wit h and because it is basically a public num ber t hat can be readily obt ained. However, t he phone num ber can be an excellent m eans of aut hent icat ion when using SMS- based services, as we discuss at t he end of t his chapt er. One way or anot her, t his inform at ion will be st ored persist ent ly at t he client . This is m andat ory due t o t he lim it ed dat a ent ry support on sm all devices. A requirem ent t o repeat edly ent er a usernam e and password wit h a clum sy keyboard obst ruct s usabilit y. Alt ernat ively, it is possible t o envisage int eract ion wit hout t he user act ually logging in t o t he syst em at all. The invocat ion m ight be t riggered by a unique I D such as a reservat ion num ber.

Lightweight Security Securit y feat ures oft en require rat her large com put at ional and m em ory resources. Use only securit y m easures t hat have lit t le im pact on bot h t he device and t he user.

Som e sm all devices can use SOAP ( or even CORBA) for rem ot e invocat ions. However, you m ust carefully consider whet her t his is a good idea. For exam ple, SOAP requires resources at alm ost every level of t he sm all device, and a significant am ount of processing power and m em ory is used for XML parsing. I n addit ion, SOAP libraries t hat are not nat ive t o t he device m ust be bundled wit h t he applicat ion, which oft en increases t he size of t he applicat ion beyond what is possible for

t he t arget device. When using SOAP, you m ust also t ake int o account t hat t he specificat ion is st ill evolving. Because im plem ent at ions for m obile devices lag back behind t he current st andard, oft en by m ore t han a year, it is unlikely t hat a st at e- of- t he- art sm all device can properly address st at eof- t he- art SOAP services in t he organizat ion. Therefore, it is sensible t o decouple t he service invocat ion in t he device from t he act ual service using an adapt er t hat resides at a gat eway server. This set up enables t he client t o connect t o t he service using t he least com m on denom inat or in t echnology, usually connect ing using HTTP or HTTPS. I fas wit h MI DP 1.0only HTTP is available, t he server can est ablish a session t hat prevent s t he repeat ed passing of users' credent ials over t he wire. The session t oken ( or login t oken) can be st ored in t he SOAP header or can be sent using a SOAP param et er. I n t he aut hors' experience, m oving from light weight J2ME SOAP im plem ent at ions, such as kSOAP, t o a HTTP- POST st yle int eract ion can reduce t he client size by approxim at ely 20kB. Given t hat t he foot print allowed by J2ME MI DP Midlet Suit es can be as low as 64kB or less, t his is effect ively a huge reduct ion in size t hat in t urn creat es t he opport unit y for a of richer and m ore user- friendly applicat ion.

Minimize Resources for Communication I n sm all devices, use resources for t he applicat ion rat her t han t he com m unicat ion. I t is bet t er t o creat e a prot ocol t ranslat or on t he server side t han t o wit hhold funct ionalit y from t he cust om er due t o resource const raint s.

I n t he check- in exam ple, m ost users will not be ready t o select t he seat s using a sm all screen and a hard- t o- handle keyboard. Using a sm all device, it is m ore likely t hat t he user will check- in right away aft er t he proper flight coupon has been select ed. Because int eract ion using m obile phones is som ewhat cum bersom e, it m ight even be possible t o check in by sim ply t yping t he coupon num ber for which t o perform t he check- in and hit t ing " cont inue." A t ypical int eract ion scenario is shown in Figure 10- 11. I n t his exam ple, t his process is achieved by elim inat ing t he need t o query t he cust om er service and t he cat erer. Not e t hat service granularit ies used by sm all devices t end t o be coarser t han services used by Web applicat ions.

Figu r e 1 0 - 1 1 . Typica l se r vice in t e r a ct ion u sin g a sm a ll de vice .

As we discussed previously, t here are various ways t o aut hent icat e t he user. I n t he check- in exam ple, it is hard t o board t he plane wit hout being t he act ual t icket holder. Thus, it is possible t o check in a single flight coupon based on an I D t hat is print ed on t he coupon, as illust rat ed in Figure 10- 12. I n such a scenario, only t he person holding t he physical flight coupon can perform t he check- in.

Figu r e 1 0 - 1 2 . Ch e ck - in u sin g a m obile de vice . I n t h is e x a m ple , t h e ch e ck - in is pe r for m e d sole ly u sin g t h e cou pon I D .

An alt ernat ive scenario wit h a richer int eract ion pat t ern is shown in Figure 10- 13. I t uses a Web applicat ion t o st ore som e conversat ional st at e, and in t he exam ple, t his is leveraged t o reduce lat ency for t he act ual check- in call. To t hat end, j ust aft er login, it dispat ches an asynchronous request t o t he pot ent ially cost ly operat ion t hat obt ains t he cust om er's of preferences.

Figu r e 1 0 - 1 3 . Usin g a pr ox y W e b a pplica t ion a s a se r vice pr ox y. As w it h W e b a pplica t ion s, t h e pr ox y ca n be u se d for st a t e a ggr e ga t ion . I t ca n a lso dispa t ch som e a syn ch r on ou s ca lls, su ch a s r e qu e st in g a ce r t a in t ype of m e a l, t o r e du ce la t e n cy w it h in t h e a ct u a l ch e ck - in ca lls. [View full size image]

When t arget ing m obile phones, it m ight also be an opt ion not t o act ually inst all t he service on t he m obile phone. Mobile phone t echnology support s opt ions such as short m essage service ( SMS) or it s succeeding t echnology, m ult im edia m essage service ( MMS) , t hat can also be used t o connect t o a service. To check in, t he user can ut ilize t he coupon num ber on t he t icket along wit h a dedicat ed t elephone num ber for checking in. An im m ediat e advant age m ight be t hat t he user can be uniquely ident ified using his phone num ber. However, t his com m unicat ion m odel has cert ain securit y rest rict ions, alt hough it m ight well be t he best opt ion for deploying consum er services such as m ult iplayer role playing gam es or dat ing services ( see Figure 10- 14) .

Figu r e 1 0 - 1 4 . En ga gin g a se r vice u sin g a n SM S pr ox y. Th is ca n pr ovide a cle a n ye t pot e n t ia lly in se cu r e w a y t o ide n t it y t h e cu st om e r .

[View full size image]

Due t o t heir coarse- grained st ruct ure, services t hat are used from sm all devices are an excellent opt ion for ot her usage scenarios whenever som e resources are at a prem ium . An exam ple is a check- in t erm inal t hat needs t o funct ion using lim it ed net work connect ivit y.

10.6. Multi-Channel Applications SOAs are ext rem ely effect ive at im plem ent ing m ult i- channel applicat ions. St rengt hs such as building a reusable, funct ional infrast ruct ure, m ast ering het erogeneit y, and providing easy access t o dat a and business logic are key fact ors for successful m ult i- channel proj ect s. A m ult i- channel applicat ion is charact erized by a funct ional core t hat can be accessed t hrough different channels by eit her hum an users or program s, as shown in Figure 10- 15. Each channel t ypically has it s own charact erist ic variant of t he basic business processes and specific access t echnology. These specifics depend direct ly on requirem ent s of t he channel's users such as sales depart m ent , back office, or service cent er. Due t o t he different requirem ent s, you will oft en find significant differences in t he channels' business processes. For exam ple, a sales represent at ive will require a different view of cust om er dat a from a back office clerk or from a service cent er agent . The processes also depend on t he access t echnology. As we have already discussed in t he previous sect ion, you will probably find t hat t he processes on a m obile device wit h a low bandwidt h connect ion t o t he funct ional core and a sm all display are different from t he processes t hat can be support ed by a Web applicat ion.

Figu r e 1 0 - 1 5 . A m u lt i- ch a n n e l a pplica t ion con sist s of a fu n ct ion a l cor e t h a t is sh a r e d by se ve r a l ch a n n e ls. Ea ch ch a n n e l pr ovide s a spe cific vie w of cor e fu n ct ion a lit y, a ccor din g t o t h e r e qu ir e m e n t s of t h e ch a n n e l u se r a n d t h e ch a n n e l t e ch n ology.

Typical m ult i- channel applicat ions can provide channels such as t he I nt ernet , various m obile channels such as WAP, B2B, fat client s, call cent ers, or voice applicat ions. They also provide cross- channel funct ionalit y, such as co- browsing or channel swit ching.

10.6.1. FUNDAMENTAL SOA A fundam ent al SOA represent s t he sim plest m odel of SOA- based m ult i- channel applicat ions ( see Chapt er 6, " The Archit ect ural Roadm ap" ) . I n m any cases, t his sim plicit y m akes it t he preferred choice of m odel. I n t his scenario, you have services neit her in t he aggregat ion layer nor in t he process layer. Consequent ly, you can also abandon ext ra t iers and t he appropriat e hardware, which can lead t o reduced lat ency and im proved applicat ion response t im es. The aut hors recom m end t he usage of t his m inim al approach where possible. This advice does not apply only t o m ult i- channel applicat ions, eit her. I t is beneficial for every SOA t o keep operat ions as sim ple as possible. Abandoning com plexit y should be a perm anent effort . Every t ier t hat can be avoided is a valuable cont ribut ion t o t he syst em 's m aint ainabilit y and perform ance ( see Figure 10- 16) .

Figu r e 1 0 - 1 6 . Th e fu n da m e n t a l SOA r e pr e se n t s a le a n se r vice a ppr oa ch t h a t ba sica lly im plie s t h e u sa ge of a pplica t ion fr on t e n ds a n d ba sic se r vice s. N o se r vice s in t h e a ggr e ga t ion la ye r or pr oce ss la ye r a r e in volve d. [View full size image]

Alt hough a lean approach is desirable, you m ight find reasons t o apply a m ore com plex approach in pract ice. Requirem ent s such as co- browsing or channel swit ching, highly com plex processes, het erogeneous backend syst em s, load balancing, and ot her reasons can force t he usage of façades or process- cent ric services.

10.6.2. SERVICE FA