Cisco ISP Essentials 1-57870-0233-X

Citation preview

Cisco I SP Esse nt ia ls Barry Raveendran Greene Philip Sm it h Publisher : Cisco Pr ess First Edit ion April 19, 2002 I SBN: 1- 58705 -0 4 1- 2, 448 pages

Fr on t Mat t er Table of Cont ent s About t he Aut hor

Cisco I OS( r ) Soft w ar e docum ent at ion is ex t ensiv e and det ailed and is oft en t oo har d for m any I nt er net ser v ice pr ov ider s ( I SPs) w ho sim ply w ant t o sw it ch on and get going. Cisco I SP Essent ials highlight s m any of t he k ey Cisco I OS feat ur es in ev er y day use in t he m aj or I SP backbones of t he world t o help new net w or k engineer s gain under st anding of t he pow er of Cisco I OS Soft w ar e and t he r ichness of feat ur es av ailable specifically for t hem . Cisco I SP Essent ials also pr ov ides a det ailed t echnical r efer ence for t he ex per t I SP engineer , w it h descr ipt ions of t he v ar ious k nobs and special feat ur es t hat hav e been specifically designed for I SPs. The configur at ion ex am ples and diagr am s descr ibe m any scenar ios, r anging fr om good oper at ional pr act ices t o net w or k secur it y . Finally a w hole appendix is dedicat ed t o using t he best pr inciples t o cov er t he configur at ion det ail of each r out er in a sm all I SP Point of Pr esence. This book is part of t he Cisco Press Net w orking Technologie s Ser ies, w hich offer s net w or k ing pr ofessionals v aluable infor m at ion for const r uct ing efficient net w or k s, under st anding new t echnologies, and building successful car eer s.

1

Cisco® I SP Essent ials Published by: Cisco Press 201 West 103rd St reet I ndianapolis, I N 46290 USA All right s reserved. No part of t his book m ay be reproduced or t ransm it t ed in any form or by any m eans, elect ronic or m echanical, including phot ocopying, recording, or by any inform at ion st orage and ret rieval syst em , wit hout writ t en perm ission from t he publisher, except for t he inclusion of brief quot at ions in a review. Pr in t e d in t h e Un it e d St a t e s of Am e r ica 1 2 3 4 5 6 7 8 9 0 Fir st Pr int ing Apr il 2 0 0 2 Libr a r y of Con gr e ss Ca t a login g- in- Pu blica t ion N u m be r : 2001090435 W a r n in g a n d D iscla im e r

This book is designed t o provide inform at ion about best com m on pract ices for I nt ernet service providers ( I SPs) . Every effort has been m ade t o m ake t his book as com plet e and as accurat e as possible, but no warrant y or fit ness is im plied. The inform at ion is provided on an “ as is” basis. The aut hors, Cisco Press, and Cisco Syst em s, I nc., shall have neit her liabilit y nor responsibilit y t o any person or ent it y wit h respect t o any loss or dam ages arising from t he inform at ion cont ained in t his book or from t he use of t he discs or program s t hat m ay accom pany it . The opinions expressed in t his book belong t o t he aut hor and are not necessarily t hose of Cisco Syst em s, I nc. Fe e d b a ck I n f or m a t io n

At Cisco Press, our goal is t o creat e in- dept h t echnical books of t he highest qualit y and value. Each book is craft ed wit h care and precision,

2

undergoing rigorous developm ent t hat involves t he unique expert ise of m em bers from t he professional t echnical com m unit y. Readers’ feedback is a nat ural cont inuat ion of t his process. I f you have any com m ent s regarding how we could im prove t he qualit y of t his book, or ot herwise alt er it t o bet t er suit your needs, you can cont act us t hrough e- m ail at [email protected]. Please m ake sure t o include t he book t it le and I SBN in your m essage. We great ly appreciat e your assist ance. Tr a d e m a r k Ack n ow le d g m e n t s

All t erm s m ent ioned in t his book t hat are known t o be t radem arks or service m arks have been appropriat ely capit alized. Cisco Press or Cisco Syst em s, I nc., cannot at t est t o t he accuracy of t his inform at ion. Use of a t erm in t his book should not be regarded as affect ing t he validit y of any t radem ark or service m ark. Pu b lisher John Wait Edit or- I n- Ch ie f John Kane Cisco Syst e m s Pr ogr a m M a n a ge m e n t Michael Hakkert Tom Geit ner William War r en Acquisit ions Edit or Am y Lew is Pr odu ct ion M a n a ge r Pat rick Kanouse D e ve lopm e nt M a na ge r

3

How ard Jones Pr oj e ct Edit or San Dee Phillips Copy Edit or Krist a Hansing Te ch n ica l Edit or s Brian Morgan and Bill Wagner Te a m Coor dina t or Tam m i Ross Book D e signe r Gina Rexrode Cove r D e sign e r Louisa Klucznik Pr odu ct ion Te a m Oct al Publishing, I nc. I nde x e r Tim Wright Cor por a t e H e a dqu a r t e r s Cisco Syst em s, I nc. 170 West Tasm an Drive San Jose, CA 95134- 1706 USA ht t p: / / w w w .cisco.com 4

Tel: 408 526- 4000 800 553- NETS ( 6387) Fax: 408 526- 4100 Eu r ope a n H e a dqu a r t e r s Cisco Syst em s Europe 11 Rue Cam ille Desm oulins 92782 I ssy- les- Moulineaux Cedex 9 France ht t p: / / www- eur ope.cisco.com Tel: 33 1 58 04 60 00 Fax: 33 1 58 04 61 00 Am e r ica s H e a dqu a r t e r s Cisco Syst em s, I nc. 170 West Tasm an Drive San Jose, CA 95134- 1706 USA ht t p: / / w w w .cisco.com Tel: 408 526- 7660 Fax: 408 527- 0883 Asia Pa cific H e a dqu a r t e r s Cisco Syst em s Aust ralia, Pt y ., Lt d 5

Level 17, 99 Walker St reet Nort h Sydney NSW 2059 Aust ralia ht t p: / / w w w .cisco.com Tel: + 61 2 8448 7100 Fax: + 61 2 9957 4350 Cisco Syst e m s h a s m or e t h a n 2 0 0 office s in t h e follow in g cou n t r ie s. Addr e sse s, ph on e n u m be r s, a n d fa x n u m be r s a r e list e d on t h e Cisco W e b sit e a t www.cisco.com / go/ offices Argent ina • Aust ralia • Aust ria • Belgium • Brazil • Bulgaria • Canada • Chile • China • Colom bia • Cost a Rica • Croat ia • C zech Republic • Denm ark • Dubai, UAE • Finland • France • Germ any • Greece • Hong Kong Hungary • I ndia • I ndonesia • I reland • I srael • I t aly • Japan • Korea • Luxem bourg • Malaysia • Mexico The Net herla nds • New Zealand • Norway • Peru • Philippines • Poland • Po rt ugal • Puert o Rico • Rom ania Russia • Saudi Arabia • Scot land • S ingapore • Slovakia • Slovenia • Sout h Africa • Spain • Sweden Swit zerland • Taiwan • Thailand • Turkey • Ukraine • Unit ed Kingd om • Unit ed St at es • Venezuela • Viet nam Zim babwe Copyright © 2000, Cisco Syst em s, I nc. All right s reserved. Access Regist rar, AccessPat h, Are You Ready, ATM Direct or, Browse wit h Me, CCDA, CCDE, CCDP, CCI E, CCNA, CCNP, CCSI , CD- PAC, CiscoLink, t he Cisco Net Works logo, t he Cisco Powered Net work logo, Cisco Syst em s Net working Academ y, Fast St ep, FireRunner, Follow Me Browsing, Form Share, GigaSt ack, I GX, I nt elligence in t he Opt ical Core, I nt ernet Quot ient , I P/ VC, iQ Breakt hrough, iQ Expert ise, iQ Fast Track, iQuick St udy, iQ Readiness Scorecard, The iQ Logo, Kernel Proxy, MGX, Nat ural Net work Viewer, Net work Regist rar, t he Net workers logo, Packet , PI X, Point and Click I nt ernet working, Policy Builder, Rat eMUX, ReyMast er, ReyView, Script Share, Secure Script , Shop wit h Me, SlideCast , SMARTnet , SVX, TrafficDirect or, TransPat h, VlanDirect or, Voice LAN, Wavelengt h Rout er, Workgroup Direct or, and Workg roup St ack are t radem arks of Cisco Syst em s, I nc.; Changing t he Way We Work, Live, Play, and Learn, Em powering t he I nt ernet Generat ion, are service m arks of Cisco Syst em s, I nc.; and Aironet , ASI ST, BPX, Cat alyst , Cisco, t he Cisco Cert ified I nt ernet work Expert Logo, Cisco

6

I OS, t he Cisco I OS logo, Cisco Press, Cisco Syst em s, Cisco Syst em s Capit al, t he Cisco Syst em s logo, Collision Free, Ent erprise/ Solver, Et herChannel, Et herSwit ch, Fast Hub, Fast Link, Fast PAD, I OS, I P/ TV, I PX, Light St ream , Light Swit ch, MI CA, Net Ranger, Post - Rout ing, PreRout ing, Regist rar, St rat aView Plus, St rat m , Swit chProbe, TeleRout er, are regist ered t radem arks of Cisco Syst em s, I nc. or it s affiliat es in t he U.S. and cert ain ot her count ries. All ot her brands, nam es, or t radem arks m ent ioned in t his docum ent or Web sit e are t he propert y of t heir respect ive owners. The use of t he word part ner does not im ply a part nership relat ionship bet ween Cisco and any ot her com pany. ( 0010R)

Cisco® I SP Essent ials About t he Aut hors About t he Technical Review ers Ack now ledgm ent s I nt r oduct ion Mot ivat ion I nt ended Audience Organizat ion Fur t her I nfor m at ion 1. Soft w ar e and Rout er Managem ent Which Cisco I OS Soft ware Version Should I Be Using? I OS Soft w ar e Managem ent Configur at ion Managem ent Com m and- Line I nt er face Det ailed Logging Net w or k Tim e Pr ot ocol Sim ple Net w or k Managem ent Pr ot ocol HTTP Ser v er Core Dum ps Conclusion Endnot es 2. Gener al Feat ur es I OS Soft w are and Loopback I nt er faces I nt erface Configurat ion I nt erface St at us Checking Cisco Express Forw arding Net Flow Turn On Nagle DNS and Rout ers Conclusion

7

Endnot es 3. Rout ing Pr ot ocols CI DR Feat ur es Select ive Packet Discard Hot St andby Rout ing Pr ot ocol I P Source Rout ing Configuring Rout ing Prot ocols I GP Configur at ion Hint s The BGP Pat h- Select ion Pr ocess [ 1] BGP Feat ures and Com m ands Applying Policy wit h BGP BGP Policy Account ing Mult iprot ocol BGP [ 5] Sum m ary Endnot es 4. Secur it y Securing t he Rout er Unneeded or Risk y I nt er face Ser v ices Cisco Discov er y Pr ot ocol Login Banners Use enable secret The ident Feat ur e SNMP Secur it y Rout er Access: Cont r olling Who Can Get int o t he Rout er Secur ing t he Rout ing Pr ot ocol Securing t he Net work Access Cont r ol List s: Gener al Sequent ial- Based ACLs BCP 38 Using Unicast RPF [ 10] Com m it t ed Access Rat e t o Rat e- Lim it or Drop Packet s [ 21] React ing t o Secur it y I ncident s Sum m ary Endnot es 5. Oper at ional Pr act ices Point - of- Pr esence Topologies Point - of- Pr esence Design Backbone Net work Design I SP Ser vices I Pv4 Addressing in an I SP Backbone I nt erior Rout ing Ext erior Rout ing Mult ihom ing Secur it y Out - of- Band Managem ent Test Laborat ory Oper at ional Consider at ions Sum m ary Endnot es A. Access List s and Regular Ex pr essions Access List Ty pes

8

I OS Soft w ar e Regular Expr essions Endnot es B. Cut - and- Past e Tem plat es Gener al Sy st em Tem plat e Gener al I nt er face Tem plat e Gener al Secur it y Tem plat e Gener al iBGP Tem plat e Gener al eBGP Tem plat e Mar t ian and RFC 1918 Net w or ks Tem plat e C. Ex am ple Configur at ions Sim ple Net work Plan Configurat ions Sum m ary D. Rout e Flap Dam ping BGP Flap Dam ping Configurat ion E. Traffic Engineering Tools I nt ernet Traffic and Net w ork Engineering Tools Ot her Useful Tools t o Manage Your Net w or k Overall I nt ernet St at us and Perform ance Tools What Ot her I SPs Are Doing Sum m ary F. Exam ple I SP Access Secur it y Migr at ion Plan Phase 1—Close Off Access t o Ev er y one Out side t he CI DR Block Phase 2—Add Ant ispoofing Filt er s t o Your Peer s Phase Thr ee—Close Off Net w or k Equipm ent t o Unaut hor ized Access Sum m ary Endnot es Glossary Glossary Technical Refer ences and Recom m ended Reading Soft w ar e and Rout er Managem ent General Feat ures Secur it y Rout ing Ot her References and Recom m ended Reading

9

About t he Aut hor s Ba r r y Ra ve e n dr a n Gr e e n e is a Senior Consult ant in t he I nt er net Ar chit ect ur es Gr oup of Consult ing Engineer ing, Office of t he CTO, Cisco Sy st em s. Cisco’s CTO Consult ing gr oup assist I SPs t hr oughout t he w or ld t o scale, gr ow , and ex pand t heir net w or k s. The assist ance is deliv er ed t hr ough consult ing, dev eloping new feat ur es, w or k ing new st andar ds ( I ETF and ot her gr oups) , and pushing for w ar d Best Com m on Pr act ices ( BCPs) t o t he I nt er net com m unit y . Bar r y ’s cur r ent t opics of int er est s ar e I SP Oper at ions and Secur it y as w ell as dev eloping t he feat ur es, funct ionalit y , and t echniques t o enhance an I SP’s success. Bar r y has been w it h Cisco since 1996, t r aveling t o all par t s of t he w or ld helping I SPs and t elcos build t he I nt er net . He is a for m er boar d m em ber for t he Asia Pacific I nt er net Associat ion ( API A) , co- cr eat or for t he APRI COT Confer ences, Pr ogr am Com m it t ee Mem ber for I TU’s Telecom 99, and facilit at or for t he cr eat ion of sev er al I nt er net eXchange Point s ( I XPs) in Asia and Pacific. Bar r y is t he co- coordinat or for Cisco’s I SP Wor k shop Pr ogr am , w hich is designed t o em pow er engineer ing t alent in I SPs all over t he w or ld. Mr . Gr eene has ov er 22 y ear s ex per ience in sy st em s int egr at ion, secur it y , oper at ions, m aint enance, m anagem ent , and t r aining on a var iet y of com put er , int er net w or k ing, and t elecom m unicat ions t echnologies. Befor e Cisco Sy st em s, Bar r y w as Deput y Dir ect or Planning and Oper at ions for Singapor e Telecom ’s SingNet I nt er net Ser v ice and t he Singapor e Telecom I nt er net Ex change ( STI X) ; Net w or k Engineer and Sy st em s I nt egr at or at Johns Hopk ins Univ er sit y / Applied Phy sics Lab ( JHU/ APL) , Net w or k Engineer and Sy st em s I nt egr at or Science Applicat ion I nt er nat ional Cor por at ion ( SAI C) , and a v et er an of t he Unit ed St at es Air For ce. Ph ilip Sm it h is a Consult ing Engineer in t he I nt er net Ar chit ect ur es Gr oup of Consult ing Engineer ing, Office of t he CTO, Cisco Sy st em s. His role includes providing consult at ion and adv ice t o I SPs pr im ar ily in t he Asia Pacific r egion and also w it h ot her pr ov ider s ar ound t he w or ld. He concent r at es specifically on net w or k st r at egies, design, t echnology , and oper at ions, as w ell as configu r at ion, scaling, and t r aining. He play s or has play ed a m aj or r ole in t r aining I SP engineer s, co- founding t he Cisco I SP/ I XP Wor k shop pr ogr am m e, and pr ov iding I SP t r aining and t ut or ials at m any net w or king event s ar ound t he w or ld, including NANOG, RI PE, APNI C, I SOC, and APRI COT confer ences. His ot her key int er est s include I Pv6, BGP, I GPs, and net w or k per for m ance and dat a analy sis. Philip has been w it h Cisco since Januar y 1998. Since j oining, he has been w or k ing t o pr om ot e and develop t he I nt er net in t he ent ir e Asia Pacific r egion and has been act iv ely inv olv ed in br inging t he I nt er net t o som e count r ies in t he r egion. He is a m em ber of t he APRI COT Ex ecut iv e Com m it t ee ( t he r egion’s annual I SP oper at ional and t echnology confer ence) as w ell as it s Pr ogr am m e Com m it t ee, co- chair of APOPS ( t he r egion’s I SP oper at ional for um ) , and chair of t w o of APNI C’s special int er est gr oups ( SI G) —t he Rout ing SI G and t he Exchange Point SI G. He also has a part icular r esear ch int er est in t he gr ow t h of t he I nt er net and pr ov ides a det ailed daily analysis of t he r out ing t able as seen in t he Asia Pacific t o t he gener al oper at or com m unit y w or ldw ide. Pr ior t o j oining Cisco, he spent fiv e y ear s at PI PEX ( now par t of UUNET’s global I SP business) , t he UK’s fir st com m er cial I SP, w her e he w as Head of Net w or k

10

Engineer ing. As is com m on w it h st ar t ups in a r apidly gr ow ing m ar ket place, Philip gained deep experience in all of t he engineering roles in an I SP, from support engineer , net w or k oper at ions, engineer ing, and dev elopm ent , befor e assum ing responsibilit y for t he ent ir e UK net w or k oper at ion. He w as one of t he fir st engineer s w or k ing in t he com m er cial I nt er net in t he UK, and he helped est ablish t he LI NX I nt ernet Exchange Point in London and played a key role in building t he m odern I nt ernet in Europe. Philip is a Doct or of Philosophy and has a Fir st Class Honour s Degr ee in Physics. A nat ive of Scot land, he now lives in Br isbane, Aust r alia.

Abou t t h e Te ch n ica l Re v ie w e r s Bill W a g n e r w or ks as a Cisco Cer t ified Syst em s I nst r uct or ( CCSI ) . He has over 22 y ear s of com put er pr ogr am m ing and dat a com m unicat ion ex per ience. He has w or k ed for cor por at ions and com panies such as I ndependent Com put er Consult ant s, Num er ax , McGr aw - Hill/ Num er ax , and St andar d and Poor s. His t eaching ex per ience st ar t ed w it h t he Chubb I nst it ut e, Pr ot ocol I nt er face, I nc., and Geot r ain. Bill is also a t echnical edit or for num er ous ot her Cisco Pr ess t it les. Br ia n M or g a n , CCI E # 4 8 6 5, CCSI , is t he Dir ect or of Dat a Net w or k Engineer ing at Allegiance Telecom , I nc. He’s been in t he net w or king indust r y for over 12 years. Pr ior t o w or k ing at Allegiance, Br ian w as an I nst r uct or / Consult ant t eaching I CND, BSCN, BSCI , CATM, CVOI CE, and BCRAN. Br ian is a co- aut hor of Cisco Press’s CCNP Rem ot e Access Ex am Cer t ificat ion Guide and t echnical edit or of num er ous ot her Cisco Pr ess t it les.

Ack now le dgm e nt s This book st ar t ed life as a sm all w hit epaper called “ I OS Essent ials,” an at t em pt t o docum ent t he v ar ious configur at ion and oper at ional best pr act ices w hich I SPs w er e using on t heir Cisco net w or k ing equipm ent . This w hit epaper has, ov er t he last few y ear s, gr ow n t hr ough sev er al v er sions int o t his book , Cisco I SP Essent ials . We w ould like t o t hank t he num er ous fr iends and colleagues in t he indust r y w ho hav e cont r ibut ed t o bot h t he w hit epaper and t his book . Many hav e cont ribut ed t heir ow n t ex t , m ade num er ous suggest ions, cont r ibut ions, and clar ificat ions, and also hav e pr ov ided t heir ow n deep r eal w or ld oper at ional ex per ience w it h t he I nt er net . Their w illingness t o help ot her s do t he r ight t hing is one of t he r easons for t he I nt er net ’s success. We’d also like t o t hank How ard Jones, our Developm ent Edit or, for t he help and suppor t he has given us. Thanks ar e also due t o Am y Lew is and John Kane of Cisco Pr ess for encour aging us and suppor t ing us t o m ake t his book possible. Ba r r y Rav eendr an Gr eene and Philip Sm it h

11

I nt roduct ion The I nt er net econom y has play ed a significant par t in t he w or ld econom y since t he m id 1990s. For m any y ear s pr ior , t he I nt er net w as t he dom ain of U.S. academ ic r esear ch and defense int er net w or k ing, and a few ent r epr eneur s ar ound t he w or ld w ho believ ed t hat a TCP/ I P- based wide- ar ea net w or k ( WAN) w ould be a viable alt er nat iv e t o t he pr iv at e w ir e net w or k s t hat businesses w er e using t o com m unicat e w it h each ot her . The m any I SP engineer s w ho lear ned t heir skills in t hat period look on t hose ear ly pioneer ing day s at t he end of t he 1980s and ear ly 1990s as som et hing special. Wor k w as inv ar iably har d, and t echnology challenges w er e seem ingly insur m ount able w hen com par ed w it h t he r elat iv e ease of use and configur at ion t hese day s. But t he sense of com pet it ion w as m or e a fr iendly r iv alr y and par t ner ship t o m ak e t he fledgling I nt er net a fun place t o be. This pioneer ing spir it , and t he desir e of t he I nt er net com m unit y t o m ak e t he I nt er net a success, has r esult ed in t he I nt er net becom ing t he m aj or par t of our liv es at t he st ar t of t he 21st cent ur y . I t ’s now a huge com m er cial net w or k , v er y com pet it iv e, w it h m any player s, sm all and lar ge, fr om all over t he w or ld, heavily involved in it s infr ast r uct ur e. Mor e people ar e t aking par t in t he I nt er net ev er y day , bot h end user s w it h t heir fir st com put er connect ing t o t he Wor ld Wide Web, and new I SPs anx ious t o becom e par t of a v er y significant gr ow t h indust r y . Fur t her m or e, t he few r em aining count r ies in t he w or ld w it hout an I nt er net connect ion ar e inv est igat ing connect ing up and ex am ining t he adv ant ages being net w or k ed w ill offer t heir local econom ies. As t he I nt er net has gr ow n in our day- t o- day consciousness, so hav e t ex t book s aspir ing t o help new com er s find t he pr ov er bial pot of gold: books ranging from beginner guides t o designing w eb pages, t o ex plaining w hat t he I nt er net is, t o descr ibing t he business pr ocess, t o becom ing a successful I SP. How ev er , t her e has been pr ecious lit t le t hat descr ibes t he configur at ion concept s and t r icks of t he t r ade t hat I SP net w ork engineers use in t heir daily lives—t her e is an ar gum ent w hich say s, “ We hav e been t oo busy fix ing t he pot holes in t he I nt er net super highw ay t o act ually spend t im e w r it ing dow n w hat w e do.”

M ot iva t ion The inspirat ion for t his book has com e fr om t hr ee sour ces. The fir st is Cisco I OS Soft w ar e. Cisco has been par t of t he I nt er net since t he I nt er net st ar t ed, building one of t he fir st dev ices t o m ov e I P pack et s acr oss a WAN. Ov er t he y ear s, I OS has gr ow n fr om being a fair ly basic piece of soft w ar e t o a highly sophist icat ed, feat ur e r ich, and ex t r em ely pow er ful r out er cont r ol suit e. A t r em endous r ange of feat ur es has been built int o t he I OS. This ex t ensiv e feat ur e set is ex cellent for public net w or k I SPs, giving t heir net w or k enginee r s a lar ge num ber of opt ions and capabilit ies t hat can be designed int o t he net w or k. While a huge num ber of feat ur es m ay be desir able, I OS also poses a pr oblem— net w or k engineer s busy r unning t heir net w or ks have a difficult t im e keeping up w it h all t he new I OS feat ur es. Many engineer s, ev en ex per ienced net w or k engineer s, do not know how , w hen, and w her e t o deploy t he var ious feat ur es in t heir net w or k. This book highlight s m any of t he key I OS feat ures in everyday use in t he m aj or I SP backbones of t he w or ld. Judicious st udy and im plem ent at ion of t he I OS pear ls

12

cont ained in t his book w ill help t o pr event pr oblem s, incr ease secur it y, im pr ove per for m ance, and ensur e t he oper at ional st abilit y of t he I nt er net . The second sour ce of inspir at ion for w r it ing t his book is t hat t her e is no com plet e r efer ence t ex t for new com er s t o t he indust r y t o t ak e r out er pr oduct s and build an I SP net w or k fr om t hem . Ther e is a gr eat deal of docum ent at ion about net w or k design pr act ices, discussing I SP business pr act ices, configur ing t he v arious rout ing pr ot ocols, and all t he higher lev el ser v ices t hat r ide on t he I nt er net . Such t ex t s as I nt er net Rout ing Ar chit ect ur es, Second Edit ion, and t he I SP Survival Guide hav e helped m any I SPs deal w it h scaling t heir backbones and get t ing t he m ost fro m t heir I SP businesses. But w hen a new com er is faced w it h a blue/ green m et al cased box fr esh out of it s packing box, a CD- ROM w it h all t he docum ent at ion for t his piece of equipm ent , t her e is t he sink ing feeling of “ w hat happens now ?” The int ent ion of t his book is t o guide bot h new com er s and ex per ienced net w or k engineer s t hr ough t he opt im al configur at ion of t hat blue/ gr een box and it s par ent net w or k , t o int egr at e it effect iv ely and secur ely int o an I SP net w or k , and t o be par t of t he I nt er net backbone. Th e f inal sour ce of inspir at ion has alr eady been t ouched on in t his int r oduct ion: We all have been t oo busy w or king t o w r it e dow n w hat w e ar e doing. Our daily w or king liv es include out r each t o new pr ov ider s, helping ex ist ing pr ov ider s m ak e t heir net w or ks bet t er , and so on. Ther e is m uch r epet it ion of concept s w hich ar e obvious, but not docum ent ed in a gener al t ex t . The “ I OS Essent ials Whit epaper ” st ar t ed t his all off, docum ent ing special Cisco I OS feat ur es t hat w er e in use alm ost exclusively in t he I SP indust r y . Many fr iends and colleagues in t he indust r y hav e encour aged us t o w r it e a book based ar ound t he w hit epaper , put t ing our ex per iences and k now ledge gained in t he indust r y since t he ear ly 1990s dow n on paper so t hat ot her s can benefit .

I n t e n de d Au die n ce Th e recom m endat ions w e m ak e in t his book focus on I SPs. The r ecom m endat ions ar e not int ended for ot her t y pes of net w or k s, w het her pr iv at e I nt er net s or ent er pr ise net w or k s connect ing t o t he I nt er net , alt hough w e ar e sur e t hat som e of t he ideas and suggest ions w e m ak e her e could be applied successfully t o such net w or k s as well. Engineer s w or king for I SPs w ill benefit m ost fr om t his t ext . ( All engineer s w ill benefit , be t hey engineer s w or k ing in t he I SPs Net w or k Oper at ions Cent er , w or k ing in t he Cust om er Help Desk, or w or k ing on t he cor e back bone it self.) All br anches of an I SP engineer ing funct ion w ill be ex posed t o t he issues and concept s cov er ed in t his book, and w e hope t hat t his w ill be a valuable r efer ence for ever yone. The final chapt er also has som e r elevanc e t o t he m or e business- or ient at ed side of t he I SP. Quit e oft en, in our exper ience, planning a net w or k is not t r eat ed quit e as ser iously as it should be. Planning is a j oint effor t bet w een net w or k engineer s and business m anager s t o ensur e t he best com pr om is e bet w een net w or k design and t he funding av ailable t o pay for it .

13

Or ga n iz a t ion Ther e ar e fiv e chapt er s as w ell as sev er al appendix es aim ed t o giv e t he r eader fur t her infor m at ion, t ips, and t em plat es r elat ing t o w hat has been cov er ed in t he paper . These chapt er s cov er t he follow ing t opics: Chapt er 1: Soft w ar e and Rout er Managem ent : I nt r oduces t he r e ader t o Cisco I OS, t he im age t r ains designed specifically for I SPs, and how t o m anage t hese on t he r out er . This chapt er also cov er s r out er m anagem ent , including configur at ion m anagem ent , t he com m and- line int er face, and handling t he st at us infor m at ion, whic h t he r out er can m ak e av ailable t o it s oper at or s. Chapt er 2: General Feat ures: I nt r oduces t he various m iscellaneous feat ures an I SP r equir es t o or ganize on t he r out er pr ior t o dealing w it h r out ing pr ot ocols and net w or k secur it y . These feat ur es include t he loopback int er face, int er face configur at ion good pr act ices, CEF, and Net Flow . Chapt er 3: Rout ing Pr ot ocols : Cov er s t he m aj or issues facing I SPs w it h t he configur at ion and feat ur e set av ailable w it h t he m aj or r out ing pr ot ocols. These include HSRP, I GP design, and t he ext ensive feat ur e set now available w it h t he BGP im plem ent at ion in Cisco I OS. Chapt er 4: Secu r it y: Cov er s t he cur r ent m aj or secur it y feat ur es and suppor t on t he r out er , and giv es an ex t ensiv e discussion on t he feat ur e set av ailable for defeat ing DoS at t ack s, w hich ar e so pr ev alent on t he I nt er net t oday . Topics cov er ed include r out er access, r out ing pr ot ocol secur it y , and net w or k secur it y . Feat ur es discussed include applicat ions of Unicast RPF and CAR. Chapt er 5: Oper at ional Pr act ices: The final chapt er cov er s how t he pr ev ious four chapt er s m esh t oget her t o help build an I SP back bone. The t ex t concent r at es on worki ng t hr ough t he t y pical pr ocesses used for building an I SP back bone, all t he w ay from net w ork design and layout , t o posit ioning and im plem ent ing higher level ser v ices. Ap p e n d ix e s: The appendix es pr ov ide addit ional r efer ence m at er ial or ex am ples t o supplem ent t he cont ent of each chapt er . I ncluded her e ar e r out e flap dam ping configur at ions, an ext ensive list of popular net w or k m anagem ent and m onit or ing t ools in use, plus a w or k ing sam ple configur at ion of a sim ple I SP net w or k using t he I OS pr inciples cover ed in t his book. The book is best r ead in or der , because each chapt er assum es k now ledge of t he cont ent cov er ed in pr ev ious chapt er s. Ex per ienced engineer s m ight be quit e happy dipping int o t he t ex t as t hey see fit . The st y le of t he book is int ended t o allow bot h ex per t s and beginner s t o feel at ease w it h t he cont ent .

Fu r t h e r I n for m a t ion This book does not set out t o sum m ar ize t he r at her copious docum ent at ion and ot her excellent m at er ials Cisco has m ade available t o t he gener al public as w ell as t o it s cust om ers. The book is based upon t he “ I OS Essent ials Whit epaper ,” w her e w e have collect ed pr elim inar y docum ent at ion of feat ur es as t hey ar e r eleased, or w e hav e

14

w r it t en our ow n ex planat ion, as no docum ent at ion ex ist ed at all. Quit e oft en Cisco’s ow n docum ent at ion has follow ed m uch lat er t han it s fir st appear ance in “ I OS Essent ials.” Along w it h t his book, t he aut hor s ar e m aint aining a w eb sit e w it h w hit epaper updat es t o t he cont ent s of t his book — ht t p: / / w w w . ispbook . com. The w eb sit e ht t p: / / w w w .cisco.com / public/ cons/ isp also cont ains ot her r efer ence m at er ials t hat m ay be useful for I SPs. Wher e t opics ar e not appar ent ly cov er ed in sufficient t echnical dept h, t he reader is encour aged t o consult t he follow ing r efer ence sour ces: • • • •

Cisco Sy st em ’s Docum ent at ion ( av ailable t o t he gener al public on Cisco’s w ebsit e at ht t p: / / w w w .cisco.com / univ er cd/ ) or on t he CD- ROM t hat com es w it h each rout er. Cisco. com. Local Cisco Sy st em s’ suppor t channels. Public discussion list s. The list t hat focuses specifically on I SPs t hat use Cisco Sy st em s equipm ent is cisco- nsp host ed by Jar ed Mauch. cisco- nsp is a m ailing list w hich has been cr eat ed specifically for I SPs t o discuss Cisco Sy st em s pr oduct s and t heir use. To subscr ibe, send an e- m ail t o: m aj or dom o@puck .net her .net with a m e ssage body cont aining: subscr ibe ci sco- nsp

15

Cha pt er 1 . Soft w a re a nd Rout er M a na gem ent This chapt er cov er s m any of t he basic quest ions t hat I SPs ask w hen t hey ar e fir st faced w it h set t ing up r out er s for t heir I nt er net business. Alt ho ugh docum ent at ion shipped w it h any it em of equipm ent pr ov ides a v er y com pr ehensiv e descr ipt ion of set up pr ocesses, m or e ex per ienced I SPs usually hav e dev eloped a m et hodology for how new har dw ar e is deploy ed on a liv ing back bone. Oft en t he v endor ’s w ellint ent ioned st ar t up pr ocess for new user s becom es m or e of a hindr ance or inconv enience in t hese sit uat ions. This chapt er does not pr ov ide an alt er nat iv e t o t he r ecom m endat ions, but it suggest s w hat I SPs should consider as t he init ial configur at ion phase for t heir net w or k equipm ent and t he pr ocesses t hat t hey should follow dur ing t heir business oper at ion. The fir st por t ion of t he chapt er cov er s t he Cisco I OS Soft w ar e and som e of t he I SP indust r y ’s cur r ent pr act ices for choosing and deploy ing t he soft w ar e. This includes w hich v er sion of oper at ing sy st em soft w ar e t he r out er s should use, how t o get t he chosen v er sion on t o t he equipm ent , and t he v ar ious st r at egies for m anagem ent of t he r out er oper at ing soft w ar e and configur at ion. The second por t ion of t he chapt er c over s aspect s of r out er m anagem ent . The user int er face t o Cisco r out er s has alw ay s been t hr ough a com m and- line int er face ( CLI ) . Back in t he ear ly days of I OS Soft w ar e, t his w as of a ver y funct ional design—t hese days m any feat ures have been added, m aking it as flexible as m any of t he m odern shells av ailable on UNI X sy st em s. Also cov er ed ar e feat ur es of r out er m anagem ent , including best pr act ices for capt ur ing logging infor m at ion, apply ing t im e sy nchr onizat ion, using t he Sim ple Net w or k Managem ent Pr ot ocol ( SNMP) , using ht t p access r at her t han t he CLI , and dealing w it h soft w ar e cr ashes.

W h ich Cisco I OS Soft w a r e V e r sion Sh ou ld I Be Usin g? I SPs and NSPs oper at e in an env ir onm ent of const ant change, ex ponent ial gr ow t h, and unpr edict able t hr eat s t o t he st abilit y of t heir backbone. The last t hing an I nt er net back bone engineer needs is buggy or unst able code on r out er s. As in any com m er cial- gr ade ser v ice pr ov iding infr ast r uct ur e, t he equipm ent for m ing t hat infr ast r uct ur e r equir es st able oper at ing soft w ar e. The I SP space, how ev er , also dem ands soft w ar e t hat w ill giv e m ar k et leader ship w hen it com es t o new feat ur es. Her ein is t he differ ence bet w een ent er pr ise businesses and I nt er net ser v ice pr ov ision: The for m er dem ands st abilit y abov e all else and w ill change only w hen necessit at ed; t ypical soft w ar e r efr esh cycles for ent er pr ise net w or ks ar e m easur ed in y ear s. The ot her k ey differ ent iat or bet w een ent er pr ise and I SP businesses is t hat I SPs ex pect t o use t he I nt er net for com m unicat ion w it h t heir soft w ar e v endor s, for accessing new im ages, speak ing t o t he t echnical assist ance cent er , or com m unicat ing dir ect ly w it h t he dev elopm ent engineer s. This div er gence fr om t he t r adit ional m odel of t he soft w ar e dev elopm ent and im plem ent at ion pr ocess im plied t hat I SPs r equir e an I OS Soft war e code t r ain specific t o t heir needs. Midw ay t hr ough t he life of t he 10.3 soft w ar e t r ain, a t eam of Cisco engineer s devot ed t o w or king w it h I SP- specific feat ur es cr eat ed a br anch of I OS Soft w ar e t hat cat er ed specifically for I SPs’ r equir em ent s. The key c haract erist ics were an I P- cent r ic

16

code base, st abilit y , quick bug fix es, easy access t o t he k ey dev elopm ent engineer s, and r apid feat ur e addit ions. The so- called “ isp- geek s” soft w ar e st ar t ed life as an unofficial I SP soft w ar e issue, but w it h t he ar r ival of t he 11.1 soft w ar e t r ain, it had m at ur ed and dev eloped int o a r elease sy st em specifically t ar get ed t o I SPs. 11.1CA w as used t o deliv er new I SP- only feat ur es m ont hs befor e t hese appear ed anyw her e else—and 11.1CC w as t he successor t o 11.1CA, used t o deliv er t he now w idely deployed CEF funct ionalit y. As I OS Soft w ar e becom es m or e feat ur e - rich, t his I SP soft w ar e t r ain concept has been fur t her dev eloped and enhanced, and it now provides a well- dev eloped and st able plat for m for all I SPs. Along w it h t he developm ent of specific I OS Soft w ar e im ages for I SPs, t he Ser v ice Pr ov ider feat ur e set w as added t o all Cisco I OS Soft w ar e r eleased. This soft w ar e is based on t he I P- only im age but w it h addit ional feat ur es for I SPs. Such soft w ar e can be r ecognized by t he “ - p- ” in t he im age nam e. This im age is usually all t hat any I SP needs t o r un. These im ages cannot be or der ed at t im e of r out er pur chase, but t hey can be dow nloaded fr om Cisco. com befor e deploy m ent of t he r out er in ser v ice. For exam ple, a 7200 or der ed by an I SP m ight com e w it h t he c7200- i- m z.120- 6 im age— t his im age should be r eplaced w it h t he Ser vice Pr ovider equivalent , c7200- p- m z.1206. These Service Provider “ - p- ” im ages ar e built for all suppor t ed r out er plat for m s, u n lik e t he m or e lim it ed plat for m suppor t av ailable on t he I SP r elease t r ains. At t he t im e of w r it ing, our r ecom m ended I OS Soft w ar e br anches for I SPs ar e t he follow ing: • • •

1 2 . 0 S— Suppor t ing t he 7200, RSP7000, 7500, and 12000 ser ies r out er s 1 2 .0 — Suppor t ing 2500, 2600, 3600, and 4000/ 4x00 ser ies r out er s [ 1] 1 2 .1 — Suppor t ing t he new har dw ar e plat for m s not suppor t ed by 12.0 ( such as 3660)

Releases 11.1CC and 11.2GS ar e no longe r r ecom m ended for I SP back bones because t hey do not suppor t t he cur r ent gener at ion of har dw ar e in use, nor w ill t hey be enhanced t o suppor t new har dw ar e or soft w ar e feat ur es. For ex am ple, 11.1CC has gained no new feat ur es since 11.1( 26) CC. Releases pr ior t o 12.0 ar e now com ing t o t he end of t heir life. Alt hough suppor t new har dw ar e or soft w ar e feat ur es. For ex am ple, 11.1CC has gained no new feat ur es since 11.1( 26) CC. Releases pr ior t o 12.0 ar e now com ing t o t he end of t heir life. Alt hough t hey ar e st ill suppor t ed by Cisco, t hey w ill not gain any new feat ur es. Migr at ion fr om t hese old r eleases should be par t of t he ongoing upgr ade planning pr ocess in all I SP net w or k s at t he m om ent . I n addit ion t o t hese soft w ar e r eleases, ot her specialized v er sions ar e av ailable, and of cour se, t her e ar e new er dev elopm ent s t han t hose list ed pr ev iously . • •

1 2 . 0 ST is a v er sion of 12.0S enhanced t o include som e of t he feat ur es of 12.0T, specifically aim ed at t hose I SPs deploy ing MPLS- based v ir t ual pr iv at e net w orks. 1 2 .2 and 1 2 . 2 T are t he successor dev elopm ent s of 12.1 and 12.1T. At t he t im e of t his w r it ing, t hese t w o r elease t r ains w er e j ust m ade available, and w e don’t r ecom m end t heir use in an I SP net w or k unless t hey have unique feat ur es not available in t he r ecom m ended t r ains. For ex am ple, 12. 2T sees t he fir st r elease of a Cisco TAC– suppor t ed I Pv 6 soft w ar e.

17

I n t he fut ur e, t her e w ill be ot her I OS Soft w ar e r eleases follow ing t hose m ent ioned her e. Consult t he Pr oduct Bullet in page on Cisco. com for up- t o- dat e inform at ion. The online supplem ent s t o t his book w ill list t he cur r ent r ecom m endat ions for I SPs. N OTE Cisco Syst em s’ m ost up- t o- dat e r ecom m endat ions on w hich I OS Soft w ar e br anch an I SP should be using ar e on t he Pr oduct Bullet in page, av ailable at Cisco. com, at ht t p: / / w w w .cisco.com / w ar p/ public/ cc/ gener al/ bullet in/ index .sht m l.

W here t o Get I nform a t ion on Relea se 1 2 .0 S Release 12.0S is now available fr om Cisco. com ’s Soft w ar e Libr ar y , at ht t p: / / cco.cisco.com / k obay ashi/ sw - cent er / sw- ios.sht m l. The follow ing URLs hav e som e addit ional det ails on t he feat ur es included in 12.0S, m igr at ion opt ions, and how t o dow nload t he soft w ar e: Cisco I OS Soft w ar e Release 12.0S new feat ur es: ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ iom j r e12/ pr odlit / 934_pb. ht m Cisco I OS Soft w ar e Release 12.0S or der ing pr ocedur es and plat for m har dw ar e suppor t : ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ iom j r e12/ pr odlit / 935_pb. ht m Cisco I OS Soft w ar e r elease not es for Release 12.0S: ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ r elnot e/ 7000fam / r n 1 2 0 s. h t m Cisco I OS Soft w ar e r elease 12.0S m igr at ion guide: ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ iom j r e12/ pr odlit / 940_pb. ht m

Furt her Reference on I OS Soft w a re Relea ses Figur es 1- 1 and 1- 2 pr ovide a visual m ap of I OS Soft w ar e r eleases up t o 12.1—t h ey also show how t he differ ent v er sions and t r ains int er r elat e. This has been and st ill is an oft en- asked quest ion in t he I SP arena and ot her m arket places in w hich Cisco is present —t hese v isual r oadm aps hav e been cr eat ed t o show t he int er r elat ion of t he differ ent I OS Soft w ar e v er sions. The cur r ent up- t o- dat e r oadm ap can be seen at ht t p: / / w w w .cisco.com / w ar p/ public/ 620/ r oadm ap.sht m l. Consult t he follow ing URLs on Cisco. com for m ore det ailed and up- t o- dat e inform at ion on I OS Soft w ar e r elease st r uct ur e: Fig u r e 1- 1 . Cisco I OS Sof t w a r e Roa d m a p u p t o Re le a se 1 2 . 1

18

Fig u r e 1- 2 . I OS Sof t w a r e Roa d m a p f r om 1 2 . 1 On w a r d

19

Cisco I OS Soft w ar e r eleases: ht t p: / / w w w .cisco.com / w ar p/ public/ 732/ Releases/ Ty pes of Cisco I OS Soft w ar e r eleases: ht t p: / / w w w . cisco. com / w ar p/ cust om er / cc/ pd/ iosw / ior e/ pr odlit / 537_pp. ht m Release designat ions defined —soft w ar e lifecy cle definit ions: ht t p: / / w w w .cisco.com / w ar p/ cust om er / 417/ 109.ht m l Soft w ar e nam ing conv ent ions for Cisco I OS Soft w ar e: ht t p: / / w w w .cisco.com / w ar p/ cust om er / 432/ 7.ht m l I OS Soft w ar e r efer ence guide: ht t p: / / w w w . cisco. com / w ar p/ public/ 620/ 1. ht m l I OS Soft w ar e feat ur e nav igat or , fr om t he “ Ser v ice and Suppor t ” page on Cisco. com: ht t p: / / w w w .cisco.com / cgi- bin/ Suppor t / Feat ur eNav / FN.pl

20

I OS Soft w a r e M a n a ge m e n t Most r out er plat for m s used in I SP backbone net w or ks have ver y flexible RAM and Flash m em or y configur at ions. For pr iv at e, ent er pr ise, or cam pus net w or k s, t he num ber of changes r equir ed in soft w ar e, new feat ur es, or ev en t he net w or k infr ast r uct ur e is sm all. The I nt er net is changing and gr ow ing daily , and a com m on m ist ak e by new I SPs is t o under specify t he equipm ent t hat t hey pur chase. This should not be t aken as a r ecom m endat ion t o buy w hat m ight seem like unneeded m em or y . I t is r ecognit ion of t he fact t hat hav ing t o upgr ade a r out er ev er y f ew m ont hs because “ unfor eseen” gr ow t h in t he I nt er net causes disr upt ion t o t he net w or k and can pot ent ially affect t he r eliabilit y of har dw ar e. Many I nt er net engineer s suppor t t he not ion t hat t he m or e oft en hum ans t ouch a piece of har dw ar e, t he less re liable t hat har dw ar e becom es.

Fla sh M em ory The Flash m em or y on a r out er is w her e t he I OS Soft w ar e im ages ar e st or ed. When a new r out er is pur chased, it has t he I OS Soft w ar e im age specified at t he t im e of ordering inst alled in Flash. Flash m em ory usually is built int o t he r out er , and som e plat for m s hav e ex pansion slot s w her e PCMCI A Flash car ds or Flash disk s can be inst alled. I t is good pr act ice t o supplem ent t he built - in Flash w it h anot her area of Flash m em or y . This can be done in at least t w o w ay s: 1. Par t it ion t he built - in Flash m em ory. This can be done using t he configurat ion com m and. For ex am ple, t he follow ing com m and w ill par t it ion t he Flash int o t w o ar eas of 8 MB each ( assum ing 16 MB of inst alled Flash m em or y and also assum ing t hat t he har dw ar e suppor t s t his t ype of part it ioning) : 2. partition flash 2 8 8 3. I nst all a Flash car d or Flash disk in one or bot h of t he ext er nal flash slot s. Hav ing m or e t han one Flash m em or y ar ea ensur es t hat t he r out er I OS Soft w ar e im age can be upgr aded w it hout affect ing t he ex ist ing im age. For ex am ple, if t her e is r oom for only one im age in Flash and it is t he im age t hat t he r out er is r unning, t he ex ist ing im age needs t o be r em ov ed befor e a new one can be inst alled. What w ould happen, say, if t he rout er crashed during t he im age upg rade? Recovery is possible w it h t he boot im age, but t his is significant ly m or e difficult t han if pr oper pr ecaut ions w er e t aken. By copying t he new im age int o t he ot her ar ea of Flash m em or y, t he I SP ensur es t hat t he net w or k funct ionalit y is m inim ally affect ed in t he event of a crash or ot her unfor eseen pr oblem s dur ing im age upgr ade. The new im age in t he second ar ea of Flash m em or y easily can be select ed, as show n in t he follow ing exam ple for t he 7500 ser ies r out er s:

boot system flash slot1:rsp-pv-mz.120-5.S boot system flash slot0:rsp-pv-mz.111-25.CC boot system flash

21

This t ells t he r out er t o boot r sp- pv- m z.120- 5.S fr om slot 1 Flash fir st . I f t hat im age cannot be found or t he Flash car d is not in slot 1, t he r out er look s for r sp- pv- m z.11125.CC in slot 0. I f t hat im age cannot be found, t he r out er boot soft w ar e look s for t he fir st im age in any of t he syst em Flash. Consider t his exam ple, on t he 36x0 series rout ers, w here t he m ain 16 MB Flash has been par t it ioned:

boot system flash flash:1:c3640-p-mz.120-5.S boot system flash flash:2:c3640-p-mz.112-19.P boot system flash This t ells t he r out er t o boot c3640- p- m z.120- 5.S fr om t he fir st Flash par t it ion. I f t he r out er cannot find t hat im age, it w ill look for c3640- p- m z.112- 19.P in t he second Flash part it ion. Fail ing t hat , it looks for t he fir st usable I OS Soft w ar e im age in Flash m em or y . This t ype of ar r angem ent ensur es t hat , in t he event of im age cor r upt ion, a pr oblem w it h t he oper at ing im age, or a r out er cr ash, som e backup im age alw ays is available on t he rout er. Pr oper pr ecaut ions ensur e m inim al net w or k dow nt im e dur ing m aint enance per iods or em er gency occasions. Dow nloading a new im age dur ing a net w or k dow n sit uat ion w it h cust om er s and m anagem ent ex er t ing pr essur e is unnecessar y ex t r a st r ess t hat easily could hav e been av oided w it h a lit t le pr ecaut ion. Com m on pr act ice is for I SPs t o leav e t he k now n w or k ing im age on one of t he Flash car ds or Flash par t it ions of t he r out er . I f deploy m ent of a new r elease ( w hich has passed t est s in t he lab envir onm ent ) exhibit s som e pr oblem or defect lat er in oper at ion, it is easy t o back t r ack t o t he old im age. Finally , it m ak es no com m er cial or oper at ional sense t o sk im p on t he am ount of Flash m em ory. As m ore of t he feat ures request ed by I SPs are added t o I OS Soft w ar e, t he im age gr ow s lar ger . Sizing Flash t o t he cur r ent im age size is a false econom y because, m or e t han lik ely , in t he near fut ur e a lar ger im age w it h new feat ures m ight require Flash m em ory larger t han w hat has been inst alled in t he rout er.

Syst em M em ory Anot her com m on pract ice am ong t he Tier 1 and Tier 2 I SPs in all r egions of t he w or ld is m ax im izing t he m em or y on ev er y r out er . Cisco r ecom m ends t he necessar y am ount of m em or y r equir ed t o r un each I OS Soft w ar e im age. Dow nloading a new im age from Cisco. com includes a quest ion ask ing user s if t hey ar e fully aw ar e of t he m em or y r equir em ent s for t he chosen im age. I gnor e t he m inim um r ecom m endat ions at your peril! For exam ple, at t he t im e of t his w r it ing, it is r ecognized t hat 128 MB of m em ory is t he m inim um r equir em ent t o oper at e a r out er car r y ing t he full I nt er net r out ing t able. Any I SP r equest ing I OS Soft w ar e r elease 12.0S is r equir ed t o ack now ledge t his fact . I OS Soft w ar e r elease 12.0S w ill st ill oper at e inside 32 MB of m em or y on a 7200 rout er and w ill carry t he m aj orit y of t he I nt ernet rout es w it h 64 MB of m em ory ( w it h

22

m inim al configur at ion and all feat ur es t ur ned off) . For ex am ple, t he BGP algor it hm s w ill use m ore m em ory if it is available t o im prove t heir efficiency. Skim ping on m em or y m eans affect ing t he per for m ance of t he r out er s and t he end r esult , w hich t he cust om er ex per iences. Many I SPs now sim ply specify m axim um m em or y w hen t hey pur chase new r out ing har dw ar e. They r ecognize t hat sending an engineer t o r em ov e pr ocessor cards cost s m oney t hr ough dow nt im e, ex t r a hum an r esour ces, and pot ent ial ser v ice disr upt ion, and it shor t ens t he lifet im e of t he equipm ent t hr ough t he hum an int er act ion. “ Fit and for get ” is t he nor m am ong m any of t he lar gest I SPs t oday.

W he n a nd H ow t o Upgr a de Sev er al I SPs upgr ade t heir r out er soft w ar e alm ost ev er y t im e Cisco r eleases a new im age. This is r ecognized by m ost indust r y oper at or s as being bad pr act ice. The only t im e t hat any I SP should be upgr ading soft w ar e is w hen it is r equir ed t o fix bugs, suppor t new har dw ar e, or im plem ent new soft w ar e feat ur es. I n m any ot her indust r ies, changing cor e oper at ing soft w ar e is seen as a m aj or ev ent not t o be under t ak en light ly . Yet for som e r eason, som e I SPs seem t o t hink t hat a for t night ly upgr ade is good pr act ice. Based on w hat m ost Tier 1 and Tier 2 I SPs now do, soft w ar e upgr ades ar e car r ied out only w hen t hey ar e r equir ed. Ex t ensiv e t est ing is car r ied out in t he t est lab ( how m any I SPs have a t est net w ork t hat looks like one of t heir PoPs, or a port ion of t heir net w or k ?) . Deploy m ent happens only aft er ex t ensiv e t est ing, and ev en t hen new im ages ar e im plem ent ed w it h caut ion on a quiet er par t of t he net w or k. For exam ple, t he soft w are versions in one PoP m ight be updat ed and left running for a w eek or a for t n igh t t o check for any issues; aft er t his init ial deploy m ent phase, t he r est of t he net w or k w ill be upgr aded. Caut ion is of par am ount im por t ance on a com m er cial- gr ade net w or k . Ev en w hen upgr ades ar e car r ied out , r em em ber t he r ecom m endat ions discussed in t his sect ion. I OS Soft w ar e m ak es it easier by giv ing back out pat hs t hr ough alt er nat iv e im ages. Nev er at t em pt an upgr ade w it hout being aw ar e of pot ent ial side effect s fr om unfor eseen pr oblem s, nev er at t em pt an upgr ade w it hout a back out plan, and nev er at t em pt an upgrade w it hout hav ing r ead t he r elease not es t hat com e w it h t he soft w ar e r elease. I t also helps t o r ead t he r elease not es for all int er m ediat e r eleases because t hat w ill giv e t he engineer good infor m at ion about w hat has changed in t he soft w ar e ov er t he r elease cy cle. Anot her pract ice im plem ent ed by m ost Tier 1 and Tier 2 I SPs is t o m inim ize t he num ber of differ ent v er sions of I OS Soft w ar e im ages r unning on t heir net w or k ’s r out er s. This is alm ost alw ay s done for adm inist r at iv e and m anagem ent r easons. Apar t fr om r educing t he num ber of pot ent ial int er oper abilit y issues due t o bugs and new feat ur es, it is easier t o t r ain oper at ions st aff on t he feat ur es and differ ences bet w een a few im ages t han it is t o t r ain t hem on t he differ ences am ong m any im ages. Typically I SPs aim t o r un no m or e t han t w o differ ent I OS Soft w ar e r eleases. One im age is t he old r elease; t he ot her is t he one on w hich t hey ar e doing t he blank et upgr ade on t he back bone. Upgr ades t end t o be phased, not car r ied out en m asse ov er night . I f t he I SPs hav e access equipm ent , such as t he AS5x 00 ser ies, or cable/ x DSL aggr egat ion dev ices, t hey w ill deploy differ ent I OS Soft w ar e im ages on

23

t hese devices. But again, if one dial box needs t o be upgr aded, I SPs t end t o upgr ade t hem all t o ensur e a consist ent I OS Soft ware release on t hat net work. A t ypical soft w ar e ver sion st r at egy is som et hing like t he follow ing: •

•

•

•

Cor e / b a ck b on e n e t w or k — One soft w ar e r elease ( x x x x- p- m z.120- 17.S1) r uns on all backbone r out er s. The soft w ar e on t hese r out er s pr obably is changed ever y six m ont hs or ev en less fr equent ly . The I nt er net cor e car r ies only I P packet s, and r ar ely ar e new feat ur es or capabilit ies added. Well- run I nt er net cor es oft en have r out er s w it h upt im es exceeding six m ont hs, som et im es even over one year. D ist r ib u t ion a n d le a se d- lin e a g g r e g a t ion la y e r — One soft w ar e r elease r uns on all r out er s. This t ends t o be t he par t of t he net w or k t hat cust om er s connect t o, so oft en new feat ur es and new ly deploy ed connect ion ser v ices dem and a m or e fr equent soft w ar e updat e cy cle. D ia l a cce ss la y e r— A com m on soft w ar e r elease is r un on all access plat for m s. As w it h t he pr ev ious ex am ple, a m or e fr equent cy cle m ight be necessar y . Som e I SPs build new infr ast r uct ur e for new ser v ices, so w hen infr ast r uct ur e is unchanging, it m ak es lit t le sense t o upgr ade soft w ar e. Som e dialup net w or k s t hat w e hav e had ex per ience w it h hav e had har dw ar e r unning t he sam e soft w ar e im age for sev er al y ear s. V PN a cce ss la y e r — A com m on soft w ar e r elease is r un on all plat for m s. This ex am ple is included because it is t he cur r ent fashion in t he indust r y . Oft en I SPs use bleeding- edge soft w ar e and har dw ar e t o deliv er VPN ser v ices, and fr equent upgr ades for new feat ur es can be necessar y fr om t im e t o t im e. Again, t he usual r ule applies: Don’t change it unless new feat ur es ar e n ecessar y ; it sav es t he cust om er s fr om going t hr ough pain.

Som e of t he bigger I SPs hav e w eek ly soft w ar e st r at egy m eet ings, w it h t he aim t o ensur e consist ency acr oss t he com pany business for soft w ar e deploy ed on t he backbone. New soft w ar e has t o be appr oved acr o ss t he engineer ing and oper at ions m anagem ent , and t hen it is deploy ed only aft er fair ly int ensiv e pr oof t est ing in t he lab. Soft w ar e ver sion consist ency m onit or ed by t he I SP’s NOC, oft en t hr ough aut om at ic or cr on- based t ools t hat log int o all t he r out er s and ot her equipm ent and gr ab t he v er sion num ber of t he r unning soft w ar e and t he cont ent s of t he r out er ’s Flash m em ory. Finally , adopt ing som e st r at egy is st r ongly r ecom m ended. Hav ing no st r at egy usually m eans t hat in t im es of cr isis dur ing net w or k pr oblem s, t he oper at ions engineer s w ill r esor t t o a r andom w alk t hr ough differ ent soft w ar e v er sions in t he desper at e hope t hat som et hing m ight w or k t o st abilize a net w or k pr oblem . Having st r ong cont r ol over soft w are versions w ill m ean t hat diagnosing net w ork proble m s can be achiev ed m or e easily .

Copying N ew I m ages t o Flash M em ory Copying a new im age int o Flash m em or y in it self isn’t a com plicat ed pr ocess, but t her e ar e a few good pr act ice point s t o be aw ar e of. The m ost im por t ant point is t o re- em phasize t hat leaving a backout im age som ew her e on t he r out er is good pr act ice and plain com m on sense. So m any net w or k out ages hav e been pr olonged because a new r out er im age failed and t he I SP didn’t leav e a back out im age on t he dev ice.

24

New im ages should be loaded int o Flash dur ing m aint enance per iods, not w hen t he r out er is live car r ying a full t r affic load. Alt hough t he act ivit y of copying an im age t o Flash w on’t im pact t he r out er ’s oper at ion, it is adv isable t o av oid enhancing t he possibilit y of an accident w hile t he net w ork is in product ion. At least an operat ional er r or dur ing a m aint enance per iod w on’t cause cust om er anger because cust om er s w er e ex pect ing dow nt im e dur ing t he m aint enance per iod ( assum ing t hat t he cust om er s w er e infor m ed in t he fir st place, anot her k ey point t hat sever al I SPs seem t o for get ! ) . Basically t w o w ay s ex ist for copy ing im ages t o and fr om t he r out er Flash. Using TFTP is t he m or e t r adit ional w ay and has been par t of I OS Soft w ar e for a ver y long t im e. FTP suppor t w as added w it h t he ar r ival of 12.0 soft w ar e; t his is som ew hat m or e efficient and flexible t han using TFTP, and it does not have TFTP’s 16 MB file size r est r ict ion. Cop y in g U sin g TFTP The com m ands t o copy soft w ar e int o Flash m em or y hav e been r efined in r eleases from 12.0, m aking t he m echanics of get t ing soft w ar e t o and fr om t he r out er sim pler and m or e consist ent in st y le. The copy com m and has been enhanced t o suppor t a URL appear ance cov er ing all sy st em dev ices in a consist ent for m at , as in t his exam ple:

beta7200#copy tftp ? bootflash: Copy to bootflash: file system disk0: Copy to disk0: file system disk1: Copy to disk1: file system flash: Copy to flash: file system ftp: Copy to ftp: file system lex: Copy to lex: file system null: Copy to null: file system nvram: Copy to nvram: file system rcp: Copy to rcp: file system running-config Update (merge with) current system configuration slot0: Copy to slot0: file system slot1: Copy to slot1: file system startup-config Copy to startup configuration system: Copy to system: file system tftp: Copy to tftp: file system beta7200#copy tftp This is som ew hat im pr ov ed ov er t he r at her inconsist ent and plat for m- dependent for m at used in pr ev ious r eleases. Befor e copying t he im age t o Flash, m ake sur e t hat t he Flash has enough space for t he new im age. Pr efer ably inst all t w o Flash dev ices or par t it ion t he ex ist ing Flash dev ice, as has been descr ibed prev iously . Use cd, de le t e , e r a se , and sq u e e z e ( depending on plat for m ) t o clear enough space on t he Flash file sy st em . Mak e sur e t hat t he par t it ion/ Flash holding t he cur r ent ly r unning im age is not t ouched. I f t her e is any pr oblem w it h t he upgr ade, a back out pat h is av ailable. And don’t for get t o set up t he b oot sy st e m x x x x x I OS Soft w ar e configur at ion com m and so t hat t he r out er is t old t o boot t he cur r ent ly r unning im age.

25

When t he flash part it ion is ready, a copy com m and can be issued t o copy t he new I OS Soft war e im age fr om t he r em ot e dev ice ( w hich could be any of t hose list ed pr ev iously ) t o t he par t it ion. An ex am ple of t he copy com m and follow s:

beta7200#copy tftp slot1: Address or name of remote host []? noc1 Source filename []? 12.0S/c7200-p-mz.120-6.S Destination filename [c7200-p-mz.120-6.S]? Accessing tftp://noc1/12.0S/c7200-p-mz.120-6.S... Loading 12.0S/c7200-p-mz.120-6.S from 192.168.3.1 (via Serial3/1): !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!! [OK - 5708964/11417600 bytes] bytes copied in 330.224 secs (17299 bytes/sec) beta7200# This w ill copy t he im age c7200- p- m z.120- 6.S fr om t he t ft p ser v er t o t he Flash dev ice in slot 1. The com m and also can be shor t ened t o t his, w hich w ill achiev e t he sam e t hing:

beta7200#copy tftp://noc1/12.0S/c7200-p-mz.120-6.S slot1: Destination filename [c7200-p-mz.120-6.S]? …etc… Not ice t hat t he r out er w ill at t em pt t o w or k out t he filenam e fr om t he URL st r ing ent er ed— t his can be helpful and can sav e t y ping. Cop y in g U sin g FTP FTP client suppor t has been added in 12.0 im ages of I OS Soft w ar e. TFTP has w ellknow n lim it at ions, w it h t he 16 MB file size becom ing an issue w it h som e of t he lar ger full- feat ur ed I OS Soft w ar e im ages now being m ade av ailable. The FTP client allow s for FTPing im ages t o and fr om an FTP ser v er . The opt ions for and t he capabilit ies of t h e ft p com m and ar e t he sam e as for t he t ft p com m and descr ibed pr ev iously . The following is an exam p le of a soft w ar e dow nload dir ect ly fr om Cisco’s FTP sit e— som et hing t hat cannot be done w it h TFTP ( t he passw or d has been r eplaced by XXX in t he exam ple) :

7206-AboveNet-SJ2#copy ftp://bgreene:[email protected] slot0: Source filename []? /cisco/ios/12.0/12.0.9S/7200/c7200-k3p-mz.1209.S.bin Destination filename [c7200-k3p-mz.120-9.S.bin]? Accessing ftp://bgreene:[email protected] //cisco/ios/12.0/12.0.9S/7200/c7200-k3pmz.120-9.S.bin...Translating "ftp.cisco.com"...domain server (207.126.96.162) [OK]

26

Loading /cisco/ios/12.0/12.0.9S/7200/c7200-k3p-mz.120-9.S.bin As I SPs hav e gained ex per ience w it h using FTP, it has v er y m uch becom e t he pr efer r ed m et hod of put t ing im ages and ot her infor m at ion ont o t he r out er . We encourage you t o look at t his m et hod, if you hav e not alr eady done so, and adopt it as cur r ent pr act ice fr om her e on. W ARN I N G I t ’s im port ant t o be aw are t hat on t he 2500 series rout ers t he im age runs from Flash, so a reload of t he rout er t o run t he BOOTROM im age is required t o upgrade. The BOOTROM im age does not suppor t FTP copies of t he I OS Soft w ar e im age ont o t he r out er , ev en t hough it is possible t o ent er t he com m and sequences list ed previously —t he BOOTROM at t em pt s t o upload t he im age using TFTP, it s only suppor t ed funct ionalit y .

Re loa d in g t h e Rou t e r s When t he im age has been successfully loaded by eit her t he TFTP or FTP m et hod and has been v er ified, set up t he r out er t o boot t he new im age ( use t he n o b oot sy st e m and b oot sy st e m com m ands descr ibed pr ev iously ) . I t is also a good idea t o configure anot her b oot sy st e m com m and point ing t o t he back up im age ( as in t he ex am ple in t he ear lier sect ion) . The st andar d w ay of im plem ent ing new soft w ar e on t he r out er is t o use t he r e loa d com m and— t his sim ply r eboot s t he r out er . Use t he com m and w it h caut ion—doing so out side a m aint enance slot w ill at t r act cust om er anger because of t he pot ent ial num ber of m inut es of dow nt im e ex per ienced. I f desir able, t he r out er ev en can be configur ed t o do a t im ed/ delay ed r eboot som et im e in t he fut ur e. Use t hat feat ur e w it h car e, t hough; it is per fect ly feasible t o do t im ed r eboot s on sever al r out er s and com plet ely br eak a por t ion of t he I SP net w or k ! We hav e seen sev er al cases of planned soft w ar e upgr ades go w r ong because t he I SP m ade a configur at ion er r or and t he r esult ing phased t im ed r eload gr adually pulled t he net w or k ov er—w it hout t he I SP’s oper at ions st aff being able t o do any t hing about it . A guideline r ule is t hat it is accept able t o do t im ed r eloads at t he edge of a net w or k , but cor e dev ices, by t heir nat ur e, ar e cr it ical for t he oper at ional int egr it y of t he back bone and should be handled appr opr iat ely w it h dir ect input from t he operat ions t eam . Th e r e loa d com m and has t hr ee opt ions t hat m any I SPs find useful. Ex am ples of t heir use are show n in Table 1- 1. Table 1-1. reload Command Examples bet a7200# reload at 17: 05 Reboot s t he r out er at 17: 05 local t im e on t he r out er . Not ice t he det ailed confir m at ion. Reload scheduled for 17: 05: 00 AEST Mon Jul 16 2001 ( in 5 hours and 10 m inut es)

27

Proceed w it h reload? [ confirm ]

Shows a m essage and t he prom pt t o confirm .

bet a7200# reload in 180 Reload Reboot s t he rout er in 180 m inut es. Again, scheduled for 16: 10: 35 AEST Mon Jul not ice t he det ailed confirm at ion m essage and 16 2001 ( in 3 hours) t he prom pt t o confirm . Proceed w it h reload? [ confirm ]

Shows a m essage and t he prom pt t o confirm .

bet a7200# r eload cancel

Cancels a pr eviously set up r eload.

Don’t for get t hat t he r e loa d ca n ce l com m and can be used t o undo any t im ed r eload—if a t im ed and phased r eload of sever al r out er s has been set up and m ust be back ed out of, ensur e t hat t her e is sufficient good access t o each r out er ( pr efer ably wit h out - of- band m anagem ent ) so t hat t he ca n ce l com m and can be im plem ent ed w it hout panicking. Also not ice t hat w hen a t im ed r eload has been set up on t he r out er , using a sh o w ve r sion com m and or sim ply logging int o t he r out er w ill show t hat a t im ed r eload has been set up— for exam ple:

[philip@pfs-pc philip]$ telnet beta7200 Trying 192.168.4.130... Connected to eth1-0.beta7200 (192.168.4.130). Escape character is '^]'. User Access Verification Username: philip Password: Reload scheduled for 17:05:00 AEST Mon Jul 16 2001 (in 1 hour and 4 minutes) beta7200> When t he new im age has been boot ed successfully , it should be put t hr ough accept ance t est s dur ing t he m aint enance slot ( t hese t est s could be as sim ple as ask ing, “ Does t he r out ing w or k ?” ) and t hen should be m onit or ed dur ing oper at ion. Don’t delet e t he previous im age—y ou k now t hat it w or k s, so if it is left on t he ot her Flash par t it ion, a back out pat h is available in case of fut ur e pr oblem s. The old im age can be delet ed aft er a decision has been m ade t o fur t her upgr ade t he I OS Soft w ar e. The benefit of configur ing t w o Flash dev ices/ par t it ions is clear t o see—ease of m aint enance! W ARN I N G Upgr ade a r out er only w hen t her e ar e bug fix es or w hen new har dw ar e suppor t or new soft w ar e feat ur es ar e r equir ed. Ot her w ise, do not t ouch t hat r out er ! “ I f it isn’t br ok en, don’t fix it !”

Con f igu r a t ion M a n a ge m e n t The follow ing sect ion discusses som e of t he configur at ion- m anagem ent feat ur es on t he r out er . This includes how t o handle t he configur at ion in t he NVRAM, how t o use

28

t he TFTP and FTP ser v er funct ions on t he r out er , and som e of t he shor t cut s available at t he CLI .

N VRAM , TFTPserver, a nd FTPserver The onboar d r out er NVRAM is used t o st or e t he r out er ’s act ive configur at ion. Most I SPs k eep an off- r out er copy of t his configur at ion, t oo. I n t he unlik ely ev ent t hat t he configurat ion is lost on t he r out er , t hey can quick ly r ecov er t he sy st em w it h t he offr out er back up. Ther e ar e sev er al opt ions for off- rout er backup of t he running configur at ion: •

•

•

•

Wr it e configur at ion t o a TFTP ser v er using t he w r it e n e t com m and. ( I n I OS Soft w ar e r elease 12.0 and m or e r ecent soft w ar e, t he w r it e n e t com m and has been super seded w it h t he m or e sophist icat ed copy funct ion. The equiv alent com m and is cop y r u n n in g t f t p :. ) Configur at ions sav ed by an oper at or ’s w r it e n e t com m and ar e k ept under aut om at ed r evision cont r ol. Com bined w it h TACACS+ aut hent icat ion ( see lat er ) , it is possible t o t r ack w ho has changed w hat and w hen. This is im por t ant for account abilit y and configur at ion back out in case of pr oblem s. An aut om at ed ( for ex am ple, UNI X cr on) j ob collect s t he configur at ion fr om t he r out er ev er y few hour s or so. The collect ed copy is k ept under r ev ision cont r ol. Changes can be flagged t o t he net w or k - m onit or ing sy st em , t o t he NOC, or t o t he oper at ions engineer s. Som e public dom ain t ools ar e av ailable t o do t his; t he best k now n and m ost popular is RANCI D ( Really Aw esom e New Cisco confI g Differ ) , at ht t p: / / w w w .shr ubber y .net / r ancid/ . Rout er configur at ions ar e built fr om a m ast er configur at ion dat abase. The r unning configur at ion is only a copy , w it h t he m ast er configur at ion k ept offr out er . Any updat es t o t he r unning configur at ion ar e m ade by alt er ing t he m ast er files ( under r evision cont r ol) ; t he new m ast er configur at ion t hen is im plem ent ed dur ing m aint enance per iods.

N O TE See Chapt er 2, “ Gener al Feat ur es,” for a discussion on loopback int er faces and for best pr act ices for configur ing t he r out er for TFTP ser v ices.

The I OS Soft w ar e com m and prom pt s t o sav e t he configur at ion ar e giv en in t he follow ing ex am ples. The sy nt ax has been significant ly changed st ar t ing w it h I OS Soft w ar e r elease 12.0, m ainly t o m ak e t he com m ands used t o t r ansfer configur at ion files and I OS Soft w ar e bet w een oper at or / NOC and t he r out er m or e consist ent . The I OS Soft w ar e com m and befor e r elease 12.0 is given in t he follow ing exam ple:

alpha7200#write network Remote host []? noc-pc Name of configuration file to write [alpha7200-confg]? Write file router2-confg on host 192.168.3.1? [confirm] Building configuration... Writing alpha7200-confg !!! [OK]

29

alpha7200# Fr om r elease 12.0 onw ar d, t he com m and t o do t he sam e t hing is copy , as given in t he follow ing exam ple:

beta7200#copy running tftp: Remote host[]? noc-pc Destination filename [beta7200-confg]? Write file tftp://noc-pc/beta7200-confg? [confirm] !!! [OK] beta7200# N OTE Th e w r it e n e t w or k com m and st ill is suppor t ed in r elease 12.0, alt hough it m ight be w it hdraw n in a fut ure release.

Since r elease 12. 0, FTP also can be used t o copy configur at ions t o an FTP ser v er . This pr ov ides m or e secur it y for t he configur at ions because t he FTP ser v er r equir es a user nam e/ passw or d. Cisco pr ov ides t w o w ay s of t o pr ov ide t he user nam e/ passw or d t o t h e FTP clien t . The fir st w ay put s t he user nam e and passw or d as par t of t he I OS Soft w ar e configur at ion. Wit h ser v ice passw or d- encr ypt ion t ur ned on, t he FTP passw or d w ould be st or ed w it h encr y pt ion t y pe 7:

ip ftp source-interface Loopback 0 ip ftp username user ip ftp password quake This allow s t he FTP com m and t o t r anspar ent ly inser t t he user nam e/ passw or d w hen connect ion t o t he FTP ser v er .

Excalibur#copy running-config ftp: Address or name of remote host []? 1.13.13.13 Destination filename [excalabur-confg]? Writing excalabur-confg !! bytes copied in 3.848 secs (1267 bytes/sec) Excalibur# The alt er nat iv e is t o include t he user nam e/ passw or d in a st andar d URL for m at :

Excalibur#copy running-config ftp://user:[email protected] Address or name of remote host [1.13.13.13]? Destination filename [excalabur-confg]? Writing excalabur-confg !! bytes copied in 4.152 secs (950 bytes/sec)

30

Excalibur#

La r ge Configur a t ions When t he NVRAM is not lar ge enough t o st or e t he r out er configur at ion, t her e is an opt ion t hat allow s t he configur at ion t o be com pr essed ( using a gzip- like com pr ession algorit hm ) :

service compress-config Use t his only if t her e is a r equir em ent t o do so. I f t he exist ing NVRAM can hold t he configur at ion uncom pr essed, do not use t his feat ur e. Som e I SPs have ext r em ely large configu r at ions, and t his feat ur e w as int r oduced t o assist t hem . I f t he r out er configur at ion has becom e ver y lar ge, it is w or t h checking w het her som e of t he new er I OS Soft w ar e feat ur es can be used. One ex am ple is t o use pr efix list s inst ead of access list s; t he for m er is m or e space- efficient in NVRAM and also is m ore efficient in oper at ion.

Com m a n d - Lin e I n t e r fa ce The I OS Soft w ar e CLI is t he t r adit ional ( and fav or ed) w ay of int er act ing w it h t he r out er t o ent er and change configur at ion and t o m onit or t he r out er ’s ope rat ion. This sect ion descr ibes t he I SP oper at or ’s use of t he CLI ; it is also possible t o use a w eb br ow ser t o int er act w it h and configur e t he r out er . Use of t he HTTP ser ver is br iefly cov er ed lat er in t he chapt er . The CLI is now ver y w ell docum ent ed in t he Cisco Univ er CD docum ent at ion set , at ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 12cgcr / fun_r / ind ex . h t m. How ev er , a few t ips and t r icks t hat ar e r egular ly used by I SPs ar e w or t h m ent ioning her e.

Edit ing Ke ys Sever al keys ar e ver y useful as shor t cut s for edit ing t he I OS Soft w ar e configur at ion. Alt hough t hese ar e cover ed in det ail in t he I OS Soft w ar e r elease 12.0 docum ent at io n set , it is useful t o point out t hose used m ost com m only, show n in Table 1- 2.

Key

Table 1-2. Common Editing Shortcuts Function

Tab

Com plet es t he com m and being t y ped in. This saves t yping effor t and is especially useful w hen t he oper at or is st ill lear ning t he I OS Soft w ar e com m an d set .

?

List s t he av ailable com m ands st ar t ing w it h t he char act er s ent er ed so far.

Up/ dow n arrow

Allow s t he oper at or t o scr oll up and dow n t hrough t he hist ory buffer.

Ct rl- A

Mov es t he cur sor t o t he beginning of t he line.

31

Ct rl- E

Mov es t he cur sor t o t he end of t he com m and line.

Ct rl- K

Delet es all char act er s fr om t he cur sor t o t he end of t he com m and line.

Ct rl- W

Delet es t he w or d t o t he left of t he cursor.

Ct rl- X

Delet es all char act er s fr om t he cur sor t o t he beginning of t he com m and line.

Esc- B

Mov es t he cur sor back one w or d.

Esc- F

Mov es t he cur sor for w ar d one w or d.

The com plet e list of com m ands can be found in t he I OS Soft w ar e docum ent at ion: ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 12cgcr / fun_r / fr pr t 1 / f r u i. h t m.

CLI St r ing Se a r ch Aft er a consider able num ber of r equest s fr om I SPs, a UNI X gr ep- lik e funct ion ( pat t er n sear ch) has been int r oduced as a new feat ur e in I OS Soft w ar e fr om r eleases 11.1CC and 12.0. I t allow s oper at or s t o sear ch for com m on expr essions in configur at ion and ot her t er m inal out put . Again, only salient point s ar e cov er ed her e because t he I OS Soft w ar e docum ent at ion now giv es m or e det ailed infor m at ion at ht t p: / / w w w .c isco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 120new ft / 120t / 1 2 0 t 1 / clipar se. h t m. The funct ion is invoked by using a ver t ical bar “ | ” , like t he UNI X p ip e com m and:

beta7200#show configuration | ? begin Begin with the line that matches exclude Exclude lines that match include Include lines that match beta7200#show configuration _ Follow ing one of t hese t hr ee opt ions, t he oper at or should ent er a r egular ex pr ession t o get a pat t er n m at ch on t he configur at ion, as in t he pr eceding ex am ple. The r egular expr essions can be single or m ult iple char act er s or a m or e com plex const r uct ion, in a sim ilar st y le t o UNI X r egular ex pr essions.

Defiant#show running-config | begin router bgp router bgp 200 no synchronization neighbor 4.1.2.1 remote-as 300 neighbor 4.1.2.1 description Link to Excalabur neighbor 4.1.2.1 send-community neighbor 4.1.2.1 version 4 neighbor 4.1.2.1 soft-reconfiguration inbound neighbor 4.1.2.1 route-map Community1 out maximum-paths 2 --MoreDur ing t he display of configur at ion or file cont ent s, t he scr een pager —More — will appear if t he out put is longer t han t he cur r ent t er m inal lengt h set t ing. I t is possible

32

t o do a r egular ex pr ession sear ch at t his pr om pt , t oo. The / k ey m at ches t he be gin k ey w or d; t he - key m eans exclude, and t he + key m eans t o include. Finally, in enable m ode it is possible t o use t he m ore com m and t o display file cont ent s. Regular ex pr essions ( as in t he pr eceding ex am ple) can be used w it h m ore . The opt ions available w it h m or e follow in t his ex ample:

beta7200#more /ascii /binary /ebcdic bootflash: disk0: disk1: flash: ftp: null: nvram: rcp: slot0: slot1: system: tftp: beta7200#

? Display binary files in ascii Force display to hex/text format Display binary files in ebcdic File to display File to display File to display File to display File to display File to display File to display File to display File to display File to display File to display File to display

By using t he | aft er t he m ore com m and and it s opt ion, it is possible t o sear ch w it hin t he file for t he st r ings of int er est in t he sam e w ay as discussed pr eviously.

D e t a ile d Loggin g Keeping logs is a com m on and accept ed operat ional pr act ice. I nt er face st at us, secur it y aler t s, env ir onm ent al condit ions, CPU pr ocess hog, and m any ot her ev ent s on t he r out er can be capt ur ed and analy zed w it h UNI X sy slog. I OS Soft w ar e has t he capabilit y of doing UNI X logging t o a UNI X sy slog ser v er. The r out er syslog for m at is com pat ible w it h BSD UNI X syslog ( found as part of m ost UNI X and Linux syst em s deploy ed t oday ) . Table 1- 3 show s a t ypical logging configura t ion for I SPs.

no logging console

Table 1-3. A Typical Logging Configuration Don’t send logs t o t he r out er console

logging buffer ed 16384

16 KB hist ory buffer on rout er

logging t r ap debugging

Cat ch debugging lev el t r aps ( in ot her w or ds, ev er y t hing)

logging facilit y local7

Sy slog facilit y on sy slog ser v er

logging 169.223.32.1

I P addr ess of y our fir st sy slog ser v er

logging 169.223.45.8

I P addr ess of y our second sy slog ser v er

To set up t he syslog daem on on a UNI X syst em , include a line such as t he follo wing in t he file / et c/ sy slog.conf:

local7.debugging

/var/log/cisco.log

33

I t is consider ed good pr act ice t o r eser ve a syslog facilit y on t he UNI X log host for each t y pe of net w or k dev ice. So, for ex am ple, back bone r out er s can use loca l7 , access ser v er s can use loca l5 , TACACS+ can use loca l3, and so on. An ex am ple of a w orking syslog configurat ion file is given here:

# Log all kernel messages to the console. kern.* /var/log/kern.log # Log everything apart from the specific entries given after this line *.debug;kern.none;mail.none;authpriv.none;local7.none\ local5.none;local4.none;local3.none /var/log/messages # The authpriv file has restricted access. authpriv.*

/var/log/secure

# Log all the mail messages in one place. mail.* /var/log/maillog # Everybody gets emergency messages. *.emerg

*

# Save mail and news errors of level err and higher in a special file. uucp,news.crit /var/log/spooler # Cisco Access server log local7.* core-log

/var/log/cisco-

# Cisco Access server log local5.* access-log

/var/log/cisco-

# NetFlowLog local4.* /var/log/netflowlog # TACACS+ local3.* /var/log/tacacs+ # Save boot messages to boot.log local1.* /var/log/boot.log Put t ing all t he logs in one huge file sim ply m akes syst em m anagem ent hard and m ak es debugging pr oblem s by sear ching t he log file nex t t o im possible. Mor e m oder n UNI X plat for m s r equir e net w or k suppor t in t he sy slog daem on t o be enabled by a r unt im e opt ion ( net w or k suppor t now is disabled by default , t o av oid secur it y pr oblem s) . Check t hat y ou see t he - r in t he sy slog com m and line, as in t he follow ing exam ple, w hen you list t he pr ocess on a UNI X syst em ( t he exam ple is for Red Hat Linux) :

34

pfs-pc$ ps ax | grep syslog 496 ? S 0:04 syslogd -r pts/5 S 0:00 grep syslog pfs-pc$ When collect ing logging infor m at ion, it is im por t ant not t o for get t o par se t he logs for any useful or net w or k cr it ical infor m at ion. I t is also essent ial t o r em em ber t o r ot at e t he logs on a daily or w eek ly basis. Som e com m er cial UNI X sy st em s hav e t his av ailable as par t of t he dist r ibut ion. I n Linux, t he log r ot at ion can be configur ed in / et c/ logr ot at e.conf —consult t he Linux m an pages for m ore inform at ion. I t ’s also possible t o configur e how m any old copies ar e r et ained, and som e I SPs have m odified t he log r ot at ion scr ipt s t o ar chiv e logs t hat hav e ex pir ed out of r ot at ion. ( I ndeed, som e business and account ing r equir em ent s st at e t hat ev idence of t ransact ions or operat ions m ust be st ored for several years —in t his case, syst em logs oft en ar e ar chiv ed on CD- ROM or ot her high- densit y st orage m edium .) By default , log m essages are not t im e - st am ped. I f t he r out er s ar e configur ed for UNI X logging, you w ill w ant det ailed t im e st am ps of for each log ent r y:

service timestamps debug datetime localtime show-timezone msec service timestamps log datetime localtime show-timezone msec This w ill pr oduce a syslog m essage t hat looks som et hing like t he follow ing:

Jul 27 15:53:23.235 AEST: %SYS-5-CONFIG_I: Configured from console by philip on console The com m and- line opt ions in t he t im e st a m ps com m and ar e as follow s: • • • • • •

de bug — All debug inform at ion is t im e - st am ped. log— All log inform at ion is t im e - st am ped. da t e t im e — The dat e and t im e ar e included in t he sy slog m essage. loca lt im e — The local t im e ( inst ead of UTC) is used in t he log m essage. sh o w- t im e zone — The t im e zone defined on t he r out er is included. ( This is useful if t he net w or k cr osses m ult iple t im e zones.) m se c— Tim e accuracy is expressed as m illiseconds, w hich is useful if NTP is configur ed.

By default , a syslog m essage cont ains t he I P addr ess of t he int er face t hat it uses t o leav e t he r out er . You can r equir e all sy slog m essages t o cont ain t he sam e I P addr ess, r egar dless of w hich int er face t hey use. Many I SPs use t he loopback I P addr ess. This keeps t heir syslogs consist ent and allow s t he m t o enhance t he secur it y of t heir syslog ser ver host ( by t he use of TCP w r apper s or r out er filt er s, for ex am ple) . To configur e sy slog t o use t he loopback as sour ce I P addr ess, ent er t his configur at ion com m and:

logging source-interface loopback0

35

N OTE See Chapt er 2 for a discussion on loopback int er faces, best pr act ices for configur ing t he log host s for syslog ser vices, and so on.

Anot her com m and t o consider is t he n o log g in g con sole com m and. Som et im es logging gener at es a t r em endous am ount of t r affic on t he console por t . Of cour se, t his happens j ust w hen y ou r eally need t o connect t o t he console por t t o t r oubleshoot w hat is causing t he t r em endous sur ge of m essages. Ther efor e, it is good pr act ice t o t ur n off console logging t o keep t he console por t fr ee for m aint enance. A com m on I SP st r at egy is t o use t he r out er v t y por t s ( w it h Telnet or Secur e Shell) for day - t oday adm inist r at ion and t hen t o r est r ict t he console por t for em er gency uses only . Oft en t his is aided or enfor ced by connect ing t he console t o t he out - of- band m anagem ent in place at t he I SP’s point of pr esence. I n r eview , w it h all t he opt ions used, a t ypical I SP logging configur at ion w ould look like t he follow ing:

service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone ! no logging console logging buffered 16384 logging trap debugging logging facility local7 logging 169.223.32.1 logging 169.223.45.8 logging source-interface loopback0 !

Syslog Topologie s I SPs use one of t w o syslog t opologies in t heir net w or ks. The fir st is a t r adit ional cent r alized sy slog infr ast r uct ur e t hat has t he sy slog ser v er s in a cent r al locat ion support ing all t he logging flow s fr om t he net w or k dev ices in t he net w or k ( see Figure 1- 3) . This is locat ed in one of t he I SP’s m aj or dat a cent ers or t he I SP’s NOC. The adv ant age w it h t his t opology is t hat all t he r aw syslog dat a is quickly available in one locat ion. The disadv ant age of t his t opology is one of scaling t he am ount of sy slog dat a com ing t o one locat ion and t he possibilit y of losing log infor m at ion sent acr oss long dist ances of t he net w or k dur ing t im es of congest ion. Fig u r e 1- 3 . Ta ilin g a Ce n t r a liz e d Sy slog File f or a Cisco Rou t e r

36

The second t opology is a dist r ibut ed t opology . Sy slog ser v er s ar e pushed out t o t he edges of t he net w or k or ar e spr ead acr oss sev er al dat a cent er s. For ex am ple, each m aj or PoP w ould hav e a sy slog ser v er for all t he dev ices in t he PoP. Sy slog dat a is eit her pr epr ocessed on t he r em ot e ser ver or pulled fr om t he dist r ibut ed ser v er s t o cent ralize t he server for processing. This t opology is m ore scalable for larger net w or k s and is less lik ely t o be im pact ed by net w or k congest ion or ot her pr oblem s in t he backbone. Eit her of t hese t w o t opologies ( and ot her t opolog ies) w ill w or k . What is im por t ant is t hat t he sy slog infor m at ion is collect ed and act ually used t o m aint ain y our net w or k .

Ana lyzing Syslog Da t a Configur ing t he r out er s t o ex por t sy slog dat a is one st ep. The nex t st ep is t o st or e t he dat a, analy ze it , and use it in day- t o- day oper at ions. I nt er face st at us, secur it y aler t s, and debugging pr oblem s ar e som e of t he m ost com m on ev ent s t hat I SPs m onit or fr om t he collect ed sy slog dat a ( an ex am ple of t he out put fr om t he collect ed syslog dat a is in Figur e 1- 3) . Som e use cust om- w r it t en Per l scr ipt s t o cr eat e sim ple r epor t s. Ot her s use m or e sophist icat ed soft w ar e t o analy ze t he sy slog dat a and cr eat e HTML r epor t s, gr aphs, and char t s. Table 1- 4 is a list of know n available soft w ar e t hat analyzes syslog dat a. Even if you ar e going t o w r it e y our ow n scr ipt s, it ’s w or t h check ing out t he com m er cial packages t o see w hat can be done w it h sy slog dat a.

Cisco Resour ce Man ager

Table 1-4. Software That Analyzes Syslog Data ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ w r 2k / r sm n/ index .sht m l

Pr iv at e I

ht t p: / / w w w .opensy st em s.com / index .asp

Cr y st al Repor t s

ht t p: / / w w w .seagat esoft w ar e.com / cr y st alr epor t s/

Net for ensics

ht t p: / / w w w .net forensics.com /

One it em t o r em em ber w it h t he I SP’s syslog infr ast r uct ur e is t his: Tim e sy nchr onizat ion is cr it ical! To com par e logs fr om t w o r out er s in differ ent par t s of t he net w or k , t he t im e on t hem m ust be sy nchr onized w it h t hat on t he sy slog ser v er . Hence, t he I SP m ust t ak e t he effor t t o deploy NTP in it s net w or k , ensur ing t hat t he ent ir e net w or k and sy st em s infr ast r uct ur e ar e in t im e sy nc.

37

N e t w or k Tim e Pr ot ocol Tim e sy nchr onizat ion acr oss t he I SP’s net w or k is one of t hose least t alk ed about y et cr it ical pieces of t he net w or k. Wit hout som e m echanism t o ensur e t hat all devices in t he net w or k ar e sy nchr onized t o ex act ly t he sam e t im e sour ce, funct ions such as accou nt ing, ev ent logging, fault analy sis, secur it y incident r esponse, and net w or k m anagem ent w ould not be possible on m or e t han one net w or k device. Whenever an I SP’s sy st em or net w or k engineer needs t o com par e t w o logs fr om t w o differ ent sy st em s, each sy st em needs a fr am e of r efer ence t o m at ch t he logs. That fr am e of r efer ence is sy nchr onized t im e. The Net w or k Tim e Pr ot ocol ( NTP) is pr obably t he m ost ov er look ed configur at ion feat ur e on an I SP’s net w or k . NTP is a hier ar chical pr ot ocol designed t o sy nchr onize t h e clock s on a net w or k of com put ing and com m unicat ion equipm ent . I t is a dy nam ic, st able, r edundant pr ot ocol used t o k eep t im e sy nchr onized bet w een net w or k devices t o a gr anular it y of 1 m s. Fir st defined in RFC 958, NTP has since been m odified t o add m ore re dundancy and secur it y . Ot her RFCs for t im e sy nchr onizat ion include t he follow ing: • • • • •

RFC 1128, “ Measur ed Per for m ance of t he Net w or k Tim e Pr ot ocol in t he I nt ernet Syst em ,” 1989 RFC 1129, “ I nt er net Tim e Sy nchr onizat ion: The Net w or k Tim e Pr ot ocol,” 1989 RFC 1165, “ Net w or k Tim e Pr ot ocol ( NTP) ov er t he OSI Rem ot e Oper at ions Ser v ice,” 1990 RFC 1305, “ Net w or k Tim e Pr ot ocol ( Ver sion 3) Specificat ion,” 1992 ( dr aft st andard) RFC 2030, “ Sim ple Net w or k Tim e Pr ot ocol ( SNTP) Ver sion 4 for I Pv4, I Pv6, and OSI ,” 1996 ( inform a t ional)

An NTP net w or k usually get s it s t im e fr om an aut hor it at iv e t im e sour ce, such as a r adio clock , a global posit ioning sy st em ( GPS) dev ice, or an at om ic clock at t ached t o a t im e server. NTP t hen dist ribut es t his t im e across t he net w ork. NTP is hierarc hical, w it h differ ent t im e ser v er s m aint aining aut hor it y lev els. The highest aut hor it y is St r at um 1. Levels of aut hor it y t hen descend fr om 2 t o a m axim um of 16. NTP is ext r em ely efficient ; no m or e t han one packet per m inut e is necessar y t o synchr onize t w o m achines t o wit hin a m illisecond of one anot her.

N TP Ar chit e ct ur e [ 2 ] I n t he NTP m odel, a num ber of pr im ar y r efer ence sour ces, synchr onized by w ir e, GPS, or radio t o nat ional st andar ds, ar e connect ed t o w idely accessible r esour ces, such as back bone gat ew ay s, and ar e oper at ed as pr im ar y t im e ser v er s. NTP pr ov ides a pr ot ocol t o pass t im ek eeping infor m at ion fr om t hese ser v er s t o ot her t im e ser v er s fr om t he I nt er net and t o c ross- check clocks and cor r ect er r or s ar ising fr om equipm ent or pr opagat ion failur es. Local- net host s or gat ew ays, act ing as secondar y t im e ser v er s, use NTP t o com m unicat e w it h one or m or e of t he pr im ar y ser v er s. To r educe t he pr ot ocol ov er head, t he secondar y ser ver s dist r ibut e t im e t o t he r em aining local- net host s. For r eliabilit y , select ed host s ar e equipped w it h less accur at e ( and less expensive) r adio clocks. These host s ar e used for backup in case of failur e of t he pr im ar y or secondar y ser v er s or t he com m unicat ion pat hs bet ween t hem .

38

The NTP “ net w or k ” consist s of a m ult iple r edundant hier ar chy of ser v er s and client s, w it h each lev el in t he hier ar chy ident ified by a st r at um num ber . This num ber specifies t he accur acy of each ser v er , w it h t he t opm ost lev el ( pr im ar y ser v er s) assigned as 1 and each lev el dow nw ar d ( secondar y ser v er s) in t he hier ar chy assigned as one gr eat er t han t he pr eceding level. St r at um 1 is populat ed w it h host s w it h bus or ser ial int er faces t o r eliable sour ces of t im e, such as r adio clock s , GPS sat ellit e t im ing r eceiver s, or at om ic clocks. St r at um 2 ser ver s m ight be com pany or cam pus ser v er s t hat obt ain t im e fr om som e num ber of pr im ar y ser v er s ov er I nt er net pat hs and pr ov ide t im e t o m any local client s. The St r at um 2 ser v er s can be configure d t o peer w it h each ot her , com par ing clock s and gener at ing a sy nchr onized t im e v alue. NTP per for m s w ell ov er t he nondet er m inist ic pat h lengt hs of pack et - sw it ched net w or ks because it m akes r obust est im at es of t hr ee key var iables in t he r elat ionship bet w een a client and a t im e ser v er . These t hr ee v ar iables ar e net w or k delay , disper sion of t im e packet exchanges ( a m easur e of m axim um clock er r or bet w een t he t w o host s) , and clock offset ( t he cor r ect ion t o apply t o a client ’s clock t o sy nchr onize it ) . Clock sy nchronizat ion at t he 10 m s level over long- dist ance ( 2000 km ) WANs and at t he 1 m s level for LANs is r out inely achieved. Ther e is no pr ov ision for peer discov er y or v ir t ual- cir cuit m anagem ent in NTP. Dat a int egrit y is provided by t he I P and UDP checksum s. No flow - cont r ol or r et r ansm ission facilit ies ar e pr ov ided or necessar y . Duplicat e det ect ion is inher ent in t he pr ocessing algor it hm s. NTP uses a syst em call on t he local host t o skew t he local syst em clock by a sm all am ount t o k eep t he clock sy nchr onized. I f t he local clock ex ceeds t he “ cor r ect ” t im e by a pr eset t hr eshold, NTP uses a syst em call t o m ake a st ep adj ust m ent of t he local clock. NTP is car eful t o av oid sy nchr onizing t o a sy st em w hose t im e m ight not be accur at e. I t avoids doing so in t w o w ays. Fir st , NTP w ill never synchr onize t o a syst em t hat is not it self sy nchr onized. Second, NTP com par es t he t im e r epor t ed by sev er al sy st em s and w ill not sy nchr onize w it h a sy st em w hose t im e is significant ly differ ent fr om t he ot hers, even if it s st rat um is lower.

Client / Server M odels a nd Associa t ion M odes NTP ser ver s can associat e w it h each ot her in a num ber of m odes. The m ode of each ser v er in t he pair indicat es t he behav ior t hat t he ot her ser v er can ex pect fr om it . An associat ion is for m ed w hen t w o peer s ex change m essages and one or bot h of t hem cr eat e and m aint ain an inst ant iat ion of t he pr ot ocol m achine. The associat ion can oper at e in one of sev er al m odes: ser v er , client , peer , and br oadcast / m ult icast . The m odes fur t her ar e classified as act iv e and passiv e. I n act iv e m odes, t he host cont inues t o send NTP m essages r egar dless of t he r eachabilit y or st r at um of it s peer . I n passive m odes, t he host sends NTP m essages only as long as it s peer is r eachable and oper at ing at a st r at um lev el less t han or equal t o t he host ; ot herwise, t he peer associat ion is dissolv ed. •

Se r v e r m od e — By operat ing in server m ode, a host ( usually a LAN t im e ser v er ) announces it s w illingness t o sy nchr onize, but not t o be sy nchr onized

39

•

•

•

by a peer . This t y pe of associat ion or dinar ily is cr eat ed upon ar r iv al of a client r equest m essage and ex ist s only t o r eply t o t hat r equest ; aft er t hat , t he associat ion is dissolved. Server m ode is a passive m ode. Clie n t m ode — By operat ing in client m ode, t he host ( usually a LAN w or k st at ion) announces it s w illingness t o be sy nchr onized by but not t o sy nchr onize t he peer . A host oper at ing in client m ode sends per iodic m essages r egar dless of t he r eachabilit y or st r at um of it s peer . Client m ode is an act ive m ode. Pe e r m od e — By operat ing in peer m ode ( also called sy m m et ric m ode) , a host announces it s w illingness t o sy nchr onize and be sy nchr onized by ot her peer s. Peer s can be configur ed as act ive ( sym m et r ic - act ive) or passive ( sym m et ric - passive) . Br oa d ca st / m u lt ica st m od e — By oper at ing in br oadcast or m ult icast m ode, t h e host ( usually a LAN t im e server operat ing on a high- speed br oadcast m edium ) announces it s w illingness t o sy nchr onize all of t he peer s, but not t o be sy nchr onized by any of t hem . Br oadcast m ode r equir es a br oadcast ser v er on t he sam e subnet , w hile m ult icast m ode r equir es suppor t for I P m ult icast on t he client m achine as w ell as connect iv it y t hr ough t he MBONE t o a m ult icast ser v er . Br oadcast and m ult icast m odes ar e act iv e m odes.

Nor m ally, one peer oper at es in an act ive m ode ( sym m et r ic - act ive, client , or broadcast / m ult icast m odes) , w hile t he ot her oper at es in a passiv e m ode ( sym m et ric - passive or ser ver m odes) , oft en w it hout pr evious configur at ion. How ev er , bot h peer s can be configur ed t o oper at e in t he sy m m et r ic - act ive m ode. An er r or condit ion r esult s w hen bot h peer s oper at e in t he sam e m ode, ex cept for t he case of sym m et ric - act ive m ode. I n t his case, each peer ignor es m essages fr om t he ot her so t hat pr ev ious associat ions, if any , w ill be dem obilized due t o r eachabilit y failure.

I m plem ent ing N TP on an I SP’s Rout e r s The t im e kept on a m achine is a crit ical resource, so w e st rongly recom m end t hat y ou use t he secur it y feat ur es of NTP t o av oid t he accident al or m alicious set t ing of an incor r ect t im e. Tw o m echanism s ar e av ailable: an access list –based r est r ict ion schem e and an encr y pt ed aut hent icat ion m echanism . The follow ing ex am ple highlight s bot h NTP secur it y opt ions. Cisco’s im plem ent at ion of NTP does not suppor t St r at um 1 ser v ice; in ot her w or ds, it is not possible t o connect a r out er r unning I OS Soft w ar e dir ect ly t o a r adio or at om ic clock . I t is r ecom m ended t hat t im e ser v ice for y our net w or k be der iv ed fr om t he public NTP ser ver s available in t he I nt er net . I f t he net w or k is isolat ed fr om t he I nt er net , Cisco’s im plem ent at ion of NTP allow s a sy st em t o be configur e d so t hat it act s as t hough it is synchr onized w it h NTP, w hen, in fact , it has det er m ined t he t im e using ot her m eans. Ot her sy st em s t hen sy nchr onize t o t hat sy st em w it h NTP. The com m and t o set up a rout er in t his way is

ntp master 1 This t ells t he r out er t hat it is t he m ast er t im e sour ce and is r unning at St r at um 1.

40

The follow ing exam ple is an NTP configur at ion on a r out er get t ing a St r at um 2 ser ver connect ion fr om 192.36.143.150 and peer ing w it h 169.223.50.14. The peer ed I P addr esses ar e t he loopback add r esses on each r out er . Each r out er is using t he loopback as t he sour ce. This m ak es secur it y easier ( not e t he access list ) .

clock timezone SST 8 ! access-list 5 permit 192.36.143.150 access-list 5 permit 169.223.50.14 access-list 5 deny any ! ntp authentication-key 1234 md5 104D000A0618 7 ntp authenticate ntp trusted-key 1234 ntp source Loopback0 ntp access-group peer 5 ntp update-calendar ntp server 192.36.143.150 ntp peer 169.223.50.14 !

N TP D e ploym e nt Ex a m ple s I SPs use sev er al designs t o deploy NTP on t heir backbones. I t is har d t o r ecom m end a best cur r ent pr act ice, but t his sect ion list s a few ex am ples: •

•

•

Fla t pe e r st r u ct u r e — All t he r out er s peer w it h each ot her , w it h a few geogr aphically separ at e r out er s configur ed t o point t o ex t er nal sy st em s. From ex per ience, t his is v er y st able, but conv er gence of t im e w ill be longer w it h each new m em ber of t he NTP m esh. The lar ger t he m esh is, t he longer it t ak es for t im e t o conv er ge. H ie r a r ch y — The BGP r out e r eflect or hier ar chy is copied for t he NTP hier ar chy . Cor e r out er s ( r out e r eflect or s) hav e a client / ser v er r elat ionship w it h ex t er nal t im e sour ces, t he r eflect or client s hav e a client / ser v er r elat ionship w it h t he cor e r out er s, t he cust om er r out er s hav e a client / ser v er r elat ionship w it h t he r eflect or client s, and so on dow n t he t r ee. Hier ar chy is pr ov en t o scale w ell. A sim ple hier ar chy t hat m at ches t he r out ing t opology pr ov ides consist ency , st abilit y , and scalabilit y—hence, it is our fav or it e t echnique. St a r— Here all t he I SP rout ers have a client / ser v er r elat ionship w it h a few t im e ser v er s in t he I SP’s back bone. The dedicat ed t im e ser v er s ar e t he cent er of t he st ar . The dedicat ed t im e ser v er s ar e usually UNI X sy st em s sy nchr onized w it h ex t er nal t im e sour ces or t heir ow n GPS r eceiv er . This set up also is r epor t ed t o be v er y st able.

Undoubt edly t her e ar e ot her possibilit ies, t oo. The m ain aim is t o go for st abilit y because t im e sy nchr onizat ion is a k ey t ool w it hin t he I SP back bone.

41

N TP in a PoP ( Ex a m ple ) Dev ices in an I SP PoP do not need t o be par t of t he backbone NTP m esh. I nst ead, t he dev ices in t he PoP ( r out er s, NAS, sw it ches, and w or k st at ions) use t he t w o cor e PoP gat ew ay r out er s as t he NTP ser ver s for t he PoP. All devices w ill use bot h r out er s as NTP sour ces, sim plify ing t he NTP configur at ion and dec reasing t he NTP conv er gence t im e in t he PoP. As can be seen in Figure 1- 4, dev ices in a PoP all need t im e sy nchr onizat ion. Account ing on t he RADI US ser v er needs t o be sy nchr onized w it h t he NAS equipm ent , w hich needs t o be sy nchr onized w it h t he sy slog ser v er , w hich needs t o be sy nchr onized w it h t he access r out er s, w hich needs t o be sy nchr onized w it h t he Net Flow collect or s, and so on. Hav ing all dev ices use t he sam e t w o servers ( one pr im ar y , one back up) ensur es t im e sy nchr onizat ion am ong all dev ices. Fig u r e 1- 4 . Ty p ica l I n t e r n e t PoP Bu ilt f or Re d u n d a n cy a n d Re lia b ilit y U sin g t h e Cor e Rou t e r s a s N TP Se r v e r s

Configur at ion is sim plified w it h only t w o ser v er s. For ex am ple, NAS 1, a Cisco 3640 wit h 96 built - in m odem s, w ould have a configurat ion highlight ed in Ex am ple 1- 2. PoP gat ew ay r out er s Cor e 1 and Cor e 2 hav e t w o configur at ion opt ions. Fir st , each dev ice in t he PoP can be m anually configur ed w it h an n t p p e e r com m and. Ev en t hough t he peer com m ands open t he gat ew ay r out er s t o hav e t he capabilit y t o allow sy nchr onizat ion, t he n t p se r v e r com m ands on t he PoP devices w ill m ake t his unlik ely . Yet , t her e is alw ay s t he chance of m aint enance- induced t r ouble ( MI T)— m isconfigur at ion on t he PoP gat ew ay or on one of t he PoP net w or k dev ices.

42

Ther efor e, a second opt ion offer s m or e pr ot ect ion. This second opt ion uses t he nt p a cce ss- group com m and t o lim it w hat can quer y , ser v e, and be an NTP peer . Exam ples 1- 1 and 1- 2 dem onst r at e how t he n t p a cce ss- group com m and is used t o add an ext r a layer of secur it y for all t he NTP peer s on t he I SP’s backbone w hile allow ing a gener al access list t o cover all t he devices in t he PoP. I f t he I SP is follow ing a logical addr essing plan, t he w hole PoP w ill be assigned one block of I P addr esses for all t he infr ast r uct ur e and loopback addr esses. This m ak es t he nt p a cce ss- gr ou p se r v e - only ACL easier t o cr eat e, w it h one ACL cover ing t he ent ir e PoP. Ex a m p le 1- 1 N TP Con f ig u r a t ion f or t h e PoP Ga t e w a y Rou t e r s ! PoP Gateway Router ! ntp authentication-key 4235 md5 ISPWorkshop ntp authenticate ntp trusted-key 4235 ! ! Lock NTP source to the uplink ! ntp source Loopback0 ntp update-calendar ! ! List of NTP Peers - Adding an additional Security Layer ! ntp access-group peer 99 ! ! Allow PoP Devices to use this router as an NTP Server ntp access-group serve-only 42 ! ! Loopback Addresses of the Backbone Routers ntp peer 200.200.1.1 ntp peer 200.200.1.2 ntp peer 200.200.1.3 ntp peer 200.200.1.4 ! Ex a m p le 1- 2 N TP D e v ice s in t h e PoP ! PoP Devices ! ntp authentication-key 4235 md5 ISPWorkshop ntp authenticate ! ntp trusted-key 4235 ! ! Lock NTP source to the uplink ntp source Loopback0 ntp update-calendar ! ! IP Addresses to Routers Core 1 and Core 2 ntp server 192.135.248.249 ntp server 192.135.248.250 !

43

Fur t he r N TP Re fe r e nce s Table 1- 5 show s som e URLs w it h fur t her point er s t o NTP infor m at ion, soft w ar e, and har dw ar e. Table 1-5. Useful NTP URLs Net w or k Tim e Pr ot ocol ( NTP) Mast er Clock for t he U.S.

ht t p: / / t ycho.usno.navy.m il/

Dat um I nc, Bancom m Tim ing Division

ht t p: / / w w w .dat um .com /

Tr ue Tim e, I nc.

ht t p: / / w w w . t r uet im e. com

The Tim e Web Serv er ( Tim e Sy nc) , by Dave Mills

ht t p: / / w w w .eecis.udel.edu/ ~ nt p/

Coet anian Sy st em s Tim e Sy nchr onizat ion ht t p: / / w w w .coet anian. com / t ss/ t ss1 0 0 . ht m Server 100

Sim ple N e t w or k M a n a ge m e n t Pr ot ocol Keeping dat a on t he healt h of an I SP’s net w or k is cr it ical t o it s sur vival as a business. For exam ple, an I SP m ust know t he load on it s backbone cir cuit s and t he load on it s cust om er cir cuit s. I t also needs t o k eep t r ack of t he pack et s lost on r out er s at v ar ious point s of t he net w or k . And it needs t o be aw ar e of long- t erm t r ends on t he ov er all gr ow t h of t he net w or k . SNMP can collect and pr ocess all of t his d at a—hence, it is a v er y cr it ical ut ilit y for net w or k engineer s. Giv en t he w ide r ange of fr eew ar e, shar ew ar e, and com m er cially av ailable SNMP t ools, all I SPs should be capable of collect ing SNMP dat a, pr ocessing it , gr aphing it , and analy zing it for pr oper t r affic engineer ing. Appendix E, “ Tr affic Engineer ing Tools,” list s point er s t o v ar ious soft w ar e and t ools on t he Net . Rem em ber t hat SNMP, especially v er sion 1, has v er y w eak secur it y ! I f SNMP w ill not be used, t ur n it off! On t he ot her hand, ensur e t hat sufficient configur at ion infor m at ion is pr esent t o cont r ol t he use of SNMP so t hat it doesn’t becom e a secur it y r isk. Most im por t ant ly, never leave a configur at ion t hat includes “ public” or “ pr iv at e” as t he com m u nit y st ring — t hese st r ings ar e so w ell k now n and ar e com m on default s on har dw ar e shipping fr om so m any v endor s t hat t hey ar e open inv it at ions t o abuse, filt er s or not .

SN M P in Re a d- Only M ode I f SNMP is used in a read- only scenar io, ensur e t hat it is set up w it h appropriat e access cont r ols. The follow ing is an ex am ple:

! access-list 98 permit 215.17.34.1 access-list 98 permit 215.17.1.1 access-list 98 deny any

44

! snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server snmp-server !

community 5nmc02m RO 98 trap-source Loopback0 trap-authentication enable traps config enable traps envmon enable traps bgp enable traps frame-relay contact Barry Raveendran Greene [[email protected]] location Core Router #1 in City Y host 215.17.34.1 5nmc02m host 215.17.1.1 5nmc02m tftp-server-list 98

Not e t he applicat ion of a cce ss- list 9 8 . The com m unit y st r ing 5 n m c0 2 m is not encr y pt ed, hence t he need t o use an access list t o cont r ol access. This is t oo oft en for got t en, and if t he com m unit y st r ing is k now n out side t he I SP, it can easily lead t o t he com pr om ise of a r out er . I n fact , som e scr ipt s av ailable on t he I nt er net allow script kiddies [ 3 ] t o pr obe a r out er and cr ack t he com m unit y nam e. Unless bot h SNMP is set up t o send a t r ap on com m unit y nam e aut hent icat ion failur e an d t h e SNMP m anagem ent dev ice is configur ed t o r eact t o t he aut hent icat ion failur e t r ap, t he scr ipt k iddies m ost lik ely w ill discov er t he com m unit y nam e. I SPs alw ay s should r em em ber t hat accept ing SNMP only fr om k now n good I P addr esses does not guar ant ee t heir secur it y . Unless t he I SP has som e v er y ser ious ant ispoofing m easur es in place, it cannot com plet ely r ely on I P addr esses for t he pr im ar y secur it y of any sy st em . I P addr esses fr equent ly ar e spoofed, so lay er ed secur it y in w hich t he syst em r elies on sever al m echanism s has pr oven t o be m or e effect iv e. Th e sn m p- se r v e r h ost configurat io n list s t he host s t o w hich SNMP infor m at ion is sent — if t her e is no m eans of collect ing SNMP t r aps, don’t configur e sn m p- se r v e r h ost , sav ing CPU cy cles and net w or k bandw idt h. I SPs should ensur e t hat t he sn m pse r v e r h ost is configur ed t o r eceiv e and r espond t o SNMP t r aps. For ex am ple, a Per l script on a PC running UNI X or Linux w it h UCD SNMP [ 4 ] could r eceiv e an SNMP env ir onm ent al t r ap r elat ing t o high r out er int er nal t emper at ur e, e- m ail it t o t he NOC alias, send an alphanum er ic page t o t he on- dut y engineer , and open a t r ouble t ick et . SNMP can be set up w it h m or e t han one com m unit y , w it h a differ ent access list . This is quit e useful, say, if different groups in t he com pany r equir e differ ent SNMP access t o t he r out er , or if a t r ansit I SP offer s SNMP r ead access t o it s I SP cust om er s on it s aggr egat ion r out er . An exam ple m ight look like t he follow ing:

! access-list access-list ! access-list access-list access-list ! snmp-server

97 permit 220.3.20.1 97 deny any 98 permit 215.17.34.1 98 permit 215.17.1.1 98 deny any community 5nmc02m RO 98

45

snmp-server community 1spxs RO 97 !

SN M P in Re a d- W rite Mode I f SNMP w ill be used in read- w r it e m ode, t hink ver y car efully about t he configur at ion and w hy t her e is a r equir em ent t o do t his. Configur at ion er r or s in t his scenar io could leave t he r out er ver y vulner able. I f possible, put an ACL at t he edge of your net w or k t o pr event out side par t ies fr om pr obing y our net w or k w it h SNMP. Many publicly and com m er cially av ailable t ools w ill scan any net w or k on t he I nt er net w it h SNMP. This could m ap out your ent ir e net w or k or discover a device t hat has had SNMP left open Re com m e n da t ion I n short , our recom m endat ion is not t o use SNMP in read- writ e m ode on any I SP r out er—t he oper at ional r isk s m or e t han out w eigh any adv ant ages.

SN M P and Com m ercial N et w ork M anagem ent Soft w are One t hing t o be aw ar e of is t hat som e com m er cial net w or k m anagem ent soft w ar e lik es t o t ak e ov er t he net w or k by doing aut odiscov er y of dev ices on t he back bone. Many I SP engineer s don’t appr ov e of t his st y le of net w or k m anagem ent and t end t o build t heir ow n t ools t hat ar e suit able for m onit or ing t he backbone. For t hose I SPs t hat r ely on com m er cial pack ages, such as HP OpenView , it is w or t h r em em ber ing and under st anding t he im pact t hat t he aut odiscov er y funct ion has. Aut odiscov er y fit s v er y nicely int o a cam pus net w or k w her e t he net w or k oper at or s hav e lit t le cont r ol over t he backbone and w hat is added t o it . How ever , t he net w or k oper at ions st aff should be in cont r ol of an I SP backbone, and all addit ions and r em ovals fr om t he back bone should be pr em edit at ed. One exam ple follow s. An I SP w as using a com m er cial package t o m onit or it s back bone and w as being sev er ely t r oubled by v er y high CPU ut ilizat ion. Aft er a lot of r esear ch, it w as discov er ed t hat t he aut odiscov er y funct ion of t he m anagem ent sy st em w as dow nloading t he ent ir e r out ing t able fr om each of t he r out er s. This I SP, of cour se, w as car r ying all 80 KB pr efixes t hat w er e pr esent at t he t im e. Giv en t he CPU hit dealing w it h a full BGP rout ing t able, it is lit t le w onder t hat an SNMP poll of t he r out ing t able w as causing high CPU usage. The w or kar ound w as t he follow ing SNMPv 2 configur at ion:

snmp-server snmp-server snmp-server snmp-server snmp-server

view cutdown internet included view cutdown ipRouteTable excluded view cutdown ipNetToMediaTable excluded view cutdown at excluded community public view cutdown RO

46

For r out er s t hat r un I OS Soft w ar e r eleases t hat don’t t ak e t he pr ec eding obj ect nam es, t he follow ing w ill also achiev e t he sam e r esult :

snmp-server snmp-server snmp-server snmp-server snmp-server

view cutdown internet included view cutdown ip.21 excluded view cutdown ip.22 excluded view cutdown mib-2.3 excluded community public view cutdown RO

This basically st ops t he r out er fr om r esponding t o SNMP quer ies r egar ding t he rout ing and ARP t ables.

H TTP Se r v e r The HTTP ser v er is a new feat ur e in 11.1CC and 12.0 soft w ar e t hat , w hen configur ed and enabled, allow s t he net w ork oper at or t o v iew and configur e t he r out er t hr ough a conv enient and easy - t o- use w eb int er face ( w it h com m on br ow ser s such as Net scape or I nt er net Explor er ) . This is int ended as an alt er nat ive t o t he CLI descr ibed ear lier in t his chapt er , m ainly aim ed at cust om er s w ho w ould pr efer t o “ point and click” t heir w ay t hr ough configur ing net w or k equipm ent . Because m ost I SPs have been using t he CLI for m any years, very few ( if any) act ually use t he HTTP ser v er capabilit y . I t is w or t h check ing t hat t he HTTP ser v er has not been enabled by default , or in er r or , or dur ing sy st em inst allat ion. This configur at ion com m and w ill ensur e t hat t he ser v er is not r unning:

no ip http server I f t her e is a need t o configur e t he HTTP ser v er because w eb- based configur at ion of t he rout er is desir ed, w e st r ongly adv ise t hat t he ser v er be configur ed w it h t he appr opr iat e secur it y—for exam ple,

ip http server ip http port 8765 ip http authentication aaa has ip http access-class

! use a non-standard port ! use the AAA authentication method which ! been configured ! access-list to protect the HTTP port

Not ice t he suggest ion of a nonst andar d por t . This adds a lit t le obscur it y t o t he w eb ser v er on t he r out er , m ak ing pot ent ial at t ack a lit t le m or e difficult . Also not ice t he access list used and t he aut hent icat ion t y pe ( AAA is discussed in t he sect ion on r out er secur it y ) . This ensur es t hat only t he per m it t ed adm inist r at iv e user s of t he r out er get access t o t he device fr om t he aut hor ized I P addr ess r ange. Re com m e n da t ion I n shor t , our r ecom m endat ion is not t o use t he HTTP ser ver on any I SP r out er or sw it ch—t he oper at ional r isk far out w eighs any adv ant age.

47

Cor e D u m ps A core dum p facilit y has been part of I OS Soft w are for several years and m any soft w ar e r eleases. The cor e dum p facilit y oper at es lik e t he UNI X v ar iant —w hen a pr ogr am cr ashes, t he m em or y im age is st or ed in a cor e file. When a r out er cr ashes, a copy of t he cor e m em or y is k ept . Befor e t he m em or y is er ased upon r eboot , t he r out er can be set up t o copy t he cor e dum p out t o a UNI X ser v er . An account ( FTP, TFTP, or RCP) and sufficient disk space ( equal t o t he am ount of m em or y on t he r out er per dum p) needs t o be set up and allocat ed. Her e is an ex am ple using FTP:

ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password 7 045802150C2E ! exception protocol ftp exception dump 169.223.32.1 ! Not e t he use of t he loopback int er face as a sour ce int er face. I t is r ecom m ended t hat access t o t he cisco account be m ade as secur e as possible. For exam ple, do not send cor e dum ps t o t he sam e FTP ser v er as t he one used t o pr ov ide gener ic anony m ous or user FTP account s. Use a w r apper for t he FTP daem on, and m ake sur e t hat only t he loopback int er faces ar e list ed in any sy st em filt er list s. Be aw ar e t hat RCP is inher ent ly insecur e and t hat it s use cannot be r ecom m ended ov er a public net w or k . Also, TFTP cor e dum ps ( w hich ar e t he default in I OS Soft w ar e) suppor t syst em m em or y sizes only up t o 16 MB. Gener ally, it is r ecom m ended t hat FTP cor e dum ps be configured w hat ever t he sit uat ion or rout er har dw ar e configur at ion is. Mor e det ailed infor m at ion for configur ing cor e dum ps on a Cisco I OS Soft w ar e –based sy st em is locat ed on t he Cisco Docum ent at ion CD. I t is publicly av ailable on t he Cisco. com sit e at ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / it g_v 1/ t r 19aa. ht m. I t includes infor m at ion needed t o t r oubleshoot pr oblem s using t he cor e du m p and sh ow st a ck s com m ands.

Con clu sion This chapt er has descr ibed som e of t he m anagem ent issues t hat I SPs should consider for t he inst allat ion and oper at ion of t heir net w or k equipm ent . I t has w or k ed t hrough m ore or less in chr onological or der t he basic st eps t hat an I SP should t ak e befor e deploy ing a new piece of equipm ent . These st eps include det er m ining w hich v er sion of oper at ing sy st em soft w ar e t o choose for t he r out er s, get t ing t he chosen v er sion on t o t he equipm ent , and em ploy ing t he v ar ious st r at egies for m anagem ent of t he r out er oper at ing soft w ar e and configur at ion. Fr equent ly used feat ur es of t he CLI w ere int roduced as w ell.

48

The chapt er concluded w it h som e basic feat ur es of r out er m anagem ent , including best pract ic es for capt ur ing logging infor m at ion, configur ing t im e sy nchr onizat ion, using SNMP, using HTTP access, and dealing w it h cr ashes.

En dn ot e s 1. Yes, t her e ar e m any I SPs in t he w or ld w hose ent ir e backbone is built on 2500s! 2. This sect ion w as w r it t en for Cisco’s DNS/ DHCP Manager . Sect ions of t he docum ent at ion on NTP hav e been included her e. The com plet e docum ent can be found at ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / iaabu/ cddm / cddm 111/ adg u id e/ n t p . h t m 3. Scr ipt k iddies ar e am at eur cr ack er s w ho use scr ipt s t o br eak int o and cause dam age t o net w or k s and sy st em s on t he I nt er net . 4. CMU SNMP has not been updat ed in a w hile, and t he pr oj ect w as t ak en ov er by UCD. How ev er , in Oct ober 2000, t he pr oj ect w as m ov ed ov er t o SourceForge and renam ed NET- SNMP. I t has been subst ant ially im proved and m odified over t he or iginal CMU and UCD ver sions. I t is by far t he best and m ost configur able sy st em—and it ’s fr ee! Visit ht t p: / / net snm p. sour cefor ge. net / for t he NET- SNMP proj ect hom e page.

49

Cha pt e r 2 . Ge ne r a l Fe a t ur e s This chapt er cover s gener al feat ur es t hat I SPs should consider for t heir r out er s and net w or k im plem ent at ions. Most ar e good design pr act ices and don’t lev er age par t icular unique Cisco I OS Soft w ar e feat ur es, but each dem onst r at es how I OS Soft w ar e can aid t he sm oot h oper at ion of an I SP’s business. Many of t he feat ur es discussed here ar e descr ibed in t he cont ex t of t he I SP soft w ar e cov er ed in Chapt er 1, “ Soft w ar e and Rout er Managem ent .” The im por t ance of t he loopback int er face should nev er be ov er look ed, especially for gener al oper at ions and m anagem ent of t he r out er . I ndeed, it is sur pr ising how few I SPs m ake use of t his t im e - sav ing r esour ce. The chapt er cont inues w it h a discussion on how t o configur e r out er int er faces and check t heir st at us. Follow ing t he discussion of basic m anagem ent configur at ion is an int r oduct ion t o t he CEF and Net Flow capabilit ies t hat I SPs should be using on t heir r out er s. The chapt er finishes w it h a br ief look at Nagle befor e discussing t he im por t ance of t he DNS in an I SP’s operat io n.

I OS Sof t w a r e a n d Loop b a ck I n t e r f a ce s The use of t he loopback int er face is m ent ioned in m any inst ances t hr oughout t his book. Alt hough t his is not a feat ur e unique t o I OS Soft w ar e, t her e ar e m any and consider able adv ant ages in m ak ing full use of t he capabilit y t hat t he loopback int er face allow s. This sect ion br ings t oget her all t he occasions w her e t he loopback int er face is m ent ioned t hr oughout t he book and descr ibes how t hey can be useful t o t he I SP net w ork engineer.

M ot ivat ion for Using t he Loopback I nt erface I SPs endeav or t o m inim ize t he unnecessar y ov er head pr esent in t heir net w or k s. This unnecessar y ov er head can be t he num ber of net w or k s car r ied in t he I GP, t he num ber of sk illed engineer ing st aff t o oper at e t he net w or k , or ev en net w or k secur it y . Th e u t ilizat ion of one feat ur e, t he loopback int er face on t he r out er , goes a long w ay t o help w it h each of t he t hr ee scenar ios m ent ioned her e. Cont r ol of t he size of t he I GP is at t ended t o by sum m ar izat ion of point - t o- point addr esses at PoP or r egional boundar ies, t he use of I P unnum ber ed on st at ic WAN int er faces, and a car efully designed net w or k addr essing plan. I SP net w or k secur it y is of par am ount im por t ance, and any t echniques t hat m ak e t he m anagem ent sim pler ar e usually w elcom ed. For ex am ple, w hen r out er s access core servers, I SPs apply filt er s or access list s t o t hese ser v er s so t hat t he r isk of com pr om ise fr om t he out side is r educed. The loopback int er face is helpful her e as w ell. I t is v er y com m on t o assign all t he I P addr esses used for loopback int er faces from one address block. For exam ple, an I SP w it h around 200 rout ers in a net w ork m ight assign a / 24 net w or k ( 253 usable addr esses) for addr essing t he loopback int er face on each r out er . I f t his is done, all dependent syst em s can be configur ed t o per m it t his addr ess r ange t o access t he par t icular funct ion concer ned, w het her it is secur it y ,

50

unnum ber ed WAN link s, or t he iBGP m esh. Som e ex am ples of t he use of t he loopback int er face in t he I SP envir onm ent follow in t he r est of t his sect ion.

BGP Updat e Source I n t he follow ing exam ple, t he iBGP m esh is built using t he loopback int er face on each r out er . The loopback doesn’t ever disappear , w hich r esult s in a m or e st able iBGP, ev en if t he under ly ing phy sical connect iv it y is less t han r eliable.

hostname gateway1 ! interface loopback 0 ip address 215.17.1.34 255.255.255.255 ! router bgp 200 neighbor 215.17.1.35 remote-as 200 neighbor 215.17.1.35 update-source loopback 0 neighbor 215.17.1.36 remote-as 200 neighbor 215.17.1.36 update-source loopback 0 !

Rout er I D I f a loopback int er face is configur ed on t he r out er , it s I P addr ess is used as t he r out er I D. This is im por t ant for ensur ing st abilit y and pr edict abilit y in t he oper at ion of t he I SP’s net work. OSPF chooses t he designat ed r out er ( DR) on a LAN as t he dev ice t hat has t he highest I P address. I f rout ers are added or rem oved from t he LAN, or if a rout er gains an int er face w it h a higher addr ess t han t hat of t he exist ing DR, t he DR likely w ill change if t he DR or back up designat ed r out er ( BDR) fails. This generally is undesirable in an I SP net w ork because I SPs prefer t o have t he DR and BDR rout ers est ablished det er m inist ically. This change in DR and BDR can be avoided by ensur ing t hat t he loopback int er face is configur ed and in use on all r out er s on t he LAN. The loopback int er face is used for t he BGP r out er I D. I f t he loopback isn’t configur ed, BGP uses t he highest I P addr ess on t he r out er . Again, because of t he ev er- changing nat ur e of an I SP net w or k , t his v alue can change, possibly r esult ing in oper at ional confusion. Configur ing and using a loopback int er face ensur es st abilit y . N OTE I f t he r out er has t w o or m or e loopback int er faces configur ed, t he r out er I D is t he highest I P addr ess of t he configur ed loopback int er faces at t he t im e of boot ing t he rout er.

51

Except ion Dum ps by FTP Cisco r out er s can be configur ed t o dum p cor e m em or y t o an FTP ser v er as par t of t he diagnost ic and debugging pr ocess. How ev er , t his cor e dum p should be t o a sy st em not r unning a public FTP ser v er , but one heav ily pr ot ect ed by filt ers ( TCP Wr apper ev en) t hat allow only t he r out er s access. I f t he loopback int er face addr ess is used as sour ce addr ess fr om t he r out er and is par t of one addr ess block , t he filt er is v er y easy t o configur e. A 200- r out er net w or k w it h 200 dispar at e I P addr esses m ak es for a v er y lar ge filt er list on t he FTP ser v er . Ex am ine t he follow ing ex am ple I OS Soft w ar e configur at ion:

ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password 7 045802150C2E exception protocol ftp exception dump 169.223.32.1

TFTP Se r ve r Acce ss TFTP is t he m ost com m on t ool for uploading and dow nloading configur at ions. The TFTP ser v er ’s secur it y is cr it ical, w hich m eans t hat y ou should alw ay s use secur it y t ools w it h I P sour ce addr esses. I OS Soft w ar e allow s TFTP t o be configur e d t o use specific I P int er faces addr ess. This allow s a fix ed ACL on t he TFTP ser v er based on a fix ed addr ess on t he r out er ( for ex am ple, t he loopback int er face) .

ip tftp source-interface Loopback0

SN M P Server Access I f SNMP is used in t he net work, t he lo opback int er face again can be br ought int o use for secur it y access issues. I f SNMP t r affic fr om t he r out er is sour ced fr om it s loopback int er face, it is easy t o pr ot ect t he SNMP m anagem ent st at ion in t he NOC. A sam ple I OS Soft w ar e configur at ion follow s:

access-list access-list access-list ! snmp-server snmp-server snmp-server snmp-server snmp-server

98 permit 215.17.34.1 98 permit 215.17.1.1 98 deny any community 5nmc02m RO 98 trap-source Loopback0 trap-authentication host 215.17.34.1 5nmc02m host 215.17.1.1 5nmc02m

TACACS/ RADI US Server Source I nt erfa ce Most I SPs use TACACS+ or RADI US for user aut hent icat ion. Ver y few define account s on t he r out er it self because t his offer s m or e oppor t unit y for t he sy st em t o be

52

com pr om ised. A w ell- pr ot ect ed TACACS+ ser ver accessed only fr om t he r out er ’s loopback int er face addr ess block offer s m or e secur it y of user and enable account s. A sam ple configur at ion for st andar d and enable passw or ds follow s:

aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec start-stop tacacs+ ! ip tacacs source-interface Loopback0 tacacs-server host 215.17.1.2 tacacs-server host 215.17.34.10 tacacs-server key CKr3t# ! When using RADI US, eit her for user adm inist r at iv e access t o t he r out er or for dial user aut hent icat ion and account ing, t he r out er configur at ion t o suppor t loopback int er faces as t he sour ce addr ess for RADI US pack et s or iginat ing fr om t he r out er looks like t his:

radius-server host 215.17.1.2 auth-port 1645 acct-port 1646 radius-server host 215.17.34.10 auth-port 1645 acct-port 1646 ip radius source-interface Loopback0 !

N et Flow Flow Export Ex por t ing t r affic t hat flow s fr om t he r out er t o a Net Flow Collect or for t r affic analy sis or billi ng pur poses is quit e com m on. Using t he loopback int er face as t he sour ce addr ess for all expor t ed t r affic flow s fr om t he r out er allow s for m or e pr ecise and less cost ly filt er ing at or near t he ser v er . A configur at ion ex am ple follow s:

ip flow-export destination 215.17.13.1 9996 ip flow-export source Loopback0 ip flow-export version 5 origin-as ! interface Fddi0/0/0 description FDDI link to IXP ip address 215.18.1.10 255.255.255.0 ip route-cache flow ip route-cache distributed no keepalive ! I nt er face FDDI 0/ 0/ 0 has been configur ed t o capt ur e flow r ecor ds. The r out er has been configur ed t o ex por t Ver sion 5–st y le flow r ecor ds t o t he host at I P addr ess 215.17.13.1 on UDP por t 9996, w it h t he sour ce addr ess being t he r out er ’s loopback int er face.

53

N TP Source I nt erface NTP is t he m eans of keeping t he clocks on all t he rout ers on t he net w ork sy nchr onized t o w it hin a few m illiseconds. I f t he loopback int er face is used as t he sour ce int er face bet w een NTP speak er s, it m ak es filt er ing and aut hent icat ion som e w hat easier t o m aint ain. Most I SPs w ant t o per m it t heir cust om er s t o sy nchr onize only w it h t heir t im e ser v er s, not ev er y one else in t he w or ld. Look at t he follow ing configur at ion ex am ple:

clock timezone SST 8 ! access-list 5 permit 192.36.143.150 access-list 5 permit 169.223.50.14 ! ntp authentication-key 1234 md5 104D000A0618 7 ntp authenticate ntp trusted-key 1234 ntp source Loopback0 ntp access-group peer 5 ntp update-calendar ntp peer 192.36.143.150 ntp peer 169.223.50.14 !

Syslog Sour ce I nterface Sy slog ser v er s also r equir e car eful pr ot ect ion on I SP back bones. Most I SPs pr efer t o see only t heir ow n sy st em s’ sy slog m essages, not any t hing fr om t he out side w or ld. Den ial- of- ser v ice at t ack s on sy slog dev ices ar e not unk now n, eit her . Pr ot ect ing t h e syslog ser ver is again m ade easier if t he know n sour ce of syslog m essages com es fr om a w ell- defined set of addr ess space—for ex am ple, t hat used by t he loopback int er faces on t he r out er s. See t he follow ing configur at ion ex am ple:

logging logging logging logging logging !

buffered 16384 trap debugging source-interface Loopback0 facility local7 169.223.32.1

Te lne t t o t he Rout e r This m ight seem t o be an odd ex am ple in a docum ent dedicat ed t o I OS Soft w ar e essent ials. How ev er , r em em ber t hat a loopback int er face on a r out er nev er changes it s st at e and r ar ely has any need t o change it s I P addr ess. Phy sical int er faces can be phy sically sw apped out or r enum ber ed, and addr ess r anges can change, but t he loopback int erface w ill alw ays be t here. So, if t he DNS is set up so t hat t he r out er nam e m aps t o t he loopback int er face addr ess, t her e is one less change t o w or r y about dur ing oper at ional and configur at ion changes elsew her e in t he I SP back bone.

54

I SP back bones ar e cont inuously dev eloping ent it ies. Her e’s an ex ample fr om t he DNS for w ar d and r ev er se zone files:

; net.galaxy zone file net.galaxy. IN

SOA

ns.net.galaxy. hostmaster.net.galaxy. ( 1998072901 ; version ==

date(YYYYMMDD)+serial

; localhost gateway1 gateway2 gateway3 ; ;etc etc

IN IN IN IN

NS NS MX MX

10800 ; Refresh (3 hours) 900 ; Retry (15 minutes) 172800 ; Expire (48 hours) 43200 ) ; Minimum (12 hours) ns0.net.galaxy. ns1.net.galaxy. 10 mail0.net.galaxy. 20 mail1.net.galaxy.

IN IN IN IN

A A A A

127.0.0.1 215.17.1.1 215.17.1.2 215.17.1.3

; 1.17.215.in-addr.arpa zone file ; 1.17.215.in-addr.arpa. IN SOA ns.net.galaxy. hostmaster.net.galaxy. ( 1998072901 ; version == date(YYYYMMDD)+serial 10800 ; Refresh (3 hours) 900 ; Retry (15 minutes) 172800 ; Expire (48 hours) 43200 ) ; Minimum (12 hours) IN NS ns0.net.galaxy. IN NS ns1.net.galaxy. IN PTR gateway1.net.galaxy. IN PTR gateway2.net.galaxy. IN PTR gateway3.net.galaxy. ; ;etc etc On t he r out er , set t he Telnet sour ce t o t he loopback int er face:

ip telnet source-interface Loopback0

RCM D t o t he Rout e r RCMD r equir es t he oper at or t o hav e t he UNI X r login/ r sh client s t o enable access t o t he r out er . Som e I SPs use RCMD for gr abbing int er face st at ist ics, uploading or dow nloading r out er configur at ions, or t ak ing a snapshot of t he r out ing t able. The r out er can be configur ed so t hat RCMD connect ions use t he loopback int er face as t he source address of all packet s leaving t he rout er:

55

ip rcmd source-interface Loopback0

I n t e r fa ce Con figu r a t ion Configur ing int er faces inv olv es m or e t han sim ply plugging in t he cable and act iv at ing t he int er face w it h t he I OS Soft w ar e com m and n o sh u t d o w n. At t ent ion should be applied t o det ails such as w het her it is a WAN or a LAN, w het her a r out ing pr ot ocol is r unning acr oss t he int er face, addr essing and m asks t o be used, and oper at or infor m at ion.

descript ion Use t he de scr ipt ion int er face com m and t o docum ent det ails such as t he cir cuit bandw idt h, t he cust om er nam e, t he dat abase ent r y m nem onic, t he cir cuit num ber t hat t he cir cuit supplier gav e y ou, and t he cable num ber . This sounds lik e ov er k ill, especially if t her e is a cust om er dat abase w it hin t he I SP or ganizat ion. Ho w ev er , it is ver y easy t o pick up all t he r elevant det ails fr om t he r out er sh ow in t e r f a ce com m and if and w hen an engineer needs t o be onsit e, w hen an engineer is aw ay fr om t he dat abase sy st em , or w hen t he dat abase is unav ailable. Ther e can nev er be t oo lit t le docum ent at ion, and docum ent at ion such as t his ensur es t hat r econst r uct ing configur at ions and diagnosing pr oblem s ar e m ade consider ably easier .

bandw idt h Don’t for get t he b a n d w id t h int er face com m and. I t is used by int er ior r out ing pr ot ocols t o decide opt im um r out ing, and it is especially im por t ant t o set t his com m and pr oper ly in t he case of back bone link s using only a por t ion of t he av ailable bandw idt h suppor t by t he int er face. For ex am ple, a ser ial int er face ( Ser ial0/ 0) on a r out er suppor t s speeds up t o 4 Mbps but has a default bandw idt h set t ing of 1.5 Mbps. I f t he backbone has different size links from 64 Kbps t o 4 Mbps and t he b a n d w id t h com m and is not used, t he int er ior r out ing pr ot ocol w ill assum e t hat all t he links have t he sam e cost and w ill calcula t e opt im um pat hs accordingly — and t his could be less t han ideal. On cust om er link s, it m ight seem t hat t his set t ing is super fluous because an int er ior r out ing pr ot ocol is never r un over a link t o a cust om er . How ever , it pr ovides ver y useful online docum ent at ion for w hat t he cir cuit bandw idt h is. Fur t her m or e, t he bandw idt h on t he cir cuit is used t o calculat e t he int er face load var iable —som e I SPs m onit or t heir cust om er int er faces loading by SNMP polls so t hat t hey can get adv ance w ar ning of pr oblem s or congest ion, or t o pr oact iv ely infor m cust om er s of necessar y upgr ades. ( Som e I SPs look at t he load v ar iable; ot her I SPs look at t he fiv e- m inut e aver age, inbound and out bound. I f you m onit or t he load var iable, you need t o set t he bandw idt h so t hat it m at ches t he t r ue cir cuit bandw idt h, not t he default configur ed on t he r out er .)

56

ip unnum be r e d Tradit ionally I SPs have used I P addresses for t he point - t o- point links on leased- line cir cuit s t o cust om er s. I ndeed, sev er al y ear s ago, befor e t he adv ent of CI DR, it w as not uncom m on t o see a / 26 or even a / 24 used for sim ple point - t o- point link addr esses. Wit h t he advent of CI DR, / 30 net w or ks have been used inst ead ( / 30 is a block of four addr esses, t w o of w hich can be used for phy sical int er faces) . How ev er , t his led t o pr oblems because I GPs of som e of t he lar ger I SPs w er e st ar t ing t o car r y sever al t housand net w or ks, affect ing conver gence t im e and r esult ing in an adm inist r at iv e and docum ent at ion night m ar e. To avoid pr oblem s w it h lar ge num ber s of / 30s float ing ar ound t he I SP’s int er nal r out ing pr ot ocol, and t o av oid t he pr oblem s of k eeping int er nal docum ent at ion consist ent w it h net w or k deploy m ent ( especially t r ue in lar ger I SPs) , m any ar e now using unnum ber ed point - t o- point links. An unnum ber ed point - t o- point link is one requiring no I P addr esses. The configur at ion is such t hat t r affic dest ined for one net w or k fr om anot her sim ply is point ed at t he ser ial int er face concer ned. ip u n n u m b e r e d is an essent ial feat ur e applicable t o point - t o- point int er faces such as Ser ial, HSSI , POS, and so on. I t enables t he use of a fix ed link ( usually fr om I SP t o cust om er ) w it hout consum ing t he usual / 30 of addr ess space, t her eby k eeping t he num ber of net w or k s r out ed by t he I GP low . The ip u n n u m b e r e d dir ect iv e specifies t hat t he point - t o- point link should use an addr ess of anot her int er face on t he r out er , t ypically a LAN or m or e usually a loopback int er face. Any net w or k s t hat m ust be r out ed t o t he cust om er ar e point ed at t he ser ial int er face r at her t han t he r em ot e addr ess of t he point - t o- point link, as w ould be done in nor m al inst ances. Ca v e a t s I SPs need t o consider som e sit uat ions befor e im plem ent ing an I P unnum ber ed syst em for t heir cust om er point - t o- point connect ions. These ar e consider at ions only —bear in m ind t hat m any I SPs have used I P unnum bered for sever al year s, m ainly so t hat t hey can cont r ol t he size of t he I GP r unning in t heir back bone net work. •

•

•

Pin g in g t h e cu st om e r — Many I SPs use m onit or ing sy st em s t hat use ping t o check t he st at us of t he leased line ( cust om er connect iv it y ) . Ev en if t he cu st omer unplugs t he LAN, an alar m w ill not be r aised on t he I SPs m anagem ent sy st em . This is because t he cust om er r out er st ill k now s t hat t he LAN I P addr ess is configur ed on t he sy st em and is “ useable.” As long as t he I P addr ess is configur ed on t he LAN, t her e w ill be no reachabilit y issues wit h using ip u n n u m b e r e d . Rou t in g pr ot ocols — I f a r out ing pr ot ocol needs t o be r un over t his link, it is oper at ionally m uch easier t o use I P addr esses. Don’t use ip u n n u m b e r e d if t he cust om er is peer ing w it h y ou using BGP across t he link or if t he link is an int er nal backbone link. Sim ply use a net w or k w it h a / 30 addr ess m ask. ( Rout ing w ill w or k over unnum ber ed links, but t he ext r a m anagem ent and oper at ional com plex it y pr obably out w eighs t he sm all addr ess space advant age gaine d.) Loop b a ck in t e r f a ce s on t h e cu st om e r ’s r ou t e r — These offer no adv ant age t o addr essing t he ping pr oblem , and t hey unnecessar ily consum e

57

addr ess space ( not t o m ent ion adding com plex it y t o t he cust om er r out er configur at ion) . ip u n n u m b e r e d Con f ig u r a t ion Ex a m ple Using t he pr eceding configur at ion com m ands, a t y pical configur at ion on t he I SP’s r out er w ould be as follow s:

interface loopback 0 description Loopback interface on Gateway Router 2 ip address 215.17.3.1 255.255.255.255 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial 5/0 description 128K HDLC link to Galaxy Publications Ltd [galpub1] WT50314E R5-0 bandwidth 128 ip unnumbered loopback 0 no ip redirects no ip directed-broadcast no ip proxy-arp ! ip route 215.34.10.0 255.255.252.0 Serial 5/0 The cust om er r out er configur at ion w ould look som et hing lik e t his:

interface Ethernet 0 description Galaxy Publications LAN ip address 215.34.10.1 255.255.252.0 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial 0 description 128K HDLC link to Galaxy Internet Inc WT50314E bandwidth 128 ip unnumbered ethernet 0 no ip redirects no ip directed-broadcast no ip proxy-arp ! ip route 0.0.0.0 0.0.0.0 Serial 0

C0

I n t his ex am ple, t he r egional or local r egist r y has allocat ed t he cust om er t he net w or k block 215.34.10.0/ 22. This is r out ed t o t he cust om er sit e w it h t he st at ic r out e point ing t o Ser ial 5/ 0. The cust om er r out er sim ply needs a default r out e point ing t o it s ser ial int er face t o ensur e a connect ion. Wit h t his configur at ion, t her e ar e no / 30s fr om point - t o- point links present in t he I GP, and t he I SP does not need t o docum ent t he link addr ess or keep a

58

t able/ dat abase up- t o- dat e. I t all m akes for easier configur at ion as w ell as easier oper at ion of t he I SP’s business. Not e t he cont ent s of t he descr ipt ion field. This ex am ple has included t he follow ing: bandw idt h of t he cir cuit

128K

encapsulat ion

H D LC

nam e of t he com pany

Ga la x y Pu b lica t ion s Lt d

dat abase m nem onic in t he I SP’s int er nal dat abase

[ g a lp u b 1 ]

t elco’s cir cuit I D

W T5 0 3 1 4 E

cable num ber

R5 - 0

All of t hese ar e online docum ent at ion, seem ingly super fluous, but v er y necessar y t o ensur e sm oot h and efficient oper at ions. All t he infor m at ion per t inent t o t h e cust om er ’s connect ion fr om t he cabling t o t he I P v alues is cont ained in t he int er face configur at ion. I f t he I SP’s dat abase is dow n or unavailable, any debug infor m at ion r equir ed by oper at or s or engineer s can be found on t he r out er it self.

I n t e r fa ce St a t u s Ch e ck in g Som e useful hidden I OS Soft w ar e com m ands enable t he oper at or t o check t he st at us of t he int er faces in I OS Soft w ar e. Thr ee useful com m ands ar e sh ow in t e r f a ce sw it ch in g, sh ow in t e r f a ce st a t s, and sh ow id b .

show int e r fa ce sw it ching The I OS Soft w ar e com m and sh ow in t e r f a ce sw it ch in g pr ovides useful infor m at ion about t he sw it ching st at us of t he r out er ’s int er faces, eit her on an indiv idual int er face basis or over t he w hole r out er . The full com m and for m at is sh ow in t e r f a ce [ int n/ n] sw it ch in g, w here an opt ional ar gum ent is t he specific int er face in quest ion. Com m and com plet ion cannot be used for sw it ch in g—it needs t o be t y ped in up t o and including t he second i. Sam ple out put m ight look like t he follow ing:

gw>show interface FastEthernet 1/0 switching FastEthernet1/0 Production LAN Throttle count 0 Drops RP 0 SP SPD Flushes Fast 0 SSE SPD Aggress Fast 0 SPD Priority Inputs 2421 Drops Protocol Path Other Process Cache misses Fast Auton/SSE IP Process Cache misses Fast Auton/SSE

Pkts In Chars In 0 0 0 0 0 0 0 5339594 516613071 5391487 256289350 1125491757 0 0

0 0 0 Pkts Out 74633

Chars Out 4477980

31653 0 5622371

2957994 0 851165330

257803747 2058541849 0 0

59

ARP Process Cache misses Fast Auton/SSE CDP Process Cache misses Fast Auton/SSE

16919 0 0 0 12449 0 0 0

1015300

34270

2056200

0 0 4083272

0 0 12440

0 0 4142520

0 0

0 0

0 0

gw> This sam ple out put show s SPD [ 1 ] act iv it y , as w ell as ot her act iv it y on t hat par t icular int er face on t he r out er . Not e t he r efer ences t o aut onom ous/ SSE sw it ching—t his is applicable only t o t he Cisco 7000 ser ies w it h Silicon Sw it ch Engine only ( a pr oduct t hat is now discont inued but w as a significant par t of t he I nt er net cor e in t he m id1990s) . Fast sw it ching r efer s t o all pack et s t hat hav e not been pr ocess- sw it ched, w hich w ould include Opt im um sw it ching, Net Flow , and CEF.

show int e r fa ce st a t s The I OS Soft w ar e com m and sh ow in t e r f a ce st a t s is t he second useful com m and t o show int er face st at us. I t show s t he num ber of pack et s and char act er s inbound and out bound on an individual r out er int er face or all of t hem . The full com m and for m at is sh ow in t e r fa ce [ int n/ n] st a t s, w her e an opt ional ar gum ent is t he specific int er face in quest ion. Com m and com plet ion cannot be used for st a t s—at least st needs t o be t y ped in at t he com m and pr om pt . Sam ple out put m ight look lik e t his:

gw>show interface stats Interface FastEthernet0/0 is disabled FastEthernet1/0 Switching path Processor Route cache Total gw>

Pkts In Chars In 5371378 521946816 256413200 1149405512 261784578 1671352328

Pkts Out Chars Out 5746126 862068168 257960291 2072462774 263706417 2934530942

As for int er face sw it ching, t he out put differ ent iat es bet w een pack et s t hat go v ia t he pr ocessor and t hose t hat hav e been pr ocessed v ia t he r out e cache. This is useful t o det er m ine t he lev el of pr ocess sw it ching t ak ing place on t he r out er . On a r out er t hat suppor t s dist r ibut ed sw it ching ( for exam ple, 7500 w it h VI P int er faces) , t he out put w ill look like t he follow ing:

gw>show interface stats FastEthernet0/1/0 Switching path Processor Route cache Distributed cache Total

Pkts In 207745 0 93 207838

Chars In 14075132 0 9729 14084861

Pkts Out 270885 0 0 270885

Chars Out 21915788 0 0 21915788

60

Not ice t hat packet s t hat have been pr o cessed v ia t he dist r ibut ed cache ar e count ed separ at ely fr om t hose handled v ia t he cent r al r out e cache and t he pr ocessor .

sh ow I D B Each int er face on t he r out er has an associat ed int er face descr ipt or block allocat ed t o it . I n t he ear ly day s, each phy sical int er face m apped t o one I DB, and r out er s gener ally could suppor t up t o 300 I DBs ( for ex am ple, t he Cisco AGS+ ) . How ev er , w it h t he incr easing num ber s of new connect ion ser v ices, and w it h ATM and Fr am e Relay pr ov iding lar ge num ber s of subint er faces, r out er s hav e had t o scale t o suppor t ing sev er al t housand I DBs. sh ow I D B r ecent ly has becom e a v isible com m and in I OS Soft w ar e ( CSCds89322) ; it allow s I SPs t o find out how m any I DBs are configured on t he rout er:

gw#show idb SW IDBs allocated (2368 bytes each) HW IDBs allocated (4040 bytes each) HWIDB#1 1 FastEthernet0/0 (HW IFINDEX, Ether) HWIDB#2 2 FastEthernet1/0 (HW IFINDEX, Ether) HWIDB#3 3 Serial2/0 (HW IFINDEX, Serial) HWIDB#4 4 Serial2/1 (HW IFINDEX, Serial) HWIDB#5 5 Serial2/2 (HW IFINDEX, Serial) HWIDB#6 6 Serial2/3 (HW IFINDEX, Serial) HWIDB#7 7 FastEthernet3/0 (HW IFINDEX, Ether) HWIDB#8 8 FastEthernet5/0 (HW IFINDEX, Ether) HWIDB#9 20 Dialer0 (HW IFINDEX, Serial) HWIDB#10 21 Loopback0 (HW IFINDEX) gw# To find out how m any I DBs ar e suppor t ed on differ ent r out er plat for m s, consult Cisco. com docum ent at ion—for ex am ple, ht t p: / / w w w . cisc o.com / w ar p/ public/ 63/ idb_lim it .ht m l. Alt hough m ost sm aller r out er plat for m s st ill suppor t only 300 I DBs at m ax im um , som e of t he lar ger plat for m s can go as high as 10,000 ( 7200/ 12.2T r elease) . These v alues m ight change as fut ur e enhancem ent s are m ade t o Cisco I OS Soft w ar e.

Cisco Ex p r e ss For w a r d in g CEF is now t he r ecom m ended for w ar ding/ sw it ching pat h for Cisco r out er s in an I SP env ir onm ent . CEF adds incr eased per for m ance, scalabilit y , and r esilience, and enables new funct ionalit y ov er t he older opt im um sw it ching. Det ails on t he operat ion and funct ionalit y of CEF ar e now cov er ed in det ail by t he I OS Soft w ar e docum ent at ion and in sev er al w hit epaper s descr ibing CEF ( see r efer ences in t he “ Technical Refer ences and Recom m ended Reading” sect ion at t he end of t his book ) . I m plem ent at ion is sim ple w it h eit her of t he follow ing com m ands ( depending on t he plat for m ) :

61

ip cef ip cef-distributed The key issue for I SPs is ensuring t hat CEF is t urned on. On m ost plat form s, CEF is not t ur ned on by default , so I SP engineer s need t o ensur e t hat CEF is t ur ned on. Table 2- 1 pr ov ides a list of t he default CEF configur at ions for var ious Cisco plat for m s. I SPs should check t heir configur at ion scr ipt s t o ensur e t hat CEF/ dCEF is t ur ned on, especially for t he 7200- based edge plat for m s such as uBR, 6400, and 5800 NAS. Table 2-1. Default Configuration for CEF on Various Platforms On This Platform. . . The Default Is. . . 2600/ 3600

CEF is not enabled.

4500/ 4700

CEF is not enabled.

7000 series wit h RSP 7000

CEF is not enabled.

7200

CEF is not enabled.

7500

CEF is enabled, but dist r ibut ed CEF is not .

7600 OSR

CEF is enab led.

12000 GSR

Dist r ibut ed CEF is enabled.

CEF will be discussed in m ore dept h in Chapt er 4, “ Secur it y . ” One of t he best secur it y t ools available for an I SP is Unicast Reverse Pat h For w ar ding. This r equir es CEF t o be act iv at ed on t he r out er because t he r ev er se pat h check is dependent on t he FI B t able, w hich is par t of t he CEF pr ocess.

N e t Flow Enabling Net Flow on r out er s pr ov ides net w or k adm inist r at or s w it h access t o pack et f low infor m at ion fr om t heir net w or k . Ex por t ed Net Flow dat a can be used for a v ar iet y of pur poses, including secur it y m onit or ing, net w or k m anagem ent , capacit y planning ( as in Figur e 2- 1) , cust om er billing, and I nt er net t r affic flow analy sis. Fig u r e 2- 1 . N e t f low in I t s Ca pa cit y - Pla n n in g Role

62

Net Flow is av ailable on all r out er plat for m s fr om t he 2600 ser ies upw ar d fr om t he 12.0 soft w ar e r elease onw ar d. I t w as fir st int r oduced in 11.1CC on t he 7200 and 7500 plat for m s. I t can be enabled on a per- int er face basis on t he r out er s, as in t he follow ing ex am ple:

interface serial 5/0 ip route-cache flow ! I f CEF is not configur ed on t he r out er , t his t ur ns off t he exist ing sw it ching pat h on t he r out er and enables Net Flow sw it ching ( basically m odified opt im um sw it ching) . I f CEF is configur ed on t he r out er , Net Flow sim ply becom es a “ flow infor m at ion gat her er ” and feat ur e acceler at or— CEF r em ains oper at ional as t he under ly ing sw it ching pr ocess.

N et Flow Feat ure Accelerat ion Net Flow feat ur e acceler at ion w or k s for a lim it ed set of feat ur es t hat can t ak e adv ant age of flow pr ocess shor t cut s. NFFA r eser v es space in t he flow cache for st at e infor m at ion belonging t o feat ur es conv er t ed t o use t he flow acceler at ion. The feat ur es can t hen at t ach per- flow st at e t o t he cache ent ry, using Net Flow as a quick w ay t o access infor m at ion t hat is flow - based. For exam ple, Net Flow policy rout ing ( NPR) uses flow acceler at ion t o elim inat e r out e- m ap check s on a per- packet basis. Net Flow feat ur e acceler at ion is t ur ned on w it h t he follow ing com m and:

ip flow-cache feature-accelerate

63

As of 12.0( 11) S, t he follow ing feat ur es hav e been conv er t ed t o w or k w it h Net Flow feat ur e acceler at ion: • • • • • • •

Num ber ed access list s Nam ed access list s I P account ing Cr y pt o decr y pt Cr ypt o encr ypt Policy rout ing WCCP r edir ect ion

N e t Flow St a t ist ics—Basics To v iew Net Flow infor m at ion on t he r out er , sim ply ent er t he com m and sh ow ip ca ch e f lo w. This display s t he cur r ent flow cache on t he t er m inal scr een ( see Ex am ple 2- 1) . Ex a m p le 2- 1 Sa m p le Ou t p u t f r om D isp la y in g Flow I n f or m a t ion on a N e t Flo w- En a b le d Rou t e r gw>sh ip cache flow IP packet size distribution (410772243 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .168 .384 .102 .160 .107 .019 .005 .003 .001 .001 .000 .000 .000 .003 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .001 .000 .035 .000 .003 .000 .000 .000 .000 .000 .000 IP Flow Switching Cache, 4456704 bytes 15074 active, 50462 inactive, 125120769 added 369493980 ager polls, 0 flow alloc failures last clearing of statistics 4d05h Protocol Total Flows Packets Bytes Idle(Sec) -------Flows /Sec /Flow /Pkt /Flow TCP-Telnet 605 0.0 44 52 9.1 TCP-FTP 3494 0.0 22 64 12.9 TCP-FTPD 4104 0.0 757 376 5.7 TCP-WWW 845158 2.3 16 281 6.8 TCP-SMTP 87119 0.2 10 201 13.1 TCP-X 59 0.0 2 68 12.0 TCP-BGP 62074 0.1 5 255 18.5 TCP-NNTP 5 0.0 3 48 19.6

Packets Active(Sec) /Sec

/Flow

0.0

8.1

0.2

9.4

8.4

34.9

39.1

4.5

2.5

4.2

0.0

0.4

0.9

9.6

0.0

8.8

64

TCP-Frag 21.2 TCP-other 7.5 UDP-DNS 19.1 UDP-NTP 19.0 UDP-TFTP 19.2 UDP-Frag 18.2 UDP-other 19.1 ICMP 19.0 IGMP 19.8 IP-other 18.7 Total: 17.9 SrcIf DstP Pkts Se2/0 0035 1 Fa1/0 2245 1 Fa1/0 0C1C 1 Se2/0 0035 1 Se2/0 0035 1 Fa1/0 0408 1 Fa1/0 0035 2 Se2/0 0035 248 Se2/0 0035 4 Fa1/0 0035 4 Fa1/0 0A6B 248 Se2/0 0035 2 Se2/0 0035 2 Fa1/0 B034 2

2

0.0

2

40

0.0

0.1

11879955

32.3

5

141

174.2

2.5

70078211

191.0

3

90

586.3

4.8

31804

0.0

1

72

0.0

0.0

327

0.0

3

153

0.0

4.8

9

0.0

4

311

0.0

22.5

41601240

113.4

2

157

301.3

4.1

498404

1.3

4

170

5.7

10.7

2

0.0

113

551

0.0

6.8

20236

0.0

4

299

0.2

12.7

125112808

341.1

3

126

1119.2

4.4

SrcIPaddress

DstIf

DstIPaddress

Pr SrcP

207.69.200.110

Fa1/0

203.37.255.121

11 2245

203.37.255.121

Se2/0

207.69.200.110

11 0035

203.37.255.97

Se2/0

169.229.128.130 11 0035

169.229.128.130 Fa1/0

203.37.255.97

11 0C1C

195.28.226.121

Fa1/0

203.37.255.97

11 0408

203.37.255.97

Se2/0

195.28.226.121

11 0035

203.37.255.97

Se2/0

163.21.134.2

11 0035

202.103.229.40

Fa1/0

203.37.255.97

11 0A6B

163.21.134.7

Fa1/0

203.37.255.97

11 0035

203.37.255.97

Se2/0

163.21.134.7

11 0035

203.37.255.97

Se2/0

202.103.229.40

11 0035

163.21.134.2

Fa1/0

203.37.255.97

11 0035

63.87.170.77

Fa1/0

203.37.255.97

11 B034

203.37.255.97

Se2/0

63.87.170.77

11 0035

The fir st par t of t he out put display s t he pack et size dist r ibut ion of t he t r affic flow ing int o t he int er faces t hat Net Flow is configur ed on. The nex t por t ion of t he out put

65

displays t he flow s, packet size, act ivit y, and so on for t he flow s per w ell- know n pr ot ocol. The final sect ion display s t he sour ce and dest inat ion int er faces/ addr esses/ por t s for t he cur r ent ly act iv e t r affic flow s. I t is also possible t o expor t t his collect ed dat a t o a sy st em t hat w ill collect t he dat a, allow ing t he I SP t o car r y out fur t her analysis. Public - dom ain soft w are is available ( cflow d fr om Caida and Net Flow Met fr om t he Univ er sit y of Auck land, for ex am ple) , as w ell as fully feat ur ed and suppor t ed com m er cial pr oduct s, such as Cisco’s Net Flow Collect or and Analy zer pack ages.

N et Flow Da t a Ex port The gr eat est benefit s of Net Flow ar e found w hen it s dat a is ex por t ed t o collect ion sy st em s and t hen ar e analy zed and pr ocessed. Cisco has adopt ed a br oad approach t o facilit at e t his act iv it y . These include donat ions for fr eew ar e collect ion/ analy sis soft w ar e, Cisco’s ow n com m er cial soft w ar e, t ools for ot her s t o cr eat e t heir ow n soft w ar e, and par t ner ships w it h com panies t hat m ak e com m er cial- gr ade billing sy st em s based on Net Flow ex por t . To ex por t t he dat a, t he follow ing configur at ion com m ands ar e r equir ed:

ip flow-export version 5 [origin-as|peer-as] ip flow-export destination x.x.x.x udp-port ip flow-export source Loopback0 The fir st com m and line set s t he export version t o 5 ( basically t his includes BGP infor m at ion such as AS num ber ) and has opt ions t o include or igin - a s or p e e r- a s in t he ex por t ed r ecor ds. Most I SPs use t he origin- a s opt ion because t hat w ill r ecor d t he or igin AS of t he pr efix or iginat ing t he flow . This has becom e a fr equent ly ask ed quest ion on t he CAI DA cflow d list , w it h I SPs for get t ing t he or ig in- a s opt ion and t hen not under st anding w hy so m any of t heir expor t ed r ecor ds have an or igin of AS 0. The second com m and line configur es t he I P addr ess of t he dest inat ion sy st em , t he Net Flow collect or sy st em , and t he UDP por t t hat t he collect or is list ening on. Most I SPs use high UDP por t s, such as 9999 or in t he 60,000s. Not e t hat because t he flow records use UDP, it is im port ant t o design t he infra st r uct ur e so t hat t he flow collect or is not t oo far aw ay fr om t he or iginat ing r out er . Som e I SPs t hat use Net Flow for billing pur poses build a separ at e m anagem ent net w or k sim ply t o suppor t t his funct ion. The t hir d com m and line or iginat es all t he flow t r affic using t he I P addr ess of t he loopback int er face. This m ak es t he cflow d configur at ion file easier t o const r uct for sev er al r out er s because m ost I SPs num ber t heir r out er loopback s out of one cont iguous block. To det er m ine t he st at us of t he flow ex por t , it is possible t o check on t he r out er t o see w hat has been sent . Obv iously t he collect or sy st em should be check ed as w ell— cflow d, for ex am ple, has ex t ensiv e inst r uct ions on how t o debug any flow ex por t pr oblem s. An ex am ple of t he usage of t he I OS Soft w ar e com mand follow s:

66

gw>sh ip flow export Flow export is enabled Exporting flows to 220.19.51.35 (9998) Exporting using source interface Loopback0 Version 5 flow records, origin-as 264038749 flows exported in 8801292 udp datagrams 0 flows failed due to lack of export packet 6079835 export packets were sent up to process level 0 export packets were punted to the RP 0 export packets were dropped due to no fib 0 export packets were dropped due to adjacency issues 0 export packets were dropped due to fragmentation failures 0 export packets were dropped due to encapsulation fixup failures A new feat ur e as of Cisco I OS Soft w ar e r elease 12.0( 5) S is Net Flow aggr egat ion, in w hich sum m ar izat ion/ aggr egat ion of t he flow r ecor ds is car r ied out on t he rout er befor e t he dat a is ex por t ed t o t he collect ing sy st em . The aim is t o r educe t he am ount of dat a going acr oss t he net w or k fr om r out er t o flow collect or , t her eby im pr ov ing t he r eliabilit y of t he collect ing sy st em . Flow aggr egat ion is enabled by t h e follow ing com m ands:

ip flow-aggregation cache as|destination-prefix|prefix|protocolport|source-prefix enabled export destination x.x.x.x UDP-port Subcom m ands r equir ed include e n a b le d , w hich sw it ches on t he flow aggr egat ion, and e x p or t d e st in a t ion, w hich list s t he host t hat w ill gat her t he aggr egat ed r ecor ds. The collect or host needs t o suppor t Net Flow Ty pe 8 r ecor ds t o be capable of r eading t he aggr egat ed infor m at ion.

Tu r n On N a gle The Nagle congest ion- cont r ol algor it hm is som et hing t hat m any ISPs t ur n on t o im pr ov e t he per for m ance of t heir Telnet sessions t o and fr om t he r out er . When using a st andar d TCP im plem ent at ion t o send k ey st r ok es bet w een m achines, TCP t ends t o send one pack et for each k ey st r ok e t y ped. On lar ger net w or k s, m any sm all packet s use up bandw idt h and cont r ibut e t o congest ion. John Nagle’s algor it hm ( RFC 896) helps alleviat e t he sm all- packet problem in TCP. I n gener al, it w or k s t his w ay : The fir st char act er t y ped aft er connect ion est ablishm ent is sent in a single pack et , but TCP holds any addit ional char act er s t yped unt il t he r eceiv er ack now ledges t he pr ev ious pack et . The second, lar ger pack et is sent , and addit ional t y ped char act er s ar e sav ed unt il t he ack now ledgm ent com es back . The effect is t o accum ulat e char act er s int o lar ger chunk s and pace t hem out t o t he net w or k at a r at e m at ching t he r ound- t r ip t im e of t he given connect ion. This m et hod is usually good for all TCP- based t r affic and helps w hen connect iv it y t o t he r out er is poor or congest ed or t he r out er it self is busier t han nor m al. How ev er , do not use t he se r vice n a gle com m and if y ou hav e XRem ot e user s on X Window sessions or sour cing v oice ov er I P t r affic or ot her r eal- t im e t r affic fr om t he r out er—per for m an ce will becom e very poor.

67

The I OS Soft w ar e com m and t o enable Nagle follow s:

service nagle N OTE Wit hout se r vice n a gle on a Cisco r out er , each char act er in a Telnet session is a separ at e CPU int er r upt . Hence, a com m and such as sh ow t e ch w ill for ce a lar ge num ber of CPU int er r upt s, im pact ing t he per for m ance of t he r out er . From a Cisco point of v iew , t he Nagle ser v ice not only helps t o opt im ize t he Telnet session but also lessens t he load on t he rout er.

D N S a nd Rout e r s The DNS m ight be an unusual t opic t o put int o a book cover ing I SP net w or k essent ials and Cisco I OS Soft w ar e best pr act ices. How ever , it is one of t he m ost over looked syst em s t opics in t he I SP indust r y —y et it is pr obably t he m ost im por t ant par t of t he public visibilit y of t he net w or k t o get r ight . I f t he DNS does not w or k, t he public t hink s t hat t he net w or k is broken—m any new spaper headlines in t he last few y ear s hav e display ed such apocr y phal headlines sim ply because of oper at ional er r or s or pr oblem s w it h t he DNS. An I SP net w or k engineer m ust pay at t ent ion t o t w o aspect s of t he DNS. The fir st is t he business of put t ing all t he nam e - t o- address- t o- nam e m appings in t he syst em so t hat r out er s can be r ecognized by t heir English- language nam es rat her t han by four bor ing decim al num ber s separ at ed by dot s. Hum ans ar en’t good at r em em ber ing t he lat t er . The second aspect is t o act ually enable suppor t for t he DNS in t he r out er s t hem selv es. This sect ion cov er s only t he r out er aspect — Chapt er 5, “ Operat ional Pr act ices,” descr ibes configur at ion and placem ent of t he DNS sy st em s t hr oughout t he I SP back bone.

M a pping I P Addresses t o N a m es Mapping dom ain nam es t o I P addr esses and v ice v er sa is one of t hose com m only ov er look ed ar eas in a new I SP’s oper at ions. Doing a t r ace fr om Aust r alia acr oss t he back bones in t he Unit ed St at es t o a sit e in t he Unit ed Kingdom gives you som et hing like Ex am ple 2- 3. Ex a m p le 2- 2 Ex a m p le Tr a ce r ou t e Acr oss t h e I n t e r n e t f r om Au st r a lia t o t h e Un it e d Kin gdom traceroute to k.root-servers.net (193.0.14.129), 30 hops max, 38 byte packets 1 fe5-0.gw.apnic.net (202.12.29.190) 0.707 ms 0.534 ms 0.497 ms 2 Serial1-0-3.cha8.Brisbane.telstra.net (139.130.64.97) 5.999 ms 5.131 ms 6.155 ms 3 GigabitEthernet5-1.cha-core4.Brisbane.telstra.net (203.50.51.1) 6.148 ms 4.972 ms 4.537 ms

68

4 ms

Pos2-0.chw-core2.Sydney.telstra.net (203.50.6.225) 19.355 ms 18.595

19.797 ms 5 Pos4-0.exi-core1.Melbourne.telstra.net (203.50.6.18) 32.120 ms 32.968 ms 32.544 ms 6 Pos5-0.way-core4.Adelaide.telstra.net (203.50.6.162) 50.088 ms 46.171 ms 44.896 ms 7 Pos6-0.wel-core3.Perth.telstra.net (203.50.6.194) 88.296 ms 75.545 ms 83.527 ms 8 GigabitEthernet4-0.wel-gw1.Perth.telstra.net (203.50.113.18) 78.172 ms 76.116 ms 75.851 ms 9 Pos1-0.paix1.PaloAlto.net.reach.com (203.50.126.30) 305.915 ms 309.617 ms 314.994 ms 10 fe0.pao0.verio.net (198.32.176.47) 308.744 ms 304.431 ms 304.230 ms 11 p4-6-0-0.r02.mclnva02.us.bb.verio.net (129.250.2.246) 380.061 ms 380.639 ms 380.292 ms 12 p16-0-0-0.r01.mclnva02.us.bb.verio.net (129.250.5.253) 384.100 ms 384.124 ms 384.382 ms 13 p4-7-2-0.r00.nycmny06.us.bb.verio.net (129.250.3.181) 390.487 ms 390.300 ms 396.328 ms 14 p4-0-2-0.r01.nycmny06.us.bb.verio.net (129.250.3.130) 390.196 ms 384.921 ms 385.245 ms 15 gxn.d3-1-0-1.r01.nycmny06.us.bb.verio.net (129.250.16.198) 321.844 ms 319.204 ms 319.252 ms 16 se6-1-0-llb-x-ny2.NY1.core.rtr.xara.net (194.143.164.45) 325.706 ms 320.925 ms 320.557 ms 17 se5-1-llb-ny1.HU1.core.rtr.xara.net (194.143.164.97) 325.264 ms 322.578 ms 321.049 ms 18 po2-0-llb-hu1.TH30.core.rtr.xara.net (194.143.164.189) 389.618 ms 390.177 ms 388.401 ms 19 gb11-0-0-llb-x-many.TH1.core.rtr.uk.xo.net (194.143.163.130) 398.421 ms 388.459 ms 390.471 ms 20 fa0-0.gxn-linx.transit1.linx.net (195.66.248.33) 388.834 ms 391.937 ms 389.687 ms 21 k.root-servers.net (193.0.14.129) 387.544 ms 391.093 ms 387.059 ms Not ice t hat each r out er I P addr ess has a cor r esponding DNS ent r y . These v er y descr ipt iv e DNS nam es help I nt er net user s and oper at or s under st and w hat is happening w it h t heir connect ions and w hich r out e t he out bound t r affic is t ak ing. The descr ipt iv e nam es ar e an inv aluable aid t o t r oubleshoot ing pr oblem s on t he net .

69

Table 2- 2 show s som e exam ples of descr ipt ive DNS for m at s used by var ious I SPs.

ISP

Table 2-2. DNS Formats Example Use of the DNS

C&W

bor der cor e4- hssi0- 0.SanFr ancisco.cw .net

BBN Planet

p2- 0. paloalt o- nbr 2.bbnplanet .net

Concer t

cor e1- h1- 0- 0.uk 1.concer t .net

Sprint

sl- bb6- dc- 1- 1- 0- T3.spr int link .net

DI GEX

sj c4- cor e5- pos4- 1.at las.digex.net

Verio

p0- 0- 0. cr 1. m t v w ca. pacific.verio.net

IIJ

ot em achi5.iij .net

Qw est

sfo- core - 03.inet .qwest .net

Telst ra BigPond

Pos5- 0- 0.cha- cor e2.Br isbane.t elst r a.net

UUNET

ATM2- 0.BR1.NYC5.ALTER.NET

Teleglobe

if - 8- 0.cor e1.New Yor k.Teleglobe.net

VSNL

E3- VSB1- LVSB.Bbone.v snl.net .in

KDD I nt ernet

gsr- ot e3.k ddnet .ad.j p

ChinaNet

p- 10- 1- 0- r1- s- bj bj - 1.cn.net

DN S Resolver in I OS Soft w are You can specify a default dom ain nam e t hat t he Cisco I OS Soft w ar e w ill use t o com plet e dom ain nam e r equest s for funct ions such as Telnet , TFTP, and ot her inst anc es of nam e com plet ion ( for exam ple, ip osp f d om a in - look u p) . You can specify eit her a single dom ain nam e or a list of dom ain nam es. Any I P host nam e t hat does not cont ain a dom ain nam e w ill hav e t he dom ain nam e t hat y ou specify appended t o it befor e being added t o t he host t able.

ip domain-name name ip domain-list name I t is also advisable t o include a nam e ser ver for t he r out er t o r esolve t he DNS r equest :

ip name-server server-address1 [[server-address2]...server-address6] Rem em ber t hat t he cur r ent pr a ct ice on t he I nt er net is t o quot e at least t w o DNS r esolv er s. The r eason is t he sam e as for any ot her sit uat ion: r edundancy . I f one DNS ser v er disappear s, t he ot her one can t ak e ov er . When bot h ar e t her e, t he r out er w ill look up t he servers in a round- robi n fashion for each r equest .

Con clu sion This chapt er cov er ed t he im por t ant st eps necessar y for an I SP t o set up it s init ial r out er configur at ion cor r ect ly . I t discussed t he im por t ance of t he loopback int er face,

70

how t o configur e int er faces, as w ell as how t o set up CEF and Net Flow. Nagle also w as cover ed, and t he chapt er concluded w it h a discussion on how an I SP uses DNS for t he net w or k back bone infr ast r uct ur e. Com bining t his chapt er w it h Chapt er 1, t he I SP now should hav e a w or k able r out er configur at ion r eady for configur ing r out ing pr ot ocols and t he secur it y feat ur es necessar y for any I SP back bone.

En dn ot e s 1. SPD is discussed in det ail in Chapt er 3, “ Rout ing Pr ot ocols. ”

71

Cha pt e r 3 . Rout ing Pr ot ocols The book so far has concent r at ed on get t ing t he cor e equipm ent in t he I SP back bone up t o a st at e in w hich it can be int roduced safely int o t he lar ger net w or k and t he I nt er net . This chapt er int r oduces t he m aj or r out ing pr ot ocols and som e of t he m ost useful feat ur es in t hose r out ing pr ot ocols t hat ar e available for I SPs. Most of t his chapt er ’s cont ent cov er s BGP, t he Bor der Gat ew ay Pr ot ocol used by I SP net w or k s t o pass r out ing infor m at ion bet w een each ot her . I n fact , BGP has gr ow n t o be m ore t han j ust a rout ing prot ocol used by net work border rout ers —BGP is now t he m echanism for t r anspor t ing noninfr ast r uct ur e pr efix es and r out ing inform at ion ar ound t he I SP’s back bone. The fir st par t of t his chapt er discusses init ial set up of t he r out er , int er ior r out ing pr ot ocols, and t he Hot St andby Rout ing Pr ot ocol ( HSRP) , a Cisco feat ur e designed t o m ake t he dual hom ing of host ing net w or ks failsafe and r edu n dan t . This chapt er assum es t hat t he I SP engineer has a w or k ing under st anding of t he cor e r out ing pr ot ocols used in t he I nt er net . I f not , please r efer t o t he online w hit epaper s or t he publicat ions list ed in t he “ Technical Refer ences and Recom m ended Reading” sect ion at t he end of t his book .

CI D R Fe a t u r e s All net w or k dev ices connect ed t o t he I nt er net m ust be CI DR- com pliant ( docum ent ed in RFCs 1812 and 2644) . Cisc o r out er s r unning Cisco I OS Soft w ar e v er sions ear lier t han Release 12.0 are m ade CI DR- com pliant if t hese t w o com m ands ar e ent er ed:

ip subnet-zero ip classless All Cisco r out er s connect ed t o t he I nt er net m ust hav e t hese com m ands t ur ned on. I OS Soft w ar e releases fr om 12.0 hav e t hese com m ands t ur ned on by default , but it is st ill w or t hw hile t o include t hem in any configur at ion t em plat e, j ust in case any oper at or r em ov es t he default s. I t m akes good sense in t he I SP oper at ions w or ld never t o r ely on any equip m ent v endor ’s default s and inst ead t o be v er y clear w hat t he r equir em ent s for y our ow n backbone ar e. Default s ar e for ent er pr ise and ot her net w or ks, not t he specialized and public I SP back bones. ( See t he sect ion t it led “ BGP Feat ur es and Com m ands” for BGP r equir em ent s.) Not e t hat t he t er m s subnet and supernet do not m ake m uch sense in t he classless I nt er net ( RFC 1812) . Unfor t unat ely, t hese t er m s ar e st ill w idely in use: •

Subnet som et im es is used t o refer t o a prefix t hat shares t he sam e init ial net w or k por t ion as anot her pr efix but has a lar ger m ask. ( For exam ple, 192.168.1.128/ 26 st ill is called a subnet of 192.168.1.0/ 24 because t he fir st 24 bit s ar e shar ed bet w een t he t w o r out es.) We prefer t he t erm subpr efix t o refer t o such a net work.

72

•

Supernet som et im es is used t o r efer t o t he aggr egat e of t w o pr efix es t hat ar e bit - w ise neighbor s. ( For ex am ple, 192.168.8.0/ 24 can be com bined w it h 192.168.9.0/ 24 t o for m t he aggr egat e [ or super net ] 192. 168. 8. 0/ 23. ) We prefer t he t erm aggr egat e t o refer t o such a net w ork.

I P Cla ssle ss The com m and ip cla ssle ss in Cisco I OS Soft w ar e affect s how t he r out er look s up r out es in t he I P r out ing t able w hen it is t r y ing t o r out e an I P pack et . I t changes t he r out e lookup fr om using t he old classful m et hod t o using classless r ules, even for classful rout ing prot ocols t hat st ill m ight be running on t he rout er. Th e Old Cla ssf u l Rou t e Look u p Ru le s The old behavior of a Cisco r out er is t o assum e t hat it know s about all t he r out es/ subnet w or k s of a dir ect ly at t ached net w or k . I t w ill do a classful r out e look up t o det er m ine a m at ch and t hen check for subnet s. For ex am ple, w hen t he r out er r eceiv es a pack et , t he net w or k por t ion of t he dest inat ion addr ess is com par ed t o t he r out ing t able for a m at ch. I f t her e is a m at ch, t he subnet s list ed for t hat classful net w ork are exam ined. I f a m at ch is found, t he packet is forw arded. I f no m at ch is found, t he packet is dropped and an I CMP Dest inat ion Unreachable m essage is r et ur ned. This is t r ue even if a default r out e exist s in t he I P r out ing t able because t he r out er assum es t hat it know s about all r out es for t hat classful net w or k. Consider t he sit uat ion if a pack et ar r iv es w it h dest inat ion addr ess 192.168.1.200 and t he rout ing t able has only t he subnet 192.168.1.0/ 26 in it . The r out er w ill at t em pt t o look for 192.168.1.0/ 24 ( t he par ent Class C net w or k of t he dest inat ion addr ess) and for w ar d t o t hat net w or k’s or igin. I f t hat Class C addr ess does not exist in t he r out ing t able and a subnet of t he Class C addr ess cov er ing 192.168.1.200 does not ex ist in t he r out ing t able, t he r out er w ill dr op t he pack et . Th e Cla ssf u l Rou t e Look u p Ru le s Th e ip cla ssle ss com m and has changed t he old behav ior for all r out ing pr ot ocols configured on t he rout er . The r out er ignor es t he class of t he dest inat ion addr ess and dir ect ly com par es t he net w or k por t ion of t he dest inat ion addr ess w it h all it s k now n r out es. I f no m at ch is found, t he packet is for w ar ded t o t he default r out e, if one ex ist s; ot her w ise, it is dropped. This lat t er act ion is required for any rout er connect ed t o t he I nt er net . Repeat ing t he pr ev ious ex am ple but using t he default classless r ules, if a pack et ar r iv es for host 192.168.1.200 and t he r out ing t able has only t he net w or k 192. 168. 1. 0/ 26, t he r out er w ill dr op t he pack et unless a default r out e has been configur ed on t he r out er . This is t he ex pect ed behav ior since t he I nt er net becam e classless in 1994.

The Zero I P Subnet The com m and ip su b n e t - zero t ells t he r out er t hat t he zer o subnet is a legit im at e subnet of t he classful net w or k being configur ed on t he r out er . For ex am ple, if t he

73

Class C net w or k 129.168.128.0 is subdiv ided int o eight subnet s, each w ould hav e a / 27 m ask for t he zer o subnet , 192.168.128.0/ 27, t hr ough t o t he sev ent h subnet , 1 9 2 . 168.128.224/ 27. The fir st and last subnet s of a classful net w or k hist or ically w er e not used because of t he pot ent ial confusion bet w een t hese and t he net w or k / br oadcast addr ess of t hat classful net w or k . Consider t his ex am ple: 192. 168. 128. 20

192. 168. 128. 0000101 0

255.255.255.224

255. 255. 255. 11100000

The fir st 3 bit s of t he final oct et ar e 0, indicat ing t he zer o subnet or 192.168.128.0. How ever , t he Class C net w or k 192.168.128.0 is w r it t en in t he sam e w ay, giving r ise t o possible confusion on t he r out er and some older TCP/ I P st ack s. So t he zer o subnet cannot be assigned t o an int er face in t his case. Now adays in t he classless I nt er net , 192.168.128.0/ 27 is sim ply anot her net w or k w it h no possibilit y of confusion w it h 192.168.128.0/ 24. Not e t hat m isconfigur at ion of t he net w or k m ask on a host on t he 192.168.128.0/ 27 subnet r esult s in t he possibilit y of confusion if t he net w or k addr ess 192.168.128.0 is used, but t his is a configur at ion er r or on t he par t of t he user , not a consequence of t he addr essing schem e as in t he case of t he classful syst em . Also, I P st ack s on v ir t ually all sy st em s connect ed t o t he I nt er net suppor t classless I P and, t her efor e, allow t he use of t he zer o subnet for addr essing pur poses. Configur ing ip su b n e t - zero on t he r out er allow s t he zer o subnet t o be used on rout er int erfaces. Mor e infor m at ion on t he zer o subnet can be found in t he Cisco. com Technical Not e at h t t p: / / w w w . cisco. com / w ar p/ public/ 105/ 40.ht m l.

Se le ct ive Pa ck e t D isca r d When a link goes int o a sat ur at ed st at e, t he r out er w ill dr op pack et s. The pr oblem is t hat t he r out er w ill dr op any t y pe of pack et s, including r out ing pr ot ocol pack et s. Select iv e Pack et Discar d ( SPD) at t em pt s t o dr op nonr out ing pack et s inst ead of r out ing pack et s w hen t he link is ov er loaded. The basic idea behind SPD is t his: I f you m ark all BGP and I GP packet s as being “ im por t ant ” and pr efer t hese pack et s ov er ot her s, y ou should pr ocess a lar ger per cent age of t he packet s t hat w ill allow r out ing and, consequent ly, keep t he BGP and I GP sessions st able. SPD can be configur ed as follow s. Som e of t hese com m ands ar e hidden fr om t he Cisco I OS Soft w ar e configur at ion helper but ar e w ell k now n t o t he I SP com m unit y . ( Not e t he differ ence bet w een t he 11.1CC and 12.0 r elease v er sions of t he com m ands.) The follow ing com m and t ur ns on SPD. The no v er sion of t he com m and t ur ns off SPD. The default is on for I OS Soft w are versions 12.0 and lat er.

74

[no] ip spd enable [no] spd enable

! for 11.1CC ! for 12.0+

The default value is 100. I t is also possible t o specify how m any high- pr ecedence pack et s y ou w ill queue ov er t he nor m al input hold queue lim it . This is t o r eser v e r oom for incom ing high- pr eceden ce pack et s. Th e com m and t o do t his is as follows:

ip spd headroom spd headroom

! for 11.1CC ! for 12.0+

I t is also possible t o set low er and upper I P pr ocess- level queue t hresholds for SPD. Wit h SSE- based SPD, low er- pr ecedence pack et s r andom ly ar e dr opped w hen t h e queue size hit s t he m inim um t hr eshold. The dr op pr obabilit y incr eases linear ly w it h t he queue size unt il t he m ax im um t hr eshold is r eached, at w hich point all low erpr ecedence pack et s ar e dr opped. For r egular SPD, low er- pr ecedence pack et s ar e dr opped w hen t he queue size r eaches t he m inim um t hr eshold. Default s ar e 50 and 75, r espect iv ely . These default v alues w er e not based on r eal- life ex per ience at t he t im e t hey w er e chosen, so t hey m ight need som e t uning. The com m and t o set t he t hr eshold levels is as follows:

[no] ip spd queue [min-threshold | max-threshold] Th e sh ow ip sp d exec com m and also show s t he cur r ent SPD m ode, t he cur r ent and m ax size of t he I P pr ocess- level input queue, and t he available headroom . SPD m ode is one of disabled, norm al, ra ndom drop, or full drop. High- pr ecedence pack et s go t o t he pr ior it y queue. A sam ple out put fr om t he r out er follow s:

alpha#show ip spd Current mode: normal. Queue min/max thresholds: 73/74, Headroom: 100, Extended Headroom: 10 IP normal queue: 0, priority queue: 0. SPD special drop mode: none alpha# sh ow in t e r f a ce sw it ch in g has som e ext ra inform at ion, t oo, w hen SPD is t urned on:

gmajor#show interface ethernet 3/0 switching Ethernet3/0 CORE BACKBONE Throttle count 0 Drops RP 0 SP SPD Flushes Fast 0 SSE SPD Aggress SPD Priority

Fast Inputs

0 88141

Drops

0 542019

0

75

SPD flushes separ at e t he r out e pr ocessor and sw it ch pr ocessor flushes, and SPD Pr ior it y list s t he pr ior it y pack et s r eceiv ed and dr opped as t he r esult of ex ceeding t he headr oom t hr eshold. Not e t hat sw it ching is a hidden opt ion for t he sh ow in t e r f a ce com m and and m ust be ent er ed in full t o be r ecognized by t he c om m and parser. SPD is act ivat ed by default on all I OS Soft w are releases from 12.0; it is r ecom m ended t hat it s configur at ion be left in t he default st at e unless y ou under st and w hat y ou ar e doing. Som e I SPs hav e gained ex t r a benefit s by m odify ing SPD param et er s; how ever , m ost sim ply leave t he default configur at ion. 11.1CC and 11.1CA suppor t ed SPD as an opt ion, and if t hose r eleases st ill ar e being used, it is best t hat SPD be sw it ched on as indicat ed her e.

H ot St a n dby Rou t in g Pr ot ocol A new feat ur e in r ecent v er sions of I OS Soft w ar e is t he Hot St andby Rout ing Pr ot ocol ( HSRP, descr ibed in t he infor m at ional RFC 2281 —see ht t p: / / w w w .iet f.or g/ r fc/ r fc2281.t xt ) . This feat ur e is especially useful on LANs w it hin t he I SP’s backbone —for ex am ple, for net w or k ser v er s, non- Cisco access ser v er s, and host ed ser v er s. The m ot ivat ion for t his pr ot ocol is t o suppor t t he need for a default gat ew ay on LAN net w or k s w hen t w o gat ew ay r out er s ar e pr ov iding connect iv it y t o t he wider net work and t he I nt er net . Rout er s t end t o suppor t t he full set of r out ing pr ot ocols. Most com put er w or k st at ions r un v ar iant s of UNI X or Window s, for w hich t her e is eit her no or m inim ally funct ional r out ing soft w ar e. Configur ing som et hing such as t he public dom ain soft w ar e GATED or t he v endor ’s ow n soft w ar e t o per for m a dy nam ic r out ing funct ion is usually an unnecessar y and unw ieldy com pr om ise. The easiest and best solut ion is t o configur e a st at ic default r out e on t he w or kst at ion or ser ver and use HSRP. Figur e 3- 1 show s a t y pical LAN w it h t w o r out er s used t o connect t o t he I SP back bone. To im plem ent HSRP, t he configur at ion r equir ed for t hese t w o r out er s look s somet hing lik e t he follow ing: Fig u r e 3- 1 . D u a l- Ga t e w a y LAN

76

Router 1: interface ethernet 0/0 description Server LAN ip address 169.223.10.1 255.255.255.0 standby 10 ip 169.223.10.254 ! Router 2: interface ethernet 0/0 description Service LAN ip address 169.223.10.2 255.255.255.0 standby 10 priority 150 standby 10 preempt standby 10 ip 169.223.10.254 ! The t w o rout ers have t heir LAN I P addr esses convent ionally defined in t he pr eceding configur at ion. How ever , anot her I P addr ess has been defined in t he com m and line st a n d b y 1 0 ip 1 6 9 .2 2 3 .1 0 .2 5 4 . This addr ess is t he addr ess of t he v ir t ual default gat ew ay defined on t he LAN. All t he sy st ems on t he LAN, apar t fr om Rout er 1 and Rout er 2, use t his addr ess as t he “ default r out e.” Rout er 2 has a st andby pr ior it y of 150, higher t han t he default of 100. Ther efor e, Rout er 2 w ill be t he default gat ew ay at all t im es unless it is unav ailable ( t hat is, dow n) . The pr e e m pt dir ect ive t ells Rout er 1 and Rout er 2 t hat Rout er 2 should be used as t he default gat ew ay w henev er possible. For ex am ple, if Rout er 2 w er e t em por ar ily out of ser v ice, it w ould t ake over from Rout er 1 w hen it is ret urned t o norm al operat ion. A pot ent ial issue her e is t hat all t he out bound t r affic goes t hr ough Rout er 2; w her eas, inbound t r affic m ay be shar ed bet w een t he t w o. Som e I SPs like t o shar e out bound t r affic bet w een t he t w o r out er s; t his is achiev ed by set t ing up t w o st andby groups as in t he follow ing exam ple:

77

Router 1: interface ethernet 0/0 description Server LAN ip address 169.223.10.1 255.255.255.0 standby 10 ip 169.223.10.254 standby 11 priority 150 standby 11 preempt standby 11 ip 169.223.10.253 ! Router 2: interface ethernet 0/0 description Service LAN ip address 169.223.10.2 255.255.255.0 standby 10 priority 150 standby 10 preempt standby 10 ip 169.223.10.254 standby 11 ip 169.223.10.253 ! Wit h t his configur at ion on t he t w o r out er s, t he I SP set s t he default r out e configur at ion on it s ser v er s so t hat t r affic is shar ed bet w een t he t w o v ir t ual default gat ew ay s. A good fir st st ar t ing point is t o configur e half of t he ser v er s t o point t o 169.223.10.254 and t he r em aining ser v er s t o point t o 169.223.10. 253. I f finer load balancing is r equir ed, t he I SP sim ply changes w hich default gat ew ay t he ser v er has configur ed on it . I f one link fails, t he ot her r out er w ill t ake over all t r affic unt il t he failed r out er r et ur ns t o nor m al oper at ion. HSRP m or e r ecent ly has been enhanced t o include t he capabilit y t o m onit or t he st at e of t he uplink s fr om t he gat ew ay r out er s. So, for ex am ple, if t he link s connect ing Rout er 1 and Rout er 2 in Figur e 3- 1 w er e ser ial connect ions t o ot her PoPs, HSRP can be configur ed t o m onit or t hose links and “ fail” one r out er if it s uplink goes dow n. The sam ple configur at ion for t his m ight be t he follow ing, w her e Ser ial 0/ 0 on Rout er 1 and Ser ial 3/ 1 on Rout er 2 ar e t r ack ed for t heir st at us:

Router 1: interface ethernet 0/0 description Server LAN ip address 169.223.10.1 255.255.255.0 standby 10 ip 169.223.10.254 standby 10 track Serial 0/0 standby 11 priority 150 standby 11 preempt standby 11 ip 169.223.10.253 standby 11 track Serial 0/0 ! Router 2: interface ethernet 0/0 description Service LAN ip address 169.223.10.2 255.255.255.0 standby 10 priority 150 standby 10 preempt

78

standby standby standby standby

10 10 11 11

ip 169.223.10.254 track Serial 3/1 ip 169.223.10.253 track Serial 3/1

! I f eit her goes dow n because of a link failur e or a r em ot e end failur e, t he r out er w ill r em ove it self fr om availabilit y, r esult ing in failover t o t he ot her r out er in t he st andby group.

I P Sou r ce Rou t in g The Cisco I OS Soft w ar e ex am ines I P header opt ions on ev er y pack et . I t suppor t s t he I P header opt ions St r ict Sour ce Rout e, Loose Sour ce Rout e, Recor d Rout e, and Tim e St am p, w hich ar e defined in RFC 791. I f t he soft w ar e finds a packet w it h one of t hese opt ions enabled, it per for m s t he appr opr iat e act ion. I f it finds a pack et w it h an inv alid opt ion, it sends an I CMP Par am et er Pr oblem m essage t o t he sour ce of t he pack et and discar ds t he pack et . I P has a feat ur e know n as sour ce r out ing t hat allow s t he sour ce I P host t o specify a r out e t hr ough t he I P net w or k. Sour ce r out ing is specified as an opt ion in t he I P header . I f sour ce r out ing is specified, t he soft w ar e for w ar ds t he pack et accor ding t o t he specified sour ce r out e. The default is t o per for m sour ce r out ing. Som e I SPs do not w ant t heir cust om er s t o hav e access t o sour ce r out ing, so it is t ur ned off at t he cust om er edge. You can do t his is w it h t he follow ing gener ic com m and:

no ip source-route Som e I SPs lik e t o hav e sour ce r out ing av ailable t o t roubleshoot problem s in t heir ow n net w or k s and neighbor ing net w or k s, especially w hen r out ing has br ok en inside one of t hose net w or k s. Ot her I SPs r equir e sour ce r out ing t o be t ur ned on w hen peer ing w it h I SPs. An ex am ple of using t r acer out e w it h sour ce r out ing enabled on t he int er m ediat e net w or ks follow s. The I SP is check ing t he pat h fr om 192.121.154.170 t o w w w .spr int .net . Th e - g opt ion specifies t hat t r acer out e should do t he pr obe as t hough pack et s ar e com ing fr om t he 192.121.154.170 sour ce addr ess. Not ice t he ! S in St ep 18—t his st at es t hat t hat host ( r out er ) is block ing I P sour ce r out ing.

Unix% /usr/local/bin/traceroute -g 192.121.154.170 www.sprint.net traceroute to www.sprint.net (208.27.196.10), 30 hops max, 40 byte packets 1 fe0-1-0.hr1.cbg1.gbb.uk.uu.net (158.43.128.192) 1.294 ms 0.703 ms 0.539 ms 2 pos0-0.cr1.cbg1.gbb.uk.uu.net (158.43.129.129) 0.748 ms 0.993 ms 0.747 ms 3 pos0-0.cr2.cbg1.gbb.uk.uu.net (158.43.129.133) 1.586 ms 1.146 ms 2.145 ms 4 pos0-2.cr2.lnd6.gbb.uk.uu.net (158.43.254.2) 4.43 ms 4.143 ms 3.731 ms

79

5 ge2-0.cr1.lnd6.gbb.uk.uu.net (158.43.254.65) 4.395 ms 4.044 ms 4.148 ms 6 POS11-0-0.GW2.LND1.Alter.Net (146.188.5.41) 4.898 ms 10.705 ms 5.082 ms 7 122.at-2-0-0.XR2.LND2.Alter.Net (146.188.15.170) 5.995 ms 6.179 ms 6.039 ms 8 194.ATM1-0-0.HR2.LND1.Alter.Net (146.188.15.129) 12.422 ms 7.229 ms 6.018 ms 9 146.188.13.165 (146.188.13.165) 7.4 ms 8.331 ms 8.376 ms 10 gblon802-ta-d5-1-0.ebone.net (192.121.154.113) 16.846 ms 15.647 ms 21.246 ms 11 gblon801-ta-p6-0-0.ebone.net (192.121.154.170) 21.513 ms 21.535 ms 21.871 ms 12 gblon304-tb-p0-1.ebone.net (195.158.226.233) 366.643 ms 358.076 ms 269.65 ms 13 usnyk106-tc-p0-3.ebone.net (195.158.229.17) 359.798 ms 204.034 ms 241.634 ms 14 sl-bb11-nyc-5-3.sprintlink.net (144.232.9.229) 380.835 ms 84.641 ms 96.338 ms 15 sl-bb10-nyc-9-0.sprintlink.net (144.232.7.1) 94.433 ms 90.512 ms 87.394 ms 16 sl-bb12-rly-7-0.sprintlink.net (144.232.9.226) 102.331 ms 88.533 ms 93.294 ms 17 sl-bb11-rly-8-0.sprintlink.net (144.232.7.213) 89.341 ms 103.678 ms 174.892 ms 18 sl-bb5-dc-4-0-0.sprintlink.net (144.232.7.166) 86.662 ms !S 88.132 ms !S * W ARN I N G As a general rule of t hum b, if you are not using I P sour ce r out ing, y ou should t ur n it off. I P sour ce r out ing is a w ell- k now n secur it y v ulner abilit y used in at t ack s against a system .

Con figu r in g Rou t in g Pr ot ocols This sect ion ex am ines t he m ost efficient and effect iv e w ay s of configur ing I GPs and BGP t o giv e gr eat est scalabilit y in I OS Soft w ar e. An I GP is t he int er ior gat ew ay pr ot ocol and is t he m edium t hr ough w hich an I SP’s backbone r out er s pass r out ing infor m at ion about t he I SP’s infr ast r uct ur e t o each ot her . As w ill be m ent ioned sever al t im es in t his book, t he design goal for an I GP is t o keep t he num ber of prefixes in it as sm all as possible. All pr efix es w hich do not belong t o t he I SP’s infr ast r uct ur e ar e put int o BGP; t hese ar e t he pr efixes t hat need local, nat ional, r egional, or global visibilit y . Essent ially t hr ee I GPs ar e in use by I SPs t oday . These ar e t he pr ot ocols I nt er m ediat e Sy st em- t o- I nt erm ediat e Syst em ( I S- I S) , Open Shor t est Pat h Fir st ( OSPF) , and Enhanced I nt er nal Gat ew ay Rout ing Pr ot ocol ( EI GRP) . The for m er ar e indust r y st andards. I S- I S w as dev eloped by OSI ( I SO 10589) and is now being enhanced by t he I ETF ( RFC 1195 is a proposed st andard) t hrough t he I S- I S Wor king Gr oup. OSPF w as developed by t he I ETF specifically for I P ( RFC 1131) and has been m uch enhanced t o t he I nt er net st andar d described in RFC 2328. EI GRP is an enhancem ent of I GRP developed by Cisco in t he lat e 1980s.

80

Only one EGP is used in t he I nt er net . BGP Ver sion 4 has been used since 1994 as t he r out ing pr ot ocol for aut onom ous sy st em s t o ex change pr efix es w it h each ot her . BGP4 is t he classless v er sion of BGP and has seen m any enhancem ent s ov er t he last few year s as t he I nt er net gr ow s ever lar ger .

Rout er I D The r out er ident ifier in an I GP is chosen fr om t he loopback int er face of t he r out er , if configur ed at t he t im e t he r out ing pr ocess is st ar t ed. I f m or e t han one loopback int er face is configur ed w it h an I P addr ess, t he r out er ident ifier is t he highest of t hose I P addr esses. I f t he loopback is not configur ed, t he highest I P addr ess configur ed on t he r out er at t he t im e t he I GP w as st ar t ed is used. I SPs pr efer st abilit y , so t he loopback int erface usually is configured and act ive on m ost I SP rout ers. The r out er I D also is used as t he last st ep of t he BGP pat h- select ion pr ocess. Ther e’s anot her r eason t o ensur e t hat t he loopback int er face is configur ed and has an I P address: I f t here is no loopback, t he rout er I D is t he highest I P address configured on t he box at t he t im e t he BGP pr ocess w as st ar t ed. ( This is pot ent ially pr oblem at ic because, if t he r out er is gaining m or e int er faces or m or e act iv at ed connect ions w it h I P addr esses assigned t o t hem , t he r out er I D can pot ent ially change if t he r out er is r eboot ed or t he BGP pr ocess is r est ar t ed.)

Choosing a n I GP A gener al discussion on t he appr opr iat e t echnical choice of I GPs for an I SP bac kbone is cur r ent ly bey ond t he scope of t his book . Many good com par isons hav e been done descr ibing t he pr os and cons of t he differ ent I GPs. A com m only quot ed ex am ple is t he pr esent at ion by Dave Kat z at t he June 2000 NANOG ( ht t p: / / w w w . nanog. or g/ m t g- 0006/ kat z.ht m l ) com par ing I S - I S and OSPF. The choice of I GP gener ally seem s t o be m ade on t he basis of ex per ience because t echnically t her e is lit t le w ay t o choose am ong t he t hr ee for m ost pr act ical purposes. Those engineers wit h a st rong I S- I S backgr ound w ill alw ays choose I S - I S. Those w it h a st r ong OSPF back gr ound w ill alw ay s choose OSPF. The r ule of t hum b seem s t o be t hat beginner s t o int er ior r out ing choose EI GRP because it is easy t o get st ar t ed. How ev er , OSPF is a bet t er choice because it for ces good I GP design t o ensur e t hat t he net w or k w ill scale. And t hose w ho ar e v er y ex per ienced t end t o choose I S - I S on Cisco r out er s because it allow s for bet t er scaling w it h m or e configur at ion opt ions t han t he ot her t wo I GPs.

Put t ing Prefix es int o t he I GP At least t hr ee possible w ays exist for inser t ing pr efixes int o t he I GP. How ever , I SPs aim t o find t he m ost efficient m et hod and t he one t hat w ill giv e gr eat est scalabilit y of t heir net w or ks. Th e n e t w or k St a t e m e n t

81

The m ost efficient , scalable, and safe m et hod is t o use t he n e t w or k st at em ent t o cover t he infr ast r uct ur e addr esses t hat have an act ive I GP r unning over t hem or r equir e t hem t o be car r ied in t he I GP. A sam ple configur at ion m ight be as follow s:

! interface loopback 0 ip address 220.220.16.1 255.255.255.255 ! interface serial 0/0 ip address 220.220.17.1 255.255.255.252 ! interface serial 1/0 ip unnumbered loopback 0 ! router ospf 100 network 220.220.16.1 0.0.0.0 area 0 network 220.220.17.0 0.0.0.3 area 0 passive-interface loopback 0 passive-interface serial 1/0 ! Not ice t he use of t he p a ssiv e- in t e r fa ce com m and t o disable OSPF neighbor discov er y on any int er faces t hat do not hav e connect ed neighbor s or t hat don’t need t o run a rout ing pr ot ocol. I t is not good pr act ice, and indeed is st r ongly discour aged, t o r edist r ibut e anyt hing int o an int er ior r out ing pr ot ocol. This is r ar ely r equir ed. r e d ist r ib u t e con n e ct e d in t o a n I GP Using t he r e d ist r ib u t e con n e ct e d com m and inj ect s t he pr efixe s assigned t o ev er y connect ed int er face int o t he I GP. Apar t fr om cr eat ing an ex t er nal net w or k t y pe, w hich, for exam ple, in OSPF is car r ied acr oss all ar eas, t he r out er per iodically m ust ex am ine t he Rout ing I nfor m at ion Base ( RI B) t o see if t her e ar e any changes t o t he connect ed st at e. This t ak es ex t r a CPU cy cles. Addit ionally , any link- st at e changes ar e passed int o t he I GP, som et hing t hat m ight not be desir able, especially in t he case of link addr esses going t o ex t er nal connect ions. I n t he OSPF ex am ple, t he ex t er nal linkst at e change is hear d ov er t he w hole net w or k . Anot her pr oblem is t hat r edist r ibut ing connect ed int er faces int o an I GP cover s all connect ed int er faces on a r out ing dev ice. Som e addr esses m ight not be desir able in an I GP, so I SPs t hen w ould have t o add a r out e m ap t o apply an access- list filt er t o t he r edist r ibut ion pr ocess. This could cause som et hing else t o go w r ong: Access list s can be delet ed by accident , result ing in prefixes being leaked int o t he I GP. Again, t his is undesir able. The r ecom m endat ion is t o av oid using r e d ist r ib u t e con n e ct e d if at all possible. Many I SPs do use r e d ist r ib u t e con n e ct e d su b n e t s r ou t e- m ap t hr ough t heir ow n ex per ience and oper at ional pr act ices, but m ost new st ar t er s in t he indust r y hav e built lar ge successful net w ork s w it hout needing t his t y pe of r edist r ibut ion. r e d ist r ib u t e st a t ic in t o a n I GP

82

Th e r e dist r ibu t e st a t ic com m and inj ect s all st at ic r out es int o t he I GP. Apar t fr om cr eat ing an ext er nal net w or k t ype, w hich, for exam ple, in OSPF is car r ied acr oss all ar eas, t he r out er per iodically m ust exam ine t he RI B t o see if t her e ar e any addit ions t o or delet ions fr om t he st at ic r out e configur at ion. This t ak es ex t r a CPU cy cles. And if t he nex t hop t o w hich t he st at ic r out e point s disappear s, t he st at ic r out e is w it hdr aw n, w it h a r esult ing w it hdr aw al fr om t he I GP. ( Of cour se, t he r ecent ly int r oduced per m anent st at ic r out e is designed t o w or k ar ound t his pr oblem , but t he pr oblem is best avoided in t he fir st place! ) The ot her pr oblem w it h r edist r ibut ing st at ic r out es int o an IGP is t hat it covers all st at ic r out es configur ed on a r out ing dev ice. Som e pr efix es m ight not be desir able in an I GP, r esult ing in a sit uat ion sim ilar t o t hat not ed in t he r e d ist r ib u t e d con n e ct e d case. The r ecom m endat ion is t o av oid using r e d ist r ib u t e conn e ct e d if at all possible. Ver y few I SPs use t his t ype of r edist r ibut ion because BGP is t he favor ed pr ot ocol for car r y ing m ost addr ess space chunk s now .

I GP Sum m arizat ion I t is good pr act ice and aids scalabilit y t o ensur e t hat sum m ar izat ion is im plem ent ed w her e it m akes sense and w henever it is possible. I GPs w or k m ost efficient ly w it h as few pr efix es as possible in t he r out ing pr ot ocol. Sm all I GPs conv er ge m or e quick ly , ensur ing r apid net w or k healing in case of link failur es. Configur ing sum m ar izat ion is differ ent for each pr ot ocol—consult t he I OS docum ent at ion on Cisco. com. For ex am ple, OSPF uses t he a r e a < n > r a n g e OSPF subcom m and.

I GP Adj a cency Cha nge Logging Neighbor st at e logging should be enabled in each I GP. This m eans t hat it becom es easier t o find out about neighbor st at es, r easons for st at e changes, and so on. For each I GP, t he I GP subcom m and l o g- a d j a ce n cy - ch a n ge s enables logging. ( Som e older v er sions of I OS Soft w ar e r equir e t he I GP t o be specified as well—for ex am ple, o sp f lo g- a d j a ce n cy - ch a n ge s.) By t he t im e t his book is published, t he com m and w ill likely be enabled by default in I OS Soft w ar e. I f logging is enabled, log m essages ar e sent t o w her ever t he logging out put has been configured. For m ost I SPs, t his w ould be a UNI X sy slog ser v er and t he int er nal logging buffer held on t he r out er—see t he sect ion “ Det ailed Logging” in Chapt er 1, “ Soft w ar e and Rout er Managem ent ,” for infor m at ion on how t o configur e t his on t he r out er . The t y pical out put w ould be sim ilar t o w hat follow s; t his ex am ple show s t he act iv at ion of a neighbor adj acency w it h a r out er connect ed t o ser ial3/ 2:

Nov 10 03:56:23.084 AEST: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.2.2 on Serial3/2 from DOWN to INIT, Received Hello Nov 10 03:56:32.620 AEST: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.2.2 on Serial3/2 from INIT to 2WAY, 2-Way Received

83

Nov 10 03:56:32.620 AEST: %OSPF-5-ADJCHG: Process Serial3/2 from 2WAY to EXSTART, AdjOK? Nov 10 03:56:32.640 AEST: %OSPF-5-ADJCHG: Process Serial3/2 from EXSTART to EXCHANGE, Negotiation Done Nov 10 03:56:32.676 AEST: %OSPF-5-ADJCHG: Process Serial3/2 from EXCHANGE to LOADING, Exchange Done Nov 10 03:56:32.676 AEST: %OSPF-5-ADJCHG: Process Serial3/2 from LOADING to FULL, Loading Done

1, Nbr 192.168.2.2 on

1, Nbr 192.168.2.2 on

1, Nbr 192.168.2.2 on

1, Nbr 192.168.2.2 on

This show s t he det ailed est ablishm ent of t he OSPF neighbor r elat ionship bet w een t w o r out er s, going fr om t he Hello st at e t hr ough 2WAY, Exchange- St ar t , and Loading t o Full adj acency. Alt hough t his sor t of det ail m ight not alw ays be r equir ed, it becom es ver y im por t ant w hen t he I S Ps ar e follow ing t he r ecom m ended pr act ice of enabling aut hent icat ion on t he I GP bet w een r out er s. ( Neighbor aut hent icat ion is covered in m ore det ail in Chapt er 4, “ Secur it y . ” )

Put t ing Prefix es int o BGP As w it h I GPs, t her e ar e at least t hr ee possible w ays of inser t ing pr efixes int o BGP. These m et hods ar e discussed in t he follow ing t hr ee sect ions. Th e n e t w or k St a t e m e n t The safest m et hod t o use is one n e t w or k st at em ent per pr efix t hat is t o be inj ect ed int o BGP. A sam ple configur at ion m ight be as follow s:

interface loopback 0 ip address 220.220.16.1 255.255.255.255 ! interface serial 1/0 ip unnumbered loopback 0 ! router bgp 100 no synchronization network 220.220.18.0 mask 255.255.252.0 ! ip route 220.220.18.0 255.255.252.0 serial 1/0 ! I t is not good pr act ice, and indeed it is st r ongly discour aged, t o r edist r ibut e any int er ior r out ing pr ot ocol int o BGP. I t is never r equir ed, and in t he past it has led t o m any ser ious accident s on t he global I nt er net . r e d ist r ib u t e con n e ct e d in t o BGP Using t he r e d ist r ib u t e con n e ct e d com m and inj ect s t he pr efix es assigned t o any connect ed int er face int o BGP. As w it h I GP, one disadv ant age is t hat t he r out er per iodically m ust exam ine t he RI B t o see if t her e ar e any changes t o t he connect ed

84

st at e, t ak ing ex t r a CPU cy cles. Addit ionally , any link- st at e changes ar e passed int o BGP, gener at ing a r out e flap on t he I nt er net r out ing t able and w ast ing consider able CPU cycles on all rout ers in t he I nt er net t hat have a view of t his pr efix. Anot her pr oblem is t hat r edist r ibut ing connect ed int er faces int o BGP cov er s all connect ed int er faces on a r out ing dev ice. I f som e addr esses ar en’t desir able in BGP, I SPs w ould need t o add a r out e m ap t o apply an access- list filt er t o t he r edist r ibut ion pr ocess. This could cause som et hing else t o go w r ong—access list s can be delet ed by accident , r esult ing in pr efixes being leaked int o BGP. Again, t his is ver y undesir able. The r ecom m endat ion is t o av oid using r e d ist r ib ut e con n e ct e d if at all possible. Ver y few I SPs use t his t ype of r edist r ibut ion because it r ar ely has any r eal use for BGP. r e d ist r ib u t e st a t ic in t o BGP Using t he r e dist r ibu t e st a t ic com m and inj ect s all st at ic r out es int o BGP. Again, t he rout er periodically m ust exam ine t he RI B t o see if t her e ar e any addit ions t o or delet ions fr om t he st at ic r out e configur at ion. This t ak es ex t r a CPU cy cles. And if t he nex t hop t o w her e t he st at ic r out e point s disappear s, t he st at ic r out e is w it hdr aw n, w it h a result ing w it hd r aw al fr om BGP. A w it hdr aw al fr om BGP r esult s in a r out e flap, as m ent ioned pr eviously. ( As in t he I GP case, t he per m anent st at ic r out e is designed t o w or k ar ound t his issue, but t he pr oblem is best avoided in t he fir st place! ) Again, redist ribut ing st at ic r out es int o BGP cover s all st at ic r out es configur ed on a r out ing device. Som e pr efixes ( such as t he RFC 1918 pr efixes) m ight not desir able in BGP, so I SPs w ould add a r out e m ap t o apply an access- list filt er t o t he r edist r ibut ion pr ocess. This can pot ent ially lead t o t he sit uat ion discussed in t he case of r e d ist r ib u t e con n e ct e d . Som e sit uat ions dem and r e dist r ibu t e st a t ic , especially w hen com m unit ies ar e at t ached t o t he pr efix es inser t ed int o t he BGP pr ocess. This appear s t o be t he pr efer r ed pr ocess am ong ISPs because t hey find it easier t o m anage t han apply ing a prefix m at ching t he rout e m ap t o individual n e t w o r k st at em ent s. The follow ing ex am ple is v er y t y pical on a cust om er aggr egat ion or edge dev ice t hat set s a com m unit y of 65534: 1234 on all t he I SP’s cust om er pr efix es:

router bgp 65534 redistribute static route-map static-to-bgp ! route-map static-to-bgp permit 5 match ip address prefix-list netblock-to-bgp set community 65534:1234 set origin igp ! ip prefix-list netblock-to-bgp permit 220.220.0.0/16 le 30 ! ip route 220.220.1.32 255.255.255.224 serial 0/0 !

85

Ther e ar e danger s w it h t his: The pr efix list could be delet ed, but t he I SP m ust w eigh t his danger against t he adm inist r at iv e inconv enience/ ov er head of set t ing com m unit ies and so on on a per- n et w or k- st at em ent basis. The r ecom m endat ion is t o t r y t o avoid using r e dist r ibu t e st a t ic if at all possible and t hen use it only if t he sit uat ion ( such as t he one descr ibed pr ev iously ) r eally w ar r ant s it .

I GP Con figu r a t ion H in t s I SPs should be aw ar e of a few configur at ion hint s w hen configur ing I GPs for t heir backbones. When t he decision bet w een EI GRP, OSPF, and I S - I S has been m ade, t he aim is t o design t he I GP for efficient oper at ion, fast failov er , and, m ost im por t ant , scalabilit y .

N e t w or k D e sign Many good docum ent s ar e av ailable on t he Cisco. com Websit e t o help w it h I GP design— det ailed design is bey ond t he scope of t his book . How ev er , it is w or t h em phasizing t he I SP engineer ’s w ell- k now n design t ips: •

Thr ee t y pes of pr efix es ex ist : - Access net work prefixes - I nfrast ruct ure prefixes - Ext ernal prefixes

• • • • • •

I GPs car r y infr ast r uct ur e pr efix es only . Access net w or k pr efix es ar e not par t of t he infr ast r uct ur e —use eit her ip u n n u m be r e d or BGP n e x t - hop- se lf if at all possible. Cust om er prefixes are never carried in an I GP; BGP is designed for t his. I GPs ar e k ept sm all for best conv er gence speed; using a good addr ess plan and sum m ar izat ion can help. iBGP and eBGP pr efix es ar e nev er dist r ibut ed int o an I GP. I GP pr efix es ar e nev er dist r ibut ed int o BGP.

Pr e fix Type s Many new com er s t o t he I nt er net indust r y r eact w it h sur pr ise w hen t he ex per ienced engineer s t alk about t he differ ent t y pes of pr efix es pr esent in an I nt er net back bone. I t is ver y im por t ant t o differ ent iat e am ong access net w or k pr efix es, infr ast r uct ur e pr efix es used for t he I SP’s back bone, and ex t er nal pr efix es t hat cov er t he r em aining uses. Acce ss N e t w or k Pr e fix e s Access net w or k pr efix es com m only ar e used in nonper m anent access net w or k s. So cust om er s connect ing using cable, ADSL, PSTN, or I SDN dialup t end t o be assigned

86

addr ess space on a dy nam ic basis. When t hey connect , PPP or DHCP is used t o assign t hem an addr ess for t he dur at ion of t heir session. When t hey t er m inat e t he connect ion, t he addr ess goes bac k int o t he pool. Access net w ork prefixes are not carried in t he I GP. I f t hey w ere carried in t he I GP, t hen ev er y t im e a cust om er connect s t o t he net w or k , one / 32 addr ess w ould be inj ect ed int o t he I GP, flooded t hr ough t he I GP back bone or ar ea, and causing a sm all CPU hit . This doesn’t scale —yet m any sm all I SPs do t his w it hout t hinking. But im agine r unning t his w it h a 10,000- port net work—sev er al hundr ed connect ions per second could br ing t he net w or k t o it s knees. We have been involved in m any cr it ical n et w ork failur es caused sim ply by announcing t he ent ir e dial addr ess space as / 32s int o t he backbone I GP. The cor r ect w ay t o car r y t he access net w or k is t o inj ect t he aggr egat e host ed by t he aggr egat ion ser v er int o t he iBGP. Whet her t his m eans r unning iBGP on t he aggr egat ion ser v er so t hat it can announce t he pr efix or r unning iBGP on t he dist r ibut ion lay er and point ing appr opr iat e st at ic r out es t o t he aggr egat ion box is up t o t he I SP net w or k in quest ion. But t his is t he cor r ect and scalable m et hod em ploy ed by mo st I SPs. Som e I SPs sell ser vices t o dial cust om er s t hat include a fixed / 32 addr ess, no m at t er w her e t hey connect in t he net w or k . Alt hough t hese ar e at t r act iv e, t he ser v ice is t echnically ill conceiv ed and is v er y har d t o deliv er . I SPs t hat do offer t hese ser v ices t end t o build a dial net w or k t hat is gat ew ay ed ont o t he cor e back bone. A separ at e I GP is used in t he dial net w or k, and it has no int er act ion w it h t he cor e I GP. The dial net w or k addr ess r ange is inj ect ed int o t he I SP backbone using iBGP. I n fr a st r u c t u r e Pr e f ix e s I nfr ast r uct ur e pr efixes ar e t he only set of pr efixes t hat should appear in t he I SP I GP. I nfr ast r uct ur e pr efix es ar e t he r out er s’ loopback int er face addr esses and t he point t o- point links connect ing all t he r out er s in t he backbone. So it shoul d be v er y easy t o pr edict t he size of t he I GP—sim ply adding t he num ber of r out er s t o t he num ber of discret e links w ill give t he num ber. Ex t e r n a l Pr e f ix e s Ex t er nal pr efix es account for all ot her pr efix t y pes not cov er ed in t his sect ion. I f t he I SP is assign ing addr ess space t o cust om er s, t hese ar e ex t er nal pr efix es. I f t he I SP is hear ing pr efix es being announced fr om peer s and cust om er s by BGP, t hese ar e ex t er nal pr efix es. All ext er nal pr efixes should be car r ied in iBGP. They ar e announced t o ot her aut onom ous sy st em s using eBGP, if t hey ar e t o be announced. They should nev er appear in t he I GP.

Configuring OSPF OSPF enfor ces a fair ly r igid design for an I SP backbone. Ar ea 0 is t he backbone ar ea and m ust exist if t here are t o be m ore t han t w o OSPF areas in a net w or k. Ar ea 0

87

pr ov ides t r ansit bet w een t he ot her ar eas, and ev er y ot her ar ea m ust be connect ed t o it . OSPF offer s a m ult it ude of ar ea t y pes, including back bone, r egular , st ub, t ot ally st ubby , and not so st ubby ar eas. Most I SPs t end t o use only back bone and regular ar eas; ver y few m ake use of OSPF int er- ar ea sum m ar izat ion capabilit ies. The r eason is t hat , t hese day s, iBGP car r ies all t he pr efix es acr oss an I SP back bone. All t hat OSPF car r ies ar e t he loopback s and t he infr ast r uct ur e addr esses, and it is oft en desir able t o see t hose acr oss t he ent ir e back bone. Loopback s do need t o be v isible acr oss t he back bone because t he loopback is usually t he iBGP nex t hop for m ost of t he pr efixes car r ied in t he backbone. An iBGP next hop cannot be a default r out e cr eat ed by OSPF or BGP. Put t ing a pr efix int o OSPF is achiev ed sim ply by cr eat ing a n e t w or k st at em ent w it h t he r elev ant net w or k , inv er se m ask , and ar ea t o w hich t he net w or k should belong. So, t o put 192.168.1.0/ 24 int o OSPF, t he com m and r equir ed is sim ply t his

Router8(config-router)# network 192.168.1.0 0.0.0.255 area 0 This pr efix cannot be put int o OSPF unless it is phy sically at t ached t o t he r out er , so announcing 192.168.1.0/ 24 needs an int er face t o use an addr ess out of t his net w or k block. To find neighbors in OSPF, sim ply m ar k t he r equir ed int er face as being nonpassiv e. The I OS Soft w ar e default is t o assum e t hat all int er faces ar e act iv e. Because m ost I SP r out er s hav e lar ge num ber s of int er faces, and OSPF t ends t o r un only on int er faces connect ed t o ot her r out er s, it is r eally im por t ant t o change OSPF’s default behavior of sear ching for neighbor s on ever y int er face so t hat t hey all ar e m ar ked as passive unless r equir ed. This is achieved by using t he p a ssiv e - in t e r fa ce de fa u lt com m and. I t is r ecom m ended t hat I SPs use t his com m and by default in any configur at ion t em plat e t hat t hey cr eat e for t heir r out er s. Ot her w ise, lar ge aggr egat ion r out er s w ast e CPU sending OSPF hellos t o nonex ist ent neighbor s on all t he int er faces t hat it has. ( Wor se, if cust om er net w or k s ar e c onnect ed and t heir neighbor ing r out er is r unning OSPF, an adj acency m ight be est ablished, w r eaking hav oc in t he I SP back bone.) A t ypical r out er configur at ion m ight look som et hing like t he follow ing:

interface FastEthernet1/0 description Ethernet backbone ip address 200.200.7.129 255.255.255.224 no ip redirects no ip directed-broadcast no ip proxy-arp ip ospf message-digest-key 1 md5 7 01100F175804 no ip mroute-cache ! interface Serial1/0 description 2 Mbps Link to Router8 bandwidth 2000 ip address 200.200.7.1 255.255.255.252

88

no no no ip no no

ip redirects ip directed-broadcast ip proxy-arp ospf message-digest-key 1 md5 7 01100F175804 ip mroute-cache fair-queue

! router ospf 100 log-adjacency-changes passive-interface default passive... no passive-interface FastEthernet 0/0 no passive-interface Serial 1/0 area 0 authentication message-digest network 200.200.7.254 0.0.0.0 area 0 those... network 200.200.7.0 0.0.0.3 area 0 to be... network 200.200.7.128 0.0.0.31 area 0 ! ip ospf name-lookup commands !

! log adjacency changes ! all interfaces are ! ...except these two ! authentication ! network statements for ! ...prefixes which need ! ...in OSPF. ! DNS resolution for show

Not ice t he OSPF configur at ion, especially t he neighbor aut hent icat ion. This w ill be cover ed in Chapt er 5, “ Ope r at ional Pr act ices.” I t is v er y im por t ant t hat aut hent icat ion be used for all r out ing pr ot ocol neighbor r elat ionships. Denial- ofser vice at t acks on r out ing pr ot ocols ar e becom ing m or e com m on as t he m or e t r adit ional av enues ar e being closed up. I f an area is not phy sically connect ed t o t he back bone ar ea, OSPF pr ov ides a concept called a v ir t ual link. I SPs som et im es need t o use vir t ual links if t heir net w or ks don’t fall int o t he rigid layout required by OSPF. The virt ual link is a bridge over an int er m ediat e ar ea, connect ing t he r em ot e ar ea t o t he back bone. A configur at ion sam ple fr om a r out er using t w o v ir t ual link s t o connect t o t he cor e back bone follow s:

router ospf 100 log-adjacency-changes area 0 authentication message-digest ! virtual link goes to Area0 area 30 authentication message-digest area 30 virtual-link 222.222.7.224 message-digest-key 1 md5 7 13061E010803 area 30 virtual-link 222.222.35.224 message-digest-key 1 md5 7 00071A150754 area 40 authentication message-digest passive-interface default no passive-interface Serial 0/0 no passive-interface Serial 0/1 network 222.222.11.224 0.0.0.0 area 30 network 222.222.17.0 0.0.0.3 area 30 network 222.222.32.0 0.0.0.3 area 40 !

89

The r out er phy sically connect s Ar ea 30 t o Ar ea 40 and requires a virt ual link t o t w o r out er s in Ar ea 0 so t hat Ar ea 40 can see t he r est of t he I SP’s back bone. Ver y lit t le else is r equir ed for configur ing OSPF. As long as t he basic design r ules ar e r em em ber ed, OSPF w or k s w ell and scales v er y nicely . Ther e is no r eal lim it t o t he num ber of r out er s t hat can be in an ar ea. How ever , it is w or t h being pr udent and designing so t hat OSPF Ar ea 0 is t he backbone of t he I SP net w or k and t he subar eas ar e t he dist r ibut ion and access lay er s of t he net w or k . A t y pical configurat ion is t o use Ar ea 0 for t he nat ional back bone and for each PoP t o hav e one ar ea of it s ow n. The m ore rout ers exist in an area, or t he m ore areas a rout er is a m em ber of, t he m ore t he CPU has t o w or k. I f t he backbone is r ichly m eshed or has unst able physical connect ions, it is bet t er t o hav e few er r out er s in an ar ea and few er ar eas connect ed t o one rout er.

Configuring I S - I S IS- I S is quit e sim ilar t o OSPF in m any w ays, and bot h use t he sam e Dij kst ra SPF algor it hm for pat h calculat ion. I m plem ent at io n is slight ly different , t hough, and I S- I S suppor t in I OS Soft w ar e has benefit ed fr om m any year s of exper ience in t he m aj or I SP back bones in t he Unit ed St at es. IS- I S does not have an ar ea concept like OSPF. I nst ead, it has t w o levels: Level 1 ( ar eas) and Lev el 2 ( t he back bone) . The I S - I S backbone is sim ply a cont iguous collect ion of Lev el 2–capable r out er s link ing Lev el 1 ar eas t oget her . Most I SPs im plem ent I S- I S using Level 2 only —t hey see lit t le benefit in t he ex t r a com plex it y t hat running bot h Level 1 and Level 2 offers. A rout er can be in Level 1 only, Level 2 only, or bot h Level 1 and Level 2. I S- I S has a link- st at e dat abase for Lev el 1 and also one for Level 2. Anot her feat ure of I S- I S is t hat it does not use I P for t ransport . I nst ead, it relies on CLNS, a pr ot ocol t hat r uns on t he w ir e alongside I P. ( For t he secur it y conscious, t his m akes I S- I S har der t o at t ack because CLNS r ar ely is r out ed acr oss t he I nt er net .) To enable CLNS on a r out er , t he global configur at ion com m and cln s r ou t in g is required. As w it h OSPF, I S- I S needs t o car r y only infr ast r uct ur e addr esses; t his m eans t he point - t o- point link s for t he back bone net w or k s and t he loopback addr esses of t he net w or k equipm ent . I S - I S aut om at ically inst alls all connect ed int er faces int o t he r out ing pr ot ocol, so no n e t w or k st at em ent s are required. Also, t o m ake I S- I S find neighbor s, sim ply act iv at e I S - I S on a rout er int erface and t he neighbor w ill be found. This is m uch easier t han handling t he passive/ no- passiv e int er face and t he n e t w or k st at em ent / m ask as found in OSPF. Because I S- I S is using CLNS, each r out er r equir es a num ber called an NSAP, a num ber t hat can be bet w een 8 and 20 by t es lar ge. Each NSAP m ust be unique acr oss t he back bone. NSAPs ar e set using t he n e t st at em ent under I S- I S. Alt hough NSAPs ar e supposed t o be officially allocat ed by OSI , m ost I SPs sim ply pick a num ber t hat w or k s for t hem and use it . OSI pr ot ocols ar e not announced acr oss t he I nt er net , so t her e is no danger of a collision bet w een NSAP addr esses used in different I SPs.

90

Th e ot h er t hing t o not e when set t ing up a new I S- I S backbone is t hat w ide m et r ics should be used. The original I S- I S used narrow m et rics ( 6 bit ) , w hich allow s only 63 differ ent v alues. Wide m et r ics ar e 32 bit , obv iously giv ing consider able m or e scope and flex ibilit y . Wide m et rics should be set as t he default in any I SP t em plat e for I SI S. I S- I S has a unifor m v alue of 10 for t he link cost ; OSPF set s t he link cost based on t he bandw idt h configur ed on t he int er face. So, t o m ak e differ ent link s hav e differ ent cost s, t he I S - I S m et ric is configured m anually. Having only 6 bit s t o play w it h is v er y r est r ict iv e, especially w it h t he lar ger back bones, so t he 32- bit m et r ic m ak es m or e sense fr om t he st ar t . A configur at ion ex am ple cor r esponding t o t he init ial pr eceding OSPF ex am ple m ight be as follow s:

clns routing ! interface FastEthernet1/0 description Ethernet backbone ip address 200.200.7.129 255.255.255.224 ip router isis CORE ! activate IS-IS on this interface isis circuit-type level-2 ! Level 2 only no ip redirects no ip directed-broadcast no ip proxy-arp no ip mroute-cache ! interface Serial1/0 description 2 Mbps Link to Router8 bandwidth 2000 ip address 200.200.7.1 255.255.255.252 ip router isis CORE ! activate IS-IS on this interface isis circuit-type level-2 ! Level 2 only no ip redirects no ip directed-broadcast no ip proxy-arp no ip mroute-cache no fair-queue ! router isis CORE net 39.1234.0000.0000.0001.00 is-type level-2-only metric-style wide log-adjacency-changes ! IS- I S has sever al ot her per for m ance opt ions t hat m ake it ideally suit ed for lar ger I SP backbones. These include t he capabilit y t o r econfigur e t im er values for lar ger t opologies, t o leak Level 2 specifics int o Level 1 ar eas, and t o suppor t m esh gr oups for NBMA clouds.

91

Configur ing EI GRP EI GRP also is used quit e ext ensively in I SP backbones, finding favor w it h I SPs t hat hav e been r equir ed t o suppor t m ult iple pr ot ocols in t he past . I t ’s also accept ed t hat EI GRP is pr obably t he easiest r out ing pr ot ocol t o get st ar t ed w it h. I t found fav or ( in t he for m of I GRP) w it h I SPs t hat didn’t w ant t o use I S- I S in t he ear lier days of t he I nt er net w hen im plem ent at ions of OSPF w er e st ill not m at ur e enough for t heir needs. EI GRP has no ar ea concept —t he net w or k r uns as one lar ge I GP. EI GRP does suppor t sum m arizat ion like t he ot her t w o I GPs; in our experience, m any I SPs t hat use EI GRP m ak e quit e significant use of sum m ar izat ion acr oss t he back bone. I t is quit e com m on for t he point - t o- point link s in each PoP t o be addr essed out of one block ; t he aggr egat e for t hat block t hen is announced as a sum m ar y along t he link s leading aw ay fr om t hat PoP. The sam e design goals ar e r equir ed for EI GRP. The I GP m ust be kept as sm all as possible , so only infr ast r uct ur e pr efixes should be inj ect ed int o EI GRP. As w it h OSPF and I S- I S, t he lar ger t he I GP r out ing t able is, t he slow er conver gence becom es and t he gr eat er t he lik elihood of inst abilit y in t he back bone becom es. The configur at ion concept s ar e ver y sim ilar t o OSPF. The n e t w or k st at em ent is used t o inj ect pr efix es int o EI GRP, and int er faces need t o be m ar k ed as passiv e or not passiv e t o disable or enable EI GRP sending r out ing updat es on t hose int er faces. The pr eceding ex am ple im plem ent ed using EI GRP looks like t he follow ing:

interface FastEthernet1/0 description Ethernet backbone ip address 200.200.7.129 255.255.255.224 no ip redirects no ip directed-broadcast no ip proxy-arp no ip mroute-cache ! interface Serial1/0 description 2 Mbps Link to Router8 bandwidth 2000 ip address 200.200.7.1 255.255.255.252 no ip redirects no ip directed-broadcast no ip proxy-arp no ip mroute-cache ip summary-address eigrp 100 200.200.7.0 255.255.255.0 no fair-queue ! router eigrp 100 passive-interface default ! all interfaces are passive... no passive-interface FastEthernet 0/0 ! ...except these two no passive-interface Serial 1/0 no default-information out ! disallow distr of default info network 200.200.7.254 0.0.0.0 area 0 ! network statements for those...

92

network 200.200.7.0 0.0.0.3 area 0 be... network 200.200.7.128 0.0.0.31 area 0 no auto-summary eigrp log-neighbor-changes !

! ...prefixes that need to ! ...in EIGRP. ! log neighbor changes

EI GRP has aut osum m ar izat ion enabled by default —t his should be disabled in an I SP back bone. Aut osum m ar y , lik e BGP, aut om at ically sum m ar izes any r edist r ibut ed pr efix t o t he classful boundar y . Clea rly, t his is not required in an I nt ernet env ir onm ent t hat has been classless for m any y ear s. Also, t he r edist r ibut ion of default infor m at ion should be disabled. Ther e is r ar ely any need for t his in an I SP back bone because ex it pat hs usually ar e det er m ined by t he inform at ion carried in iBGP. EI GRP suppor t s logging of neighbor changes in addit ion t o logging of neighbor w ar nings. As w it h t he ot her t w o I GPs, logging m or e infor m at ion of pr oblem s is generally advisable in an I SP net work. Th e su m m a r y - a ddr e ss line in t he pr ev ious configur at ion for ser ial1/ 0 announces 200.200.7.0/ 24 t o t he neighbor on t hat link r at her t han announcing any subnet s. This leads t o gr eat er efficiencies in t he oper at ion of EI GRP and can help r educe t he overall size of t he backbone I GP rout ing t able.

D e sign Sum m a r y The t hr ee pr eceding subsect ions gav e br ief ex am ples of configur ing each of t he t hr ee popular I GPs t o oper at e in an I SP back bone. Quest ions oft en ar e ask ed about w hich I GP is bet t er or w hich is easier t o configur e. Hopefully t hese exam ples have show n t hat t her e is lit t le t o choose w hen it com es t o configur at ion, and t he benefit s in an I SP back bone usually com e dow n t o t he scalabilit y of each I GP and t he fam iliar it y t hat t he operat ors have w it h t hem . CAU T I O N Be sur e t o r em em ber t his one im port ant point : Keep t he size of t he I GP rout ing t able as sm all as possible!

I f I SPs follow t his caut ion, t hey w ill hav e no pr oblem scaling t he back bone. As a gener al r ule of t hum b, an I GP w it h m or e t han 5000 r out ing t able ent r ies w ill have t rouble scaling m uch fur t her , w ill hav e t r ouble dealing w it h cir cuit br eak s, and w ill dem onst r at e a significant slow dow n in conv er gence. This 5000 lim it has pr ov en t r ue in our exper ience in t he net w or ks t hat w e have been involved in —it is independent of pr ocessor pow er and is m or e r epr esent at iv e of t he t y pical back bone t opologies used by m edium- size I SPs t oday . I t is wort h bearing in m ind t hat t he largest I SPs carry m ore t han 5000 prefixes in t heir back bones. How ev er , t his is achiev ed by v er y car eful design and handcr aft ing of m any of t he I GP par am et er s. I ndeed, m ost of t he lar gest back bones in t he I nt ernet t oday are using I S- I S as t he I GP, a t est am ent t o it s long deploy m ent , good scalabilit y feat ur es, and t he oper at ional ex per ience gained since t he lat e 1980s.

93

Th e BGP Pa t h - Se le ct ion Pr oce ss [ 1 ] BGP r out er s t ypically r eceive m ult iple pat hs t o t he sam e dest inat ion. The BGP best pat h algor it hm decides w hich is t he best pat h t o act ually inst all in t he I P rout ing t able and, t her efor e, t he r out e t o use for for w ar ding t r affic. The follow ing t ex t sum m arizes t his algorit hm . For t he pur poses of under st anding t his algor it hm , begin by assum ing t hat all received pat hs for part icular prefix are ar r anged in a list , sim ilar t o t he out put of t he sh ow ip b g p pr efix com m and. Som e pat hs r eceiv ed by t he r out er ar e not consider ed candidat es for t he best pat h. Such pat hs t y pically do not hav e t he v alid flag in t he out put of sh ow ip b g p x .x .x .x. I t is im p or t ant t o under st and t hese cases befor e look ing at t he algor it hm it self. 1. I gnor e pat hs m ar ked as “ not synchr onized” in t he out put of sh ow ip b g p x . x . x . x. I f BGP sy nchr onizat ion is enabled—w hich is t he cur r ent default in I OS Soft w ar e — t here m ust be a m at ch for t he prefix in t he I P rout ing t able for an int er nal ( t hat is, iBGP) pat h t o be consider ed a v alid pat h. Most I SPs w ill w ant t o disable sy nchr onizat ion using t he n o sy n ch r on iz a t ion BGP subcom m and. 2. I gnor e pat hs for w hich t he NEXT_HOP is inaccessible. This is why it is im por t ant t o hav e an I GP r out e t o t he NEXT_HOP associat ed w it h t he pat h. 3. I gnor e pat hs fr om an eBGP neighbor if t he local AS appear s in t he AS pat h. Such pat hs ar e denied upon ingr ess int o t he r out er and ar e not even inst alled in t he BGP RI B. The sam e applies t o any pat h denied by r out ing policy im plem ent ed t hr ough access, pr efix , AS pat h, or com m unit y list s, unless inbound soft reconfigurat ion is configured for t he neighbor. 4. I f b g p b e st p a t h e n f or ce - fir st - a s is enabled and t he UPDATE does not c ont ain t he AS of t he neighbor as t he fir st AS num ber in t he AS_SEQUENCE, send a NOTI FI CATI ON and close t he session. 5. I gnor e pat hs m ar k ed as “ ( r eceiv ed- only) ” in t he out put of sh ow ip bgp x . x . x . x. This pat h has been r ej ect ed by policy but has been st or ed by t he r out er because soft- r econfigur at ion inbound has been configur ed for t he neighbor sending t he pat h. 6. I gnore pat hs wit h a next - hop m et ric m arked as inaccessible.

The BGP Best - Pat h Algorit hm for I OS Soft w are Assign t he fir st v alid pat h as t he cur r ent best pat h. Now com par e t he best pat h w it h t he next pat h in list , unt il t he end of t he list of valid pat hs is r eached. 1. Pr efer t he pat h w it h t he lar gest w eight . Not e t hat w e igh t is a Cisco specific par am et er , local t o t he r out er on w hich it is configur ed. 2. Pr efer t he pat h w it h t he lar gest LOCAL_PREF. 3. Pr efer t he pat h t hat w as locally or iginat ed t hr ough a n e t w o r k or a g g r e g a t e BGP subcom m and or t hr ough r edist r ibut ion fr om an I GP. 4. Pr efer locally sour ced net w or k / r edist r ibut ed pat hs ov er locally gener at ed aggr egat es. 5. Prefer t he pat h w it h t he shor t est AS pat h. A. This st ep is skipped if b g p b e st p a t h a s- pa t h ignor e is configur ed.

94

6. 7.

8.

9. 10.

11.

12.

B. An AS_SET count s as one AS, no m at t er how m any aut onom ous syst em s ar e in t he set . The AS_CONFED_SEQUENCE is not included in t he AS pat h lengt h. Pr efer t he pat h w it h t he low est or igin t ype: I GP is low er t han EGP, and EGP is low er t han I NCOMPLETE. Prefer t he pat h w it h t he low est MED. A. This com par ison is done only if t he fir st ( t hat is, neighbor ing) AS is t he sam e in t he t w o pat hs; any confeder at ion sub- aut onom ous sy st em s ar e ignor ed. I n ot her w or ds, MEDs ar e com par ed only if t he fir st AS in t he AS_SEQUENCE is t he sam e; any pr eceding AS_CONFED_SEQUENCE is ignor ed. B. I f b g p a lw a y s- com pa r e - m e d is enabled, MEDs ar e com par ed for all pat hs. This k nob needs t o be enabled over t he ent ir e AS. Ot her w ise, r out ing loops could occur . C. I f b g p b e st p a t h m e d con f e d is enabled, MEDs ar e com par ed for all pat hs t hat consist only of AS_CONFED_SEQUENCE ( t hat is, pat hs or iginat ed w it hin t he local confeder at ion) . D. Pat hs r eceiv ed w it h no MED are assigned a MED of 0, unless b g p be st pa t h m issin g- is- w or st is enabled, in w hich case t hey effect iv ely ar e consider ed t o hav e ( alt hough not act ually assigned) a MED of 4,294,967,295. Any r out e r eceived fr om a neighbor w it h a MED of 4 , 2 9 4 , 9 6 7 , 2 9 5 w ill hav e t he MED changed t o 4,294,967,294 befor e inser t ion int o t he BGP t able. E. BGP Det er m inist ic MED ( see t he sect ion “ BGP Det er m inist ic MED” lat er in t his chapt er) also can influence t his st ep. Pr efer t he eBGP ov er iBGP pat hs. Not e t hat pat hs cont aining AS_CONFED_SEQUENCE ar e local t o t he confeder at ion and, t her efor e, ar e t r eat ed as int er nal pat hs. Ther e is no dist inct ion bet w een confeder at ion ex t er nal and confeder at ion int ernal. Prefer t he pat h w it h t he low est I GP m et ric t o t he BGP next hop. I f m a x im um - pa t hs N is enabled and t here are m ult iple ex t er nal/ confeder at ion- ex t er nal pat hs fr om t he sam e neighbor ing AS/ sub- AS, t hen inser t up t o N m ost r ecent ly r eceiv ed pat hs in t he I P rout ing t able. This allow s eBGP m ult ipat h load sharing. The m axim um value of N is cur r ent ly 6; t he default v alue, w it h t he k nob disabled, is 1. The oldest r eceiv ed pat h is m ar k ed as t he best pat h in t he out put of sh ow ip b g p x . x . x . x, and t he equivale nt of n e x t - hop- se lf is per for m ed befor e for w ar ding t his best pat h on t o int er nal peer s. Pr efer t he pat h t hat w as r eceiv ed fir st ( t hat is, t he oldest one) . A. This st ep m inim izes r out e flapping because a new er pat h w ill not displace an older one, even if it ot her w ise w ould be select ed on account of t he addit ional decision crit eria below . I t m akes m ore sense t o apply t he addit ional decision st eps only below t o iBGP pat hs, t o ensur e a consist ent best - pat h decision w it hin t he net w or k and t her eby avoid loops. B. This st ep is skipped if b g p b e st p a t h com p a r e - r ou t e r id is enabled. C. This st ep is skipped if t he ROUTER_I D is t he sam e because t he r out er s w er e r eceived fr om t he sam e r out er . D. This st ep is skipped if t her e is no cur r ent best pat h. An exam ple of losing t he cur r ent best pat h occur s w hen t he neighbor offer ing t he pat h goes dow n. Pr efer t he r out e com ing fr om t he BGP r out er w it h t he low est r out er I D. The r out er I D is t he highest I P addr ess on t he r out er , w it h pr efer ence given t o

95

loopback int erfaces if one or m ore are configur ed. I t also can be set m anually t hr ough b g p r o u t e r- id x . x . x . x. Not e t hat if a pat h cont ains r out e r eflect or at t r ibut es, t he or iginat or I D is subst it ut ed for t he r out er I D in t he pat hselect ion pr ocess. 13. I f t he or iginat or / r out er I D is t he sam e, pr efer t he pat h w it h t he m inim um clust er I D lengt h. This w ill be pr esent in BGP r out e- r eflect or envir onm ent s only, and it allow s client s t o peer w it h r out e r eflect or s/ client s in ot her clust er s. I n t his scenar io, t he client m ust be aw ar e of t he r out e r eflect or– spec ific BGP at t r ibut es. 14. Pr efer t he pat h com ing fr om t he low est neighbor addr ess. This is t he I P addr ess used in t he BGP neighbor configur at ion, and it cor r esponds t o t he addr ess t hat t he r em ot e peer uses in t he TCP connect ion w it h t he local r out er .

BGP Fe a t u r e s a n d Com m a n ds BGP is t he hear t of t he I nt er net . I t is t he essent ial pr ot ocol t hat keeps it all glued t oget her . Yet , because t he only t hing y ou can guar ant ee on t he I nt er net is change, BGP needs consist ent updat es w it h new feat ur es and funct ionalit y t o help I SPs m anage and scale t heir net w or k s. This sect ion det ails som e of t he ear ly docum ent at ion and configur at ion not es for I SP net w or k engineer s r at her t han being a r eplacem ent for t he docum ent at ion at Cisco.c om ( ht t p: / / w w w . cisco. com / univ er cd/ hom e/ hom e. ht m) . We w ill cover t hese new feat ur es/ com m ands br iefly in t his sect ion. I SP engineer s need t o under st and w hen t o use t hese com m ands on t heir back bones. Hence, det ailed ex am ples on how t hese com m ands ar e used can be found in Using t he Bor der Gat ew ay Pr ot ocol for I nt er dom ain Rout ing ( h t t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / ics/ icsbgp4. ht m) .

St able iBGP Configurat ion An I SP’s backbone should be built using an I GP t o car r y int er nal infr ast r uct ur e addr essing and using iBGP t o car r y access net w or k s, cust om er net w or k s, and Regional Regist ry – assigned aggr egat es. Ther e is t he obv ious dist inct ion in ser v ices and funct ion, but , m or e im por t ant , I GPs conver ge fast er t han iBGP and r espond t o changes in condit ions ( physical link st at us) m or e quickly. This design is deployed by m any I SPs. A com m on er r or is t o use I P addr esses of Et her net or FDDI int er faces as t he r em ot e peer addr esses. Ther e could be no issue w hen t he r em ot e addr ess is act iv e, but as soon as t he int er face goes dow n, t he peer ing is t or n dow n because t he addr ess is no longer r eachable. Most I SP back bone designs hav e r out er s connect ed by at least t w o int er faces t o t he r est of t he net w or k . I t w ould be unfor t unat e t o hav e t he iBGP peer ing go dow n if one WAN link had gone dow n w hen, in fact , t he r out er is per fect ly accessible t hr ough anot her connect ion. The design using t w o exit pat hs per rout er should be t ak en adv ant age of. This is done by using t he loopback int er face, a “ r eal” int er face but w it hout any phy sical connect iv it y . The int er face alw ay s ex ist s, and it can be shut dow n only by t he I OS Soft ware sh u t d ow n com m and. Choosing t he loopback int er face and assigning it a host I P addr ess ( / 32) guar ant ees a m or e st able iBGP, r egar dless of t he under ly ing phy sical net w or k issues t ak en car e of by t he I GP.

96

hostname gateway1 ! interface loopback 0 ip address 215.17.1.34 255.255.255.255 ! router bgp 200 neighbor 215.17.1.35 remote-as 200 neighbor 215.17.1.35 description iBGP with CR2 neighbor 215.17.1.35 update-source loopback 0 neighbor 215.17.1.36 remote-as 200 neighbor 215.17.1.36 description iBGP with CR3 neighbor 215.17.1.36 update-source loopback 0 ! Rout er s 215.17.1.35 and 215.17.1.36 see t he BGP updat es com ing fr om addr ess 215.17.1.34 of gat ew ay 1. Lik ew ise, gat ew ay 1 hear s t he BGP updat es com ing fr om 215.17.1.35 and 215.17.1.36 ( loopback int er faces configur ed on t hose r out er s) . This configur at ion is independent of t he back bone infr ast r uct ur e and is not dependent on t he phy sical connect iv it y of t he r out er s. The loopback int er faces on an I SP net w or k usually ar e addr essed out of one block of addr ess space, w it h each loopback addr ess car r ied in t he I SP’s I GP as a / 32 addr ess. The r easons for assigning loopback int er face addr esses out of one block w ill becom e appar ent lat er in t his chapt er . Not ice t he use of t he de scr ipt ion com m and. This is new in I OS Soft w ar e Releases 11.1CC and 12.0, allow ing a descr ipt ion of t he BGP peer ing t o be ent er ed int o t he r out er configur at ion. I t s use is st r ongly encour aged because it m ak es under st anding and debugging of configur at ion and oper at ion so m uch easier. Not e t hat , for eBGP configur at ion, I P addr esses of r eal int er faces gener ally ar e used. This is because no I GP is r unning bet w een I SP back bones or aut onom ous sy st em s. eBGP peer ing r out er s have no w ay of finding out about ext er nal net w or ks, apar t fro m t he ot her end of a point - t o- point WAN link t hat w ill be linking t he t w o t oget her.

BGP Aut osum m ary I n I OS Soft w ar e, aut osum m ar izat ion is t ur ned on by default for all pr efix es t hat ar e r edist r ibut ed int o BGP fr om ot her r out ing pr ot ocols. This feat ur e aut omat ically sum m ar izes subpr efix es t o t he classful net w or k boundar ies w hen cr ossing classful net w or k boundar ies. The I nt er net r egist r ies now ar e allocat ing fr om t he for m er Class A space; an I SP t oday is m or e likely t o be allocat ed a / 18 I Pv4 addr ess fr om w hat used t o be t he Class A space. BGP’s default behavior is t o t ake t hat / 18 and adver t ise a / 8. Wit hout t he BGP com m and n o a u t osu m m a r y , BGP aut osum m ar izes t he / 18 int o a / 8. At t he least , t his w ill cause confusion on t he I nt er net , but w or se, it pot ent ially w ill at t r act ot her I SPs’ unr out able t r affic t o t he local back bone, w it h due consequences on cir cuit and sy st em s loading. Her e’s an ex am ple: An I SP is allocat ed 24.10.0.0/ 18. The I SP suballocat es t his / 18 for it s cust om er s. The I SP w ant s t o adv er t ise t he / 18 t o t he I nt er net . BGP’s default behav ior w ould be t o aut om at ically sum m ar ize t he / 18 int o t he classful boundar y ,

97

24.0.0.0/ 8, t he old Class A addr ess. The pr oblem is t hat ot her I SPs ar e also get t ing / 18 allocat ions fr om t he I Pv4 Regist r y. I n t oday’s classle ss I nt er net w or ld, in w hich t he for m er Class A space is being efficient ly suballocat ed, I SP and ent er pr ise back bones t hat use BGP and r edist r ibut e pr efix es fr om ot her r out ing pr ot ocols hav e t o use n o a u t o- su m m a r y . I t should be not ed t hat redist ribut ion int o BGP for any I SP backbone is st r ongly discour aged! Ther efor e, I SPs t hat ar e not doing r edist r ibut ion can safely om it t he no a u t o- su m m a r y com m and. How ev er , as w it h ev er y t hing else, it is good pr act ice t o include t he com m and; it pr ev ent s fut ur e accident s.

BGP Synchronizat ion The ( hist or ical) default in I OS Soft w ar e is for BGP not t o adver t ise a r out e unt il all r out er s w it hin t he AS hav e lear ned about t he r out e t hr ough an I GP. I n t oday ’s I nt er net , and cer t ainly since t he m id- 1990s, I SPs have designed t heir net w or k s so t hat t he iBGP car r ies cust om er and I nt er net pr efix es and t he I GP car r ies t he infr ast r uct ur e addr esses. I t is v er y uncom m on for a pr efix t o appear bot h because of an I GP and because of iBGP. So sy nchr onizat ion m ust be t ur ned off for all I SP configur at ions. To do t his, use t his BGP configur at ion n o sy n ch r o n iz a t io n com m and.

BGP Com m unit y Form at Many I SPs m ak e ex t ensiv e use of BGP com m unit ies for r out ing policy decision m aking. A BGP com m unit y t ut or ial is beyond t he scope of t his book, but I SP engineer s should be aw ar e of t he t w o for m at s suppor t ed in I OS Soft w ar e. The or iginal for m at for a com m unit y num ber t ook t he for m of a 32- bit int eger . Mor e r ecent ly , t he r epr esent at ion of a com m unit y w as r edefined in t he I nt er net space as being t w o 16- bit int egers separ at ed by a colon, per t he BGP st andar d. The fir st 16bit num ber is accept ed as being t he I SP’s AS ( because com m unit y num ber s ar e ex por t able bet w een aut onom ous sy st em s w it h t he se n d- com m u n it y BGP dir ect iv e) . The second 16- bit num ber is used t o r epr esent differ ent policies t hat t he I SP w ant s t o im plem ent . Not e t hat som e st andar d com m unit ies ar e defined as com m on or cur r ent pr act ice; t hese ar e docum ent ed in RFC 1998. The configur at ion com m and is t his

ip bgp-community new-format This com m and conv er t s comm unit ies fr om look ing som et hing lik e 13107210 t o a m or e hum an- fr iendly 200: 10. I t should be not ed t hat BGP does not car e w hich form at is used int ernally; t he field is st ill 32 bit s t ot al. This new form at is purely for appear ance and pr act icalit y pur poses.

98

BGP N eighbor Shut dow n A new feat ur e int r oduced int o 11.1CC and 12.0 soft w ar e is t he capabilit y t o shut dow n a BGP peer ing w it hout act ually r em ov ing t he configur at ion. Pr ev iously , t he only w ay t o disable a BGP peer ing w as t o delet e t he configur at ion fr om t he r out er . This w as v er y disr upt iv e t o t he r out er ’s funct ioning, and it significant ly incr eased t he lik elihood of m ak ing m ist ak es w hen r einst at ing t he configur at ion at a lat er st age. A neighbor ing peer ing is shut dow n w it h t his com m and ex am ple:

router bgp 200 neighbor 169.223.10.1 shutdown To r einst at e t he peer ing w hen t he pr oblem or r eason for shut dow n has been r em ov ed, sim ply ent er t he opposit e com m and:

router bgp 200 no neighbor 169.223.10.1 shutdown Not ice t hat t he com m and is n o n e ig h b or 1 6 9 .2 2 3 . 1 0 .1 sh u t d ow n , w it h t h e no com ing first inst ead of t he possibly m ore int uit ive n e ig h b or 1 6 9 .2 2 3 .1 0 .1 n o sh u t d ow n. Be ver y aw ar e of t his because it is possible t o for get or m ist ype t he sh u t d ow n por t ion of t he com m and, t hus delet ing t he w hole BGP configur at ion for t hat neighbor. All user s of BGP ar e encour aged t o use neighbor shut dow n r at her t han delet ing t he configur at ion—it gr eat ly enhances t he ease and r eliable oper at ion of t he net w or k.

BGP Dynam ic Reconfigurat ion Tw o m et hods ar e av ailable now t o dy nam ically r eset a BGP peer ing session w it hout t ear ing dow n t he ent ir e peer ing. Nor m ally , w hen an I SP r equir es changing t he policy in a BGP peer ing, t he peer ing it self has t o be t or n dow n so t hat t he new policy can be im plem ent ed. For peer ings ex changing a lar ge num ber of r out es in t he I nt er net , t his can be ext r em ely disr upt ive, put t ing load on t he CPU of bot h r out er s involved and r esult ing in a r out ing flap t hr ough t he back bone as t he I SP’s net w or k announcem ent s ar e w it hdr aw n and t hen r einst at ed. The first and older m et hod is soft r econfigur at ion; t he second and st r ongly r ecom m ended m et hod is r out e r efr esh. I t is now consider ed best cur r ent pr act ice for I SPs t hat hav e ex t er nal BGP peer ings t o use dy nam ic r econfigur at ion. The im pact on t heir net w or k s, t heir cust om ers’ per ceiv ed ser v ice qualit y , and t he I nt er net is t oo gr eat w it hout t his feat ur e. Consider t he im pact and effect s in t he follow ing sect ion discussing BGP r out e flap dam ping. Also, it is r ecom m ended t o discour age all oper at ions st aff fr om doing a har d clear of BGP unless it is an absolut e last r esor t . Sof t Re con f ig u r a t ion

99

BGP soft r econfigur at ion capabilit y w as int r oduced in I OS Soft w ar e Releases 11.1CC, 11.2P, and 12.0. Wit h t his feat ur e enabled, aft er t he policy changes hav e been m ade, t he I SP sim ply can im plem ent a soft r eset of t he peer ing w it hout hav ing t o t ear it dow n. To suppor t soft r econfigur at ion on a peer ing, t he r out er r equir es one ex t r a configur at ion com m and, for ex am ple:

router bgp 200 neighbor 215.17.3.1 neighbor 215.17.3.1 neighbor 215.17.3.1 neighbor 215.17.3.1 !

remote-as 210 soft-reconfiguration in route-map in-filter in route-map out-filter out

I f t he policy on t he peer ing m ust be changed, t he I SP m ak es t he changes t o t he r out e- m ap configur at ion in t he pr evious exam ple and t hen sim ply issues t he com m and cle a r ip b g p n e ig h b or 2 1 5 .1 7 .3 .1 sof t . I f only t he inbound or out bound policy needs t o be changed, t he cle a r com m and can be supplem ent ed w it h in or out dir ect iv es. Not ice t he pr eceding configur at ion. Soft r econfigur at ion is r equir e d only inbound; out bound does not need t o be ex plicit ly configur ed because, t o m ak e out bound policy configur at ion changes, t he BGP pr ocess sim ply has t o send an incr em ent al updat e t o it s neighbor. One cav eat is t hat soft r econfigur at ion can r equir e a lot mor e r out er m em or y because t he r out er m ust st or e pr efix es t hat it has r eceiv ed befor e t he BGP inbound policy is im plem ent ed. I f t he inbound policy is com plex or t her e ar e m ult iple peer ings, it is possible t hat alm ost t w ice t he am ount of m em or y w ill be r equir ed by t he BGP pr ocess t han if soft r econfigur at ion w as not configur ed. Rou t e Re f r e sh A new feat ur e available fr om I OS Soft w ar e Release 12.0( 5) S is r out e r efr esh ( docum ent ed in RFC 2918) . The concept is sim ilar t o soft r econfigur at ion, but t his is a capabilit y shar ed bet w een t w o BGP speak er s ( as opposed t o soft r econfigur at ion, w hich is configur ed on t he local r out er only ) and it is negot iat ed aut om at ically at t he t im e t he BGP session is br ought up. To find out w het her r out e r efr esh is suppor t ed, check t he BGP neighbor using t he follow ing com m and:

alpha>sh ip bgp neighbors 192.168.4.130 BGP neighbor is 192.168.4.130, remote AS 2830, external link Index 1, Offset 0, Mask 0x2 Community attribute sent to this neighbor BGP version 4, remote router ID 192.168.11.1 BGP state = Established, table version = 207, up for 16w1d Last read 00:00:01, last send 00:00:08 Hold time 30, keepalive interval 10 seconds Configured hold time is 30, keepalive interval is 10 seconds Default information originate Unicast default sent, multicast default not sent Neighbor NLRI negotiation:

100

Configured for unicast routes only Peer negotiated unicast routes only Exchanging unicast routes only Received route refresh capability(new) from peer ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Minimum time between advertisement runs is 30 seconds Received 1648106 messages, 0 notifications, 0 in queue Sent 1648064 messages, 0 notifications, 0 in queue Prefix advertised 125, suppressed 1, withdrawn 66 Route refresh request: received 0, sent 1 Connections established 2; dropped 1 Last reset 16w1d, due to Peer closed the session Number of unicast/multicast prefixes received 3/0 The por t ion highlight ed w it h ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ show s t hat t he r out e r efr esh capabilit y has been negot iat ed bet w een t he t w o BGP neighbor s. The “ ( new ) ” in t he ca pa bilit y st at em ent indicat es t hat t he r out er s suppor t t he I ANA - assigned rout e r efr esh capabilit y code ( w hich has a value of 2, descr ibed in RFC 2918) r at her t han t h e Cisco- specific code w hen t he feat ur e fir st w as dev eloped by Cisco ( w hich w ould show up as “ ( old) ” ) . ( All t he I ANA- assigned BGP capabilit y codes ar e list ed at h t t p: / / w w w . iana. or g/ assignm ent s/ capabilit y- codes. ) I f t he local r out er r equir es a fr esh v iew of t he r out ing t able, it can send a r out er efr esh r equest t o t he neighbor ing BGP peer . This w ould be r equir ed, for exam ple, w hen t he inbound r out ing policy has been c hanged. Upon r eceipt of t he r out e- refresh r equest , t he r em ot e r out er w ould send it s list of pr efix es t o t he r equest ing r out er . The r out e r efr esh capabilit y r equir es no ex t r a m em or y on t he local r out er . Wher e t he capabilit y ex ist s bet w een speak er s, it is st r ongly r ecom m ended t hat t his is chosen ov er soft r econfigur at ion ( because t he lat t er r equir es m or e m em or y t o st or e t he inbound pr efix es r eceiv ed fr om t he r em ot e peer ) . To r equest a r out e r efr esh inbound, use t his com m and:

clear ip bgp [neighbor] in No ot her configur at ion is r equir ed. To r eset t he BGP session out bound using r out e r efr esh, sim ply use t his com m and:

clear ip bgp [neighbor] out

BGP Rout e Reflect ors and t he BGP Clust er I D One m echanism t hat is available for scaling t he iBGP m esh is t o set up a r out e r eflect or clust er- based sy st em acr oss t he I SP back bone. A r out e- r eflect or clust er t ypically is m ade up of one or m ore rout ers as t he reflect or, w it h t he rem aining r out er s in t he clust er configur ed as client s. These client s need t o peer only w it h t he reflect or, not any ot her rout er in t he iBGP m esh. A t ypical exam ple is t hat show n in Figur e 3- 2. Fig u r e 3- 2 . BGP Rou t e - Re f le ct or Clu st e r

101

How ev er , m ost I SPs choose t o im plem ent clust er s w it h t w o r out e r eflect or s, as in Figure 3- 3. This giv es t hem r edundancy in t he clust er if one r out e r eflect or fails. Fig u r e 3- 3 . BGP Rou t e - Re f le ct or Clu st e r w it h Tw o Rou t e Re f le ct or s

Net wor k designer s should be aw ar e of som e cav eat s w hen configur ing r out e r eflect or s. As soon as a r out er is configur ed as a r out e r eflect or , it is assigned a clust er ident ifier aut om at ically by t he BGP pr ocess. This clust er I D is t he BGP r out er I D, usually t he loopback int er face addr ess of t he r out er . The goal of a clust er I D is only t o r educe t he pr opagat ion of BGP updat es bet w een r out e r eflect or s w hen t he sam e client uses m ult iple r out e r eflect or s. A r out e r eflect or w ill not accept an updat e com ing fr om anot her rout e reflect or t hat has t he sam e I D ( t he I D is st ored in t he clu st e r- list at t r ibut e) . This is because t his updat e has been r eceiv ed alr eady fr om t he client ( because t he client peer s w it h all r out e r eflect or s of t he clust er ) . Now , w hat if t he clust er I D in bot h r out e r eflect or s isn’t t he sam e? I n t hat case, a client connect ed t o t w o r out e r eflect or s sends it s updat e t o bot h, and each r out e r eflect or sends t his updat e t o t he ot her . The r esult is t hat t hey r eceiv e t he sam e updat e t w ice. Now r out e r eflect ors act as BGP r out er s. When t hey discov er t hat t he updat e is ex act ly t he sam e, t hey select j ust one ( t he one w it h t he shor t est clust er list ) and pr opagat e t hat one t o ot her neighbor s. Ther e w ill be no r out ing loops because w e ar e t alk ing about ident ical updat es: sam e net / m ask , sam e nex t hop. Som e I SPs choose t o change t he default clust er I D t o v alues t hat let t hem m or e easily ident ify r egions/ clust er s in t he back bone. And quit e oft en t hey set t he clust er

102

I D of t he r out e r eflect or s in each PoP t o be t he sam e v alue. They should do so only wit h t he purpose of t he clust er I D kept in m ind: • •

Each client m ust peer w it h bot h r out e r eflect or s. Failur e t o do so could r esult in r out ing loops. The phy sical pat h fr om t he client t o t he r out e r eflect or m ust not be t hrough anot her r out e r eflect or in t he clust er . Failur e t o adher e t o t his w ill r esult in a rout ing black hole.

I s t he overhead of one ext ra rout ing updat e w ort h sacrificing for an increased risk in r out ing black holes because of net w or k t opology ? Not e t he last point : I t seem s t o be accept ed w isdom and, indeed, is oft en docum ent ed t hat t he clust er I D should be t he sam e on bot h r out e r eflect or s in a clust er . This is pot ent ially danger ous and could cause m or e pr oblem s t han leaving t he st at us quo. ( One ex am ple m ight be t he sit uat ion in w hich one reflect or is phy sically r eachable fr om t he client only t hr ough t he ot her r eflect or . The best pat h could be v alid t hr ough t he clust er I D but could be unr eachable.) Finally , t her e is lit t le w r ong w it h hav ing ov er lapping r out e- reflect or clust er s, as show n in Figur e 3- 4. Many I SPs t hat hav e inst alled a r out e- r eflect or hier ar chy acr oss t heir back bones hav e done so v er y successfully sim ply using t he default I OS Soft w ar e v alues for t he clust er I D. The ear ly I OS Soft w ar e r est r ict ion of a r out er eflect or client being able t o belong t o only one r out e r eflect or now has been r em ov ed, so set t ing up client s t o be m em ber s of t w o clust er s is com m on pr act ice a cross t he I nt er net . I f in doubt , t he quest ion t o ask is t his: “ I s t he over head of one ext r a r out ing updat e w or t h sacr ificing for an incr eased r isk in r out ing black holes because of net w or k t opology?” Net w or k t opology changes on a m ont hly or m or e fr equent basis in an I SP backbone, so w hat m ight have been a perfect design on t he fir st day pot ent ially could becom e a liabilit y lat er . Fig u r e 3- 4 . Ov e r la p p in g Rou t e - Re fle ct or Clu st e r s

n e x t -h op- se lf The BGP com m and n e x t - h o p- se lf sim ply does as it suggest s. The next - hop announcem ent s fr om t his r out er ar e set t o t he local r out er I D r at her t han t he I P

103

addr ess of t he or igin of t he pr efix. To configur e n e x t - hop- se lf, use t he follow ing BGP configur at ion:

router bgp 200 neighbor 215.17.3.1 remote-as 210 neighbor 215.17.3.1 next-hop-self ! Tw o com m on im plem ent at ion ex am ples ar e giv en in t he follow ing sect ions. The fir st deals w it h ex t er nal connect ions, such as t hose t o ot her net w orks or I nt ernet ex change point s; t he second deals w it h aggr egat ion r out er s, in w hich cust om er link s ar e t er m inat ed ont o t he I SP back bone. Ex t e r n a l Con n e ct ion s I f a pr efix is hear d fr om an ext er nal net w or k, t he next hop is pr eser ved t hr oughou t t he I GP. How ever, set t ing n e x t - hop- se lf on t he bor der r out er t hat is dist r ibut ing t he eBGP prefix int o iBGP m eans t hat t he ext ernal prefix w ill receive t he rout er I D ( loopback int er face I P addr ess) of t he bor der r out er r at her t han t he ext er nal next hop. Sever al I SPs choose t o do t his r at her t han t r anspor t t he ext er nal point - t o- point links acr oss t he back bone. I SPs w it h a lar ge num ber of BGP cust om er s or ex t er nal pr iv at e peer s w ill use t his m echanism—each peer pot ent ially can add anot her / 30 t o t he I GP acr oss t he back bone. Rem ov ing t hese ex t r a / 30s helps t o k eep t he I GP t able as sm all as possible. This feat ur e also is used especially by I SPs at ex change point s t o ensur e consist ency and r eliabilit y of connect ions acr oss t he ex change. I ndeed, it is considered a st r ongly r ecom m ended pr act ice because it st ops t he pot ent ially fr audulent act iv it y of st ealing bandw idt h at t he exchange point . I f t he I SP is car r ying t he exchange- point LAN addr ess acr oss it s backbone in t he I GP ( or in iBGP) , t hen it is announcing reachabilit y of t he ex change acr oss t he ent ir e net w or k . So a peer ing cust om er ( usually I SP) t hat is also pr esent at t he ex change point could send t r affic t o t his I SP and hav e it successfully deliv er ed t o t he ex change point using t he I SP’s infr ast r uct ur e r at her t han his ow n poor er connect ion. A sam ple configur at ion using a GRE t unnel, w hich could be used by such a per pet r at or , follow s. Figur e 3- 5 giv es a pict ur e of how t his m ight be done. Fig u r e 3- 5 . GRE Tu n n e ls a t I X Ps/ N APs

104

Router D: interface tunnel 0 ip address 221.0.1.1 255.255.255.252 tunnel source 220.0.0.2 tunnel destination 169.223.0.2 ! ip route 169.223.0.2 255.255.255.255 220.0.0.1 Router B: interface tunnel 0 ip address 221.0.1.2 255.255.255.252 tunnel source 169.223.0.2 tunnel destination 220.0.0.2 ! ip route 220.0.0.2 255.255.255.255 169.223.0.1 The GRE t unnel is used bot h t o defeat r ev er se pat h for w ar ding check s t hat AS 109 m ay be car r ying out and t o hide t he t r affic fr om AS 109. Rever se pat h for w ar ding checks are covered in m ore det ail in Chapt er 4. Hopefully , t his sect ion has dem onst r at ed how im por t ant t he n e x t - hop- se lf BGP set t ing is. I f t he I SP in AS 109 had used n e x t - hop- se lf for t he I XP LAN rat her t han car r y ing 169.223.0/ 24 acr oss it s back bone, t he cust om er in AS 2830 w ould not have been able t o const r uct t he GRE t unnel as point ing t he st at ic r out e fr om Rout er D t o Rout er C. Rout er C w ould not k now how t o get t o t he 169.223.0/ 24 net w or k , so all t he t r affic w ould hav e been dr opped on t he floor . The sam e goes for Rout er B: I f Rout er B does not k now how t o get t o t he 220.0.0.0/ 30 point - t o- point link, t he t unnel again could not have been built . I n fact , it is very com m on for an I SP rout er at an I XP t o car r y not hing m or e t han t he I SP’s int er nal net w or k pr efix es, car r y ing a default r out e is equiv alent t o say ing t hat t he r out er k now s how t o get t o t he ent ir e I nt er net . Aggr e ga t ion Rou t e r s

105

I f a pr efix is inj ect ed int o t he iBGP at a gat ew ay r out er ( t he st andar d w ay of inj ect ing a cust om er prefix int o t he iBGP) , t he next - hop addr ess is t he I P addr ess of t he point t o- point link bet w een t he gat ew ay aggr egat ion r out er and t he cust om er . This m eans t hat t he iBGP has a large num ber of next - hop addr esses t o r esolv e fr om t he I GP ( not bad in it self) and t hat t he I GP w ill be lar ger , r esult ing in slow er conv er gence and great er pot ent ial inst abilit y in case of inst abilit y or failures in t he net w ork. The obvious solut ion t o t his is t o use I P unnum ber ed. The next - hop address in t hat case will be t he rout er I D. However, m any I S Ps t hat st ar t ed off using I P addr esses on point - t o- point links have a lot of w or k t o do t o m igr at e t o I P unnum ber ed on all t heir cust om er point - t o- point connect ions. A quick er fix , w hich let s t hem scale t he I GP but does not r equir e undue hast e ( t hat is, panic! ) in r enum ber ing lar ge num ber s of cust om er int er faces is t o set n e x t - hop- se lf on t he iBGP peer s of t he gat ew ay r out er . The cust om er pr efixes t hen w ill appear in t he iBGP w it h t he loopback addr ess of t he aggr egat ion r out er , sim plify ing t he r out e look ups and, m or e im por t ant , m eaning t hat t he I GP no longer has t o car r y all t he point - t o- point link addresses used for cust om er connect ions. ( This also assum es t hat t he point - t o- point link addr esses have been assigned in a block per r out er and ar e t r anspor t ed by t he iBGP. I f t he point - t o- point addresses are not in t he iBGP, t he cust om er point - t o- point link w ill not be pingable, w hich could be an unaccept able sit uat ion for pr ov ider s t hat m onit or t he st at e of t heir cust om er link s.)

BGP Rout e Fla p Da m ping Rout e flap dam ping ( int r oduced in Cisco I OS Soft w ar e at Release 11.0) is a m echanism for m inim izing t he inst abilit y caused by r out e flapping. A r out e flap occur s w hen a BGP net w or k pr efix is w it hdr aw n and r eannounced; specifically, t his happens w hen a BGP speaker hear s a WI THDRAW follow ed by an UPDATE for a pr efix . ( A peer ing w it h an eBGP neighbor being r eset does not count as a flap.) Whenever a net w or k goes dow n, t he r est of t he I nt er net is t old about it . Hence, BGP pr opagat es t his st at e change t hr oughout t he I nt ernet ( unless, of cour se, I SPs in t he pat h do som et hing t o block t his pr opagat ion t hr ough r out e filt er s or som et hing sim ilar ) . I f t he st at e change is caused by fault y cir cuit s ( fr equent ly going up and dow n) or fr om m isconfigur ed r out ing ( r edist r ibut ing t he I GP int o t he EGP) , t he I nt ernet w ould ex per ience sev er al hundr ed BGP st at e changes a second. For ev er y st at e change, BGP m ust allocat e t im e t o pr ocess t he w or k and pass on t he changes t o all ot her BGP neighbor s. This places unnecessar y ex t r a st r ain on t he back bone r out er s r ecom put ing best pat hs and pr opagat ing t hese changes. Hence, t he t ool t o cont r ol and m inim ize t he effect of rout e flaps is BGP dam ping. Com m a n d Sy n t a x The follow ing ar e t he com m ands used t o cont r ol r out e dam ping:

bgp dampening [[route-map map-name] | [half-life-time reuse-value suppress-value maximum-suppress-time]]

106

• • • • • •

half - life- t im e— Has a range of 1 t o 45 m inut es; current default is 15 m inut es. r euse- value— Has a range of 1 t o 20,000; default is 750. suppr ess- v alue— Has a r ange of 1 t o 20,000; default is 2000. m ax- suppr ess- t im e— Giv es t he m ax im um dur at ion t hat a r out e can be suppressed. I t s range is 1 t o 255; t he default is four t im es t he half - life t im e ( 60 m inut es) . sh ow ip bgp da m pe n e d- pa t h s — Display s all t he dam ped r out es, w it h t he t im e r em aining t o unsuppr ess. This is ver y useful for finding out w hich sit es ar e hav ing inst abilit y pr oblem s. cle a r ip b g p d a m p e n in g [ a d d r e ss m a sk ] — Clear s t he dam ping r elat ed infor m at ion. This also unsuppr esses t he suppr essed r out es and is a v er y useful t ool w hen one of y our cust om er s calls about an unr eachable net w or k t hat has been suppr essed.

A r out e m ap can be associat ed w it h BGP dam ping t o select ively apply t he dam ping par am et er s if cer t ain cr it er ia ar e found. Ex am ple select iv e dam ping cr it er ia include m at ching on t he follow ing: • • •

A specific I P r out e An AS pat h A BGP com m unit y

Adj ust ing t he dam ping t im er s becom es essent ial w hen adm inist r at or s cannot affor d t o have a long out age for a specific r out e. BGP dam ping w it h r out e m aps is a pow er ful t ool t o select ively penalize ill- behav ed r out es in a user- configurable and cont rolled m anner. Recom m ended r out e flap dam ping par am et er s for use in t he I nt er net w er e com bined in a docum ent by t he RI PE Rout ing Working Group ( rout in g- [email protected] ) and are av ailable at ht t p: / / w w w .r ipe.net / docs/ r ipe- 229.ht m l. These values are used by m any Eur opean and U.S. I SPs and ar e based on t he oper at ional ex per ienced gained in t he indust r y . Det ailed ex am ples of BGP dam ping t echniques w it h r out e m aps can be found in t he book I nt er net Rout ing Ar chit ect ur es, Second Edit ion, by Sam Halabi and Danny McPher son. How ev er , it is v er y unusual for I SPs t o use m or e t han t he default I OS Soft w are set t ing or t he set t ings cont ained in RI PE- 229. I m p le m e n t a t ion For each flap, a penalt y ( 1000) is applied t o t he r out e. For an at t r ibut e change, a penalt y of 500 is applied t o t he r out e. As soon as t he penalt y ex ceeds t he suppr ess lim it , t he adver t isem ent of t he r out e is suppr essed. The penalt y is ex ponent ially decayed based on a pr econfigur ed half - life t im e. When t he penalt y decr eases below t he r euse lim it , it is unsuppr essed. A pict or ial descr ipt ion of flap dam ping is giv en in Figur e 3- 6. Fig u r e 3- 6 . BGP Rou t e Fla p D a m p in g

107

The r out es ex t er nal t o an AS lear ned t hr ough iBGP w ill not be dam ped. This is t o av oid t he iBGP peer s hav ing higher penalt y for r out es ex t er nal t o t he AS. I t also ensur es t hat only dir ect ly connect ed eBGP peer s w ill have r out e flap dam ping applied t o t hem , ev en t hough t he configur at ion on t he rout er is applied on a global basis. Any rout e t hat flaps at a rat e m ore t han t he half - life t im e event ually w ill be suppr essed. The penalt y is decay ed at a gr anular it y of 5 seconds, and t he ent r ies ar e unsuppr essed w it h t he gr anular it y of 10 seconds. Fur t her m or e, w hen t he penalt y is less t han half of t he r euse lim it , t he dam ping infor m at ion is pur ged. The m ax im um suppr ess t im e is t he final im por t ant par am et er used for configur ing r out e flap dam ping. I t is t he m axim um t im e t hat a pr efix can be suppr essed, an d t he ot her t hr ee par am et er s m ust be chosen accor dingly. I t is also a key par am et er in t he soft w ar e im plem ent at ion for flap dam ping ( see RFC 2439 for a det ailed discussion) . I f t he half - life t im e is 15 m inut es and t he m axim um suppress t im e is 60 m inut es, t he r euse lim it is used t o com put e w hat t he m ax im um possible penalt y can be. All r out eflap penalt ies cannot exceed t his m axim um value. I t is m ost im por t ant for t he suppr ess lim it t o be less t han t he m ax im um possible v alue t hat t he r out e flap penalt y can be; ot her w ise, no pr efix es w ill be suppr essed. Par am et er s chosen should be check ed t o ensur e t hat t hey achiev e t he desir ed effect . ( A m ax im um is set for t he v alue of t he penalt y so t hat if a pr efix is oscillat ing fr equent ly , t he r esult ant penalt y at t ach ed t o it doesn’t gener at e a huge num ber and, t her efor e, becom e unlikely ever t o be r eused.) The m at hem at ical equat ion m ight be as show n in Figur e 3- 7. Fig u r e 3- 7 . Ca lcu la t in g t h e M a x im u m Possib le Pe n a lt y

108

For t he I OS Soft w ar e default configur at ion of b g p d a m p e n in g 1 5 7 5 0 2 0 0 0 6 0 , t his w ould m ake t he m axim um possible penalt y of 12,000. So, even if t he pr efix car r ies on flapping aft er t he penalt y has r eached 12,000, t he penalt y w ill not be incr eased any m or e. These ar e som e exam ples of good and bad flap- dam ping par am et er s: •

• •

•

b g p d a m p e n in g 3 0 2 0 0 0 3 0 0 0 6 0 — These v alues ar e v alid. The m axim um suppress t im e is 60 m inut es, t he reuse lim it is 2000, and t he m axim um possible penalt y is 8000. A suppr ess lim it of 3000 m eans t hat penalt ies easily can ex ceed t he suppr ess lim it . ( A m ax im um possible penalt y of 8000 decay s t o 4000 aft er one half - life [ 30 m inut es] and t o 2000 aft er t w o half - lives [ 60 m inut es] . ) b g p d a m p e n in g 1 5 7 5 0 3 0 0 0 4 5 — These values are valid. The m axim um suppress t im e is 45 m inut es, t he reuse lim it is 750, and t he m axim um possible penalt y is 6000. The suppr ess lim it of 3000 is easily reachable. b g p d a m p e n in g 1 5 5 0 0 2 5 0 0 3 0 — These values ar e illegal and w on’t give r ise t o any pr efixes being suppr essed because of flaps. The m axim um suppress t im e is 30 m inut es, so t o reach t he reuse lim it of 500 wit h a half - life t im e of 15 m inut es, t he penalt y m ust r each 2000. How ever , t he suppr ess lim it has been set t o 2500, so no pr efix es w ill be suppr essed. b g p d a m p e n in g 3 0 7 5 0 3 0 0 0 6 0 — These values ar e valid but w on’t give rise t o any prefixes being suppressed because of flaps. The m a xim um suppress t im e is 60 m inut es, so t o reach t he reuse lim it of 750 wit h a half - life t im e of 30 m inut es, t he suppr ess lim it m ust be 3000. How ever , pr efixes ar e suppr essed only aft er t he penalt y r eaches 3000. Because t he penalt y is adj ust ed ev er y fiv e sec onds, t he pr efix w ill be suppr essed only if an updat e ar r iv es w it hin fiv e seconds of t he w it hdr aw al. This is ex t r em ely unlik ely , so no pr efix es w ill be suppr essed w it h t his ex am ple.

D e sig n in g Fla p D a m p in g Pa r a m e t e r s I t is im por t ant t o not e t hat pr efix es are suppr essed only w hen t hey ar e hear d fr om a neighbor and hav e a flap hist or y and a penalt y v alue t hat w ill cause t hem t o be suppr essed. A pr efix at t r act s a penalt y of 1000 w hen t he WI THDRAW is hear d fr om t he neighbor ing eBGP peer . How ever , because t he pr e fix has been w it hdr aw n, it is not pr esent in t he BGP t able apar t fr om having a hist or y ent r y, so it w ill not be dam ped. The penalt y w ill decay , how ev er . When t he pr efix is r eannounced by t he neighbor , if t he penalt y is st ill above t he suppr ess lim it , t he pr efix w ill be suppr essed. I f t he penalt y is below t he suppr ess lim it , t he pr efix w ill not be suppr essed. I t is im por t ant t o r em em ber t his w hen designing flap- dam ping par am et er s. The m ax im um possible penalt y should be set consider ably lar ger t han t he suppr e ss lim it ; ot her w ise, t her e is t he possibilit y of a slow ly oscillat ing pr efix not being suppr essed as int ended. For exam ple, b g p d a m p e n in g 3 0 7 5 1 3 0 0 0 6 0 w ill gener at e a m ax im um possible penalt y of 3004, w hich is only four higher t han t he suppr ess lim it . The fast est t hat t he prefix w ould reappear in t he BGP t able is one m inut e lat er t he next t im e t he BGP pr ocess r uns, and t hat int er v ening t im e could hav e at t r act ed ar ound 100 point s of decay , t her eby br inging t he penalt y below 3000 and not causing any dam ping t o

109

happen. I f b g p d a m p e n in g 3 0 8 0 0 3 0 0 0 6 0 w as used, t he penalt y w ould st ill be abov e 3000 and t he pr efix w ould be suppr essed as int ended. BGP Fla p St a t ist ics I t is possible t o m onit or t he flaps of all t he pat hs t hat ar e flapping. The st at ist ics w ill be lost w hen t he r out e is not suppr essed and st able for at least one half - life t im e. The display looks like t he follow ing:

cerdiwen#sh ip bgp neighbors 171.69.232.56 flap-statistics BGP table version is 18, local router ID is 172.19.82.53 Status codes: s suppressed, d damped, h history, * valid, > best, i internal Origin codes: i - IGP, e - EGP, ? - incomplete Network *> 5.0.0.0 *> 6.0.0.0

From 171.69.232.56 171.69.232.56

Flaps Duration Sup-time Path 1 0:02:21 300 2 0:03:21 300

The follow ing ar e t he new com m ands t hat display flap st at ist ics: • • • • • •

sh ow ip b g p f la p- st a t ist ics— Display s flap st at ist ics for all t he pat hs sh ow ip b g p f la p- st a t ist ics r e g e x p [ r e g e x p ] — Display s flap st at ist ics for all pat hs t hat m at ch t he r egular expr ession sh ow ip b g p f la p- st a t ist ics f ilt e r- list [ list ] — Display s flap st at ist ics for all pat hs t hat pass t he filt er sh ow ip b g p f la p- st a t ist ics x . x . x . x [ m . m . m . m ]— Display s flap st at ist ics for a single ent ry sh o w ip b g p f la p- st a t ist ics x . x . x . x m . m . m . m lon g e r- prefix — Displays flap st at ist ics for m or e specific ent r ies sh ow ip b g p n e ig h b or x .x .x .x f la p - st a t ist ics— Display s flap st at ist ics for all pat hs fr om a neighbor

N OTE Because w e m aint ain infor m at ion about only one pat h for a neighbor , t he sh ow ip bgp f la p- st a t ist ics neighbor could show a differ ent pat h for t he sam e net w or k lay er r eachabilit y infor m at ion ( NLRI ) .

The follow ing com m ands could be used t o clear t he flap st at ist ics: • • • •

cle a r ip bgp fla p- st a t ist ics— Clear s flap st at ist ics for all r out es cle a r ip bgp fla p- st a t ist ics r e g e x p [ r e g ] — Clear s flap st at ist ics for all t he pat hs t hat m at ch t he r egular expr ession cle a r ip bgp fla p- st a t ist ics f ilt e r- list [ list ] — Clear s flap st at ist ics for all t he pat hs t hat pass t he filt er cle a r ip bgp fla p- st a t ist ics x . x . x . x [ m . m . m . m ]— Clear s flap st at ist ics for a single ent ry

110

•

cle a r ip b g p x . x . x . x f la p- st a t ist ics— Clear s flap st at ist ics for all pat hs fr om a n eigh bor

BGP N eighbor Aut hent ica t ion You can inv ok e MD5 aut hent icat ion bet w een t w o BGP peer s. This feat ur e m ust be configur ed w it h t he sam e passw or d on bot h BGP peer s; ot her w ise, t he connect ion bet w een t hem w ill not be m ade. I nv ok ing aut hent icat ion causes t he Cisco I OS Soft w ar e t o gener at e and check t he MD5 digest of ev er y segm ent sent on t he TCP connect ion. I f aut hent icat ion is inv ok ed and a segm ent fails aut hent icat ion, a m essage appear s on t he console. Configur ing a passw or d for a neighbor causes an ex ist ing session t o be t or n dow n and a new one t o be est ablished. I f you specify a BGP peer gr oup by using t he p e e rgr ou p nam e ar gum ent , all t he m em ber s of t he peer gr oup w ill inher it t he char act er ist ic configur ed w it h t his com m and. I f a r out er has a passw or d configur ed for a neighbor but t he neighbor rout er does not , a m essage such as t he follow ing appear s on t he console w hile t he r out er s at t em pt t o est ablish a BGP session bet w een them :

%TCP-6-BADAUTH: No MD5 digest from [peer's IP address]: 11003 to [local router's IP address]:179 Sim ilarly, if t he t w o rout ers have differ ent passw or ds configur ed, a m essage such as t he follow ing appear s on t he console:

%TCP-6-BADAUTH: Invalid MD5 digest from [peer's IP address]: 11004 to [local router's IP address]:179 The follow ing ex am ple specifies t hat t he r out er and it s BGP peer at 145.2.2.2 invoke MD5 aut hent icat ion on t he TCP connect ion bet w een t hem :

router bgp 109 neighbor 145.2.2.2 password v61ne0qkel33&

BGP MED N ot Set When an MED is not set on a r out e, Cisco I OS Soft w ar e has alw ay s assum ed t hat t he MED is 0. Som e ot her v endor s hav e assum ed t hat t he MED is 4,294,967,295 ( 2 3 2 – 1) . This diver gence can r esult in eBGP r out ing loops bet w een Cisco r out er s and ot her vendor s’ r out er s. This confusion w as t he r esult of a lack of any definit ion in t he BGP st andar d on w hat t o do if a MED w as not set . The m ost r ecent I ETF decision r egar ding BGP MED assigns a value of infinit y t o t he m issing MED, m aking t he r out e lacking t he MED var iable t he least pr efer r ed. The default behavior of BGP r out er s r unning Cisco I OS Soft w ar e is t o t r eat r out es w it hout t he MED at t r ibut e as hav ing a MED of 0, m aking t he r out e lacking t he MED var iable t he m ost pr efer r ed. To configur e t he r out er t o confor m t o t he belat ed I ETF st andar d, use t his com m and ( new in I OS Soft w ar e Releases 12.0 and 12.1 but not in 12.0S) :

111

router bgp 109 bgp bestpath missing-as-worst I SPs should ser iously consider w het her t hey need t o im plem ent t his com m and, giv en t he lar ge deploy m ent of r out er s in t he I nt er net t hat ar e using t he Cisco default . I m plem ent ing t his com m and w it hout consider ing t he consequences w ill hav e j ust as ser ious of an im pact as not being aw ar e of t he sit uat ion in t he fir st place.

BGP Det erm inist ic M ED I f bgp de t e r m in ist ic - m e d is not enabled, t he or der in w hich r out es ar e r eceiv ed could im pact MED- based best - pat h decisions. This can occur w hen t he sam e rout e is r eceiv ed fr om m ult iple aut onom ous sy st em s or confeder at ion sub- aut onom ous sy st em s, w it h ex act ly t he sam e pat h lengt h and differ ent MEDs. For ex am ple, consider t he follow ing r out es, r eceiv ed in t he or der show n: • • •

A— ASPATH 1, MED 100, int ernal, I GP m et ric t o NEXT_HOP 10 B— ASPATH 2, MED 150, int ernal, I GP m et ric t o NEXT_HOP 5 C— ASPATH 1, MED 200, ex t er nal

I f t he pat hs w er e r eceived in t his or der , a BGP- speak ing r out er w it hout det er m inist ic MED configur ed w ould choose pat h B ov er pat h A because of a low er I GP m et r ic t o r each t he NEXT_HOP ( see St ep 10 in t he BGP pat h- select ion pr ocess) . I t w ould t hen prefer pat h C t o pat h B because it is ext ernal. How ever, pat h C has a higher MED t han pat h A. Enabling b g p d e t e r m in ist ic - m e d r em ov es any t em por al dependency of MED- based best pat h decisions. I t ensur es t hat cor r ect MED com par ison is m ade acr oss all r out es r eceiv ed fr om t he sam e AS and t hat pat h A is chosen as t he best pat h in t he ex am ple. Not e t hat if bgp a lw a y s- com pa r e - m e d is enabled, BGP MED decisions alw ay s w ill be det er m inist ic. Cisco st r ongly r ecom m ends enabling b g p d e t e r m in ist ic - m e d in all new net w ork deploy m ent s. I SP engineer s also should consider r et r oact iv ely inst alling t his configur at ion but should m ak e sur e t hat t her e is no unfor eseen oper at ional im pact in doing so.

Com paring Rout er I Ds As par t of t he st andar d pat h- select ion pr ocess in I OS Soft w ar e, t he r out er does not sw it ch bet w een t w o eBGP pat hs based solely upon t he r out er I D. Consider a sit uat ion in w hich a r out er hear s t w o announcem ent s of a par t icular pr efix fr om t w o differ ent neighbor s. I f t he pat h- select ion pr ocess det er m ines t hat all at t r ibut es ar e ident ical apar t fr om t he r out er I D, it cur r ent ly t ak es t he pat h w it h t he oldest ent r y in t he rout ing t able, not t he ent r y w it h t he low est r out er I D. This choice w as m ade t o m aint ain st abilit y in oper at ional back bones ov er t he last decade. How ev er , it is act ually count er t o RFC 1771, w hich st at es t hat t he r out er I Ds alw ay s should be com pared.

112

A new BGP opt ion has been added ( fully docum ent ed in CSCdr 47086) fr om I OS Soft w ar e Releases 12.0( 11) S and 12.1( 3) t hat m ak es t he pat h- select ion pr ocess com pliant w it h t he RFC:

bgp bestpath compare-routerid This t ells t he BGP pat h- select ion pr ocess t o alw ay s com par e r out er I Ds at t he appr opr iat e st age. The I OS Soft w ar e default r em ains unchanged.

BGP net w ork St at em ent A hist or ical lim it at ion w as placed on t he num ber of n e t w or k st at em ent s t hat could be applied t o t he BGP configur at ion. I n t he ear ly days of I OS Soft w ar e, in w h ich t h e aver age r out er had at m ost 30 int er faces, t he lim it of 200 n e t w o r k st at em ent s seem ed a lar ge num ber com par ed w it h t he num ber of net w or k s t hat such a r out er could or iginat e. How ever , in t he last five year s or so, r out er s have gr ow n t hr ough hav ing hundr eds of int er faces t o t he t housands of int er faces t hat w e see on som e of t he plat for m s t oday . Obv iously , 200 n e t w o r k st at em ent s is a sev er e lim it at ion ( and, unfor t unat ely, has r esult ed in m any I SPs heading dow n t he pat h of r edist r ibut ing prefixes from ot her r out ing pr ot ocols int o BGP) . CSCdj 57631 r em ov ed t he 200 n e t w or k st at em ent lim it for BGP ( as w ell as OSPF, RI P, and EI GRP) in lat e 1997 in I OS Soft w ar e Releases 11.1CC and 11.3, and now it is r ecom m ended pr act ice for all new deploy m ent s t o use t he n e t w or k st at em ent t o inj ect pr efix es int o BGP.

Rem oving Priva t e Aut onom ous Syst em s Som e I SPs use pr iv at e aut onom ous sy st em s w it hin t heir net w or k s ( t y pically but not exclusively for cust om er s w ho m ult ihom e ont o t he backbone) . A new BGP opt ion ( CSCdi64489) pr ev ent s any pr iv at e aut onom ous sy st em s fr om being leak ed t o t he I nt er net :

router bgp 109 neighbor 145.2.2.2 remove-private-AS I f t he BGP updat e has pr ivat e aut onom ous syst em s [ 2] in t he AS pat h, t his opt ion r em ov es t he pr iv at e AS num ber s. Not e t hat it w ill not w or k if t her e ar e pr iv at e and public aut onom ous sy st em s in t he AS pat h; t hus, t his opt ion cannot be used w hen a net work is using a privat e AS for it s BGP yet is pro v iding t r ansit t o t he public I nt er net ( t his is consider ed a configur at ion er r or ) . Not e t hat in t he case of BGP confeder at ions, t he pr ivat e AS w ill be r em oved as long as t he pr ivat e AS appear s aft er ( and out side) t he confeder at ion por t ion of t he AS pat h. I t should be not ed t hat t his com m and can be used only for eBGP peer s; it has no effect on iBGP peer s. Befor e t he ex ist ence of t his com m and, m ost I SPs used a com binat ion of AS pat h access list s and pr ox y announcem ent s t o r em ov e pr iv at e aut onom ous sy st em s. That pr ocess w as clum sier and oft en pr one t o er r or . For t his r eason, r e m ov e - p r iv a t e - AS is st r ongly r ecom m ended t o r em ov e pr iv at e aut onom ous sy st em s fr om t he public I nt er net , and it is r ecom m ended t o be included as par t of t he default BGP configur at ion t em plat e on all new and cur r ent net w or k deploy m ent s. The configur at ion t em plat e in Appendix B, “ Cut - and- Past e Tem plat es,” includes t his.

113

For r efer ence and com par ison pur poses, t he old m et hod is shown here:

router bgp 109 network 220.10.0.0 mask 255.255.224.0 neighbor 145.2.2.2 filter-list 5 out ! ip as-path access-list 5 deny ^(65534_)+$ ! ip route 220.10.0.0 255.255.224.0 null0 250 This configur at ion has t he sam e effect as using one line r e m ov e - p r iv a t e - AS for t he AS 65534 peer, but it is m ore prone t o error by requiring a filt er list and a st at ic pullup r out e t o ensur e t hat t he / 19 pr efix appear s in t he r out ing t able.

BGP local- a s A r ecent addit ion t o BGP is t he loca l- a s neighbor opt ion, w hich allow s t he AS num ber of t he net w or k t o be changed for eBGP peer ings. I t isn’t possible in I OS Soft w are t o configure BGP t o run in m ore t han one AS; t he local AS feat ure provides a solut ion t o t his r equir em ent . Con figu r a t ion Rout er A is in AS 109, and Rout er B is in AS 159. However, when A peers wit h B, it uses AS 210 as it s AS num ber. As far as B is concerned, it is peering wit h a rout er in AS 210. Operat ionally, t his is equivalent t o Rout er B peering w it h a rout er in AS 210, and t hat rout er peer ing w it h Rout er A in AS 109. The AS pat h for all pr efixes lear ned from Rout er B as seen on Rout er A w ould be 210_159. AS 210 is insert ed int o t he AS pat h sequence. The AS pat h for all pr efixes lear ned fr om Rout er A by Rout er B w ould be 210_109. Rout er A configur at ion:

router bgp 109 neighbor 145.2.2.2 remote-as 159 neighbor 145.2.2.2 local-as 210 Rout er B configurat ion:

router bgp 159 neighbor 144.2.2.1 remote-as 210 loca l- a s can be configur ed per eBGP peer or per peer gr oup. I t cannot be configur ed for individual m em bers of a peer group. Also, lo ca l- a s cannot be configur ed w it h t he AS num ber of t he BGP pr ocess of t he local r out er , nor can it be configur ed w it h t he AS num ber of t he BGP pr ocess on t he r em ot e r out er . lo ca l- a s cannot be used bet w een t w o confeder at ion eBGP peer s, eit her—t hey m ust be t r ue eBGP peer s. M ot iva t ion

114

Th e loca l- a s feat ur e m ost oft en is used and w as r equest ed by I SPs t hat ar e act ive in acquir ing ot her I SPs. I f an I SP pur chases anot her I SP, a significant am ount of w or k goes int o r enum ber ing t he acquir ed I SP’s back bone int o t he sam e AS num ber as t hat of t he pur chaser . How ever , in som e sit uat ions t he acquir ed I SP m ight have agr eem ent s w it h peer s or a specific eBGP configur at ion t hat is har d t o m igr at e t o t he new AS ( for polit ical or t echnical r easons, or bot h) . The loca l- a s opt ion can m ake t he peering rout er in AS 109 look as t hough it really is in AS 210, t he I SP net w ork t hat w as pur chased, unt il t he polit ical and t echnical issues w it h t he int er pr ov ider peer can be sor t ed ou t .

BGP N e ighbor Cha nge s I t is possible t o log neighbor st at e changes t o a UNI X syslog ser ver . This is ext rem ely useful for m ost syslog- based m onit or ing syst em s because it gives ear ly w ar ning of pr oblem s w it h iBGP peer s, and m or e especially ext er nal BGP neighbor s. The logging is enabled by t he follow ing com m ands:

router bgp 109 bgp log-neighbor-changes Not e t hat , as of I OS Soft w ar e Releases 12.1( 4) and 12.0( 12) ST, t he default act ion is t o log all BGP neighbor changes ( see DDTS CSCdm 59903) . At t im e of w r it ing, t he default w as not yet included in I OS Soft w are Release 12.0S. Also not e t hat , as of CSCdr 54231, BGP not ificat ion m essages ar e included w hen b g p log- n e ig h b or- ch a n g e s is configur ed. Befor e CSCdr 54231, t he only w ay t o see t hese m essages w as t o enable d e b u g ip b g p , w hich w as a bit lim it ing for m ost I SPs! This alt er at ion has been included in I OS Soft w ar e Releases fr om 12.0( 10) S3, 12.1( 3) , and 11.1( 34) CC.

Lim it ing t he N um ber of Prefix es from a N eighbor At t im es, eit her t hr ough configur at ion er r or or t hrough a blat ant at t ack on t he I nt er net , t he global default fr ee r out ing t able j um ped t o t w o t o t hr ee t im es it s size. This caused sev er e pr oblem s on sect ions of t he I nt er net . The BGP neighbor com m and m a x im u m - prefix w as added t o help net w or k s safeguar d against t h ese sor t s of pr oblem s. This com m and allow s you t o configur e a m axim um num ber of pr efixes t hat a BGP r out er is allow ed t o r eceive fr om a peer . I t adds anot her m echanism ( in addit ion t o dist r ibut e list s, filt er list s, and r out e m aps) t o cont r ol prefixes r eceiv ed fr om a peer . When t he num ber of r eceiv ed pr efix es ex ceeds t he m ax im um num ber configur ed, t he r out er t er m inat es t he peer ing ( by default ) . How ever , if t he keyw or d w a r n in g- only is configur ed, t he r out er inst ead only sends a log m essage but cont inues peer ing w it h t he sender . I f t he peer ing is t er m inat ed, t he peer st ay s dow n unt il t he cle a r ip bgp com m and is issued. I n t he follow ing ex am ple, t he m ax im um num ber of pr efix es allow ed fr om t he neighbor at 129.140.6.6 is set t o 150,000 ( t he global I nt er net r o ut e t able w as ar ound 106,000 at t he t im e of t his w r it ing) :

115

router bgp 109 network 131.108.0.0 neighbor 129.140.6.6 maximum-prefix 150000 Th e m a x im um - prefix com m and sends log m essages also, so any ov er r un can be t r apped by a m anagem ent syst em t hat m on it or s t he r out er ’s syslog out put . One m essage is sent w hen t he num ber of pr efix es r eceiv ed r eaches t he configur ed t hr eshold v alue:

%BGP-4-MAXPFX: No. of unicast prefix received from 129.140.6.6 reaches 113021, max The default t hr eshold is 75 per cent . T his v alue can be changed by specify ing t he t hr eshold per cent age in t he m a x im um - pr e fix line. The follow ing ex am ple set s t he t hr eshold t o 95 per cent :

router bgp 109 bgp log-neighbor-changes neighbor 129.140.6.6 maximum-prefix 120000 95 Anot her m essage is sent w hen t he num ber of pr efix es r eceiv ed ex ceeds t he m ax im um num ber of pr efix es configur ed. Logging of neighbor changes is included for com plet eness in t his ex am ple:

%BGP-3-MAXPFXEXCEED: No. of unicast prefix received from 129.140.6.6: 123411 exceed limit 120000 %BGP-5-ADJCHANGE: neighbor 129.140.6.6 Down – BGP Notification Sent

Lim it ing t he AS Pat h Lengt h from a N eighbor Oft en I SPs ar e r equir ed t o lim it t he lengt h of t he AS pat h t hat t hey r eceiv e fr om t heir peer s, especially w hen t r y ing t o configu r e m ult ihom ing w it h good load shar ing. The har d w ay t o do t his is t o use an AS pat h access list w it h a v er y long m at ch st ring —for exam ple, som et hing like t his

router bgp 109 neighbor 192.168.1.1 remote-as 65534 neighbor 192.168.1.1 filter-list 1 in ! ip as-path access-list 1 permit ^[0-9]*_[0-9]*_[0-9]* _[0-9]*_[0-9]*_[0-9]*_[0-9]*_[0-9]*_[0-9]*_[0-9]+$ ! This w ill per m it all pr efixes w it h an AS pat h less t han or equal t o 10 ASNs. How ever , t his is quit e ugly, and t he problem is easier t o solve w it h t he m a x a s- lim it com m and:

116

router bgp 109 neighbor 192.168.1.1 remote-as 65534 neighbor 192.168.1.1 maxas-limit 10 ! This does alm ost t he sam e t hing. The m ain differ ence bet w een t he AS pat h filt er ex am ple and t he m a x a s- lim it com m and is t hat , in t he lat t er case, all r eceived pr efixes ar e included in t he BGP t able, but only t hose w it h an AS pat h lengt h less t han or equal t o 10 ASNs ar e included in t he BGP pat h- select ion pr ocess. ( At t he t im e of t his w rit ing, m a x a s- lim it is a hidden com m and.)

BGP fa st - e x t e r na l-fa llove r By default , if t he phy sical connect ion t o t he eBGP neighbor goes dow n, t he peer ing relat ionship is reset im m ediat ely. By adding t he n o bgp fa st - e x t e r n a l- f a llo v e r configur at ion, t he peer ing is held open for t he dur at ion of t he BGP k eepaliv e t im er . This configur at ion is desir able, if not essent ial, in t he case of long- dist ance peering links or unreliable or long- lat ency connect ions t o ot her aut onom ous sy st em s, and w hen I SPs pr efer st abilit y over conver gence speed in lar ge net w or ks.

router bgp 109 no bgp fast-external-fallover Not e t hat t he fast fallov er for ex t er nal peer s only applies t o dir ect eBGP peer ings. Those peerings using eBGP- m ult ihop ar e not subj ect ed t o fast - ext ernal- fallover pr ocesses. Also not e t hat fast - ext ernal- fallover does not apply t o iBGP peer ings. W ARN I N G This configur at ion opt ion should be used w it h car e. I t is r ecom m ended t hat fa st e x t e r n a l- f a llov e r be used only for links t o an I SP’s upst ream provider or over unreliable links. Because t he failover is slow er, it is possible t o black hole r out es for up t o t hr ee m inut es. This could pr ov e pr oblem at ic, for ex am ple, on a link t o a m ult ihom ed cust om er , w her e peer ing m ight hav e suffer ed an unint ent ional r eset because of hum an act iv it y .

BGP Peer Group [ 3 ] The m aj or benefit of BGP peer gr oups is a r educt ion of r esour ces ( CPU load and m em or y ) r equir ed in updat e gener at ion. Peer gr oups also sim plify BGP configur at ion. Wit h BGP peer gr oups, t he r out ing t able is w alked only once per peer gr oup, and updat es ar e r eplicat ed t o all ot her peer- group m em bers t hat are in sync. Depending on t he num ber of m em bers, t he num ber of prefixes in t he t able, and t he num ber of pr efix es adv er t ised, t his could significant ly r educe t he load. Thus, it is highly r ecom m ended t hat peer s w it h ident ical out bound announcem ent policies be gr ouped int o peer gr oups.

117

Re q u ir e m e n t s All m em ber s of a peer gr oup m ust shar e ident ical out bound announcem ent policies ( for ex am ple, dist r ibut e list s, filt er list s, and r out e m aps) , ex cept for t he or iginat ing default , w hich is handled on a per- peer basis ev en for peer- gr oup m em ber s. The inbound updat e policy can be cust om ized for each indiv idual m em ber of a peer group. A peer group m ust be eit her int er nal ( w it h iBGP m em ber s) or ext er nal ( w it h eBGP m em ber s) . Mem ber s of an ex t er nal peer gr oup hav e differ ent AS num ber s. H ist or ica l Lim it a t ion s Sever al lim it at ions exist ed w it h BGP peer gr oups in older I OS Soft w ar e ver sions: • • •

I f used for client s of a r out e r eflect or , all t he client s should be fully m eshed. I f used as an eBGP peer gr oup, t r ansit cannot be pr ov ided am ong t he peergroup m em bers. All t he eBGP peer- gr oup m em ber s should be fr om t he sam e subnet t o avoid nonconnect ed next - hop announcem ent s.

I nconsist ent r out ing w ould occur if t hese lim it at ions w er e not follow ed. They hav e been r em ov ed st ar t ing w it h t he follow ing I OS Soft w ar e v er sions: 11.1( 18) CC, 11.3( 4) , and 12.0. Only t he r out er on w hich t he peer gr oups ar e defined needs t o be upgr aded t o t he new code. Ty pica l Pe e r Gr ou p U sa ge I SP net w or k engineer s gr oup BGP peer s on a r out er int o peer gr oups based on t heir out bound updat e policies. A list of peer gr oups com m only by I SPs follow s: • • • • •

N or m a l iBGP pe e r gr ou p — For nor m al iBGP peer s. iBGP clie n t p e e r g r ou p — For r eflect ion peer s on a r out e r eflect or . e BGP fu ll r ou t e s— For peer s t o r eceiv e full I nt er net r out es. e BGP cu st om e r r ou t e s— For peer s t o r eceiv e r out es fr om dir ect cust om er s of t he I SP only . Som e m em ber s can be configur ed w it h de fa ult - or igin a t i o n t o r eceiv e t he default r out e as w ell as t he cust om er r out es. e BGP d e f a u lt r ou t e s— For peer s t o r eceiv e t he default r out e and possibly a few ot her r out es.

BGP Pe e r Gr ou p Ex a m p le s This ex am ple show s an iBGP peer gr oup for a r out er inside an I SP’s backbone:

router bgp 109 neighbor internal neighbor internal neighbor internal neighbor internal

peer-group remote-as 109 update-source loopback 0 send-community

118

neighbor neighbor neighbor neighbor neighbor neighbor

internal route-map send-domestic out internal filter-list 1 out 131.108.10.1 peer-group internal 131.108.20.1 peer-group internal 131.108.30.1 peer-group internal 131.108.30.1 filter-list 3 in

! This exam ple show s an eBGP peer Gr oup for a r out er peer ing w it h sever al I SPs, all w it h t he sam e adv er t isem ent policies:

router bgp 109 neighbor external-peer peer-group neighbor external send-community neighbor external-peer route-map set-metric out neighbor external-peer route-map filter-peer in neighbor 160.89.1.2 remote-as 200 neighbor 160.89.1.2 peer-group external-peer neighbor 160.89.1.4 remote-as 300 neighbor 160.89.1.4 peer-group external-peer ! The iBGP and eBGP configur at ion t em plat es in Appendix B m ake use of peer groups as ex am ples. I t is r ecom m ended t hat y ou t o t r y use peer gr oups w her ev er possible because t he incr eased r eadabilit y of t he configur at ion and m odest per for m ance benefit s for t he r out er ar e w or t hw hile. One im port ant point t o not e is t he int er act ion bet w een r out e m aps and peer gr oups. I f a rout e m ap is applied inside a peer group, a rout e m ap applied t o t he act ual peer is accept ed by t he par ser but isn’t act ually used. I n ot her w or ds, t he r out e m ap configur ed inside a peer gr oup ov er r ides t he r out e m ap configur ed on t he act ual n eigh bor—for ex am ple:

router bgp 109 neighbor external-peer peer-group neighbor external send-community neighbor external-peer route-map set-metric out neighbor external-peer route-map filter-peer in neighbor 160.89.1.2 remote-as 200 neighbor 160.89.1.2 peer-group external-peer neighbor 160.89.1.2 route-map set-policy out ! The r out e m ap se t - policy w ill be ignor ed in t he pr ev ious configur at ion because t he peer group rout e m ap se t - m et ric w ill ov er r ide it .

BGP M ult ipat h The BGP im plem ent at ion in I OS Soft w ar e suppor t s t hr ee w ays of load shar ing over par allel cir cuit s. Tw o of t hem ar e applicable t o eBGP. eBGP m ult ihop has been used in I OS Soft w ar e for sev er al y ear s and is t he com m on w ay of set t ing up an eBGP

119

peer ing w it hout using t he dir ect ly connect ed peer addr esses; t he com m on use of t his is for load shar ing ov er par allel peer ing cir cuit s. Mor e r ecent ly , eBGP m ult ipat h w as added t o give an alt er nat ive m echanism for load shar ing w it hout som e of t he side effect s exper ienced w it h eBGP m ult ihop. The t hir d m ult ipat h feat ur e is a ver y r ecent addit ion t o I OS Soft w ar e and is applicable t o iBGP only . I t allow s load shar ing ov er par allel pat hs w it hin an I SP’s net w or k ( independent of any load shar ing av ailable t hr ough t he I GP) . e BGP M u lt ipa t h Wit h I OS Soft w ar e Release 11.1CC cam e a new BGP feat ur e t hat allow s m or e t han one pat h t o t he sam e dest inat ion t o be inst alled in t he for w ar ding t able. This feat ur e, called eBGP m ult ipat h, is designed t o allo w I SPs t o load- shar e over ext er nal cir cuit s t o eBGP neighbor s. When t he bor der r out er has m or e t han one pat h t o t he sam e ex t er nal net w or k , t he BGP pat h- select ion pr ocess m akes a decision at St ep 11, inst alling t he oldest received pat h int o t he RI B. So only one of t he ext er nal pat hs is used. eBGP m ult ipat h allow s t he r out er t o inst all up t o a m axim um of six pat hs in t he for w ar ding t able. To enable t his feat ur e, configur e m a x im u m - pa t hs under r ou t e r bgp as follow s:

router bgp 100 maximum-paths 1-6 ! Tw o or m or e eBGP pat hs ar e consider ed candidat es for all m ult ipat h at t r ibut es ( w eight , localpr ef, AS- PATH [ ent ir e at t r ibut e, not j ust lengt h] ) . Or igin and MED ar e t he sam e. Care is required when using eBGP m ult ipat h. I f t he ext ernal peer or peers are car r y ing t he full r out ing t able, t he local r out er suppor t ing m ult ipat h w ill r eceive t w o full BGP feeds and w ill require sufficient m em ory t o support t his. This is w hy it is m ore norm al for I SPs t o use an eBGP- m ult ihop configur at ion t o suppor t load shar ing ov er par allel pat hs t o an ext ernal peer. e BGP M u lt ih op Consider t his ex am ple. AS 100 connect s t o AS 200 using t hr ee par allel cir cuit s bet w een t heir t w o bor der r out er s. An eBGP m ult ipat h I OS Soft w ar e configur at ion ex am ple is giv en her e t hat w ill let t he bor der r out er in AS 100 t o load- share out bound t r affic on t he t hr ee cir cuit s:

interface serial 1/0 ip address 1.1.1.2 255.255.255.252 ! interface serial 1/1 ip address 1.1.1.6 255.255.255.252 ! interface serial 1/2

120

ip address 1.1.1.10 255.255.255.252 ! router bgp 100 neighbor 1.1.1.1 remote-as 200 neighbor 1.1.1.1 prefix-list AS200peer in neighbor 1.1.1.5 remote-as 200 neighbor 1.1.1.5 prefix-list AS200peer in neighbor 1.1.1.9 remote-as 200 neighbor 1.1.1.9 prefix-list AS200peer in maximum-paths 3 ! I f AS 200 is delivering t he full rout ing t able, t he AS 100 border rout er w ill require sufficient m em or y t o suppor t t he t hr ee full view s. The equivalent configurat ion using eBGP m ult ihop is m ore m em ory - efficient in t he case of large rout ing t ables being exchanged; it is given her e:

interface loopback 0 ip address 1.1.0.1 255.255.255.255 ! interface serial 1/0 ip address 1.1.1.2 255.255.255.252 ! interface serial 1/1 ip address 1.1.1.6 255.255.255.252 ! interface serial 1/2 ip address 1.1.1.10 255.255.255.252 ! router bgp 100 neighbor 1.1.1.255 remote-as 200 neighbor 1.1.1.255 ebgp-multihop 2 neighbor 1.1.1.255 update-source loopback 0 neighbor 1.1.1.255 prefix-list AS200peer in ! ip route 1.1.1.255 255.255.255.255 serial 1/0 ip route 1.1.1.255 255.255.255.255 serial 1/1 ip route 1.1.1.255 255.255.255.255 serial 1/2 ! Not ice t hat t he eBGP peer ing is set up bet w een t he loopback int er faces of t he t w o rout ers. There is only one BGP session, so it is m ore efficient in rout er m em ory. Not ice t he e bgp- m u lt ih op configur at ion. Som e I SPs allow 255 for t he m axim um num ber of int er m ediat e hops; how ev er , it is r ecom m ended t hat , t o av oid pot ent ial pr oblem s w it h r out ing loops or black holes, t he m ult ihop par am et er be set t o t he act ual number of hops r equir ed bet w een t he t w o r out er s. This exam ple is t he m ost com m on use of eBGP m ult ihop. Som e I SPs also use eBGP m ult ihop for all t heir cust om er BGP connect ions. I t is our ex per ience t hat t her e is lit t le t o be gained by doing t his because m ost ISPs hav e separ at e aggr egat ion point s for BGP cust om er s and st at ically connect ed cust om er s. Ter m inat ing cust om er BGP

121

connect ions is usually m inim al ov er head com par ed w it h t he back bone iBGP sessions present in m ost I SP net w orks. iBGP M u lt ipa t h A new feat ur e added t o I OS Soft w ar e Release 12.0ST allow s iBGP t o gain a m ult ipat h feat ur e sim ilar t o w hat has been av ailable for eBGP since 11.1CC. When t her e ar e m ult iple bor der BGP r out er s w it h r eachabilit y infor m at ion hear d over eBGP, and if no local policy is applied, t he bor der r out er s w ill choose t heir eBGP pat hs as best pat h. They adv er t ise t hat best pat h inside t he I SP net w or k . So, fr om a pur e cor e r out er ’s per spect iv e, t her e can be m ult iple pat hs t o t he sam e dest inat ion, but only one pat h w ill be select ed as t he best and be used for for w ar ding. I f t he m ult iple pat hs ar e equidist ant , it is som et im es desir able t o use t hem for load balancing. To enable t his feat ur e, configur e m a x im u m - pa t h s ibgp under rout er BGP as follows:

router bgp 100 maximum-paths ibgp 1-6 ! Tw o or m or e iBGP pat hs ar e consider ed candidat es for m ult ipat h if t he follow ing cr it er ia ar e m et : • •

All at t r ibut es ( w eight , localpr ef, AS- PATH [ ent ir e at t r ibut e, not j ust lengt h] , Or igin, MED, and I GP dist ance ar e t he sam e. The nex t hops ar e differ ent .

The best - pat h calculat ion cont inues unt il t he last st ep of t he BGP pat h- select ion pr ocess, so t he best pat h t hat is adv er t ised w ill r em ain t he sam e w it h and w it hout t his feat ur e t ur ned on. Out of t he candidat e m ult ipat hs, t he best pat h is alw ay s insert ed int o t he RI B. Ot her candidat e m ult ipat hs ar e inser t ed depending on t he num ber of pat hs allow ed int o t he RI B t hr ough t he m a x im um - p a t h s ib g p com m and.

Apply in g Policy w it h BGP The m ain differ ent iat or bet w een BGP and an I GP is t hat BGP can be used t o apply policies t o t he ex change of r out ing infor m at ion bet w een t w o neighbor ing r out er s. This sect ion consider s t he policy opt ions av ailable, fr om t he int r oduct ion of pr efix list filt er ing and t he applicat ion of r out e m aps, t o t he use of BGP policy account ing t o char act erize t raffic flowing int o and out of a net work.

Using Prefix List s in BGP Rout e Filt ering [ 4 ] The pr efix list feat ur e offer s significant per for m ance im pr ov em ent ( in t erm s of CPU consum ed) ov er t he access list in r out e filt er ing of r out ing pr ot ocols. I t also pr ov ides for fast er loading of lar ge list s and suppor t for incr em ent al configur at ion. I n addit ion,

122

t he com m and- line int erface is m uch m ore int uit ive. This feat ure is av ailable in I OS Soft w ar e ver sions fr om 11.1 ( 17) CC, 11.3( 3) , and 12.0. Accor ding t o one I SP t hat has done som e in- dept h per for m ance t est ing of pr efix list s and access list s, a 7507/ RSP4 r unning 11.1( 20) CC t ook m or e t han 15 m inut es t o boot using ext ended ACLs for filt er ing BGP r out es. The sam e t est w it h pr efix list s had t he r out er boot ing and fully oper at ional in less t han 5 m inut es. The configur at ion involved ar ound 95,000 lines of ACL ( or pr efix list s) t ot al for all neighbor s, a configur at ion ar ound 6 MB in t ot al size. Giv en t his t y pe of ex per ience, it is easy t o see w hy m ost I SPs now ar e using pr efix list s r at her t han ACLs for pr efix filt er ing on t heir BGP peering sessions. The pr efix list pr eser v es sev er al k ey feat ur es of t he access list : • • •

Configur at ion of eit her pe r m it or deny Or der dependency—fir st m at ch w ins Filt er ing on pr efix lengt h, bot h ex act m at ch and r ange m at ch

How ev er , pr efix list s, or pr efix list s in r out e m aps, do not suppor t pack et filt er ing. This sect ion pr esent s t he det ailed configur at ion com m ands and sev er al applicat ions of t he prefix list in rout e filt ering. Con f ig u r a t ion Com m a n d s Thr ee configur at ion com m ands ar e r elat ed t o t he pr efix list . The follow ing com m and can be used t o delet e a pr efix list :

no ip prefix-list list-name Her e, list - nam e is t he st r ing ident ifier of a pr efix list . This next com m and can be used t o add or delet e a t ext descr ipt ion for a pr efix list :

[no] ip prefix-list list-name description text The follow ing com m and can be used t o configur e or delet e an ent r y of a pr efix list :

[no] ip prefix-list list-name [seq seq-value] deny|permit \ network /len [ge ge-value] [le le-value] Sev er al com m and at t r ibut es ex ist , as can be seen in t he t hr ee pr eceding ex am ples. These at t r ibut es have t he follow ing m eanings: • •

list - nam e— Mandat or y . This is t he st r ing ident ifier of a pr efix list . se q seq- value— Opt ional. This at t r ibut e can be used t o specify t he sequence num ber of an ent ry of a prefix list . By default , t he ent ries of a prefix list w ould have sequence values of 5, 10, 15 and so on. I n t he absence of a

123

• • • •

specified sequence v alue, t he ent r y w ould be assigned w it h a sequence num ber of ( Cur r ent _Max+ 5) . de n y| pe r m it — Mandat or y . The act ion is t ak en once a m at ch is found. net w ork/ len — Mandat or y. This is t he pr efix ( t hat is, net work and prefix lengt h) . Mult iple policies ( ex act m at ch or r ange m at ch) w it h differ ent sequence num ber s can be configur ed for t he sam e n et w or k/ len. ge ge- value— Opt ional. le le- v alue— Opt ional.

Bot h ge and le ar e opt ional. They can be used t o specify t he range of t he prefix lengt h t o be m at ched for pr efix es t hat ar e m or e specific t han net w ork/ len. An ex act m at ch is assum ed w hen neit her ge nor le is specified. The r ange is assum ed t o be from ge- v alue t o 32 if only t he ge at t r ibut e is specified. The r ange is assum ed t o be from le n t o le- value if only t he le at t r ibut e is specified. A specified ge- value or le- v alue m ust sat isfy t he follow ing condit ion:

len < ge-value < le-value can be subdivided int o t hr ee sect ions: • •

•

Com m a n d s g lob a l t o t h e r ou t e r — These com m ands affect t he oper at ion of t he BGP global t o t he box. Exam ples: bgp de t e r m in ist ic - m e d and b g p clu st e r- i d. Com m a n d s t o id e n t if y t h e n e ig h b or s/ p e e r g r ou p s— These com m ands define t he neighbor or peer gr oup ( w hich ar e accessible fr om t he default r out ing t able) by specify ing t he r em ot e AS, eBGP m ult ihop, updat e sour ce, and so on. Ex am ples: n e ig h b o r x . x . x . x r e m ot e - a s and n e i g h b o r x .x .x .x d e scr ip t ion. Com m a n ds pe r a ddr e ss f a m ily — Tw o set s of com m ands can be applied t o addr ess fam ilies. - Globa l t o pe r a ddr e ss fa m ily — These com m ands ar e neighborindependent and allow t he behav ior of BGP t o be changed for t hat specific addr ess fam ily . Pr efix es t o be sour ced ( using a n e t w or k st at em ent or r edist r ibut ion) under t his addr ess fam ily fit int o t his cat egor y . Ex am ples: n e t w or k x . x . x . x, r e d ist r ib u t e d v m r p, and b g p sca n- t im e . - Pe r- n e ig h b or / p e e r g r ou p— These com m ands configur e policy for t he neighbor s or peer gr oups w it h dist r ibut e list s, pr efix list s, or r out e m aps. The

137

neighbor s also can be configur ed as client s or can be added as m em bers of a peer gr oup. The neighbor s need t o be ex plicit ly act iv at ed t o ex change t he m ult ipr ot ocol BGP pr efix es. Ex am ples: n e i g h b o r x .x .x .x filt e r- list , n e ig h b or x . x . x . x r ou t e - m a p , and n e i g h b o r x . x . x . x a ct iv a t e .

Com m and Group Organizat ion Th e com m ands list ed in t he fir st pr eceding gr oup hav e no am biguit y , so t hey all can appear under t he r o u t e r b g p < a s> definit ion. The com m ands list ed in t he second gr oup hav e no am biguit y eit her , so t hey can follow t he fir st gr oup of com m ands under t he m ain BGP definit ion ( w it h t he except ion of VPNs) . The com m ands list ed in t he t hir d gr oup can hav e pot ent ial am biguit y and t hus ar e list ed under a new addr ess fam ily subm ode:

router bgp address-family afi sub-afi ... exit-address-family exit A neighbor c an have different rout e m ap or prefix list st at em ent s, one per address fam ily. For configur ing I Pv4 unicast BGP ( vanilla BGP) policy for t he neighbor s, it is possible t o configur e t hem follow ing t he Gr oup 2 com m ands. This is v er y sim ilar t o w hat is present in t he old CLI . The addr ess fam ily I Pv4 unicast is im plicit . I t is also possible t o configur e t he I Pv 4 unicast BGP policy using t he addr ess fam ily subm ode. I n sh o w r u n n in g- con fig, addr ess fam ily I Pv 4 unicast is show n, and I Pv 4 unicast global ( 3a) and policy ( 3b) com m ands ar e list ed w it hin t he m ode. Under t he addr ess fam ily subm ode, t he com m ands of Gr oup 3a, w ill appear fir st . These ar e com m ands t hat ar e global t o t his addr ess fam ily . Follow ing t he Gr oup 3a com m ands ar e t he com m ands of Gr oup 3b. These com m a nds im plem ent t he policy t o t he neighbor s for t hat addr ess fam ily. Befor e any policy is defined t o a neighbor under an addr ess fam ily , t he follow ing neighbor should be act iv at ed for t hat addr ess fam ily . The com m and t o do t hat is as follow s:

router bgp address-family afi sub-afi neighbor x.x.x.x activate ... exit-address-family exit The new configur at ion st r uct ur e look s lik e t his:

router bgp 1 bgp deterministic-med bgp bestpath med confed neighbor ebgp peer-group neighbor 1.1.1.1 remote-as 1

-> -> -> ->

Global Global Peer group defn, Global Neighbor defn, Global

138

neighbor 2.2.2.2 remote-as 2 neighbor 3.3.3.3 remote-as 3 ! address-family ipv4 unicast unicast bgp scan-time 45 aggregate-address 50.0.0.0 255.255.0.0 neighbor ebgp activate IPv4 uni neighbor ebgp route-map ebgp-ucast-out out policy neighbor 1.1.1.1 activate IPv4 uni neighbor 1.1.1.1 peer-group ebgp IPv4 uni neighbor 1.1.1.1 route-map peer-ucast-in in policy neighbor 2.2.2.2 activate IPv4 uni neighbor 2.2.2.2 route-reflector-client unicast neighbor 3.3.3.3 activate IPv4 uni neighbor 3.3.3.3 peer-group ebgp IPv4 uni ! address-family ipv4 multicast network 100.0.0.0 multicast redistribute dvmrp route-map redist-map multicast neighbor ebgp activate IPv4 mult neighbor 1.1.1.1 peer-group ebgp IPv4 mult neighbor 1.1.1.1 route-map peer-mcast-in in policy neighbor 3.3.3.3 peer-group ebgp IPv4 mult exit-address-family !

-> Neighbor defn, Global -> Neighbor defn, Global -> Address family IPv4 -> Global to IPv4 unicast -> Global to IPv4 unicast -> Activate neighbor for -> Peer group IPv4 unicast -> Activate neighbor for -> Neighbor membership -> Neighbor IPv4 unicast -> Activate neighbor for -> RR client - IPv4-> Activate neighbor for -> Neighbor membership -

-> Address family submode -> Global to IPv4 -> Global to IPv4 -> Activate Neighbor for -> Neighbor membership -> Neighbor IPv4 multicast -> Neighbor membership -

For m or e det ails on t hese com m ands, r efer t o t he docum ent at ion on Cisco. com.

Com pa rison Bet w een Old a nd N ew St yles The follow ing sect ions com par e t he old and new st y les of BGP com m ands, specifically show ing t he im pr ov em ent s t o t he CLI w hen using t he addr ess fam ily gr oupings m ent ioned pr ev iously . a ct iv a t e

139

Old st yle: This com m and w as not pr esent in I OS Soft w ar e Release 12.0S. The neighbor w as act iv at ed for I Pv 4 BGP aut om at ically . How ever , if t he m ult ipr ot ocol BGP w er e t o be enabled, t he nlri keyw ord in t he n e ig h b or com m and w as used:

Router(config-router)#neighbor 1.2.3.4 remote-as 10 nlri unicast multicast I f t he n lr i k ey w or d w as not specified, t he r out er ex changed I Pv 4 pr efix es only . How ev er , if t he n lr i k ey w or d w as specified w it h t he m u lt ica st opt ion only, only t he I Pv 4 m ult icast session w as act iv at ed. This w ay , only t he unicast , or only t he m ult icast , or bot h sessions could be act iv at ed. Addr ess fam ily st y le: To enable an addr ess fam ily for t he neighbor , t he a ct iv a t e com m and is used in t he r out er configur at ion or addr ess fam ily subm ode. The neighbor s t hat ar e defined under t he r out er configur at ion m ode aut om at ically ar e act iv at ed for I Pv 4. For all ot her addr ess fam ilies, t he neighbor s m ust be act iv at ed ex plicit ly . To deact iv at e a neighbor for an addr ess fam ily , use t he no form of t his com m and:

Router(config-router)#address-family ipv4 multicast Router(config-router-af)# neighbor 1.2.3.4 activate n e t w or k Old st yle: I n t he old- st yle CLI , t he w ay t o adver t ise a net w or k over BGP or m ult ipr ot ocol BGP w as t o use t he n e t w or k com m and. The com m and had an NLRI ex t ension t o specify w het her t he net w or k w as t o be adv er t ised ov er BGP or m ult ipr ot ocol BGP or bot h. The absence of t he n lr i keyw or d im plied I Pv4 unicast BGP only —for ex am ple:

Router(config)#router bgp 10 Router(config-router)#network 1.0.0.0 mask 255.0.0.0 nlri unicast multicast Router(config-router)#network 2.0.0.0 mask 255.0.0.0 Router(config-router)#network 3.0.0.0 mask 255.0.0.0 nlri multicast Addr ess fam ily st y le: I n t his st y le, t he pr esence of t he addr ess fam ily subm ode obv iat es t he need for t he nlri k ey w or d. To adv er t ise a net w or k ov er I Pv 4 BGP, t he n e t w or k com m and m ust be specified in r out er configur at ion m ode. For t he net work t o be adv er t ised ov er t he m ult ipr ot ocol BGP session, t he n e t w o r k com m and m ust specified under t he I Pv4 m ult icast addr ess fam ily subm ode. The follow ing adv er t ises a net w or k t o all t he neighbor s in t he I Pv 4 addr ess fam ily :

Router(config-router)#network 1.0.0.0 mask 255.0.0.0 Router(config-router)#network 2.0.0.0 mask 255.0.0.0 To advert ise a net w ork in t he I Pv4 m ult icast address fam ily, do t his

140

Router(config-router)#address-family ipv4 multicast Router(config-router-af)#network 1.0.0.0 mask 255.0.0.0 Router(config-router-af)#network 3.0.0.0 mask 255.0.0.0 This w ay , net w or k s can be adv er t ised independent ly for t he BGP and m ult ipr ot ocol BGP sessions Pe e r Gr ou ps Old st yle: A peer gr oup w as defined in r out er configur at ion m ode. To enable t he peer group t o ex change m ult ipr ot ocol BGP pr efix es, t he n lr i k ey w or d w as used. Using t he nlri k ey w or d, y ou could specify BGP or m ult ipr ot ocol BGP. I f t he n lr i k ey w or d w as not specified, it im plied only I Pv4 unicast BGP. The peer gr oup m em ber s aut om at ically inher it ed t he unicast or m ult icast capabilit y of t he peer gr oup.

Router(config)#router bgp 10 Router(config-router)#neighbor test peer-group nlri unicast multicast Router(config-router)#neighbor 1.2.3.4 remote-as 20 Router(config-router)#neighbor 1.2.3.4 peer-group test Addr ess fam ily st y le: The peer gr oup is defined under r out er configur at ion m ode. How ev er because w e hav e addr ess fam ily subm odes, t he need for t he nlri keyw or d is obviat ed. The peer gr oup or it s m em ber s need t o be act ivat ed in t he I Pv4 m ult icast addr ess fam ily t o enable t he exchange of m ult icast BGP pr efixes. As w it h t h e n e igh bor com m and, a peer gr oup and it s m em ber s ar e act iv at ed by default for I Pv 4 BGP. This behav ior can be ov er r idden by t he no for m of t he a ct iv a t e com m and.

Router(config)#router bgp 10 Router(config-router)#neighbor test peer-group Router(config-router)#neighbor 1.2.3.4 remote-as 20 Router(config-router)#neighbor 1.2.3.4 peer-group test Router(config-router)#address-family ipv4 multicast Router(config-router-af)#neighbor test activate Router(config-router-af)#neighbor 1.2.3.4 peer-group test Rou t e M a ps Old st yle: A single r out e m ap w as used t o specify policies for all addr ess fam ilies. This r out e m ap t hen w as applied eit her as t he inbound or t he out bound r out e of a peer or a peer gr oup. Rout ing policies r elat ing t o t he t w o addr ess fam ilies t hat could be car r ied in t he BGP session ( I Pv4 unicast and I Pv4 m ult icast ) w er e r epr esent ed using a single rout e m ap by specifying t he m a t ch n lr i keyword on t he rout e m ap par agr aph. The m a t ch n lr i clause in a r out e m ap had t he follow ing sem ant ics:

match nlri multicast match nlri multicast unicast multicast)

-> Matches only multicast updates -> Matches any update (both unicast &

141

match nlri unicast or (match nlri unspecified)

-> Matches only unicast updates.

The follow ing ex am ple show s how t o configur e BGP so t hat any m ult icast r out es fr om neighbor 1.1.1.1 w ill be accept ed if t hey m at ch access list 1:

router bgp 109 neighbor 1.1.1.1 remote-as 1 nlri unicast multicast neighbor 1.1.1.1 route-map in filter-some-multicast ! route-map filter-some-multicast match nlri multicast match ip address 1 ! Addr ess fam ily st y le: One of t he im por t ant r easons for m igr at ing configur at ions fr om t he old m ode t o t he addr ess family m ode is t hat rout ing policies expressed using t he m a t ch n lr i keyw or d in t he par agr aphs of t he r out e m ap soon becom e unm anageable w hen com plicat ed and differ ing policies hav e t o be ex pr essed for differ ent addr ess fam ilies. The new I OS Soft w ar e ver sions m ust suppor t m or e t han t w o addr ess fam ilies, w hich r equir es an elegant w ay of configur ing policies for each addr ess fam ily. Having one r out e m ap for expr essing all policies w as seen t o be inelegant and nonscalable. The int r oduct ion of a new par ser m ode for each addr ess fam ily facilit at ed t he int r oduct ion of a new w ay of configur ing policies on a per- AF basis— t hat is, sim ply use a r out e m ap st at em ent for t he neighbor under each of t he addr ess fam ily m odes. Not only can r out e m aps be specified on a per- addr ess- fam ily basis, but all t he filt er ing r ules ( pr efix list , dist r ibut e list , and AS pat h filt er list ) can be t oo. Wit h t he new m ode of configur ing policies, t he keyw or d nlri is not r equir ed; t he par ser w ould r ej ect t he occur r ence of t he m a t ch n lr i clause in t he rout e m ap par agr aph. The pr eceding ex am ple policy can be ex pr essed in t he addr ess fam ily st y le as follows:

router bgp 109 neighbor 1.1.1.1 remote-as 1 address-family ipv4 multicast neighbor 1.1.1.1 activate neighbor 1.1.1.1 route-map filter-some-multicast in ! route-map filter-some-multicast permit 10 match ip address 1 ! Re dist r ibu t ion Old st yle: Redist r ibut ion is t he pr ocess of im por t ing r out es fr om one r out ing pr ot ocol t o anot her . When r out es ar e r edist r ibut ed int o BGP using t he re dist r ibu t e st at em ent , t he BGP t able ( Loc - RI B, as specified in RFC 1771) int o w hich t he rout es hav e t o be r edist r ibut ed m ust be specified. This BGP t able can be eit her t he unicast BGP t able or t he m ult icast BGP t able. The old st yle of specifying t he t able int o w hich

142

a rout e m ust go used t he se t n lr i clause in t he r edist r ibut ion r out e m ap. The se t nlri clause in t he r edist r ibut ion r out e m ap has t he follow ing sem ant ics:

set nlri multicast the set nlri unicast multicast set nlri unicast or unicast (set nlri unspecified)

-> redistributes the matching prefix to multicast table. -> redistributes the matching prefix to both unicast and multicast tables. -> redistributes matching prefix to table.

The follow ing exam ple show s how t o configur e r edist r ibut ion in BGP so t hat all connect ed pr efixes m at ching access list 1 in t he r out ing t able go int o t he m ult ipr ot ocol BGP t able:

router bgp 109 redistribute connected route-map mbgp-source-map ! route-map mbgp-source-map match ip address 1 set nlri multicast ! Addr ess fam ily st y le: Wit h t he int r oduct ion of addr ess fam ily m ode, t he addr ess fam ily m ode under w hich t he r e dist r ibu t e com m and is specified det er m ines t he t able int o w hich t he r edist r ibut ed pr efix es ar e inj ect ed. So, if t he r e dist r ibu t e st at em ent is under t he a ddr e ss- f a m ily ip v 4 m u lt ica st m ode, t he redist ribut ed pr efix es get s int o t he m ult ipr ot ocol BGP t able, and so on. Hence, t he old st y le r edist r ibut ion configur at ion t r anslat es t o t he follow ing in t he addr ess fam ily st y le:

router bgp 109 ! address-family ipv4 multicast redistribute connected route-map mbgp-source-map ! route-map mbgp-source-map match ip address 1 ! Not e t hat w it h t he int r oduct ion of t he r e dist r ibu t e st at em ent w it hin t he addr ess fam ily m ode, t he clause se t n lr i is not r equir ed and t he par ser r ej ect s t he pr esence of se t n lr i in t he r out e m ap par agr aph. Rou t e Re f le ct or Old st yle: I n t he old st y le CLI , a r out e r eflect or w as specified for all address fam ilies and t he r out e r eflect or r eflect ed r out es of all addr ess fam ilies t hat it has negot iat ed w it h it s client s. The r out e r eflect or k now s t hat it m ust r eflect r out es fr om and t o

143

client s by specify ing r out e - r e f le ct o r- clie nt for a par t icular neighbor or iBGP peer gr oup. The follow ing is an exam ple in w hich t he iBGP peer 1.1.1.1 is m ade a r out er eflect or client for bot h unicast and m ult icast I Pv 4 pr efix es:

router bgp 109 neighbor 1.1.1.1 remote-as 109 nlri unicast multicast neighbor 1.1.1.1 route-reflector-client ! Addr ess fam ily st y le: I n t he addr ess fam ily st y le of configur ing r out er r eflect or s, t he fact t hat a peer ( or peer- gr oup) is a r out e- r eflect or client is addr ess fam ily dependent ; t his m eans t hat it is configu r ed in t he addr ess fam ily m ode. That is, j ust because a peer is a r out e- reflect or client in I Pv4 unicast m ode does not m ake it aut om at ically a r out e r eflect or client in t he I Pv 4 m ult icast m ode. This m ust be specified by configur ing t he client in t he I Pv 4 mult icast addr ess fam ily m ode. Thus, t he pr ev ious configur at ion, w hich m ak es 1.1.1.1 a client for bot h unicast and m ult icast client , can be ex pr essed as follow s:

router bgp 109 neighbor 1.1.1.1 remote-as 109 neighbor 1.1.1.1 route-reflector-client ! address-family ipv4 multicast neighbor 1.1.1.1 activate neighbor 1.1.1.1 route-reflector-client ! This giv es t he oper at or flex ibilit y t o m ak e a r out er t he r out e r eflect or for only cer t ain addr ess fam ilies. As such, t he r out e- r eflect or t opologies for differ ent addr ess fam ilies can be differ ent in t he cor e. Aggr e ga t ion Old st yle: I n t he old- st y le CLI , an m ult ipr ot ocol BGP aggr egat e is configur ed t he sam e w ay t hat y ou configur e for unicast BGP using t he a g g r e g a t e - a ddr e ss com m and. The a ggr e ga t e - a ddr e ss c om m and w as enhanced t o specify w het her t he aggr egat e addr ess should be applied t o BGP or m ult ipr ot ocol BGP using t he n lr i k ey w or d in t he a ggr e ga t e - a ddr e ss com m and. The follow ing is an ex am ple of gener at ing aggr egat es in t he m ult ipr ot ocol BGP t able:

router bgp 109 aggregate-address 174.0.0.0 255.0.0.0 as-set nlri multicast ! The NLRI opt ions t hat can be specified on an aggr egat e addr ess com m and ar e m u lt ica st , u n ica st m u lt ica st , and u n ica st , w hich gener at es aggr egat es in t he m ult ipr ot ocol BGP t able, bot h BGP and m ult iprot ocol BGP t ables, or j ust t he BGP t able, r espect iv ely . The absence of t he nlri keyword in t he a g g r e g a t e - a ddr e ss com m and causes t he aggr egat e t o be gener at ed in t he unicast BGP t able.

144

Addr ess fam ily st y le: The pr esence of individual m odes for differ ent addr ess fam ilies elim inat es t he need for t he n lr i k ey w or d in t he a ggr e ga t e - a d d r e ss com m and. The addr ess fam ily m ode under w hich t he aggr egat e is specified det er m ines t he t able w her e t he aggr egat ed pr efix should be gener at ed. Hence, t he pr ev ious aggr egat e can be gener at ed in addr ess fam ily m ode as follow s:

router bgp 109 address-family ipv4 multicast aggregate-address 174.0.0.0 255.0.0.0 as-set !

Upgra ding t o t he N ew CLI For a com plet e list of t he com m ands av ailable in t he new CLI , consult t he I OS Com m and docum ent at ion at Cisco. com. Upgrading t he CLI requires inst alling an im age t hat suppor t s t he r eor ganized BGP par ser ; t hese ar e 12.0ST and im ages fr om 12.1 onw ar d. The exist ing BGP com m ands suppor t ed under t he old CLI can be conv er t ed int o t he new st y le quit e easily . To have a sm oot h upgr ade pat h, suppor t has been added t o par se t he old 12.0Sst y le com m ands t hat had t he nlri keyw or d. These com m ands ar e • • • •

n e ig h b o r n e t w or k a ggr e ga t e se t and m a t ch nlri in rout e m aps

The only cav eat is t hat t he old st y le com m ands can be used as long as no new feat ur es need t o be act iv at ed. I n t hat ev ent , t he old st y le BGP com m ands need t o be t r anslat ed t o t he new st yle ( discussed m om ent ar ily) . When in t he new m ode, t he old com m ands no longer w ill be accept ed. I n ot her w or ds, t he old ( NLRI st y le) and t he new ( addr ess fam ily st y le) com m ands cannot be m ix ed in t he sam e configur at ion. As t he r out er par ses t he com m ands, it locks int o a m ode based on t he fir st com m and t hat uniquely ident ifies a m ode ( for exam ple, an n lr i keyw or d or an addr ess fam ily k ey w or d) . Aft er it is lock ed int o a m ode, t he par ser only accept s com m ands t hat ar e com pat ible w it h t hat m ode. To m igr at e t o t he new com m and set , t he bgp u pgr a de - cli com m and m ust be ent ered in rout er configurat ion m ode:

Router(conf-rout)#bgp upgrade-cli This com m and t r anslat es t he old configur at ion t o t he new one. A w r it e m e m or y m ust be done t o sav e t his configur at ion. ( The bgp u pgr a de - cl i com m and is not show n in t he sh ow r u n n ing- con f ig . )

145

Exam ples of t he N ew CLI in Use This sect ion giv es som e ex am ples of t he r ev ised CLI . They hav e been t ak en fr om r eal, live r unning configur at ions w or king in net w or ks t oday. The fir st ex am ple show s how BGP suppor t ing I Pv 6 is configur ed. This r out er is a part of t he 6BONE, t he I Pv 6 ex per im ent al back bone:

! router bgp 4608 no synchronization no bgp default ipv4-unicast ! IPv4 uni bgp log-neighbor-changes bgp dampening neighbor UPSTREAMS peer-group neighbor iBGP-peers peer-group neighbor 2001:200:0:1805::2 remote-as 2500 neighbor 3FFE:800::FFF9:0:0:9 remote-as 4554 neighbor 3FFE:C00:E:11::1 remote-as 109 neighbor 3FFE:C00:800F:FFFF::2 remote-as 4608 ! address-family ipv6 ! family neighbor UPSTREAMS activate ! neighbor UPSTREAMS send-community neighbor UPSTREAMS prefix-list block-my-sla in neighbor UPSTREAMS prefix-list announce-my-sla out neighbor iBGP-peers activate ! neighbor iBGP-peers next-hop-self neighbor iBGP-peers send-community neighbor 2001:200:0:1805::2 peer-group UPSTREAMS neighbor 3FFE:800::FFF9:0:0:9 peer-group UPSTREAMS neighbor 3FFE:C00:E:11::1 peer-group UPSTREAMS neighbor 3FFE:C00:800F:FFFF::2 peer-group iBGP-peers network 3FFE:C00:800F::/48 exit-address-family !

allow more than

set up IPv6 addrdefine UPSTREAMS

define iBGP peers

Not ice how t he configur at ion is separ at ed int o dist inct par t s. This r out er could support I Pv4 BGP peering as w ell if t he or ganizat ion chose t o do t his. All t hat is configur ed her e is I Pv 6 suppor t , so t he I Pv 6- specific com m ands ar e included under t he I Pv 6 addr ess fam ily . Not e t he a ct iv a t e dir ect iv e—t his is needed t o act iv at e any peer ing inside an addr ess fam ily. The second exam ple is a sim ple MPLS/ VPN exam ple t aken fr om t he Cisco Syst em s’ I SP Wor k shop pr ogr am . Tw o VPNs hav e been set up acr oss t he I SP back bone; t he I SP backbone is m ade up of t hree P rout ers and t hree PE rout ers. The P rout ers form t he I SP cor e, and t he PE r out er s deal w it h t he VPN. This configur at ion is fr om one of t he PE r out er s at t he edge of t he I SP net w or k ( PE st ands for “ pr ov ider edge” ) . The t w o BGP peer ings ar e w it h t he ot her t w o PE r out er s configur ed in t his net w or k . The P rout ers t ake no part in t he iBGP configur at ion r equir ed for t he VPN.

146

! router bgp 100 no synchronization neighbor 200.200.0.12 remote-as 100 neighbor 200.200.0.12 update-source Loopback0 neighbor 200.200.0.13 remote-as 100 neighbor 200.200.0.13 update-source Loopback0 no auto-summary ! address-family ipv4 vrf VPN1 ! define VPN1 and its addresses no auto-summary no synchronization network 172.16.1.0 mask 255.255.255.0 network 172.16.2.0 mask 255.255.255.252 exit-address-family ! address-family ipv4 vrf VPN2 ! define VPN2 and its addresses no auto-summary no synchronization network 172.16.1.0 mask 255.255.255.0 network 172.16.2.0 mask 255.255.255.252 exit-address-family ! address-family vpnv4 ! now set up the VPN address-fam neighbor 200.200.0.12 activate neighbor 200.200.0.12 send-community extended neighbor 200.200.0.13 activate neighbor 200.200.0.13 send-community extended no auto-summary exit-address-family ! Not ice again how t he configur at ion is neat ly separ at ed, m aking it m uch easier t o read and underst and. Wit h t he old- st yle CLI , it w ould have been ver y har d t o configure t his.

Su m m a r y This chapt er cov er ed t he m aj or r out ing pr ot ocol feat ur es t hat I SPs need t o consider for t heir back bone net w or k s. I t discussed gener al r out ing feat ur es t hat t he r out er r equir es, cov er ed t he basic back gr ound for using I GPs and BGP, and show ed t he good design pr act ices for im plem ent ing I GPs. The chapt er t hen consider ed t he I SPfriendly feat ur es of BGP in consider able det ail. Most of t hese feat ur es ar e new or r ecent addit ions t o I OS Soft w ar e and w ar r ant in- dept h discussion. The chapt er also looked at t he policy feat ures available for BGP in I OS Soft ware and a m echanism for doing I P t raffic account ing using BGP at t r ibut es. The chapt er finished by look ing at t he new CLI available for BGP, designed t o suppor t m ult iple addr ess fam ilies.

147

En dn ot e s 1. This sect ion or iginally w as w r it t en by Mar k Tur ner ( m arkt @cisco. com) an d can be found on Cisco. com at ht t p: / / w w w .cisco.com / w ar p/ public/ 459/ 25.sht m l. 2. Pr ivat e aut onom ous syst em s ar e t hose AS num b er s bet w een 64512 and 65534. AS 65535 is reserved by I ANA. 3. Thanks t o Enke Chen for pr oviding m ost of t he t ext descr ibing BGP peer gr oups. 4. The cor e of t his sect ion is by Br uce R. Babcock ( bbabcock @cisco. com) and Enke Chen. 5. This sect ion is based on t he addr ess fam ily r elease not es or iginally w r it t en by Srihari Ram achandra.

148

Cha pt e r 4 . Se cur it y This chapt er on Cisco I OS Soft w ar e secur it y feat ur es assum es t hat t he I SP engineer has a w orking gr asp of t he fundam ent als of sy st em secur it y . I f not , t he m at er ials list ed in t he r efer ence sect ion should be r eview ed fir st t o help gain an under st anding of som e of t he fundam ent als. I t is also im por t ant t o not e t hat t he follow ing sect ions ar e int ended t o supplem ent , not r eplace, Cisco docum ent at ion. I t is assum ed t hat t he I SP engineer w ill r ead such docum ent at ion in par allel w it h t his chapt er . This chapt er descr ibes t ools t hat all I SPs should consider for t heir ov er all secur it y ar chit ect ur es. Most of t hese t ools ar e passiv e t ools. When configur ed, t hey w ill help pr event secur it y pr oblem s fr om happening and m ake it m or e difficult t o cause m ischief on an I SP’s net w ork. The chapt er is split int o sev er al sect ions. These cov er I SP secur it y in gener al, t he pr ocess of secur ing t he r out er , t he r out ing pr ot ocol, and t he net w or k . Aft er t his, t he chapt er look s in m or e det ail at how t o im plem ent t he unicast r ev er se pat h for w ar ding ( RPF) check and how t o use Com m it t ed Access Rat e ( CAR) t o deal w it h at t ack s in pr ogr ess. The chapt er concludes w it h a br ief discussion on how t o appr oach I SP secur it y and how t o r eact t o incident s under w ay . The v ar ious sect ions are t it led as follow s: • • • • • •

Secur ing t he Rout er Secur ing t he Rout ing Pr ot ocol Secur ing t he Net w or k Unicast RPF Using CAR t o Count er DoS At t ack s React ing t o Secur it y I ncident s

Subdiv iding t he t ex t in t his w ay suppor t s t he for m at ion of a clear st r at egy for an I SP t o w or k on t o m inim ize t he effect s of a secur it y incident . Secur ing an ent er pr ise net w or k fr om I nt er net t hr eat s is easy w hen com par ed w it h t he pr oblem s of secur it y facing an I SP. When an ent er pr ise net w or k connect s t o t he I nt er net , t her e is essent ially one I nt er net secur it y pr oblem—prot ect ing your net w ork fr om out side int r usion. To achiev e I nt er net secur it y obj ect iv es, an ent erprise m ust balance t r ade- offs w it h connect iv it y , accessibilit y , per for m ance, and secur it y . An I SP’s secur it y concer ns ar e m uch br oader . The I SP business is all about t r anspar ent , cost - effect ive, and high- per for m ance I nt er net connect iv it y . Secur it y m easur es w ill affect t he I SP’s net w or k oper at ion, yet secur it y t hr eat s ar e r eal and need t o be pr ot ect ed against . I SPs ar e v er y v isible t ar get s for m alicious, v indict iv e, and cr im inal at t ack s, so t hey m ust pr ot ect t hem selv es, help pr ot ect t heir cust om er s, and m inim ize t he r isk of t heir cust om er s becom ing pr oblem s t o ot her s on t he I nt er net ( see Figur e 4- 1) . Fig u r e 4- 1 . An I SP’s Se cu r it y Th r e a t s Com e f r om All D ir e ct ion s

149

N OTE No net w or k is ever fully secu r e or pr ot ect ed. Ther e is alw ays a r isk fact or , especially for I SPs w hose j ob it is t o m ov e ot her people’s pack et s acr oss t heir net w or k s ( ot her people’s pack et s ar e cust om er s’ pack et s) . What I SPs can ex pect t o hav e ar e t ools t o build r esist an ce. I SPs need t o r esist at t ack s and int r usion at t em pt s t o t heir net w or k s, r esist ing long enough for int er nal secur it y- react ion pro cedur es t o be act iv at ed t o t r ack t he incident and apply count er m easur es. The t ools descr ibed in t his chapt er help t o build r esist an ce and secu r it y.

Se cu r in g t h e Rou t e r Ensur ing t hat each dev ice on t he net w or k is as secur e as possible is one of t he fir st secur it y t ask s. This m eans t hat t he m ode of access, feat ur es/ ser v ices t ur ned on, and t he configur at ion on t he r out er all need t o be r eview ed fr om a m ind- set of secur it y . Sim ple secur it y pr inciples such as “ if you ar e not using it , do not t ur n it on” can be applied t o r out er s and sw it ches j ust as easily as t hey can be applied t o a UNI X ser v er . I ssues w it h how people gain access t o t he net w or k dev ices and how t hat access is audit ed need t o be consider ed as w ell. I n t oday ’s I nt er net , at t ack s dir ect ed at t h e n et wor k dev ice it self ar e a r eal t hr eat ; hence, t ools t o help r ide out t he dir ect at t ack m ust be consider ed. Secur ing t he r out er / sw it ch is t he fir st t hing t hat needs t o be com plet ed befor e any t ools used t o defend t he net w or k ar e im plem ent ed. Ot herwise, t he ro ut er / sw it ch becom es t he back door or Achilles’ heel of t he net w or k .

150

Unneeded or Risk y Globa l Services Many of t he built - in ser vices in I OS Soft w ar e ar e not needed in an I SP backbone env ir onm ent . These feat ur es should be t ur ned off in y our default configur a t ion. Turn t hem on only if t her e ar e ex plicit r equir em ent s.

no no no no no

ip finger service pad service udp-small-servers service tcp-small-servers ip bootp server

Som e of t hese ser v ices w ill be pr econfigur ed in I OS Soft w ar e ( depending on t he release) and can be t ur ned off by default , but I SPs should ensur e t hat t hey ar e explicit ly t ur ned off in t he m ast er configur at ion files. The w hit epaper / field aler t “ Defining St r at egies t o Pr ot ect Against UDP Diagnost ic Por t Den ial- of- Ser v ice At t ack s” descr ibes t he secur it y r isk and pr ov ides point er s t o public discussion on t he I SP Oper at ions for um s. This w hit epaper is post ed at ht t p: / / w w w .cisco.com / w ar p/ public/ 707/ 3.ht m l and is v isible t o t he general public ( in ot her words, no Cisco. com login is r equir ed) . The ser vices t hat should not be used ar e descr ibed in m or e det ail her e. For supplem ent ar y infor m at ion, r efer t o t he I OS Soft w ar e docum ent at io n. •

• •

•

n o ip f in g e r disables t he pr ocess t hat list ens for finger r equest s fr om r em ot e host s. Only I SP per sonnel nor m ally access t he back bone r out er s, and t her e are ot her and bet t er m eans of t racking who is logged in. Besides, finger is a known securit y risk in t he I nt er net because it div ulges det ailed infor m at ion on people logged int o a syst em . ( I n I OS Soft w ar e r eleases ear lier t han 12.0, t his com m and w as n o se r v ice f in g e r. ) se r v ice pa d is not r equir ed. I t r efer s back t o t he days of X.25 net w or king; in r ecen t v ersions of I OS Soft w are, n o se r v ice p a d has becom e t he default . The sm all TCP and UDP ser v er s ar e t hose w it h por t num ber s below 20. Ty pical ser v ices include echo and discar d por t s, w it h t he for m er echoing all packet s sent t o it and t he lat t er t hr ow ing aw ay all pack et s sent t o it . I f t hey ar e enabled and act iv e, t hey could be used t o car r y out successful DoS at t ack s —t heir use w ill div er t CPU r esour ces aw ay fr om ot her pr ocesses, w hich w ill cause pr oblem s for t he connect ed net w or k s and I nt er net ser v ice depende nt on t hat rout er. The boot p ser v ice pr ov ides suppor t for sy st em s t hat find t heir configur at ion using t he boot p pr ocess. This com m only is used in LANs ( X t er m inals com m only use boot p, for ex am ple) and nev er on t he WAN. I t should be disabled.

Un n e e de d or Risk y I n t e r f a ce Se r v ice s Som e I P feat ur es ar e gr eat for cam pus LANs but do not m ake sense on an I SP back bone. Abuse of t hese funct ions by “ cy ber punk s” incr eases t he I SP’s secur it y r isk .

151

All int er faces on an I SP’s back bone r out er should hav e t he follow ing configured by default :

no ip redirects no ip directed-broadcast no ip proxy-arp An ex planat ion for each of t hese int er face com m ands follow s: • •

•

n o ip r e d ir e ct s m eans t hat t he r out er w ill not send r edir ect m essages if t he I OS Soft w ar e is for ced t o r esend a pack et t hr ough t he sam e int er face on w hich it w as r eceiv ed. n o ip dir e ct e d- b r oa d ca st m eans t hat t he t r anslat ion of dir ect ed br oadcast t o phy sical br oadcast s is disabled. I f enabled, a br oadcast t o a par t icular net w or k could be dir ect ed at a r out er int er face, pr oducing effect s t hat could be undesir able and pot ent ially har m ful. An ex am ple of t he ill effect s of dir ect ed br oadcast s being enabled is t he so- called sm ur f at t ack . For m or e infor m at ion about sm ur f, see Cr aig Huegen’s sm ur f Web sit e at ht t p: / / w w w . pent ics. net / . Since I OS Soft ware 12.0, n o ip d ir e ct e d br oa dca st has becom e t he default on all r out er int er faces. n o ip p r ox y - a r p disables t he proxy ARP funct ion. Proxy ARP is defined in RFC 1027 and is used by t he r out er t o help host s w it h no r out ing capabilit y det er m ine t he MAC addr esses of host s on ot her net w or k s or subnet s. For ex am ple, if t he r out er r eceiv es an ARP r equest for a host t hat is not on t he sam e int er face as t he ARP r equest sender , and if t he r out er has all of it s r out es t o t hat host t hr ough ot her int er faces, it gener at es a pr ox y ARP r eply pack et giv ing it s ow n local MAC addr ess. The host t hat sent t he ARP r equest t hen sends it s pack et s t o t he r out er , w hich for w ar ds t hem t o t he int ended host . This is basically t he r out er say ing t hat it k now s how t o get t o t he host being r equest ed by t he ARP r equest sender . This configur at ion could be undesir able on an I SP back bone because I SP net w or k s car r y ex plicit r out ing infor m at ion for all dest inat ions on t he Int er net . Rely ing on pr ox y ARP could r esult in an I nt er net back bone r out er car r y ing a huge MAC addr ess t able, pot ent ially hinder ing t he r out er ’s per for m ance.

Cisco D iscove r y Pr ot ocol The Cisco Discov er y Pr ot ocol ( CDP) is used for som e net w or k- m anagem ent funct ions on m ainly cam pus LANs ( alt hough it oft en is used on sm aller office LANs) . I t allow s any sy st em on a dir ect ly connect ed segm ent t o discov er t hat t he equipm ent is m anufact ur ed by Cisco ( a r out er , sw it ch, and so on) and t o det er m ine infor m at ion such as t he m odel num ber and t he soft w are version running. This is very useful in som e inst ances, but it does not m ake ver y m uch sense on an I SP’s backbone because t he I SP should be com plet ely aw ar e of w hat is inst alled and w hat soft w ar e versions are running! Th e infor m at ion av ailable fr om CDP does not t hr eat en secur it y , as such, but at t acker s could use CDP as an int elligence t ool. Know n bugs ar e t ar get ed t hr ough t he discov er y of t he specific v er sion of soft w ar e on t he net w or k dev ice. As can be seen in Ex am ple 4- 1, CDP easily highlight s t he specific I OS Soft w ar e v er sion on t he rout er.

152

Ex a m p le 4- 1 I n f or m a t ion Th a t Ca n Be Ga in e d f r om CD P Defiant#show cdp neighbors detail ------------------------Device ID: Excalabur Entry address(es): IP address: 4.1.2.1 Platform: cisco RSP2, Capabilities: Router Interface: FastEthernet1/1, Port ID (outgoing port): FastEthernet4/1/0 Holdtime : 154 sec Version : Cisco Internetwork Operating System Software IOS (tm) RSP Software (RSP-K3PV-M), Version 12.0(9.5)S, EARLY DEPLOYMENT MAINTENANCE INTERIM SOFTWARE Copyright (c) 1986-2000 by cisco Systems, Inc. Compiled Fri 03-Mar-00 19:28 by htseng Defiant# CDP can be disabled using t his global com m and:

no cdp run I f CDP is r equir ed on an I SP’s net w or k, it is possible t o leave CDP r unning but disable t he pr ot ocol on a per- int er face basis. This int er face configur at ion com m and disables CDP on a par t icular int er face:

no cdp enable I t is st r ongly r ecom m ended t hat CDP be disabled on all public - facing int er faces, w het her t hose face ex change point s, upst r eam I SP, or ev en cust om er s. Not e t hat CDP is enabled by default on 11.1CA, 11.1CC, and m or e r ecent soft w ar e.

Login Ba n n e r s Much over looked, but im por t ant in t he age of t he com m er cial I SP, is t he ba n n e r login com m and. This feat ur e is par t of t he ba n n e r com m and set , w hich display s t ex t w hen user s connect t o t he r out er . ba n n e r login display s t ex t w hen a user fir st init iat es a Telnet session t o t he rout er. I t m ight seem t rivial, but a lack of a banner is as effect iv e a secur it y dev ice as a banner t elling connect ed sessions t hat only t hose t hat ar e aut hor ized ar e per m it t ed t o connect . Som e I SPs ar e using banner s w it h m essage cont ent sim ilar t o t he one t hat follow s. Any I SP should consider w het her it s int er est is ser ved best by including a banner w it h an official w ar ning or not hing at all. I t is good pr act ice not t o ident ify t oo m uch about t he syst em it self in t he banner . ( Things such as “ j oes- rout er” m ight not be such a good idea because t hey could giv e a hint about t he user / ow ner of t he syst em and any user I Ds or passw or ds on it .)

153

banner login ^ Authorized access only This system is the property of Galactic Internet Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +99 876 543210 for help. ^ Anot her t y pe of banner av ailable is t he ex ec banner , display ed at t he t im e a user has successfully aut hent icat ed and logged in. Ther e ar e sev er al ot her ex am ples of t he banner com m and, as can be seen in t he follow ing r out er out put :

alpha7200(config)#banner ? LINE c banner-text c, where ‘c' is a delimiting character exec Set EXEC process creation banner incoming Set incoming terminal line banner login Set login banner motd Set Message of the Day banner alpha7200(config)#banner For ex am ple, a not e t o all engineer ing st aff on a back bone r out er aft er t hey hav e logged int o t he rout er m ight be as follow s:

banner exec ^ PLEASE NOTE – THIS ROUTER SHOULD NOT HAVE A DEFAULT ROUTE! It is used to connect paying peers. These 'customers' should not be able to default to us. The configuration of this router is NON-STANDARD Contact Network Engineering +99 876 543234 for more information. ^

Use e n a ble se cr e t Use t he e n a b le se cr e t password in lieu of t he e n a ble pa ssw or d com m and. The encr y pt ion algor it hm Ty pe 7 used in e n a b le p a ssw or d and se r v ice p a ssw or de n cr y pt ion is r ev er sible. The e n a b le se cr e t com m and pr ov ides bet t er secur it y by st or ing t he enable passw or d using a nonr ev er sible cr y pt ogr aphic funct ion. The added layer of secur it y encr ypt ion t hat it pr ovides is useful in envir onm ent s w her e t he passw or d cr osses t he net w or k or is st or ed on a TFTP ser ver .

154

service password-encryption enable secret [removed] no enable password CAU T I O N Do not rem ove t he e n a b le p a ssw or d as in t he pr evious exam ple if t he boot ROMs or boot im age of t he r out er does not suppor t t he e n a b le se cr e t configur at ion. The use of e n a ble se cr e t is support ed in I OS Soft ware Re lease 11.0 and lat er . Wit h an older boot ROM and n o e n a b le p a ssw or d , it is possible t o gain access t o t he r out er w it hout supply ing any passw or d if t he r out er ends up r unning t he boot im age because of som e net w or k pr oblem or m alfunct ion. A net w or k ’s fir st line of defense is t he r out er s used, and any one w ant ing t o com pr om ise a net w or k m or e t han lik ely w ill st ar t w it h t he r out er r at her t han any sy st em behind t hat r out er ( w her e configur at ions m ight be st or ed) .

Alm ost all passw or ds and ot her aut hent icat ion st r ings in Cisco I OS Soft w ar e configur at ion files ar e encr y pt ed using t he w eak , r ev er sible schem e used for user passw or ds. To det er m ine w hich schem e has been used t o encr y pt a specific passw or d, check t he digit pr eceding t he encr y pt ed st r ing in t he configur at ion file. I f t hat digit is a 7, t he passw or d has been encr y pt ed using t he w eak algor it hm . I f t he digit is a 5, t he passw or d has been hashed using t he st r onger MD5 algor it hm . Even t hough e n a b le se cr e t is used for t he enable passw or d, do not for get se r v ice pa ssw or d- e n cr y p t ion so t hat t he r em aining passw or ds ar e st or ed in t he configur at ion w it h Type 7 encr ypt ion r at her t han in plain t ext . Weak encr ypt ion is bet t er t han none at all. For ex am ple, in t he follow ing configur at ion com m and, t he e n a b le se cr e t com m and has been hashed w it h MD5:

enable secret 5 $1$iUjJ$cDZ03KKGh7mHfX2RSbDqP. I n t his com m and, how ev er , t he passw or d has been encr y pt ed using t he w eak r ever sible algor it hm :

username jbash password 7 07362E590E1B1C041B1E124C0A2F2E206832752E1A01134D Because sev er al v er sions of code hav e been designed t o br eak t he w eak encr y pt ion on a Cisco r out er , I SPs ar e st r ongly encour aged t o use ot her st r at egies for passw or ds t hat ar e not pr ot ect ed by st r ong encr y pt ion. Cisco I OS Soft w ar e suppor t s Kerberos, TACACS+ , and RADI US aut hent icat ion ar chit ect ur es, so t he opt ion is open t o use AAA t o access t he r out er inst ead of hav ing user nam es on t he r out er it self.

155

Th e ide n t Fe a t u r e I dent ificat ion ( ident ) suppor t allow s y ou t o quer y a Tr ansm ission Cont r ol Pr ot ocol ( TCP) port for ident ificat ion. This feat ur e enables an insecur e pr ot ocol, descr ibed in RFC 1413, t o r epor t t he ident it y of a client init iat ing a TCP connect ion and a host r esponding t o t he connect ion. Figur e 4- 2 giv es an ex am ple of t he com m unicat ion pr ocess bet w een client and ser v er and show s how ident fit s in as t he “ aut hor izat ion” funct ion. No at t em pt is m ade t o pr ot ect against unaut hor ized quer ies. This com m and should be enabled only if t he consequences and t he adv ant ages in t he local sit uat ion ar e under st ood. Fig u r e 4- 2 . Ov e r v ie w of id e n t Pr ot ocol

ip ident Som e I SP bac kbone engineer s like ident . Ot her s do not . New I SP engineer s ar e r ecom m ended t o look int o ident , r ead t he RFC, t r y it , and see if it fit s as a secur it y t ool on t he back bone. ident is not suppor t ed in t he 12.0S fam ily but m ight be suppor t ed in 12.2S.

SN M P Se cu r it y At t he t im e of w r it ing, ev en w it h t he addit ional encr y pt ion added w it h SNMPv 3, m ost I SPs st ill use SNMPv1. The appar ent r eason t hat no one has m oved t o SNMPv3 is t hat t he r isk assessm ent has nev er out w eighed t he k ey encr y pt ion coor dinat ion require d for SNMPv 3’s full capabilit ies. The per ceiv ed w eak secur it y in SNMPv 1 depends on four int er linked configur at ions t hat m inim ize t he secur it y r isk: •

•

Re a d- on ly a cce ss— I SPs nev er configur e SNMP w it h w r it e access. All SNMP is configur ed for r ead only . This reduces t he r isk of int elligence gat her ing ( som eone r eading SNMP MI Bs) and DoS against SNMP ( flooding t he SNMP por t w it h pack et s) . Acce ss list p r ot e ct in g SN M P— Access list s r equir e t hat t he sour ce pack et of t he SNMP r equest m at ch t he ACL. Toget her w it h r ead- only access, t hese t w o st eps m ak e it ex t r em ely difficult for som eone t o do blind pr obes int o SNMP. I f t he per son k now s t he SNMP com m unit y , a spoofed pack et can get t hr ough t he ACL and can t r igger SNMP act iv it y . Yet , because SNMP is only in read- only m ode, t he v alue of t hat act iv it y is lim it ed.

156

N ot e I n a blind pr obe, som eone sends spoofed sour ce pack et s t o get t hr ough an ACL and does not ex pect a r esponse back . The at t ack er is guessing t hat t he infor m at ion sent w ill t r igger an ev ent or secur it y loophole.

•

•

Com p le x com m u n it y n a m e s— Com m unit y nam es, lik e any access passw or d, can be guessed. Because a com m unit y does not nam e passw or ds ( t hat is, y ou do not hav e t o t y pe t hem in t o connect ) , long, com plex , and m ix ed- char act er com m unit y nam es can be used. For ex ample, an SNMP com m unit y nam e of cisco is m uch easier t o guess t han a com plex com m unit y nam e of 123$# Cisco# $456. This com plex it y m ak es SNMP com m unit ies m or e r esist ant t o guessing. Com bining r ead- only access, ACLs, and com plex com m unit y nam es pr ov ides a high degr ee of r esist ance t o at t ack . SN M P a u t h e n t ica t ion f a ilu r e t r a p— Som e I SPs add a check t o t he t hr ee st eps used t o pr ot ect SNMPv 1. They do t his t hr ough t he SNMP aut hent icat ion failur e t r ap. When configur ed, SNMPv 1 sends an aut hent icat ion failur e t r ap whenev er a pack et w it h an incor r ect com m unit y st r ing is r eceiv ed. So, if som eone is spoofing t he ACL and t r y ing t o guess t he SNMP com m unit y , SNMP w ill send aut hent icat ion t r aps t o an SNMP t r ap ser v er . The ser v er t hat r eceiv es t he aut hent icat ion failur e t r aps t hen can t ak e t he addit ional st ep of aler t ing t he oper at ions or secur it y t eam t hat a num ber of SNMP aut hent icat ion failur es ar e happening on a specific dev ice.

Used t oget her , t hese four t echniques ar e t he pr im ar y defense used by m ost I SPs t oday . As SNMPv 3 becom es m or e w idely deploy ed, it is ex pect ed t hat a fift h t echnique of MD5 aut hent icat ion w ill be added t o t he sy st em of feat ur es used t o pr ot ect SNMP.

Using t he t r a p- source loopba ck 0 I SP best com m on pr act ices ( BCPs) has SNMP use t he net w or k dev ice’s loopback int er face as t he sour ce and dest inat ion addr esses for all SNMP t r affic. SNMP ser v er secur it y is easier t o m anage if t he SNMP t r aps or iginat ing fr om t he net w or k device use t he loopback int er face. This allow s SNMP t r aps t o pass pack et filt er s pr ot ect ing t he SNMP ser v er s ev en w hen t her e is a t opology change on t he net w or k . Ot her w ise, SNMP t r aps use t he sour ce addr ess of t he int er face in w hich t he t r ap leav es t he net w or k dev ice. For ex am ple, if a net w or k dev ice has t w o pat hs t o get t o t he SNMP ser v er—int erf aces Ser ial1 and Ser ial3—and t he pat h t hr ough Ser ial3 goes dow n, SNMP t r ap pack et s leav ing int er face Ser ial1 w ill use Ser ial1’s I P addr ess as t he sour ce. This m eans t hat t he pack et filt er s pr ot ect ing t he SNMP ser v er s w ill need t o have source address ent rie s for bot h Ser ial1 and Ser ial3. Alt er nat iv ely , if t he SNMP t r aps use t he loopback int er face’s addr ess as t he sour ce, it does not m at t er w hich int er face t he pack et s ex it ; t hey w ill alw ay s hav e t he sam e sour ce addr ess. Or iginat ing SNMP t r ap sour ces t o use t he loopback int er face is achiev ed w it h t he follow ing com m and:

157

snmp-server trap-source loopback 0

Rou t e r Acce ss: Con t r ollin g W h o Ca n Ge t in t o t h e Rou t e r Access t o r out er s t o car r y out adm inist r at iv e funct ions can be achiev ed eit her phy sically t hr ough t he console por t or r em ot ely t hr ough a v ir t ual t er m inal ( VTY) . The nex t few subsect ions discuss t he pr inciples and m inim um secur e configur at ions t hat should be used for access t o t he net w or k dev ice.

Pr in ciple s The VTY por t s on t he r out er ar e int ended as t he pr im ar y m eans of access t o t he dev ice. The console por t gener ally is used only for last - r esor t access, w it h t he com m on set up being t hat t he console is plugged int o a “ console ser v er ” t hat giv es t he I SP w hat is know n as out - of- band access. Most v er sions of IOS Soft w ar e hav e suppor t for fiv e VTY por t s on t he r out er—t hese por t s ar e t he m ost com m on w ay of accessing t he dev ice, ar e accessible acr oss t he net w or k , and suppor t m ult iple pr ot ocols. I SPs com m only use Telnet , w it h suppor t for Secur e Shell ( SSH) added f rom I OS Soft w ar e Releases 12.0S and 12.1T. The follow ing configur at ion guidelines ar e com m on sense but st ill ar e w or t h t ouching upon: •

•

•

Use access cont r ol list s ( ACLs) t o r est r ict Telnet connect ions t o t hose fr om sour ce net w or k s t hat y ou t r ust . This is not foolproof, but it adds a layer of difficult y . I t is also r ecom m ended t hat y ou include ant ispoofing filt er s on t he edge of y our net w or k t o pr ev ent spoof at t em pt s fr om out side y our net w or k . I m plem ent user nam e/ passw or d pair s inst ead of t he t r adit ional VTY passw ordonly t echnique of logging int o a r out er . Using bot h a user nam e and a passw or d incr eases t he lev el of effor t need t o use br ut e for ce t o cr ack t he passw or d. I deally , an AAA pr ot ocol ( RADI US, TACACS+ , or Ker ber os) should be used. I f an AAA prot ocol is used, t he user nam e/ passw or d pair can be used as a backup in case t he AAA ser ver s ar e not w or king or ar e not accessible. I nclude shor t er inact iv it y t im eout s. The inact iv it y t im eout m inim izes som e of t he r isk w hen t he car eless oper at or leav es his t er m inal logged int o t he rout er.

Each of t hese w ill be ex am ined in m or e det ail in t he follow ing sect ions.

VTY a nd Console Port Tim eout s By default , t he t im eout applied t o all connect ions t o t he VTY, console, and AUX por t s on a r out er is 10 m inut es. This t im eout is cont rolled by t he e x e c- t im e ou t com m and, as in t his exam ple:

line con 0 exec-timeout 5 0 line aux 0 exec-timeout 10 0

158

line vty 0 4 exec-timeout 5 0 ! Her e t he r out er has been t old t o disconnect console por t and VTY connect ions t hat have been idle for m or e t han 5 m inut es and 0 seconds. The aux iliar y por t t im eout has been set t o 10 m inut es. Not ice t hat set t ing t he idle t im eout t o 0 m eans t hat t he session w ill be left connect ed indefinit ely . This gener ally is r egar ded as bad pr act ice because it w ill hog t he few av ailable por t s on t he r out er and could cause m aint enance access pr oblem s in case of em er gencies. Fur t her m or e, enabling TCP k eepaliv es on incom ing connect ions ensur es t hat any sessions left hanging by a r em ot e sy st em cr ash or disconnect ion w ill not block or use up t he av ailable r out er VTY por t s. The follow ing configur at ion com m and w ill ensur e t hat sessions ar e not left hanging:

service tcp-keepalives-in

Access List s on t he VTY Port s I t is im por t ant t o secur e t he VTY por t s used for Telnet access w it h a st andard ACL. By default , t her e ar e no access cont r ols on any of t he VTY por t s. I f t his is left t his w ay and a passw or d is applied [ 1] t o t he VTY por t , t he r out er w ill be w ide open t o any one w ho at t em pt s a br ut e- for ce cr ack against t he passw or d. The follow ing configur at ion w it h a cce ss- list 3 is t y pical of a bet t er appr oach:

aaa new-model aaa authentication login Cisco-Lab local ! username Cisco1 password 7 11041811051B13 ! access-list 3 permit 215.17.1.0 0.0.0.255 access-list 3 permit 215.17.34.0 0.0.0.255 access-list 3 deny any ! line vty 0 4 access-class 3 in exec-timeout 5 0 transport input telnet ssh transport output none transport preferred none login authentication Cisco-Lab history size 256 ! a cce ss- list 3 defines t he net w or k s 215.17.1/ 24 and 215.17.34/ 24 as t he only ones w it h access t o t hese VTYs ( t hese net w or ks could be t he adm inist r at ion or NOC net w or ks at t w o locat ions, for exam ple) . I n t he pr eceding ex am ple, a t im eout of 5 m inut es is applied t o t he int er face. The second field is for seconds for finer

159

gr anular it y . I SPs gener ally pick t he best t im eout v alues accor ding t o ex per ience and t he oper at ing env ir onm ent . Also, all unnecessary t r anspor t s ar e r em oved; user s of VTYs r equir e only char act er access t o t he r out er , not hing else. ( Ot her av ailable t r anspor t s such as pad, r login, and V120 ar e not r equir ed on an I SP backbone r out er .) I t is good pr act ice t o configur e necessar y t r anspor t s on a per- in t er faceapplicat ion basis —dialup user s r equir e only I P t r anspor t , for ex am ple. I n t his case, only Telnet has been per m it t ed t o t he VTY por t , and no out bound connect ions ar e per m it t ed. I f t he r out er suppor t s m or e t han fiv e VTYs, do not for get t hem! IP- only soft w ar e ( - iand - p- code r eleases) suppor t only fiv e, but ot her feat ur e set s can suppor t 64 or as m any as 1024 VTYs. Be sur e t o apply access list s t o all of t hem , if t hey ar e configur ed. The com m and lin e v t y 0 4 cov er s t he fir st fiv e VTY por t s. To find out how m any VTYs t he r out er w ill suppor t , ent er t he follow ing com m and in configur at ion m ode:

beta7200(config)#line vty 0 ? Last Line number

beta7200(config)#line vty 0 _ The r out er w ill pr int t he last t w o lines, list ing t he VTY r ange suppor t ed. I t t hen w ill giv e y ou t he configur e com m and pr om pt . Use t his t o ensur e t hat all t he VTYs on t he r out er act ually ar e pr ot ect ed by an access list . a cce ss- list 3 pr ov ides sim ple pr ot ect ion. An ACL t hat w ill pr ov ide m or e det ailed audit ing is

access-list access-list access-list access-list

199 199 199 199

permit permit deny deny

tcp 215.17.1.0 0.0.0.255 any log tcp 215.17.34.0 0.0.0.255 any log tcp any any range 0 65535 log ip any any log

This ex t ended ACL ( x ACL) does ev er y t hing t hat a cce ss- list 3 does, but in addit ion, TCP and I P connect ions ar e logged t o t he sy slog file. The adv ant age w it h t his x ACL is t hat at t em pt s t o scan or br eak int o t he r out er ar e logged w it h t he TCP/ I P infor m at ion ( t hat is, t he sour ce addr ess) of t he per pet r at or . Bot h valid and unaut hor ized at t em pt s ar e logged, and t his is one w ay t o check w het her people ar e at t em pt ing a br ut e- for ce br eak- in t o t he r out er . On t he syslog ser ver , t he log file can be analyzed for abnor m al login at t em pt s—see t he follow ing ex am ple:

Dec 28 17:29:34.917: %SEC-6-IPACCESSLOGP: list 199 permitted tcp 144.254.193.62(1749) -> 0.0.0.0(23), 1 packet

VTY Acce ss a nd SSH [ 2 ] Before I OS Soft w are Releases 12.0S and 12.1T, t he only m et hod r eally used t o access t he VTY por t s w as Telnet . r login has been used by som e I SPs, especially for

160

execut ing one- off com m ands, but t he pr ot ocol is insecur e and can’t be r ecom m ended for any public net w or k . SSH v er sion 1 suppor t now has been added, giv ing I SPs gr eat er flex ibilit y and som e secur it y w hen accessing t heir equipm ent acr oss t he I nt er net . SSH w ill for m an encr y pt ed t unnel bet w een t he client and t he I OS Soft w ar e SSH ser v er . Because t his t unnel is for m ed befor e any aut hent icat ion is r equir ed, t he aut hent icat ion pr ocess bet w een t he user and t he r out er is encr y pt ed. This pr ov ides passw or d confident ialit y as t hey ar e t r ansm it t ed bet w een t he client and t he r out er , as w ell as confident ialit y of t he ent ir e session. The SSH t unnel pr ov ides r em ot e t er m inal ( VTY) access t o t he I OS Soft w ar e. Any of t he aut hent icat ion m et hods used w it hin I OS Soft w ar e can be applied t o t he VTYs. This includes local passw or ds, TACACS+ , and RADI US. User nam e/ passw or d com binat ions, eit her local or t hr ough an AAA ( TACACS+ or RADI US) , ar e a pr er equisit e t o SSH w or king in I OS Soft w ar e. Befor e SSH can be configur ed, t he r out er needs t o be r unning a cr y pt ogr aphic im age t hat suppor t s SSH. The st andar d I SP ( - p- ) im ages do not hav e SSH suppor t because t he U.S. gov er nm ent r est r ict s t he ex por t of 3DES. The cr y pt ogr aphic im ages ar e m ade available on Cisco. com aft er an appr ov al applicat ion has been subm it t ed. The appr ov al applicat ion for m can be found at ht t p: / / w w w .cisco.com / cgibin/ Soft w ar e/ Cr y pt o/ cr y pt o_m ain.pl. Aft er t he applicat ion has been appr ov ed, per m ission w ill be gr ant ed t o dow nload t he necessar y im ages. When t he appr opr iat e cr ypt ogr aphic im age is r unning, SSH needs t o be set up on t he r out er . The follow ing sequence of configur at ion com m ands giv es an ex am ple of how t his can be achiev ed:

beta7200(config)#crypto key generate rsa beta7200(config)#exit beta7200#write terminal Select a key size of at least 1024 bit s. Saving t he configur at ion is necessar y for t he k ey t o be sav ed and act iv at ed. Aft er t his, add ssh as t he input t r anspor t on t he VTYs:

line vty 0 4 transport input telnet ssh When t his configurat ion has been com p let ed, it is possible t o use SSH t o access t he r out er . I f t he I OS Soft w ar e im age suppor t ing DES is being used, t he SSH client on t he oper at or ’s sy st em needs t o suppor t DES ( m ost SSH client s hav e t his disabled by default because it no longer is consider ed ver y secur e) . This could r equir e t he client t o be r ecom piled ( if t his is an opt ion) . N OTE A user nam e/ passw or d pair m ust be configur ed on t he r out er befor e SSH access w ill w or k . How ev er , it is st r ongly r ecom m ended t hat AAA be used t o aut hent icat e user s ( discussed lat er ) because t his is t he pr efer r ed w ay of secur ing t he r out er .

161

The follow ing is t he UNI X com m and t o connect t o t he r out er bet a7200 using SSH w it h t he user nam e philip.

ssh beta7200 –l philip Since I OS Soft w ar e Release 12.0( 6) S ( CSCdr 82377) , SSH c an be used in reverse Telnet m ode as w ell w hen connect ing t o out - of- band access r out er s. For exam ple, t he follow ing com m and w ill connect t o VTY por t 16 using SSH:

ssh router telnet 10.0.0.1 2016 SSH w ill t er m inat e on t he out - of- band access r out er and t hen connect t hr ough r ev er se Telnet int o t he console connect ed t o t he dev ice. SSH also can be used in bat ch com m ands:

ssh -x -t -c [des|3des] -l username ipaddr "show ip cache addr mask verbose flow" I n Release 12.0( 9) S ( CSCdp73127) , an SSH client [ 3] w as added t o I OS Soft w ar e. This allow s an engineer t o use SSH t o get int o a r out er and t hen use SSH fr om t hat r out er t o hop t o anot her r out er . I t is com m on for back bone engineer s t o go fr om one r out er t o anot her w hile in t he m idst of t r oubleshoot ing. The SSH client allow s back bone engineer s t o use t he sam e t r oubleshoot ing t echniques w it hout loss of capabilit y . The com m and sy nt ax for t he I OS Soft w ar e SSH client is as follow s:

ssh [-l userid] [-c des|3des] [-o numberofpasswdprompts n] [-p portnum] ipaddr|hostname [IOS command] The det ails of t he com m and synt ax ar e • • •

• •

- l userid is t he user t o log in as on t he r em ot e m achine. The default is t he cur r ent user I D. - c d e s| 3 de s specifies t he cipher t o use for encr y pt ing t he session. 3DES is encr y pt - decrypt - encr ypt w it h t hr ee differ ent keys. The default is 3DES if t his algorit hm is included in t he im age; ot herw ise, t he default is DES. - o specifies ot her opt ions. Ther e is only one cur r ent ly, and t hat is num ber ofpassw d pr om pt s n, w hich specifies t he num ber of passw or d pr om pt s befor e ending t he at t em pt ed session. The ser v er also lim it s t he num ber of at t em pt s t o 5, so it is useless t o set t his v alue lar ger t han 5. Ther efor e, t he range is set at 1 t o 5 and t he default is 3, w hich is also t he I OS Soft w are ser v er default . - p port num specifies t he por t t o connect t o on t he r em ot e host . The default is 22. ipaddr | host nam e is t he r em ot e m achine I P addr ess or host nam e.

162

•

I OS com m and is an I OS Soft w ar e ex ec com m and enclosed in quot at ion m ar k s. This w ill be ex ecut ed on connect ion, and t he connect ion w ill be t er m inat ed w hen t he com m and has com plet ed.

User Aut hent icat ion I t is good pr act ice t o r egist er each individual user w it h a separ at e user I D. I f a gener ic account is set up, it is easier for it t o fall int o t he w r ong hands, and t her e is v ir t ually no account abilit y . This r esult s in abuse of access and pot ent ial m alfunct ion of t he net w or k. I n addit ion, if t he default passw or d- only login is used, it becom es v er y easy t o use a br ut e- for ce cr ack ut ilit y t o get t he passw or d. A user nam e/ passw or d pair m ak es br ut e- for ce t echniques har der but not im possible. This ex am ple show s one w ay of configur ing user I Ds. I t is pr act ical for net w or k s of a few r out er s, but it does not scale and it suffer s fr om t he w eak Type 7 encr ypt ion. ( This encr y pt ion t y pe should be av oided but is giv en her e for com plet eness.) St ar t by configur ing t he follow ing on t he r out er :

username joe password 7 045802150C2E username jim password 7 0317B21895FE ! line vty 0 4 login local ! That changes t he login pr om pt sequence fr om t his

Password: t o t his

Username: Password: Her e, t he user nam e r equest ed is fr om t hose list ed pr ev ious. Each user w ill hav e t o supply t he passw or d upon r equ est . At t he t im e of t his w r it ing, MD5- st y le encr y pt ion w as added t o t he u se r n a m e configur at ion com m and in som e v er sions of I OS Soft w ar e, so it is now possible t o st or e user nam e/ passw or d pair s on t he r out er t o be cer t ain t hat t he passw or ds w ill not be reverse- engineer ed. The configur at ion looks like t he follow ing:

username joe secret 5 $1$j6Ac$3KarJszBV3VMaL/2Nio3E. username jim secret 5 $1$LPV2$QO4NwAudy0/4AHHHQHvWj0 ! line vty 0 4 login local

163

! Alt hough t his adds secur it y t o t he passw or ds st ored on t he rout er, we st ill r ecom m end using a net w or k- based aut hent icat ion sy st em , as descr ibed in t he nex t sect ion.

Using AAA t o Secure t he Rout er The pr efer r ed and r ecom m ended m et hod of secur ing access t o t he r out er is t o use an AAA pr ot ocol such as T ACACS+ , RADI US, or Kerberos. Here t he usernam es and passw or ds for all t he user s w ho hav e access t o t he r out er s ar e held at a cent r al locat ion, off t he r out er . This has sev er al adv ant ages: • • • •

Recall t hat t he encr y pt ion m et hod 7 is r ev er sible. Any one w ho has ac cess t o t he r out er configur at ion pot ent ially could w or k out t he passw or d and gain access t o t he syst em . I f t her e is a new user or a user leav es t he I SP, it is easy t o change t he passw or d dat abase once. Changing it on m any differ ent r out er s becom es a considerable t ask. Passw or ds ar e held in UNI X encr y pt ed for m at on t he cent r al AAA ser v er . The algorit hm for UNI X passw ord encrypt ion is not reversible and t hus is m ore secur e. All accesses ar e logged t o t he AAA ser ver . I n fact , som e AAA soft w ar e w ill allow all act ions on t he r out er t o be logged.

Com m er cial Cisco ACS soft w ar e is available for Window s NT and Sun Solar is sy st em s. A fr eely av ailable TACACS+ ser v er for UNI X plat for m s is pr ov ided on Cisco’s engineer ing FTP sit e at ft p: / / ft peng.cisco.com / pub/ t acacs/ t ac_plus.F4.0.4.alpha.t ar .Z. This can be com piled and built on virt ually all UNI X syst em s —in fact , m any I SPs hav e int egr at ed t his int o t heir ow n aut hent icat ion pro cesses. On t he r out er , a t y pical TACACS+ configur at ion w ould be as follow s:

! aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ ! ip tacacs source-interface Loopback0 tacacs-server host 221.17.1.2 tacacs-server host 221.17.34.10 tacacs-server key CKr3t# ! To ex plain t hese com m ands, t he a u t h e n t ica t ion login st at es t hat TACACS+ should be used for login aut hent icat ion. I f t he TACACS+ ser v er s ar e not r eachable, t he local enable secr et passw or d is used. The a u t h e n t ica t ion e n a ble com m an d st at es t h at TACACS+ should be used t o aut hent icat e t he use of t he ena ble com m and. The

164

enable passw or d should be t ak en fr om t he TACACS+ ser v er befor e using t he lo cal enable secr et passw or d. Not e t he use of t he loopback int er face as t he sour ce of TACACS+ r equest s ( r easons as in pr ev ious ex am ples) and t he use of t w o TACACS+ ser v er s for r edundancy and r esilience. I f t he r out er is r unning an I OS Soft w ar e v er sion t hat does not support TACACS+ (pre- 11.0) , it is st r ongly advised t hat it be upgr aded t o at least ver sion 11.0 because t he m or e r ecent soft w ar e has m or e feat ur es appr opr iat e t o I SPs, as discussed in t his chapt er . I f t he I SP engineer ing oper at ions t eam does decide t o st or e som e local user nam es and passw or ds on t he r out er , using t he new u se r n a m e nam e se cr e t secret opt ion, t he configur at ion m ight look som et hing lik e t he follow ing:

! aaa new-model aaa authentication login default tacacs+ local enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ ! username joe secret 5 $1$j6Ac$3KarJszBV3VMaL/2Nio3E. username jim secret 5 $1$LPV2$QO4NwAudy0/4AHHHQHvWj0 ! ip tacacs source-interface Loopback0 tacacs-server host 221.17.1.2 tacacs-server host 221.17.34.10 tacacs-server key CKr3t# ! Not e t he loca l in t he a u t h e n t ica t ion log in line. I f t he TACACS+ ser ver is unavailable, t he r out er w ill expect eit her of t w o local account s j oe or j im t o be used for t he login and aut hent icat ion.

Rout er Com m a nd Audit ing Suppose t hat it has been a bad day at t he office. You hav e had t o fir e an engineer on t he oper at ions t eam . I t w as a r ough ex per ience, w it h w or ds ex changed. You hav e br iefed y our t eam , and people ar e st ar t ing t o change t hings on t he net work, r em ov ing t he ex- em ploy ee’s access. Suddenly alar m s t r igger on all sor t s of equipm ent in t he NOC. A net w or k- w ide out age is spr eading fast . I s it an at t ack? I s it a r out ing pr ot ocol collapse? What is happening? Devices ar e failing t hr oughout t he net w or k, and cust om er lines ar e r inging off t he hook. Som eone fr om your t eam uses t he out - of- band access t o get int o one of t he affect ed sw it ches. Hor r or ! The ent ir e configur at ion has been er ased along w it h t he soft w ar e im age. Ot her sw it ches and r out er s ar e check ed—t hey also hav e t heir configur at ions er ased and soft w ar e im ages er ased. I t is sabot age! You k now w ho did it , but how ar e y ou going t o pr ov e it ! Yes, sabot age fr om disgr unt led em ploy ees has happened in t he past and w ill happen again. The t w o pr im ar y adv ant ages of using TACACS+—cent r alized account cont r ol

165

and com m and audit ing—should pr ov ide enough m ot iv at ion for I SPs t o pr ot ect t hem selv es fr om sit uat ions such as t his one. Wit h cent r alized cont r ol, t he em ploy ee’s account can be r em ov ed before he is giv en not ice. I f t he em ploy ee had back- door access t hr ough anot her account , any com m ands r un on t he sw it ches and r out er s w ould be audit ed, pr ov iding r ecor ds t hat can be used in a cour t of law . I n fact , fr om a secur it y point of view , AAA should be view ed as aut hent icat ion, aut hor izat ion, and audit ing—w hich, of cour se, is one of t he st r engt hs t hat TACACS+ has over RADI US in t he secur it y dom ain. AAA on t he r out er and a TACACS+ ser v er can be configur ed t o t r ack all com m ands or a lim it ed set of com m ands t y ped int o t he r out er . AAA com m and account ing pr ov ides infor m at ion about t he ex ec shell com m ands for a specified pr iv ilege lev el t hat ar e being ex ecut ed on a r out er . Each com m and account ing r ecor d includes a list of t he com m ands ex ecut ed for t hat pr iv ilege lev el, as w ell as t he dat e and t im e t hat each com m and w as ex ecut ed and t he user w ho ex ecut ed it . The follow ing ex am ple show s t he infor m at ion cont ained in a TACACS+ com m and account ing r ecor d for pr ivilege level 1:

Wed Jun 25 03:46:47 1997 172.16.25.15 fgeorge 5622329430/4327528 stop task_id=3 service=shell priv-lvl=1 version Wed Jun 25 03:46:58 1997 172.16.25.15 fgeorge 5622329430/4327528 stop task_id=4 service=shell priv-lvl=1 interfaces Ethernet

Wed Jun 25 03:47:03 1997 172.16.25.15 fgeorge 5622329430/4327528 stop task_id=5 service=shell priv-lvl=1 route

tty3 cmd=show tty3 cmd=show

tty3 cmd=show ip

The follow ing exam ple show s t he inf or m at ion cont ained in a TACACS+ com m and account ing r ecor d for pr ivilege level 15:

Wed Jun 25 03:47:17 1997 172.16.25.15 fgeorge 5622329430/4327528 stop task_id=6 service=shell priv-lvl=15 terminal Wed Jun 25 03:47:21 1997 172.16.25.15 fgeorge 5622329430/4327528 stop task_id=7 service=shell priv-lvl=15 Serial 0 Wed Jun 25 03:47:29 1997 172.16.25.15 fgeorge 5622329430/4327528 stop task_id=8 service=shell priv-lvl=15 address 1.1.1.1 255.255.255.0

tty3 cmd=configure tty3 cmd=interface tty3 cmd=ip

The next exam ple, show n in Figur e 4- 3, is t aken from t he Cisco Secur e AAA Ser v er and dem onst r at es t he adv ant age of com m and audit ing. The display on t he Cisco

166

Secur e soft w ar e pr oduct is a sim ple sum m ar y of w hat w as happening, at w hat t im e, and fr om w her e. Fig u r e 4- 3 Com m a n d Au d it in g on t h e Rou t e r Th r ou g h Cisco Se cu re a n d TACACS+ bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 pfs tty0 pfs tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty0 bgreene tty66 bgreene tty66 bgreene tty66

NOC 4 NOC 5 NOC 6 NOC 8 NOC 11 NOC 12 NOC 14 NOC 16 NOC 17 NOC 18 NOC 20 NOC 21 NOC 22 NOC 23 NOC 24 NOC 25 NOC 32 NOC 35 NOC 45 NOC 46

enable 210.210.51.224 exit 210.210.51.224 no aaa accounting exe Worksho 210.210.51.224 exit 210.210.51.224 enable 210.210.51.224 exit 210.210.51.224 enable 210.210.51.224 show accounting 210.210.51.224 writeterminal 210.210.51.224 configure 210.210.51.224 exit 210.210.51.224 write terminal 210.210.51.224 configure 210.210.51.224 aaa new-model 210.210.51.224 aaa authorization commands 0 de 210.210.51.224 exit 210.210.51.224 ping 210.210.51.224 show running-config 210.210.51.224 router ospf 210 210.210.51.224 debug ip ospf events 210.210.51.224

0

shell

0

shell

0

shell

0

shell

0

shell

0

shell

0

shell

15

shell

15

shell

15

shell

0

shell

15

shell

15

shell

15

shell

15

shell

0

shell

15

shell

15

shell

15

shell

15

shell

Configur at ion cont r ol and audit of w ho has done w hat and w hen on t he r out er s is t he k ey obj ect iv e for using AAA com m and account ing on an I SP’s back bone.

aaa aaa aaa aaa

new-model authentication login default tacacs+ enable authentication enable default tacacs+ enable accounting command 15 start-stop tacacs+

167

aaa accounting exec start-stop tacacs+ ! ip tacacs source-interface Loopback0 tacacs-server host 215.17.1.2 tacacs-server host 215.17.34.10 tacacs-server key CKr3t# N OTE When com m and account ing is enabled, all com m ands ( t hat is, k ey st r ok es) sent t o t he r out er in enable m ode ar e logged in t he account ing file on t he account ing host . Be aw ar e t hat , w hen changing sensit iv e configur at ions on t he r out er , t hese changes are recorded in t he account ing host log file. One exam ple is w here t he last resort passw or d ( enable secr et ) is changed dur ing an online session on t he r out er ; t he new password will be recorded in full in t he account ing file. Of cour se, t he r ecom m ended w ay t o change such a last - r esor t passw or d is t o use TFTP t o copy t he necessar y configur at ion ( including t he fir st line disabling t he account ing com m and and t he last line reenabling it again! ) from a TFTP server and never t o m a ke such changes live on t he rout er.

On e -Tim e Pa ssw ord One adv ant age of ensur ing t hat t he I SP’s oper at ions t eam uses TACACS+ t o access t o t he net w or k ’s infr ast r uct ur e is t hat one- t im e passw or d ( OTP) t echniques can be used t o pr ovide an addit ional layer of secur it y . The com m on appr oach usually is t o use a sm ar t car d in t he possession of t he indiv idual being aut hent icat ed. That indiv idual ent er s a per sonal ident ificat ion num ber ( PI N) int o t he sm ar t car d. The sm art card ret urns a t im e - sensit iv e passw or d t hat c an be used as t he aut hent icat ion passw or d ( see Figur e 4- 4) . This passw or d can be used only once, pr ev ent ing passw or d sniffing. Fig u r e 4- 4 . OTP Sy st e m s Ar e U se d in I SP Ope r a t ion s

OTP t echniques pr ev ent an at t ack er fr om guessing a user ’s passw or d t hr ough br ut e for ce. Because t he passw or d r et ur ned by t he OTP car d is valid only once and for only a specific w indow of t im e, for ced passw or d guessing is pr act ically im possible ( besides t he fact t hat AAA sy st em s such as TACACS+ w ill deact iv at e a user ’s account w it h t oo

168

m any failed login at t em pt s) . OTP syst em s also m inim ize t he pr oblem s of st aff w r it ing dow n t heir passw or ds. I t is nat ur al hum an behav ior t o w r it e dow n com plex passw or ds in case t hey ar e for got t en. You see people put t ing passw or ds on a file in a lapt op, in a planner, in a PDA, on a desk, or at a desk at hom e. Eac h of t hese m em or y aides could be st olen or br ok en int o, opening t he r isk of som eone using t hat passw or d as a w ay t o br eak int o t he net w or k . OTP sy st em s for ce t he indiv idual t o r em em ber one passw or d for t he OTP car d and t hen use it r epeat edly . I f t he in div idual does not hav e t he car d, he cannot get a passw or d and w on’t be able t o get int o t he syst em . The com binat ion of ACL, TACACS+ , SSH, and OTP com bines t o pr ov ide a st r ong secur it y sy st em t o pr ot ect an I SP’s net w or k infr ast r uct ur e. W h a t OTP Sy st e m s Ar e Su p por t e d ? Several t hird- par t y OTP sy st em s ar e suppor t ed by Cisco’s com m er cial TACACS+ ser v er ( CiscoSecur e ACS) at t he t im e of t his w r it ing. CiscoSecur e ACS v 2.3 for Window s NT suppor t s t ok en car d ser v er s fr om t he follow ing: • • • •

Secur it y Dy nam ics Axent Technologie s Safew or d ( Secur e Com put ing) CRYPTOCar d ( included)

CiscoSecur e ACS v 2.2 for UNI X suppor t s t ok en car d ser v er s fr om t he follow ing: • • •

CRYPTOCar d ( included) Secur e Com put ing Secur it y Dy nam ics

Due diligence is r ecom m ended. Hence, check t he lat est docum ent at ion on w w w . cisco. com for t he m ost cur r ent list of suppor t ed OTP sy st em s. OTP Con f ig u r a t ion H in t s OTP logins r esult in t r affic t o TACACS+ ser v er , t o t he OTP ser v ice, back t o t he TACACS+ ser v er , and t hen t o t he net w or k dev ice. Hence, t he net w or k dev ice should be configur ed w it h a m inim um 10- second TACACS+ t im eout set t ing:

tacacs+ timeout 10 The default TACACS+ t im eout set t ing of 1 second w ill cause OTP logins t o fail a lar ge per cent age of t he t im e. For com plet e and det ailed infor m at ion on OTP configur at ion w it h CiscoSecur e ACS, check t he online docum ent at ion at w w w . cisco. com. Look for t he chapt er t it led “ Tok en Ser v er Suppor t ” ( ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / access/ acs_soft / cs_unx / csu23ug / 9 _ t ok en . h t m) .

169

M anaging I CM P Unreachables from t he Rout er The I CMP Dest inat ion Unr eachable m essages are key m essages used t o det erm ine t he st at e of t he net w or k . I CMP Unr eachables ar e r esponses sent by a r out er / host / sw it ch w henev er t he dest inat ion host addr ess, pr ot ocol unr eachable, or dest inat ion net w or k s ar e not list ed in t he for w ar d t able ( FI B) or serv ices by t h e dev ice. I CMP Unr eachables ar e a nor m al funct ion of t he TCP/ I P pr ot ocol suit e. Yet , t his nor m al st at e has been ex ploit ed t o ov er load net w or k dev ices, specifically sending pack et s t hat w ould r equir e t he dev ice t o r espond w it h v olum es of I CMP Unreachable m essages. As of DDTS CSCdr 46528, Cisco has r at ionalized handling of I CMP Unr eachables so t hat t hey ar e m anaged as close t o t he int er face as possible. On dist r ibut ed r out ing ar chit ect ur es ( plat for m s such as t he 7500, 7600, 10000, and 12000) , t he line car d ( LC) or v er sat ile int er face pr ocessor ( VI P) handles t he I CMP Unr eachable r eplies. This isolat es any I CMP Unr eachable ov er load DoS/ DDoS at t ack s t o t he LC/ VI P, fr eeing t he r out e pr ocessor ( RP) fr om unnecessar y bur den. I n addit ion t o t his archit ect ura l t hem e, Cisco pr ov ides t w o feat ur es t hat I SPs should consider t o lim it t he effect s of an I CMP Unr eachable DoS/ DDoS at t ack on a r out er . These ar e t he I CMP Unr eachable r at e- lim it ing feat ure and t he n o ip u n r e a ch a b le s int er face com m and. I CM P U n r e a ch a ble Ra t e Lim it in g DDTS CSCdp28161 added an I CMP Unr eachable r at e- lim it ing feat ur e in I OS Soft w ar e Release 12.0( 8) S. This feat ur e is t ur ned on by default and is not seen in t he CLI unless t he default r at e lim it is changed. This feat ur e r at e lim it s t he r out er ’s responses t o I CMP Unr eachable m essages leav ing t he r out er . The obj ect iv e is t o k eep t he r out er fr om being ov er w helm ed w it h an I CMP Unr eachable ov er load ( t hat is, a DoS at t ack against t he r out er ) . By default , t he r out er r at e lim it s I CMP Unr eachables t o one per 500 m illiseconds. Befor e I OS Soft w ar e Release 12.0( 8) S, t his w as not configur able. Now m any I SPs can set t he r out er s t o r espond t o one I CMP Unr eachable ev er y 2000 m s, pr ov iding gr eat er r esist ance t o an I CMP Unr eachable ov er load. A new global com m and is int r oduced t o cont r ol t he I CMP Unr eachable r at e. This com m and is hidden unless t he default s ar e changed. The default is t o hav e one I CMP Unreachable m essage per 500 m s.

ip icmp rate-limit unreachable [DF] 1-4294967295 no ip icmp rate-limit unreachable [DF] The DF opt ion r at e lim it s t he I CMP Unr eachable m essage w it h code 4, fr agm ent at ion needed, and DF set . Consult at ion w it h sev er al I SPs r esult ed in t he r ecom m endat ion t hat I SPs should set t his r at e lim it t o one ever y t w o seconds. For exam ple, t he follow ing e x am ple set s t he r at e of t he I CMP Unr eachable t o one m essage per 2000 m s w it h DF set :

170

ip icmp rate-limit unreachable DF 2000 N o I P U n r e a ch a b le s For a long t im e, Cisco r out er s had t he configur at ion capabilit y t o t ur n off I CMP Unr eachable r esponse. This w as done w it h t he int er face com m and n o ip u n r e a ch a b le s. Whet her t his is done is an oper at ional decision of t he I SP—som e do and som e do not . The r out er r equir em ent s RFC ( RFC1812) say s t hat each dev ice should r espond w it h I CMP Unr eachables, but w hen a net w or k oper at or ex per iences an at t ack against a r out er , RFC nicet ies get left behind. What can be r ecom m ended or consider ed is t hat n o ip u n r e a ch a b le s be applied t o t he special int er faces used on I SP r out er s: Loopback and Null0. For exam ple, m any I SPs use st at ic rout es t o Null0 for t heir ent ire CI DR block t o lock up t heir BGP adv er t isem ent s. This Null0 r out e can be ex ploit ed by an I CMP Unr eachable at t ack on t he par t of t he CI DR block t hat has y et t o be allocat ed and act iv at ed. By default , t he r out er r esponds w it h ICMP Unreachable m essages ( rat e lim it ed t o one per 500 m s) on t he unallocat ed/ act iv at ed par t of t he st at ic t o Null0. Tur ning off I CMP Unr eachables pr ev ent s t his fr om happening, black- holing t he pack et s at t hat r out er .

interface Null0 no ip unreachables ! ip route dest-to-drop mask Null0 N OTE A r ecent audit ar ound I OS Soft w ar e Releases 12.0( 16) S and 12.1( 6) E ensur ed t hat t h e n o ip u n r e a ch a b le s com m and w or ks on all line car ds and r out er s. Pr eviously t his w as not t he case for sev er al line car ds, r equir ing fix es t o be put in place.

Building a N ew Rout er or Sw it ch This final par t of t he r out er secur it y sect ion gives a dem onst r at ion of how an I SP engineer should go about deploy ing a new I P dev ice on an I SP back bone. Ther e is a pr oper , sensible, w ell- defined process for doing t his —and not j ust one t hat inv olv es plugging it in, sw it ching it on, and t hen figur ing out w hat t o do, as is done by far t oo m any I SP engineer s t oday . Th e Pr oce ss The pr ocess t hat should be follow ed goes som et hing lik e t he follow ing st eps. Th e t echnique is used by sev er al I SPs and is par t of t he Cisco Sy st em s I SP Wor k shops t hat have been r unning in m any par t s of t he w or ld for t he pr evious sever al year s. 1. Sw it ch o n— Mak e sur e t hat t he r out er is not connect ed t o any LAN or WAN. Connect t he c onsole por t t o t he com put er or t er m inal ser v er dev ice, and t hen

171

pow er up. When t he new r out er ask s t o ent er configur at ion m ode, answ er no t o get t o t he Rou t e r > com m and pr om pt . Ent er enable m ode. ( The quickst ar t configur at ion guide is gr eat for new user s t o Cisco rout ers but is som ew hat point less for an I SP t hat is going t o copy cust om ized t em plat es ont o t he dev ice aft er it is secur ely connect ed t o t he LAN.) 2. Se t t h e r ou t e r h ost n a m e — This is so t hat you know w hat you ar e t yping on and w hat y ou ar e t r y ing t o configur e. I n t he daily life of a busy engineer , being diver t ed br iefly t o anot her issue w hile in t he m iddle of configur ing t he r out er and t hen com ing back and not k now ing w hat y ou ar e w or k ing on, can be a secur it y r isk or can cause disast er on t he I SP backbone by configuring t he w r ong dev ice. 3. Se t p a ssw or d s— This m eans set t ing an e n a b le se cr e t , put t ing a passw or d on all t he VTY por t s on t he r out er , and enabling se r v ice p a ssw or d e n cr y pt ion . Obv iously , configur ing a net w or k- based aut hent icat ion sy st em m akes no sense at t he m om ent because t her e is no net w or k connect ed! 4. D isa b le u n n e ce ssa r y se r v ice s— Rem ove global and per- int er face ser v ices t hat should not be present on a I SP’s rout er. This includes HTTPD and CDP if t hey ar e r unning by default . 5. Se t ba n n e r s — Set any login banner s t hat ar e r equir ed for t he VTYs or ot her port s of t he rout er. 6. Con figu r e a cce ss list s — Access list s for pr ot ect ing t he VTY por t s should be configur ed on t he r out er . Assum ing t hat SNMP is disabled at t his st age, an access list t o prot ec t t he SNMP ser v ice on t he r out er is not r equir ed. The VTY access list should allow connect ions fr om t he NOC or m anagem ent syst em s, not hing m or e. Access list s for any int er faces t hat w ill be br ought liv e t o com plet e t he configur at ion also should be inst alle d. Most new r out er s hav e a single LAN connect ion act iv at ed so t hat t he r est of t he configur at ion can be inst alled. This is usually all t hat is r equir ed at t his st age. 7. N ow p lu g in t o t h e n e t w or k — Only at t his st age has t he r out er been sufficient ly secur ed so t hat it can be connect ed t o t he net w or k . The VTYs ar e pr ot ect ed t o allow only per m it t ed I P addr esses, and t he LAN/ WAN int er face is pr ot ect ed so t hat only per m it t ed I P addr esses can connect t o t he r out er . Minim al r out ing can be configur ed, w het her t his is par t of t he I SP’s I GP pr ocess or a sim ple st at ic r out e for default gat ew ay . 8. Con f ig u r e TACACS+ — Now t hat t he net w or k connect ion has been act iv at ed, t he net w or k - based aut hent icat ion sy st em can be act iv at ed. Don’t forget t o rem ove any locally added usernam e / passw or d pair s and any passw or ds t hat hav e been added t o t he r out er VTY por t s. Don’t for get t o log off and log on again t o check t hat t he configur at ion act ually w or k s. 9. Con f igu r e N TP a n d loggin g — Tim e sy nchr onizat ion should be configur ed, along wit h syslo gging on t he r out er . This is so t hat log m essages of any issues can be capt ur ed on t he log host s, t o keep a full r ecor d of w hat is happening on t he rout er.

172

10. Con f ig u r e SN M P— Many I SPs use t ools such as MRTG, so now is t he t im e t o configur e SNMP access and any necessar y access list s t o suppor t secur e access t o t he SNMP funct ion. 11. Con f ig u r e r e m a in in g in t e r f a ce s— Any ot her int er faces t hat ar e r equir ed for t he r out er ’s oper at ion now should be configur ed. 12. Con f igu r e r ou t in g pr ot ocols — Now is t he t im e t o int r oduce an I GP ( if not alr eady configur ed ear lier ) and iBGP, if r equir ed. 13. Fin ish e d! — Use a net w or k secur it y t ool such as SAI NT t o check t he secur it y of t he r out er . I n t he absence of any such t ools, ask colleagues t o check t he configur at ion for com plet eness, or run a diff against t he configur at ion of a sim ilarly configured rout er running successfully in t he net w ork. These hint s apply t o r out er s and sw it ches. Not e t hat alt hough som e of t hem could be release- dependent , it is w or t h spending t im e t o analy ze w hat t he soft ware release w ill and w ill not suppor t , and t o m ak e allow ances in t he pr ev ious check list for t hose point s. Fu ll Ex a m ple The follow ing is a full ex am ple of a secur e configur at ion t em plat e put t oget her fr om all t he t echniques described so far:

service password-encryption service tcp-keepalives-in ! aaa new-model aaa authentication login default tacacs+ enable aaa authentication login Cisco-Lab local enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ ! username Cisco1 password 7 11041811051B13 enable secret ! access-list 199 permit tcp 221.17.1.0 0.0.0.255 any access-list 199 permit tcp 221.17.34.0 0.0.0.255 any access-list 199 deny tcp any any range 0 65535 log access-list 199 deny ip any any log ! ip tacacs source-interface Loopback0 tacacs-server host 221.17.1.2 tacacs-server host 221.17.34.10 tacacs-server key CKr3t# ! line vty 0 4 access-class 199 in exec-timeout 5 0 transport input telnet ssh transport output none transport preferred none

173

login authentication Cisco-Lab history size 256 !

Se cu r in g t h e Rou t in g Pr ot ocol How do you know t hat t he r out ing updat es fr om one of your int er nal backbone r out er s r eally cam e fr om a neighbor ing r out er on y our backbone? You do not —unless neighbor aut hent icat ion is used. Theor et ically , r out ing infor m at ion can be spoofed and inj ect ed int o an I SP’s back bone. The hor r or of seeing a nor m al 110,000- ent r y r out ing t able j um p t o 200,000 or 500,000 ent r ies—and t hen seeing t hese pr opagat e all over t he I nt ernet —has been quiet ly t alk ed about in t he halls of I nt er net oper at ions m eet ings. I SPs ar e st r ongly encour aged t o pr ev ent t heir r out er s fr om r eceiv ing fr audulent r out e updat es by pr ot ect ing t he r out ing pr ot ocol updat es. Cisco IOS Soft w ar e has sev er al t ools t o help pr ot ect t he r out ing pr ot ocols fr om int ent ional or unint ent ional at t ack s. The pr im ar y m echanism for pr ot ect ing r out e pr ot ocol updat es is r out er aut hent icat ion. MD5 is a v aluable t ool t hat w ill v alidat e t he aut hent icat io n of a r out ing updat e. The second t ool is ext ended ACLs. ACLs should be used st r at egically t hr oughout t he net w or k t o v alidat e t he sour ce/ dest inat ion address of packet s headed for t he I GP and EGP port s. ACLs m ake it m ore difficult for DoS/ DDoS at t ack s t o t ar get t he r out ing pr ot ocol. Finally , specific com m ands and BCPs in t he r out ing pr ot ocols help pr ot ect t hem fr om at t ack . For ex am ple, BGP’s m a x im um - pr e fix com m and aler t s t he oper at or s and opt ionally shut s dow n t he BGP session w henever t he m axim um pr efix lim it is r eached. This pr ot ect s t he r out er fr om being ov er w helm ed by t he num ber of updat es or hav ing it s m em or y com plet ely consum ed, pot ent ially cr ashing t he r out er . This sect ion r ev iew s t hese t ools and t echniques so t hat I SP engineer s w ill hav e a bet t er underst anding of how t o pr ot ect t heir r out ing pr ot ocols.

Aut hent ica t ing Rout ing Prot ocol Upda t es Neighbor r out er aut hent icat ion is par t of an I SP’s t ot al secur it y plan. This sect ion descr ibes w hat neighbor r out er aut hent icat ion is, how it w or k s, and w hy it should be used t o incr ease ov er all net w or k secur it y . Docum ent at ion det ails can be found at ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc / pr oduct / soft w ar e/ ios122/ 122cgcr / fsecur _c / fot her sf/ scfr out r . ht m. Be n e f it s of N e ig h b or Au t h e n t ica t ion When configur ed, neighbor aut hent icat ion occur s w henev er r out ing updat es ar e ex changed bet w een neighbor r out er s. This aut hent icat ion ensur es t hat a r out er r eceiv es r eliable r out ing infor m at ion fr om a t r ust ed sour ce. Wit hout neighbor aut hent icat ion, unaut hor ized or deliber at ely m alicious r out ing updat es could com pr om ise t he secur it y of y our net w or k t r affic. A secur it y com pr om ise could occur if an unfriendly par t y div er t s or analy zes y our net w or k t r affic. For ex am ple, an unaut hor ized r out er could send a fict it ious r out ing updat e t o conv ince y our r out er t o send t r affic t o an incor r ect dest inat ion. This div er t ed t r affic could be analy zed t o learn confident ial infor m at ion of y our or ganizat ion or m er ely could be used t o disr upt

174

y our or ganizat ion’s capabilit y t o effect iv ely com m unicat e using t he net w or k . Neighbor aut hent icat ion pr ev ent s any such fr audulent r out e updat es fr om being received by your rout er. Pr ot ocols Th a t U se N e ig h b or Au t h e n t ica t ion Neighbor aut hent icat ion can be configur ed for t he follow ing r out ing pr ot ocols: • • • • • •

Bor der Gat ew ay Pr ot ocol ( BGP) DRP Server Agent I nt er m ediat e Sy st em- t o- I nt erm ediat e Syst em ( I S- I S) I P Enhanced I nt er ior Gat ew ay Rout ing Pr ot ocol ( EI GRP) Open Shor t est Pat h Fir st ( OSPF) Rout ing I nfor m at ion Pr ot ocol Ver sion 2 ( RI Pv2)

W h e n t o Con f ig u r e N e ig h b or Au t h e n t ica t ion You should configur e any r out er for neighbor aut hent icat ion if t hat r out er m eet s all of t hese condit ions: • • • •

The r out er uses any of t he r out ing pr ot ocols pr ev iously m ent ioned. I t is conceiv able t hat t he r out er m ight r eceiv e a false r out e updat e. I f t he r out er r eceived a false r out e updat e, your net w or k m ight be com pr om ised. I f y ou configur e a r out er for neighbor aut hent icat ion, y ou also need t o configur e t he neighbor r out er for neighbor aut hent icat ion.

H ow N e igh bor Au t h e n t ica t ion W or k s When neighbor aut hent icat ion has been configur ed on a r out er , t he r out er aut hent icat es t he sour ce of each r out ing updat e pack et t hat it r eceiv es. This is accom plished by t he ex change of an aut hent icat ing k ey ( som et im es r efer r ed t o as a passw or d) t hat is know n t o bot h t he sending and t he r eceiving r out er s ( as show n in Figur e 4- 5) . Tw o t y pes of neighbor aut hent icat ion ar e used: plain- t ex t aut hent icat ion and Message Digest Algor it hm Ver sion 5 ( MD5) aut hent icat ion. Bot h for m s w or k in t he sam e w ay , w it h t he ex cept ion t hat MD5 sends a “ m essage digest ” inst ead of t he aut hent icat ing key it self. The m essage digest is cr eat ed using t he key and a m essage, but t he key it self is not sent , prevent ing it from being read w hile it is being t r ansm it t ed. Plain- t ex t aut hent icat ion sends t he aut hent icat ing k ey it self ov er t he wire. Figu r e 4 - 5 . Rou t e r Au t h e n t ica t ion

175

N OTE Plain- t ex t aut hent icat ion is not r ecom m ended for use as par t of y our secur it y st r at egy. I t s pr im ar y use is t o avoid accident al changes t o t he r out ing infr ast r uct ur e. Using MD5 aut hent icat ion, how ev er , is a r ecom m ended secur it y pr act ice.

CAU T I O N As w it h all k ey s, passw or ds, and ot her secur it y secr et s, it is im per at iv e t hat y ou closely guard t he keys used in neighbor aut hent icat ion. The secur it y benefit s of t his feat ur e r ely upon all aut hent icat ing keys being kept confident ial. Also, w hen per for m ing r out er- m anagem ent t asks t hr ough SNMP, do not ignor e t he r isk associat ed w it h sending k ey s using nonencr y pt ed SNMP.

Pla in- Te x t Au t h e n t ica t ion Each par t icipat ing neighbor r out er m ust shar e an aut hent icat ing key. This key is specified on each r out er dur ing configur at ion. Mult iple keys can be specified w it h som e pr ot ocols; each key m ust be ident ified by a key num ber . I n gener al, w hen a r out ing updat e is sent , t he follow ing aut hent icat ion sequence occur s: St e p 1 . A r out er sends a r out ing updat e w it h a k ey and t he cor r esponding k ey num ber t o t he neighbor r out er . For pr ot ocols t hat can hav e only one k ey , t he key num ber is always 0. St e p 2 . The r eceiv ing ( neighbor ) r out er check s t he r eceiv ed k ey against t he sam e key st ored in it s own m em ory. St e p 3 . I f t he t w o k ey s m at ch, t he r eceiv ing r out er accept s t he r out ing updat e pack et . I f t he t w o k ey s did not m at ch, t he r out ing updat e pack et is r ej ect ed. M D 5 Au t h e n t ica t ion

176

MD5 aut hent icat ion w or ks sim ilar ly t o plain- t ex t aut hent icat ion, ex cept t hat t he k ey is nev er sent ov er t he w ir e. I nst ead, t he r out er uses t he MD5 algor it hm t o pr oduce a “ m essage digest ” of t he k ey ( also called a “ hash” ) . The message digest t hen is sent inst ead of t he k ey it self. This ensur es t hat nobody can eav esdr op on t he line and lear n k ey s dur ing t r ansm ission. An ex am ple of t he sequence inv olv ed in gener at ing t he hash is displayed in Figur e 4- 6 ( for t he originat ing rout er) and Figur e 4- 7 ( for t he dest inat ion r out er ) . These pr ot ocols use MD5 aut hent ic at ion: Fig u r e 4- 6 . H ow Rou t in g Pr ot ocol Au t h e n t ica t ion Op e r a t e s—Or ig in a t in g Ro u t e r

Fig u r e 4- 7 . H ow Rou t in g Pr ot ocol Au t h e n t ica t ion Op e r a t e s—D e st in a t ion Ro u t e r

177

• • • • •

OSPF RI P Ver sion 2 BGP EI GRP IS- IS

Rou t in g Pr ot ocol Au t h e n t ica t ion Su m m a r y The follow ing ar e som e ex am ples of how r out er aut hent icat ion is configur ed in OSPF, IS- I S, and BGP. OSPF:

interface ethernet1 ip address 10.1.1.1 255.255.255.0 ip ospf message-digest-key 100 md5 cisco ! router ospf 1 network 10.1.1.0 0.0.0.255 area 0 area 0 authentication message-digest IS- I S:

interface ethernet0 ip address 10.1.1.1 255.255.255.0 ip router isis isis password cisco level-2

178

BGP:

router bgp 200 no synchronization neighbor 4.1.2.1 remote-as 300 neighbor 4.1.2.1 description Link to Excalabur neighbor 4.1.2.1 send-community neighbor 4.1.2.1 version 4 neighbor 4.1.2.1 soft-reconfiguration inbound neighbor 4.1.2.1 route-map Community1 out neighbor 4.1.2.1 password cisco

Se cu r in g t h e N e t w or k The next st ep in I SP secur it y is building r esist ance t o at t acks and int r usions int o t he net w ork. For an I SP, t his can be divided int o t hree m aj or areas: • • •

Rou t e filt e r in g — Cont r olling w hich r out es ar e adver t ised and r eceived fr om sour ces out side y our aut onom ous sy st em ( AS) Pa ck e t f ilt e r in g— Ensur ing t hat pack et s t hat ent er t he net w or k are valid and hav e a sour ce addr ess m at ching t he or igin Ra t e lim it in g— Lim it ing t he am ount of som e t y pes of t r affic ( such as I CMP) t hat has no v alid r eason t o consum e huge chunk s of bandw idt h

Each t echnique is consider ed in t ur n in t he follow ing sect ions.

Egress a nd I ngress Filt ering Egr ess and ingr ess filt er ing ar e a cr it ical par t of an I SP’s r out er- configur at ion st r at egy . These t er m s ar e r elat iv e t o t he net w or k ingr ess/ egr ess filt er ing ar e applied: •

I ngr ess filt er ing applies filt er s t o t r affic com ing int o a net w ork from out side ( see Figure 4- 8) . This can be fr om an I SP’s cust om er s or t r affic fr om t he I nt er net at lar ge. Fig u r e 4- 8 . I n g r e ss Filt e r in g

179

•

Egress filt ering applies a filt er for all t r affic leav ing an I SP’s net w or k s ( see Figur e 4- 9) . I t is applied t o infor m at ion leav ing t he net w or k t o t he I nt er net or cust om er net w or k s. Fig u r e 4- 9 . Eg r e ss Filt e r in g

Be m indful t hat t hese t er m s ar e r elat iv e t o t he specific net w or k ’s point of v iew . For ex am ple, I SP B’s egr ess t r affic is I SP A’s ingr ess t r affic. I ngr ess/ egr ess filt er s help pr ot ect an I SP’s r esour ces and it s cust om er s’ net w or k s, allow s it t o enforce policy, and m inim izes t he r isk of being t he net w or k chosen by hack er s t o launch an at t ack on ot her net w or k s. I SPs ar e st r ongly encour aged t o dev elop st r at egies using egr ess and ingr ess filt er ing t o pr ot ect t hem selv es fr om t heir cust om er s and t he I nt er net at lar ge. By pr ot ect ing t hem selv es, t he I SPs ar e w or k ing t ow ar d pr ot ect ing t he I nt er net in gener al. BCP 38/ RFC 2827, “ Net w or k I ngr ess Filt er ing: Defeat ing Denial- of- Ser v ice At t ack s Which em ploy I P Sour ce Addr ess Spoofing” ( by P. Fer guson and D. Senie, May 2000, ht t p: / / w w w .iet f.or g/ r fc/ r fc2827.t xt ) pr ov ides gener al guidelines for all I SPs on

180

ingr ess and egr ess filt er ing. The I nt er net com m unit y now consider s t his docum ent best cur r ent pr act ice, and all I SPs should follow it s r ecom m endat ions.

Rout e Filt e r ing Rout e filt er ing r em ov es specific r out es fr om r out ing pr ot ocol adv er t isem ent s. Fr om t he cont ex t of t he I nt er net , r out e filt er ing is applied t o t he BGP ingr ess and egr ess adv er t isem ent s w it h an I SP’s cust om er s and peer s. I n t oday ’s I nt er net , r out e filt er ing is done t o m eet t w o obj ect ives. The fir st obj ect ive is t o keep specific net w or k s t hat should not be adv er t ised on t he net fr om pr opagat ing t o ot her net w or k s. For ex am ple, RFC 1918 addr esses ( w hich docum ent s pr iv at e addr ess space) , t he host subnet ( 127.0.0.0/ 8) , and m ult icast addr esses ( if m ult icast BGP is not used) should be filt er ed fr om an I SP’s ingr ess and egr ess adv er t isem ent s. [ 4] The second obj ect iv e is t o filt er sm aller net w or k adv er t isem ent s out of lar ger CI DR blocks. Som e I SPs leak out a m ore specific rout e from a larger CI DR block ( som et im es int ent ionally for oper at ional r easons, but m or e oft en accident ally or t hr ough lack of k now ledge) . These leak s incr ease t he r at e of gr ow t h of t he I nt er net Rout e Table. To discour age r out e leaking, som e I SPs r out e filt er on t he pr efix boundar ies of t he Regional I nt er net Regist r y ’s ( RI R’s) m inim al I Pv 4 allocat ion block . [ 5] The I SPs t hat do t his oft en ar e r efer r ed t o as net police—hence, t he nam e of t his t ype of r out ing filt er is called net police filt er ing. Exam ples of a r out ing filt er t o keep unr egist er ed block s out and net police filt er ing ar e pr ov ided in t his sect ion. N e t w or k s Th a t Sh ou ld N ot Be Ad v e r t ise d on t h e I n t e r n e t As m ent ioned ear lier , som e net w or k s ar e r eser v ed for special funct ions on t he I nt er net . These net w or k s should not appear in t he I nt er net Rout e Table; ex am ples are • • • • • • •

0.0.0.0/ 0 and 0.0.0.0/ 8 —Default and net w or k 0 ( unique and now hist or ical pr oper t ies) 127. 0. 0. 0/ 8—Host loopback 192.0.2.0/ 24 —TEST- NET gener ally used for exam ples in vendor docum ent at ion 10.0.0.0/ 8, 172.16.0.0/ 12, and 192.168.0.0/ 16 —RFC 1918 pr ivat e addr esses 169.254.0.0/ 16 —End- node aut oconfigur at ion net w or k in t he absence of DHCP Any adver t isem ent s less t han / 24 Any net w or k fr om an I nt er net eXchange Point ( I XP) m edium , such as t he Et her net segm ent

These last t w o could be cont r ov ersial: •

•

A / 24 ( t he hist or ical Class C addr ess) has been t he sm allest block of I Pv 4 addr esses ev er allocat ed fr om an I P r egist r y . Hence, t her e should be no adv er t isem ent s less t han a / 24 on t he I nt er net . Yet , for w hat ev er r eason, net w or k s below / 24 do appear in t he I nt er net Rout e Table. So it is up t o t he I SP t o decide w het her t o filt er t hem . Filt er ing t he I P net w or k s of t he I XPs is an opt ion t hat m any I SPs select . I XPs gener ally ar e not accust om ed t o pr ov iding t r ansit t o t he I nt er net , so t he I XP LAN does not need global visibilit y. For t his r eason, m any I SPs filt er t hese pr efixes on t he ingr ess and egr ess.

181

I n his I nt er net Dr aft , Bill Manning has docum ent ed t he net w or k s t hat hav e special uses in t he I nt er net . The cur r ent ver sion is ht t p: / / w w w .iet f.or g/ int er net - dr aft s/ dr aft m anning- dsua- 07.t xt at t he t im e of t his w r it ing. [ 6] The follow ing is a list of som e of t he US I XP net w or ks in ACL for m at . [ 7] Any one w ho w ant s t o im plem ent t his should fir st check t hat t hese ar e st ill v alid I XP net w or k blocks and should be pr epar ed t o put in t he effor t t o m aint ain t hem . Also, it m ight be w or t h ext ending t his list t o include all t he exchange point s in t he w or ld ( w w w .ep.net ) , a pot ent ially harder j ob.

! MAE-East primary IP block access-list 150 deny ip 192.41.177.0 0.0.0.255 ! ! MAE-East secondary IP block access-list 150 deny ip 198.32.186.0 0.0.0.255 ! ! MFS Portion of MAE-West access-list 150 deny ip 198.32.136.0 0.0.0.255 ! ! NASA/AMES - NASA Portion of MAE-West access-list 150 deny ip 198.32.184.0 0.0.0.255 ! ! Ameritech Advanced Data Services - AADS access-list 150 deny ip 198.32.130.0 0.0.0.255 ! ! Digital's Palo Alto Internet eXchange - PAIX access-list 150 deny ip 198.32.176.0 0.0.0.255 ! ! Sprint NAP - Pennsauken access-list 150 deny ip 192.157.69.0 0.0.0.255 ! ! Pac*Bell NAP access-list 150 deny ip 198.32.128.0 0.0.0.255 ! ! MAE-LA access-list 150 deny ip 198.32.146.0 0.0.0.255 ! ! MAE-Dallas access-list 150 deny ip 198.32.138.0 0.0.0.255 ! ! MAE-Houston access-list 150 deny ip 198.32.150.0 0.0.0.255 !

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

0.0.0.0 255.255.255.0

As discussed in Chapt er 3, “ Rout ing Pr ot ocols,” t he use of BGP dist r ibut e list is consider ed depr ecat ed by m any I SPs. The new er BGP pr efix list s ar e significant ly quick er t o load int o t he r out er and offer gr eat er usabilit y . They ar e also som ew hat easier t o configur e ( using nat ur al net w or k / pr efix lengt h r epr esent at ion) and use nam es rat her t han num b er s, m aking it easier t o keep t r ack of t he differ ent filt er s on t he r out er ( nam ed access list s also use nam es but hav e no per for m ance adv ant age ov er st andar d access list s.) Her e is an exam ple configur at ion and filt er using a BGP pr efix list :

182

router bgp 200 no synchronization neighbor 220.220.4.1 remote-as 210 neighbor 220.220.4.1 version 4 neighbor 220.220.4.1 prefix-list rfc1918-sua in neighbor 220.220.4.1 prefix-list rfc1918-sua out neighbor 222.222.8.1 remote-as 220 neighbor 222.222.8.1 version 4 neighbor 222.222.8.1 prefix-list rfc1918-sua in neighbor 222.222.8.1 prefix-list rfc1918-sua out no auto-summary ! ip prefix-list rfc1918-sua deny 0.0.0.0/0 ip prefix-list rfc1918-sua deny 0.0.0.0/8 le 32 Zero ip prefix-list rfc1918-sua deny 10.0.0.0/8 le 32 ip prefix-list rfc1918-sua deny 127.0.0.0/8 le 32 ip prefix-list rfc1918-sua deny 169.254.0.0/16 le 32 Autoconf ip prefix-list rfc1918-sua deny 172.16.0.0/12 le 32 ip prefix-list rfc1918-sua deny 192.0.2.0.0/24 le 32 ip prefix-list rfc1918-sua deny 192.168.0.0/16 le 32 ip prefix-list rfc1918-sua deny 224.0.0.0/3 le 32 ip prefix-list rfc1918-sua deny 0.0.0.0/0 ge 25 >/24 ip prefix-list rfc1918-sua permit 0.0.0.0/0 le 32 Everything Else !

! Default ! Network ! RFC1918 ! Hostnet ! Non-DHCP ! ! ! ! !

RFC1918 TESTNET RFC1918 Multicast prefixes

!

Ef f e ct s of CI D R- iz a t io n One of t he m ost cr it ical issues t hat could t hr eat en t he st abilit y of t he I nt er net is t he size of t he global I nt er net Rout e Table. The t able’s gr ow t h influences scalabilit y , incr eases oper at ional/ capit al cost , and has posed a secur it y r isk t o t he I nt er net . For a var iet y of r easons, I SPs t hr oughout t he w or ld hav e inj ect ed all sor t s of net w or k s int o t he I nt er net , r anging fr om / 8s ( old Class A’s) t o / 32s ( host r out es) . The r esult is r apid gr ow t h of t he I nt er net Rout e Table. Classless int er dom ain r out ing ( CI DR) has had a not iceable and significant im pact on t he gro w t h of t he t able. CI DR is a com binat ion of r ev ised I Pv 4 allocat ion policies ( for inst ance, t oday ’s nor m of pr ov ider- based addr essing) , updat es t o r out ing pr ot ocols ( RI Pv2, OSPF, I S- I S, and BGP) , and new address- aggr egat ion t echniques t o ensur e t hat only t he lar ge allocat ed blocks of addr esses get adver t ised int o t he global I nt ernet Rout e Table. The im pact of CI DR m oved t he grow t h rat e from an ex ponent ial cur v e t o linear gr ow t h. Ev en t hough t he int r oduct ion of CI DR did hav e an effect , it did not hav e as long t erm an effect as m any pr ovider s and indust r y obser ver s hoped it w ould. The linear gr ow t h r at e w as st ill fair ly st eep. Som e oper at or s on t he I nt er net began t o feel t hat t he r eal pr oblem w as lazy I SPs t hat eit her did not car e or did not k now how t o do proper aggr egat ion. Tw o appr oaches w er e t r ied t o encour age I SPs t o do t he r ight t hing. The fir st appr oach w as t o post of t he I nt er net CI DR Repor t t o t he cor e oper at ional m ailing list s w eekly. The second appr oach w as t o im plem ent filt er s on

183

t he r out es announced by each I SP, basically accept ing only t he m aj or allocat ed CI DR block s and no m or e specifics. Bot h appr oaches ar e in use t oday and ar e it em s t hat all I SPs need t o be aware of and consider. The CI DR Repor t fr om Tony Bat es ( t bat es@cisco. com) is a w eek ly analy sis of t he r at e of gr ow t h of t he I nt er net Rout e Table. The r epor t , sent t o all t he m aj or operat ions’ m ailing list s, provides t he nam e of I SPs inj ect ing t he m ost prefixes, aggr egat ed and unaggr egat ed. Ev er y one saw w ho w as causing t he gr ow t h and t hen could apply peer pr essur e t o hav e t he cor r ect ions m ade. Figur e 4- 10 ( t he up- t o- dat e v er sion of t his gr aph can be seen at ht t p: / / w w w .t elst r a.net / ops/ bgp) show s t he gr ow t h of t he I nt er net Rout e Table. Bet w een lat e 1995 and 1998, peer pr essur e helped t o st abilize t he gr ow t h, and indeed helped slow dow n t he gr ow t h r at e, by lat e 1998, as can be seen in Figur e 4- 10. As new er r out er s w it h fast er pr ocessor s and m or e m em or y capacit y w er e deploy ed, I SPs becam e less concer ned w it h t he gr ow t h in t he rout ing t able and hence lost int er est in pr ov iding or accept ing peer pr essur e. The CI DR r epor t s ar e int er est ing but ar e losing t heir ear lier im pact , and t hey ar guably ar e no longer ser v ing t heir or iginal funct ion. Fig u r e 4- 1 0 . BGP Rou t e Ta b le ( N ov e m b e r 1 5 , 2 0 0 1 ) f r om Te lst r a ’s AS 1 2 2 1 Po in t o f V ie w

The follow ing is a list of sit es collect ing dat a on t he I nt er net Rout e Table: Tony Bat es’s Daily CI DR Report : ht t p: / / w w w .em ploy ees.or g: 80/ ~ t bat es/ cidr - report .ht m l Philip Sm it h’s Daily Rout ing Analysis: ht t p: / / w w w . apnic. net / st at s/ bgp/ Geoff Hust on’s BGP Table Dat a:

184

ht t p: / / w w w . t elst r a. net / ops/ bgp/ bgp- act ive.ht m l The second appr oach t o encour aging I SPs t o use pr oper aggr egat ion w as t o inst all inbound pre fix filt er s on r out e announcem ent s. Sean Dor an ( sm d@clock . or g) w as one of t he fir st t o not ice and act on t he fact t hat t he new CI DR- based allocat ion policies fr om t he t hr ee RI Rs w er e not affect ing t he I SPs’ announcem ent s t o t h e I nt er net . I n Sean’s w or ds, What t r igger ed t he filt er w as t he obser v at ion t hat t he block s fr eshly allocat ed by all t hr ee r egist r ies w er e v er y poor ly aggr egat ed. Mor e annoy ingly , t hose allocat ed t o Spr int ’s pr incipal peer s ( m ost not ably I nt er net MCI ) dem onst r at ed t he w or st aggr egat ion; in one case, a / 14 w as announced alm ost ex clusiv ely as pr efix es no shor t er t han 19 bit s.[ 8] I n essence, CI DR was not being im plem e nt ed as or iginally int ended. Spr int placed one of t he fir st filt er s t hat enfor ced t he RI R’s default allocat ion size ( see Table 4- 1) . This filt er blocked any announcem e nt fr om any I SP t hat w as longer t han it s allocat ed address block. For exam ple, if an I SP w as allocat ed a / 19 from RI PE NCC’s 62/ 8 CI DR block, t his is all t hat should be announced t o t he I nt er net . The RI PE NCC had published it s m inim um allocat ion size, so Spr int could const r uct a filt er t hat w ould accept only a / 19 or less from 62/ 8. So a / 18 w ould pass t he filt er, but a / 20, / 21, / 22, / 23, / 24, and so on w ould not pass. The oper at ional effect s of t hese filt er s hav e been v ar iable. Sev er al I SPs j oined Spr int in im plem ent ing such filt er s. All t hr ee RI Rs hav e assist ed t he I SP com m unit y by publishing t heir default allocat ion sizes for all of t heir CI DR block s ( see Table 4- 1 f or a list of URLs) . Today, all t hree RI Rs have a m inim al allocat ion size of / 20 on t heir new CI DR blocks, m aking it pot ent ially easier for any I SP t o filt er on t he r egist r y allocat ion sizes if desir ed. Table 4- 1 show s w her e t o get t he list infor m at ion on each RI R’s m inim al allocat ion size for each of t he m acr o addr ess block s.

APNI C

Table 4-1. URLs to the RIR’s Default Allocation Sizes ht t p: / / w w w . apnic. net / docs/ add- m anage- policy .ht m l# 6.9

ARI N

ht t p: / / w w w .ar in.net / r egser v / I PSt at s.ht m l

RI PE NCC

ft p: / / ft p.r ipe.net / r ipe/ docs/ r ipe- 211.t xt

D o N e t Police Filt e r s H e lp Se cu r e a N e t w or k ? I n som e incident s on t he I nt er net , a r ogue r out er ( int ent ionally or unint ent ionally ) has deaggr egat ed t he addr ess bloc ks and st ar t ed adver t ising lot s of sm all, m or e specific net w or k s. I n one incident , t he I nt er net Rout e Table gr ew fr om 50K r out es t o m or e t han 120K r out es over a per iod of sever al m inut es. I SP secur it y people have been m indful t hat t his sor t of at t ack is t heor et ically feasible. Yet it also has been pr ov en t hat t he net police filt er is a det er r ent t o t his sor t of at t ack . For t his at t ack t o w or k, a r out er at t ached t o t he I nt er net and speaking BGP m ust t ake a lar ge CI DR block ( say, a / 19) and adver t ise m any m ore specifics ( say , all possible / 32s—8192 net w or k s) . A net police filt er ensur ing t hat a / 19 or less ( e.g., / 18) w ould pass t he filt er and t hat all ot her m or e specifics w ould be dr opped. The r esult is t hat t he at t ack w ould have m inim al effect on t he I SP w it h t he net police filt er, w hile ot her I SPs

185

w it hout t he net police filt er w ould get a huge influx of new announcem ent s, ov er loading t he m em or y of t heir r out er s and pot ent ially cr eat ing chaos on t he net . N OTE Depending on t he num ber of r out es adv er t ised, t here m ight be addit ional load on t he BGP pr ocess as it dr ops all t hese unw ant ed net w or k s. Also, if BGP soft r econfigur at ion is t ur ned on, each of t he dr opped net w or k s st ill w ould get sav ed.

N e ga t iv e I m pa ct of N e t Police Filt e r s The m aj or dow nside, if not complet e show st opper , for im plem ent ing net police filt er s on a net w ork is t heir im pact on legit im at e m ult ihom ing. This is especially felt in dev eloping par t s of t he I nt er net w her e new I SPs hav e r eceiv ed t heir fir st allocat ion from t he regional regist ry and hav e a basic r equir em ent t o be m ult ihom ed as par t of t heir business. Their / 20 net w or k block is all t hey hav e; if t hey subdiv ide t his, t hey fall foul of t he net police filt er s t hat t hat have been put in place. I t is for t his r eason t hat m any I SP operat ors out side t he Unit ed St at es fr ow n upon t he concept . Many hour s hav e been w ast ed t r y ing t o m ak e m ult ihom ing w or k , only t o be t hw ar t ed by net police filt er s im plem ent ed by an I SP in t he Unit ed St at es. As w it h all m ult ihom ing scenar ios, t he I SP alw ays should anno unce it s net w or k block. This ensur es connect iv it y t o t he I nt er net . How ev er , announcing t w o / 21s, t he fir st st ep t o basic inbound load shar ing of t r affic, w ill be im pact ed by t heir upst r eam s im plem ent ing RI R m inim um allocat ion- size filt ers. Encount ering any sit uat ion like t his w ill r equir e cooper at ion of upst r eam I SPs, or , in t he w or st case, consider ing m ov ing cust om elsew her e. Cr e a t in g You r Ow n N e t Police Filt e r I SPs t hat w ant t o cr eat e t heir ow n net police filt er s ar e st r ongly encour aged t o do t he follow ing: • • • • •

Con sid e r t h e im p a ct — Shut t ing t he door doesn’t m ake t he st or m go aw ay —it shut s out ev er y t hing, good and bad. M a in t a in a n a ccu r a t e list — Consult each of t he RI R’s published CI DR block s for t he default allocat ions. Ensur e t hat y ou hav e an up- t o- date list , and cr eat e a pr ocess for v alidat ing y our filt er w it h fut ur e updat es of t he RI R’s list . Con su lt w it h colle a gu e s — Consult w it h y our peer s about t he list . I f possible, com par e w hat ot her s hav e done w it h w hat y ou w ant t o accom plish. Pu b lish p olicy — A Web page w it h t he filt er policy is ext rem ely helpful. Refer t o t he NANOG list of filt er policies as an exam ple ( ht t p: / / w w w .nanog.or g/ filt er .ht m l) . Also put a com m ent in y our AS obj ect . Ke e p it u p- t o- d a t e !— This is t he m ost im port ant part —and oft en t he har dest t o do. Be sensit iv e t o com plaint s, especially about r eachabilit y fr om your cust om ers or t heir cust om ers —w it hout t hem , y ou hav e no business.

186

The follow ing is j ust one exam ple of a net police filt er . I SPs can use t his as a st ar t ing point but ar e st r ongly encour aged t o ver ify t hat ever yt hing in t he list w ill w or k in t heir env ir onm ent s.

! !! ip ip ip ip ip ip ! !! ip ip ip ip ip ! !! ip ip ip ip ip ip ip ip ip ! !! ip ! !!

RIPE prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list

FILTER FILTER FILTER FILTER FILTER FILTER

permit permit permit permit permit permit

62.0.0.0/8 ge 9 le 20 80.0.0.0/7 ge 9 le 20 193.0.0.0/8 ge 9 le 20 194.0.0.0/7 ge 9 le 20 212.0.0.0/7 ge 9 le 20 217.0.0.0/8 ge 9 le 20

APNIC prefix-list prefix-list prefix-list prefix-list prefix-list

FILTER FILTER FILTER FILTER FILTER

permit permit permit permit permit

61.0.0.0/8 ge 9 le 20 202.0.0.0/7 ge 9 le 20 210.0.0.0/7 ge 9 le 20 218.0.0.0/7 ge 9 le 20 220.0.0.0/8 ge 9 le 20

ARIN prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list

FILTER FILTER FILTER FILTER FILTER FILTER FILTER FILTER FILTER

permit permit permit permit permit permit permit permit permit

24.0.0.0/8 ge 9 le 20 63.0.0.0/8 ge 9 le 20 64.0.0.0/6 ge 9 le 20 68.0.0.0/8 ge 9 le 20 199.0.0.0/8 ge 9 le 20 200.0.0.0/8 ge 9 le 20 204.0.0.0/6 ge 9 le 20 208.0.0.0/7 ge 9 le 20 216.0.0.0/8 ge 9 le 20

General - Filter anything greater then /24 prefix-list FILTER deny 0.0.0.0/0 ge 25 Other filters...

Not ice t hat only announcem ent s of pr efix sizes bet w een / 9 and / 20 ar e per m it t ed. No r egist r y w ill allocat e t he ent ir e / 8 block t o one I SP ( at least , not so far ) , so a / 8 announcem ent shouldn’t be seen fr om t hese block s. Lik ew ise, because a / 20 is t he m inim um allocat ion by t he r egist r ies, not hing sm aller should be seen. Som e I SPs ar gue t hat t he pr eceding r anges should be / 9 t hr ough / 21 or / 22, t o allow t he possibilit y of im plem ent ing m ult ihom ing for t he sm aller I SPs. Pr ov ider s t hat ar e im plem ent ing t hese filt er s should consider w hat t hey r easonably w ant t o per m it in t heir net work.

Pa ck e t Filt e r ing DoS at t ack s, spoofing, and ot her for m s of at t ack s ar e on t he incr ease on t he I nt er net . Many of t hese at t ack s can be t hw ar t ed t hr ough t he j udicious use of ingr ess ( pack et s or iginat ing fr om y our net w or k ) and egr ess ( pack et s ar r iv ing fr om t he I nt er net ) pack et filt er ing. I n pack et filt er ing, t he r out er or sw it ch inspect s t he pack et ’s sour ce or dest inat ion infor m at ion and eit her passes or dr ops a pack et based

187

on a rule set . I P addresses and port num bers are t he m ost com m on inform at ion used in t he r ule set . N OTE This sect ion cover s t he sim ple case of single - hom ed dow nst r eam cust om er s. Mor e t hought is req uir ed w hen applying packet filt er ing for I SPs t hat ar e m ult ihom ed. For exam ple, w hen t he cust om er I SP’s link t o your net w or k goes dow n, you w ill see pack et s fr om t hat net w or k com ing fr om “ t he I nt er net .” Also be aw ar e t hat a m ult ihom ed cust om er m ight r equire special r out ing t o im plem ent load shar ing; in t hat case, y ou w ill again see t hat I SP’s t r affic com ing fr om “ t he I nt er net .”

Cisco has sev er al w ay s of inspect ing and dr opping pack et s: • • •

ACLs — An access list is used t o m at ch on t he sour ce/ dest inat ion I P add r ess, t he pr ot ocol, or TOS t o dr op a pack et . Bla ck- h ole f ilt e r in g ( f or w a r d in g t o N u ll0 ) — The for w ar d t able r out es t he pack et based on dest inat ion I P addr ess t o t he adj acency Null0, dr opping t he p ack et . Unica st RPF — An MTRI E lookup int o t he FI B is done based on t he sour ce I P addr ess. I f t he sour ce I P addr ess does not m at ch t he infor m at ion in t he for w ar d t able, t he pack et is dr opped. N ot e An MTRI E is a st orage m echanism m ade up of nodes and leaves, used in CEF.

•

Com m it t e d Acce ss Ra t e ( CAR) — When t he com m it t ed r at e is set t o 0, all packet s ar e dr opped. CAR can m at ch packet s based on ACLs or MTRI E lookups int o t he FI B’s QoS- I D field.

Each of t hese pack et - dropping t echniques has various packet - per- second ( PPS) per for m ance im pact s. Like any secur it y t ool, r isks need t o be w eighed against t he benefit s. N OTE A per for m ance im pact could affect t he for w ar ding speed of t he r out er w hen m any filt er s ar e applied. Cisco’s new er sw it ching t echnologies help m inim ize t he per for m ance im pact . Tur bo ACLs ( com piled access list s) are available from I OS Soft w ar e Release 12.0( 6) S and giv e super ior per for m ance for access list s m or e t han 10 ent r ies long. Due car e and consider at ion should be t ak en w henev er t he access list st ar t s get t ing bey ond 50 ent r ies. How ev er , it should be st at ed t hat t r ading a few m icr oseconds of I P for w ar ding speed for t he safet y of m inim izing t he im pact of DoS at t ack s could pr ov e t o be w or t hw hile.

188

Acce ss Con t r ol List s: Ge n e r a l Se qu e n t ia l - Ba se d ACLs ACLs ar e t he fir st opt ion for a lot of or ganizat ions looking t o enfor ce policy and pr ov ide a lay er of secur it y . At t he sam e t im e, ACLs hav e t he gr eat est scaling, per for m ance, and secur it y r isk . Gener ally , t r adit ional ACLs ar e pr ocessed sequent ially ( see Figur e 4- 11) . Any r out er pr ocessing a pack et t hr ough an ACL look s for a m at ch on a list of ACL ent ries; each line of t he ACL m ust be checked unt il t here is a m at ch. I f t he ACL is an ext ended ver sion, t he I P payload, por t s, and applicat ion also m ust be checked sequent ially ( see Figur e 4- 12) . Because m any policy and secur it y filt er s ar e ACLs w it h a lot of deny st at em ent s ( see Exam ple 4- 2) , a v er y long ACL incr eases t he pack et lat ency and consum es m or e CPU cy cles. Fig u r e 4- 1 1 . St a n da r d Acce ss List —I n com in g Pa ck e t s

Fig u r e 4- 1 2 . Ex t e n de d ACLs —Log ica l Flow f or Ev e r y Pa ck e t

189

N OTE The gener al pr inciple of ACLs and t he processing involved are issues for all rout ers, not j ust Cisco r out er s. Cisco has t ools such as Net Flow and Opt im um sw it ching t hat incr ease t he per for m ance of t he ACL check s, y et ev en t hese hav e t heir lim it s.

Ex a m p le 4- 2 St a n d a r d Acce ss List w it h Lot s of d e n y St a t e m e n t s access-list access-list access-list . . many deny . access-list access-list

25 deny 165.21.10.10 25 deny 171.68 34.1 25 deny 192.34.5.10 entries 25 deny 141.43.10.100 25 permit all

As t he lengt h of t he policy filt er incr eases, t he bur den on t he pr ocessing pow er of t he r out er r eaches a point at w hich it is at 99 per cent CPU ut ilizat ion. Added t o t his is t he 100 per cent t o 200 per cent gr ow t h r at e of t he I nt er net . This m eans t hat as t he PPS

190

load on t he r out er incr eases fr om t he nat ur al gr ow t h of t he net w or k, t he CPU load and pack et lat ency fr om ACL pr ocessing also incr ease, affect ing t he ov er all end- t oend bandw idt h per for m ance. To k eep t he ACL a r elev ant t ool on t he I nt er net , im pr ov em ent s need t o be added t o allow ACLs t o scale. Fast er pr ocessor s, dist r ibut ed pr ocessing, dedicat ed ASI C/ TCAMs, and special flow - based swit ching t echnologies hav e been added t o v ar ious Cisco pr oduct s t o allow ACLs t o cont inue t o k eep pace w it h t he I nt er net ’s gr ow t h. Yet , w hile k eeping pace w it h t he gr ow t h of t he I nt er net , it is under st ood t hat sequent ial ACLs hav e lim it at ions. One w ay ar ound t hese lim it at ions is t o use a new w ay t o pr ocess pack et s t hr ough an ACL: Tur bo ACLs.

Access Cont rol List s: Turbo ACLs Tur bo ACLs use a t echnique t hat t ak es a st andard or ex t ended ACL, cr eat es a set of dat a t ables, and com piles t hem for r unt im e pr ocessing. For t his r eason, Tur bo ACLs also ar e r efer r ed t o as com piled ACLs. Tur bo ACLs do not change t he “ fir st m at ch w ins” char act er ist ic of all ACLs. I nst ead, t hey r educe t he num ber of CPU oper at ions t o find a m at ch, allow ing for lar ger ACLs t o be used w it hout an incr ease in packet lat ency . This pr ov ides I SPs w it h a t ool t o allow lar ge ACLs w it hout a significant per for m ance im pact on t he r out er . As seen in Figur e 4- 13, w hen a Tur bo ACL is act iv at ed in a r out er , it t ak es t he st andar d ACL input , cr eat es t ables based on t he ACLs ent r ies, and com piles t he t ables t o allow an ar r ay ed m at ch. The r esult is t hat a m at ch is achieved in five st eps, no m at t er w hat t he size of t he ACL is. This also m eans t hat a Turbo ACL’s advant ages becom e clear only w hen t he ACL is longer t han five ent r ies. So ACLs w it h 3 or 5 lines w ould out per for m Tur bo ACLs, but Turbo ACLs wit h 300 t o 500 lines w ould out per for m sequent ially sear ched ACLs. Figur e 4- 14 show s t he r esult of one st udy on t he per for m ance differ ence bet w een sequent ial ACLs and Tur bo ACLs. As t he lengt h of t he ACL incr eases, t he t im e t hat it t ak es a sequent ial ACL t o m at ch incr eases ( assum ing a last - line m at ch) . Tur bo ACLs pr ov ide consist ency t hr oughout t he lengt h of t he ACL. The lim it in t he lengt h of a Tur bo ACL has mor e t o do w it h t he har dw ar e per for m ance env elope; m em or y , TCAM size, ASI C size, buffer ing, and CPU cycles are all fact ors t hat lim it t he m axim um size of a Turbo ACL. Fig u r e 4- 1 3 . Tu r b o ACLs: Pr oce ssin g Ta k e s Fiv e St e p s, Re g a r d le ss of Siz e

191

Fig u r e 4- 1 4 . Tu r b o ACL Pe r f or m a n ce Con sist e n cy

Tu r bo ACL Con f ig u r a t ion D e t a ils a n d Re f e r e n ce s To act iv at e Tur bo ACLs in a Cisco r out er , t he follow ing global com m and is used:

access-list compiled This applies Tur bo ACLs t o all ACLs on t he rout er, no m at t er w hat t heir size is. Turbo ACLs w er e int r oduced in I OS Soft w ar e Release 12.0( 6) S. Key w or d sear ches on Cisco. com or ht t p: / / w w w . cisc o.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 120new ft / 120lim i t / 1 2 0 s/ 1 2 0 s6 / t u r boacl. h t m w ill pr ovide configur at ion infor m at ion on Tur bo ACLs. Full det ails of how Tur bo ACLs w or k w er e giv en by aut hor Andr ew McRae ( am cr ae@cisco. com) , in t he paper “ High- Speed Pack et Classificat ion,” pr esent ed at t he Aust r alian UNI X User s Gr oup nat ional confer ence in Sept em ber 1999 ( ht t p: / / w w w .em ploy ees.or g/ ~ am cr ae/ paper s/ pack et _class/ ) .

192

ASI C- Based ACLs As t he bandw idt h and PPS r at es incr eased, har dw ar e v endor s st ar t ed t o consider ot her pack et - filt er ing t echnologies. Unt il quit e r ecent ly, all packet filt er ing w as done in soft w ar e based ar ound a CPU. This could be a cent r ally sw it ched ar chit ect ur e ( such as a Cisco 72XX) or a dist r ibut ed sw it ch ar chit ect ur e w it h line car ds t hat w er e CPU- based ( such as a Cisco 75XX w it h VI P car ds, a Cisco 76XX w it h flex WAN car ds, or a Cisco 12XXX w it h Engine 0 line cards) . The alt er nat ive t o doing packet filt er ing in soft w ar e loaded on a CPU is t o m icr ocode t he ACL ont o an ASI C. ASI C- based ACLs use t he st r engt hs of specifically designed har dw ar e t o acceler at e t he r esolut ion of an ACL. Som e ASI Cs could be m ission- specific, w it h ACLs added as a supplem ent ( for ex am ple, Salsa on t he Cisco 12000’s Engine 1 line car ds) . Ot her ASI Cs ar e v er y specific and ar e opt im ized j ust for ACL look up ( for ex am ple, TCAMs on t he Cisco 7600 and fut ur e pr oduct s) . The im pact is t hat ACLs in ASI C hav e differ ent per for m ance char act er ist ics t han ACLs pr ocessed by gener al soft w ar e on a CPU: •

•

•

The fir st differ ence is per for m ance, w hich show s a dr am at ic im pr ov em ent . For som e, it is an incr ease in t he r at e of PPS but is im pact ed by t he dept h of t he ACL ( as m ent ioned ear lier , t he dept h of an ACL im pact s t he per for m ance of som e ACLs, t ak ing longer t o r each a m at ch) . Ot her s show no PPS im pact , allow ing full sw it ching up t o t he m axim um dept h of t he ACL ( t his is 15,000 lines in a 7600’s TCAM) . The second char act er is t ic differ ence is t hat som e capabilit ies of t he ACL ar e lim it ed. For ex am ple, ASI C- based ACLs usually r equir e a pr ecom pilat ion on t he r out er befor e it is loaded int o t he ASI C. This pr ecom pilat ion happens behind t he scenes aft er an ACL has been updat ed. How ev er , by com piling t he ACL, som e per- ACE infor m at ion is lost . For ex am ple, an oper at or can get t he aggr egat e count er s on m ost ASI C- based ACLs but cannot get t he per- ACE count er s ( ACE st ands for access cont r ol ent r y , a single line in an access cont r ol list ). Finally , ASI C- based ACLs have an ACE dept h lim it . Soft w ar e - based ACLs can rely on shared m em ory of a line card or a rout e processor, but hardware based ACLs ar e r est r ict ed t o t he m em or y design size of t he ASI C. Ther efor e, t he num ber of ACEs in a soft w ar e - based ACL pot ent ially can be very large: w her eas, t he size of t he har dw ar e ACE pot ent ially is lim it ed. This put s a m ax lim it on t he num ber of line ( ACEs) t hat an ACL can handle. For som e ASI Cs, t his num ber is low —t he 12000’s Engine 2 PSA ASI C can up t o 448 ACEs. For ot her ASI Cs, it is high—t he 7600’s TCAM can go up t o 15,000 ACEs.

Tak en t oget her , t hese ASI C- based ACL char act er ist ics offer t he oper at or new st r engt hs but also new lim it at ions. ( I t is a com m on secur it y pr inciple t hat no new lev el of secur it y c om es w it hout new lim it at ion.) Operat ors need t o be m indful of t hese lim it at ions as t hey use ACLs in t heir sy st em of apply ing policy and secur it y t hrough t heir net works. Sa lsa ACLs in t h e Cisco 1 2 0 0 0 En g in e 1 Lin e Ca r d [ 9 ] The Salsa ASI C is a specialized chip t hat assist s t he line car d’s CPU in pack et pr ocessing ( feat ur es) and r out e- look up ( for w ar ding) funct ions. I ngr ess ( input ) ACLs ar e t he key packet - processing feat ure t hat t he ASI C opt im izes. By doing t he input

193

ACL lookup as it does r out e lookup, t he Salsa ASI C fr ees up t he line car d’s CPU, allow ing for consider ably fast er packet pr ocessing. Because t his is only a fir st gener at ion im plem ent at ion of ACLs in ASI Cs, som e lim it at ions st ill ex ist : • • •

Only ingr ess ACLs ar e suppor t ed. Subint er faces ( such as VLANs or Fr am e Relay ) ar e not suppor t ed. Salsa uses t he shar ed m em or y of t he line car d, so t her e ar e no har d m ax im um lim it s t o t he num ber of ACE ent r ies. Salsa suppor t for ACLs is enabled by t he configur at ion com m and a cce ss- list h a r d w a r e sa lsa .

I f t his com m and is not in t he configur at ion, ACLs w ill st ill w or k, but t hey w ill be evaluat ed on t he line car d’s CPU and t hus t he line car d w ill show m uch slow er pack et - for w ar ding per for mance t han w ould hav e been possible w it h t he ASI C- based ACLs. PSA ACLs in t h e Cisco 1 2 0 0 0 En g in e 2 Lin e Ca r d The Cisco 12000’s Engine 2 line car d t ook t he appr oach of using t he spar e capacit y in t he PSA for w ar ding ASI C t o apply ACLs. Specific m icr ocode needs t o be loaded w hen t he har dw ar e- based ACL is applied t o t he PSA. The r esult is significant PPS per for m ance, w it h ACL dept h det er m ined t o be accept able for m ost I SP oper at ions ( ar ound 128 lines of ACE w it h 448- line capabilit y) . Alt hough t her e is a PPS adv ant age w it h t his im plem ent at ion, t her e ar e also lim it at ions. I f t hese lim it at ions ar e ex ceeded, t he line car d is designed t o m ov e t he ACL’s funct ion out of t he PSA for w ar ding ASI C int o t he line car d’s CPU, im pact ing t he pot ent ial for w ar ding r at e. Check w it h t he lat est GSR pr oduct docum ent at ion for t hese lim it at ions. ( The lim it at ions usually evolve ar ound m ax ACE lim it s ver sus dCEF t able size.) When a PSA ACL is applied t o a r out er , it is enabled by default . The gener al I OS Soft w ar e com m and n o a cce ss list h a rd w a r e p sa can be used t o disable t his feat ur e.

Using ACLs for Egress Pack et Filt ering: Prevent ing Tra nsm ission of I nva lid I P Addresses Egr ess pack et filt er ing ensur es t hat t he pack et s t hat y ou send out t o ot her net w or k s ( I SPs or cust om er s) ar e v alid. I t can be applied eit her on t he gat ew ay ( upst r eam— peer ing r out er s) or on t he cust om er edge. By filt er ing packet s on your r out er s t hat connect y our net w or k t o t he I nt er net ( see Figur e 4- 15) , y ou can per m it only pack et s w it h v alid sour ce I P addr esses t o leav e y our net w or k and get int o t he I nt er net . For ex am ple, if y our net w or k consist s of net w or k 165.21.0.0 and y our r out er connect s t o your I SP using a ser ial 0/ 1 int er face, you can apply t he access list as follow s: Fig u r e 4- 1 5 . Eg r e ss Pa ck e t Filt e r in g on t h e U p st r e a m Ga t e w a y Rou t e r

194

access-list 110 permit ip 165.21.0.0 0.0.255.255 any access-list 110 deny ip any any log ! interface serial 0/1 description Upstream Connection to ISP A ip access-group 110 out ! The last line of t he access list det er m ines w het her t her e is any t r affic w it h an invalid sour ce addr ess ent er ing t he I nt er net . I f t her e ar e any m at ches, t hey w ill be logged. I t is not cr ucial t o hav e t his line, but it helps locat e t he sour ce and ex t ent of t he possible at t ack s. Som e I SPs consider t he opt ion of egr ess filt er ing on t he cust om er edge of t heir n et w orks ( see Figur e 4- 16) . I n t his case, t he obj ect iv e is t o ensur e t hat pack et s bound for t he cust om er do not hav e a sour ce I P addr ess w it hin t he r ange of addr esses allocat ed by t he I SP t o t he cust om er . For ex am ple, a pack et w it h a dest inat ion addr ess of 165.21.10.1 should not hav e a sour ce addr ess of 165.21.10.1. These ar e spoofed pack et s and should be dr opped. The follow ing is an ex am ple of an egr ess packet - filt er ACL on t he cust om er ’s int er face: Fig u r e 4- 1 6 . Eg r e ss Pa ck e t Filt e r in g on t h e Cu st om e r Ed g e

access-list 121 deny ip 165.21.0.0 0.0.255.255 any

195

access-list 121 permit ip any any log ! interface serial 1/1/1.3 description T1 Link to Customer Acme Computer systems ip access-group 121 out ! The k ey fact or t hat lim it s t his t echnique is t he m agnit ude of t he im plem ent at ion. I SPs w it h 10,000 business cust omer s do not see t he benefit s gained by m anaging 10,000 ACLs, especially w hen t he cust om er is encour aged t o im plem ent ingr ess filt er ing on it s gat ew ay r out er .

Using ACLs for I ngress Pack et Filt ering: Prevent ing Recept ion of I nva lid I P Addresses I ngress packe t filt er ing v alidat es t he pack et s fr om t he out side w or ld ( I SPs and cust om er s) int o and acr oss y our net w or k . For an I SP, t he out side w or ld is any place out side t he I SP’s cont r ol. Obv iously , pack et s fr om ot her I SPs ar e fr om t he out side w or ld and ar e not t o be im plicit ly t rust ed. This also m eans t hat packet s from an I SP’s cust om er s ar e fr om t he out side w or ld. Just because a net w or k is a cust om er of an I SP does not m ake it a t rust ed net work. For I SPs t hat pr ov ide ser v ice t o end net w or k s, w e highly r ecom m end t he validat ion of incom ing pack et s fr om client s. This can be accom plished by t he use of inbound pack et filt er s on y our bor der r out er s. Figur e 4- 17 is an ex am ple in w hich an I SP has one gat ew ay t o it s upst r eam net w or k . Cust om er s w it h a net w or k num ber of 165.21.10.0/ 24 should not see any pack et s com ing fr om t he I nt er net w it h 165.21.10.1 as t he sour ce addr ess. These pack et s ar e at t em pt s at spoofing and should be dr opped. The follow ing exam ple show s a sam ple filt er for net w ork 165.21.0.0 w it h filt er s for pr iv at e and r ogue r out es: Fig u r e 4- 1 7 . I n g r e ss Pa ck e t Filt e r in g on t h e U p st r e a m Ga t e w a y Rou t e r

access-list access-list access-list access-list

111 111 111 111

deny deny deny deny

ip ip ip ip

host 0.0.0.0 any log 127.0.0.0 0.255.255.255 any log 10.0.0.0 0.255.255.255 any log 172.16.0.0 0.15.255.255 any log

196

access-list 111 deny ip 192.168.0.0 0.0.255.255 any log access-list 111 deny ip 165.21.0.0 0.0.255.255 any log access-list 111 permit ip any any ! interface serial 1/0 ip access-group 111 in ! All t he ant ispoofing, pr iv at e addr ess, and r ogue filt er s hav e a n y l og m at ches. I t is not cr ucial t o hav e t his line, but it helps locat e t he sour ce and ex t ent of t he possible pr obes or at t ack s. Figure 4- 18 pr ov ides an illust r at ion of ingr ess filt er ing on t he upst ream gat ew ay rout er. Fig u r e 4- 1 8 . I n g r e ss Pa ck e t Filt e r in g on t h e I SP/ Cu st om e r Ed g e ( I ETF BCP Re com m e n d a t ion )

Alt hough t he ex am ple j ust giv en w ill w or k for all cust om er s connect ing t o t he I nt er net and m any sm all- t o m edium- size I SPs, it is in addit ion t o w hat is r ecom m ended in BCP 38/ RFC 2827 “ Net w or k I ngr ess Filt er ing: Defeat ing Denial- ofSer vice At t acks Which Em ploy I P Sour ce Addr ess Spoofing,” by P. Fer guson and D. Senie. BCP 38 r ecom m ends ingr ess filt er ing on t he I SP/ cust om er edge of t he net w or k . This filt er ing ensur es t hat a sour ce addr ess of a pack et m at ches t he I P addr ess block allocat ed t o t he cust om er . For ex am ple, a cust om er w ho has been allocat ed an I P addr ess block of 165.21.10.0/ 24 should not send any pack et s w it h a sour ce addr ess of 192.168.1.1. The follow ing is t he ex am ple of an ingr ess pack et filt ering ACL for t he cust om er:

access-list 122 permit ip 165.21.0.0 0.0.255.255 any access-list 122 deny ip any any log ! interface serial 1/1/1.3 description T1 Link to Customer Acme Computer systems ip access-group 121 in ! Not ice t hat t he ACL is one line long and per m it s only aut hor ized pack et s. The second line of t he ACL, d e n y ip a n y a n y log , is opt ional. The one- or t wo- line ACL is easier

197

t o im plem ent . The second de n y a n y line is opt ional, but it does capt ur e t he pack et s t hat ar e denied and logs t hem in t he r out er ’s sy slog. The obj ect iv e is t o log pack et s t hat ar e in violat ion of t he access list , allow ing for t he det ect ion of com pr om ised syst em s. The lim it at ion of t his t echnique ar ises w hen t her e ar e t housands of leased- line cust om er s; how is it possible t o m anage t housands of ACLs in a m anner t hat is scalable for t he r out er , for cust om er suppor t per sonnel, and for t he engineer ing st aff r unning t he net w or k ? The answ er is not t o use t he access list t echnique, but t o use unicast RPF, w hich is discussed in dept h lat er in t his chapt er .

Bla ck -H ole Rout ing as a Pack et Filt er ( Forw arding t o N u ll0 ) Anot her w ay of im plem ent ing dest inat ion- based packet filt er ing on a r out er is t o cr eat e a specific list of st at ic host r out es and point t hem t o t he pseudo- int er face N ull0 . This t echnique com m only is r efer r ed t o as black- hole rout ing. Null0 is a pseudo- int er face, w hich funct ions sim ilar ly t o t he null dev ices av ailable on m ost oper at ing syst em s. This int er face is alw ays up and can never for w ar d or r eceive t raffic. Alt hough Null0 is a pseudo- int er face, w it hin CEF it is not a v alid in t er face. Hence, w henever a r out e is point ed t o Null0, it w ill be dr opped t hr ough CEF and dCEF’s for w ar ding pr ocess w it h no pr ocessor ov er head. The null int er face pr ov ides an alt er nat iv e m et hod of filt er ing t r affic. You can av oid t he over head involved w it h using access list s by dir ect ing undesir ed net w or k t r affic t o t he null int er face. The follow ing ex am ple configur es a null int er face for I P r out e 127.0.0.0/ 16 and t he specific host 171.68.10.1 ( subnet m ask 255.255.255.255) :

interface Null0 no icmp unreachables ! ip route 127.0.0.0 255.0.0.0 null 0 ip route 171.68.10.1 255.255.255.255 null 0 Th e n o icm p u n r e a ch a b le s com m and is used t o pr ev ent unnecessar y r eplies w henev er t r affic is passed t o t he Null0 int er face. Not e t hat r at e lim it ing should be applied t o t he icm p - unr eachables opt ion so t hat t he r out er CPU does not get bogged dow n dealing w it h responses t o discarded t raffic. Consensus from I SPs using icm p unr eachables r ecom m ends t he follow ing configur at ion:

ip icmp rate-limit unreachable DF 2000 ! interface Null0 no icmp unreachables ! This r at e- lim it s t he icm p - unr eachables r esponse t o one ev er y t w o seconds, w it h t he DF bit set ( t he I OS Soft w are default is 500 m s) .

198

Figur e 4- 19 giv es a gr aphic ex am ple of how t his black- list filt er ing t echnique w or k s, com par ing t he pack et pat hs w it h using an access list . By using t he black- hole rout ing t echnique t o im plem ent black- list filt er ing ( no pun int ended) , t he st r engt h of t he r out er is ut ilized t o dr op packet s bound for for bidden sit es. A r out er ’s pr im ar y funct ion is t o for w ar d pack et s, not filt er pack et s. The black- hole rout ing t echnique uses t he pack et - for w ar ding pow er t o dr op all packet s bound for sit es on t he blac k list . Fig u r e 4- 1 9 . U sin g St a t ic H ost Rou t e s t o N u ll0 f or Bla ck- List Filt e r in g

Tw o m ain dr aw back s t o t his t echnique ex ist . Fir st , access t o all serv ices at a giv en sit e m ust be r est r ict ed. The at t r act ion of ex t ended ACLs is t he fine gr anular it y giv en t o filt er at t he applicat ion lev el. Black- hole r out ing does not hav e t his gr anular it y . I t filt er s access for all packet s bound t o a specific host or subnet . Second, it is har d t o by pass t he black- hole r out ing t echnique. Any or ganizat ion t hat w ant s t o by pass t he black list act ually m ust find a w ay t o bypass t he filt er ing r out er ’s for w ar ding t able. Com pensat ing for eit her lim it at ion is a nont r ivial t ask.

BCP 3 8 Usin g Un ica st RPF [ 1 0 ] BCP 38/ RFC 2827 pr ov ides a pr oact iv e st ep t o pr ev ent pr oblem s caused by pack et s w it h m alfor m ed or for ged I P sour ce addr esses passing t hr ou gh a r out er . I t is a w onder ful secur it y concept and w ill pr ev ent m any secur it y pr oblem s on t oday ’s I nt er net . How ev er , I SPs st ill hav e not w idely adopt ed BCP 38, m ost ly because of concer ns about scalabilit y and t he im pact of t he t echnique on t he m anagem ent of t heir r out er s. Consider an I SP t hat w ould like t o do BCP 38 w it h ACLs but is aggr egat ing 10,000 leased- line cust om er s ont o it s backbone. This m eans t hat 10,000 separ at e ACLs need t o be cr eat ed, m aint ained, and updat ed as new addr esses ar e allocat ed t o t he I SP. Now consider an I SP w it h 100,000 leased- line cust om er s, 500,000 leased- line cust om er s, and so on. You can easily see t he scalabilit y pr oblem posed w hen t r y ing t o im plem ent BCP 38. Ev en scr ipt ed as par t of configur at ion- m anagem ent t ools,

199

im plem ent in g BCP 38 using ACLs adds scalabilit y bur dens for an over w or ked I SP oper at ions and deploy m ent t eam . Som e w ay t o aut om at ically per for m t he BCP 38 check w it hout t he headache of ACLs is needed. This is w her e unicast r ev er se pat h for w ar ding ( uRPF) pr ov ides t he per fect solut ion. uRPF or iginally w as cr eat ed t o aut om at ically achiev e t he BCP 38 pack et filt er ing w it hout t he headaches of ACLs. I t is a one- line int er face configur at ion t hat can be placed in an I SP’s deploy m ent scr ipt s. Hence, any inst allat ion engineer c an apply BCP 38 filt er ing as soon as t he leased line is inst alled t o t he sit e of t he new cust om er . No ACLs, no need t o m aint ain ACLs, and no need t o change ACLs w henev er new pr efix es ar e allocat ed t o t he cust om er .

Ba ck gr ound uRPF is a CEF feat ur e t hat uses t he infor m at ion in t he FI B t o aut om at ically per for m t he BCP 38 check s. The or iginal st r ict m ode uRPF w as designed for t he cust om er / I SP edge of t he net w or k ( see Figure 4- 20 ) . The obj ect iv e w as t o design a feat ur e t hat can easily be aut om at ed in t he cust om er pr ov isioning sy st em , scale as new addr esses block s ar e allocat ed t o t he cust om er , and w or k w it h t he MTRI E- based CEF sw it ching. uRPF m eet s t hese obj ect iv es, ev en w hen t he cust om er s ar e m ult ihom ed t o one or m or e upst r eam I SPs. Or iginally im plem ent ed in I OS Soft w ar e 11.1( 17) CC ( av ailable fr om Mar ch 1998) , it has since been im plem ent ed t hr oughout t he I OS Soft w ar e code t r ain and w or k s on r out er s fr om t he lar gest t o t he sm allest . Deploy m ent of uRPF r anges fr om a sm all ent er pr ise net w or k using it t o enhance a sit e’s secur it y t o t he lar gest Tier 1 I SPs in business t oday . uRPF is t he pr ov en and scalable I SP secur it y t ool for BCP 38/ RFC 2827 deploy m ent , elim inat ing ex cuses t hat I SPs have used in t he past for not deploying ingr ess packet filt er ing on cust om er s. I t is accept ed by m any secur it y ex per t s t hat com plet ely im plem ent ing uRPF check s on t he edge of t he I nt er net ( w her e end sit es connect t o t he I SPs) w ould elim inat e a signific ant por t ion of t he m iscr eant behav ior on t he I nt er net t oday . Fig u r e 4- 2 0 . Or ig in a l u RPF D e p loy m e n t W a s on t h e Cu st om e r / I SP Ed g e

H ow u RPF W or k s: St r ict M od e u RPF When uRPF is enabled on an int er face, t he r out er v er ifies t hat all pack et s r eceiv ed on t hat int er face hav e a sour ce addr ess t hat is r eachable t hr ough t hat int er face. This

200

“ look back w ar d” capabilit y is av ailable only w hen CEF is enabled because t he lookup r elies on t he pr esence of t he FI B. uRPF ensur es t hat t her e is a r ev er se- pat h r out e t o t he input int er face of t he pack et . I f t her e is a r ev er se- pat h r out e, t he pack et is for w ar ded as nor m al. I f t her e is not a r ever se- pat h r out e, t he packet is dr opped. When a pack et is r eceiv ed by a r out er ’s int er face w it h uRPF, t he follow ing occur s: 1. uRPF does an MTRI E lookup on t he sour ce addr ess in t he FI B. I t pulls up t he next - hop and adj acency infor m at ion fr om t he sour ce addr ess’s MTRI E. 2. The infor m at ion fr om t he MTRI E lookup is used t o validat e t he packet ’s ret urn pat h t hr ough t he inbound int er face. This is r efer r ed t o as a FI B + adj acency check or a st r ict m ode uRPF check. 3. I f t he FI B + adj acency check on t he sour ce m at ches, t he pack et cont inues forwarding t hro ugh t he r out er ( see Figur e 4- 21) . Fig u r e 4- 2 1 . u RPF V a lid a t in g I P Sou r ce Ad d r e sse s

4. I f t he FI B + adj acency check does not m at ch, t he pack et ent er s t he dr op sequence ( see Figure 4- 22) . These ar e pack et s w hose sour ce addr esses do not m at ch t he inform at ion in t he FI B. Fig u r e 4- 2 2 . u RPF D r op p in g Pa ck e t s t h a t Fa il V e r if ica t ion

201

5. I f an uRPF ACL is applied, t he pack et is pr ocessed t hrough t hat feat ure ACL befor e final dr opping. This ACL could be configur ed t o ov er r ule t he uRPF check and pass t he pack et . [ 11] 6. CEF t able ( FI B) lookup is car r ied out for pack et for w ar ding, passing pack et s t hat m at ch t he FI B + adj acency check or dr opping pack et s t hat ar e spoofed sour ces. For ex am ple, if a cust om er sends a pack et w it h t he sour ce addr ess of 210.210.1.1 fr om int er face FDDI 2/ 0/ 0, RPF check s t he FI B t o see if 210.210.1.1 has a pat h t o FDDI 2/ 0/ 0. On t he ot her hand, if a pack et w it h a sour ce addr ess 144.64.21.1 ar r ives on FDDI 2/ 0/ 0, it w ould not m at ch t he infor m at ion in t he FI B glued t o t he FDDI 2/ 0/ 0 adj acency . The FI B has pr efix 210.210.0.0/ 16 as t he only pr efix glued t o t he FDDI 2/ 0/ 0 adj acency . So sour ce addr ess 144.64.21.1 is a spoofed addr ess. uRPF w ill fail t his pack et and dr op it . One of uRPF’s gr eat adv ant ages is t hat it dy nam ically adapt s t o changes in t he FI B caused by changes in t he RI B r esult ing fr om updat es fr om t he v ar ious r out ing pr ot ocol dat abases. uRPF has far low er per for m ance im pact as an ant ispoofing t ool com par ed w it h t he access list appr oach, has m inim al CPU ov er head, and oper at es at a few per cent less t han t he t y pical CEF sw it ching r at es. The use of t he MTRI E look up in CEF allow s it t o be coded in new , super fast - for w ar ding ASI Cs needed t o achiev e t he MPPS r at es for OC- 12 and higher cir cuit s. uRPF also r equir es less oper at ional m aint enance t han t r adit ional appr oaches t hat use I P access or ex t ended access list s. I t can be added t o t he cust om er ’s default int er face configur at ion on t he I SP’s r out er ( r em em ber t hat t his w ill w or k only if t he r out er has CEF configur ed) :

! Configuration template for customer interfaces description [enter description of interface] no ip redirects

202

no ip direct broadcast no ip proxy-arp ip verify unicast reverse-path bandwidth [bandwidth in kbps] Th e ip v e r ify u n ica st r e v e r se - pa t h int er face com m and can be added t o t he sam e deploy m ent scr ipt s as all t he ot her int er face it em s added w hen a new cust om er is inst alled. Because uRPF dy nam ically w or k s w it h t he FI B t able, t he inst allat ion engineer has no r eason t o configur e any ot her access list s. All t hat needs t o be done is t o add t he cust om er ’s r out e. uRPF is com pat ible w it h ot her CEF feat ur es such as per- pack et / per- dest inat ion load shar ing, CAR/ QPPB, WCCPv 2, and BGP policy account ing. uRPF w as fir st suppor t ed in I OS Soft w ar e Release 11.1( 17) CC CEF im ages on t he RSP 7000, 7200, and 7500 plat for m s. I t is not suppor t ed in any I OS Soft w ar e Release 11.2 or 11.3 im ages, but it has since been deployed on all r eleases of I OS Soft w ar e fr om 12.0 t hat suppor t CEF. This includes ev er y t hing fr om t he lar gest r out er s t o net w or k access ser v er s ( cable, x DSL, and dialup) and t he sm allest CPE dev ices. RPF Con f ig u r a t ion D e t a ils ( a s of I OS Sof t w a r e V e r sion 1 2 .0 ( 1 0 ) S1 ) To use uRPF, enable CEF sw it ching or CEF dist r ibut ed sw it ching in t he r out er . Ther e is no need t o configur e t he input int er face for CEF sw it ching. This is because uRPF has been im plem ent ed as a sear ch t hr ough t he FI B using t he sour ce I P addr ess. As long as CEF is r unning on t he r out er , indiv idual int er faces can be configur ed w it h ot her swit ching m odes. RPF is an input - side funct ion t hat is enabled on an int er face or subint erface suppor t ing any t y pe of encapsulat ion and t hat oper at es on I P pack et s received by t he rout er. W ARN I N G I t is very im port ant for CEF t o be t urned on globally in t he rout er. RPF w ill not w ork w it hout CEF.

Configur e RPF on t he int er face using t he follow ing int er face com m and synt ax:

[no] ip verify unicast reverse-path [ACL] For exam ple, do t his on a leased- line aggr egat ion r out er :

ip cef ! or "ip cef distributed" for an RSP+VIP based box ! interface serial 5/0/0 ip verify unicast reverse-path

203

As anot her exam ple, t he AS 5800 suppor t s CEF in I OS Soft w ar e Release 12.0. The in t e r fa ce gr ou p- a syn c com m and m akes it even easier t o apply uRPF on all t he dialup por t s:

ip cef ! interface Group-Async1 ip verify unicast reverse-path Use t he com m and sh o w ce f in t e r f a ce in t er face t o verify t hat RPF is operat ional:

Use sh ow ip in t e r f a ce in t er f ace t o find specific dr ops on a int er face ( as of I OS Soft w ar e Release 12.0( 10) S1) :

Excalabur#sh ip inter fastEthernet 4/1/0 FastEthernet4/1/0 is up, line protocol is up . . Unicast RPF ACL 100 55 unicast RPF drops 0 unicast RPF suppressed drops A count er is m aint ained t o count t he num ber of discar ds because of RPF. The v alue of t he count er is display ed as par t of t he out put fr om t he com m and:

show ip traffic The RPF dr op count er is included in t he I P st at ist ics sect ion:

ISP-LAB-7505-3#sh ip traffic IP statistics: Rcvd: 1471590 total, 887368 local destination 0 format errors, 0 checksum errors, 301274 bad hop count

204

0 unknown protocol, 0 not a gateway 0 security failures, 0 bad options, 0 with options Opts: 0 end, 0 nop, 0 basic security, 0 loose source route 0 timestamp, 0 extended security, 0 record route 0 stream ID, 0 strict source route, 0 alert, 0 other Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble 0 fragmented, 0 couldn't fragment Bcast: 205233 received, 0 sent Mcast: 463292 received, 462118 sent Sent: 990158 generated, 282938 forwarded Drop: 3 encapsulation failed, 0 unresolved, 0 no adjacency 0 no route, 0 unicast RPF, 0 forced drop ^^^^^^^^^^^^^^ ACL Op t ion ( a d d e d in I OS Sof t w a r e Re le a se 1 2 .0 ( 1 0 ) S1 ) [ 1 2 ] The opt ional ACL par am et er t o t he com m and can be used t o cont r ol t he ex act behavior w hen t he r eceived fr am e fails t he sour ce I P addr ess check. The ACL can be eit her a st andar d or an ex t ended I P access list : [ 13]



IP IP IP IP

standard extended standard extended

access access access access

list list list (expanded range) list (expanded range)

I f an ACL is specified, w hen ( and only w hen) a pack et fails a uRPF check t he ACL is check ed t o see if t he pack et should be dr opped ( using a deny ACL) or for w ar ded ( using a per m it ACL) . I n bot h cases, t he pack et is count ed as befor e. ACL logging ( log and log- in p u t ) and m at ch count s oper at e as nor m al—for ex am ple:

ip cef distributed ! interface ethernet 0/1/1 ip address 192.168.200.1 255.255.255.0 ip verify unicast reverse-path 197 ! access-list 197 permit ip 192.168.200.0 0.0.0.255 any log-input Fr am es sour ced fr om 192.168.200.10 ar r iv ing at Et her net 0/ 1/ 1 ar e for w ar ded ( because of perm it ) , logged by t he ACL, and count ed per int er face. Fr am es sour ced fr om 192.168.201.100 ar r iv ing at Et her net 0/ 1/ 1 ar e dr opped ( because of deny ) , logged by t he ACL, and count ed per int er face and globally. Count ing is seen per int er face:

Router> show ip interface ethernet 0/1/1 | include RPF Unicast RPF ACL 197 1 unicast RPF drop

205

1 unicast RPF suppressed drop Count ing also is seen globally:

Router> show ip traffic | include RPF 0 no route, 1 unicast RPF, 0 forced drop I n addit ion, count ing is seen per ACL:

Router> show access-lists Extended IP access list 197 permit ip 192.168.200.0 0.0.0.255 any log-input (100 match) uRPF’s ACL feat ur e has t w o pr im ar y funct ions. The fir st and obv ious funct ion is t o allow for ex cept ions. Som e net w or k s m ight need t o get t hr ough t he uRPF check , so t he ACL allow s a by pass t echnique. The second funct ion is t o ident ify spoof pack et s. uRPF w ill not send any not ificat ions of w hich packet s it is dr opping. Count er s w ill incr em ent , so t he oper at or w ill not ice ex cessiv e uRPF dr ops. I f t he oper at or w ant s t o quest ion w hat is being dr opped, an ACL can be int r oduced t o det er m ine w het her t he dr ops ar e v alid ( spoofed sour ce addr esses) or in er r or ( v alid pack et s being dr opped) . I n t he follow ing ex am ple, uRPF applies each pack et t hat fails t he r ev er se pat ch for w ar ding check t o ACL 171. The ACL st ill dr ops t he packet , but it also logs t he packet in t he ACL count er s and t he log file on t he pr ocessor or VI P/ line car d.

interface ethernet 0/1/1 ip address 192.168.200.1 255.255.255.0 ip verify unicast reverse-path 171 ! access-list 171 deny icmp any any echo log-input access-list 171 deny icmp any any echo-reply log-input access-list 171 deny udp any any eq echo log-input access-list 171 deny udp any eq echo any log-input access-list 171 deny tcp any any established log-input access-list 171 deny tcp any any log-input access-list 171 deny ip any any log-input Excalabur#sh controllers vip 4 logging show logging from Slot 4: . . 4d00h: %SEC-6-IPACCESSLOGNP: list 171 denied 0 20.1.1.1 -> 255.255.255.255, 1 packet . u RPF’s D e bu g Opt ion s Dr opped uRPF pack et s can be capt ur ed w it h a d e b u g ip ce f d r op s r p f a cl st at em ent . This debug opt ion w as added only int o I OS Soft w ar e Release 12.0S and w as not par t of t he or iginal 11.1CC im plem ent at ion. The w ay t hat t his uRPF debug t ool is used depends on w het her t he rout er is a VI P or non- VI P plat for m . On Cisco 7500s w it h VI P car ds, debug r esult s do not leav e t he VI P. To see t he r esult s of a

206

debug on a VI P, use an ACL on d e b u g ip ce f d r op s r p f w it h t he log- in pu t func t ion applied t o t he ACL. You t hen can use sh ow con t r olle r s v ip num ber loggin g t o see t he r esult s of t he debug. Ex am ple 4- 3 pr ov ides an ex am ple of sh ow con t r olle r s v ip 1 log g in g, t oget her w it h t he d e b u g ip ce f d r op s r p f 8 8 com m and. Ex a m p le 4- 3 U sin g sh ow con t r olle r v ip 1 log g in g on a Cisco 7 5 0 0 V I P Ca r d Thundershild#config Configuring from terminal, memory, or network [terminal]? Enter configuration commands, one per line. End with CNTL/Z. Thundershild(config)# Thundershild(config)#access-list 88 permit 1.19.1.4 0.0.0.0 log Thundershild(config)#exit Thundershild#debug ip cef drops rpf 88 Thundershild#sh controller vip 1 logging Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns) Console logging: level debugging, 59 messages logged Monitor logging: level debugging, 0 messages logged Buffer logging: level debugging, 65 messages logged Log Buffer (8192 bytes): smallest_local_pool_entries = 192, global highest_local_visible_bandwidth = 100000 . . . 2d16h: CEF-Drop: Packet from 1.19.1.4 via rpf check 2d16h: CEF-Drop: Packet from 1.19.1.4 via rpf check 2d16h: CEF-Drop: Packet from 1.19.1.4 via rpf check 2d16h: CEF-Drop: Packet from 1.19.1.4 via rpf check 2d16h: CEF-Drop: Packet from 1.19.1.4 via rpf check . . Thundershild#no debug ip cef drops rpf 88

particles = 618

FastEthernet1/0/0 -- unicast FastEthernet1/0/0 -- unicast FastEthernet1/0/0 -- unicast FastEthernet1/0/0 -- unicast FastEthernet1/0/0 -- unicast

W ARN I N G Car e m ust be t aken w it h any am ount of debug infor m at ion logging funct ions of a r out er . sev er al t ens of m egaby t es of

use of t he debug feat ur e on a pr oduct ion r out er . The easily ov er w helm s t he capabilit y of t he console and This is especially t rue w hen t he r out er is handling DoS t r affic.

Rout ing Ta ble s Re quir e m e nt s uRPF needs accur at e infor m at ion in t he FI B t o w or k pr oper ly. The fundam ent al r equir em ent for uRPF t o w or k com m only is st at ed by Cisco st aff in t his w ay : “ [ A]

207

v alid and pr efer r ed pat h m ust ex ist in t he for w ar ding t able t hat m at ches t he sour ce addr ess t o t he input int er face.” This does not m ean t hat t he r out er m ust hav e t he ent ir e I nt er net Rout e Table. The am ount of rout ing inform at ion needed in t he CEF t able s depends on w here uRPF is configur ed and w hat funct ions t he r out er play s in t he I SP’s net w or k . For ex am ple, a r out er t hat is a leased- line aggr egat ion r out er for cust om er s needs only t he infor m at ion based on t he st at ic r out es int r oduced int o t he I GP or iBGP ( depending on w hich t echnique is used in t he net w or k ) . uRPF w ould be configur ed on t he cust om er s’ int er faces—hence, t he requirem ent for m inim al rout ing inform at ion. I n anot her scenario, a single - hom ed I SP can place uRPF on t he gat ew ay link t o t he I nt ernet . The full I nt er net Rout e Table w ould be r equir ed in t his case. This w ould help pr ot ect t he I SP fr om ex t er nal DoS at t ack s t hat use addr esses not in t he I nt er net Rout e Table. u RPF Ex ce pt ion s Som e sour ce I P addr esses should be allow ed t hr ough t he uRPF filt er ing ( see Exam ple 4- 4) . uRPF w ill now allow pack et s w it h 0.0.0.0 sour ce and 255.255.255.255 dest inat ion t o pass so t hat BOOTP and DHCP st ill can funct ion. This feat ur e ( CSCdk80591) w as added fr om I OS Soft w ar e Release 12.0( 3.05) ( but is not in 11.1CC) . Also, if t he dest inat ion addr ess is m ult icast , uRPF ex em pt s t hose pack et s. Ex a m p le 4- 4 u RPF Alg or it h m a s of I OS Sof t w a r e 1 2 .0 ( 9 ) S lookup source address in forwarding database if the source address is reachable via the source interface pass the packet else if the source is 0.0.0.0 and destination is a 255.255.255.255 /* BOOTP and DHCP */ pass the packet else if destination is multicast pass the packet else drop the packet

BCP 3 8 I m plem ent at ion w it h uRPF St rict M ode uRPF’s key BCP 38 im plem ent at ion pr inciples follow : • • •

A r out e m ust exist in t he FI B m at ching t he pr efix t o t he int er face. This can be done t hr ough a connect ed int er face, a st at ic r out e, a n e t w or k st at em ent ( BGP, OSPF, RI Pv 2, and so on) , or dy nam ic r out ing updat es. Tr affic fr om t he int er face m ust m at ch t he pr efix es for t he int er face. I f t here are m ult iple ent ries for t he prefix in t he rout e t ables, t he prefix local t o t he r out er im plem ent ing uRPF must be pr efer r ed ( using BGP w eight w it h m ult ihom ed cust om er s) .

Giv en t hese t hr ee im plem ent at ion pr inciples, uRPF becom es a t ool t hat I SPs can use not only for t heir cust om er s, but also for t heir dow nst r eam I SPs—ev en if t he dow nst r eam I SP has ot her connect ions t o t he I nt er net .

208

u RPF St r ict M ode w it h a Sin gle - H om e d Le a se d - Lin e Cu st om e r s Single - hom ed cust om er s ar e by far t he v ast m aj or it y of leased- line cust om ers. For t hese cust om er s, uRPF can be par t of t he default int er face configur at ion applied when t he circ uit first is inst alled. Leased- line cust om er aggr egat ion r out er s ar e ideal wit h single - hom ed cust om er s. [ 14] I n t his t opology , t he cust om er aggr egat ion rout ers need not have t he full I nt er net Rout e Table; t hey sim ply need t he infor m at ion on t he r out ing pr efix es assigned t o t he cust om er . [ 15] Hence, infor m at ion int r oduced int o t he I GP or iBGP ( depending on t he w ay y ou add cust om er r out es int o your net w or k) w ould be enough for uRPF t o do it s j ob. Using Figur e 4- 23, a t ypical configur at ion on t he I S P’s r out er w ould be as follow s ( assum ing t hat CEF is t ur ned on) : Fig u r e 4- 2 3 . Single - H om e d Cu st om e r u RPF f or I n g r e ss Filt e r in g

interface loopback 0 description Loopback interface on Gateway Router 2 ip address 215.17.3.1 255.255.255.255 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial 5/0 description 128K HDLC link to Galaxy Publications Ltd [galpub1] WT50314E R5-0 bandwidth 128 ip unnumbered loopback 0 ip verify unicast reverse-path ! Unicast RPF activated here no ip redirects no ip directed-broadcast no ip proxy-arp ! ip route 215.34.10.0 255.255.252.0 Serial 5/0 u RPF St r ict M od e w it h M u lt ih om e d Le a se d - Lin e Cu st om e r s ( On e I SP) uRPF st r ict m ode act ually w or k s w it h m ult ihom ed leased- line cust om ers! I t w orks w it h t he asy m m et r ic t r affic flow s bet w een t he cust om er and t he I SP! I t is a com m on m y t h per pet uat ed by m any people t hat uRPF does not w or k w hen you have m ult ihom ing or w hen y ou hav e asy m m et r ic t r affic flow s. Engineer s w ho j um p t o t hat conclusion t end t o not t hink t hr ough t he pr oblem or do not have t im e t o t hink about t he pr oblem . Ther e is also som e lack of under st anding about how t he RI B and t he FI B int er act w it h t oday’s best - pat h forw arding rout ing. uRPF w or k s w it h best pat h for w ar ding. For uRPF t o w or k cor r ect ly , t he one pat h in t he FI B m ust be t he pat h t hat point s back out of t he int er face closest t o t he m ult ihom ed

209

cust om er . So, for uRPF t o w or k for m ult ihom ed cust om er s w her e t r affic t r ends t o be asy m m et r ic, t he FI B m ust align w it h how t he cust om er is connect ed t o t he r out er . And t hat is easier t o achiev e t han is com m only per ceiv ed in t he indust r y t oday . D e t a ils Be h in d u RPF, M u lt ih om e d Cu st om e r s, a n d Asy m m e t r ica l Rou t in g Under st anding w hat is happening w it h r out ing on t he I nt er net is essent ial t o t he configur at ion of uRPF st r ict m ode on m ult ihom ed leased- line cust om er s. For st ar t er s, r ealize t hat asy m m et r ical r out ing is v er y com m on for a m ult ihom ed leased- line cust om er ( see Figur e 4- 24 ) . When t r affic t r av els ov er t he I nt er net asy m m et r ically , it usually m eans t hat pack et s w ill t ak e one pat h t o get t o t he dest inat ion and anot her pat h t o r et ur n t o t he sour ce fr om t he dest inat ion. TCP/ I P, of cour se, w or k s per fect ly w ell w it h asy m m et r ical r out ing. I f m ult iple pat hs ar e used t o get t o t he dest inat ion and cause t he pack et s t o ar r iv e out of or der , TCP r eassem bles t he pack et s in t heir pr oper or der . Asym m et r ical t r affic flow s ar e nor m al and happen all t he t im e in t he I nt er net t oday ; in t he v ast m aj or it y of cases, asy m m et r ic flow s do not hav e any per ceiv able im pact on t he effect iv eness of t he client / ser v er comm unicat ions. Because it is so com m on, asym m et r ical r out ing has been used as t he r eason w hy uRPF w ill not w or k on m ult ihom ed cust om er connect ions. Fig u r e 4- 2 4 . Ty p ica l Asy m m e t r ica l Rou t in g Ex a m p le

Rout ing on t oday’s I nt er net is based on t he concept of best - pat h forwarding. Best pat h forw arding st art s in t he rout er’s RI B. The RI B receives rout ing updat es from t he r out ing pr ot ocols r unning in t he r out er . These, in t ur n, r eceive t heir updat es fr om ot her rout ers in t he net w ork ( t heir neighbors) . Wit h m ult iple rout ing updat es from m ult iple r out er s, t he RI B m ight have a sit uat ion in w hich t her e ar e m ult iple updat es on t he sam e pr efix . When t his happens, t he RI B has a m eans of figuring out w hich of t he m ult iple ent r ies for t his pr efix is t he best pat h. This best pat h is subm it t ed t o t he FI B for consider at ion ( see Figur e 4- 25) . The r esult is t hat t he r out er has select ed t he best pat h for for w ar ding a pack et fr om t he inbound int er face. Fig u r e 4- 2 5 . Be st - Pa t h For w a r d in g I s W h y W e H a v e Asy m m e t r ica l Rou t in g a n d I m p a ct s on u RPF D e p loy m e n t

210

Every rout er on t he I nt ernet m akes t his best - pat h decision. Because t hese decisions ar e done fr om a point of view of t he r out er ’s posit ion in t he I nt er net , t he r esult ing best pat h m ight not m at ch w hat t he best pat h is on ot her r out er s. Rout ing pr ot ocols ar e cr eat ed w it h t he m eans t o ensur e t hat t hese independent best - pat h decisions do not cause r out ing loops. Alt hough r out ing loops ar e not a pr oblem , differ ent best pat h decisions m ight be a pr oblem for a net w or k . As a r esult , som e r out ing pr ot ocols have a m eans of influencing t he best - pat h decision acr oss a net w or k bounded by an aut onom ous net w or k . For ex am ple, BGP’s Local Pr efer ence at t r ibut e allow s v alues t o be placed on pr efix updat es. These values can be used t o ensure t hat every BGP RI B inside t he AS select s t he sam e best pat h for t hat pr efix . So t he best - pat h result s are effect ive net w or k- w ide inst ead of on j ust one rout er. N OTE Rout ing loops occur w hen t w o or m or e r out er s select t he best pat h back t o t he ot her. This causes pack et s t o ping pong back and for t h unt il t he pack et ’s TTL t im es out and dr ops t he pack et .

BGP at t r ibut es such as Local Pr efer ence allow t he I SP t o cont r ol t he flow of t r affic bet w een it s net w or k and ot her pr ov ider s’ net w or k s. I t does not allow t hem t o effect iv ely influence t he best - pat h decisions of ot her net w or ks. So, in one com m on for m of asym m et r ical r out ing, each I SP m akes it s ow n best - pat h decision. At t his t im e, t he t ools for one I SP t o affect t he best - pat h decisions of anot her I SP ar e coar se and ar e not guar ant eed t o w or k. [ 16] Because each r out er , and per haps each I SP, is m aking it s ow n best - pat h decision and lit t le can be done t o effect ively influence t his on an I nt ernet - w ide scale, asy m m et r ic t r affic flow is t he r esult . This affect s w her e uRPF can be applied in t he net work.

211

When a r out er r eceiv es a pack et fr om anot her r out er , it does not car e w hich int er face it cam e in on. The r out er ex am ines t he dest inat ion addr ess, look s up t he r out e in t he FI B, and for w ar ds t he pack et out anot her int er face. Wit h uRPF t ur ned on, t he r out er does need t o car e w hich int er face t he pack et w as r eceiv ed fr om . To m ak e t his w or k , t he RI B needs t o pr ov ide infor m at ion t o t he FI B t o allow uRPF t o w or k. Wor king t hr ough an exam ple is t he best w ay t o dem onst r at e how t his applies. Figur e 4- 26 shows a m ult ihom ed cust om e r using a com binat ion of split adv er t isem ent and adv anced com m unit ies t o t r affic - engineer t he net w ork. This is an effect iv e m ult ihom ing t echnique t hat pr ov ides good t r affic engineer ing cont r ol t o t he cust om er w hile k eeping t he I SP’s oper at ional ov er head for suppor t ing t he m ult ihom ed cust om er t o a m inim um . The RI B on Rout er A r eceiv es t he advert isem ent from Rout er C for t he t w o prefixes. The rout e m aps used for RFC 1998- st y le com m unit y usage set t he local pr efer ence for t hese pr efix es in t he I SP’s net work. [ 17] I n t his w ay , t he local pr efer ence v alues ar e use t o det er m ine w hich of t he m ult iple copies of t he r out e—say, 169.21.0.0/ 17 —is sent t o t he FI B. Fig u r e 4- 2 6 . BGP Loca l Pr e f e r e n ce Af f e ct s Tr a f f ic Flow w it h M u lt ih om e d Cu st om e r s

N OTE I n split adv er t isem ent , a m ult ihom ed cust om er split s it s I Pv 4 addr ess block int o t w o par t s. Each par t is adv er t ised out each of t he m ult ihom ed link s w it h differ ent at t r ibut es applied. These at t r ibut es could be BGP com m unit ies t hat affect local pr efs ( RFC 1998 and v ar iant s) , t he BGP com m unit y no- a dv e r t ise , AS pat h pr epend, or ot her BGP- based t raffic - engineer ing t echniques. Split adv er t isem ent ensur es t hat t he ent ir e addr ess block is r eachable in case one of t he links goes dow n.

I n t his ex am ple, t he cust om er is using per- pr efix load balancing t o send t r affic t o t he I nt er net . So pack et s arriving on Rout er A could be from t he 169.21.0.0/ 17 block or t he 169.21.128.0/ 17 block . I f a pack et bound for 169.21.128.0/ 17 ar r iv es on Rout er A, Rout er A w ould for w ar d t he pack et based on t he pr efix ’s local pr efer ence v alue,

212

result ing in t he packet being sent t o Rout er B and t hen t o Rout er C. This adds an ex t r a hop but has no per ceiv able im pact on t he per for m ance of t he pack et flow . How ev er , t his w ill im pact uRPF’s capabilit y t o oper at e as ex pect ed. Because t he I SP has no cont r ol ov er t he cust om er ’s use of per- pack et or per- flow load balancing, t her e can be no effect iv e or scalable w ay t o for ce all 169.21.0.0/ 17 pack et s out one link or 169.21.128.0/ 17 pack et s out t he ot her link . [ 18] So t he packet s ar r iving on Rout er A fr om Rout er C could be fr om any addr ess in t he 169.21.0.0/ 16 addr ess block . Yet uRPF on Rout er A w ould see pack et s in t he 169.21.0.0/ 17 com ing fr om Rout er C as being v alid. Pack et s fr om 169.21.128.0/ 17 w ou ld get dr opped. A m odificat ion t o t he configur at ion on Rout er A is needed t o ensur e t hat pack et s fr om any addr ess in t he / 16 w ould be passed by uRPF. Figur e 4- 27 illust r at es t he solut ion. The BGP opt ion w e igh t ov er r ides t he local pr efer ence v alue on t he r out er . I t affect s only t hat one r out er and t he r out e r eceiv ed fr om t he BGP neighbor . Local pr efer ence st ill w or ks as nor m al in t he net w or k. Wit h t he BGP w e igh t com m and applied t o t he BGP session bet w een Rout er C and Rout er A, bot h 169.21.0.0/ 17 and 169.21.128.0/ 17 w ill be subm it t ed t o t he FI B as t heir best pat hs ar e dir ect ly t o Rout er C ( ver sus t hr ough Rout er B for t he 169.21.128.0/ 17 w it h local pr efer ence applied) . This w ill allow any pack et fr om 169.21.0.0/ 16 t o pass t hr ough uRPF st r ict m ode as it w as t ur ned on Rout er A’s int er face t hat connect ed t o Rout er C. Fig u r e 4- 2 7 . BGP W e ig h t Ap p lie d t o t h e Cu st om e r Pe e r in g Allow s u RPF t o W or k

The oper at ional im pact of using BGP w eight on t he configur at ion is m inim al. The “ ext r a” hop is elim inat ed if a packet t hat should go t o Rout er B ar r ives on Rout er A. So t he cust om er m ight lik e t his added efficiency . The addit ion t o t he configur at ion is m inim al because it is one line added t o t he BGP configur at ion—easily scr ipt ed. The only got cha is educat ing t he cust om er and ensur ing t hat he k now s how m ult ihom ing and t his t echnique w or k. Of cour se t hat is t he pur pose of t his book—helping I SPs and cust om er s t o I SPs do t he r ight t hing and under st and w hy t hey ar e doing it . Chapt er

213

5, “ Oper at ional Pr act ices, ” cov er s m ult ihom ing ex am ples and r ecom m ended pr act ices in det ail. I n sum m ar y , BGP w eight ensur es t hat t he r out e local t o t he r out er is t he pr efer r ed pat h. BGP w eight is local t o t he rout er and easily is configured on t he BGP neighbor configur at ion. The BGP w eight pr efer s t he local connect ion even t hough BGP local pr efer ence is point ing t he best pat h t o t he ot her m ult ihom ed r out er . Set t ing t he BGP w eight has m inim al, if any , effect on t he t r affic flow s bet w een t he I SP and t he m ult ihom ed cust om er . Yet it is enough t o align t he FI B so t hat uRPF can accom plish BCP 38 filt er ing on m ult ihom ed cust om er s w it h asy m m et r ical r out ing. W or k in g Ex a m p le of u RPF, M u lt ih om e d Cu st om e r s, a n d Asy m m e t r ica l Rou t in g I n t his ex am ple, depict ed in Figur e 4- 28, t he ent er pr ise cust om er of t he I SP is m ult ihom ed int o t w o differ ent r out er s. BGP is used w it h a pr iv at e ASN assigned by t he I SP. The ent erprise’s I P address block w ould be allocat ed from t he I SP or from an I P regist ry. As t he rout e is advert ised int o t he I SP’s rout ers, an int ernal BGP w eight is applied. This ensur es t hat if t her e is a t ie bet w een t w o ident ical pr efix es, t he one direct ly from Rout er C w ill be preferred on t he local rout er and ent er ed int o t he FI B. Fig u r e 4- 2 8 . M u lt ih om e d Le a se d- Lin e Cu st om e r a n d u RPF

I n Ex am ple 4- 5, t he cust om er div ides it s allocat ed / 16 addr ess block int o t w o / 17s, allow ing for som e lev el of t r affic engineer ing. The cust om er adv er t ises t he / 16 and t w o / 17s out bot h links ( r equir ed t o have uRPF w or k on t he I SP’s ingr ess) . Pr efer ence is com m unicat ed t o t he I SP t hr ough BGP com m unit ies. The I SP uses t he com m unit ies adv er t ised fr om t he cust om er t o set t he BGP local pr efer ence of t he / 17, m aking one pr efer r ed over t he ot her . [ 19] Bot h / 17s hav e t he BGP com m unit y no- ex port t o ensur e t hat t hey st ay inside t heir upst r eam I SPs.

214

Som e basic r est r ict ions gov er n apply ing uRPF t o t hese m ult ihom ed cust om er s: •

•

•

Cust om er s should not be m ult ihom ed t o t he sam e rout er. This is com m on sense—it br eak s t he pur pose of building a r edundant ser v ice for t he cust om er . I f t he sam e r out er is used, t he cir cuit s should be configur ed for par allel pat hs ( using som et hing along t he lines of CEF load balancing) . uRPF w or k s w it h par allel pat h cir cuit s bet w een t w o r out er s. Cust om er s need t o ensur e t hat t he pack et s flow ing up t he link ( out t o t he I nt er net ) m at ch t he pr efixes adver t ised out t he link. Ot her w ise, uRPF w ill filt er t hose pack et s. Adv er t ising t he sam e r out es out bot h link s is t he best w ay t o m ak e sur e t hat t his w ill happen. The t raffic - engineer ing t r ick of split t ing t he I P addr ess space in half, w it h each half adver t ising one link, cannot be used. For exam ple, t he ent er pr ise cust om er cannot adv er t ise 169.23.0.0/ 17 and 169. 23. 0. 0/ 16 out one upst ream link and 169.23.128.0/ 17 and 169.23.0.0/ 16 out t he ot her link. This w ould br eak uRPF—unless an ACL is applied t o uRPF t o allow for t his case. The r ecom m ended t echnique is highlight ed in Ex am ple 4- 5 using BGP com m unit ies t o set local pr efer ence. [ 20]

Ex a m p le 4- 5 M u lt ih om e d u RPF Router A interface serial 1/0/1 description Link to Acme Computer's Router C ip address 192.168.3.2 255.255.255.252 ip verify unicast reverse-path no ip redirects no ip directed-broadcast no ip proxy-arp ip route-cache distributed router bgp 109 . neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 .

remote-as 65000 description Multihomed Customer – Acme Computers update-source Loopback0 send-community soft-reconfiguration inbound route-map set-customer-local-pref in weight 255

ip route 192.168.10.3 255.255.255.255 serial 1/0/1 ip bgp-community new-format

Router B interface serial 6/1/1 description Link to Acme Computer's Router C ip address 192.168.3.6 255.255.255.252 ip verify unicast reverse-path no ip redirects no ip directed-broadcast

215

no ip proxy-arp ip route-cache distributed router bgp 109 . neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 neighbor 192.168.10.3 .

remote-as 65000 description Multihomed Customer – Acme Computers update-source Loopback0 send-community soft-reconfiguration inbound route-map set-customer-local-pref in weight 255

ip route 192.168.10.3 255.255.255.255 serial 6/1/1 ip bgp-community new-format Router C ! interface serial 1/0 description Link to Upstream Router A ip address 192.168.3.1 255.255.255.252 ip verify unicast reverse-path no ip redirects no ip directed-broadcast no ip proxy-arp ip load-sharing per-destination ip route-cache distributed

interface serial 1/1 description Link to Upstream ISP Router B ip address 192.168.3.5 255.255.255.252 ip verify unicast reverse-path no ip redirects no ip directed-broadcast no ip proxy-arp ip load-sharing per-destination ip route-cache distributed

router bgp 65000 no synchronization network 169.21.0.0 network 169.21.0.0 mask 255.255.128.0 network 169.21.128.0 mask 255.255.128.0 neighbor 171.70.18.100 remote-as 109 neighbor 171.70.18.100 description Upstream Connection #1 neighbor 171.70.18.100 update-source Loopback0 neighbor 171.70.10.100 send-community neighbor 171.70.18.100 soft-reconfiguration inbound neighbor 171.70.18.100 route-map Router-A-Community out neighbor 171.70.18.200 remote-as 109 neighbor 171.70.18.200 description Upstream Connection #2 neighbor 171.70.18.200 update-source Loopback0 neighbor 171.70.18.200 send-community neighbor 171.70.18.200 soft-reconfiguration inbound

216

neighbor 171.70.18.200 route-map Router-B-Community out maximum-paths 2 no auto-summary ! ip route 169.21.0.0 0.0.255.255 Null 0 ip route 169.21.0.0 0.0.127.255 Null 0 ip route 169.21.128.0 0.0.127.255 Null 0 ip route 171.70.18.100 255.255.255.255 serial 1/0 ip route 171.70.18.200 255.255.255.255 serial 1/1 ip bgp-community new-format ! access-list 50 permit 169.21.0.0 0.0.127.255 access-list 51 permit 169.21.128.0 0.0.127.255 ! route-map Router-A-Community permit 10 match ip address 51 set community 109:70 no-export ! route-map Router-A-Community permit 20 match ip address 50 set community 109:100 no-export ! route-map Router-A-Community permit 30 ! route-map Router-B-Community permit 10 match ip address 50 set community 109:70 no-export ! route-map Router-B-Community permit 20 match ip address 51 set community 109:100 no-export ! route-map Router-B-Community permit 30 M u lt ih om e d Le a se d - Lin e Cu st om e r s ( Tw o I SPs) uRPF also w or k s w it h a m ult ihom ed cust om er t hat has a connect ion t o t w o differ ent I SPs. Figur e 4- 29 show s how t he dow nst r eam cust om er ( ent er pr ise or I SP) connect s t o t w o upst r eam I SPs. These t w o I SPs, Alpha and Bet a, int er connect w it h each ot her at v ar ious places in t he w or ld ( com bining pr iv at e peer ing, I XP peer ing, and t r ansit ) . Ther efor e, each I SP w ill hav e t w o BGP ent r ies for t he dow nst r eam cust om er ’s pr efix . Yet eac h I SP w ould select t he shor t est - pat h ent r y fr om t he BGP t able as t he best pat h, enabling uRFP t o w ork. Fig u r e 4- 2 9 . En t e r p r ise Cu st om e r M u lt ih om e d t o Tw o I SPs

217

BGP w eight should be used in Rout ers A and B for all t he prefixes being advert ised fr om Rout er C. This is necessar y t o pr ovide a safeguar d against AS pat h pr epending. I t is nor m al pr act ice for m ult ihom ed cust om er s t o use t he AS pat h- prepending t echnique t o affect t he balance of t he incom ing t r affic flow s. I n som e cases t he pr epending of ASNs w ould br eak t he uRPF. For ex am ple, t he dow nst r eam cust om er pr epends enough ASNs t o it s adv er t isem ent s t o Rout er A t hat Rout er A’s best pat h t o Rout er C w ould be t hrough Rout er B. This m eans t hat t he Rout er A–C for w ar ding pat h act ually w ould select a Rout er A– B–C for w ar ding pat h. uRPF w ould not hav e a valid pat h for sour ce addr esses com ing up t he Rout er C–A link , effect iv ely block ing t he dow nst r eam cust om er ’s out bound t r affic on t he Rout er C–A link. A BGP w eight ( see Ex am ple 4- 5) applied on Rout er s A and B w ould over r ide t he local effect s of AS pat h pr epends. Not e t hat t his is a case in w hich t he cust om er ’s t r affic w ould be asy m m et r ic t hr ough t he t w o upst r eam connect ions. Rout er C w ould for w ar d t r affic based on t he shor t est pat h infor m at ion pr ovided by Rout ers A and B. Som e t raffic flow s w ould exit Rout er C–A link yet ret urn t hr ough t he Rout er C– B link ( and v ice v er sa) . I n bot h cases of asy m m et r ical flow s, uRPF w ill block unaut hor ized t r affic fr om t he ent er pr ise cust om er’s net w ork.

Com m it t e d Acce ss Ra t e t o Ra t e- Lim it or D r op Pa ck e t s [ 2 1 ] The use of a QoS t ool such as CAR as a secur it y t ool is a r esult of t he t y pes of r ecent at t ack s on t he I nt er net . I n 1997, a new gener at ion of at t ack s w as launched on t he I nt er net : t he sm ur f at t ack . This is a net work- level DoS at t ack nam ed aft er it s exploit pr ogr am . A sm ur f at t ack is built ar ound I CMP echo pack et s; t he ot her com m on at t ack is fr aggle, w hich uses UDP echo packet s in t he sam e m anner as t he I CMP echo pack et s. Fr aggle w as a sim ple r ew r it e of t he sm urf program .

The Sm urf At t a ck The pr inciple behind a sm ur f at t ack is sur pr isingly sim ple. A per pet r at or sends a lar ge am ount of I CMP echo ( ping) t r affic t o specific I P br oadcast addr esses. All t he I CMP echo pack et s hav e t he spoofed sour ce addr ess of a v ic t im . I f t he r out ing dev ice deliv er ing t r affic t o t hose br oadcast addr esses per for m s t he I P br oadcast –t o–Layer 2 t r anslat ion, t he I CMP br oadcast funct ion w ill be for w ar ded t o all host s on t he Lay er 2 m edium ( see Figur e 4- 30 ) . Fig u r e 4- 3 0 . H ow Sm u r f Use s Am plifie r s

218

Each host on t hat I P net w or k w ill t ak e t he I CMP echo r equest and r eply t o it w it h an echo r eply. This m ult iplies t he inbound t r affic by t he num ber of host s r esponding. On a m ult i- access br oadcast net w or k, pot ent ially hundr eds of m achines could be r eplying t o each packet , r esult ing in w hat is called an at t ac k from a “ sm urf am plifier net work.” The sy st em s m ost com m only hit by t hese t y pes of at t ack s ar e I nt er net Relay Chat ( I RC) ser v er s, specific Web sit es, and t heir pr ov ider s. Tw o par t ies ar e hur t by t his at t ack : • •

The int er m ediar y ( br oadcast ) dev ices, called t he am plifiers The spoofed addr ess t ar get , t he v ict im

The v ict im is t he t ar get of t he lar ge am ount of t r affic t hat t he am plifier s gener at e. Consider a scenar io t hat paint s a dr am at ic pict ur e of t he danger ous nat ur e of t his at t ack . Assum e a co- locat ion sw it ched net w or k w it h 100 host s and an at t acker w it h a T1 cir cuit available. The at t acker sends, say, a 768 Kbps st r eam of I CMP echo ( ping) pack et s, w it h a spoofed sour ce addr ess of t he v ict im , t o t he br oadcast addr ess of t he bounce or am plifier sit e. These ping pack et s hit t he bounce sit e’s br oadcast net w or k of 100 host s; each host r esponds t o t he pack et , cr eat ing 100 ping r eplies out bound. I f you m ult iply t he bandw idt h, you w ill see t hat 76.8 Mbps is gener at ed out bound fr om t he bounce sit e. Because of t he spoofed source address of t he originat ing pack et s, t hese r eply pack et s ar e dir ect ed t ow ar d t he v ict im ( see Figur e 4- 30. ) This scenar io dem onst r at es t w o t hings: •

The am plifier sit e loses a lar ge am ount of out bound bandw idt h t o t he I CMP packet s being used for t he sm ur f at t ack. I f t he sit e does not have 76.8 Mbps

219

•

of bandw idt h, it s links w ill go int o sat ur at ion and it w ill be t he vict im of a ver y effect ive DoS at t ack on it s servic es. I f t he am plifier sit e has sufficient out bound bandw idt h, t he vict im w ill be pounded by 76.8 Mbps of inbound t r affic. I f t he v ict im doesn’t hav e sufficient inbound bandw idt h, it s link w ill go int o sat ur at ion and w ill becom e unusable, and it s upst ream I SP’s aggr egat ion r out er t hat it connect s t o w ill spend a lar ge am ount of t im e t r y ing t o deal w it h t he ex cess t r affic t hat w on’t fit t hr ough t he cust om er ’s cir cuit .

This sit uat ion can be m ade w orse if t he perpet rat or has ident ified m ore t han one sm urf am plif ier sit e. This t hen becom es a dist r ibut ed DoS at t ack , w hich is har der t o t r ack and affect s m or e I SPs along t he pack et pat h. Consider t he second scenar io. The per pet r at or has a sim ple dialup connect ion av ailable and is going t o car r y out a DoS at t ack on a par t icular Web sit e. Analog dialups have a m axim um of 33.6 Kbps available for out bound bandw idt h, so assum e t hat t he at t ack er w ill use all of t his for t he at t ack . The at t ack er also has ident ified 20 sm ur f am plifier net w or k s ar ound t he I nt er net . I f each am plifier net w or k has 100 host s on it , t he at t ack er can m ult iply his out bound pack et flow 2000 t im es, or conver t a 33.6 Kbps ping flood int o 67.2 Mbps. Also, because 20 sm ur f am plifier sit es hav e been ident ified ar ound t he I nt er net , t his becom es m uch har der for t he Web sit e ow ner and t he host I SP t o act ually ident ify . I SPs ar e m ult ihom ed t o t he I nt er net , and it w ill look as t hough t her e ar e ping at t ack s com ing in t hr ough all upst r eam connect ions fr om all ov er t he I nt er net . The host I SP w ill see t he affect on all it s links ( depending on it s link capacit y) , but t he im pact w ill be m ost sever e w her e t he Web sit e connect s t o t he I SP’s back bone. This second scenar io dem onst r at es how danger ous t he sm ur f at t ack can be ev en fr om a “ sim ple” dialup user . And w it h m any count ries in t he w orld now act ively pr om ot ing DSL and cable ser v ices, t he bandw idt hs av ailable t o t he “ casual” user ar e becom ing ev er gr eat er , so t he chances of a ser ious DoS at t ack fr om a r elat iv ely inexper ienced or low - key sour ce is m or e likely. ( Com m on or igins for DoS at t acks in t he past hav e included univ er sit y com put er r oom s and ot her net w or k s w it h public access and less r igid secur it y t han t y pically found in t he cor por at e env ir onm ent . These t w o scenar ios dem onst r at e how cr it ically im por t ant it is for I SPs t o do t he follow ing: • • •

I m plem ent uRPF on all t heir cust om er connect ions Consider r at e- lim it ing I CMP inbound on t heir net w orks Make sur e t hat dir ect br oadcast s ar e t ur ned off on all net w or k links

Failur e t o do any of t hese sim ply cont r ibut es t o t he abuse of resources and facilit ies by m any people on t he I nt er net t oday .

Ra t e - Lim it ing w it h CAR I t is an inevit able consequence of being par t of t he I nt er net t hat ever y I SP at som e point w ill ex per ience a DoS at t ack . I t is im per at iv e t hat I SPs hav e t ools and pr ocedur es in place t o r espond t o t hese DoS at t ack s, pr efer ably befor e an at t ack occurs. CAR is one such t ool.

220

CAR is a funct ionalit y t hat w or ks w it h Cisco Expr ess For w ar ding, found in I OS Soft w ar e Releases 11.1CC and onw ar d fr om 12.0. I t allow s net w or k oper at or s t o r at e- lim it cer t ain t y pes of t r affic t o specific sour ces or dest inat ions. The m ain adv ant age of CAR is t hat it can w or k on pack et s as t hey ar r iv e on t he r out er ’s int er face, dr opping or r at e- lim it ing t he DoS flow befor e any ot her packet pr ocessing. Th e f ollow ing URLs pr ovide det ails on CAR: Com m it t ed Access Rat e ( CAR) : ht t p: / / w w w .cisco.com / w ar p/ public/ 732/ Tech/ car / index .ht m l Configur ing Com m it t ed Access Rat e: ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios122/ 122cgcr / fqos_c/ f qcpr t 1 / qcfcar . ht m The follow ing ar e t w o ex am ples t hat show how CAR can be used t o r at e- lim it par t icular t y pes of t r affic t hat should not see high v olum es in an I SP net w or k . The fir st ex am ple r at e- lim it s I CMP; t he second ex am ple r at e- lim it s TCP SYN. Ex a m p le 1 An I SP has filt er ed it s I RC ser ver fr om r eceiving I CMP echo- r eply pack et s t o pr ot ect it . Now m any at t ack er s ar e going aft er t he cust om er ’s dev ices t o fill som e net w or k segm ent s. The I SP chose t o use CAR t o lim it all I CMP echo and echo- r eply t r affic r eceiv ed at t he borders t o 256 Kbps. An exam ple follow s:

! traffic we want to limit access-list 102 permit icmp any any echo access-list 102 permit icmp any any echo-reply ! interface configurations for borders interface Serial3/0/0 rate-limit input access-group 102 256000 8000 8000 conform-action transmit exceedaction drop This lim it s I CMP echo and echo- reply t raffic t o 256 Kbps, w it h a m axim um burst of 8000 by t es; all pack et s ex ceeding t his lim it ar e dr opped. Mult iple r at e- lim it com m ands can be added t o an int er face t o cont r ol ot her k inds of t r affic as w ell. The com m and sh ow in t e r f a ce in t er f ace- nam e r a t e - lim it show s t he st at ist ics for r at e- lim it ing; cle a r cou n t e r s int er face- nam e clears t he st at ist ics for a fresh look. Ex a m p le 2

221

You can use CAR t o lim it TCP SYN floods t o par t icular host s, w it hout im peding ex ist ing connect ions. Som e at t ack er s hav e st ar t ed using v er y high st r eam s of TCP SYN pack et s t o har m sy st em s. This ex am ple lim it s TCP SYN pack et s dir ect ed at host 10.0.0.1 t o 8 Kbps and allow s a m axim um burst size of 8000 byt es:

! We don't want to limit established TCP sessions -- non-SYN packets ! access-list 103 deny tcp any host 10.0.0.1 established ! ! We do want to limit the rest of TCP (this really only includes SYNs) ! access-list 103 permit tcp any host 10.0.0.1 ! ! interface configurations for network borders ! interface Serial3/0/0 rate-limit input access-group 103 8000 8000 8000 conform-action transmit exceedaction drop ! I SP CAR Con f ig u r a t ion Te m p la t e I t is r ecom m ended t hat I SPs ser iously consider inst alling a CAR configur at ion on t h eir bor der r out er s t o deal w it h I CMP and TCP SYN at t acks. Ther e w ill be a sm all ov er head t o handle t his, but t he sm all ov er head is bet t er t han hav ing t he net w or k connect iv it y disr upt ed or com plet ely disabled because of a DoS at t ack . The configur at ion t em plat e m ight be som et hing like t he follow ing:

! traffic we want to limit access-list 102 permit icmp any any echo access-list 102 permit icmp any any echo-reply ! access-list 103 deny tcp any any established access-list 103 permit tcp any any ! ! serial interface configuration interface Serial3/0/0 description Link to UPSTREAM A rate-limit input access-group 102 256000 8000 8000 conform-action transmit exceedaction drop rate-limit input access-group 103 8000 8000 8000 conform-action transmit exceedaction drop ! interface Serial 3/1/0 description Link to UPSTREAM B rate-limit input access-group 102 256000 8000 8000 conform-action transmit exceedaction drop

222

rate-limit input access-group 103 8000 8000 8000 conform-action transmit exceedaction drop ! The t em plat e r at e- lim it s I CMP echo and echo- reply t o 256 Kbps and TCP SYN t o 8 Kbps. Any one w ho im plem ent s t his t em plat e should choose v alues t hat ar e appr opr iat e t o cir cuit capacit ies and t he t y pe of t r affic nor m ally ex pect ed on t he net w or k . The ex am ple her e is t ak en fr om a r out er t hat has a 2- Mbps uplink cir cuit t o bot h it s upst r eam s. For t he pr evious exam ple, t he out put fr om t he r out er m ight look som et hing like t he follow ing:

gw#show interface serial 3/0/0 rate-limit Serial3/0/0 "Link to UPSTREAM A" Input matches: access-group 103 params: 8000 bps, 8000 limit, 8000 extended limit conformed 275702 packets, 14948676 bytes; action: transmit exceeded 0 packets, 0 bytes; action: drop last packet: 52ms ago, current burst: 4 bytes last cleared 03:29:31 ago, conformed 9000 bps, exceeded 0 bps matches: access-group 102 params: 256000 bps, 8000 limit, 8000 extended limit conformed 14017 packets, 4390628 bytes; action: transmit exceeded 137 packets, 48783 bytes; action: drop last packet: 1624ms ago, current burst: 0 bytes last cleared 03:29:20 ago, conformed 2000 bps, exceeded 0 bps gw# These cor r esponding access list hit s show

gw>sh access-list 102 Extended IP access list 102 (Compiled) permit icmp any any echo (1464042 matches) permit icmp any any echo-reply (3 matches) gw>sh access-list 103 Extended IP access list 103 (Compiled) deny tcp any any established (1426506 matches) permit tcp any any (278199 matches) gw> This show s all t he pack et s t hat hav e m at ched t he access list and t o w hich t he r at elim it ing par am et er s w ould hav e been applied. The sh ow in t e r f a ce ex cer pt pr ev iously show ed t hat 137 pack et s hav e ex ceed t he 256 Kbps r at e set in t he r a t e lim it com m and, indicat ing quit e a lar ge num ber of incom ing I CMP echoes on t he inbound int er face.

223

Sm urf Defense Sum m a ry This chapt er has discussed sev er al effect iv e t ools for m inim izing sm ur f at t ack s on t he I nt er net . The t op t hr ee passiv e t ools ar e list ed her e. You should r ev iew each sect ion for det ails—each is st r ongly r ecom m ended for consider at ion in any I SP net w or k . • • •

I nt er face ser v ices ( no I P dir ect ed br oadcast ) Egr ess and ingr ess filt er ing ( including uRPF) CAR

Act iv e defenses ar e t ools, t echniques, or pr ocedur es ex ecut ed dur ing at t ack s. They ar e used t o lim it or block t he at t ack in pr ogr ess. I n m any inst ances, t hese t ools w ill hav e an effect on ot her I nt er net applicat ions and ser v ices, y et t he t r ade- off is bet w een no I nt er net ser v ices and lim it ed disr upt ion. I t is highly r ecom m ended t hat t he I SP docum ent and t r ain st aff on t he use of t hese t ools. That w ay , t he I SP’s NOC can quickly r espond t o an at t ack in pr ogr ess. Cisco I OS Soft w ar e has secur it y t ools t o cov er t he br oad r ange of net w or k ing, and m any new secur it y t ools ar e being added as t he indust r y dem ands t hem . Many of t hese t ools ar e sit uat ion- specific, w it h som e being w r it t en t o help ent er pr ise net w or k s and ot her s being m or e suit ed for I SP env ir onm ent . CAR is per haps t he best t ool t o help as an act iv e sm ur f defense; ex am ples of CAR configur at ion in a passiv e sit uat ion hav e been descr ibed alr eady . To use CAR as an act iv e t ool, addit ional inst ances and access list s can be added as r equir ed along t he lines of t he pr ev ious ex am ples.

Re a ct in g t o Secu r it y I n cide n t s Based on t he ev olv ing secur it y t hr eat s in t he I nt er net , w hat should t he I SP’s obj ect ive be? The I SP has sever al decisions t o m ake, som e shor t - t erm , m ediumt erm , and long- t er m . This sect ion look s at som e of t he appr oaches t hat t he I SP should consider. I n all cases, t he fir st policy decision m ust about w hat t o do in t he im m ediat e sit uat ion. I t can be eit her t o dr op it , r at e- lim it it , or do not hing at all. There is an im m ediat e t hr eat t o t he net w or k , so t he I SP m ust r eact im m ediat ely t o pr ot ect t h e int egr it y of it s ow n business. Oft en t his decision is m ade by looking at t he nat ur e of t he at t ack , deciding w het her t her e w ill be an im m ediat e im pact on t he net w or k , and t hen r eact ing by block ing t he at t ack , r at e- lim it ing t he pack et s causing t he at t ack, or sim ply deciding t hat t he at t ack w on’t im pact t he net w or k .

Appr oa ch e s This sect ion descr ibes t he appr oach t hat all I SPs should consider for t heir over all secur it y ar chit ect ur e. Most of t he t ools descr ibed in t his chapt er ar e passiv e t ools. When configur ed, t hey w ill help pr ev ent secur it y pr oblem s fr om happening and w ill m ake it m ore difficult t o cause m ischief on t he I SP’s net w ork. The dist inct phases of dealing w it h a secur it y incident ar e t he follow ing:

224

•

•

• •

•

Ph a se 1 : Pr e pa r a t ion — The m ost cr it ical phase cov er s all t he configur at ions, pr ocedur es, dat a collect ion, ex er cises, and ot her fact or s t hat pr epar e an I SP’s oper at ions and secur it y t eam for handling a secur it y incident . Ph a se 2 : I d e n t if ica t ion— This t r igger s a secur it y incident . Ev ent s such as t hose fr om m onit or ing t ools t hat show an incr ease of act ivit y, a cust om er calling t he NOC, or t he ident ificat ion of an at t ack ar e t he fir st st ep in t he incidence r esponse. Ph a se 3 : Cla ssif ica t ion— Know t hy enem y . Classificat ion digs int o w hat t ype of at t ack is being m ade against a t ar get on t he net w or k. This under st anding is essent ial t o t r acing back and r eact ing t o t he incident . Ph a se 4 : Tr a ce b a ck— Tr acing t he incident t o an ent r y point int o t he I SP or acr oss t he I nt er net t o t he act ual sour ce is an essent ial par t of incident r esponse. The r eact ion phase depends on t he r esult s of a t r aceback t hr ough t he net work. Ph a se 5 : Re a ct ion — React ion is w hat y ou do t o r esolv e t he at t ack . I t could be as sim ple as choosing not t o r eact or as ext r em e as unplugging a connect ion t o a net w or k ( cust om er or peer ) .

These five phases allow for a hier ar chy of w hat an I SP should do t o m inim ize t he effect s of any secur it y incident .

Som e Ex a m ple s The fir st policy decision t hat an I SP w ill need t o m ake is w hat t o do w it h t he packet s w hen an at t ack is ident ified. When DDoS at t ack s ar e ident ified and classified, t he I SP m ust m ak e a decision. Classificat ion w ill pr ov ide t he DDoS flow ’s sour ce I P addresses—hence, a t ar get for filt er ing. How ever , do you filt er all t he packet s fr om t hat sourc e addr ess, filt er som e of t he packet s fr om t hat sour ce addr ess, or j ust r at e- lim it t he at t ack from t hat address? The sour ce I P addr esses fr om DDoS at t ack s can hav e one of four char act er ist ics. The char act er ist ic of t he sour ce w ill be a fact or in t he decision on coar se pack et dr ops, det ailed packet dr ops, r at e lim it ing, or no act ion at all. •

•

Sp o o f e d RFC 1 9 1 8 a n d sp e cia l- u se a d d r e sse s— These ar e t he easiest addresses t o spot and drop. They are w ell know n, are not in t he global I nt ernet Rout e Table, and easily can be dr opped w it h passiv e pack et - filt ering t ools. Many net w or k s hav e ingr ess/ egr ess pack et filt er s t hat block t hese pack et s fr om pr opagat ing t hr ough t he I nt er net . Unfor t unat ely , t hese passiv e packet filt ers are not ubiquit ous. So som et im es an I SP w ill see DDoS at t ack s w it h t hese addr esses. The r eact ion decision is easy : Dr op t hese. Sp oof e d a d d r e sse s t h a t a r e n ot in t h e g lob a l I n t e r n e t Rou t e Ta b le — Cyber punks have found w ays t o get ar ound t he w ell- k now n pack et filt er s t hat drop well- known RFC 1918 and special- use addr esses. The m ost com m on t echnique is t o use addr esses t hat have not been allocat ed t o any I SP or added t o t he global I nt er net Rout e Table. RI Rs allocat e block s of I P addr esses t o I SPs and subr egist r ies. These RI R allocat ions com e fr om delegat ions of aut hor it y fr om t he I nt er net Assigned Num ber s Aut hor it y ( I ANA) . The list of I Pv 4 addr ess block s t hat hav e not been act iv at ed is easily accessible at ht t p: / / w w w .isi.edu/ in- not es/ iana/ assignm ent s/ ipv 4- addr ess- sp ace. This m ak es it easy for som eone t o cr eat e an at t ack w it h spoofed addr esses based

225

•

•

on t his list . Because t hese addr esses ar e not in t he global I nt er net Rout e Table, a sim ple check in t he forw arding t able easily can aid in t heir ident ificat ion. Som e t ools, such as uRPF, m ake t his check st r aight for w ar d, m ak ing it har der for t hese at t ack s t o be successful. The r eact ion w hen faced w it h t hese sor t s of at t ack s is t o dr op t he pack et s. V a lid a d d r e ss f r om a D D oS a ge n t — Som e at t ack er s w ill sacr ifice t he DDoS agent t hat t hey hav e penet r at ed t o achiev e t heir obj ect iv es. I n t his case, t he sour ce addr ess w ill be t he act ual addr ess of t he penet r at ed sy st em . For som e at t ack s ( such as TCP SYN and ACK at t ack s) , t hese ar e easy t o ident ify and classify . Ot her at t ack s, designed t o ov er w helm t he t ar get w it h v alid t r affic, ar e not as easy t o ident ify and classify , especially w hen t he at t ack s ar e based on sim ple HTTP GET r equest . When a par t icular sour ce addr ess has been classified as a DDoS agent , t he I SP needs t o decide w het her t o dr op all t he pack et s fr om t hat penet r at ed m achine, dr op only t he at t ack ing t r affic flow , or j ust r at e- lim it t he at t ack. This is a valid com put er w it h a per son behind it . Hence, dr opping all packet s fr o m t his source address m ight seem ex t r em e. Yet it is a penet r at ed m achine, m ak ing it a t hr eat t hat needs t o be r esolv ed. The r eact ion decision for t hese sor t s of at t ack s is m ix ed: Most can be dr opped; a few can j ust be r at e- lim it ed. Sp oof e d a d d r e ss u sin g a v a lid a d d r e ss f r o m so m e w h e r e e lse o n t h e I n t e r n e t — The m ost difficult sour ce addr esses t o r eact t o ar e t hose t hat ar e valid addr esses but t hat ar e spoofing anot her sit e. This is a case in w hich you w ill get a r ev er se DDoS at t ack . The at t ack er ’s act ual t arget is not t he t ar get t hat t he DDoS agent s send pack et s t o, but t he r eal t ar get is t he one w hose addr esses ar e being spoofed. For ex am ple, t he r eal t ar get is an ent er pr ise w it h sev er al connect ions t o t he I nt er net . The at t ack er k now s t hat sev er al connect ions, r edundancy, I DS t ools, fir ew alls, and ot her t r icks m ake t he ent er pr ise a t ough t ar get t o t ak e dow n. So, inst ead of t r y ing t o t ak e dow n t he ent er pr ise’s net w or k, t he at t acker w or ks t o get ot her s t o isolat e it fr om t he net . DDoS at t ack s spoofing t he ent erpr ise’s sour ce I P r ange fr equent ly ar e at t em pt ed.

These ar e j ust a few ex am ples of t echniques used by cy ber punk s t o im plem ent DoS at t ack s —t her e ar e ot her s based on sim ilar ideas. The m or e t he I SP is aw ar e of t he capabilit ies and oppor t unit ies av ailable t o t hese people, t he bet t er prepared and prot ect ed it w ill be if any problem occurs on it s net w ork.

Su m m a r y This chapt er cov er ed r out er secur it y , r out ing pr ot ocol secur it y , and gener al net w or k secur it y . I t t hen w ent int o som e of t he t echniques and configur at ion con cept s t h at I SPs should be consider ing and im plem ent ing in t heir back bones, especially t he significance of BCP 38/ RFC 2827. I t discussed in dept h t he gr eat im por t ance of uRPF and gav e configur at ion ex am ples of t he differ ent applicat ions and scenar ios in w hich uRPF can be applied. The chapt er concluded by discussing t he use of CAR as an ant iDoS t ool. The “ Technical Refer ences and Recom m ended Reading” sect ion at t he end of t his book has m any r efer ences and point er s t o fur t her t echnical docum ent s discussing I SP secur it y issues. This is a v er y lar ge subj ect t hat quit e easily could j ust ify a w hole book in it self. We hope t hat t his chapt er has given you an over view of w hat is

226

av ailable in I OS Soft w ar e so t hat I SPs can pr ot ect t hem selv es, t heir cust om er s, and t he I nt er net .

En dn ot e s 1. The Telnet ser v er is disabled on any VTY por t t hat does not hav e a passw or d or som e ot her aut hent icat ion configur ed. 2. Sect ion enhancem ent s ar e com pliment s of Chr is M. Lonvick ( clonv ick @cisco. com) . 3. SSH client and server also w as included in I OS Soft w are from 12.1T ( and in t he m ainline 12.2 release) . 4. Exper im ent s w it h m ult icast BGP ( MBGP) now have begun on som e I SP sit es. Deploym ent of MBGP r equir es a r et hink and r edesign of t he egr ess and ingr ess r out e filt er s. 5. Cur r ent ly , t he m inim al allocat ion block for t he RI Rs consist s of / 20s. 6. Because I nt er net dr aft s ex pir e six m ont hs aft er publicat ion, it is w or t h check ing w het her t her e is a r evised dr aft or w het her an RFC docum ent has r eplaced t he dr aft befor e im plem ent ing t his list . 7. Thank s t o Pat r ick W. Gilm or e of Pr ior i Net w or k s for supply ing t he list . 8. NANOG Mailing List —Subj ect : Re: t oo m any rout es. From : Sean M. Doran < sm d@clock .or g > Dat e: 10 Sep 1997 20: 33: 09 – 0400 ( see w w w .nanog.or g for m ailing list archives) . 9. Based on work done by Rakesh Dubey ( r dubey @cisco. com) an d St ev e LeGault ( slegault @cisco. com) . 10. Or iginal docum ent at ion on uRPF w as per for m ed by Br uce R. Babcock ( bbabcock @cisco. com) . 11. This t echnique is used in som e m ult ihom ing configur at ions. 12. This sect ion w as t ak en fr om t he r elease not es of CSCdp76668 by t he engineer w ho coded t his funct ion, Neil Jar vis ( n j ar v is@cisco. com) . 13. Ther e w as a bug in I OS Soft w ar e Release 12.0( 10) S. The Help opt ion in I OS Soft w ar e display s only t he st andar d and ex t ended st andar d access list s as opt ions. St andar d/ ex panded and ex t ended/ ex panded ACLs bot h w or k . 14. For m ult ihom ed cust om er s, see t he sect ion on uRPF lim it at ions. 15. The cust om er’s assigned I P bl ock ( t hat is, r out ing pr efix es) usually is inser t ed int o t he I SP’s net w or k in one of sever al w ays. Any of t hese w ays w ill w or k as t he infor m at ion is passed t o t he CEF t ables.

227

16. At t he t im e of t his publicat ion, w or k has st ar t ed in t he I ETF t o consider som e new opt ions for one I SP t o influence t he best - pat h decision of anot her I SP. 17. Because local pr efer ence is a nont r ansit ive at t r ibut e w or king only inside one AS, m any cust om er s ar e encour aged t o include t he BGP com m unit y noex port w it h t heir split adv er t isem ent s along w it h t he adver t isem ent of t heir ent ir e block s. That w ay , t hey r eceiv e t he benefit s of t r affic - engineered m ult ihom ing w it h t he I SP w hile keeping t he m or e specific pr efixes fr om t he split - adv er t isem ent t echnique off t he r est of t he I nt er net . 18. Policy- based r out ing could be used, but it is neit her efficient ( because of t he per for m ance penalt y ) nor scalable ( consider how t his w ould be im plem ent ed for t housands of leased- line cust om er s) . 19. RFC 1998, “ An Applicat ion of t he BGP Com m unit y At t r ibut e in Mult i- home Rout ing,” now is used w idely in t he I SP com m unit y. 20. I bid. 21. This sect ion is an edit ed ver sion of Cr aig Huegen’s w or k on sm ur f and fr ag pr ot ect ion. For t he lat est infor m at ion, r efer t o Cr aig’s page at ht t p: / / w w w . pent ics. net / . Cr aig can be cont act ed at ch u egen @cisco. com.

228

Cha pt e r 5 . Ope r a t iona l Pr a ct ice s So far t his book has giv en specific and det ailed adv ice about r out er- configurat ion best pr act ices for I SP back bones. I t is im por t ant t o see t hese suggest ions in t he bigger pict ur e of an act ual I SP business. This chapt er cov er s som e of t he oper at ional issues r egar ding t he est ablishm ent of an I SP back bone, t he choices m ade, t he posit ioning of har dw ar e, t he configur at ion of soft w ar e, and t he est ablishm ent of ex t er nal r elat ionships. Aspect s of I SP business oper at ions hav e been cov er ed in ot her publicat ions, such as I SP Survival Guide: St rat egies for Running a Com pet it ive I SP, by Geoff Hust on, and w on’t be cov er ed her e. The chapt er fills t he gap bet w een t he high- lev el business st r at egy and t he box es of equipm ent t hat ar r iv e at a br andnew st ar t up’s headquar t er s. This final chapt er guides you t hr ough t ypical point - of- pr esence ( PoP) t opologies, PoP design, back bone net w or k design, and t he posit ioning and configur at ion of I SP ser vices. Defining a w or kable and scalable addr essing plan appar ent ly is a har d t ask for sev er al new com er s t o t he I nt er net , so t his t opic is cov er ed as w ell. Tw o sect ions deal wit h rout ing pr ot ocols: The fir st cov er s int er ior r out ing pr ot ocol design, and t he second look s at ex t er ior r elat ionships and giv es som e sim ple ex am ples for efficient and effect iv e m ult ihom ing. Secur it y is an im por t ant t opic, so som e basic consider at ions ar e pr esent ed. The chapt er concludes w it h a look at ar eas t hat m ost new I SPs for get about : out - of- band m anagem ent , a t est labor at or y, and oper at ional st andar ds.

Poin t - of - Pr e se n ce Top olog ie s A PoP is not hing m or e t han a physical locat ion w her e an I SP has equipm ent . This can r ange fr om a sm all pile of equipm ent in t he cor ner of an office t o w hole floor s of lar ge collocat ion cent er s, som et im es called car r ier hot els, in m aj or cit ies. Regar dless of t he shape or im plem ent at ion of t hese PoPs, successful I SPs hav e est ablished a v er y clear st r at egy for const r uct ing a sm oot hly oper at ing PoP. This sect ion cov er s som e of t his st r at egy . Rout er s and ot her equipm ent ar e subdiv ided int o clear oper at ing unit s. These unit s are t ypically core, dist ribut ion, and access unit s, oft en augm e nt ed by ot her unit s such as bor der , VPN, br oadband, and collocat ion/ host ing unit s.

Core The PoP cor e has dev ices called cor e r out er s. These suppor t high- bandw idt h links only , int er connect ions bet w een sim ilar cor e dev ices, connect ions t o ot her unit s w it hin t he PoP, and connect ions t o ot her PoPs. I t is v er y com m on for I SPs t o inst all t w o core rout ers—one dev ice im plies a single point of failur e, som et hing t hat high- qualit y and high- av ailabilit y net w or k s w on’t t oler at e. The definit ion of high bandw idt h depends on t he r egion of t he w or ld. I n som e count r ies, int r acor e connect ions oft en ar e m ade at Gigabit Et her net , 2.4- Gbps, or 9. 6- Gbps speeds. I n ot her count r ies, 10- Mbps Et her net is st ill com m only used. Regar dless of t he t echnology or speeds av ailable, t he cor e net work clearly is ident ified—and w ould look som et hing like t he diagr am in Figur e 5- 1.

229

Fig u r e 5- 1 . PoP Cor e

Not ice t hat t her e ar e t w o r out er s w it h one high- speed dir ect int er connect and link s out side t he PoP t o ot her locat ions. The dir ect high- speed int er connect is used so t hat t r ansit t r affic t hr ough t he PoP has t he m inim um am ount of har dw ar e t o t r av er se. Choosing t o place an Et her net sw it ch as t he t r ansit pat h cr eat es a single point of failur e for t hat PoP because a failur e in t he sw it ch ( or sw it ches) w ill r esult in t r ansit no longer being available. The m edium used for t he high- speed int er connect now aday s is eit her Et her net or POS; t he lat t er cur r ent ly offer s higher speeds t han Et her net , but , at t he t im e of t his w r it ing, 10- Gbps Et her net w as in dev elopm ent , r eady t o be a viable com pet it or t o shor t - reach OC192/ STM64 POS connect ions. Som e I SPs choose t o m ak e connect ions t o r em ot e PoPs on one cor e r out er only . The adv ant age is t hat one less hop is display ed in a r out e t r ace, but t his m ak es t he r out er a single point of failur e. Of cour se, if t he PoP is connect ed t o t hr ee or m ore PoPs, t he I SP could st udy t r ansit t r affic flow s t o opt im ize t he pat h for t hose and t o m aint ain a sensible back up st r at egy in case of equipm ent failur e.

D ist r ibut ion The dist r ibut ion layer is one st ep r em oved fr om t he cor e and get s it s nam e fr om it s funct ion of act ing as a dist r ibut ion lay er bet w een t he cor e r out er s and t he access par t of t he net w or k. I ndeed, m any sm all- t o m edium- size I SPs don’t hav e any dist r ibut ion lay er ; t hey sim ply connect t he access par t of t he net w or k t o t he cor e. I t all de pends on t he size of t he PoP. The dist ribut ion layer can be m ade up of t wo or m ore rout ers —quit e oft en t her e could be consider ably m or e. I SPs conscious of pr oviding qualit y ser vices t o t heir cust om er s oft en connect pr em ium cust om er s dir ect ly t o t his dist r ibut ion layer, by passing any bandw idt h aggr egat ion t hat is t ak ing place at t he access lay er . The phy sical m edium used for connect ing dist r ibut ion t o cor e v ar ies. FDDI w as v er y popular in t he early t o m id- 1990s, but since t he advent of high- speed connect ions such as DS3/ E3 and Ocx / STMn bandw idt h, it has w aned in popular it y t o t he ex t ent t hat it is vir t ually not used anyw her e. Tw o popular m edia t ypes ar e in use t oday: sw it ched Et her net and DTP r ings. The for m er t ype usually is built in t he for m of Et her net sw it ches suppor t ing speeds fr om 10 Mbps t o 1 Gbps. The lat t er t y pe is

230

available fr om speeds of 622 bps t o 2.5 Gbps and using t he Spat ial Reuse Pr ot ocol ( SRP) ar guably is m or e efficient and m or e r eliable t han using sw it ched Et her net . ( Many I SPs pr efer passiv e t echnology such as FDDI and DPT t o act iv e t echnology such as sw it ched Et her net because t her e is one less har dw ar e or soft w ar e dev ice in t he net w or k t o go w r ong.) The ot her pr ev iously popular m edia t y pe for PoP int er connect s w as ATM. How ev er , ATM also has fallen by t he w ay side because t he ex t r a it em s of ex pensiv e equipm ent and ATM’s well- know n inefficiencies ( popular ly know n as cell t ax) w hen car r ying I P have m ade it quit e unpopular in m any inst allat ions. The relat ive cheapness of Et her net sw it ches com par ed w it h t he cost of ATM t o car r y out t he sam e funct ion has r esult ed in Gigabit Et her net sw it ches r eplacing ATM sw it ches in I SP cor es in m any cases. Figur e 5- 2 show s a t y pical PoP w it h t he cor e and dist r ibut ion lay er s dr aw n in det ail. Point - t o- point links have been chosen as t he cor e - t o- dist r ibut ion int er connect m edium ; t hese eit her can be back- t o- back Et her net connect ions or POS int er faces on t he rout ers. Point - t o- point lin ks oft en ar e chosen over alt er nat ive m et hods because t her e is only a sim ple cable j oining t he dev ices. As m ent ioned ear lier , inser t ing ex t r a dev ices int o t he I SP net w or k sim ply m eans t hat som et hing else in t he pack et pat h could go w r ong. This design is very fault t oler ant : I f t he cable fails or one of t he cor e r out er s fails, t he back up pat h t hr ough alt er nat iv e connect ions or an alt er nat iv e cor e r out er is available. This design is also ver y sim ple, in t hat no ot her pow er ed devices are linking t he core and dist r ibut ion lay er s of t he net w or k . Fig u r e 5- 2 . PoP D ist r ib u t ion La y e r

The alt er nat iv es designs ar e t o eit her place Et her net sw it ches in t he cor e, as shown in Figur e 5- 3, or use DPT, as shown in Figur e 5- 4. I nst alling t w o lar ge Et her net sw it ches as t he PoP cor e dev ice is quit e a com m on occur r ence for I SPs w it h m ediumsize net w or k infr ast r uct ur es. They sav e on t he ex pense of inst alling a lar ge num ber of high- speed por t s on t heir cor e r out er s, but t hey hav e t he addit ional ex pense of pur chasing t w o high- per for m ance Et her net sw it ches t o giv e t hem t he desir ed funct ionalit y . The com m on design is t o oper at e t he sw it ches as Lay er 2 dev ices, j oined by Et her Channel or Gigabit Et her net and separ at ed int o t w o VLANs. The solid

231

lines in Figur e 5- 3 indicat e t he fir st VLAN; t he dot t ed lines indicat e t he second VLAN. The r out ing configur at ion can be set up so t hat one VLAN is t he pr im ar y and t he ot her is backup, or t he t w o VLANs can be load- shared. This design is also very fault t oler ant . I f eit her sw it ch fails or eit her cor e r out er fails, t her e is a backup pat h t hr ough t he ot her dev ice of t he pair . Ther e is addit ional com plex it y ov er t he design used in Figur e 5- 2, but quit e oft en t he cor e sw it ches hav e m any VLANs configur ed on t hem t o ser v e ot her funct ions in t he access lay er of t he PoP. Fig u r e 5- 3 . PoP D ist r ibu t ion La y e r w it h Et h e r n e t Sw it ch e s

Fig u r e 5- 4 . PoP D ist r ibu t ion La y e r U sin g D PT

The second alt er nat iv e is t o use Dy nam ic Pack et Tr anspor t ( DPT) , as show n in Figur e 5- 4. DPT is t he com m er cial nam e for an opt ical r ing t opolo gy dev eloped by Cisco and

232

now r ecognized as t he I EEE 802.17 specificat ion. DPT uses SRP t o achiev e opt im um use of t he r ing: The m or e nodes t hat ar e added t o t he r ing, t he gr eat er t he pot ent ial t hroughput of t he ring is w it h good design of t he ingress and egr ess pat hs. Not ice t he int erleaving of rout ers in Figur e 5- 4. To get t he best adv ant age of DPT and SRP, t he connect ions show n ar e t he m ost efficient . Tr affic flow s from t he dist ribut ion layer t o t he net w or k cor e. Each dist r ibut ion r out er has a dir ect pat h t o t he net w or k cor e, ensur ing m ax im um ut ilizat ion of t he bandw idt h av ailable. Only w hen t he connect ion fr om dist r ibut ion t o cor e br eaks in t his t opology w ill t he bandw idt h fr om dist r ibut ion r out er s t o cor e be halv ed. This t y pe of t opology has pr ov en v er y effect iv e in sev er al I SPs’ PoPs in r ecent y ear s. I t is consider ably m or e cost - efficient t han using point - t opoint POS link s or Gigabit Et her net or using Et her net sw it ches—and, of cour se, it is m uch m or e r eliable because physical failur es of t he r ing ar e dealt w it h at t he link layer and not at Layer 3. I n choosing a PoP design for t he dist r ibut ion lay er and it s int er connect ion w it h t he cor e, bear in m ind t he pr inciples discussed her e. The few er har dw ar e dev ices t her e ar e in t he pack et pat h, t he less lik ely t her e w ill be a pr oblem caused by soft w ar e or har dw ar e failur e of t hose int er m ediat e dev ices. Each m et hod is com m only used; each m et hod has it s st rong support ers and opponent s. The design in Figur e 5- 2 is ver y com m on, but t he one illust r at ed in Fig u r e 5- 3 is also popular because it sav es t he I SP from deploying m any high- speed int erfaces for point - t o- point connect ions in t he PoP. Likew ise, t he design in Figur e 5- 4 is popular w it h I SPs w ho hav e fav or ed t he use of FDDI in t he past and w ho r ecognize t he benefit s t hat DPT/ SRP can give over bot h point - t o- point link s and Et her net sw it ches.

Acce ss This is t he edge of t he I SP net w or k facing t he cust om er . The access lay er can be m ade up of all kinds of devices, fr om a sim ple r out er suppor t ing PSTN m odem banks t o cable - aggr egat ion dev ices suppor t ing br oadband cust om er s. For t his r eason, I SPs t end t o subdivide t heir access layer s int o m or e m anageable unit s. Lar ger businesses oft en hav e differ ent business unit s r unning differ ent access ser v ices, and it is useful t o giv e t hese differ ent unit s m anagem ent access t o t he dev ices t hey ar e ser v icing. Quit e oft en it is possible t o see DI AL ( PSTN and I SDN) , low - speed perm anent line ( 64 Kbps t o 512 Kbps) , high- speed per m anent line ( 2 Mbps t o 8 Mbps) , and br oadband ( ADSL and cable) locat ed in separ at e access point s. I SPs suppor t ing cust om er s w it h ev en gr eat er bandw idt h needs w ill sim ply add m or e cat egor ies t o t hose list ed—in t he Unit ed St at es, it is com m on t o see a 45 Mbps access layer , a 155 Mbps access lay er , and ev en higher- bandw idt h access lay er s. Figur e 5- 5 giv es an ex am ple of an access net w or k . It show s t w o Et her net sw it ches used t o connect sev er al r em ot e access ser v er s ( RAS) for dial- in users. RAS servers ar e gener ally unsophist icat ed w hen it com es t o r unning r out ing pr ot ocols. All t hey r equir e is an addr ess block assigned t o t hem for pr oviding addr ess space for dial- in user s. The dist r ibut ion r out er s w ill inj ect t his pr efix int o t he I SP’s iBGP r out ing pr ot ocol r unning acr oss t he back bone. Fig u r e 5- 5 . PoP Acce ss N e t w or k

233

Most I SPs don’t bot her t oo m uch w it h t he r edundancy of t he RAS. Sever al w ill pr ov ision t w o sw it ches, but it is r ar e for t hem t o pr ov ision t w o Et her net s out of t he RAS. The t op- end RAS devices have t w o Et her net s, so bet t er r edu ndancy can be pr ov ided. How ev er , in m ost cases, I SPs t end t o use low - end unit s—and large num ber s of t hese because failur es ar e act ually easier t o deal w it h. For ot her t ypes of access net w or ks, t her e w ill be ot her designs. Cable and xDSL access require a m uch m or e sophist icat ed PoP design; det ails of t hese ar e cov er ed lat er.

H ost in g Anot her v er y com m on ser v ice offer ing is Web host ing. I n t he ear ly day s of t he I nt er net , t his w as sim ply t he I SP agr eeing t o allow t he cust om er t o br ing in som e unspecified ser v er and connect it t o t he I SP’s back bone. This w as an at t r act iv e ser v ice offer ing because t he bandw idt h needs fr om t he cont ent cont ained on t he ser v er w er e gr eat er t han t he cust om er could affor d t o supply t o it s ow n pr em ises or t hat t he t elco easily could pr o vision. As t he I nt er net has m at ur ed, w hole businesses hav e been cr eat ed dedicat ed t o Web host ing—indeed, t he w hole net w or k design for such an or ganizat ion is com plex and det ailed. For I SPs t hat sim ply ar e int er est ed in host ing a few t o a hundr ed or so serv er s for cust om er s ( t he av er age m iddle of t he r oad I SP) , t he follow ing design t ips are helpful. The m ost im por t ant t hing t o r ealize is t hat t he I SP k now s absolut ely not hing about t he cust om er ’s sy st em s: w hat soft w ar e is r unning, w hat ser v ices ar e being host ed, and w hat ot her t hings m ight be offer ed. I n planning such a ser v ice, I SPs need t o

234

car efully consider w hat t hey ar e offer ing. Det ailed design for a Web host ing net w or k is not cov er ed; only t he equipm ent placem ent is consider ed her e. Host ed ser v ices should be t r eat ed lik e any ot her connect ion t o t he access par t of t he I SP net w or k . I f host ing is cr it ical t o t he business, it can be connect ed dir ect ly t o t he cor e of t he I SP net w or k using high- speed links. Redundancy in r out er s connect ing t he host ing LAN t o t he cor e is necessar y: I f one r out er fails, t he ot her w ill pr ovide ongoing connect iv it y . The sam e is t r ue for any sw it ched Et her net infr ast r uct ur e being used on t he host ing LAN. Figur e 5- 6 show s an ex am ple of how t he Web host ing net w or k fit s int o t he I SP PoP. Som e I SPs connect t he host ing lay er dir ect ly t o t he PoP cor e r out er s. Be aw ar e, how ev er , t hat t his put s a consider able filt er ing bur den on t he cor e r out er s, som et hing t hat pot ent ially could com pr om ise r est of t he PoP. Fig u r e 5- 6 . PoP W e b H ost in g N e t w or k

Com m e nt a r y Figur e 5- 6 t ypifies t he PoP design t hat I SPs gener ally use in t heir PoPs. The sm aller I SPs obv iously w on’t hav e as m uch sophist icat ion and segr egat ion of equipm ent , and t he lar ger I SPs w ill have a significant ly lar ger lay out . Many t y pes of access lay er s ar e possible —t w o hav e been show n by w ay of ex am ple, alt hough ot her access lay er s ar e v er y sim ilar in im plem ent at ion.

235

Poin t - of - Pr e se n ce D e sign Hav ing est ablished t he pr inciple of subdiv iding t he PoP int o unit s depending on funct ion and m anagem ent , t he next im por t ant pr inciple of PoP design is t o est ablish a basic lay out and r eplicat e t his acr oss t he ent ir e back bone net w or k . Successful I SPs w or k on t he pr inciple of hav ing only t hr ee or four differ ent designs for each PoP. This ensur es t hat t he net w or k is not t oo com plex , allow ing st aff t r aining t o be giv en on each design. I t also ensur es t hat t he engineer ing and oper at ions st aff ar e aw ar e t hat t he r est of t he I SP’s back bone w ill follow one of t he few chosen lay out s. The ot her adv ant age of hav ing only a few differ ent designs com es at t he point of deploy ing new infr ast r uct ur e. I nst ead of hav ing t o t ak e t im e for t he senior design engineer s t o com e up w it h a new design, t he I SP oper at ions t eam sim ply can inst r uct t he equipm ent deploym ent par t of t he oper at ion t o inst all a PoP in Cit y A using Design B. The new PoP is deliver ed w it h a basic configur at ion r eady for t he net w or k ing t eam t o int egr at e int o t he back bone. The net w or k ing t eam has no sur pr ises because t he design confor m s t o one of t he previously agreed- upon st andar ds. I n m anpow er t er m s, t his m eans t hat expensive st aff m em ber s ar e not t ied up w it h or der ing equipm ent , building r acks, or exam ining collocat ion space. I SPs t end t o choose one design for an av er age- size PoP—a scaleddow n v er sion of t his becom es t he sm all PoP, and a scaled- up version becom es t he lar ge PoP. Ther e is lit t le benefit t o hav ing any fur t her opt ions; as w as show n pr ev iously , PoPs ar e m ade out of cor e, dist r ibut ion, and access lay er s, so hav ing t hree desig ns t hat incorporat e t his layering is about as m uch as can be realist ically done w it hout t he m odel br eaking dow n. A fur t her adv ant age is t hat t he sear ch for collocat ion space, or r oom space t o inst all t he facilit y, is act ually sim plified. I f t he design spec ificat ion dict at es a part icular dim ension w it h par t icular elect r ical r equir em ent s, par t icular air - condit ioning r equir em ent s, and pr ecise secur it y r equir em ent s, it becom es significant ly easier t o w or k t hr ough a checklist t o est ablish a suit able locat ion. No t hing is w orse t han going halfw ay t hr ough a build t o discov er som e fundam ent al design flaw about t he facilit y . These t hr ee issues ar e pr obably t he k ey com ponent s t hat m ost I SPs w or r y about . Ther e ar e undoubt edly m any ot her s, depending on t he dept h of det ail required, but t hese ar e fundam ent al t o designing t his par t icular par t of t he I SP infr ast r uct ur e.

Ba ck bon e N e t w or k D e sign Ty pically t w o st y les of back bone design ar e in use t oday . The fir st em ploy s a st art ype net w or k, in w hich t he I SP is based in one cit y and sim ply inst alls point - t o- point cir cuit s t o t he ot her locat ions. This isn’t a w ide- ar ea net w or k, as such, but m or e of a collapsed back bone in t he m aj or headquar t er s hub. The adv ant age of t his st y le is t hat m anagem ent is easier and t he net w or k is not c om plex. Each sit e has one exit pat h: t o t he m iddle. How ev er , t his design is fundam ent ally flaw ed w hen it com es t o offer ing any level of ser vice qualit y. I f any link br eaks in t he st ar , t he r em ot e sit e is cut off unt il t he link is r est or ed. I f t he cent r al node has a pr oblem , t he ent ir e net w or k goes dow n. For t his r eason, a st ar net w or k is not used m uch beyond t he init ial incept ion of t he I SP business and, in fact , is act iv ely discour aged in som e cir cles t hese day s.

236

Figur e 5- 7 show s an ex am ple of a net w or k built using a st ar t opology . Not ice t he single connect ion fr om each PoP back t o headquar t er s, w her e t he m aj or facilit y is. I f any cir cuit fails, t he PoP t hat it connect s t o t he cor e is disconnect ed and r em ains out of service unt il t he link is repaired again. Even m ore problem at ic, if t he headquart ers sit e has a pr oblem , t he ent ir e I nt er net connect ion ser vice t hat t his I SP is offer ing is affect ed because all ex t er nal connect ions fr om t his I SP go t hr ough t he headquar t er s. Refer r ing t o t his figur e, it is ver y easy t o see w hy m ost I SPs opt for som e redundancy in t he backbone. This is not a scalable net w ork, nor is it a viable business m odel. Fig u r e 5- 7 . St a r N e t w or k Topology

The alt er nat ive is t he w ell- t r ied and ver y t r ust ed backbone cor e design. Maj or locat ions ar e select ed as t he I SP cor e back bone, and t he r est of t he net work is dist r ibut ed ar ound t hat . For exam ple, an I SP m ight decide t hat it s six m aj or cit ies w ill for m t he cor e of it s backbone and w ill build high- speed link s bet w een t hese cit ies. The r est of t he net w or k w ill be built ar ound t his cor e lay er . Som e people call t his a dist r ibut ion net w or k because it m im ics t he dist r ibut ion design used in PoPs, as m ent ioned pr ev iously . Each sit e in t he dist r ibut ion lay er w ill hav e r edundant link s t o t he cor e —one prim ary pat h w ill be a high- band link t o t he cor e, and one back up pat h w ill go t o anot her dist r ibut ion layer sit e. I f t he int er dist r ibut ion sit e link goes dow n, t he pr im ar y pat h funct ions as nor m al for bot h sit es. I f one pr im ar y pat h goes dow n, t he sit e has backup t hr ough t he ot her sit e and it s pr im ar y link t o t he cor e . Figur e 5- 8 show s an ex am ple of how t he sit es in Figur e 5- 7 could be connect ed pr ov iding r edundancy t o each sit e. Not ice t hat each sit e has t w o ex it pat hs, and t he net w or k it self has t w o separ at e connect ions t o t he I nt er net . Thr ee ex t r a dom est ic circuit s are required, and one m ore upst ream circuit is required. How ever, t he m aj or differ ence is t hat t he net w or k in Figur e 5- 8 can be r un w it h 99.5 per cent guar ant eed av ailabilit y ; it is not at t he m er cy of any one cir cuit going dow n, and it is ev en resilient against com plet e sit e failur e and upst r eam cir cuit failur e.

237

Fig u r e 5- 8 . M e sh Ba ck bon e

Look ing ar ound t he m any I nt er net back bones t oday , it is easy t o see t hat m any I SPs have adopt ed t he lat t er design. New com er s t o t he I nt er net business ar e st ill sk ept ical about it because t hey see m or e t han t he m inim um necessar y out lay of equipm ent and r esour ces. Ther e ar e t r ade- offs: I f t her e is a desir e t o hav e t he net w or k 100 per cent av ailable, r edundancy is r equir ed, no m at t er how good any vendor m ight claim t hat it s equipm ent or infr ast r uct ur e r eliabilit y is. I f only 80 per cent av ailabilit y is r equir ed, building a cheap net w or k w ill be j ust fine. I n our exper ience, m ost new com er s t r y t o do t he for m er using t he lat t er concept s, and finding an accept able com pr om ise is oft en t he m aj or challenge. A fur t her point t hat oft en is m issing in net w or k designs is t hat m any new com er s ar e com plet ely sold on t he possibilit ies of t he new leading- edge t echnologies being offer ed. MPLS is per haps t he m ost r ecent ex am ple: I P back bone pr ov ider s ( usually I SPs) now can com pet e w it h t he incum bent t elcos t o pr ov ide VPN t echnologies t o cust om er s w ho pr ev iously w er e t he dom ain of t he t elcos only . These new - world I SPs w ould deploy MPLS on t op of t heir exist ing net w or k infr ast r uct ur es—or w or se, lit er ally t hr ow t oget her t he m inim um I P infr ast r uct ur e necessar y t o suppor t an MPLS backbone. This st r at egy alw ays is doom ed t o failur e. The m ost im por t ant point w hen it com es t o designing a net w or k is t hat it m ust be r obust and foolpr oof enough t hat any ov er lay net w or k such as MPLS can oper at e r eliably and t r anspar ent ly .

I SP Se r vice s The ar t of put t ing t oget her t he sy st em ser v ices r equir ed t o suppor t t he I nt er net infrast ruct ure probably could fill a book in it s ow n right . Our aim here is t o t ry t o ex plain t o t he net w or k engineer how t o pr oper ly posit ion and set up t he net w or k por t ion of t he v ar ious ser v ices t hat an I SP should be offer ing t o it s cust om er base.

238

Meet ing net w or k engineer s w ho dism iss ser v ices such as t he DNS as “ Oh, it ’s t he sy st em s engineer s pr oblem ” is som ew hat disconcer t ing because t he posit ioning and connect ion of t hese ser v ices t o t he I nt er net back bone is act ually quit e im por t ant . Net w or k engineer s need t o be aw ar e of sev er al I SP ser v ices. The t hr ee m ost im por t ant ones t oday ar e t he DNS, m ail, and new s ser vices. Alt hough m any pr inciples ar e t he sam e for each ser v ice, it is w or t h cov er ing each in t ur n.

DNS The DNS is t he public face of t he I nt er net , quit e lit er ally . The com m on com plaint from first - t im e user s of t he I nt er net if t hey can’t get t o a sit e is, “ The I nt er net is dow n.” I n fact all t hat could be w rong is t hat t he nam e isn’t in t he DNS, or t he DNS hasn’t been set up pr oper ly , or t her e is som e infr ast r uct ur e pr oblem t hat r ender s t he DNS unusable. All t hr ee scenar ios ar e ver y com m on. The DNS has t hr ee im por t ant par t s: t he pr im ar y nam eser v er , t he secondar y nam eser v er , and t he caching nam eser v er ( or r esolv er , as som e call it ) . Each at t r act s a slight ly differ ent deploy m ent st r at egy , and each is r eally im por t ant t o deploy properly on an I SP net work. The basic pr inciple for each of t hese t y pes of ser v er s is r edundancy . I f one ser v er goes dow n, t he I nt er net com m unit y w ant s ( if not r equi res) t hat you have som e backup so t hat nam e resolut ion st ill w orks. I f you have one DNS server and it goes dow n, t he sy st em s using t he I nt er net hav e no w ay of m apping any nam e t hat y ou give t o an address—nor do t hey have any w ay of m apping addr esses on I nt er n et infr ast r uct ur e int o nam es. The for m er m ak es t he I nt er net appear dow n; t he lat t er m ak es t he I nt er net look lik e a j um ble of num ber s r at her t han t he nam es t hat w e hum ans can handle m ore easily. Pr im a r y D N S Only one pr im ar y DNS ser v er ex ist s for each domain nam e. I t is t he sy st em t hat holds t he m ast er delegat ion r ecor ds of t he nam e fr om t he par ent zone. For ex am ple, Cisco w ill hav e a pr im ar y nam eser v er for t he zone cisco. com. Gener al I SP design is t o place t he pr im ar y DNS ser ver in a ver y safe par t of t he net work. Safe usually equat es t o secure, w it h I SPs put t ing t heir prim ary DNS behind dedicat ed fir ew alls, on secur e LANs, and so on. Because t her e is only one pr im ar y DNS ser v er car r y ing all t he r ecor ds for t he zone and any ot her zones t he I SP chooses t o host , it is look ed aft er v er y w ell. Specificat ions hav e been incr easing ov er t he year s. A key point t hat w e have not iced is t hat m em or y and net w or k I / O speed ar e cr ucially im por t ant , especially for sy st em s car r ying a lar ge am ount of infor m at ion. I SPs oft en host zones on behalf of t heir cust om er s or host zones for t heir peer I SPs as par t of t he peer ing agr eem ent t hey hav e ( m or e discussion in t he follow ing sect ion on secondar y DNS) . Depending on t he size of t he files in quest ion, it oft en is consider ed good pr act ice t o use differ ent phy sical har dw ar e for local and cust om er zones files and for peer zone files. The r eason is scalabilit y : Tw o sy st em s w ill scale bet t er t han one, and t he I SP’s cust om er zone files don’t get any per for m ance im pact caused by overloading because of a peer’s problem .

239

Finally , t he pr im ar y DNS ser v er host s t w o t y pes of zone files. The for w ar d zone m aps nam es t o addr esses so t hat user s can cont act r em ot e locat ions w it hout hav ing t o r em em ber t he act ual addr esses ( t hese ar e har der t o r em em ber t han nam es, and addr esses t end t o change m or e oft en t han nam es any how ) . The r ev er se zone m aps addr esses t o nam es so t hat any look up on an addr ess w ill r ev eal t he nam e of t he sy st em in quest ion. Rev er se zones ar e used in t r oubleshoot ing connect ion pr oblem s as w ell as for DNS consist ency w or k—som e sit es w ill not allow connect ions t o be m ade unless t he for w ar d DNS ent r y m aps t o t he r ever se DNS ent r y. Se con da r y D N S As t he nam e suggest s, t he secondar y DNS oper at es in a suppor t ing r ole t o t he pr im ar y DNS. Ther e can be only one pr im ar y DNS for any zone, but t her e can be sev er al secondar ies t hat hold back up infor m at ion. The secondar y ser v er s get t heir infor m at ion fr om t he pr im ar ies—any change on t he pr im ar y aut om at ically is sy nchr onized t o t he secondar y as par t of t he DNS updat e pr ocess. I SPs usually can cope w it h set t ing up a pr im ar y DNS, but w hen t he v ar ious nam e r egist r ies t ell t hem t hat t hey need t w o nam eser v er s, t he pr oblem s begin. The fir st com m on m ist ak e is t o put t he DNS secondar y sy st em on t he sam e LAN connect ed t o t he sam e sw it ch as t he pr im ar y ser v er . I f t he LAN goes dow n, t he sw it ch br eak s, or t he gat ew ay r out er s t o t he out side w or ld br eak , bot h t he pr im ar y and t he secondar y DNS ar e disabled. We r et ur n t o our “ t he I nt er net is dow n” pr oblem . The solut ion t o t his is sim ple: Put t he secondar y DNS elsew her e in t he net w or k. Som e exam ples ar e discussed lat er in t his sect ion. The second com m on m ist ake is t o sim ply not bot her w it h a secondary DNS. An I P address is m a de up for t he benefit of t he dom ain r egist r ies ( w hich r ar ely check t hat it act ually r esponds w it h cor r ect zone infor m at ion) , and t he r esult is a sit uat ion in w hich 50 per cent of all DNS lookups w ill be aim ed at a nonexist ent host . This w ill m ake all zones ser v ed by t he pr im ar y nam eser v er look as t hough t hey ar e sluggish because of long delay s w hile 50 per cent of t hem hav e t o t im e out t he nonex ist ent server. The t hir d com m on m ist ake is t o set up only t w o DNS ser ver s. The fir st com m on m ist ake is avoided by put t ing t he t w o ser v er s on differ ent par t s of t he net w or k , but oft en t his is on differ ent LANs in t he sam e PoP; t he t w o syst em s m ight be in t he sam e r ack and m ight be fed fr om t he sam e pow er sour ce! Ther e is no r edundancy here, and it is a w ast e of t im e t o c onsider t his opt ion. The m inim ally cor r ect w ay of deploy ing DNS is t o hav e one pr im ar y and at least one secondar y . Tw o, t hr ee, or four secondar ies ar e ev en bet t er . Consider t his ex am ple. An I SP has set up DNS and chooses t o hav e t hr ee secondar ies. One secondary is locat ed in a r em ot e PoP in t he back bone. Anot her secondar y is locat ed in anot her r em ot e PoP, m aybe close t o an exchange point or ot her ext er nal link t o it s backbone. The four t h secondar y is locat ed out side t he st at e or count r y t hat t he I SP is based in. This is t he ult im at e in r edundancy . I f t he upst r eam link s br eak , t he zones t hat t he I SP is suppor t ing don’t disappear , so t he I SP and it s cust om er s do not disappear fr om t he I nt er net . The nam eser v er ov er seas w ill r espond w it h addr esses, and ser v ices such as SMTP w ill t ake car e of queuing e- m ail unt il t he link r et ur ns ( in t he e- m ail case, t his is far m ore desirable t han having e- m ail bounce w it h Unknow n Dom ain) .

240

The follow ing exam ple is how APNI C, t he Regional I nt er net Regist r y ( RI R) for t he Asia Pacif ic r egion, handles it s DNS set up for t he apnic.net zone:

$ dig apnic.net ns ;; ANSWER SECTION: apnic.net. apnic.net. apnic.net. apnic.net.

50m44s 50m44s 50m44s 50m44s

;; ADDITIONAL SECTION: svc00.apnic.net. ns.ripe.net. rs.arin.net. ns.apnic.net.

1d23h53m25s 1d23h54m46s 1d23h53m25s 1d9h29m16s

IN IN IN IN

NS NS NS NS

svc00.apnic.net. ns.ripe.net. rs.arin.net. ns.apnic.net.

IN IN IN IN

A A A A

202.12.28.131 193.0.0.193 192.149.252.21 203.37.255.97

$ The four sy st em s list ed her e beside t he NS r ecor ds ar e on differ ent cont inent s! The sy st em sv c00.apnic.net is in Tok y o, Japan; ns.r ipe.net is in Am st er dam , Net her lands; r s.ar in.net is in Washingt on D.C., U.S.A.; and ns.apnic.net is in Br isbane, Aust r alia. This is pr obably one of t he best exam ples of DNS r edundancy in t he I nt er net t oday . Ca ch in g D N S The t hir d t ype of DNS ser vice is gener ally a m yst er y t o m any of t he sm aller I SPs because it is har dly discussed out side DNS oper at or cir cles and out side lar ger I SPs. The caching DNS is a sy st em t hat m aint ains a cache of DNS infor m at ion; it doesn’t pr ov ide a secondar y funct ion for any zone files, but it k now s w her e t o go t o r et r iev e t he inform at ion. Caching nam eser v er s t y pically ar e used by t he I SP t o answ er day - t o- day DNS quer ies. End host s usually ask for a DNS “ r esolv er ” w hen t hey ar e being configur ed. The r esolver is t he caching nam eser ver . Not e t hat pr im ar y and secondar y DNS servers also w ill answ er r esolv er quer ies, but as net w or k s scale, t his is gener ally undesirable —t he last t hing t hat any I SP w ant s is for it s DNS ser vice t o be sw am ped by m iscellaneous DNS queries. A few good st r at egies exist for deploying a caching nam eser ver syst em in an I SP back bone. One com m on one is t o deploy a caching DNS at each PoP. The har dw ar e r equir em ent s ar en’t significant —a fast pr ocessor , lar ge m em or y, and a fast I / O ar e t he m ain needs and, at t he t im e of w r it ing, easily could be sat isfied by a 1U r ackm ount PC av ailable fr om a num ber of differ ent v endor s. The addr ess of t he caching DNS w ould be dist r ibut ed t o cust om er s at t he PoP t o use as t he DNS r esolv er so t hat one sy st em is dedicat ed t o a PoP. I f r edundancy is r equir ed, dedicat ing t w o ser v er s per PoP m ight work: Lookups are done in a round- robin m anner, so it doesn’t m ake m uch sense t o put t he second r esolv er at anot her locat ion—t his gener at es a sm all am ount of ex t r a t r affic on t he back bone.

241

A fur t her enhancem ent of t he st r at egy using t w o r esolv er s per PoP is t o giv e t w o unique addr esses t o all t he r esolv er s in t he I SP back bone. These t w o addr esses ar e r out ed as / 32 host addr esses acr oss t he I SP back bone. Each PoP w ould hav e t w o r esolv er s, so cust om er s connect ing t o one PoP w ould use bot h r esolv er s t her e. I f t hese sy st em s bot h disappear ed because of m aint enance or an incident , no r econfigur at ion of t he cust om er sy st em s w ould be necessar y —t he I SP’s int erior r out ing pr ot ocol sim ply w ould r eadj ust t o point t he pat h t o t he nex t near est sy st em s. An added benefit is t hat t he cust om er w ould not obser v e any dow nt im e in t he DNS, im pr ov ing it s per cept ion of t he QoS offer ed. This is a use of t he so- called any cast addr ess in I Pv4. Many syst em s share t he sam e I P addr ess. I f one sy st em disappear s, t he int er ior r out ing pr ot ocol ensur es t hat t he nex t near est sy st em w ill r eceiv e t he t r affic. Fr om our ow n ex per ience, t his set up has w or k ed ex t r em ely w ell and has pr ov ed it self t o be t he m ost r eliable w ay of deliv er ing DNS r esolv er ser v ices t o cust om er s. ( Not ice t hat each r esolv er sy st em r equir es it s ow n unique addr ess also; ot her w ise, it w ill be incapable of com m unicat ing w it h ot her ent it ies in t he I nt er net .)

M a il Mail is t he second m ost im por t ant ser v ice on t he I nt er net aft er Web br ow sing. Jour nalist s alw ay s t alk about sur fing t he Web; m ost new PC pur chases ar e t he r esult of people w ant ing t o dial up and look at I nt er net Web sit es t o find infor m at ion. Keeping in t ouch is t he second m aj or r eason for buying a PC, and e- m ail is t he popular w ay of doing so. Also, m any com panies now use e- m ail as t he prim ary m et hod of dist r ibut ing infor m at ion t o cust om er s and dist r ibut ing infor m at ion int er nally . So, if e- m ail br eak s, get s lost , or bounces, t he effect is v er y not iceable. Receiv ing t he bounce m essage four hour s aft er sending an e- m ail is very frust r at ing and adds t o t he per cept ion t hat t he I nt er net is unr eliable. I n fact , all t hat m ight be w r ong is t hat t he r em ot e m ail host has been disconnect ed. Configur ing m ail ser ver s t o fit int o an I SP backbone should not be t hat har d for a net w or k engineer . Yet again, follow ing t he r ule t hat t w o or m or e of ever yt hing is good w ill ensur e t hat t her e is a r eliable e- m ail ser vice. I ndeed, som e I SPs ar e alm ost as diligent w it h t heir m ail ser ver designs as t hey ar e w it h DNS. The DNS is helpful in pr oviding r edundancy for m ail sy st em s using t he concept of an MX ( or m ail exchanger) record. A dom ain can and should have m ore t han one MX r ecor d. A num ber specifier follow s, giv ing t he pr ior it y of t he m ail sy st em—t he low er t he value is, t he higher in priorit y t he m ail rela y is. The one w it h t he low est v alue is t he pr im ar y MX, t he int ended dest inat ion of t he e- m ail. Look ing at w hat som e or ganizat ions hav e done t o configur e t heir e- m ail is helpful. The follow ing ex am ple show s PI PEX’s ar r angem ent and is a good guide for how it should be done:

$ dig pipex.net mx

242

;; ANSWER SECTION: pipex.net. 8H IN MX pipex.net. 8H IN MX lfallback1.lnd.ops.eu.uu.net. pipex.net. 8H IN MX

10 shed.pipex.net. 90 99 fallback.mail.pipex.net.

;; ADDITIONAL SECTION: shed.pipex.net. 8H IN A 158.43.128.176 lfallback1.lnd.ops.eu.uu.net. 57m17s IN A 62.189.34.30 fallback.mail.pipex.net. 1H IN A 158.43.128.81 fallback.mail.pipex.net. 1H IN A 62.189.34.25 fallback.mail.pipex.net. 1H IN A 62.189.34.30 $ The pr im ar y device has an MX of 10, so m ail for user @pipex.net w ill be deliver ed t o t his syst em if it is available. I f t he m achine does not r espond, m ail goes t o t he fallback syst em w it h MX of 90. That syst em has a differ ent I P addr ess t han t he pr im ar y MX and is act ually in a differ ent physical locat ion on t he I SP’s net w or k. I f t hat sy st em also fails, PI PEX has pr ov ided a dedicat ed fallback sy st em w it h t hr ee separ at e I P addr esses. I SPs t hat ar e pr ov iding connect ion ser v ices for cust om er s r eally should offer MX r elay ser v ices for t heir cust om er s t hat ar e r unning t heir ow n SMTP gat ew ay s. Ot herwise, t he cust om er has only it s own m ail host for all e- m ail—if t he host goes dow n, e- m ail is queued up on t he sending syst em . I f t he host does not com e back on t im e, t he e- m ail w ill be bounced t o t he sender. A t ypical set up for all MX records should look like t he follow ing:

Customer.com Customer.com Customer.com

MX 5 MX 10 MX 20

mail.customer.com. relay0.isp.net. relay1.isp.net.

Mail for t he cust om er is deliv er ed dir ect ly t o t he cust om er ’s m ail host . I f it goes dow n or t he cir cuit t o t he cust om er becom es unav ailable, t he I SP’s r elay sy st em s t ake over in or der of pr ior it y specified by t he MX values. ( Not e t hat if an I SP is supply ing r elay ing ser v ices for it s cust om er s, it m ust be v er y car eful t hat it s m ail soft w ar e is configur ed t o allow r elaying only for it s ow n cust om er s. Open r elays ar e consider ed ex t r em ely bad pr act ice and oft en r esult in t he per pet r at or ’s net w or k being added t o a r out ing black list .)

New s Usenet New s has been ar ound for as long as t he I nt er net has exist ed in it s var ious for m s. I t s m ain popular it y cent er s on t he non r eal- t im e dist r ibut ion of infor m at ion on v ir t ually all possible t opics t hat hum ank ind could be int er est ed in. Pr ov iding a Usenet New s ( or sim ply new s) ser v ice alw ay s has been a challenge for I SPs. I n our per cept ion, t he new s hey day w as t he m id- 1990s, w it h huge v olum es of infor m at ion and m at er ials being dist r ibut ed around t he I nt er net . A t y pical full new sfeed w as consum ing ar ound 20 GB per day , m ak ing it bot h a challenge t o st or e and a challenge t o deliv er on t he t y pical cir cuit capacit ies at t he t im e. I t w as quit e

243

com m on for cust om er s dem anding a full new sfeed t o t r y t o sign up t o at least a 128Kbps cir cuit j ust t o ensur e t hat t her e w as sufficient capacit y t o deliv er new s. Now adays t he new sfeed volum e is only m ar ginally lar ger t han it w as in t he m id1990s, but t y pical cir cuit capacit ies and har d- disk sizes are one or t w o or der s of m agnit ude lar ger . The pr oblem of deliv er ing a new sfeed is not so gr eat , but t he lessons of t he ear lier year s ar e w ell w or t h heeding t o ensur e a scalable and r eliable ser v ice. N e t w or k D e sig n Good at t ent ion needs t o be paid t o t he design of t he new s- delivery net work. For a net w or k engineer , it should not be som et hing t hat is sim ply left t o t he syst em s gr oup t o w or k out for t hem selves because t he volum e of m at er ial t hat can be deliver ed oft en can cause chok e point s in t he back bone. Fur t her m or e, pay ing close at t ent ion t o t he design w ill ensur e t hat t he cust om er ’s ex per ience is a good one and w ill not add m or e fuel t o t he “ I nt er net is slow ” per cept ion. Figur e 5- 9 show s an ex am ple of how an I SP w ould im plem ent a scalable new sdist r ibut ion sy st em acr oss it s back bone. One sy st em is dedicat ed t o r eceiv ing new sfeeds fr om ex t er nal I SPs, w het her t his is peer or upst r eam or ot her I SPs t hat hav e agr eed t o pr ov ide a new sfeed. This sy st em usually is closed off fr om t he r est of t he I nt er net so t hat only t he per m it t ed host s get access t o t he sy st em ; t his pr ev ent s unw ant ed connect ions fr om slow ing t he sy st em . This single sy st em dist r ibut es t he r eceiv ed new s t o t he r est of t he new sfeeder sy st em s acr oss t he back bone. Each PoP could have one or m or e feeder syst em s, and a hier ar chy could be set up in each PoP if t her e is sufficient dem and for new sfeeds fr om cust om er s. The new sfeeder sy st em s t hen dist r ibut e t he feeds t o cust om er s. De pending on t he har dw ar e and new s soft w ar e chosen, t his could be anyt hing fr om 10 t o m or e t han 100 feeds per ser ver system . Fig u r e 5- 9 . I n b ou n d N e w s D ist r ib u t ion

244

Figur e 5- 10 show s t he r ev er se cy cle. Cust om er sit es t hat post new s it em s t o t he Usenet sy st em send t heir feeds t o a dedicat ed ser v er in t he I SP net w or k . One syst em is show n in t he figur e, but quit e oft en I SPs w ill set up one syst em per PoP sim ply t o r eceiv e t hese feeds. I t depends on how m uch v olum e t her e is t o handle and how m any cust om er s t her e ar e. The new s- collect or sy st em t hen sends it s feed t o t he new s- dist r ibut ion sy st em , and t he cy cle cont inues. I f t he new s is m eant t o be sent t o t he I nt er net , t he dist r ibut ion sy st em sends t he it em s t o it s ex t er nal peer s. Fig u r e 5- 1 0 . Cu st om e r N e w s Re d ist r ib u t ion

Com m e n t a r y The new s- dist ribut ion design described in t his sect ion is quit e sim ple, wit h it s m ain adv ant age being scalabilit y . Com put er s ar e r elat iv ely cheap t hese day s, so hav ing lot s of sm aller CPUs and disk s is gener ally bet t er policy for deploy m ent t han using one subst ant ial sy st em . Obv ious enhancem ent s t o t his design occur w hen t he I SP pr ov ides a sy st em for online br ow sing of new s—again t his should be a dedicat ed sy st em , separ at e fr om t he sy st em s pr oviding or receiving newsfeeds from cust om er s. Depending on t he Usenet new s soft w ar e being used, it should be possible t o configur e t he sy st em s t o per m it or deny online br ow sing as appr opr iat e.

Keeping Soft w a re Up-t o- Dat e A com m on error m ade by m any I SPs is r ely ing on t he ser v er soft w ar e t hat is dist r ibut ed w it h t heir UNI X syst em s r at her t han keeping t he ver sions cur r ent . Mail and DNS ar e t w o ex t r em ely high- pr ofile ser vices, and t he developer s and user com m unit y cont inuously ar e m ak ing enhancem ent s and bug fixes.

245

I t ’s quit e com m on for oper at ions st aff t o subscr ibe t he I SP t o t he CERT adv isor y m ailing list —her e all adv isor ies about k now n secur it y pr oblem s w it h soft w ar e on UNI X- and Window s- based oper at ing syst em s ar e announced. Cisco also m akes use of t he CERT advisor y pr ocess t o announce det ails of any secur it y pr oblem s discov er ed w it h t he I OS Soft w ar e. Pay ing at t ent ion t o t he adv isor ies is a v er y pr udent t hing t o do: People w ho w ant t o at t ack sy st em s or I SP net w or k s also pay at t ent ion t o t he CERT advisor y lis t and oft en at t em pt t o ex ploit a new ly announced secur it y bug in hopes of gaining access t o a syst em or a net w or k. When a pr oblem is announced on t he CERT list , I SPs ar e st r ongly encour aged t o upgr ade t he affect ed soft w ar e ( not e t hat a CERT adv isor y gener a lly is not m ade unt il t her e is a pat ch or a w or k ar ound fr om t he v endor inv olv ed) . For t his r eason, st ick ing t o v endor- shipped m ail and DNS applicat ions is not a good st r at egy . Oper at ing sy st em r eleases t ak e a snapshot of t he cur r ent applicat ion st at us, and int erim versions rarely are released ( unless a m aj or problem is discov er ed) . Most I SPs t ak e t he st andar d oper at ing sy st em r elease and t hen com pile or inst all t he cur r ent lat est ver sion of t he ser ver soft w ar e. Mail ( SMTP) and DNS have been list ed as t he ke y applicat ions t o be aw ar e of, but ot her cr it ical ones can include new s, FTP, Secur e Shell, PoP3, and HTTPD. ( Any publicly available TCP por t needs t he lat est ser v er soft w ar e list ening on t he por t t o be cur r ent at all t im es.) Because t he sour ce has been com piled by t he I SP syst em s engineers, it becom es very sim ple t o pat ch t he v er sion t hat is in oper at ion, and t his gener ally t ak es no m or e t han a few m inut es of work. Befor e deploym ent , t est ing of t he fix clear ly is r equir ed, especially if t he I SP has any unusual oper at ional r equir em ent s or has m ade som e ot her changes. But t he fir st r equir em ent is alw ay s secur it y and int egr it y of t he ser v ices suppor t ed, so dow nt im e w hile pat ching and t est ing is clear ly pr efer able t o hav ing a v ulner able sy st em st ill in ser v ice.

I Pv 4 Addr e ssin g in a n I SP Ba ck bon e Designing an I P addressing plan for an I SP backbone is probably one of t he m ore com plex issues t hat an I SP faces w hen ent er ing t he com pet it iv e I nt er net m ar k et place. I nt er net gr ow t h in r ecent y ear s has been ex plosiv e, and cov er age is now w ell bey ond t hat of academ ic and gov er nm ent or ganizat ions in t he Unit ed St at es. Because t he finit e am ount of I Pv 4 addr ess space av ailable, t his gr ow t h has r esult ed in t he com m unit y ’s desir e t o be m or e car eful about how t he r esour ce is shared am ong it s user s. The I ANA is responsible for t he dist ribut ion of all I P address space; I ANA is now one of t he funct ions of I CANN. To t his end, it has delegat ed t he t hr ee RI Rs ( ARI N, RI PE NCC, and APNI C) t o car r y out it s r esponsibilit ies in t he t hr ee r eg ions of t he w or ld. The t hr ee RI Rs and t he user com m unit y ar e w or k ing t o ensur e t hat each user of public addr ess space is efficient w it h ut ilizat ion, t o be fair t o all w ho r equir e t his finit e r esour ce. Thr ee RI Rs ex ist at t he t im e of w r it ing; t heir geogr aph ical reach is show n in Figur e 5- 11. The geogr aphical r each of t he t w o em er ging RI Rs, LACNI C ( Lat in Am er ican and Car ibbean NI C) and Afr iNI C ( Afr ica NI C) , is show n in t he figure as well. Fig u r e 5- 1 1 . RI R Ar e a s

246

I t is hoped t hat t his sect ion w ill encour age I SPs t o consider how t o design a scalable addr essing plan. Conser v at ion and efficient ut ilizat ion of addr ess space oft en ar e seen as pr oblem at ic and ev en undesir able by I SPs t r y ing t o m inim ize t he num ber of prefixes carried around in t heir net w ork. This is only an ex am ple of t he consider at ions necessar y w hen designing t h e addr essing plan for an I SP net w or k. I t does not advise on how t o go t hr ough t he pr ocess of apply ing for addr ess space fr om t he t hr ee r egional addr ess r egist r ies or how t o configur e t he BGP r out ing pr ot ocol. I t is assum ed t hat t he r eader is fam iliar w it h t er m s such as CI DR, classless addr essing, and t he / N net w or k m ask schem e. Not e t hat ant iquat ed t er m inology such as Class C is not used her e and should not be used in t he v ocabular y of t he I nt er net .

Business M odel and I P Address Space I t gener ally is accept ed t hat t o t hink ahead one and t w o y ear s w hen designing an I SP net w or k is r easonable. Bey ond t hat , giv en t he r apid gr ow t h of t he I nt er net , it is v er y difficult t o pr edict w hat new t echnologies w ill be av ailable or w hich dir ect ion t he business m ight be headed. I n t he early 1990s, available address space w as m ore lim it ed t han it is t oday. CI DR hadn’t yet been deployed on t he I nt er net backbone, and Class B space w as r apidly r unning out . Ther e w as r eal concer n about t he r at e of addr ess space consum pt ion, and t he gr ow t h of t he I nt er net Rout e Table w as ex ponent ial. Som e pr edict ions w er e m ade t hat I Pv 4 space w ould r un out in 1998. This concer n spur r ed on t he int r oduct ion of CI DR and t he classless I nt er net . Since t hen, t he r egist r ies hav e been m aking classless net w or k assignm ent s, and DHCP has m ade addr essing and renum bering on t he LAN very t rivial. What w as a problem in 1994 w as no longer one in 1996. An I SP m ight go int o business t o ser vice a par t icular m ar ket place. How ever , local oper at ing condit ions m ight change, or new “ killer applicat ions” m ight appear on t he

247

I nt er net . As a r esult , t he I SP’s or iginal business obj ect iv e could change com plet ely . Fur t her m or e, addr ess space r equir em ent s usually change w hen t his happens. The t hr ee I nt er net r egist r ies look for an est im at e of addr ess r equir em ent s for a y ear or m or e ahead and m ake allocat ions on t hat basis. They do so because m ost I SP business plans oper at e ar ound sim ilar concept s, and t his m eans t hat I SPs don’t hav e t o const ant ly r ev isit t he r egist r ies t o get t heir nex t allocat ions. I t is im por t ant t o sit back for a few m inut es and look ser iously at t he av enues of gr ow t h for any I SP business. This isn’t easy in a m ar k et place t hat is gr ow ing ex ponent ially . How ev er , it pay s in t er m s of engineer ing t im e t o t r y t o get a realist ic pict ur e of w her e t he business w ill lead. Think about w het her t her e likely w ill be m or e PoPs or lar ger PoPs, m or e backbone links, new ser vices, new equipm ent w it h incr eased por t densit y , and so on. An I SP oper at ion usually has a business plan t o guar ant ee funding fr om sponsor s or shar eholder s. The ex ist ence of a business plan im plies t hat som e t hought has been giv en t o t he dir ect ion of gr ow t h of t he net w or k ; t he t w o ar e int er link ed.

Addr e ss Pla n I t is a const ant surprise t o m any seasoned cam paigners how lit t le at t ent ion indust r y new com er s pay t o dev eloping a sensible and coher ent addr essing plan for t he back bone. I SPs oft en spend m ont hs of det ailed design for t heir back bone, com plet ely for get t ing t o put a plan t oget her . A w eek or few befor e t hey go liv e, t hey r ealize t hat addr ess space is r equir ed, and t he ensuing panic w it h applicat ions t o t he r egist r ies r esult s in an inevit able slow ing of t he deploym ent plans. Designing a net w or k and t hink ing of addr ess space r equir em ent s ar e pr ocesses t hat happen at t he sam e t im e. Docum ent ing r equir em ent s at t hat ear ly st age m ak es t he docum ent at ion subm ission t o t he RI Rs significant ly easier , w it h allocat ions m ade in v er y shor t t im e scales. This sect ion docum ent s a sim ple exam ple of an addr ess plan for a gr ow ing I SP business. Obv iously , it is int ended only as a guide, but under st anding t he pr inciples w ill help in designing m ost ot her t ypes of net w or ks. This plan cover s only t he I SP’s infr ast r uct ur e. Too oft en, I SPs becom e bogged dow n in second- guessing how m any cu st om ers t hey w ill have and how big t hese cust om ers m ight be w hen t rying t o give t heir est im at es t o t he r egist r ies. The r egist r ies ar e concer ned only w it h t he addr ess space t hat t he I SP w ill use for infr ast r uct ur e at t he fir st applicat ion; t hey assum e t h at t h e I SP w ill apply for addr ess space t hat it can assign t o it s cust om er s as w ell. Thr ee net w or k plans ar e pr esent ed: t he cur r ent plan, t he plan aft er six m ont hs, and t he plan aft er one y ear . The addr essing schem a should t ak e each plan int o account because it helps t he I SP plan t he net w ork grow t h over t he com ing 12 m ont hs. Not ice, how ev er , t hat each sect ion pr oduces a sum m ar y of r equir ed addr ess space. When t he t hr ee plans ar e dev eloped, t hey should be docum ent ed in a for m at sim ilar t o t his t ext . Anot her im por t ant t enet of an I SP’s business is t hat t her e can nev er be t oo m uch docum ent at ion. Good docum ent at ion such as t his helps t he engineer ing t eam w or k ing for t he I SP t o under st and t he gr ow t h plans and helps t hem engineer t he r out ing pr ot ocols as needed. I t helps t he addr ess r egist r ies under st and t he addr ess space r equir em ent s and eases t he assignm ent pr ocess.

248

I t has been assum ed all along t hat t he I SP is follow ing t he essent ial feat ur es of configur ing r out ing pr ot ocols; t y pical ex am ples can be found in Cisco’s ISP/ I XP Wor kshop t ut or ials and w er e cover ed in Chapt er 3, “ Rout ing Pr ot ocols.” I n gener al, net works are designed wit h an I GP ( such as OSPF or I S- I S) t o carry point - t o- point link addr esses, loopback int er face addr esses, and LAN addr esses. BGP is used t o car r y t he I SP’s cust om er net w or k s ( iBGP) and any ex t er nally lear ned net w or k s ( eBGP) . ( Alt hough BGP pr obably w on’t be used unt il t he I SP m ult ihom es [ connect s t o m or e t han one upst ream I SP] , it pay s t o follow t he design pr inciple in case BGP w ill be im plem ent ed in t he fut ur e.) N e t w or k Pla n : St a r t in g Of f The fir st st age inv olv es look ing at t he net w or k design at t he st ar t of t he I SP’s oper at ion. Figur e 5- 12 giv es an ex am ple net w or k—it has four r out er s, t hr ee sw it ches w it h som e host s connect ed t o t hem , and som e cust om er leased- line connect ions. Ther e is also a dialup r out er . Finally, t he net w or k has a link t o an upst r eam I SP. This is a sim ple net w or k w it h four sm all PoPs at init ial r ollout . Fig u r e 5- 1 2 . N e t w or k Pla n a t D e ploy m e n t

Also on t he figur e ar e t he sizes of t he subnet s allocat ed t o each por t ion of t he net w or k. I n det ail, t hese ar e as follow s: •

• •

WAN point - t o- point links have been assigned a / 30. Ther e ar e t w o host s on a point - t o- point link, so t he m axim um address space required is a / 30. Assigning a lar ger subnet w ould r esult in w ast ed addr ess space. ( I f t her e is t r ouble calculat ing how m any host s can fit int o a subnet , r efer t o Table 5- 1, w hich has a few exam ples of subnet t ing a / 24 addr ess block.) LANs hav e been assigned only t he addr ess space t hat t hey r equir e. One loopback int erface is assigned per rout er. Table 5-1. Subnet Sizes and Host Counts for a /24 Block Network Mask Subnets Host Count

/ 24

1

254

249

/ 25

2

126

/ 26

4

62

/ 27

8

30

/ 28

16

14

/ 29

32

6

/ 30

64

2

/ 31

128

None

/ 32

255

1

The / 30 address required for t he I SP’s upst ream link usually is assigned by t he upst r eam I SP and is not r equir ed her e. I t is assum ed t hat I P unnum ber ed is used t o c onfigure t he point - t o- point link s going t o a cust om er sit e. I t also is assum ed t hat one loopback int er face is assigned t o each of t he four r out er s. Fur t her m or e, it is assum ed t hat each Et her net sw it ch has been assigned an I P addr ess for adm inist r at iv e access pur poses. These assum pt ions ar e com m on pr act ice in m ost I SPs for t hese r easons: •

• •

Wit h I P unnum ber ed, t he point - t o- point link bet w een t he I SP and t he cust om er rout ers is not assigned an I P address. Non- Cisco equipm ent w ill use ot her convent ions but is sim i lar ly capable. Using no addr ess m eans t hat t her e is one less net w or k in t he I SP’s I GP—I GP design alw ays aim s t o have m inim al net w or k s in t he int er ior r out ing t able for efficiency and conv er gence speed. ( Som e I SPs have st ar t ed t o use / 31s on point - t o- point links—t his sav es a bit m or e addr ess space but st ill m eans t hat t he addr ess has t o be car r ied ar ound in t he I GP. See RFC 3021 for m ore inform at ion on 31- bit pr efix es.) A loopback int er face on a r out er div or ces t he r out er ’s adm inist r at iv e funct ions fr om any of it s phy sical int er faces, t her eby guar ant eeing cont inued adm inist r at iv e access in case of any phy sical link failur e. All LAN equipm ent is assigned an I P addr ess for adm inist r at iv e access. Alt hough all equipm ent w ill have a ser ial console, oft en an I P- cap able int er face is ext r em ely useful as a fir st line of ent r y for adm inist r at ion funct ions ( m any I SPs r eser v e console access as a last r esor t ) .

At t his init ial st age, w e have five / 30s, one / 29, one / 28, and t w o / 27s, plus a / 30 required for t he four loopbac k addr esses ( loopback int er faces ar e assigned a / 32) . You can follow t he sum s t hr ough in det ail her e: 1. 2. 3. 4.

Six / 30s m ake one / 28 and one / 29. One / 28 and one / 29 plus t he single / 29 m ake t w o / 28s. The t w o / 28s plus t he single / 28 m ake one / 27 and one / 28. One / 27 and one / 28, t oget her w it h t he t w o / 27s, m ake one / 26, one / 27, and one / 28. 5. The next usable addr ess boundar y fr om one / 26, one / 27, and one / 28 is a single / 25.

I n conclusion, t his net w or k infr ast r uct ur e could be num ber ed out of a single / 25 of address space. N OTE

250

This plan consider s only t he I SP’s infr ast r uct ur e. Pr ov iding addr ess space for t he I SP’s cust om ers t o use is a separat e considerat ion and is covered in m ore det ail in t he succeeding sect ions.

N e t w or k Pla n : Aft e r Six M on t h s Aft er t he fir st six m ont hs of oper at ion, t he I SP aim s t o show m oder at e gr ow t h in it s net w or k and est im at es t hat it s net w or k infr ast r uct ur e w ill show addit ional equipm ent and sit es. The net w or k is int ended t o gr ow t o t hat show n in Figur e 5- 13. The figur e also docum ent s t he addr ess plan int ended for t he net w or k at t he end of t he fir st year. Fig u r e 5- 1 3 . N e t w or k Pla n a t En d of Six M on t h s

The changes t o t he net w or k include adding t w o new sit es and upgr ading t he dialup infr ast r uct ur e t o suppor t m or e m odem lines. Ther e ar e now six sit es, alt hough t he I SP st ill has not planned for any physical r edundancy at each sit e. Not ice t hat t he I P addr essing for leased- line cust om er s st ill uses I P unnum ber ed. Look ing at t he ut ilizat ion at t his st age, t her e ar e eight / 30s for point - t o- point links, t w o / 28s, t w o / 27s, and t w o / 26s. Ther e ar e also t w o new devices now requiring loopback int er faces ( m ak ing a r equir em ent for a / 29) . You can follow t he sum s t hr ough in det ail her e: 1. 2. 3. 4. 5. 6.

Eight / 30s m ake one / 27. Two / 28s m ake one / 27. Two / 27s m ake one / 26 Two / 26s m ake one / 25. Adding all of t hese t oget her m akes a t ot al of a / 24. The ext r a / 29 r equir ed for loopback int er faces also should be count ed, m aking t he t ot al requirem ent a / 23.

251

I n conclusion, t he addr ess space r equir em ent for t his pr oj ect ed net w or k at t he end of six m ont hs is a / 23. N e t w or k Pla n : En d of Fir st Y e a r The final st age t o consider is t he plan at t he end of t he fir st y ear of oper at ion. Again, t his is a pr oj ect ion on w hat t he business could be lik e. I t is assum ed t hat t he I SP has st ar t ed t o do Web host ing for it s cust om er s and is inv est ing in a lar ge dialup net w ork ( w hich w e shall say t hat t he business plan calls for ) . The I SP also plans t o deploy t w o of ev er y t hing by t he end of t he fir st y ear of oper at ion. This affor ds incr eased r edundancy and r esiliency in t he net w or k . ( This isn’t m eant t o be a t ut or ial in net w or k design—t hese elem ent s sim ply ar e included t o show how t he addr essing plan m ight be done.) The I SP also plans t o t ake on it s fir st BGP cust om er s. Now t hat it has r edundancy in t he PoP, in t he back bone, and t o it s upst r eam s, it feels t hat it can offer t he ser vice at a sufficient qualit y t o be accept able t o cust om er s. This m eans t hat it w ill r equir e addr ess space for point - t o- point link addr esses. Figur e 5- 14 show s how t he net w or k has gr ow n and show s t he addr essing t hat goes w it h it . Som e v alues hav e been om it t ed for clar it y . Each sit e has dual r out er s now , and t he link s bet w een t hem also r equir e a / 30 of addr ess space each. The I SP has t w o upst r eam connect ions now , each com ing fr om differ ent sit es. This is also par t of t he I SP r edundancy planning. By t he end of t he fir st year , t he business is fully funct ioning and t he I SP needs all t he har dw ar e and infr ast r uct ur e r edundancy t hat it can obt ain. Fig u r e 5- 1 4 . N e t w or k Pla n a t En d of Fir st Ye a r

Looking at t he ut ilizat ion now , t here are 14 / 30s, 2 / 28s, 2 / 26s, 2 / 24s, and 12 / 32s for loopback int er face addr esses ( m ak ing a r equir em ent for a / 28) . Ther e ar e 30 leased- line cust om er s w ho w ill need point - t o- point link addr esses. Also, t he sw it ches

252

now ar e dual- hom ed t o t he r out er s, and t he I SP is using t he Hot St andby Rout ing Pr ot ocol ( HSRP) t o cr eat e a v ir t ual default gat ew ay for t he LAN devices. Using HSRP consum es anot her / 32 per LAN for t he v ir t ual gat ew ay addr ess, and I SPs gener ally configur e t w o st andby gr oups per LAN t o load- shar e bet w een t he t w o r out er s. This needs t o be r em em ber ed in t he LAN host count . Also, t he LAN host count includes t he I P addr ess r equir ed for t he r out er in ev er y case. Going t hr ough t he sum s in det ail giv es t he follow ing: 1. For t he loopbacks, 14 / 30s m ake 1 / 27, 1 / 28, and 1 / 29. 2. For cust om er point - t o- point links, 30 / 30s m ake 1 / 26, 1 / 27, 1 / 28, and 1 / 29. 3. Two / 28s m ake one / 27. 4. Two / 26s m ake one / 25. 5. Two / 24s m ake one / 23. 6. Twelve / 32s m ake one / 28. 7. From 1 and 2, t he t wo / 29s m ake one / 28. 8. From 1, 2, 6, and 7, t he four / 28s m ake one / 26. 9. From 1 and 2, t he t wo / 27s m ake one / 26. 10. From 8 and 9, t he t wo / 26s m ake one / 25. 11. From 4 and 10, t he t wo / 25s m ake one / 24. 12. From 2, 3, 4, and 5, t his m akes a t ot al of one / 23, one / 24, one / 25, one / 26, and one / 27. 13. The nex t av ailable addr ess size t hat t his can fit int o is a / 22. I n conclusion, t he addr ess r equir em ent s aft er t he fir st y ear ar e a / 22. Not ice how t he net w or k subnet s hav e been pack ed as efficient ly as possible. This is quit e a com plex net w or k now , y et w it h j udicious use of addr essing, it has consum ed less t han / 22 of addr ess space. Fur t her m or e, because t he I SP now has t w o connect ions t o t he I nt er net , it w ould hav e applied for an AS num ber and used t hat t o m ult ihom e t o t he t w o upst r eam I SPs. This allow s t he lay er ing of t he net w or k r out ing pr ot ocols, as w ill be descr ibed lat er. Not ice also t he cont inued use of I P unnum ber ed. The I SP should m ake a r easonable est im at e of how m any link s I P unnum ber ed can be used for . Som et im es cust om er r out er s m ight not be capable of suppor t ing t he concept . Ot her cust om er s m ight r equir e use of eBGP t o ex change r out ing infor m at ion w it h t he I SP and t hus w ill r equir e a / 30 addr ess for t he point - t o- point link . I n t his case, t he I SP has est im at ed t hat t her e w ill be 30 such cust om er s links. By convent ion, t he links t o t he upst r eam I SPs ar e num ber ed using addr ess space fr om t he upst ream . Also not ice t hat I P addr esses ar e dy nam ically assigned on t he dialup access ser v er s. Whereas t he I SP m ight be operat ing at a rat io of 20 users per m odem , t he r equir em ent is only for one I P addr ess t o be available per m odem line. Also not ice t hat w here a PRI m ight have 32 channels, generally only 30 are available for use in an I SDN/ PSTN ser v ice ( t he ot her 2 ar e used for signaling) —quit e conv enient w hen it com es t o I P addr essing.

253

Put t ing Toget her an Address- Deploym ent Plan The pr eceding exam ples est ablished t he addr ess r equir em ent s for t he sam ple I SP net w or k t hat has been t he subj ect of t his sect ion. The final t hing t hat needs t o be done is t o convert it int o a reasonable infrast ruct ure - assignm ent pr ocess. The follow ing sect ions consider t hese part icular net w or k addr essing r equir em ent s: • • • •

Loopback int er face on t he r out er WAN links LANs Cust om er net w or ks

Loop b a ck I n t e r f a ce s The loopback int er face on t he r out er is alw ays t he fir st consider at ion on an I SP net w or k . I t is a helpful gener al- purpose feat ure used for m any t hings, including iBGP peer ing and sour ce addr ess for pack et s or iginat ing fr om t he r out er ( useful for aut hent icat ion or filt er ing) . The ear ly chapt er s of t his book m ade m any r efer ences t o t he loopback int er face, and a w hole sect ion w as devot ed t o t he benefit s of using it s address as t he sour ce of all I P packet s originat ing from t he rout er. By t he end of t he fir st year , t he exam ple I SP est im at ed t hat it w ould have 12 r out er s in it s net w or k , so it is r easonable t o assign a / 28 t o be used for t hat purpose from t he ver y beginning. I t is advisable t o assign a single block for t his pur pose; alt hough som e I SPs num ber loopback s out of r andom chunk s of addr ess space, t he m or e com m on pr act ice is t o set aside a single block . I t becom es v er y easy t o set up filt ers on t he r out er s and NOC equipm ent so t hat only r out er s and t he NOC equipm ent get access t o each ot her . The loopback / 32s w ill be carried around only in t he I GP, and t hey pose m inim al ov er head t o t he r out ing pr ot ocol chosen. W AN Link s Th e secon d consider at ion is t he WAN links. Unless t he net w or k easily can be separ at ed int o dist inct r egions, t her e is lit t le t o be gained by sum m ar izing t he WAN / 30 addr ess space. Also, because WAN addr ess space gr ow s m or e slow ly t han ot her par t s of t he net w or k , WAN link s should be addr essed out of anot her cont iguous block fr om t he allocat ion ( say , t he chunk nex t t o t he loopback net w or k block ) . WAN link s and loopback s addr ess block s should be cont iguous; t hey gener ally don’t need m uch opt im um r out abilit y out side t he I SP’s back bone because t hey ar e used only t o connect t he back bone. They should be r eachable, but t hey ar e not significant load- gener at ing dest inat ions as far as t he I nt er net is concer ned. This concept w ill be r evisit ed in t he “ Mult ihom ing ” sect ion lat er in t his chapt er . LAN s Finally w e com e t o t he LANs and int er- r out er links. LANs pr obably gain host s fast er t han ot her par t s of t he net w or k, so it is pr udent t o allow gr eat er headr oo m here t han

254

elsew her e in t he net w or k . Again, no benefit can be had fr om sum m ar izat ion unless t he net w or k can be split int o a r egional lay out . Unless t here is a good reason not t o do so, DHCP should be used t o assign I P addr esses. A DHCP ser v er can t ak e care of changing I P addr esses and net w or k m asks on a LAN, as w ell as changes in t he DNS r esolver addr esses. Cu st om e r N e t w or k s Cust om er net w or k assignm ent s hav e not been cov er ed her e because t his sect ion concent r at es only on infr ast r uct ur e. How ever , w hen an I S P is running BGP wit hin it s net w ork, t here is no benefit t o be had by eit her aggregat ing or assigning net w orks on a r egional basis. Cust om er s m ov e and w ant t heir connect ions t o m ov e t o differ ent par t s of t he count r y , so t her e is lit t le pur pose in t r y ing t o allocat e addr ess space t o t hem r egionally . Besides, BGP easily can handle huge num ber s of cust om er- assigned net w orks. Nat ur ally , all t hese cust om er net w or k s w ould be aggr egat ed on t he edge of t he I SP’s net w or k . The RI Rs w ill assign block s of addr ess space t o t heir I SP m em bership, and t he I nt er net com m unit y ex pect s all I SPs t o announce t hese block s only t o ot her I SPs. Announcing m or e specific net w or ks fr om a block gener ally is fr ow ned upon unless t her e is a genuine need t o do so w hen aiding sit uat ions such as m ult ihom ing. Pla n Su m m a r y Follow ing t he pr ev ious st r at egy , it should be quit e st r aight for w ar d t o consist ent ly num ber a new I SP back bone. Figur e 5- 15 show s how t his st r at egy could apply t o t he scenar io descr ibed. The I SP can j ust ify a / 23 for t he infr ast r uct ur e including loopback s, so t he addr ess dist r ibut ion look s som et hing lik e t hat in Figur e 5- 15. All t hr ee r egist r ies m ake a m inim um allocat ion of a / 20 at t he t im e of t his w r it ing, so t his ex am ple assum es t hat t he I SP w ould r eceiv e a / 20 w it h t he pr ev iously docum ent ed plan. Fig u r e 5- 1 5 . Addr e ss- D e p loy m e n t Pla n

Pla n n in g f or Fu t u r e Gr ow t h When t he addr ess block in Figur e 5- 15 has been used up ( at t he t im e of t his w rit ing, w hen t he 80 per cent ut ilizat ion level has been r eached) , t he I SP should apply for addit ional addr ess space. I f good r ecor ds hav e been k ept and t he docum ent at ion has been subm it t ed t o t he RI R, obt aining t he nex t block should be r elat iv ely st r aight for w ar d. The RI Rs gener ally at t em pt t o allocat e enough addr ess space t hat

255

t he I SP m akes a year ly r equest t o r eceive r esour ces. For exam ple, if t he I SP has consum ed r esour ces fast er t han docum ent ed in t he r ollout plan, t he I SP could m ake a lar ger r equest t he second t im e r ound and r eceiv e t he lar ger allocat ion. I n t he cur r ent exam ple, it is assum ed t hat t he second allocat ion m ade is t he sam e size as t he or iginal allocat ion. The RI Rs quit e oft en at t em pt t o allocat e cont iguous address spa ce—it ’s not guar ant eed, but t hey usually at t em pt t o do t his. The m ain m ot ivat ion is t o encour age CI DR- izat ion on behalf of t he I SP r at her t han t o m ake life conv enient . Wit h t he second addr ess block being cont iguous, as show n in Figur e 5- 16, t he I SP sim ply can ex t end t he loopback addr ess r ange for t he ex t r a equipm ent being inst alled and can k eep all t he addr esses cont iguous. The infr ast r uct ur e can be ex panded int o t he new block as show n, or t he I SP can com plet ely r enum ber t he infr ast r uct ur e int o t he new addr ess block ( it is r elat ively easy t o r enum ber I SP infr ast r uct ur e because t he v ast m aj or it y of it is point - t o- point links bet w een t he r out er s and because t he secondar y address support m akes t his j ob very sim ple) . Cust om er assignm ent s also can com m ence fr om t he t op end of t he block . Fig u r e 5- 1 6 . Addr e ss- D e p loy m e n t Pla n Af t e r Se con d Alloca t ion

When t he 80 per cent ut ilizat ion has been r eached, t he nex t applicat ion can be m ade, and so t he cy cle goes. I t m ight be quit e lik ely t hat t hir d and fut ur e assignm ent s w ould not be cont iguous, so t he I SP should not r ely on ex panding t he / 19 out int o a / 18. I f it happens, consider it good for t une and t ak e it as encour agem ent t o announce only a / 18 t o t he I nt er net !

Address Spa ce for Cust om ers Cust om er s ar e assigned addr ess space by t he I SP accor ding t o t he policies set out by t he RI R of w hich t he I SP is a m em ber . As an appr ox im at e guide, I SPs ar e ex pect ed t o follow t he sam e pr ocedur es w it h t heir cust om er s as t hey hav e follow ed w it h t he RI R in obt aining addr ess space. So, w hen t he I SP m ak es fir st t echnical cont act w it h t he cust om er , an assessm ent of addr ess space r equir em ent s is m ade accor ding t o need. Need should not be confused w it h want . St aff m em bers from m any or ganizat ions w ho hav e been br ought up in t he classful I Pv 4 w or ld st ill being t aught by m any academ ic inst it ut ions m ight believ e t hat an Et her net can be addr essed w it h only a / 24 net w or k ( t he for m er Class C) . I n m any docum ent ed inst ances, oper at ions st aff of an or ganizat ion hav e num ber ed a back bone using / 24s for Et her net s and point - t o- point WAN links and becom e ver y upset w hen t heir request s are t urned dow n by I SPs. I n our ex per ience, sev er al pot ent ial cust om er s hav e cancelled ser v ice because of r efusal t o m eet t he dem ands for addr ess space. The best sy st em for assigning addr ess space t o cust om er s is t o fir st est ablish w hat range of addr ess space cust om er s ar e lik ely t o need. Will it be any t hing fr om / 24 t o

256

/ 28, or / 18 t o / 26, or / 18 t o / 28, for exam ple? Aft er t he m inim um and t he m axim um hav e been est ablished, cr eat ing a m at r ix of t he differ ent sizes and assigning fr om t he m at r ix is pr obably t he m ost efficient w ay because it m axim izes ut ilizat ion. Ther e ar e no har d- and- fast rules—t he t opic has been one of m uch discussion on m any I SP m ailing list s. A sim ple exam ple m ight be som et hing like t he follow ing: • • • • • • • • •

Cust om er Cust om er Cust om er Cust om er Cust om er Cust om er Cust om er Cust om er Cust om er

1 2 3 4 5 6 7 8 9

needs needs needs needs needs needs needs needs needs

a a a a a a a a a

/ 28. / 26. / 24. / 25. / 27. / 24. / 23. / 28. / 26.

Assigning t hese linear ly w on’t w or k because addr esses need t o be assigned accor ding t o bit boundar ies. The schem a show n in Figur e 5- 17 could be one appr oach t o t ak e. The I SP has decided t hat t he lar gest cust om er lik ely t o connect t o t he net w ork w ill require a / 23 of address space. So t he I SP part it ions t he block int o / 23 chunk s and t hen fills each chunk w it h t he addr ess r equir em ent s of t he cust om er s as t hey com e in. Not ice t he at t ent ion t o bit boundar ies: A com m on m ist ak e is t o assign addr ess space acr oss bit boundar ies, m ak ing t he cust om er st at ic r out e and iBGP set up on t he aggr egat ion r out er s unj ust ifiably har der . Fig u r e 5- 1 7 . Ex a m p le of Cu st om e r Assig n m e n t s

Som e I SPs hav e w r it t en scr ipt s t o do t his assignm ent aut om at ically fr om t heir addr ess block s as par t of t heir cust om er sign- on and pr ov isioning sy st em s. Ot her s sim ply m aint ain a lar ge spr eadsheet kept up- t o- dat e w it h each assignm ent m ade. Som e of t he scr ipt s ar e av ailable on t he I nt er net . A com m only used ex am ple is t r ee, av ailable by anony m ous FTP fr om I SI . [ 1]

Applying t o t he RI Rs or Upst re a m I SP for Addr e sse s The docum ent at ion and applicat ion pr ocess for obt aining addr ess space is differ ent for each of t he t hr ee RI Rs, so it is not cov er ed in t his t ex t . How ev er , t he act ual pr ocess should be r elat ively st r aight for w ar d. The first st ep in m aking any applicat ion for addr ess space is t o appr oach t he upst r eam I SP. Many ar e m em ber s of one of t he t hr ee RI Rs and so can act on behalf

257

of t he RI R t o assign addr ess space. Som e m ight char ge a sm all fee for t he ser v ice. When t he upst r eam I SP cannot offer t his ser vice or r efuses t o offer t his ser vice, t he nex t st ep is t o appr oach t he RI R ser v ing t he r egion w her e t he new business is based. The RI R fir st ex pect s t he new I SP t o sign up as a m em ber . Mem ber ship does not guar ant ee any allocat ion of r esour ces, but it is sim ply t he m echanism necessary t o set up cont act infor m at ion, billing det ails, and so for t h. The applicat ion pr ocess com es next . The for m s t o m ake t he applicat ion for r esour ces ar e kept on t he RI R’s Web sit e; j oining as a m em ber will help clarify exa ct ly w hat has t o be done nex t . I t is good pr act ice t o allow a few w eek s bet w een com plet ing applicat ion for m em ber ship and act ually ex pect ing t o r eceiv e r esour ces. Set t ing up m em ber ship oft en r equir es v er ificat ion of business st at us, est ablishm ent of billing pr ocesses, and so on; t his all can t ak e t im e. The applicat ion for r esour ces also can t ak e t im e because oft en plans t hat seem obv ious t o t he designer m ight not be so obv ious t o a t hir d par t y t hat has t o m ak e addr ess- allocat ion decisions based on t hem . I f t he det ail list ed pr ev iously is follow ed, t her e should be sufficient infor m at ion av ailable for t he r egist r y host m ast er t eam t o under st and t he r equir em ent s of t he back bone. I SPs should be pr epar ed t o disclose net w or k plans ( RI Rs sign a nondisclosure agreem e nt w it h m em ber ship) and designs, as w ell as pur chase or der s or r eceipt s for t he equipm ent r equir ing addr ess space. All of t his infor m at ion helps t he v er ificat ion pr ocess and ensur es t im ely allocat ion of r esour ces.

Conclusion This sect ion gav e an ex am ple of how t o w or k out an addr essing schem e for a dev eloping I SP net w or k . This is int ended t o help t he gr ow ing I SP business w or k out how t o apply addr essing t o it s net w or k and how t o allocat e assigned addr ess space t o it s infr ast r uct ur e. I ndeed, follow ing t hese pr ocesses should aid in t he applicat ion pr ocess for addr ess space fr om t he RI Rs.

I n t e r ior Rou t in g One of t he har dest quest ions t hat a v endor is posed at t he t im e of an RFP is w hich I GP t he new I SP net w or k should choose. As m ent ioned pr ev iously , t he choice should be m ade for t echnical r easons, not for any ot her s. Fur t her m or e, t he choice should be m ade by t he I SP because t he engineer ing and oper at ions st aff m ust r un t he net w or k aft er t he equipm ent has been inst alled. For all int ent s and pur poses, t her e is lit t le t o choose am ong EI GRP, OSPF, and I S- I S. Because EI GRP is a Cisco- propriet ary I GP, it s deploy m ent in t he I SP w or ld is som ew hat less com m on t han t he deploy m ent of OSPF and I S- I S. OSPF and I S- I S have a lot in com m on for m ost net work im plem ent at ions t oday , and t he choice bet w een t he t w o r eally com es dow n t o w hich I GP has t he m ost oper at ional ex per ience w it h t he st aff em ploy ed. I t ’s quit e int er est ing t o obser ve t he spr ead of I S - I S ar ound t he I nt er net . As engineer s leave t he est ablished I SPs and m ove t o new jobs, t hey im plem ent I S - I S as t he I GP of choice sim ply because t hey ar e m ost fam iliar w it h it . A sim ilar t hing can be seen for OSPF, r esult ing in int er est ing debat es over w hich I GP is bet t er .

258

The I SP I GP Versus BGP Model Chapt er 3 cov er ed t he det ails of set t ing up t he configur at ion for t he t hr ee popular I GPs. I f t hose configur at ion guidelines ar e follow ed, t her e is v er y lit t le else t o say about I GP choice and configur at ion. The key t o a scalable net w or k is sim ple: Keep t he I GP sm all. BGP is designed t o carry a large num ber of prefixes around t he I SP back bone. To t hat end, iBGP is consider ed by som e net w or k engineer s t o be t he int er ior r out ing pr ot ocol for t heir net w or k . The act ual m odel used is quit e sim ple and is best displayed in Figur e 5- 18: Fig u r e 5- 1 8 . Rou t in g Pr ot ocol Re la t ion sh ips

• •

•

The I GP car r ies infr ast r uct ur e pr efix es, t y pically back bone point - t o- point links and r out er loopback int er face addr esses. iBGP car r ies cust om er- assigned addr ess block s, access net w or k addr ess pools, and any ot her pr efix es t hat do not need t o be car r ied in t he I GP. iBGP also is used t o car r y som e or all of t he I nt er net Rout e Table ( depending on t he I SPs int er nal policy ) . eBGP is used t o car r y pr efix es bet w een I SPs and t o im plem ent r out ing policy bet w een I SPs.

This m odel is ver y differ ent fr om ear lier m odels used in t he infancy of t he I nt er net , in w hich t he I GP car r ied all pr efix es in t he I SP back bone and BGP w as r est r ict ed t o sim ply ex changing pr efix es bet w een differ ent aut onom ous sy st em s. The r elat iv e lack of scalabilit y of I GPs and t he gr eat scalabilit y now av ailable in iBGP t hr ough r out e r eflect or s and confeder at ions m eans t hat iBGP is an excellent t ool for car r ying pr efixes acr oss t he I SP’s backbone. When first faced w it h t his m odel, m any net w ork engineers ar e sk ept ical about r unning iBGP t o all cor ner s of t heir backbone. A com m on m isper cept ion in t he I nt er net com m unit y is t hat subst ant ial r out er s ar e r equir ed befor e BGP can be r un — m any engineer s for get t hat sev er al I SP back bones hav e been built out of not hing m or e t han Cisco 2500 r out er s t hat hav e lim it ed RAM and CPU capabilit ies. I ndeed, t he 2500 shar es m any feat ur es w it h it s pr edecessor , t he I GS, w hich w as used in m any I SP backbones in t he ear ly 1990s. Because iBGP suppor t s r out ing policy and r out e filt er capabilit ies, t her e is no need t o car r y t he full r out ing t able acr oss t he ent ire I SP backbone. I t is quit e com m on for I SPs t o lim it full or large rout ing t able suppor t t o t he cor e of t heir net w or k and sim ply announce t heir dom est ic pr efix es t o t he r est of t heir backbone. A Cisco 2500 rout er, for exam ple, is quit e happy running BGP w it h 10,000 pr efix es; it w ill be quit e slow at pr ocessing updat es, but t hen t her e are unlikely t o be m any updat es inside an I SP’s backbone. Updat es are m ost likely t o

259

be gener at ed by new cust om er s being added t o t he I SP’s net w or k , har dly a persecond or ev en per- m inut e occurrence in all but t he largest of net w orks. A t ypical deploym ent scenar io is t he follow ing: 1. An I SP inst alls all t he loopback addr esses in t he back bone int o t he IGP. The I SP also inst alls all t he point - t o- point link addresses in t he backbone int o t he I GP. 2. All r out er s in t he back bone par t icipat e in t he I GP. Any t hat cannot par t icipat e in t he I GP gener ally hav e st at ic default r out es point ing t o t hose t hat can ( t hese ar e t y pically dum b access- aggr egat ion dev ices) . 3. The I SP set s up iBGP acr oss t he ent ir e I SP back bone using r out e r eflect or s ( our pr efer r ed t echnique) or confeder at ions. iBGP is configur ed t o peer t he r out er s w it h t he loopback int er faces. As t he loopback s are car r ied in I GP, t he r out er s can see each ot her because t her e is an ent r y in t he for w ar ding t able cour t esy of t he I GP. 4. The I SP im plem ent s a policy for iBGP. All dom est ic pr efix es ( t he I SP’s ow n pr efix es not in t he I GP and addr ess space assigned t o it s cust om ers) are t agged w it h a cer t ain com m unit y . 5. The r out e r eflect or s hav e filt er s so t hat t he client s r eceiv e only t he r out es t hat belong t o t his special com m unit y . This w ay , t he I SP ensur es t hat t he cor e r out er s ( t he r out e r eflect or s) ar e t he only ones car r y ing t he full r out ing t able. The r em aining r out er s in t he backbone, t he client s in each clust er , car r y only t he r educed list of pr efix es. 6. The client s use a couple of pr edet er m ined net w or k s as default net w or k s so t hat t hey hav e an ex it pat h t o t he back bone c or e. The t w o special net w or ks could belong t o t he I SP it self or, m ore likely, w ill belong t o t he upst ream I SPs. I f one of t hese net w or k s disappear s, t he second w ill be av ailable as a backup. This t ype of deploym ent m akes for a very scalable net w ork and all ow s t he I SP t o im plem ent BGP on v ir t ually ev er y r out er dev ice w it hin it s ow n back bone. This m et hod is m uch preferred over having an iBGP island in t he m iddle of t he net w ork and eit her point ing st at ic r out es or using r edist r ibut ion fr om ot her r out ing pr ot oc ols at t he edge. The lat t er has pr oven it self r isky and unr eliable in m any sit uat ions.

Sca ling I nt erior Rout ing Prot ocols This sect ion m ost ly is concer ned w it h t he scaling of iBGP. I GP scalabilit y for t he m aj or it y of net w or k s is easy t o achiev e: Keep t he num ber of pr efixes sm all. iBGP is m or e of a challenge w it hout using t w o im por t ant feat ur es, r out e r eflect or s and peer gr oups. Rou t e Re fle ct or s The fir st essent ial feat ur e is t he r out e r eflect or . Rout e r eflect or s ar e par t of t he BGP st andar d and ar e descr ibe d in RFC 2796. I t is gener ally r ecom m ended t hat new I SP net w or k inst allat ions t oday inst all r out e r eflect or s fr om day one. Failing t hat , it is also ver y easy t o m igr at e fr om using a full iBGP m esh t o using r out e r eflect or s.

260

Refer r ing t o Figure 5- 1, t he gener al design pr inciple is t o set up t he cor e r out er s as t he r out e r eflect or s and hav e t he r em aining r out er s in t he PoP configur ed as r out er eflect or client s. Recall fr om Chapt er 3 t hat it is quit e com m on for I SPs t o hav e t w o r out e- r eflect or clust er s per PoP, w it h each r out er in t he PoP being a client of t w o r out e r eflect or s ( inst ead of set t ing t he clust er I D of t he t w o r out er r eflect or s t o be t he sam e v alue) . This configur at ion goes against alm ost all docum ent ed r ecom m endat ions ov er t he last few y ear s, but it is act ually t he cor r ect w ay of configur ing r out e r eflect or s and alm ost guar ant ees av oidance of rout ing loops. The t w o cor e r out er s in t he PoP should t alk st andar d iBGP w it h each ot her and also should be fully m eshed w it h t he r em aining r out er s t hat ar e not clust er client s in t he PoP and t he r est of t he net w or k . A com m on net w or k design is t o fully m esh t he core and bor der r out er s in a net w or k, w it h t he r em aining r out er s in t he dist r ibut ion and access layer s being client s ( as in t he exam ple in Figur e 5- 19) . This scales v er y w ell for all but t he largest net works — and, in t hose cases, a r out e- reflect or hierarchy pr ov ides t he solut ion for dist r ibut ion of pr efix es acr oss t he back bone. Fig u r e 5- 1 9 . Rou t e - Re fle ct or La y ou t

261

BGP Pe e r Gr ou ps The second essent ial feat ur e is t he BGP peer gr oup. This gr oups t he BGP peer s w it h t he sam e out bound policy int o one gr oup. The nor m al sit uat ion w it hout using peer groups is for t he r out er t o calculat e t he updat e t o be sent t o each neighbor indiv idually . For a low num ber of peer s, t her e is pr obably not m uch im pact on t he r out er CPU, but as t he num ber of neighbor s incr eases, so does t he bur den on t he CPU—for ex am ple, 50 neighbor s m eans t hat 50 updat es hav e t o be com put ed and sent individually. When neighbor s ar e gr ouped accor ding t o out bound policy w it hin a peer gr oup, t he r out er calculat es t he updat e once per peer gr oup and t hen sends t he updat e t o all m em ber s w it hin t he gr oup. When peer gr oups fir st appear ed in I OS Soft w ar e, t her e w as a v er y m ar k ed incr ease in iBGP neighbor est ablishm ent and iBGP conv er gence r at e on t he Mot or ola CPU- based r out er s. And w it h r ecent enhancem ent s t o I OS Soft w ar e suppor t for peer gr oups, it is now consider ed by m any engineer s st andar d pr act ice t o use peer gr oups w henever configur ing iBGP. The ot her adv ant age of peer gr oups is a cosm et ic one: They im pr ov e t he r eadabilit y of t he r out er configur at ion and m ake it m uch less pr one t o t he int r oduct ion of errors by t he oper at or or t he net w or k engineer ing st aff. This is a consider able adv ant age, especially in t he busy oper at ional env ir onm ent . I t is also quit e com m on for I SPs t o cr eat e cent r alized t em plat es w it h t heir peer- gr oup configur at ion. Updat es t o t he peer gr oups ar e done cent r ally , t hus ensur ing t hat all r out er s r eceiv e consist ent changes t o any r equir ed iBGP configur at ion. Com par e t his t o m aking changes t o ever y iBGP session for every rout er in t he net work. A 50- r out er back bone needs one line changed in a cent r alized peer gr oup and needs t his change t o be applied t o 50 r out er s; com par e t his t o 2450 changes r equir ed if t he I SP w as not using peer gr oups. I t is easy t o see w hich is t he pr efer r ed opt ion. N OTE Peer gr oups also can be used for eBGP neighbor s. This is less com m on because I SPs usually hav e v er y specific policies t ow ar d each of t he ex t er nal peer s, but in pr inciple, peer gr oups can be used her e, t oo.

Ex t e r ior Rou t in g Connect ions fr om one I SP t o anot her t ak e t w o for m s: The I SP eit her point s a st at ic default r out e t o t he neighbor or uses BGP. The use of a st at ic default r out e is per haps t he m ost com m on w ay t hat t he edge of t he I nt er net connect s t o t he nex t lay er . Ent er pr ises connect ing t o t he I nt er net oft en hav e lit t le m or e r equir em ent t han a st at ic default r out e t o an I SP; t he I SP t ak es car e of r out ing t he ent er pr ise’s t r affic and announcing it s pr efix es t o t he r est of t he I nt er net . This is also t r ue for sm aller I SPs: They don’t r equir e a sophist icat ed connect ion and sim ply r ely on t heir upst r eam I SP for all t heir connect iv it y needs. When an I SP ( and, in som e cases, an ent er pr ise) hav e m or e sophist icat ed needs t han a st at ic default r out e can offer t hem , t hey can inv est igat e t he use of BGP as t he ext er ior r out ing pr ot ocol. BGP allow s t he net w or k m or e t han one ex it point by

262

ex changing r out ing infor m at ion w it h t he upst r eam pr ov ider s t hat t he net w or k connect s t o. This t hen allow s t he net w or k t o be m ult ihom ed.

AS N um ber Mult ihom ing it self is cover ed in t he next sect ion—a huge num ber of opt ions are available , and t hat sect ion cov er s som e of t he configur at ion concept s in m or e det ail. Using BGP t o m ult ihom e r equir es t he I SP t o acquir e an AS num ber ( ASN) . This can be obt ained in t he sam e w ay in w hich I Pv 4 addr ess space can be obt ained. The t hr ee r egional r egist r ies pr ov ide ASNs t o t heir m em ber ship upon applicat ion. The basic r equir em ent s ar e t hat t he or ganizat ion or I SP hav e at least t w o separ at e connect ions t o t he I nt er net , eit her t o at least t w o different I SPs or t o an I SP and an I nt er net eXchange Point ( I XP) . The applicat ion should list t he ASNs t hat w ill be connect ed, allow ing t he r egist r ies t o v er ify t he int ended connect ion fr om t he I nt er net Rout e Table. I SPs also can apply for ASNs on behalf of t heir cust om er s —t he RI R Web pages should be check ed for cur r ent infor m at ion. At t he t im e of t his w r it ing, ar ound 30 per cent of t he av ailable ASN space has been dist r ibut ed t o or ganizat ions connect ed t o t he I nt er net —and ar ound 12,000 of t hose ASNs act ually ar e being announced t o t he r out ing t able. The last t w o y ear s saw a rem arkable grow t h in m ult ihom ing as t he edge of t he I nt ernet required m ore reliable connect ions t o t he cor e —a sur e sign t hat t he I nt er net is becom ing business- cr it ical for m any or ganizat ions. I t is also ar gued by sev er al t hat t he “ m iddle r eaches” of t he I nt er net hav e show n a m ar k ed dow nt ur n in ser v ice qualit y and av ailabilit y , for cing or ganizat ions pr eviously cont ent w it h a single - hom ed st r at egy t o look for r edundancy by connect ing t o anot her I SP. To cat er for t he gr ow t h in ASN usage, an ex t ension t o t he AS space has been pr oposed: r eplacing t he 2- by t e ASN w it h a 4- byt e ASN. A m igrat ion st rat egy hasn’t been w or k ed out fully y et , but sev er al pr oposals indicat e t hat it shouldn’t be t oo difficult w hen t he t im e com es for t he r egist r ies t o assign 4- by t e ASNs.

Sca la ble Ex t e r na l Pe e r ing Most of t he BGP scalabilit y effor t s hav e been applied w it hin t he I SP net w or k . eBGP is a point - t o- point pr ocess bet w een t w o I SPs, so scaling t he pr ot ocol her e is less com plex . How ev er , new com er s t o t he indust r y need t o consider t w o com m only used and im por t ant t echniques for scaling ex t er nal BGP peer ing bet w een I SPs. Rou t e Re f r e sh Ca p a b ilit y The rout e refresh capabilit y is a long- aw ait ed t echnique in t he oper at or indust r y t hat let s eBGP ( and, indeed, iBGP) neighbors apply a new polic y t o t he BGP session w it hout t ear ing dow n t he w hole peer ing. For m any y ear s, t he only t echnique av ailable w hen an I SP w ant ed t o apply a new policy t o a peer ing session w it h a neighbor w as t o shut dow n t he BGP peer ing and t hen bring it up again. Consider w hat happens in t he I nt er net w hen t his is done:

263

•

•

•

Shut t ing dow n a peer ing m eans t hat all t he pr efixes r eceived fr om t he neighboring I SP m ust be rem oved from t he BGP t able and t he forw arding t able on t he local r out er . For t he full r out ing t able, t his t akes CPU t im e t o achiev e. The pr efix es t hat hav e been r em ov ed fr om t he local r out er t hen hav e t o be r em oved fr om all t he ot her r out er s in t he I SP backbone t aking par t in t he full iBGP m esh ( subj ect , of cour se, t o any policy applied bet w een iBGP neighbor s) . This t akes CPU load on all t he rout ers in t he backbone. I f t he I SP w as pr ov iding t r ansit t o t he I nt er net , t hese changes ar e pr opagat ed t o t he I SP’s t r ansit cust om er s. This m eans m or e changes and m or e im pact on t heir r out er CPUs. And t his change pr opagat es t hr ough all t he neighbor ing ASNs t hat r elied on t he r out ing infor m at ion r eceiv ed fr om t his peer ing t hat w as t or n dow n.

This ex am ple sim ply show s w hat happens t o pr efix es r eceiv ed fr om t he peer . The r ipple effect is quit e pr onounced! The sam e t hing happens in t he opposit e dir ect ion: •

•

Shut t ing dow n t he peer ing m eans t hat t he I SP’s pr efix es t hat w er e announced t o t he peer ar e w it hdr aw n fr om t hat peer . The local I SP effect iv ely disappear s fr om t he neighbor , r elying on alt er nat ive pat hs, if t hey exist , or com plet ely disappear ing fr om t he I nt er net if t hey don’t . The neighbor ing I SP w it hdr aw s t he pr efixes fr om it s BGP and for w ar ding t ables and t hen w it hdr aw s t hem fr om all announcem ent s m ade t o neighbor s.

Com bining t he t w o effect s, sim ply shut t ing dow n t he BGP peer ing cr eat es a significant r ipple effect t hr ough t he I nt er net on bot h sides of t he peer ing in quest ion. I f t her e w er e m ult iple pat hs fr om t he I SP t o t he I nt er net , m any r out er s have t o r ecalculat e t he best pat h, affect ing t r affic flow and appar ent I nt er net “ per for m anc e” t o t he end user. As discussed in t he follow ing sect ion, m any I SPs also apply rout e flap dam ping. I f a r out e flaps, t he pr efix possibly w ill be dam ped ( not r eannounced by t he peer ev en t hough it r eappear s in t he BGP t able) . To sim ply apply som e new polic y on t he eBGP peer ing, t he I SP has caused lot s of pot ent ial pr oblem s t o hit t he I nt er net : • • • •

A r out ing pr efix w it hdr aw al sur ge occur s t hr ough t he local back bone and peers. An I SP’s pr efix es ar e w it hdr aw n fr om t he I nt er net . The pot ent ial of flap dam ping ex ist s. Cust om er s see t r affic flow being int er r upt ed for m inut es—in t he w or st case, flow st ops alt oget her

The sim ple solut ion t o t his pr oblem is a new capabilit y int r oduced int o BGP know n as r out e r efr esh. This capabilit y allow s peer ing r out er s t o r eset policy bet w een each ot her w it hout hav ing t o t ear dow n t he session. ( Cisco had an ear lier v er sion of t his called soft r econfigur at ion, w hich w as im plem ent ed only on t he local r out er and r equir ed pot ent ially double t he am ount of m em or y t o st or e t he pr efixes r eceived fr om a neighbor .)

264

When t he I SP oper at or w ant s t o change t he policy , t he new policy is applied t o t he r out er configur at ion and t hen t he r out e r efr esh r equest is sent t o t he neighbor ing r out er . The neighbor ing r out er t hen sim ply sends it s ent ir e announcem ent back t o it s peer , allow ing t he peer t o r ebuild it s BGP t able using t he new policy . This is such a sim ple ex t ension t o t he BGP funct ionalit y , but t her e is now gr eat er possibilit y for st abilit y in t he I nt er net Rout e Table. We w ould like t o see t he har d cle aring of BGP m ade as difficult t o do as r eboot ing t he r out er ( going t hr ough a Q&A session on t he r out er t o confir m t hat t he oper at or is absolut ely sur e) , w it h t he explicit aim of encour aging m or e I SPs t o r em em ber t o use t he r out e r efr esh capabilit y . As cov ered in t he pr ev ious chapt er , t he com m and t o do t his is cle a r ip b g p neighbor in| out , hopefully som et hing t hat can be “ im plant ed” in all I SP engineer s befor e t hey t ouch any oper at ional net w or k s. BGP Fla p D a m p in g A consequence of t he chur ning in t he I nt er net caused by indiscr im inat e clear ing of BGP sessions, by unst able infr ast r uct ur e, and by ant isocial configur at ion pr act ices is t he r equest for BGP flap dam ping t o be av ailable in I OS Soft w ar e. Pr efix es t hat appear and disappear fr om t he I nt er net Rout e Table cause a CPU hit on t he rout er: Wit hdr aw ing a pr efix m eans t hat it has t o be w it hdr aw n fr om t he BGP and for w ar ding t ables; t he pr efix also has t o be w it hdr aw n fr om neighbor s and so on t hr oughout all t he BGP speak er s at t ached t o t his net w or k . The m echanics of flap dam ping w er e discussed in t he pr ev ious chapt er , and t hat should be st udied in case t he m echanism is not clearly underst ood. Every flap at t r act s a fix ed penalt y ; t he penalt y is decay ed ex ponent ially . Lim it s ar e set for w hen t he prefix is suppressed and w hen it can be m ade av ailable t o t he I nt er net again. The pur pose of flap dam ping is t o penalize t hose pr efix es t hat w on’t r em ain st able. The m or e t hey flap, t he m or e likely t hey ar e t o disappear fr om t he I nt er net Rout e Table. I SPs t hat im plem ent t he Cisco default s w ill see t heir pr efixes disappear fr om t he I nt er net for a m axim um of 60 m inut es. This is a long t im e t o explain aw ay t o cust om er s, so t he incent iv e is t her e t o ensur e t hat st abilit y is guar ant eed w hen m ak ing announcem ent s t o t he I nt er net . Flap dam ping usually is im plem ent ed on t he I SP’s peer ing and bor der r out er s, basically all r out er s w it h eBGP sessions. I OS Soft w ar e does not suppor t flap dam ping for iBGP, and, in any case, t he concept does not m ak e sense. I f I SPs ar e experiencing rout e flaps caused by t he cor e net w or k , t hey need t o ex am ine t heir net w or k design and oper at ional pr act ices r at her t han t r y ing t o use an I OS Soft w ar e feat ure t o fix self - m ade problem s. The eBGP- speak ing r out er s w ill r equir e slight ly m or e CPU ( depending on t he num ber of peer s and t he num ber of pr efix es r eceiv ed) , but t he big adv ant age is t hat t he r est of t he net w or k w ill see a r educt ion in CPU r equir em ent because t he w or st flaps ar e k ept out of t he cor e net w or k . I n t he m id- 1990s, w hen r out e flap dam ping fir st w as im plement ed in I OS Soft w ar e, a t y pical I SP back bone in our ex per ience saw som et hing lik e 15 per cent r educt ion in CPU r equir em ent for t he Mot or ola CPU–based r out er s deployed. The bor der r out er had t he biggest , fast est CPU av ailable at t he t im e, and t his t ook t he hit from t he inst abilit ies caused by t he pr oblem s t hat som e of t he U.S. back bones w er e ex per iencing at t he t im e. The r est of t he local back bone w as v er y st able, w it h r out e fluct uat ions r educing not iceably . Most im por t ant , t he lev el of cust om er com plaint s

265

about st r ange r out ing and t r affic disappear ing r educed as w ell, alt hough som e educat ion w as necessar y about w hat r out e flap dam ping w ould act ually m ean. ( An unr eachable dest inat ion could be caused by a net w or k being disconnect ed, or t hat net w or k could be fla p- dam ped. Or it could be caused by t he locally or iginat ed pr efix being suppr essed by som e I SP som ew her e along t he pat h because of flap dam ping. These par t icular point s need t o be r em em ber ed by oper at or s, especially cust om er suppor t cent er s and net w or k ope r at ions cent er s t hat deal w it h cust om er com plaint s about unr eachabilit ies in t he I nt er net .) An I SP can m ak e t w o choices w hen im plem ent ing flap dam ping. The fir st is t o use t he I OS Soft w ar e default v alues; t hese ar e consider ed quit e sev er e by som e m em ber s of t he com m unit y , suppr essing a pr efix aft er it has flapped t hr ee t im es in t he space of a few m inut es. The second opt ion is t o use t he RI PE- 229 [ 2] v alues, which im plem e nt pr ogr essiv e flap dam ping depending on pr efix lengt h. Sm aller pr efix es ar e hit har der t han lar ger pr efix es, based on oper at ional ex per ience gained t hr ough t he last few y ear s. The RI PE- 229 v alues also allow t hr ee flaps befor e t he pr efix is suppr essed, allow ing a lit t le m or e leew ay for soft w ar e upgr ades and ot her m aint enance act ivit ies in t he I SP net w or k.

M u lt ih om in g I n m ult ihom ing, a net w or k has m or e t han one connect ion t o anot her independent net w ork. This can t ake t he form of one I SP having m ore t han one connect ion t o anot her I SP or hav ing connect ions t o m or e t han one ot her I SP. For t he net w or k engineer , configur ing m ult ihom ing w it h BGP poses a challenge t hat , t o m any , seem s j ust t oo har d. Whet her it is because BGP has been t out ed as being difficult or bec ause so m any configur at ion knobs now ar e available, m ult ihom ing has becom e som et hing of a feared subj ect for m any I SPs. Wr it ing a definit ive t ext on m ult ihom ing is not st r aight for w ar d —t here are so m any possible sit uat ions and so m any differ ent t ypes of int er pr ov ider r elat ionships t hat cov er ing t hem all is not r eally feasible. I nst ead, t his sect ion guides y ou in t he basics of est ablishing a BGP session w it h ot her net w or ks and ot her I SPs, w it h t he pr inciple t hat once t he basics hav e been est ablished and underst ood, enhancem ent of t he configur at ion t o deal w it h m or e par t icular or com plex scenar ios becom es som ew hat easier . When faced w it h a com plex m ult ihom ing pr oblem , t oo m any engineer s t r y t o deal w it h it w it hout r egar d t o t he basic needs. The pr inciple of KI S S ( “ Keep I t Sim ple, St upid” ) has been m ent ioned befor e and is ver y applicable t o m ult ihom ing. A few configur at ion scenar ios w ill be cov er ed. These for m t he basis for v ir t ually all m ult ihom ing r equir em ent s bet w een I SPs. • • •

M u lt ih om in g t o t h e sa m e I SP— This is applicable t o t he ent er pr ise cust om er and t he sm aller I SP t hat has j ust st ar t ed in business. They hav e one upst r eam cir cuit and now r equir e a second one. M u lt ih om in g t o a d if f e r e n t I SP— As w it h t he pr ev ious case, t his can apply t o bot h ent er pr ise and I SP cust om er s. I t r epr esent s t he nex t lev el in com plex it y of being par t of t he I nt er net . U sin g com m u n it ie s f or m u lt ih om in g a n d t r a f f ic e n g in e e r in g— Aft er t he I SP has becom e com for t able w it h it s m ult ihom ing configur at ions, using com m unit ies w ill give it gr eat er cont r ol over engineer ing t r affic flow s in and out of t he net w or k.

266

•

M u lt ih om in g f or con t e n t p r ov id e r s — This is a unique sit uat ion in w hich out bound t r affic flow s ar e som ew hat lar ger t han inbound flow s. This r equir es car eful consider at ion because w hat m ight be good pr act ice in t he pr evious ex am ples w ill not be sufficient in t his case.

Ba sics Befor e pushing ahead and inst alling a new cir cuit , som e of t he basics of m ult ihom ing m ust be under st ood and r em em ber ed: •

•

•

•

The new cir cuit should go t o a differ ent PoP of t he upst r eam I SP. I f it goes t o t he sam e upst r eam PoP, sit e r edundancy of t he upst r eam is lost . Com plet e PoP out ages can occur because of elect r ical, oper at ional, or env ir onm ent al pr oblem s. I f t he upst r eam I SP has only one PoP, r equest or insist t hat t he new connect ion be t er m inat ed in a differ ent r out er , one w it h a differ ent pow er sour ce and an alt er nat iv e pat h out of t he net w or k fr om t hat of t he or iginal connect ion. This isn’t as good as t he pr ev ious r equir em ent , but it ensur es t hat m aint enance of t he upst r eam r out er doesn’t defeat t he point of t he cust om er m ult ihom ing. The new connect ion should be t er m inat ed in a differ ent local r out er . Ter m inat ing a m ult ihom ed connect ion on t he sam e local r out er cr eat es a single point of failur e in t hat r out er—and equip m ent failur es ( w het her soft w ar e, har dw ar e, or oper at ional) ar e j ust as com m on as infr ast r uct ur e failur es ( for ex am ple, in cir cuit s) . I f t he net w or k being m ult ihom ed is pr esent in m or e t han one locat ion, t her e is ext r a benefit t o be gained by inst alling t he new connect ion in a differ ent locat ion: sit e r edundancy . I f possible, pur chase t he new cir cuit fr om a differ ent cir cuit pr ov ider . We oft en hav e seen net w or k s being m ult ihom ed at t he I P lev el, but t he cir cuit pr ov iding t he back up goes t hr ough t he sam e t elc o equipm ent from t he cabinet all t he w ay t o t he ex change and t o t he upst r eam I SP. Som e t elcos w ill be happy t o guar ant ee div er se r out ing ( as a paid ser v ice) , but if a choice of car r ier s is available, using t w o m akes a lot m or e sense. One car r ier can have a com plet e out age of t he net w or k, r esult ing in only one of t he local net w or k’s link s being affect ed.

Lik ew ise, t he r out ing configur at ion has som e basic pr inciples t hat need t o be under st ood and r em em ber ed: • • •

BGP needs t o be used. A lot of m ult ihom ing t echnically can be achiev ed by using st at ic r out es, but t hat t echnique is v er y inflex ible, and failov er is oft en not as effect ive as w it h BGP. Dow nloading t he full r out ing t able is not r equir ed. This is a ver y com m on m isunder st anding in t he indust r y t oday . Cisco IOS Soft w ar e has sufficient t ools av ailable, but t heir applicat ion needs t o be under st ood pr oper ly . Pr ov ided t hat t he follow ing t hr ee “ r ules” ar e adher ed t o, t her e should be lit t le difficult y under st anding t he basics of m ult ihom ing: - I P prefix list s are used for filt er ing pr efix es ( don’t use access list s because t hey ar e har der t o under st and, ar e slow er t o im plem ent in t he r out er , and hav e been t he sour ce of m any configur at ion er r or s in t he past ) .

267

- AS pat h access list s ( or filt er list s) are used t o filt er AS pat hs. AS pat h filt ering is a very useful t ool for m any m ult ihom ing sit uat ions encount ered. - Rout e m aps ar e used t o im plem ent policy . ( Rout e m aps also can be used t o apply filt er ing, but t his is r ecom m ended only aft er y ou fully under st and how t o use BGP in t he cont ex t discussed her e.) •

Sever al BGP at t r ibut es can and should be used t o facilit at e pr oper load balancing w hen m ult ihom ing: - Local pr efer ence is applied t o r out ing announcem ent s com ing fr om t he neighbor ing AS, t o allow t he local AS t o influence t he out bound pat h t hat t r affic w ill t ak e. The default local pr efer ence is 100; t he highest local pr efer ence w ins. - MED ( m et r ic) is applied t o r out ing announcem ent s sent t o t he neighbor ing AS, t o allow t he local AS t o influence t he inbound pat h t hat t r affic t o t h e net w or k w ill t ake. The low est m et r ic w ins. I f t he m et r ic is not set , Cisco I OS Soft w ar e assum es t hat t he m et r ic is 0. The MED at t r ibut e is used only bet w een locally connect ed aut onom ous sy st em s. - AS pat h prepend is used in a sim ilar way t o MED, but it has global scope. Apply ing an AS pat h pr epend t o out bound r out ing announcem ent s sent t o t he neighbor ing AS m ak es t he neighbor ing AS see t he pat h t o t he local AS as being m or e AS hops aw ay and, t her efor e, less pr efer r ed t han any alt er nat ive pat h t hat has few er AS hops in it . The AS ( or aut onom ous syst em s) pr epended should be t he local AS; ot her w ise, unpr edict able r esult s could occur. ( BGP loop det ect ion m ight com e int o play in det erm ining opt im um or suit able pat hs.) - Com m unit ies ar e used ext ensively for t r ansm it t ing policy opt ions bet w een I SPs. Com m unit y v alues need agr eem ent bet w een neighbor ing or par t icipat ing I SPs, as t he exam ples lat er in t his chapt er show .

•

•

Announce t he RI R- allocat ed net w or k block . I t is v er y im por t ant t hat t he net w or k block ( or block s) t hat hav e been allocat ed by t he RI R, or upst r eam I SP ar e announced in one piece. For ex am ple, if t he net w or k 221.10.0.0/ 19 has been allocat ed t o t he local I SP, t hat net w or k m ust be announced as a / 19. The r egist r ies don’t m ake t his a r equir em ent , but not doing t his w ill incr ease t he chances t hat t he pr efix w ill be dam ped or filt er ed by I SPs elsew her e in t he I nt er net . Be pr udent w it h net w or k- block subpr efix es t hat hav e t o be announced. The only r equir em ent t o leak subpr efix es t o t he I nt er net Rout e Table is for t r affic engineer ing. And t r affic engineer ing is a delicat e t ask , as t he ex am ples lat er in t his sect ion show . The unaccept able face of t r affic engineer ing has r esult ed in a r out ing t able of ar ound 108,000 pr efixes, of w hich a lar ge pr opor t ion could be excluded if t he origin net works had been m ore careful wit h t heir announcem ent s. ( Announcing a / 19 net w or k block as 32 / 24 net w or ks is like t aking a m allet t o crack a very sm all nut —w hat is m ore, it rarely w orks as w ell as hoped.)

268

Bear ing t hese point s in m ind, t he r est of t he sect ion exam ines m ult ihom ing scenar ios and pr ovides som e r out er configur at ion ideas.

M ult ihom ing Opt ions Befor e look ing at det ailed ex am ples, it ’s w or t h consider ing t he differ ent t y pes of m ult ihom ing possible and t he r out ing pr ot ocols t hat should be considered for t hem . St u b N e t w or k This is w her e a net w or k is connect ed t o t he I SP t hr ough a single connect ion or t hr ough m ult iple par allel cir cuit s bet w een t he cust om er r out er and t he I SP r out er . I n t his case, t here is no need for BGP—BGP can be used, if desired, but it is sim ply unnecessar y, and oft en a st ub net w or k is easier t o configur e and m anage using st at ic r out ing pr ot ocols and appr opr iat e net w or k equipm ent ( if av ailable or applicable) . The exam ple in Figur e 5- 20 show s a net w or k connect ed w it h t w o par allel cir cuit s t o t he upst r eam I SP. This ex am ple does not r equir e BGP ( alt hough BGP could be used, if desir ed) . A public AS num ber w ill not be assigned t o t he cust om er net w or k by t he r egist r ies for t his pur pose. Fig u r e 5- 2 0 . St u b N e t w or k

The com m on solut ion t o t his issue is t o eit her use st at ic default rout es t o load- share on t he par allel cir cuit s, or use a har dw ar e dev ice at bot h ends of t he par allel cir cuit s t o m ult iplex t he t w o pat hs int o one higher- bandw idt h cir cuit pr esent ed t o t he r out er s. The ot her alt er nat iv e adopt ed by som e pr ov ider s is t o use Mult ilink PPP t o bond t he t w o cir cuit s t oget her . The fir st opt ion is t he easiest t o use because it offer s t he least com plicat ion and has t he sm allest chance of som et hing going w r ong. How ev er , I SPs in r egions w her e high- bandw idt h cir cuit s ar e not av ailable oft en have t o m ult iplex ( MUX) t oget her m any low er bandw idt h cir cuit s j ust t o fulfill t heir r equir em ent s, and quit e oft en t hese I SPs pur chase T1/ E1 MUXs ( for ex am ple) t o j oin anyt hing up t o 16 T1/ E1 cir cuit s int o a higher- bandw idt h pipe. Ther e ar e sev er al pr oduct s on t he m ar k et , and fr om our ex per ience, using a har dw ar e dev ice oft en ends up being sim pler and m ore reliable t han using t he rout er t o do t he sam e t hing.

269

M u lt ih om e d St u b N e t w or k I f t he net w or k in t he pr eceding exam ple is m ult ihom ed on t o t heir upst ream by connect ing t o differ ent r out er s in t heir upst r eam ’s net w or k , BGP w ill hav e t o be used. Som e I SPs use an I GP for t his funct ion ( w e have seen RI P, EI GRP, and OSPF all being used) , but t his pr act ice is st r ongly discour aged because it is a v er y serious pot ent ial sour ce of m isconfigur at ion and pr oblem s in t he I SP back bone. The chances of hav ing t he cust om er ’s I GP leak ing int o t he I SP’s I GP ar e v er y gr eat ; such configur at ions need ex t r em ely car eful filt er ing pr act ices on behalf of t he I SP. ( Som e I SPs use a differ ent I GP t han t he one t hey use in t heir back bone, and t hat can allev iat e t he pr oblem of cr oss- pollut ion.) The pr efer r ed, if not cor r ect , w ay t o do t his is t o use BGP. When m ult ihom ing t o t he sam e I SP, it should be not ed t hat only a pr ivat e ASN is r equir ed. [ 3] There is no need for a public ASN because only t he upst r eam I SP w ill see t he AS. The upst r eam I SP m ust st r ip t he pr ivat e AS fr om any announcem ent s m ade t o t he I nt er net ( using t h e r e m ov e - pr iva t e - AS BGP neighbor com m and) . This sav es a v isit t o t he upst ream I SP, Local I nt ernet Regist ry, or t he RI R t o apply for an ASN ( and under RI R policies at t he t im e of t his w r it ing, t his t y pe of m ult ihom ing w ill not qualify for an ASN) . Figur e 5- 21 giv es an ex am ple in w hich t he cust om er net w or k has t hr ee connect ions t o t he upst r eam , w it h one connect ion going t o a differ ent r out er / PoP in t he upst r eam ’s back bone. Fig u r e 5- 2 1 . D iv e r se ly M u lt ih om e d St u b N e t w or k

The load shar ing on t he t w o par allel connect ions in Figur e 5- 21 can be achiev ed in a var iet y of w ays. Apar t fr om using a har dw ar e MUX or Mult ilink PPP, t he use of BGP is now a possibilit y . A couple of BGP configur at ion opt ions ar e av ailable: The fir st is eBGP m ult ihop, and t he second is BGP m ult ipat h. e BGP M u lt ih op

270

eBGP m ult ihop is an eBGP peer ing bet w een t he loopback int er faces ( or ot her int er face not on t he dem ar cat ion zone bet w een t he t w o net w or k s) of r out er s in t he t w o net w or k s. The configur at ion could be som et hing like t he follow ing:

Router A: router bgp 65534 neighbor 1.1.1.1 remote-as 100 neighbor 1.1.1.1 ebgp-multihop 5 ! ip route 1.1.1.1 255.255.255.255 serial ip route 1.1.1.1 255.255.255.255 serial ip route 1.1.1.1 255.255.255.255 serial ! Router B: router bgp 100 neighbor 2.2.2.2 remote-as 65534 neighbor 2.2.2.2 ebgp-multihop 5 ! ip route 2.2.2.2 255.255.255.255 serial ip route 2.2.2.2 255.255.255.255 serial ip route 2.2.2.2 255.255.255.255 serial !

1/0 1/1 1/2

0/0 0/1 0/2

The addr esses 1.1.1.1 and 2.2.2.2 ar e t he addr esses of t he loopback int er faces on r out er s B and A, r espect iv ely . Because t her e is no dir ect pat h bet w een t he t w o addr esses, a st at ic r out e needs t o be configur ed on bot h r out er s point ing t o eac h ot her ’s loopback addr esses. Thr ee pat hs ar e av ailable, so t hr ee st at ic r out es need t o be inst alled. I f one int er face goes dow n, t he st at ic r out e t hr ough t he ot her int er face st ill w ill be available. The second essent ial com m and is t he e bgp- m u lt ih op st at ement for BGP. BGP peer s nor m ally bet w een addr esses on t he dem ar cat ion zone; t he m u lt ih op st at em ent ensur es t hat BGP w ill set a higher TTL ( t he num ber 5 in t he ex am ple set s a TTL of 5) so t hat a BGP peer ing can be est ablished bet w een r out er s w it h sev er al int er m ediat e hops. The num ber 5 has been used as an exam ple only —som e I SPs use 255 as t heir default value, but w e gener ally r ecom m end not set t ing t he TTL hop m uch higher t han is st r ict ly necessar y . BGP M u lt ipa t h The second opt ion available is t o use BGP m ult ipat h. This should be consider ed as an alt ernat ive t o using eBGP m ult ihop and is an opt ion used by several I SPs also. The configur at ion could be som et hing lik e t he follow ing:

Router A: router bgp 65534 neighbor 1.2.3.1 remote-as 100 neighbor 1.2.3.5 remote-as 100 neighbor 1.2.3.9 remote-as 100 maximum-paths 3 !

271

Router B: router bgp 100 neighbor 1.2.3.2 remote-as 65534 neighbor 1.2.3.6 remote-as 65534 neighbor 1.2.3.10 remote-as 65534 maximum-paths 3 ! Addr esses 1.2.3.1, 1.2.3.5, and 1.2.3.9 ar e t he addr esses on Rout er A for t he point t o- point link bet w een A and B. Rout er B has t he cor r esponding / 30 addr esses. The only ext r a r equir em ent over a st andar d BGP configur at ion is t he m a x im u m - p a t h s 3 dir ect iv e— t his t ells t he rout er t o inst all t hree parallel pat hs int o t he FI B. The dow nside t o t he m ult ipat h com m and is t hat m ult iple BGP sessions ar e r equir ed. I n t he r ar e cir cum st ances in w hich t he full r out ing t able is ex changed bet w een peer s, t his can r equir e a consider able am ount of m em or y on t he rout er t o st ore m any copies of t he r out ing t able r at her t han t he one copy t hat w ould hav e been r equir ed for t he eBGP m ult ihop opt ion. Ge n e r a l M u lt ih om in g The pr ev ious t w o ex am ples ar e quit e specific and offer t he solut ions t o m ult ihom ing in t hose sit uat ions. I n t he I nt ernet at large, m ult ihom ing is generally m ore com plex or has m or e sophist icat ed r equir em ent s. Figure 5- 22 show s a m or e t ypical exam ple of t he t y pe of int er connect ions t hat m ight ex ist . Fig u r e 5- 2 2 . Ge n e r a l M u lt ih om in g

272

I n t his case, t her e is only one solut ion: Use BGP w it h a public ASN. The bulk of t h e r est of t his sect ion on m ult ihom ing discusses t he configur at ion opt ions and t ools t hat ar e av ailable. Befor e inv est igat ing t hese, it is r eally im por t ant t o r em em ber t o k eep it sim ple. The sim plest w ay t o st ar t off figur ing out any m ult ihom ing configur a t ion solut ion is t o st ar t w it h default s and t he m inim um configur at ion, and t hen build it up int o som et hing m or e com plex . Too oft en net w or k engineer s head int o t he r ealm s of t he m ost sophist icat ed configur at ions, full r out ing t ables, and so on, j ust t o solv e t he sim plest of pr oblem s. Com plexit y m eans gr eat er difficult y in m aint aining t he configur at ion, t r aining suppor t st aff t o handle it , and t r aining oper at ions st aff t o t r oubleshoot w hen t her e is a m aj or pr oblem and t he gur u is off on holidays out of m obile phone or e- m ail cover age. Because t his is pr obably t he har dest par t of an int er pr ov ider r elat ionship, it is w or t h spending t he t im e t o w or k out t he configur at ion necessary. Som e effort is required; t here is no m agic solut ion ( alt hough m any I SPs and end sit es oft en w ish for one) .

M ult ihom ing t o t he Sam e I SP This t ype of m ult ihom ing quit e oft en is over looked in favor of m ult ihom ing a net w or k t o differ ent I SPs. How ever , if t he upst r eam I SP is a pr ovider of r easonable qualit y w hose net w or k design, r esilience, and r edundancy can ex per ience no not iceable ser v ice out age, m ult ihom ing using a second connect ion m ak es a gr eat deal of sense t echnically because t he BGP r equir em ent s ar e som ew hat sim pler t han any ot her sit uat ion. ( I t oft en m akes a gr eat deal of sense financially because m any I SPs offer discount s for second cir cuit s t o t heir back bones.) I n t his sit uat ion, as m ent ioned previously, a privat e AS is all t hat is required for BGP. The r egist r ies w ill not assign an AS for t his case because t he AS w ill not hav e, or require, a global view . The upst ream I SP proxy - aggr egat es ( announces t he cust om er addr ess space fr om it s ow n AS) for t he cust om er ; for all int ent s and pur poses, t he I nt er net sees t he cust om er as a sim ple st at ically connect ed ent it y . Ther e is no need for det ailed local t opology infor m at ion par t icular t o only t he I SP and it s cust om er t o be announced t o t he w hole I nt er net . En d Sit e s Tw o t y pes of end sit es ex ist as far as t r affic flow s ar e concer ned. The fir st is t he I SP t hat is connect ing cust om er s t o t he I nt er net . The t r affic flow in t his case is m ost ly inbound; a t y pical v olum e r at io m ight be 70 per cent inbound and 30 per cent out bound. The second is t he cont ent pr ov ider , and t r affic flow lev els t y pically ar e r ev er sed. This sect ion consider s only t he fir st scenar io; t he lat er sect ion on “ Out bound Tr affic Loadshar ing” consider s t he case for t he cont ent pr ov ider . Pr im a r y a n d Ba ck u p Pa t h s The fir st exam ple consider ed ( show n in Figur e 5- 23) is one w it h t w o pat hs bet w een t he net w or ks: One pat h is used as t he pr im ar y link, and t he ot her pat h is used ex clusiv ely for back up. This sit uat ion is used com m only w hen t he pr im ar y pat h has a high bandw idt h and t he backup pat h is of low bandw idt h or poor lat ency and is sufficient only w hen t he m ain link has failed.

273

Fig u r e 5- 2 3 . Pr im a r y a n d Ba ck u p Pa t h s t o t h e Sa m e I SP

The pr im ar y pat h is bet w een Rout er A and Rout er C, w it h t he back up pat h bet w een Rout er B and Rout er D. AS 65534 has an addr ess block of it s ow n—say , 221.10.0.0/ 19 for t his ex am ple. AS 65534 m ust announce it s aggr egat e block t o AS 100—it is v er y im por t ant t hat t he aggr egat e alw ays be announced because t he sm aller t he pr efix is, t he m or e likely som e I SPs ar e t o filt er it ( t her eby causing r eachabilit y pr oblem s for t he net w or k announcing subpr efix es) . The configur at ion of t he v ar ious r out er s follow s:

Router A: router bgp 65534 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.2 remote-as 100 neighbor 222.222.10.2 description RouterC neighbor 222.222.10.2 prefix-list aggregate out neighbor 222.222.10.2 prefix-list default in ! ip prefix-list aggregate permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! Router B: router bgp 65534 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.6 remote-as 100 neighbor 222.222.10.6 description RouterD neighbor 222.222.10.6 prefix-list aggregate out neighbor 222.222.10.6 route-map routerD-out out neighbor 222.222.10.6 prefix-list default in neighbor 222.222.10.6 route-map routerD-in in ! ip prefix-list aggregate permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! route-map routerD-out permit 10 match ip address prefix-list aggregate set metric 10 route-map routerD-out permit 20

274

! route-map routerD-in permit 10 set local-preference 90 ! Router C: router bgp 100 neighbor 222.222.10.1 remote-as 65534 neighbor 222.222.10.1 default-originate neighbor 222.222.10.1 prefix-list Customer in neighbor 222.222.10.1 prefix-list default out ! ip prefix-list Customer permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 !

Router D: router bgp 100 neighbor 222.222.10.5 remote-as 65534 neighbor 222.222.10.5 default-originate neighbor 222.222.10.5 prefix-list Customer in neighbor 222.222.10.5 prefix-list default out ! ip prefix-list Customer permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! The configur at ion on Rout er A is quit e sim ple. The out bound announcem ent m ade is t o send t he / 19 aggr egat e upst r eam ; inbound, t he r out er accept s only t he default r out e fr om t he upst r eam . For all int ent s and pur poses, t he default r out e is ex act ly t he sam e as t he full r out ing t able; it says t hat t he I SP announcing t he default know s how t o get t o t he w hole I nt er net . One pr efix quot ing t he default is m uch more pr efer able t han 108,000 pr efix es t o do t he sam e t hing. Rout er B is t he chosen backup pat h bet w een t he t w o net w or ks. As m ent ioned ear lier , local pr efer ence is used t o set a pr efer ence on inbound announcem ent s t o det er m ine out bound t r affic flow . MED is used t o set a pr efer ence on out bound announcem ent s t o influence inbound t r affic flow . So t he configur at ion on Rout er B does j ust t hat : • •

A local pr efer ence of 90 is set on t he inbound announcem ent s. As a r esult , any pr efix es lear ned w ill hav e a low er pr ior it y t han t hose learned from Rout er A. A MED of 10 is set on out bound announcem ent s. AS 100 sees t he pr efix es fr om AS 65534 w it h an MED of 10, a higher value t han hear d t hr ough t he ot her pat h and, t her efor e, less pr efer r ed.

Rout er C and Rout er D share very similar configur at ions: The upst r eam I SP has left all t he m ult ihom ing opt ions t o it s cust om er . The aggr egat e is allow ed in, t he default is sent out , and ot her w ise t he configur at ion is quit e sim ple. I t also is easy t o m aint ain —any changes r equir ed in t he m ult ihom ing ar e left t o t he cust om er , r educing t he suppor t bur den.

275

I f t he cust om er does not w ant t o deal w it h t he com plex it y of configur ing t he m ult ihom ing, t he configur at ion can be sw apped ar ound ( so t hat t he MED and local prefs go on AS 100, w it h t he configurat ion on AS 65534 r out er s m ade “ sim ple” ) . Rout er E’s configur at ion needs t o be set so t hat it st r ips t he pr iv at e AS out of any announcem ent s t o t he I nt er net . The configur at ion ex am ple follow s:

Router E: router bgp 100 neighbor 222.222.10.17 neighbor 222.222.10.17 neighbor 222.222.10.17 ! ip prefix-list Customer

remote-as 200 remove-private-AS prefix-list Customer out permit 221.10.0.0/19

! Not e t he r e m ov e - pr iva t e - a s dir ect ive on t he peer ing w it h AS 200. This st r ips t he AS 65534 fr om t he / 19 aggr egat e announced by t he cust om er of AS 100 and announces it as t hough it or iginat ed fr om AS 100. Loa d Sh a r in g The second ex am ple, display ed in Figur e 5- 24, r efines t he pr ev ious one so t hat bot h link s bet w een t he t w o I SPs can be used t o car r y t r affic at all t im es. This m or e com m only is used in t he I nt er net because m any long- dist ance and int er nat ional circuit s are st ill subst ant ially m or e ex pensiv e per k ilom et er t han t hose in m et r opolit an ar eas. Fig u r e 5- 2 4 . Loa d Sh a r in g t o t h e Sa m e I SP

As befor e, AS 65534 announces it s aggr egat e t o t he upst r eam I SP. To achiev e t he load shar ing, t he aggr egat e is also split int o t w o pieces, t w o / 20 net w or ks. One / 20 is announced out of one link, and t he ot her / 20 is announced out of t he ot her link. The configur at ion ex am ples ar e

Router A: router bgp 65534 network 221.10.0.0 mask 255.255.224.0

276

network 221.10.0.0 mask 255.255.240.0 neighbor 222.222.10.2 remote-as 100 neighbor 222.222.10.2 prefix-list routerC out neighbor 222.222.10.2 prefix-list default in ! ip ip ip ! ip ip !

prefix-list default permit 0.0.0.0/0 prefix-list routerC permit 221.10.0.0/20 prefix-list routerC permit 221.10.0.0/19 route 221.10.0.0 255.255.240.0 null0 route 221.10.0.0 255.255.224.0 null0

Router C: router bgp 100 neighbor 222.222.10.1 remote-as 65534 neighbor 222.222.10.1 default-originate neighbor 222.222.10.1 prefix-list Customer in neighbor 222.222.10.1 prefix-list default out ! ip prefix-list Customer permit 221.10.0.0/19 le 20 ip prefix-list default permit 0.0.0.0/0 ! Rout er A inser t s t he / 19 and t he fir st / 20 int o t he BGP pr ocess. Not ice t hat t he t w o st at ic r ou t es t o place t he t w o pr efixes ar e in t he r out ing t able so t hat t he pr efixes go int o t he BGP t able. Rout er A announces t he / 19 and t he fir st / 20 ( t he pr efix list called r out er C t akes car e of t his) t o Rout er C. Rout er B’s configur at ion is alm ost ident ical t o t hat of Rout er A; it s configurat ion uses t he ot her / 20. As befor e, t he upst r eam I SP configur at ion has been k ept deliber at ely sim ple. The upst r eam I SP sim ply allow s t he aggr egat e and any / 20 subpr efix fr om it s cust om er and announces only t he default r out e t o it s cust om er. This is a r easonable fir st cut at m aking t his t ype of load shar ing w or k. No local pr efer ences, MEDs, or AS pat h m anipulat ion is r equir ed. I f AS 65534 r equir es fur t her m anipulat ion t o balance out t r affic pat t er ns, it could t r y subdiv iding t he / 19 int o / 21 subpr efix es and announcing t hose t o t he upst r eam . This w ould r equir e changes in t he BGP configur at ion and pr efix list r anges t o suit . Quit e oft en an upst r eam I SP sim ply allow s t he aggr egat e and a r ange of subpr efix sizes fr om it s cust om er so t hat t he cust om er can m anipulat e t he load shar ing as r equir ed. Rout er E configur at ion is unchanged fr om t he pr evious exam ple: AS 100 only announces t he cust om er aggr egat e and r em ov es t he pr iv at e AS fr om t he AS pat h. Ther e is nev er any need t o announce t he cust om er subprefixes in a sit uat ion like t his. M u lt ip le D u a l- H om e d Cu st om e r s ( RFC 2 2 7 0 ) RFC 2270– based m ult ihom ing is an ext ension of t he pr evious t w o exam ples, descr ibing how t o scale a sit uat ion in w hich m ult iple cust om er s ar e m ult ihom ing ont o t he I SP back bone.

277

Figur e 5- 25 show s how m ult iple cust om er w ould m ult ihom e ont o t he back bone of t he sam e upst r eam I SP. The diagr am also show s t hat t he sam e ASN is used—t his is not a dr aw ing er r or , but it is genuinely all t hat is r equir ed. The configur at ion w ill be based on t hat used in t he pr evious sect ion, so AS 100 r out er s w ill be announcing only a default r out e t o cust om er s; t her e is no ot her r out ing infor m at ion and no r out ing infor m at ion fr om one cust om er t o t he ot her , so t her e is no chance of BGP loop det ect ion com ing int o play . Fig u r e 5- 2 5 . RFC 2 2 7 0 M u lt ih om in g

The pr inciples ar e t he sam e as befor e: Each cust om er announces it s aggr egat e t o AS 100 and any subpr efixes t hat ar e necessar y t o m ake t he load shar ing on t he dual links w or k. The configur at ion exam ples ar e

Router A1: router bgp 65534 network 221.10.0.0 mask 255.255.224.0 network 221.10.0.0 mask 255.255.240.0 neighbor 222.222.10.2 remote-as 100 neighbor 222.222.10.2 prefix-list routerC out neighbor 222.222.10.2 prefix-list default in

278

! ip ip ip ! ip ip !

prefix-list default permit 0.0.0.0/0 prefix-list routerC permit 221.10.0.0/20 prefix-list routerC permit 221.10.0.0/19 route 221.10.0.0 255.255.240.0 null0 route 221.10.0.0 255.255.224.0 null0

Router C: router bgp 100 neighbor bgp-customers peer-group neighbor bgp-customers remote-as 65534 neighbor bgp-customers default-originate neighbor bgp-customers prefix-list default out neighbor 222.222.10.1 peer-group bgp-customers neighbor 222.222.10.1 description Customer One neighbor 222.222.10.1 prefix-list Customer1 in neighbor 222.222.10.9 peer-group bgp-customers neighbor 222.222.10.9 description Customer Two neighbor 222.222.10.9 prefix-list Customer2 in neighbor 222.222.10.17 peer-group bgp-customers neighbor 222.222.10.17 description Customer Three neighbor 222.222.10.17 prefix-list Customer3 in ... ! ip prefix-list Customer1 permit 221.10.0.0/19 le 20 ip prefix-list Customer2 permit 221.16.64.0/19 le 20 ip prefix-list Customer3 permit 221.14.192.0/19 le 20 ... ip prefix-list default permit 0.0.0.0/0 ! Router E: router bgp 109 neighbor 222.222.10.17 remote-as 110 neighbor 222.222.10.17 remove-private-AS neighbor 222.222.10.17 prefix-list Customers out ! ip prefix-list Customers permit 221.10.0.0/19 ip prefix-list Customers permit 221.16.64.0/19 ip prefix-list Customers permit 221.14.192.0/19 ! The Rout er A1 configur at ion is t he t em plat e for all t he Rout er A’s in t he net w or k . The Rout er B’s also all shar e t he sam e configur at ion, j ust using t he ot her / 20 fr om t he aggregat e block. The int erest ing configurat ion is for Rout er C. Rat her t han handcraft an individual configur at ion for each cust om er , a peer gr oup is cr eat ed and applied for each. Aft er all, each cust om er has t he sam e out bound announcem ent , a default r out e, so a peer group can be used easily ( it also is m ore efficient for t he rout er and is m uch easier t o read) . Each cust om er has it s ow n address block t hat is filt ered in t he inbound BGP configur at ion.

279

Rout er D configur at ion is almost ident ical t o t hat of Rout er C. I ndeed, only t he point t o- point link addr esses for t he BGP peer ings ar e differ ent . Rout er E announces j ust t he aggr egat es t o t he I nt er net , st r ipping out t he pr iv at e AS fr om t he AS pat h. Not e t hat if t he aggr egat es all come out of AS 100’s addr ess space, t hat addr ess space should be announced t o t he I nt er net inst ead of t he indiv idual announcem ent s fr om t he cust om er s. Ther e is no point announcing t he subpr efixes; even t hough it does no har m , it is unnecessar y infor m at ion on t h e r out ing t able and adds t o t he bur den t hat all t he I SPs car r y ing t he full r out ing t able m ust deal w it h. A configurat ion of Rout er E in t his case m ight be as follow s:

router bgp 109 neighbor 222.222.10.17 remote-as 110 neighbor 222.222.10.17 remove-private-AS neighbor 222.222.10.17 prefix-list my-aggregate out ! ip prefix-list my-aggregate permit 221.8.0.0/13 ! Rem ov al of t he pr iv at e AS isn’t ev en r equir ed because t he subpr efix es ar e st r ipped out by t he pr efix list . How ever , w e consider it good pr act ice t o include t he com m and in all eBGP configur at ions, j ust in case of any configur at ion er r or s or any inst ances in w hich t he pr efix list fails t o include all pr efixes.

M ult ihom ing t o Different I SPs When t w o connect ions t o an upst r eam I SP is not sufficient , net w or k s choose t o m ult ihom e bet w een differ ent I SPs. This is som ew hat har der t o achiev e, and it does need a lit t le m or e car e because t he effect s ar e seen ov er t he w hole I nt er net . I t is under st ood t hat a public ASN is r equir ed for t his t ype of m ult ihom ing, alt hough it is per fect ly feasible t o use a pr iv at e AS if bot h upst r eam par t ies agr ee t o it . Ther e ar e sev er al inst ances of t his in t he I nt er net , but by far t he m or e com m on pr act ice is t o use a public ASN. Many of t he configur at ion hint s cover ed in t he pr ev ious sect ion apply her e, but w it h suit able m odificat ion. Sev er al com m on ex am ples ar e consider ed nex t . Pr im a r y a n d Ba ck u p Pa t h s The fir st ex am ple consider ed is on in w hich t her e ar e t w o pat hs fr om t he end sit e t o t he I nt er net : One pat h is used as t he prim ary link, and t he ot her pat h is used ex clusiv ely for back up. This sit uat ion is com m only used w hen t he pr im ar y pat h has a high bandw idt h, w hen t he back up pat h is of low bandw idt h or poor lat ency , or w hen t he upst ream has a low - qualit y or expensive t r a nsit net w or k and is sufficient only when t he m ain link has failed. I n Figur e 5- 26 t he m ain link is bet ween Rout ers A and C; t he link bet ween Rout ers B and D is t he back up link . To configur e t he r out er s t o achiev e t his t r affic flow , an AS pat h pr epend is applied t o out bound announcem ent s fr om Rout er B—and local pr efer ence is applied inbound t o announcem ent s r eceiv ed by Rout er B. Tak ing an I nt er net view , if a pr epend of one AS is applied on Rout er B, t he pat h from t he

280

I nt er net t hr ough AS 108 t o Rout er B w ill appear t o be one hop longer t han t he pat h t hr ough AS 109 and Rout er A. How ever , t r affic fr om AS 108 t o AS 107 should go t hr ough AS 109 r at her t han follow ing t he dir ect pat h: A prepend of one AS will m ake t he dir ect pat h st ill shor t er t han t he pat h t hr ough t he I nt er net ( w hich m ight hav e one or m or e aut onom ous sy st em s in t he pat h) . A t w o- AS prepend will look t he sam e lengt h ( assum ing one AS in t he I nt er net ) , but a t hr ee- AS prepend will m ake t he pat h t hr ough AS 109 look shor t er . So t his is t he v alue used in t he configur at ion t hat follows. Fig u r e 5- 2 6 . Pr im a r y / Ba ck u p t o D iffe r e n t I SPs

Router A: router bgp 107 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.1 remote-as 109 neighbor 222.222.10.1 prefix-list aggregate out neighbor 222.222.10.1 prefix-list default in ! ip prefix-list aggregate permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 !

Router B: router bgp 107 network 221.10.0.0 neighbor 220.1.5.1 neighbor 220.1.5.1 neighbor 220.1.5.1 neighbor 220.1.5.1

mask 255.255.224.0 remote-as 108 prefix-list aggregate out route-map routerD-out out prefix-list default in

281

neighbor 220.1.5.1 route-map routerD-in in ! ip prefix-list aggregate permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! route-map routerD-out permit 10 set as-path prepend 107 107 107 ! route-map routerD-in permit 10 set local-preference 80 ! The configur at ions for Rout er C and Rout er D are not given: They are not very differ ent fr om any of t he pr ev ious ex am ples, in t hat t hey should accept t he AS 107 aggr egat e only and announce j ust a default r out e t o AS 107. As m ent ioned ear lier , w hen t he bulk of t r affic flo w is int o t he end sit e, t here is lit t le requirem ent for out bound load shar ing for AS 107 beyond w hat near est exit w ill offer . An exam ple of t he I GP configur at ion for t his scenar io follow s in t he case st udy sect ion lat er in t his chapt er . Loa d Sh a r in g The second ex am ple r efines t he pr ev ious one so t hat bot h link s t o t he t w o upst r eam I SPs can be used t o car r y t r affic at all t im es. Figur e 5- 27 show s t he scenar io. The w ay t o im plem ent load sharing here is t o st art by subdividing t he / 19 int o t w o / 20s— t he / 19 aggr egat e st ill is announced out of each link , but t he announcem ent of a / 20 on each link w ill ensur e t hat t r affic for t hat / 20 w ill follow t hat pat h by pr efer ence. Fig u re 5 - 2 7 . Loa d Sh a r in g

282

The configur at ions of t he r out er s ar e giv en as follow s:

Router A: router bgp 107 network 221.10.0.0 mask 255.255.224.0 network 221.10.0.0 mask 255.255.240.0 neighbor 222.222.10.1 remote-as 109 neighbor 222.222.10.1 prefix-list firstblock out neighbor 222.222.10.1 prefix-list default in ! ip prefix-list default permit 0.0.0.0/0 ! ip prefix-list firstblock permit 221.10.0.0/20 ip prefix-list firstblock permit 221.10.0.0/19 ! Router B: router bgp 107 network 221.10.0.0 mask 255.255.224.0 network 221.10.16.0 mask 255.255.240.0 neighbor 220.1.5.1 remote-as 108 neighbor 220.1.5.1 prefix-list secondblock out neighbor 220.1.5.1 prefix-list default in ! ip prefix-list default permit 0.0.0.0/0 ! ip prefix-list secondblock permit 221.10.16.0/20 ip prefix-list secondblock permit 221.10.0.0/19 ! Not ice fr om t he Rout er A configur at ion t hat t he fir st / 20 is announced out t he link t o AS 109. All t r affic for t his / 20 w ill pr efer t his pat h, over r iding any pat h fr om t he announcem ent of t he / 19. I f t he link t o AS 109 fails, t r affic w ill diver t t o t he pat h t hr ough AS 108 ( because of t he announcem ent of t he / 19 on t hat pat h) . A sim ilar sit uat ion is t rue for t he pat h t hrough Rout er B t o AS 108. The announc em ent of t he second / 20 t hr ough t hat pat h ensur es t hat all inbound t r affic for t hat half of t he / 19 w ill t ak e t his pat h. This is a r udim ent ar y fir st cut at set t ing up load shar ing bet w een t w o upst r eam I SPs. I t is pr efer able t o st ar t her e and slice t he addr ess space sm aller as needed, r at her t han chopping t he / 19 int o 32 / 24s t o t r y t o m ake som et hing w or k fr om t her e. Sim plicit y in configur at ion m ak es t he lik elihood of som et hing w or k ing som ew hat gr eat er . Not ice t hat announcing a subpr efix of a / 19 quit e lik ely w ill hav e less chance of being pr opagat ed acr oss t he ent ir e I nt er net t han using t he / 19 w ould hav e. This is w hy it is cr it ical t hat t he / 19 alw ay s is announced t hr ough ev er y pat h. I f m ore fine t uning of t he load sharing is required, t he follow ing exam ple giv es an idea of som e of t he opt ions t hat m ight be av ailable. I n t he pr ev ious case, t r affic fr om AS 108 t o t he fir st / 20 block w ill t ake t he I nt er net pat h t o AS 109 r at her t han going dir ect ly over t he link t o AS 109. This is subopt im al, and I SPs alw ays t ry t o ensur e t hat r out ing is sy m m et r ic and t hat t r affic pat hs ar e as shor t as possible ( in t he absence of any polit ical r equir em ent s) . The follow ing ex am ple again subdiv ides t he / 19 int o t w o / 20s, but only one of t he / 20s is announced out bound, on t he link t o AS

283

108. The / 19 is announced on t his pat h w it h an AS pat h prepend. The link t o AS 109 sees only t he / 19 being announced.

Router A: router bgp 107 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.1 remote-as 109 neighbor 222.222.10.1 prefix-list default in neighbor 222.222.10.1 prefix-list aggregate out ! ip prefix-list aggregate permit 221.10.0.0/19 ! Router B: router bgp 107 network 221.10.0.0 mask 255.255.224.0 network 221.10.16.0 mask 255.255.240.0 neighbor 220.1.5.1 remote-as 108 neighbor 220.1.5.1 prefix-list default in neighbor 220.1.5.1 prefix-list subblocks out neighbor 220.1.5.1 route-map routerD out ! route-map routerD permit 10 match ip address prefix-list aggregate set as-path prepend 107 107 route-map routerD permit 20 ! ip prefix-list subblocks permit 221.10.0.0/19 le 20 ip prefix-list aggregate permit 221.10.0.0/19 ! Tr affic for t he / 20 w ill alw ays com e int o AS 107 t hr ough AS 108. I f t he link t o AS 108 fails, t r affic w ill com e in t hr ough t he link t o AS 109. Tr affic for t he r est of t he addr ess space w ill com e in t hr ough t he link t o AS 109 —t he AS pat h pr epend ensur es t hat t his happens. Manipulat ing t he size of t he pr epend allow s AS 107 t o cont r ol w hich pat h inbound t r affic w ill t ak e int o t he net w or k . A single pr epend w ill at t r act m or e t r affic; w hereas, t w o or t hree or m ore w ill reduc e t he am ount seen for t he second / 20.

Out bound Tra ffic Loa d Sha ring The pr ev ious ex am ples dealt w it h sit uat ions in w hich t he cust om er w as t r y ing t o deal w it h load sharing inbound t raffic —and paying lit t le at t ent ion t o dealing w it h out bound t r affic, apar t fr om using t he near est exit . This sect ion goes int o how t o load- share out bound t r affic in m or e det ail; it ’s m or e applicable t o cont ent pr ovider s at t he edge of t he net w or k or I SPs t hat ar e par t of t he t r ansit pat h t hr ough t he I nt er net cor e. I SPs st r iv e t o balance t r affic on t heir cir cuit s as m uch as possible; congest ion im plies incr eased lat ency and poor er per for m ance for cust om er s. They also st r iv e t o k eep t r affic as sy m m et r ical as possible. Alt hough t her e is no per for m ance dow nside t o hav ing asy m m et r ic t r a ffic, it is gener ally bet t er t o k eep flow s as sy m m et r ic as possible, t o m ake t he net w or k easier t o scale and easier t o m anage. Of cour se, som e ex t r em e sit uat ions ex ist in ar eas of poor er upst r eam connect iv it y , w her e asy m m et r ic

284

flow s m ight be highly undesirable. The case st udy ex am ple in t his sect ion illust r at es su ch a case. The com m on w isdom r egar ding t he best w ay t o balance out bound t r affic for any m ult ihom ed net w or k is t o giv e t his net w or k a full v iew of t he I nt er net Rout e Table dow n all t he pat hs t o t he net w ork. Alt hough t his undoubt edly w ill w ork, it is far from t he m ost opt im al solut ion. A decade ago, w hen t he r out ing t able had 10,000 pr efixes, t his m ight have been viable opt ion. But pur chasing a r out er w it h huge m em or y and lar ge CPU sim ply t o pr ocess t he r out ing updat es and do t he best - pat h select ion in a r easonable am ount of t im e is sim ply not a v iable opt ion for m any sm all- t o m edium- size I SPs. What ’s m or e, it is sim ply unnecessar y , as t he ex am ples in t his sect ion show . On e U p st r e a m I SP a n d On e Loca l P e e r To st ar t t he list of ex am ples, consider per haps t he sim plest and m ost com m on m ult ihom ing t hat a sm all I SP w ill im plem ent , as show n in Figur e 5- 28. The I SP connect s t o it s upst r eam I SP, and it needs t o connect t o it s local ( com pet ing) I SP so t hat dom est ic t r affic does not use t he ex pensiv e upst r eam connect ions. Fig u r e 5- 2 8 . On e U p st r e a m I SP a n d On e Loca l Pe e r

The com m on w ay in w hich t his is configur ed t oday is for t he upst r eam I SP t o send t he full r out ing t able t o AS 109 and for t he local I SP t o send all it s pr efixes t o AS 109. But w hy does AS 109 need t o see t he full I nt ernet t able? There is only one pat h t o t he I nt er net ; t he peer ing w it h t he local I SP giv es AS 109 access t o only local r out es, and t hat is t he only infor m at ion t hat AS 109 act ually needs, bey ond a default r out e t o t he r est of t he I nt er net . The ro ut er configur at ion for t his exam ple is as follow s:

Router A: router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.2 remote-as 108 neighbor 222.222.10.2 prefix-list my-block out neighbor 222.222.10.2 prefix-list AS108-peer in neighbor 222.222.10.2 filter-list 10 in

285

! ip ip ip ! ip ! ip !

prefix-list AS108-peer permit 222.5.16.0/19 prefix-list AS108-peer permit 221.240.0.0/20 prefix-list my-block permit 221.10.0.0/19 route 221.10.0.0 255.255.224.0 null0 as-path access-list 10 permit ^(108_)+$

Router C: router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.1 remote-as 107 neighbor 222.222.10.1 prefix-list default in neighbor 222.222.10.1 prefix-list my-block out ! ip prefix-list my-block permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! ip route 221.10.0.0 255.255.224.0 null0 ! The configur at ion of Rout er A has bot h a pr efix list and an AS pat h access list pr esent on t he inbound par t of t he BGP configur at ion. The pr efix list w ill specifically filt er t he pr efix es com ing fr om AS 108, w hile t he AS pat h access list w ill filt er t he pr efixes so t hat only t hose or iginat ed by AS 108 w ill be per m it t ed. This is an exam ple of a double applicat ion of inbound policy. St r ict ly, only t he pr efix list is necessar y , but t he I SP has chosen t o include t he AS pat h filt er in case of any configur at ion pr oblem ( such as t he pr efix list accident ally being er ased) . Som e I SPs include only t he AS pat h filt er ; how ever , t his is ver y t r ust ing of t he neighbor ing AS because it sim ply says t hat all pr efixes or iginat ed by t he neighbor ing AS ar e t o be allow ed in. This does not pr ov ide any safet y against t he neighbor ing AS accident ally or iginat ing pr efix es t hat it should not be. The configur at ion of Rout er C is v er y st r aight for w ar d: The default r out e is per m it t ed inbound, and only t he local addr ess block is per m it t ed out bound. The r esult is a ver y lean r out ing t able w it hin AS 109. This is all t hat is r equir ed t o set up m ult ihom ing bet w een a neighbor ing I SP and t he upst r eam I SP. Tw o U p st r e a m s I SPs a n d On e Loca l Pe e r The second exam ple adds a second upst r eam I SP t o t he pr ev ious configur at ion, as show n in Figur e 5- 29. This is anot her com m on configur at ion, usually an enhancem ent of t he pr ev ious case in w hich t he net w or k w ant s som e r esilience for it s I nt er net connect ion. Fig u r e 5- 2 9 . Tw o Upst r e a m I SPs

286

Again, t he com m on solut ion t o t his pr oblem is for bot h upst r eam I SPs t o pr ov ide t he full rout ing t able t o AS 109, as can be show n in t he follow ing rout er configurat ion ex am ples:

Router C: router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.1 remote-as 107 neighbor 222.222.10.1 prefix-list rfc1918-deny in neighbor 222.222.10.1 prefix-list my-block out neighbor 222.222.10.1 route-map AS107-loadshare in ! ip prefix-list my-block permit 221.10.0.0/19 ! See Appendix B for the RFC1918 list ! ip route 221.10.0.0 255.255.224.0 null0 ! ip as-path access-list 10 permit ^(107_)+$ ip as-path access-list 10 permit ^(107_)+_[0-9]+$ ! route-map AS107-loadshare permit 10 match ip as-path 10 set local-preference 120 route-map AS107-loadshare permit 20 set local-preference 80 ! Router D: router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.5 remote-as 106 neighbor 222.222.10.5 prefix-list rfc1918-deny in neighbor 222.222.10.5 prefix-list my-block out ! ip prefix-list my-block permit 221.10.0.0/19 ! See Appendix B for RFC1918 list ! This configur at ion t akes full r out es fr om bot h upst r eam I SPs. Rout er D t akes ever yt hing and m odifies not hing. Rout er C t akes ever yt hing, but for all pr efixes

287

or iginat ed by AS 107 and AS 107’s im m ediat ely neighbor ing aut onom ous sy st em s, it set s a local pr efer ence of 120. This is so t hat all t r affic t o AS 107 and it s im m ediat ely neighbor ing aut onom ous sy st em s w ill go out t he link t o AS 107. The r em aining pr efix es lear ned fr om AS 107 hav e t heir local pr efer ence set t o 80 so t hat t he pat h t hrough AS 107 is less preferred t han t he pat h t hrough AS 106. Obviously, t his w on’t give per fect load shar ing, but it w ill give you an idea of w her e t o st ar t and w hat t he t hought pr ocesses should be in achiev ing a r easonable out bound balance of t r affic. The alt ernat ive and m uc h pr efer r ed w ay of configur ing BGP for t his net w or k is not t o use t he full r out ing t able. Rem em ber fr om befor e t hat a default r out e is equiv alent t o t he full r out ing t able, for all int ent s and pur poses; it r equir es a subst ant ially r educed am ount of r out er m em or y and CPU t im e t o pr ocess. The configur at ion for t he light er v er sion follow s:

Router C: router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.1 remote-as 107 neighbor 222.222.10.1 prefix-list rfc1918-nodef-deny in neighbor 222.222.10.1 prefix-list my-block out neighbor 222.222.10.1 filter-list 10 in neighbor 222.222.10.1 route-map tag-default-low in ! ip prefix-list my-block permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! ip route 221.10.0.0 255.255.224.0 null0 ! ip as-path access-list 10 permit ^(107_)+$ ip as-path access-list 10 permit ^(107_)+_[0-9]+$ ! route-map tag-default-low permit 10 match ip address prefix-list default set local-preference 80 route-map tag-default-low permit 20 ! Router D: router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.5 remote-as 106 neighbor 222.222.10.5 prefix-list default in neighbor 222.222.10.5 prefix-list my-block out ! ip prefix-list my-block permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! ip route 221.10.0.0 255.255.224.0 null0 ! Rout er D r eceives only a default r out e fr om AS 106 and sends only t he / 19 net w or k block or iginat ed by AS 109; t his is a v er y st r aight for w ar d configur at ion.

288

Rout er C is slight ly m ore com plex, but again t his m eans car r y ing a subst ant ially r educed r out ing t able ov er t he v er sion show n ear lier . The r fc1918- nodef- deny prefix list allow s all pr efixes int o Rout er C, apar t fr om t he RFC 1918 list docum ent ed in t he appendix es ( t he default r out e also is per m it t ed int o Rout er C) . Aft er t he pr efix list has been applied, t he filt er list ( AS pat h access list ) block s pr efix es t hat hav e not been or iginat ed by AS 107 or AS 107’s im m ediat ely neighbor ing aut onom ous syst em s. Finally, t he r out e m ap is applied t o set a low local pr efer ence on t he default r out e t hat AS 107 is announcing t o Rout er C. The end r esult of t his is t hat t he default r out e is t hr ough Rout er D, w it h specific pat hs being lear ned fr om Rout er C and backup default t hr ough Rout er C. We have seen t his t y pe of configurat ion in act ion in several sm aller I SPs, and it is very effect ive w it h sm all r out er s, sm all am ount s of m em or y, and lit t le CPU pow er . I n all t est s, t he failov er occur s m or e quick ly in t hat t he r out er CPUs hav e t o spend less t im e pr ocessing best pat hs, less t im e sending iBGP updat es t o peer s, and so on; t his is t he pr efer r ed and r ecom m ended pat h t o m ult ihom ing bet w een t w o upst r eam I SPs. Not ice t hat Rout er A has scar cely been m ent ioned—not hing changes w it h it s configur at ion. The pat h t o AS 108 alw ays w ill be shor t er t han any alt er nat iv e pat h t hr ough t he upst r eam I SPs. Also be aw ar e t hat som e upst r eam I SPs m ight be unw illing t o announce a default r out e ( usually for fear t hat a cust om er w ill cr eat e a configur at ion er r or t hat som ehow will dam age t he net work) . I n t his case, t he alt er nat iv e is t o use t he full t able and a pr efix t hat can be used as a default r out e—a sim ple w or k ar ound. See t he case st udy t hat follow s for anot her m echanism t o get ar ound t his pr oblem . M u lt ip le U p st r e a m I SPs a n d I X P The t hird exam ple adds a lit t le m or e com plicat ion t o t he net w or k and is pr obably one of t he m or e ext r em e cases found on t he I nt er net t oday, as show n in Figur e 5- 30. Fig u r e 5- 3 0 . M u lt iple Upst r e a m I SPs

289

This is an exam ple of a well- connect ed I SP, pr esent at t he local ex change point , w it h a few pr ivat e peer s, som e r egional I SP connect io ns pr oviding backup t r ansit , and a couple of upst r eam / Tier 1 I SPs pr ov iding I nt er net t r ansit . The configur at ions for t his ar en’t t oo har d t o gener at e, eit her . Follow ing t he pr ev ious ex am ples, t he discussion w it hin t he net w or k engineer ing t eam t o pr oduce t he configur at ion should go like t he follow ing: • •

vLoca l pe e r s — Tak e all local r out es fr om t hem , and leav e t hem unt ouched. Loca l I XPs — Take all local r out es fr om t hem , and leave t hem unt ouched. I f t her e is a pr ivat e peer ing w it h an I XP m em ber , w e pr efer t he pat h ov er t he I XP: - Announce t he net w or k over t he pr ivat e peer ing w it h a longer AS pat h ( pr epend of one) or a user a higher MED, if t he peer suppor t s MEDs. - Tag t he pr efix es lear ned ov er t he pr iv at e peer ing w it h a low er local pr efer ence. [ 4] ( This keeps t he I XP configur at ion consist ent for all peer s, allow ing you t o use peer gr oups on t he I XP r out er s.)

•

•

Re gion a l pe e r s — Tak e t heir local r out es because t hey offer bet t er connect iv it y t o t hese r egional net w or k s t han t he upst r eam r out es w ill, and t his w ill be cheaper financially . May be t ak e t heir neighbor ing AS r out es as w ell, if t he pat h is bet t er . Ask for a default r out e, and t ag it w it h a low local pr efer en ce—say, 60. Upst r e a m I SPs — Take only a default rout e from one upst ream I SP and leav e it t agged w it h t he default local- pr efer ence of 100. For t he ot her upst r eam I SP, t ak e it s or iginat ed r out es and t he or iginat ed r out es of it s im m ediat ely neighbor ing aut onom ous sy st em s. A lso ask for a default r out e; t ag t his w it h a local pr efer ence of 80.

I n all cases, AS 109 announces it s addr ess block t o t he I nt er net ( if m or e m anipulat ion of inbound t r affic is r equir ed, consider t he ex am ples ear lier in t his sect ion) . The sum m ar y of t his is quit e sim ple: Local t r affic st ay s local, r egional t r affic st ay s r egional, t he pr im ar y pat h is t o one upst r eam I SP, and t he backup is t o t he ot her upst r eam I SP. I f bot h upst r eam I SPs go dow n, t he backup is t hr ough t he r egional peer s. All t his can be done w it hout t ak ing t he full r out ing t able fr om any of t he neighbor ing aut onom ous sy st em s. The ex am ples t hat w e hav e seen in act ion hav e a r out ing t able of about 22,000 pr efix es, one fift h of t he full r out ing t able av ailable at t he t im e of w r it ing. The I SPs in quest ion also hav e r epor t ed no oper at ional or reachabilit y problem s so far. Ca se St u dy The follow ing case st udy im plem ent s t he t echniques descr ibed pr ev iously for an I SP t hat r ecent ly needed t o m ult ihom e t o t he I nt er net . The scenar io is quit e sim ple, but it cov er s a sit uat ion t hat causes m any new com er s t o BGP and t he I nt er net quit e a lot of t rouble.

290

The net work in Figur e 5- 31 show s t he net w or k layout . AS 5400 is based in Eur ope, and AS 2516 is based in Japan—t his in it self poses a challenge because t he m ult ihom ing is bet w een ent it ies t hat ar e quit e lit er ally on opposit e sides of t he globe. Bot h cir cuit s ar e of equal capacit y , m ak ing t he configur at ion som ew hat easier t han it m ight hav e ot her w ise been. Fig u r e 5- 3 1 . Ca se St u d y N e t w or k

The appr oach t ak en is quit e sim ple, t hough, and is based on t hat has been descr ibed alr eady: Keep it sim ple! I deally, t he t w o upst r eam I SPs w ill send a default r out e, and one upst ream I SP w ill send a m inim al rout ing t able of t he prefixes from t he net w ork and t hose or iginat ed by local peer s. So, in t heor y , t his should all be configur able w it h a m inim al rout ing t able on t he AS 17660 net w ork. Realit y is slight ly differ ent , t hough. AS 5400 can offer only t he full r out ing t able t o any BGP cust om er , so t his m ust be t aken int o account in t he configur at ion of Rout er A. AS 2516 can offer eit her ev er yt hing or j ust t he default r out e, so t he opt ion w as t ak en t o r equest j ust a default r out e. The fir st cut at pr oducing a w or k ing configur at ion car r ied out t he follow ing t ask s: 1. Going t o sev er al look ing glasses in t he cent er of t he I nt er net gav e a good idea of how far AS 2516 and AS 5400 w er e fr om t he “ cent er .” The definit ion of t he “ cent er ” is har d, but look ing glasses at m aj or U.S. ex change point s pr obably give t he m ost accur at e definit ion of w her e t he cent er m ight be—or at least get close. Analy sis of t he r out ing t able at t he looking glasses show ed t hat AS 2516 w as, on av er age, one AS hop closer t o t he cent er t han AS 5400. The r esult ing configur at ion for out bound announcem ent s becam e t he follow ing: - To AS 5400, j ust announce t he aggr egat e block. - To AS 2516, announce t he aggr egat e block w it h a single pr epend of AS 17660.

291

2. The next st ep w as t o r equest AS 2516 t o send j ust a default r out e. When t his w as av ailable, t he peer ing w it h AS 2516 w as br ought up. 3. AS 5400 could send only t he full r out ing t able. Adv ant age of t his w as t ak en t o st r ip aw ay m ost of t he r out ing t able, t o leave a sensible num ber of pat hs t o balance t he load shar ing of out bound t r affic bet w een t he t w o pat hs. - Look ing at t he m any r out ing t able analy ses being done on t he I nt er net at t he m om ent , t he decision w as m ade t o t ak e t he t op fiv e cont r ibut or s t o t he r out ing t able and ex clude any pr efix es t hat had t hose aut onom ous sy st em s in t he pat h. - Aft er t his, only pr efixes or iginat ed by AS 5400, it s im m ediat ely neighbor ing aut onom ous sy st em s, and t heir im m ediat e neighbor s w er e accept ed. ( AS 5400 is a m aj or t ransit backbone in Europe, so t his allow ed pat hs from peers and peer ing Eur opean net w or k s in.) This pr oduced a r out ing t able of ar ound 30,000 prefixes on Rout er A. This w asn’t a bad fir st at t em pt . Inbound t r affic t ur ned out t o be alm ost per fect ly balanced, ar ound 500 Kbps on aver age on bot h of t he cir cuit s. Ther e w as som e t im e of- day var iat ion t hat seem ed a bit unusual, but on t he w hole, t r affic w as loadbalanced quit e w ell. Out bound t r affic w as sk ew ed t ow ar d t he AS 5400 connect ion, t hough, and t her e w as clear ev idence of sev er e asy m m et r ic r out ing, w it h t r affic t o t he Unit ed St at es out bound on t he AS 5400 pat h but r et ur n t r affic inbound on t he AS 2516 pat h. To ev en t his out m or e, t he follow ing st eps w ere t aken: 1. Going back t o t he r out ing analy sis dat a, t he nex t 3 aut onom ous sy st em s in t he list of t he t op 20 w er e added t o t he AS pat h filt er list . This now pr oduced a r out ing t able of ar ound 14,000 pr efixes. 2. OSPF w as configur ed t o be t he conv ey or of t he default r out e because it w as not possible t o hear a default fr om AS 5400. The pr im ar y pat h w as t hr ough AS 2516 ( low I GP m et ric) , wit h t he backup t hrough AS 5400 ( high I GP m et r ic) . The r esult ing load balancing w as alm ost per fect , w it h out bound t r affic w ell balanced at about 200 Kbps on each pat h. Fur t her inv est igat ion show ed t hat m ost t r affic t o and fr om Eur ope w as using t he AS 5400 pat h, w hile m ost t r affic t o Asia and t he West Coast of t he Unit ed St at es w as using t he AS 2516 pat h. All in all, t his w as quit e a good r esult , and all w it h j ust 13,000 pr efixes in t he BGP t able on t he r out er s. The r out er s used w er e 2611s w it h 64 MB of m em or y . Adv ice fr om pr ospect iv e upst r eam I SPs t o AS 17660 w as t hat t hey w ould r equir e a r out er w it h a m inim um of 128 MB of m em ory, wit h t he 7204 being t he r ecom m ended m inim um m odel for any net work t o m ult ihom e. The final configur at ions of t he r out er s w er e as follow s:

Router A: router ospf 100 log-adjacency-changes passive-interface default no passive-interface Ethernet0/0

292

default-information originate metric 20 ! router bgp 17660 no synchronization no bgp fast-external-fallover bgp log-neighbor-changes bgp deterministic-med neighbor core-ibgp peer-group neighbor core-ibgp remote-as 17660 neighbor core-ibgp update-source Loopback0 neighbor core-ibgp next-hop-self neighbor core-ibgp-partial peer-group neighbor core-ibgp-partial remote-as 17660 neighbor core-ibgp-partial update-source Loopback0 neighbor core-ibgp-partial next-hop-self neighbor core-ibgp-partial send-community neighbor core-ibgp-partial prefix-list partial-ibgp out neighbor 166.49.165.13 remote-as 5400 neighbor 166.49.165.13 description eBGP multihop to AS5400 neighbor 166.49.165.13 ebgp-multihop 5 neighbor 166.49.165.13 update-source Loopback0 neighbor 166.49.165.13 prefix-list in-filter in neighbor 166.49.165.13 prefix-list out-filter out neighbor 166.49.165.13 filter-list 1 in neighbor 166.49.165.13 filter-list 3 out neighbor 202.144.159.193 peer-group core-ibgp neighbor 202.144.159.197 peer-group core-ibgp-partial neighbor 202.144.159.198 peer-group core-ibgp-partial ! ip prefix-list in-filter deny rfc1918etc in ip prefix-list out-filter permit 202.144.128.0/19 ip prefix-list partial-ibgp permit 202.144.128.0/19 le 32 ! ip route 0.0.0.0 0.0.0.0 serial 0/0 254 ip as-path access-list 1 deny _701_ ip as-path access-list 1 deny _1_ ip as-path access-list 1 deny _7018_ ip as-path access-list 1 deny _1239_ ip as-path access-list 1 deny _7046_ ip as-path access-list 1 deny _209_ ip as-path access-list 1 deny _2914_ ip as-path access-list 1 deny _3549_ ip as-path access-list 1 permit _5400$ ip as-path access-list 1 permit _5400_[0-9]+$ ip as-path access-list 1 permit _5400_[0-9]+_[0-9]+$ ip as-path access-list 1 deny .* ip as-path access-list 3 permit ^$ !

Router B: router ospf 100 log-adjacency-changes passive-interface default no passive-interface Ethernet0/0

293

default-information originate ! router bgp 17660 no synchronization no auto-summary no bgp fast-external-fallover bgp log-neighbor-changes bgp deterministic-med neighbor core-ibgp peer-group neighbor core-ibgp remote-as 17660 neighbor core-ibgp update-source Loopback0 neighbor core-ibgp next-hop-self neighbor core-ibgp-partial peer-group neighbor core-ibgp-partial remote-as 17660 neighbor core-ibgp-partial update-source Loopback0 neighbor core-ibgp-partial next-hop-self neighbor core-ibgp-partial send-community neighbor core-ibgp-partial prefix-list partial-ibgp out neighbor 202.144.159.192 peer-group core-ibgp neighbor 202.144.159.197 peer-group core-ibgp-partial neighbor 202.144.159.198 peer-group core-ibgp-partial neighbor 210.132.92.165 remote-as 2516 neighbor 210.132.92.165 description eBGP peering neighbor 210.132.92.165 soft-reconfiguration inbound neighbor 210.132.92.165 prefix-list default-route in neighbor 210.132.92.165 prefix-list out-filter out neighbor 210.132.92.165 route-map as2516-out out neighbor 210.132.92.165 maximum-prefix 100 neighbor 210.132.92.165 filter-list 2 in neighbor 210.132.92.165 filter-list 3 out ! ! ip prefix-list default-route permit 0.0.0.0/0 ip prefix-list out-filter permit 202.144.128.0/19 ip prefix-list partial-ibgp permit 202.144.128.0/19 le 32 ! ip as-path access-list 2 permit _2516$ ip as-path access-list 2 deny .* ip as-path access-list 3 permit ^$ ! route-map as2516-out permit 10 set as-path prepend 17660 ! Som e com m ent s on t he configur at ions ar e necessar y . For Rout er A: •

• •

Notic e t he default r out e point ing t o ser ial0/ 0 w it h dist ance 254. As long as t he ser ial int er face is up ( line pr ot ocol is up) , t he st at ic r out e w ill be in t he r out ing t able, t her eby causing t he de fa u lt - in for m a t ion or igin a t e in t he OSPF configur at ion t o announce t he default int o t he net work. Not ice t hat t he OSPF m et r ic has been set t o 20. eBGP m ult ihop is used as a r equir em ent fr om AS 5400. Not e t hat fiv e TCP hops have been select ed r at her t han 255, as is used by m any I SPs—it ’s bet t er pr act ice t o set t he num ber of hops r equir ed r at her t han j ust use t he lar gest num ber possible.

294

• •

Not ice in t he eBGP peer ing t hat bot h an out bound filt er list and an out bound pr efix list ar e used. This is double insur ance; if one “ goes m issing,” t he ot her one is t her e as a safet y net . Not e t he t w o peer gr oups. core - ib g p sends t he full pr efixes t o Rout er B, w hereas core - ibgp- p a r t ia l sends only int er nal pr efix es ( subpr efix es of t he aggr egat e block ) t o t he r em aining iBGP speak er s in t he net w or k . ( This net w or k has t w o r out e r eflect or s, hence t he bor der r out er s t alk only iBGP w it h t he r out e r eflect or s.)

Som e com m ent s ar e needed for Rout er B, in addit ion t o t hose for Rout er A: • •

•

Not e t hat t he default r out e is hear d fr om AS 2516. The m a x im u m - pr e fix e s BGP opt ion has been set in t he unlik ely ev ent t hat ot her par t s of t he configur at ion accident ally ar e r em ov ed. Not e t hat OSPF or iginat es a default r out e w it h m et r ic 10. This is a higher m et r ic t han t he one fr om Rout er A, so it w ill be t he pr efer r ed default r out e t hr ough t he back bone. As long as t he default is hear d fr om AS 2516, t her e w ill be a default in t he r out ing t able, w hich m eans t hat OSPF w ill or iginat e it s default t hr ough t he back bone. Not ice t he AS pat h pr epend on t he out bound announcem ent . A single 17660 is included in t he out bound announcem ent t o m ak e t he inbound t r affic flow sy m m et r ic on bot h r out er s.

Finally , a br ief look at t he t w o MRTG gr aphs for t he loading on t hese t w o r out er s’ ex t er nal int er faces show s t he r elat iv e balance bet w een t he t w o r out er s. I nbound and out bound loads are reasonably sy m m et r ic, alt hough closer ex am inat ion could r ev eal t hat t he Eur opean pat h has m or e inbound t r affic lat er in t he day t han t he Asian pat h. Reasons for t his ar e unclear at t he t im e of w r it ing—w e w ill hav e t o sit dow n and look at t he Net Flow log infor m at ion in m ore det ail. The shaded area in Figur e 5- 32 and Figur e 5- 33 r epr esent s t he inbound t raffic load; w her eas, t he solid line r epr esent s t he out bound t r affic load. Fig u r e 5- 3 2 . Rou t e r A U p st r e a m Lin k Loa d

Fig u r e 5- 3 3 . Rou t e r B Up st r e a m Lin k Loa d

295

Ov er t im e, t his configur at ion w ill hav e t o be m onit or ed because t r affic pat t er ns do change as net w or k s gr ow and t he client ele on t he net w ork com e and go. But as a case st udy for good m ult ihom ing pr act ice, t his is one of t he best effor t s seen on t he I nt er net t oday . Unlik e t he claim s fr om som e of t he pot ent ial upst r eam I SPs of AS 17660, all t his has been achievable w it h m inor rout er plat for m s and m inim um m em or y r equir em ent s ( t he t w o 2600s each st ill hav e ar ound 32 MB of m em or y spar e, plent y of r oom for fut ur e gr ow t h and r efinem ent of t he m ult ihom ing needs) .

Using Com m unit ie s Com m unit ies ar e used ex t ensiv ely in t he I nt er net indust r y for ex changing and facilit at ing policy bet w een I SPs. This sect ion only br iefly look s at som e of t he possible uses of com m unit ies and giv es som e point er s on som e im plem ent at ions of com m unit ies in t he I nt er net at lar ge. The fir st public docum ent at ion of com m unit y usage in t he I SP com m unit y w as RFC 1998, w hich docum ent ed how t he t hen I nt er net MCI used com m unit ies for cust om er s t hat m ult ihom ed ont o it s back bone. Since t his RFC appear ed, m any I SPs have applied t he concept and developed com m unit y policies for t heir ow n net w or k r equir em ent s. Ther e isn’t r eally any docum ent at ion descr ibing t he differ ent policies, but r esear ching I SP oper at ions Web sit es and t he I nt er net Rout ing Regist r y r eveals m any of t he policies in use t oday. RFC 1 9 9 8 The principle behind RFC 1998 is quit e sim ple: Make life easy for t he upst r eam I SP by st andar dizing configur at ions on t he back bone aggr egat ion r out er s, and giv e t he cust om er m or e scope t o m odify it s m ult ihom ing configur at ion w it hout inv olv ing t he upst ream I SP. Four com m unit ies ar e defined; all of t hem allow t he cust om er t o m odify t he localpr efer ence of pr efixes in t he upst r eam I SP net w or k. I f t he upst r eam I SP has AS num ber ASx , t hen t he follow ing com m unit ies ar e defined: • •

ASx :1 0 0 — Send t his com m unit y t o t he upst r eam I SP t o indicat e t hat t his is t he pr efer r ed pat h ( t he default local pr efer ence is 100) . ASx :9 0 — Send t his com m unit y t o t he upst r eam I SP so t hat t his pr efix is t agged w it h a local pr efer ence of 90. This is t he backup pat h if dual hom ed ont o ASx.

296

• •

ASx :8 0 — Send t his com m unit y t o t he upst r eam I SP so t hat t agged w it h a local pr efer ence of 80. This is t he back up pat h anot her I SP and t he pat h lengt hs are equal. ASx :7 0 — Send t his com m unit y t o t he upst r eam I SP so t hat t agged w it h a local pr efer ence of 70. This is t he backup pat h anot her.

t his pr efix is if m ult ihom ed t o t his pr efix is if m ult ihom ed t o

A w or k ed ex am ple is t he easiest w ay t o see t hese com m unit y set t ings in act ion. Consider t he diagram in Figur e 5- 23, a sit uat ion in w hich a sm all net w or k has a pr im ar y pat h and a back up pat h t o it s upst r eam I SP. Pr ev iously , a configur at ion w as descr ibed t hat used local pr efer ence and MED set t ing by t he local net w or k t o achiev e t he desir ed configur at ion. The sam e result can be achieved using RFC 1998- st y le com m unit ies; t he r out er configur at ions ar e

Router A: router bgp 65534 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.2 remote-as 109 neighbor 222.222.10.2 description RouterC neighbor 222.222.10.2 prefix-list aggregate out neighbor 222.222.10.2 route-map routerC-out out neighbor 222.222.10.2 prefix-list default in neighbor 222.222.10.2 route-map routerC-in in ! ip prefix-list aggregate permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! route-map routerC-out permit 10 match ip address prefix-list aggregate set community 109:100 route-map routerC-out permit 20 ! route-map routerC-in permit 10 set local-preference 100 !

Router B: router bgp 65534 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.6 remote-as 109 neighbor 222.222.10.6 description RouterD neighbor 222.222.10.6 send-community neighbor 222.222.10.6 prefix-list aggregate out neighbor 222.222.10.6 route-map routerD-out out neighbor 222.222.10.6 prefix-list default in neighbor 222.222.10.6 route-map routerD-in in ! ip prefix-list aggregate permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! route-map routerD-out permit 10 match ip address prefix-list aggregate set community 109:90

297

route-map routerD-out permit 20 ! route-map routerD-in permit 10 set local-preference 90 !

Router C: router bgp 109 neighbor 222.222.10.1 remote-as 65534 neighbor 222.222.10.1 default-originate neighbor 222.222.10.1 prefix-list Customer in neighbor 222.222.10.1 route-map bgp-cust-in in neighbor 222.222.10.1 prefix-list default out ! ip prefix-list Customer permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! ip community-list 70 permit 109:70 ip community-list 80 permit 109:80 ip community-list 90 permit 109:90 ! route-map bgp-cust-in permit 10 match community 70 set local-preference 70 route-map bgp-cust-in permit 20 match community 80 set local-preference 80 route-map bgp-cust-in permit 30 match community 90 set local-preference 90 route-map bgp-cust-in permit 40 set local-preference 100 !

Router D: router bgp 109 neighbor 222.222.10.5 remote-as 65534 neighbor 222.222.10.5 default-originate neighbor 222.222.10.5 prefix-list Customer in neighbor 222.222.10.5 route-map bgp-cust-in in neighbor 222.222.10.5 prefix-list default out ! ip prefix-list Customer permit 221.10.0.0/19 ip prefix-list default permit 0.0.0.0/0 ! ip community-list 70 permit 109:70 ip community-list 80 permit 109:80 ip community-list 90 permit 109:90 ! route-map bgp-cust-in permit 10 match community 70 set local-preference 70 route-map bgp-cust-in permit 20 match community 80 set local-preference 80

298

route-map bgp-cust-in match community 90 set local-preference route-map bgp-cust-in set local-preference !

permit 30 90 permit 40 100

Not ice t he sim ilar it y bet w een t he configur at ions for Rout er C and Rout er D. Apar t fr om t he I P addr esses on t he point - t o- point links, t hey are exact ly t he sam e. This is an adv ant age for t he upst r eam I SP because it has a scalable solut ion t o it s differ ent cust om ers’ m ult ihom ing needs. I t s cust om ers can m odify w hich is t he prim ary pat h and w hich is t he back up pat h sim ply by sending differ ent com m unit ies: 109: 100 m eans t hat t his is t he pr im ar y pat h, and 109: 90 m eans t hat t his is t he back up pat h. All t he configu rat ion w ork is done on Rout ers A and B. Rout er A has t he prim ary pat h and so sends com m unit y 109: 100 t o Rout er C. Rout er C m at ches t his com m unit y in t h e bgp- cu st - in r out e m ap and set s t he local pr efer ence of t he pr efix t o 100 ( w hich is t he default , in any case) . Rout er B has t he back up pat h, so it sends com m unit y 109: 90 t o Rout er D and set s local pr efer ence 90 t o any inbound announcem ent s. I f t he cust om er net w or k in AS 65534 w ant s t o change t he pat hs, t he engineer s t her e sim ply need t o sw ap ar ound w hich comm unit ies ar e sent and w hich local pr efer ences ar e set on Rout er s A and B. The NOC of AS 109 does not need t o be involved, t hus low er ing t he suppor t bur den of cust om er s, low er ing t he cost s of oper at ing t he business, w it h gener al benefit s t o bot h t he business and t he cust om er s. I SP Com m u n it y U sa g e RFC 1998 w as w r it t en sev er al y ear s ago, and since t hen, I SPs hav e r efined and enhanced w hat t hey use com m unit ies for . Many ex am ples ex ist on t he I nt er net , and a few of t hem t hat w er e publicly v isible at t he t im e of t his w r it ing ar e docum ent ed here. The fir st exam ple is fr om AS 2764, an Aust r alian- based I SP. The com m unit y policies ar e docum ent ed in t he AS obj ect st or ed in t he I nt er net Rout ing Regist r y:

aut-num: as-name: descr: admin-c: tech-c: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: notify:

AS2764 ASN-CONNECT-NET connect.com.au pty ltd CC89 MP151 Community Definition -----------------------------------------------2764:1 Announce to "domestic" rate ASes only 2764:2 Don't announce outside local PoP 2764:3 Lower local preference by 25 2764:4 Lower local preference by 15 2764:5 Lower local preference by 5 2764:6 Announce to non customers with "no-export" 2764:7 Only announce route to customers 2764:8 Announce route over satellite link [email protected]

299

mnt-by: changed: source:

CONNECT-AU [email protected] 19990506 CCAIR

The r em ar k s sect ion descr ibes t he com m unit ies suppor t ed. For ex am ple, if a cust om er of AS 2764 sends a pr efix w it h com m unit y of 2764: 2 set , AS 2764 w ill announce t he pr efix only w it hin t he local PoP. I f a cust om er sends a pr efix w it h com m unit y 2764: 8 set , t he pr efix w ill be announced only over AS 2764’s sat ellit e connect ion t o t he Unit ed St at es, and so on. The second ex am ple is fr om AS 702, a Eur opean- based I SP. I t s com m unit y policies also ar e docum ent ed in it s AS obj ect , st or ed in t he I nt er net Rout ing Regist r y:

aut-num: as-name: descr: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: mnt-by: changed: source:

AS702 AS702 UUNET - Commercial IP service provider in Europe ------------------------------------------------------------UUNET uses the following communities with its customers: 702:80 Set Local Pref 80 within AS702 702:120 Set Local Pref 120 within AS702 702:20 Announce only to UUNET ASes and UUNET customers 702:30 Keep within Europe, don't announce to other UUNET ASs 702:1 Prepend AS702 once at edges of UUNET to Peers 702:2 Prepend AS702 twice at edges of UUNET to Peers 702:3 Prepend AS702 thrice at edges of UUNET to Peers Details of UUNET's peering policy and how to get in touch with UUNET regarding peering policy matters can be found at: http://www.uu.net/peering/ ------------------------------------------------------------UUNET-MNT [email protected] 20010928 RIPE

Not ice t hat AS 702 suppor t s one of t he RFC 1998 v alues: 702: 80. I f t hat com m unit y is at t ached t o a pr efix sent t o AS 702 by a cust om er , AS 702 w ill set t he local pr efer ence t o 80. 702: 20 and 702: 30 ar e int er est ing because t hey det er m ine boundar ies about how far a pr efix w ill be announced w it hin t he AS 702 net w or k . Finally , t he 702: 1, 702: 2, and 702: 3 com m unit ies det er m ine t he num ber of aut onom ous syst em s pr epended in any announcem ent s t hat AS 702 m akes of cust om er pr efix es t o AS 702’s peer s. This giv es cust om er s t he opt ion of a pr im ar y pat h t o and fr om AS 702, but t hey possibly have a backup pat h t hr ough anot her I SP w hen t hey m ult ihom e. Or t hey can use t his com m unit y t o help fine- t une t heir m ult ihom ing load shar ing bet w een AS 702 and t heir ot her upst r eam I SPs. The t hir d exam ple is one of t he m ost com pr ehensive seen in use in t he I nt er net at t his t im e. An excer pt is given in t he follow ing:

aut-num: as-name: descr: remarks: remarks:

AS5400 CIPCORE Concert European Core Network Communities scheme: The following BGP communities can be set by Concert BGP

300

remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks: remarks:

customers to affect announcements to major peerings. Community to Not announce

To peer:

Community to AS prepend 5400

5400:1000 5400:1001 5400:1002 5400:1003 5400:1005 5400:1006 5400:1007 5400:1008 5400:1009 5400:1010

European peers Ebone (AS1755) Eunet (AS286) Unisource (AS3300) UUnet (AS702) Carrier1 (AS8918) SupportNet (8582) AT&T (AS2686) Level 3 (AS9057) RIPE (AS3333)

5400:2000 5400:2001 5400:2002 5400:2003 5400:2005 5400:2006 5400:2007 5400:2008 5400:2009 5400:2010

... remarks: notify: mnt-by: source:

5400:1100 US peers [email protected] CIP-MNT RIPE

5400:2100

Cust om er s of AS 5400 hav e a lar ge v ar iet y of com m unit ies av ailable t o t hem , allow ing pr efix announcem ent s t o be m ade t o all of AS 5400’s peer s and t o m ult ihom ing load balancing t o be det er m ined bet w een AS 5400 and t heir ot her upst ream I SPs. This m odel is based on t hat used by AS 3257, w hose det ailed com m unit y policy is list ed at ht t p: / / w w w . as3257. net / ht m l/ com m unit ies. ht m. Alt hough im plem ent ing t his m ight seem unduly com plicat ed ( it is not t hat har d, j ust a very large rout e m ap) , t he benefit s for t he net w orks t hat use t his sort of schem e is t hat t heir cust om er s hav e a gr eat deal of liber t y w hen it com es t o configur ing t heir m ult ihom ing needs. Com m u n it ie s Con clu sion These exam ples hopeful ly have show n som e of t he benefit s of using com m unit ies in I SP net w or ks. A lar ge num ber of opt ions ar e possible, not j ust for m ult ihom ing, as has been cov er ed her e. Com m unit ies hav e been used t o color differ ent pr efix es for announcem ent w it hin an I SP’s ow n back bone, t o r eplace com plex ex t er nal filt er s on bor der r out er s, and t o r em ov e t he gener at ion of filt er s fr om t he bor der r out er s t o t he aggr egat ion r out er s w her e t he cust om er s fir st connect t o t he back bone. All t hese benefit s significant ly r educe t he com p lexit y of oper at ing an I SP backbone and, fr om our ex per ience, significant ly sim plify t he w hole cust om er- pr ov isioning pr ocess w it hin an I SP oper at ion. You ar e st r ongly encour aged t o st udy t he use of com m unit ies fur t her —t h e Cisco. com sit e has a consider able r ange of m at er ials, and t he Consult ing Engineer ing pages on t he Cisco. com sit e have m or e exam ples of I SP applicat ions.

Se cu r it y Chapt er 4, “ Secur it y ,” cov er ed in consider able det ail t he consider at ions t hat an I SP m ust m ake, and t he t ools available, t o pr ot ect t he I SP’s por t ion of t he I nt er net fr om pr oblem s t hat can affect t he net w or k’s per for m ance and secur it y, and t he secur it y of it s cust om er s. How ev er , t his book has not specifically cov er ed t he secur it y

301

consider at ions t hat an I SP needs t o m ak e at t he basic lev el of connect ing cust om er s and ser v er s t o t he back bone. This sect ion cov er s som e of t he bet t er secur it y pr act ices and includes sam ple t em plat es t hat can be used as a basis for t he secur it y pr ov isions on t he net w or k . This is not int ended t o be t he last w or d in secur it y ( m any t ex t s cov er net w or k secur it y in consider able dept h) , but it is int ended t o be t he absolut e m inim um t hat an I SP m ust inst all on it s backbone and for it s cust om ers.

I SP Border Pack et Filt ers The decision of w het her t o inst all pack et filt er s on t he net w or k bor der usually depends on t he size of t he I SP oper at ion and w het her t he design and oper at ions t eam feels t hat such filt er s can ser v e any useful pur pose t o pr ot ect t he back bone. There is no hard- and- fast r ule, but w e hav e found t hat sm aller I SPs t end t o im plem ent quit e sev er e filt er s on t heir net w or k edges, w hile t he lar gest I SPs pr obably im plem ent only one or t w o key filt er s t o pr event DOS at t acks on t heir net w orks. Much of any I SP filt er ing is inst alled on t he aggr egat ion r out er w her e t he cust om er s connect t o t he back bone. The r est of an I SP’s infr ast r uct ur e is cor e net w or k equipm ent , and t hat is pr ot ect ed by it s ow n set of access filt er s. I SPs t hat put w eak filt er ing on aggr egat ion r out er s ( no one should be put t ing any t hing w eak on t his par t of t he net w ork) t end t o need m uch m ore carefully t hought - out filt ering on t heir bor der r out er s. Appendix C, “ Ex am ple Configur at ions,” pr ov ides a t y pical configur at ion ex am ple of an “ I SP Essent ials” applicat ion in a sm all I SP: The bor der rout er t her e has det ailed filt er ing so t hat unw ant ed pack et s ar e not per m it t ed int o t he I SP back bone. Assum ing t hat t he I SP has net w or k block 221.19.0.0/ 19, a m inim um inbound access list on an ext er nal- facing peer ing int er face m ight be som et hing like t he follow ing:

! access-list access-list access-list access-list access-list access-list access-list access-list access-list !

100 100 100 100 100 100 100 100 100

deny ip 221.19.0.0 0.0.31.255 any deny ip 0.0.0.0 0.255.255.255.255 any deny ip 10.0.0.0 0.255.255.255 any deny ip 127.0.0.0 0.255.255.255 any deny ip 169.254.0.0 0.0.255.255 any deny ip 172.16.0.0 0.15.255.255 any deny ip 192.0.2.0 0.0.0.255 any deny ip 192.168.0.0 0.0.255.255 any permit ip any any

This block s all out side pack et s a w it h sour ce addr ess fr om t he local net w or k addr ess space ( t o block DOS at t ack s) , block s all RFC 1918 addr ess space, and block s ot her addr ess space, such as t he loopback net w or k and t he aut oconfigur at ion net w or k .

302

The out bound filt er act ually is quit e sim ilar t o t his, again filt er ing special- use address space and t he RFC 1918 block s. Tr affic w it h t hose addr esses as sour ce addr esses should not be leak ed out int o t he I nt er net :

! access-list access-list access-list access-list access-list access-list access-list !

101 101 101 101 101 101 101

deny ip 10.0.0.0 0.255.255.255 any deny ip 127.0.0.0 0.255.255.255 any deny ip 169.254.0.0 0.0.255.255 any deny ip 172.16.0.0 0.15.255.255 any deny ip 192.168.0.0 0.0.255.255 any deny ip 192.0.2.0.0.0.0.255.255 any permit ip any any

The configur at ion m ight be applied t o t he r out er lik e t he follow ing:

! interface hssi 5/0 description 34Mbps link to Galactic Internet ip address 221.1.2.1 255.255.255.252 ip access-group 100 in ! inbound filter ip access-group 101 out ! outbound filter no ip directed-broadcast ! access-list compiled ! compile access-lists ! Chapt er 4 cov er ed a r ange of ot her filt er ing opt ions available t o I SPs and t he har dw ar e suppor t av ailable in differ ent plat for m s t o deal w it h DOS at t ack s. The pr eceding list is pr obably sufficient for m ost I SPs and, w hen used as a com piled access list on higher- end plat for m s, does not pose m uch of a per for m ance penalt y on t he r out er s.

Aggregat ion Rout er Filt ers The m inim um inbound filt er t hat any I SP should be apply ing t o t he cust om er- facing int er faces on it s aggr egat ion r out er s is t he unicast RPF check . This ensur es t hat all pack et s com ing fr om t he cust om er ar e check ed t o m ak e sur e t hat t heir or igin addr ess com es out of t he addr ess block assigned t o t he cust om er . This check is m uch m or e efficient t han applying any inbound filt er s on t he aggr egat ion r out er , and it is a r ecom m ended best pr act ice t hr oughout t his book. I f t he cust om er has addr ess space 221.4.0.0/ 22, t he int er face configur at ion t o achiev e inbound filt er ing is as follows:

! interface serial 1/0 description 2Mbps link to Planet Toyshop ip unnumbered Loopback0 ip verify unicast reverse-path no ip direct-broadcast

! inbound filter

303

! ip route 221.4.0.0 255.255.252.0 serial 1/0 ! The I SP opt ionally could inst all an out bound filt er as w ell ( if t he cust om er r out er is incapable of doing filt er ing) , but m ost I SPs generally supply t em plat es or sam ple filt er s for t heir cust om er s’ use. The t y pical out bound filt er w ould look lik e t he inbound filt er discussed in t he nex t sect ion. Not ice t hat if t he cust om er uses BGP and is m ult ihom ed, t he uRPF check configur at ion needs a lit t le m ore planning —t his w as discussed in m uch m ore det ail in t he pr ev ious chapt er . But for t he v ast m aj or it y of I SPs and cust om er connect ions, t his configur at ion is all t hat is sufficient . I f ev er y connect ed end sit e had an RPF check applied on unicast t r affic, t her e w ould be a m ar ked r educt ion in t he am ount of spoofed cont ent on t he I nt er net t oday .

Cust om er Rout er Filt ers As a ser v ice t o t heir cust om er s, all I SPs should be supply ing sam ple filt er s for r out er s t hat ar e used t o connect per m anent ly connect ed net w or k s t o t he I nt er net . I f t he cust om er s ar e using r out er s t hat ar e incapable of filt er ing, t o quot e an I SP over hear d r ecent ly, “ t hat device should be r eplaced w it h a r eal r out er .” I t is an unfor t unat e fact t oday t hat t oo m any people equat e secur it y w it h fir ew alls and com plet ely fail t o r em em ber t hat a r out er is a v er y sophist icat ed fir st - line securit y dev ice in it s ow n r ight . A t ypical cust om er t em plat e m ight look like t he follow ing:

interface ethernet 0 description backbone LAN ip address 221.4.3.254 255.255.252.0 no ip directed-broadcast no ip proxy-arp no ip redirects ! interface serial 0 description Connection to Planet ISP ip unnumbered Ethernet 0 ip access-group 100 in ip access-group 101 out no ip directed-broadcast ! access-list 100 permit icmp any any access-list 100 permit tcp any any established access-list 100 permit tcp any any eq 22 access-list 100 permit tcp any host 221.4.0.1 eq access-list 100 permit tcp any host 221.4.0.2 eq access-list 100 permit udp any host 221.4.0.3 eq access-list 100 permit tcp any host 221.4.0.3 eq access-list 100 permit udp any any eq ntp access-list 100 deny udp any any eq 2049 access-list 100 permit udp any any gt 1023

www smtp domain domain

304

access-list 100 deny ip any any log ! access-list 101 permit ip 221.4.0.0 0.0.3.255 any access-list 101 deny ip any any log ! List 100 is a ver y t ight list , allow ing only a lim it ed num ber of connect ions: • • • • • • • • • •

Perm it I CMP Per m it any est ablished TCP sessions ( sessions t hat hav e been init iat ed from inside t he cust om er net w ork) Per m it Secur e Shell Per m it WWW t o t he appoint ed Web ser v er ( and m ak e sur e t hat t he Web ser v er is secur ed and r unning t he lat est soft w ar e) Per m it SMTP t o t he appoint ed m ail host ( and m ak e sur e t hat t he m ail sy st em is not r unning as a r elay, is secur ed, and is r unning t he lat est soft w ar e) Per m it DNS look ups and zone t r ansfer s t o t he appoint ed DNS ser v er ( and m ak e sur e t hat t he nam eser v er is r unning t he m ost r ecent soft w ar e and is secur ed) Per m it NTP Block UDP port 2049 ( NFS por t ) Per m it UDP on unpr ivileged por t s ( t his could be a secur it y r isk, but m ost I SPs and cust om er s ar e cont ent w it h t his) Block everyt hing else, and log it

List 101 is t he basic list , ensur ing t hat only t r affic fr om legit im at e addr ess r anges is per m it t ed out int o t he I nt er net . Don’t fall int o t he t r ap of assum ing t hat t he upst r eam I SP w ill filt er “ illegal” t r affic. This is t he absolut e m inim um necessar y configur at ion for any end- sit e rout er connect ing any sit e t o t he I nt er net . Obv iously , it is not t he last w or d in high secur it y, but w e consider it t o be m andat or y for any new I nt er net - connect ed sit e. Firew alls and ot her int er nal filt er ing also should be consider ed and im plem ent ed w it hin t he net work.

I SP Se r ve r Conside r a t ions Ser v er s w it hin t he I SP or cust omer net w or k need t o be pr oper ly set up and secur ed befor e t hey ar e plugged int o any LAN—or ev en connect ed t o t he I nt er net . I SP ser v er s should be connect ed behind a r out er w it h st r ong filt er s, nev er plugged int o t he cor e of t he backbone, as has been m ent ioned pr ev iously . Using a r out er giv es t he I SP a chance t o inst all st r ong filt er s t o pr ot ect t he ser v er s. For ex am ple, if t he ser ver is a Web ser ver , only por t 80 should be visible t o t he out side w or ld, and so on. The act ual ser v er s t hem selv es r equir e, at m inim um , TCP w r apper s t o be inst alled and all nonessent ial ser v ices t o be sw it ched off. At t he t im e of t his w r it ing, t he m ost recent release of Red Hat Linux included a default firew all inst allat ion in t he inst all pr ocess. This is v er y significant because now oper at ing syst em vendor s ar e t aking secur it y m or e ser iously . A sam ple of an ipchains configur at ion is giv en her e:

305

:input ACCEPT :forward ACCEPT :output ACCEPT -A input -s 0/0 -A input -p tcp -A input -p tcp -A input -p tcp -A input -p udp -A input -p udp -A input -p tcp -A input -p tcp -A input -p udp -A input -p udp -A input -p tcp -A input -p tcp

-d -s -s -s -s -s -s -s -s -s -s -s

0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0

-i -d -d -d -d -d -d -d -d -d -d -d

lo -j ACCEPT 0/0 ssh -y -j ACCEPT 0/0 http -y -j ACCEPT 0/0 smtp -y -j ACCEPT 0/0 ntp -j ACCEPT 0/0 domain -j ACCEPT 0/0 0:1023 -y -j REJECT 0/0 2049 -y -j REJECT 0/0 0:1023 -j REJECT 0/0 2049 -j REJECT 0/0 6000:6009 -y -j REJECT 0/0 7100 -y -j REJECT

The follow ing list descr ibes t he act ions of each line of t he ipchains configur at ion: • • • • • • • • • • • • •

Accept all pack et s fr om t he loopback int er face. Accept incom ing Secur e Shell connect ions. Accept incom ing Web connect ions ( because I ’m a Web ser v er ) . Accept incom ing SMTP connect ions ( because I ’m a m a il host ) . Accept incom ing NTP connect ions ( for t im e sy nchr onizat ion) . Accept incom ing DNS connect ions. Block TCP connect ions t o pr iv ileged TCP por t s. Block TCP connect ions t o por t 2049 ( Sun NFS) . Block UDP connect ions t o pr iv ileged UDP por t s. Block UDP connect ions t o port 2049 ( Sun NFS) . Block TCP connect ions t o por t s 6000 t o 6009 ( X Window s) . Block TCP connect ions t o por t 7100. Not ice t he –y on all t he TCP connect ions. This allow s only new TCP connect ions t o be m ade ( SYN bit set , ACQ and FI N bit s clear ed) .

I f t he ipchains/ ipt ables fir ew all soft w ar e is not par t of t he Linux dist r ibut ion, it is easy enough t o dow nload fr om m any of t he I nt er net m ir r or sit es. For com m er cial UNI X sy st em s, t he TCP w r apper soft w ar e is fr eely dow nloadable fr om t he I nt er net . I nst allat ion inst r uct ions and ex am ples ar en’t giv en her e —t her e ar e sufficient exam ples on t he I nt er net , as a sim ple sear ch on Google show s. I f non- UNI X–based oper at ing sy st em s ar e being used, secur it y needs t o be considered even m ore carefully —t hose oper at ing sy st em s t end t o be aim ed at t he m ass m ar k et , w her e conv enience of t he desk t op is consider ed alm ost at t he exclusion of any sem blance of secur it y. The sam e r ules apply her e. Any syst em s being connect ed t o t he public I nt er net need sufficient filt er s; pur chasing fir ew all soft w ar e for t hese sy st em s is an ex cellent choice.

Fir e w a lls Fir ew alls hav e t heir place w it hin t he I nt er net back bone, m or e gener ally for pr ot ect ing end- sit e net w or ks t han for use w it hin I SP backbones. Som e I SPs inst all fir ew alls bet w een t heir net w or k s and t he I nt er net , but t hese ar e r ar e and hav e v er y par t icular

306

r easons for doing so. Most I SPs r ely on t he t y pe of pack et filt er ing descr ibed pr ev iously . How ev er , for end sit es, a fir ew all has becom e alm ost t he de fact o r equir em ent in t he I nt er net t oday. I n t he early t o m id- 1990s, fir ew alls w er e for t he secur it y- obsessed, but t oday , w it h t he lar ge num ber of at t ack s on net w or k s, ex ploit ed host s, alt er ed Web sit es, and so on, t he fir ew all has becom e necessar y for t he lar gest cor por at ion down t o t he hom e user sit t ing at end of a dialup or ADSL link . The per sonal fir ew alls av ailable w it h Linux hav e been m ent ioned alr eady , and an ex am ple of t he configur at ion for ipchains w as giv en pr ev iously . Fir ew alls t ak e all for m s, fr om dedicat ed har dw ar e such as t he Cisc o PI X dev ice, t o soft w ar e such as ipchains r unning on Linux sy st em s. What ev er t ur ns out t o be t he m ost appr opr iat e solut ion, it s use is st r ongly r ecom m ended for any edge sit e connect ing t o t he I nt er net .

Re m ot e Acce ss For I SP engineer s, t he best w ay of accessing a net w or k in r ecent year s has t o be use Secur e Shell. I n t he ear ly day s of t he I nt er net , Telnet w as quit e popular , but it now has lar gely been abandoned in t he developed I nt er net as being far t oo insecur e and r isk y t o use. Com m unicat ion bet w een host and client is unencr y pt ed; passw or ds t o log in ar e unencr y pt ed. Any one sniffing pack et s or snooping on a net w or k has im m ediat e access t o t he net w or k , a r isk t hat is t oo big for m ost I SPs. The pr efer r ed m et hod is t o set up t he Secur e Shell ser ver on a chosen host ( or t w o) inside t he net w or k . The t r av eling engineer t hen can dial up and has secur e access t o t he cor e back bone t hr ough t hese ser v er s. Rout er s now hav e Secur e Shell ser v er and client suppor t on t hem , so Telnet can be disabled on r out er s for access purposes. Secur e Shell soft w ar e is fr eely av ailable: OpenSSH is dow nloadable fr om m any m ir r or sit es on t he I nt er net and is an ex am ple of one of t he m any pack ages av ailable on t he I nt er net . Of cour se, m any com m er cial im plem ent at ions of Secur e Shell ex ist as w ell, but it is r eally up t o t he I SP oper at or t o decide w hat t o use. I n t he spir it of t he I nt er net , m any I SP engineer s pr efer t o use Open Sour ce public - dom ain soft w ar e. Som e or ganizat ions ar e look ing at using I PSec - based solut ions for r em ot e access. Alt hough t hese w or k v er y w ell, t her e seem s t o be a pr efer ence t o use sim pler , less com plicat ed soft w ar e for r em ot e access. I PSec solut ions pr ov ide a dedicat ed t unnel bet w een I SP engineer lapt op and t he hom e sit e; w her eas, SSH can t unnel t he select ed TCP por t s, leav ing dir ect access t o t he I nt er net for t he r est of t he engineer ’s needs. ( This isn’t m eant t o be a com par ison or discussion of t he capabilit ies of SSH and I PSec; it ’s m or e a r ealit y com par ison of w hat I SP engineer s ar e doing t oday.) Lik ew ise, Secur e Shell client s ar e com m er cially av ailable or fr eely dow nloadable fr om t he I nt er net , and t hey suppor t m ost oper at ing sy st em s av ailable t oday . Telnet , RSH, and sim ilar insecur e r em ot e access pr ot ocols can be consigned t o hist or y for good.

307

Out - o f- Ba n d M a n a ge m e n t The out - of- band m anagem ent net w or k is t he m ost necessar y par t of t he I SP’s oper at ion, yet it is so r ar ely found out side t he lar gest oper at ions. Out of band quit e sim ply m eans t hat t he I SP oper at ions st aff has a m eans of get t ing access t o t he net work infrast r uct ur e w hen t he m ain link s t o t he sit e ar e dow n. When I SPs w it hout an out - of- band net w or k ar e asked w hat t heir st r at egy is w hen t hey can’t get in- band access t o t heir equipm ent , t he r esponse is oft en along t he lines of sending som eone w it h a lapt op t o connect t o t he console por t of t he r out er . Alt hough t his m ight w or k for a cent r ally m anaged sit e, it is a ludicr ous concept for any I SP t hat has m or e t han one locat ion. As m ent ioned ear lier in t his chapt er , t he goal of any I SP is t o pr ovide as close t o 100 per cent av ailabilit y of t he net w or k as is feasible. Put t ing an engineer in a car or an airplane t o fix a problem can m eans hours of dow nt im e, ir r it at ed cust om er s, and loss of r evenue. But it is quit e am azing t hat so m any I SPs r ely on t his st r at egy w hile claim ing four or fiv e 9s ( 99.99 per cent or 99.999 per cent ) of net w or k av ailabilit y .

M ode m The out - of- band net w ork can t ake m any form s. The sim plest is inst alling a m odem on t he console por t of t he equipm ent in t he r em ot e sit e and t hen cascading t he aux iliar y por t s t o t he console por t s of t he r em aining equipm ent . This w or k s as a useful last r esor t , but it get s ver y unw ieldy w it h m or e t han t w o or t hr ee it em s of equipm ent . Put t ing t w o or t hr ee m odem s in t he PoP can help w it h t his, but t he solut ion generally is not v er y scalable. At least t he pr inciple of out - of- band connect ion is t her e, t hough. I f all t he connect ions t o t he sit e go dow n, t he I SP engineer s st ill can dial int o t he PoP, connect t o a r out er , and m aybe t r y t o diagnose w hat t he pr oblem s m ight be. Also, if t hey ar e w or k ing r em ot ely on a piece of equipm ent , m ak ing a m ist ak e t hat cut s off access easily can be r epair ed w it h a sim ple connect ion t hr ough t he dialup lines. I ndeed, one of t he best w ay s of est ablishing w het her t her e has been a com plet e sit e failure is det er m ining t hat t he dialup m odem isn’t r esponding. Then t her e is pr obably lit t le point in dispat ching an engineer ; t im e t hen is bet t er spent cont act ing t he sit e m anagem ent t eam t o r est or e pow er or cor r ect ing w hat ev er failur e m ight hav e happened.

Con sole Se r ve r The next logical st ep from using one m odem or m ult iple m odem s is t o inst all a console ser v er . This is a dev ice t o w hich all t he equipm ent consoles ar e connect ed. The m ost popular console server in recent years has been t he Cisco 2511 rout er—it has 1 Et her net , 2 WANs, and 16 asy nchr onous ser ial por t s. Or iginally int ended as a dialup rout er, it has long since been ret ired from t hat funct ion in m ost I SPs and now is used as a console ser ver t hr oughout m any I SP backbones. The configur at ion is act ually quit e sim ple. The Et her net por t of t he r out er is connect ed t o t he m anagem ent LAN in t he PoP, pr ov iding access t o for t he NOC

308

engineer s w hen t he net w or k is fully funct ional. When t he PoP is disconnect ed fr om t he back bone, t he m odem at t ached t o t he aux iliar y por t allow s dialup connect ions t o be m ade t o t he r out er , pr ov iding full access t o each piece of equipm ent in t he PoP. A sam ple configur at ion for t he console ser v er follow s:

! ip alias 221.1.0.1 2001 ip alias 221.1.0.2 2002 ip alias 221.1.0.3 2003 ip alias 221.1.0.4 2004 ip alias 221.1.0.5 2005 ip alias 221.1.0.6 2006 ip alias 221.1.0.7 2007 ip alias 221.1.0.8 2008 ip alias 221.1.0.9 2009 ip alias 221.1.0.10 2010 ip alias 221.1.0.11 2011 ip alias 221.1.0.12 2012 ip alias 221.1.0.13 2013 ip alias 221.1.0.14 2014 ip alias 221.1.0.15 2015 ip alias 221.1.0.16 2016 ! line 1 location Console of PoP-cr1 no exec transport preferred none transport input telnet line 2 location Console of PoP-cr2 no exec transport preferred none transport input telnet line 3 location Console of PoP-br1 no exec transport preferred none transport input telnet line 4 location Console of PoP-gw1 no exec transport preferred none transport input telnet line 5 location Console of PoP-gw2 no exec transport preferred none transport input telnet line 6 16 location Spare no exec transport preferred none transport input telnet !

309

Not e t he use of t he ip a lia s com m ands in t he configur at ion. This allow s t he I SP t o insert a m apping of t he console por t nam e t o t he addr ess and por t so t hat t he NOC oper at ions st aff can access t he console por t in quest ion w it hout hav ing t o r em em ber addr esses or por t num ber s. For ex am ple, t he nam e console.pop- cr 1.isp.net could be m apped t o t he addr ess 221.0.1.1 in t he I SP’s DNS.

Ou t -of- Ba nd I SDN Som e I SPs use I SDN as t he m eans of accessing t heir out - of- band m anagem ent sy st em . I SDN is r eadily av ailable in m any count r ies, oft en at a cost not t oo dissim ilar t o PSTN, but w it hout t he need for an ex t er nal m odem at t ached t o t he r out er . Ex per ience has show n t hat m odem s can becom e fault y , so hav ing t he enhanced r eliabilit y av ailable t hr ough I SDN is an adv ant age for som e. An added adv ant age is t hat I SDN suppor t s 128 Kbps, m ak ing r eal- t im e upgr ades of r out er or sw it ch im ages t hat m uch m ore pract ical t han over a V.34 m odem link.

Ou t -of- Ba nd Cir cuit s The t er m inal ser v er usually is connect ed t o t he m anagem ent LAN in t he PoP. But if access t o t he PoP is com plet ely disconnect ed fr om t he out side w or ld because t he I SP’s equi pm ent com plet ely fails or all t he t elco cir cuit s fail, t he I SP st ill w ill need access t o t he equipm ent t o aid in r est or ing t he ser v ice. ( For ex am ple, t he t w o cor e r out er s m ight hav e cr ashed, causing all ex t er nal access t o be disconnect ed. Regaining access t o t hese r out er s giv es t he NOC som e chance of r est or ing connect iv it y t o t he PoP. ) This is w her e I SPs w ill inst all a separ at e WAN backbone on a differ ent t elco infr ast r uct ur e sim ply t o ser v ice t he out - of- band m anagem ent net w or k. ( Of cour se, w hen t he ent ir e t elco infr ast r uct ur e fails int o t he sit e, t her e is lit t le t hat any one can do and lit t le advant age t o having an out - of- band net w or k pr ovision t hat can w or k ar ound t his pr oblem .) The ser ial por t s on t he 2511 r out er ar e connect ed t o t he r em ot e sit es in w hat ever efficient pat t er n is r equir ed and ar e connect ed dir ect ly t o t he NOC for im m ediat e access t o t he r out er s. Redundancy also can be built int o t he net w or k , and an I GP can be used t o car r y r out ing infor m at ion for t his net w or k.

Test ing Out of Ba nd I t goes w it hout say ing t hat t he out - of- band provisioning at each I SP PoP should be t est ed on a r egular basis. Alt hough t he r egular access t o t he out - of- band net work m ight be over t he I SP’s backbone t hr ough it s m anagem ent net w or k, it is r eally im por t ant t hat t he back up pat hs are t est ed as w ell. This is especially t rue w hen a m odem or I SDN is used as t he backup pat h. Ext ernal m odem s can be not or iously unr eliable w hen t hey ar e needed m ost , so doing a daily check t o ensur e t hat t he out - of- band phone line and m odem work is an essent ial par t of any NOC’s act iv it ies. Lik ew ise, check ing t hat t he I SDN link is w or k ing is good pr act ice. I n som e count r ies, t he t elco has been k now n t o “ r et ir e” an I SDN line t hat has not been used for som e t im e, so w hen t he I SP requires it , a “ servic e

310

unobt ainable” m essage is r eceiv ed r at her t han t he connect ion t hat w as hoped for . A daily check m ak es sense her e, t oo.

Com m e nt a r y Given how st r aight for w ar d it is t o inst all out - of- band m anagem ent for a net work, it is cont inuously sur pr ising t hat so m any ISPs t r y t o m anage w it hout it . Using t he expense of a t er m inal ser ver as an excuse for not doing it is quit e unj ust ified, consider ing t he ex pense differ ence bet w een t he equipm ent for t he r est of t he net w or k and t he out - of- band dev ice com par ed t o t he r ev enue gener at ed by t he business. Most I SPs act ually w onder how t hey m anaged w it hout t he out - of- band facilit y aft er t his has been dem onst r at ed t o t hem and t he capabilit y has been inst alled on t he net work.

Te st La bor a t or y As w it h out - of- band m anagem ent , t he ot her essent ial part of an I SP’s operat ion is t he t est labor at or y. A lar ge num ber of I SPs m ake one a cor e r equir em ent of t heir business, y et it is v er y sur pr ising how m any new com er s t o t he indust r y seem t o t hink t hat t hey can m anage fine w it hout one. The t est laborat ory is used for several purposes: • • • • •

To t est new har dw ar e befor e it is deploy ed in t he field To t est new soft w ar e befor e it is deployed in t he field To t est failur e scenar ios, w or k ar ounds, and so on, t o av oid hav ing t o do t hese t est s on t he live net w or k To t est new pr oduct s, especially connect ion ser v ices befor e t hey ar e deploy ed To r un pilot s or bet a v er sions of ser v ices, fut ur e soft w ar e, and fut ur e h ar dw ar e pr odu ct s

No doubt ot her possibilit ies ex ist , t oo, but t hese ar e t he m ain ones consider ed by I SPs. Any t hing new being dev eloped for t he I SP business has t he pot ent ial t o im pact exist ing ser vices. As m ent ioned ear lier in t he chapt er , if t he I SP’s goal is t o oper at e a business w it h as close t o 100 per cent upt im e as possible, doing any t hing t o affect t his goal w ill r esult in angr y cust om er s and loss of business. Hav ing a locat ion t o t est concept s w it hout im pact ing on t he day - t o- day business is crucial t o t he operat ional fut ur e of t he com pany .

Test ing N ew Hardw are and Soft w are Wit h t he const ant developm ent of new t echnology , connect ion and back bone dev ices ( r out er s, sw it ches, and so on) ar e becom ing ev er m or e sophist icat ed and fast er . New int er faces ar e being dev eloped, new WAN t echnologies ar e deploy ed, and pow er ful new feat ur es ar e being added t o oper at in g sy st em soft w ar e. An I SP w it hout a t est labor at or y has lit t le choice but t o connect new pr oduct s ( pot ent ially running new soft w are) direct ly t o it s net w ork backbone. Many m aj or

311

t r ouble spot s in I nt er net hist or y hav e been caused by such int r oduct ions int o liv e r unning net w or ks. Whet her it w as bugs in r out er soft w ar e or bugs in equipm ent such as ATM or Fr am e Relay sw it ches, all hav e r esult ed in dow nt im e ex t ending t o sev er al day s and affect ing m or e t han j ust t he I SP t hat im plem ent ed a change w it hout any t est ing. Per for m ing t he cor r ect t est pr ocedur e befor e any t hing new is int r oduced t o t he liv e oper at ional net w or k inv olv es a consider able am ount of t im e in t he t est lab befor e car eful deploy m ent t o t he back bone. The follow ing list is t y pical of m any net w or k s: St e p 1 . I nst all har dw ar e/ soft w ar e in t he t est lab. St e p 2 . Run for one w eek and obser v e per for m ance. Wor k w it h t he v endor t o r esolv e any issues. I f t her e ar e soft w ar e bugs, obt ain a fix ed v er sion of t he soft w ar e and r epeat t his st ep. St e p 3 . Aft er a successful r un for a w eek, im plem ent som e load on t he t est lab net w or k. The best w ay of doing t his is t hr ough a t r affic gener at or . Again, if t here are perform ance issues under load, revert t o St ep 2 and carry on unt il t he load t est can r un successfully for a w eek. St e p 4 . Deploy t he new har dw ar e/ soft w ar e on a less cr it ical par t of t he back bone. For ex am ple, inst all soft w ar e at a PoP w it h few er cust om er s or a lower revenue- gener at ing r at e. I f t her e ar e liv e pr oblem s, t he r ev enue im pact is m inim ized in t im e of failure. I f t her e ar e pr oblem s w it hin a w eek of t his t est , r ev er t t o St ep 2 aft er r esolv ing pr oblem s w it h t he v endor . St e p 5 . Engineer ing appr ov al follow s. The soft w ar e or har dw ar e is deem ed suit able for w ider deploy m ent in t he back bone, so a phased inst allat ion over a few w eek s can be planned. Again, caut ion is necessar y . - I f new r out er soft w ar e is being deployed, PoPs designed w it h har dw ar e r edundancy should have only one of t he pair of r out er s upgr aded. I f t her e is a m aj or failur e, t he ot her r out er in t he pair can t ake over t he load and provided cont inued ser vice unt il t he pr oblem is fixed. - I f new har dw ar e is being deploy ed, t he har dw ar e r edundancy adv ant age in t he PoP should be used. For ex am ple, if t w o cor e r out er s ar e t o r eceiv e new WAN int er face car ds, inst all one car d one w eek and t he ot her car d t he follow ing w eek . This ensur es t hat if t her e is a field pr oblem w it h t he car d or t he soft w ar e dr iv er s, t he ot her r out er w ill be in a st able, k now n st at e. St e p 6 . Cont inue t his pr ocess unt il t he w hole net w or k has been deploy ed w it h t he new har dw ar e/ soft w ar e. The t est lab is a cr it ical par t of t his deploy m ent phase. For ev er y st ep in w hich a pr oblem has been obser v ed, t he t est lab is used t o t r y t o w or k out w hat t he pr oblem is. The m ain net w ork receives m inim al impact , and t he cust om er s connect ed t o it ar e m inim ally affect ed.

312

D e signing a Te st La b Fr om t he descr ipt ion of t he pr ev ious t est scenar io, it should be quit e clear w hat com ponent s m ake up an I SP’s t est lab. Som e I SPs build a r eplica of one of t heir PoPs; ot her s sim ply hav e a few of t he m aj or dev ices used on t heir back bone connect ed in a sim ple net w ork. Pr efer ence usually is giv en t o t he PoP r eplica design because it becom es v er y sim ple t o r eplicat e pr oblem s t hat occur on t he live net w or k. Sever al I SPs even hav e t h is t est lab as part of t heir backbones —t he lab w on’t t ak e an act iv e par t in t he I GP or t he iBGP, but it is close enough t hat t his could be done, if necessar y. Fur t her m or e, t his t est lab can be added t o t he I SP’s oper at ional pr ocedur es so t hat st aff t raining on new net w or k designs and oper at ional pr act ices can t ak e place on “ liv e” net w or k equipm ent t hat looks like t he real net w ork. A m aj or issue w it h som e I SPs is j ust ifying funding for a t est net w ork. For experienced I nt ernet engineers, t his is an ext ra or dinar y issue t o hav e. Pr ov iding an “ alw ay s on” ser v ice w it h a t echnology t hat is dev eloping r apidly m eans t hat hav ing spar e equipm ent ar ound t o t est t hings on has alw ay s been par t of t heir t ool set . To find new com er s naiv ely assum ing t hat t hey w ill nev er t est any t hing is quit e har d t o under st and—and t his is not par t icular ly helped by equipm ent v endor s w ho do not act iv ely suppor t t he need for such a t est com ponent in new back bones. How ev er t he funding m odel is designed in any business, t he engineer s r esponsible for t he net w or k need t o be cr eat iv e in ar r anging for t heir t est net w or k r equir em ent s. •

•

•

Rapidly gr ow ing net w or k s could pur chase equipm ent in adv ance of r equir em ent s and inst all it in t he t est lab. When t he equipm ent is needed in t he back bone, new t est equipm ent can be pur chased and t he old t est equipm ent can be m ov ed fr om t he lab t o t he liv e net w or k . The ot her adv ant age of t his t echnique is t hat equipm ent has had “ bur n- in” t im e —if it has been working for t hree m ont hs in t he lab, it is known t o work and t her e ar e unlikely t o be sur pr ises. Many I SPs m aint ain a st ock of spar es, w het her t hey ar e spar e par t s or w hole r out er s/ sw it ches. They opt for br eak / fix m aint enance cont r act s fr om t heir v endor s ( w hich m eans t hat t hey can send fault y par t s back t o base for r eplacem ent ) and r eplace fault y har dw ar e fr om t heir st ock . I nst ead of hav ing t he st ock sit t ing in a cupboar d or st or e r oom , m any I SPs opt t o have “ hot spar es,” w it h t he equipm ent pow er ed up and oper at ing as t he t est lab. This does m ean t hat t he t est lab is r aided w hen a back bone com ponent fails, but r aiding a noncr it ical par t of t he liv e net w or k is pr efer able t o an out age. Vendor s such as Cisco offer heav ily discount ed pr ices on equipm ent t hat w ill only ever be used in a t est laborat ory. The I SP m ust undert ak e t hat t he equipm ent w ill be used only for t est ing pur poses, but t hat is a sm all pr ice t o pay for allow ing t he business t o ensur e t hat it can t est new pr oduct s, soft w ar e, ser v ices w it hout t he m aj or financial bur den of buy ing st andar d- pr ice equipm ent pure ly for t est ing pur poses.

313

Com m e nt a r y Giv en t he suggest ions in t his sect ion, it is quit e easy t o j ust ify t he inst allat ion of a t est lab. A t est lab should be consider ed as essent ial t o t he I SP net w or k as t he m aj or par t of t he backbone, and it cont r ibut es subst ant ially t o t he oper at ional r eliabilit y of t he net work.

Ope r a t ion a l Con side r a t ion s Why design t he w or ld’s best net w or k w hen good oper at ional pr act ices hav e not been consider ed? This m ight be such an innocent quest ion, but it is surprising how m any new I SPs com plet ely for get about any oper at ional pr act ices for t heir net w or k s. The best designed net w or k can w or k only as w ell as t he oper at or s w ho ar e r unning it . Lik ew ise, good oper at ional pr act ices oft en can m ak e up for a lot of deficiencies in t he physical lay out of net w or k s. This sect ion highlight s som e of t he issues t hat need t o be consider ed for t he operat ional part of an I SP net w ork.

Maint enance Changes should nev er ev er be m ade on t he liv e r unning net w or k—per iod. Oper at or s w ho m ake configur at ion changes on t he liv e r unning net w or k dur ing peak t r affic per iods cause m ost pr oblem s, accident s, and disast er s on I SP back bones. A new I SP should est ablish w hen m aint enance w ill be car r ied out on t he net w or k — any w her e on t he net w or k . These t im es should be published on t he operat ions Web pages. Cust om er s also should be t old quit e clear ly and specifically w hen t he at - risk per iods ar e in t he back bone so t hat ex pect at ions ar e set and t her e ar e no sur pr ises. When should m aint enance per iods be set ? Fir st , t he t im e of day m ust be est ablished. This is done by looking at t raffic profiles—t he per iod of low est t r affic is t he t im e w hen t he m aint enance should be car r ied out . For m ost t y pical net w or k s, t his is bet w een 4 a.m . and 7 a.m . Next , t he day of t he w eek needs t o be est ablished: •

•

•

Doing m aint enance on a Monday m akes lit t le sense unless t he oper at ions t eam w ant s t o w or k all w eekend pr epar ing for it . Besides, for sm aller I SPs w it hout a 24- hour oper at ions t eam on sit e, get t ing up and going t o t he sit e at 4 a.m . on a Monday m ornin g is psy chologically har d t o do. Doing m aint enance on a Friday m akes lit t le sense, t oo. Apart from t he w ellk now n I P engineer m ax im um of nev er t ouching any t hing on Fr iday , doing w or k on Fr iday inv ar iably m eans spending all w eek end clear ing up any pr oblem s t hat m ight have occur r ed dur ing t he m aint enance w or k. So, t his leav es Tuesday , Wednesday , and Thur sday . Tw o per iods per w eek m ake m or e sense t han j ust one. A Tuesday m aint enance follow ed by a Thur sday m aint enance allow s for spillov er of Tuesday ’s w or k t o Thursday.

314

I ndeed, a Tuesday/ Thursday opt ion is t he popular opt ion for m any I SPs in t he indust r y t oday . Tuesday m aint enance allow s a Tuesday / Wednesday cleanup, and Thur sday m aint enance allow s a Thur sday / Fr iday cleanup, not im pinging on t he w eekend. Her e’s t he sum m ar y of t his: All m aint enance should be car r ied out on Tuesday and Thur sday, bet w een t he hour s of 4 a.m . and 7 a.m . I f t his m akes sense for t he net w or k, publicize it and don’t do any w or k, not m at t er how t r ivial, on t he backbone equipm ent out side t hese hours.

N et w ork Operat ions Versus Cust om er Support Net w or k oper at ions ar e v er y differ ent fr om cust om er suppor t . Tr y ing t o suppor t t he net w or k w it h t he sam e t eam t hat is suppor t ing cust om er s m eans t hat t he net w or k w ill becom e neglect ed in t he face of cust om er dem ands. And in t im es of net w ork failur e, cust om er suppor t w ill spend all it s t im e dealing w it h phone calls r at her t han fixing t he breakage in t he net w ork. As t hey gr ow fr om being m or e t han a few - person operat ion, m ost I SPs very quickly split cu st om er suppor t fr om t he net w or k oper at ions t eam . Net Ops or t he NOC hav e a differ ent cont act num ber ( or differ ent ent r y in t he call- m anagem ent syst em ) and gener ally cannot be cont act ed by cust om er s. Cust om er suppor t is usually answ er able t o t he sales or ganizat ion w it hin t he I SP; w her eas, t he NOC is answ er able t o t he net w or k engineer ing or oper at ions div ision. They hav e differ ent pr ior it ies, focuses, and r esponsibilit ies. All t his m eans t hat t he net w or k w ill be fixed in t he t im e of cr isis, w it hout cust om er s get t ing in t he w ay of st aff t r y ing t o solv e t he pr oblem s. Escalat ion fr om cust om er suppor t can be t o t he NOC, if it is clear t hat t he pr oblem t hat a cust om er is ex per iencing is caused by a m isconfigur at ion on t he back bone, ex t er nal connect ions, or a pr oblem som ew her e w it hin t he back bone. This escalat ion pat h should be m ade clear w it hin t he I SP oper at ion. Escalat ion fr om t he NOC can be t o t he net w or k oper at ions and net w or k engineer ing t eam s. I t is unusual, and should be discour aged, for t he cust om er suppor t t eam t o escalat e dir ect ly t o engineer ing because t he NOC is t he t eam t r ained t o deal w it h t he first - level problem s wit hin t he net work. I f an I SP cannot affor d t o have a NOC ( because it is t oo ear ly in t he st ar t up process) , arm ing engineers w it h a pager or a m obi le phone and oper at ing an on- call r ot at ion is t he best alt er nat iv e. This is ant isocial for t he engineer s w ho ar e on t he r ot at ion ( due t o t he pot ent ial fr equency of calls and ser ious disr upt ion t o sleep and non- w or k act iv it ies) , so t his should be consider ed only as a last r esor t or should be done only dur ing t he init ial st ar t up st ages in any net w or k.

En gin e e r in g The final t eam r equir ed w it hin an I SP is t he engineer ing t eam . This t eam designs t he net w or k, plans t he next phase, and fixes any oper at ional pr oble m s t hat cannot be handled by t he NOC. Quit e oft en, lar ger I SPs div ide engineer ing int o sy st em s and net w or k engineer ing, and t hey oft en div ide t his fur t her bet w een engineer ing and

315

oper at ions. This leaves a w ell- focused t eam for each par t of t he I SP net w or k , w it h no conflict of int er est and a good oper at ional pr ocess.

Cha nge M a na gem ent Change m anagem ent is about docum ent ing w hat w or k w ill be done in t he net w or k , w hat t he im pact w ill be, w hat t he backout pat h w ill be, and how long it w ill t ake. Docum ent ing changes ensur es t hat t he net w or k is fully docum ent ed, t hat t her e is a case hist or y of changes m ade, and t hat t he or igin of new pr oblem s t hat ar ise can be t raced back m ore easily. Ba ck gr ou n d When a change- m anagem ent pr ocess is int r oduced int o an I SP business, it oft en is seen as a r est r ict ion t o pr ogr ess, st opping changes and put t ing added bur eaucr acy in t he pat h of good business and gr ow t h. How ev er , w it h m or e people oper at ing t he net w or k, it becom es har der t o t r ack t he changes being m ade in t he net w or k, even if a st r ict m aint enance per iod had been est ablished. I SPs ar e not t he only or ganizat ions w it h a change- m anagem ent pr ocess—ev en m edium t o lar ge cor por at e net w or k s hav e such a sy st em . Ex am ples of a change- m anagem ent sy st em can be found at t hese locat ions: ht t p: / / w w w .cisco.com / w ar p/ cust om er / 432/ change- m gm t .ht m l ht t p: / / w w w .cisco.com / w ar p/ public/ 126/ chm gm t . h t m l Bot h docum ent t he necessar y qualit ies of a change- m anagem ent sy st em and t he pr ocesses and best pr act ices t hat should be im plem ent ed. I SP Pr a ct ice s Change- m anagem ent m eet ings t ake place t he day befor e t he m aint enance is due t o happen, w it h t he net w ork m anager being able t o assess w het her one pr oposed change w ould collide or conflict w it h any ot her pr oposed changes. Docum ent ing t he change pr ocess and t he back out pr ocess, ensur es t hat r em ot e hands can do t he work, t hat no part of t he process is m issed o r ov er look ed, and t hat t her e is a higher lik elihood of t he change act ually w or k ing as pr oposed. When t he w or k is act ually due t o t ak e place dur ing t he m aint enance per iods, t he oper at ions st aff m em ber s r esponsible for im plem ent ing changes follow t he det aile d list of t ask s as docum ent ed in t he change- m anagem ent pr ocess. Quit e oft en t he engineer w ho descr ibed t he changes necessar y also is pr esent , eit her leading t he t eam or ser ving as par t of t he t eam of engineer s m aking t he changes. Alt hough it is r ecom m ended t o include ev er y conceiv able det ail, m or e t han lik ely som e ov er sight has been m ade in t he m or e com plicat ed t asks, and it is oft en easier for t he inst igat or of t he changes t o be inv olv ed fr om t he st ar t r at her t han hav ing t o be r aised fr om sleep in t he sm all hours of t he m orning. Aft er t he w or k has been com plet ed, docum ent ing t he w or k done in t he changem anagem ent for m is r ecom m ended. For ex am ple, if som et hing goes w r ong or t her e

316

ar e par t icular ov er- sight s, a fut ur e w or k plan can r ev iew past ex per iences t o ensure t hat t he issues faced pr ev iously do not r ecur . I t is our ex per ience t hat a net w or k can funct ion pr oper ly only w hen a changem anagem ent pr ocess has been est ablished. I f one engineer is r unning ev er y t hing, such a pr ocess is pr obably ov er k ill, but a business w it h ev en half a dozen engineer s in oper at ions r esponsible for differ ent par t s of t he net w or k m eans t hat such a syst em is necessar y. A net w or k has so m any com plex int er act ions t hat only a r eview of pr oposed w or k can fully est ablish w het her one change w ill hav e an im pact any w her e else on t he backbone.

Su m m a r y This chapt er pr ovided an over view of m any of t he facet s of designing and oper at ing an I SP back bone. This included PoP t opologies and design, back bone design, I SP ser v ices, and t he dev elopm ent of IPv 4 addr essing plans. The chapt er also cov er ed int er ior and ex t er ior r out ing pr ot ocols, int r oduced sim ple t echniques for m ult ihom ing, and t ouched on som e of t he basic r equir em ent s for secur it y on an I SP back bone. The r em aining par t s of t he chapt er considered out - of- band m anagem ent for t he net w ork and t he im por t ance of a t est lab, befor e finishing w it h a br ief look at som e of t he k ey operat ional requirem ent s in t he net w ork. Consider ably m or e det ail is possible in t hese t opics—t his chapt er has sim ply been an ov er v iew t o aid new com er s t o t he I SP indust r y in get t ing t he best st ar t for t heir business, t o help t hem be aw ar e of needs, and t o offer a fast t r ack int o t he w or ld t hat is t he I nt er net .

En dn ot e s 1. The exact URL is ft p: / / ft p.isi.edi/ t ools/ r a . The ver sion at t he t im e of t his w r it ing w as t r ee- 2.1.5.t ar .Z. 2. RI PE- 229 docum ent s t he r ecom m ended coor dinat ed r out e flap dam ping par am et er s ( ft p: / / ft p.r ipe.net / r ipe/ docs/ r ipe- 229.txt ) . The t echnical back gr ound of r out e- flap dam ping is discussed in Chapt er 3. 3. The privat e AS range is 64512 t o 65534. AS 65535 is r eser v ed. 4. Not e t hat som e I SPs pr efer t o send t r affic over pr ivat e peer ings r at her t han t heir I XP peer ings. This case is only an ex am ple, and policies bet w een t he local AS and it s pr ivat e peer s should be set up accor ding t o local condit ions and r equir em ent s. For ex am ple, an I SP m ight w ant t o send all t r affic ov er t he I X apart from delay- sensit iv e t r affic or t r affic fr om special cust om er s t hat goes over t he privat e peering link.

317

Appendix A. Access List s a nd Regula r Ex pressions This appendix pr ov ides a handy r efer ence for t he access list t y pes suppor t ed in Cisco I OS Soft w ar e at t he t im e of t his w r it ing. I t also pr ov ides som e ex am ples of BGP p at h- filt ering regular expressions in com m on use. As always, refer t o t he online docum ent at ion for t he m ost up- t o- dat e inform at ion.

Acce ss List Ty pe s < 1- 9 9 >

I P st andard access list

< 100- 1 9 9 >

I P ex t ended access list

< 200- 2 9 9 >

Pr ot ocol t y pe- code access list

< 700- 7 9 9 >

48- bit MAC addr ess access list

< 1100- 1199>

Ext ended 48- bit MAC addr ess access list

< 1300- 1999>

I P st andar d access list ( expanded r ange)

< 2000- 2699>

I P ex t ended access list ( ex panded r ange)

com piled

Enables I P access- list com pilat ion ( new fr om 12.0( 6) S)

r at e- lim it

Sim ple rat e lim it –specific access list

perm it

Specifies pack et s t o for w ar d

deny

Specifies pack et s t o r ej ect

dynam ic

Specifies a dy nam ic list of per m it s or denies

< 0- 2 5 5 >

An I P pr ot ocol num ber

ahp

Aut hent icat ion Header Pr ot ocol

eigrp

Cisco’s EI GRP r out ing pr ot ocol

esp

Encapsulat ion Secur it y Pay load

gre

Cisco’s GRE t unneling

icmp

I nt ernet Cont r ol Message Pr ot ocol

igm p

I nt er net Gat ew ay Message Pr ot ocol

igrp

Cisco’s I GRP r out ing pr ot ocol

ip

Any I nt er net pr ot ocol

ipinip

I P in I P t unneling

nos

KA9Q NOS–com pat ible I P ov er I P t unneling

ospf

OSPF r out ing pr ot ocol

p cp

Pay load Com pr ession Prot ocol

pim

Pr ot ocol I ndependent Mult icast

t cp

Tr ansm ission Cont r ol Pr ot ocol

udp

User Dat agr am Pr ot ocol

a. b. c. d

Sour ce or dest inat ion addr ess

any

Any sour ce host

host

A single sour ce host ( equiv alent t o a.b.c.d 255.255.255.255)

log

Log m at ches against t his ent r y

log- input

Log m at ches against t his ent r y , including input int er face

pr eceden ce

Mat ches pack et s w it h giv en pr ecedence v alue

t os

Mat ches pack et s w it h giv en TOS v alue

318

Ot her opt ions ex ist , depending on w hich upper- lay er pr ot ocol ( TCP or UDP) has been chosen. For ex am ple, TCP has t hese fur t her opt ions for configur ing an access list : ack, eq, est ablished, fin, gt , log, log- input , lt , neq, pr ecedence, psh, r ange, r st , sy n, t os, and urg.

I OS Soft w a r e Re gu la r Ex pr e ssion s Here are som e exam ples of re gular expressions used for BGP peerings on I SP rout ers t oday. Refer t o t he docum ent at ion for m ore in- dept h discussion and det ailed ex am ples. ^ 200$

Mat ches AS200 only

.*

Mat ches any t hing

.+

Mat ches at least one char act er

^*

Mat ches all pat hs, including t he local AS

^$

Mat ches t he local AS only

^ 200_

Mat ches all aut onom ous sy st em s r eceiv ed fr om AS200

_200_

Mat ches all pat hs cont aining AS200

_200$

Mat ches all pat hs w it h AS200 as t he or igin

^ 200_210$

Mat ches AS210 or igin and r eceived fr om AS200 only

_200_210_

Mat ches all pat hs t hat hav e been t hr ough AS200 and AS210 link

^ ( 200_) + $

Mat ches at least one of AS200 ( or m ult iple occur r ences of one AS) ( usually fr om AS_PATH st uffing [ 1 ] )

^ ( _[ 0- 9] + ) $

Mat ches at least one AS ( or m ult iple occur r ences of one AS)

_\ ( 65350\ ) _

Mat ches all pat hs hav ing confeder at ion sub- AS 65350 in t he p at h

^ [ 0- 9] + $

Mat ches AS_PATH lengt h of 1

^ [ 0- 9] + _[ 0- 9] + $

Mat ches AS_PATH lengt h of 2

^ [ 0- 9] * _[ 0- 9] + $

Mat ches AS_PATH lengt h of 1 or 2

^ [ 0- 9] * _[ 0- 9 ] * $

Mat ches AS_PATH lengt h of 1 or 2 ( or 0) [ 2 ]

^ [ 0- 9] + _[ 0- 9 ] + _ [ 09] + $

Mat ches AS_PATH lengt h of 3

_( 701| 1800) _

Mat ches any t hing t hat has gone t hr ough AS701 or AS1800

_1849( _.+ _) 12163$

Mat ches any t hing of or igin AS12163 and t r ansit ed t hr ough AS1849

I SPs t hat ar e using ut ilit ies such as t he RAToolSet ( see Appendix E, “ Tr affic Engineer ing Tools” ) t o gener at e t heir BGP configur at ions w ill see subst ant ially m or e sophist icat ed r egular ex pr essions—it is unusual for anyone t o gener at e by hand any t hing m or e sophist icat ed t han w hat has been list ed pr eviously.

319

En dn ot e s 1. AS pat h st uffing m eans seeing an AS pat h such as 200_200_200 for a net w or k announcem ent . This is com m only used w hen defining par t icular r out ing policy, such as w it h load shar ing. 2. Zer o is in br acket s because announcem ent s r eceiv ed fr om a neighbor ing AS w ill include an AS in t he pat h. But if t he filt er is used for iBGP pat h filt er ing, it w ill also m at ch t he local AS.

320

Appendix B. Cut - a n d- Pa st e Te m pla t e s The follow ing ar e som e cut - and- past e t em plat es t hat y ou can m odify t o configur e your rout ers. Make sure t hat you change any sam ple I P addresses or AS num bers used in t he t em plat es t o m at ch y our ow n addr essing! Do not use t he addr esses in t he ex am ples because t hey ar e invalid. As descr ibed in t he main t ex t , it is consider ed good pr act ice t o set up a configur at ion t em plat e for each class of r out er r unning in t he net w or k . Use t hese t em plat es, t ak en fr om r unning configur at ions in I SP back bones t oday , t o const r uct y our ow n t em plat es suit able for your needs.

Ge n e r a l Sy st e m Te m p la t e Not e t he inclusion of Cisco Ex pr ess For w ar ding ( CEF) in t his t em plat e ( see Exam ple B- 1) . CEF gives access t o im por t ant feat ur es such as Unicast RPF. The 7500 ser ies r out er s suppor t t he dist r ibut ed CEF opt ion —if CEF is r equir ed, don’t for get t he dist r ibu t e d k ey w or d. Som e r out er s hav e CEF enabled by default , so t he ip ce f dir ect iv e pr obably is not r equir ed in t he t em plat e. ( I t ’s good pr act ice t o include it any w ay , j ust in case any fut ur e default s change or CEF accident ally has been sw it ched off. ) Ex a m ple B- 1 Ge n e r a l Sy st e m Te m pla t e ! ! General stuff we don't want ! no service finger ! replaced with ip finger from 12.0 no service pad no service udp-small-servers no service tcp-small-servers no ip finger no ip bootp server no ip source-route no cdp run ! ! Nagle ! service nagle ! ! Full timestamping ! service timestamps debug datetime localtime show-timezone msec service timestamps log datetime localtime show-timezone msec ! ! Keepalives ! service tcp-keepalives-in ! ! All routers need a loopback ! interface loopback 0 description Loopback interface for router XY

321

ip address y.y.y.y 255.255.255.255 ! ! Enable logging with two loghosts using facility local4 ! no logging console logging buffered 16384 logging trap debugging logging source-interface loopback 0 logging facility local4 logging x.x.x.A logging x.x.x.B ! ! Make sure we are classless ! – these commands are default from 12.0 – best included anyway ! ip subnet-zero ip classless ! !!!! Enable Cisco Express Forwarding !!!! ! ip cef !

Ge n e r a l I n t e r f a ce Te m p la t e Many I SPs now m ake sur e t hat CEF is r unning on t heir r out er s by default , and t hey specify ip v e r ify u n ica st r e v e r se - pa t h on all t heir cust om er- facing int er faces. Ex am ple B- 2 includes t he configur at ion. I f CEF is not enabled on t he r out er , t he com m and par ser w ill not accept t he U n ica st RPF ch e ck com m and. Ex a m ple B- 2 Ge n e r a l I n t e r fa ce Te m pla t e ! ! Customer Facing Interfaces ! interface serial 0/0 description BW Connection to XYZ, Circuit ID, Cable ID. bandwidth BW ip verify unicast reverse-path no ip redirects no ip directed-broadcast no ip proxy-arp no cdp enable ! interface hssi 1/0 description BW Connection to ABC, Circuit ID, Cable ID. bandwidth BW ip verify unicast reverse-path no ip redirects no ip directed-broadcast no ip proxy-arp no cdp enable ! ! Core Network Interfaces

322

! interface fastethernet 2/0 description Core link to Router2, X-over Ethernet no ip redirects no ip directed-broadcast no ip proxy-arp no cdp enable !

Ge n e r a l Se cu r it y Te m p la t e Not ice t hat TACACS+ is included in t he t em plat e in Ex am ple B- 3. Using a cent ralized aut hent icat ion sy st em is st r ongly r ecom m ended. Such a sy st em scales and is m oder at ely secur e, unlike st or ing individual or shar ed passw or ds on ever y r out er in t he back bone. Ex a m ple B- 3 Ge n e r a l Se cu r it y Te m p la t e ! ! General stuff ! service password-encryption enable secret no enable password ! ! Enable AAA and TACACS+ ! aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable ! ! And set up the TACACS+ authentication – two servers ! ip tacacs source-interface loopback 0 tacacs-server host z.z.z.A tacacs-server host z.z.z.B tacacs-server key ! ! Enable Secure Shell ! – need to run "crypto key generate rsa" before applying this template ! ip ssh time-out 120 ip ssh authentication-retries 3 ! ! Protect the console ports – list NOC and other permitted addresses in acl 198 ! access-list 198 permit ip n.n.n.n m.m.m.m any access-list 198 deny ip any any log ! line con 0 location Router Console – use as last resort exec-timeout 1 0 transport preferred none

323

tracsport input none line aux 0 location Connected to XY-BR router Console – access through reverse telnet no exec transport preferred none transport input telnet line vty 0 4 location Router VTY ports – only telnet and SSH permitted access-class 198 in transport preferred none transport input telnet ssh transport output telnet ssh !

Ge n e r a l iBGP Te m p la t e This gener ic t em plat e ( see Exam ple B- 4) should be used for configu ring iBGP on a r out er . I t also includes a sam ple configur at ion for a peer gr oup. Not ice t he configur at ion of t he BGP v er sion ( w hich som e consider being ult r apar anoid and ot her s consider good pr act ice) and also t he use of n e x t- hop- se lf for iBGP peer ings. Ex a m ple B- 4 Ge n e r a l iBGP Te m p la t e ! ! Set up our AS ! router bgp 65280 ! ! Defaults we need to fix ! no synchronization no auto-summary ! ! Catch all logs ! bgp log-neighbor-changes ! ! Set up peer-group ! neighbor ibgp-peer peer-group neighbor ibgp-peer description Internal BGP neighbor ibgp-peer remote-as 65280 neighbor ibgp-peer update-source loopback 0 neighbor ibgp-peer next-hop-self neighbor ibgp-peer version 4 neighbor ibgp-peer send-community internally neighbor ibgp-peer password !

peers

! ultra paranoid ! send communities ! use password on peering

324

Ge n e r a l e BGP Te m p la t e This gener ic t em plat e should be used for configur ing eBGP on a r out er . I t also includes a sam ple configur at ion for an ex t er nal BGP peer- gr oup. The eBGP t em plat e w ill differ depending on w het her t he rout er is a border rout er or a cust om er aggr egat ion r out er , so it is a good idea t o set up a t em plat e for each scenar io encount er ed. Not ice t he int r oduct ion of r out e flap dam ping—using t he RI PE- 229 par am et er s is consider ed by m any t o be bet t er t han using t he Cisco default s because t hey ar e less aggr essiv e. See Appendix D, “ Rout e Flap Damping,” for t he det ailed flap- dam ping configurat ion. The rout e m aps and prefix list s m ent ioned in Ex am ple B- 5 are for apply ing policy and r out e filt er ing as appr opr iat e. Ex a m ple B- 5 Ge n e r a l e BGP Te m p la t e ! ! Set up our AS ! router bgp 65280 ! ! Use BGP damping, preferably using RIPE-210 values ! bgp dampening graded-flap-damp ! ! Defaults we need to fix ! no synchronization no auto-summary ! ! Catch all logs ! bgp log-neighbor-changes ! ! enable deterministic MED ! bgp deterministic-med ! ! Set up peer-group ! neighbor ebgp-peer peer-group neighbor ebgp-peer description Generic eBGP peergroup neighbor ebgp-peer remove-private-AS neighbor ebgp-peer version 4 neighbor ebgp-peer prefix-list isp-out out neighbor ebgp-peer prefix-list isp-in in neighbor ebgp-peer route-map out-policy out neighbor ebgp-peer route-map in-policy in neighbor ebgp-peer password neighbor ebgp-peer maximum-prefix 150000 !

325

M a r t ia n a n d RFC 1 9 1 8 N e t w or k s Te m pla t e This list r epr esent s t he com m on filt er ing pr act ice of sev er al I SPs. I t includes default , m ult icast , and RFC 1918 net w or k s, as w ell as t he so- called Mar t ian net w or k s. The list w as docum ent ed in t he I nt er net dr aft by Bill Manning, w hich, at t he t im e of w r it ing, w as found at w w w .iet f.or g/ int er net - dr aft s/ dr aft- m anning- dsua- 07.t xt . The use of t hese filt ers on inbound and out bound int er faces of bor der r out er s is r ecom m ended. I t is also r ecom m ended t hat t hese pr efix es be used on BGP peer ings w her e t he I SP is r eceiving or sending full pr efixes t o a neighbor ing aut onom ous syst em . The r ecom m ended list of filt ers will change over t im e —t he list is quot ed at t he t im e of t his w rit ing, m id- 2001. You ar e encour aged t o check t he com panion w eb sit e ( w w w . ispbook . com) and w hit epaper supplem ent s t o t his book for any updat es befor e im plem ent ing any of t hese filt er s. Tw o ex am ples of t he filt er s ar e giv en ( see Exam ples B- 6 and B- 7) : The fir st ex am ple uses access list s, now consider ed depr ecat ed for BGP r out e filt er ing; t he ot her ex am ple uses pr efix list s, w hich ar e easier t o under st and and m uch m or e efficient for t he r out er t o pr ocess.

I P Access List Exam ple Ex a m ple B- 6 Acce ss List t o D e n y RFC 1 9 1 8 a n d M a r t ia n N e t w or k s ! access-list 150 access-list 150 0.255.255.255 access-list 150 0.255.255.255 access-list 150 0.0.255.255 access-list 150 0.15.255.255 access-list 150 access-list 150 0.0.255.255 access-list 150 31.255.255.255 access-list 150 access-list 150 !

deny deny

ip 0.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255 ip 10.0.0.0 0.255.255.255 255.0.0.0

deny

ip 127.0.0.0 0.255.255.255 255.0.0.0

deny

ip 169.254.0.0 0.0.255.255 255.255.0.0

deny

ip 172.16.0.0 0.15.255.255 255.240.0.0

deny deny

ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255 ip 192.168.0.0 0.0.255.255 255.255.0.0

deny

ip 224.0.0.0 31.255.255.255 224.0.0.0

deny ip any 255.255.255.128 0.0.0.127 permit ip any any

I P Prefix List Exam ple Ex a m ple B- 7 Pr e f ix List t o D e n y RFC 1 9 1 8 a n d M a r t ia n N e t w or k s ! ip prefix-list announced ip prefix-list ip prefix-list ip prefix-list ip prefix-list

rfc1918-sua description Networks which shouldn't be rfc1918-sua rfc1918-sua rfc1918-sua rfc1918-sua

deny deny deny deny

0.0.0.0/8 le 32 10.0.0.0/8 le 32 127.0.0.0/8 le 32 169.254.0.0/16 le 32

326

ip ip ip ip ip ip !

prefix-list prefix-list prefix-list prefix-list prefix-list prefix-list

rfc1918-sua rfc1918-sua rfc1918-sua rfc1918-sua rfc1918-sua rfc1918-sua

deny 172.16.0.0/12 le 32 deny 192.0.2.0/24 le 32 deny 192.168.0.0/16 le 32 deny 224.0.0.0/3 le 32 deny 0.0.0.0/0 ge 25 permit 0.0.0.0/0 le 32

327

Appendix C. Ex a m ple Configura t ions This appendix aims t o giv e I SPs lear ning about t he ar t of const r uct ing an I SP caliber back bone a lit t le guidance on som e of t he configur at ion st eps and design hint s t hat hav e been cov er ed t hr oughout t he “ I OS Essent ials” w hit epaper and t his book . The com m on elem ent s of an ISP’s net w or k have all been included, including bor der , aggr egat ion, and dial access r out er s. Clear ly it is alm ost im possible t o giv e sam ple configur at ions suit able for all I SPs. The follow ing aim s t o giv e guidance t o sm all or m edium I SPs t hat hav e not been in operat ions long so t hat t hey gain t he general pr inciples of w hat is r equir ed for t he net w or k r out er s.

Sim ple N e t w or k Pla n Figure C- 1 show s a sim ple net w or k diagr am of a basic I SP point of pr esence ( PoP) , w hich w ill be used in t hese exam ples. I t has t he key elem ent s of an I SP PoP: a bor der r out er , t w o cor e r out er s, aggr egat ion r out er s ( for leased- line or per m anent ly connect ed cust om er s) , t w o ser v ice r out er s ( for w eb host ing and t he I SP’s ow n ser v ices) , a dial aggr egat ion r out er , and a r out er t hat connect s t o t he net w or k oper at ions cent er . Obv iously , as I SPs gr ow , t heir net w or k w ill be m or e sophist icat ed t han t his, and t heir configurat ion needs likew ise w ill grow t o m a t ch. How ev er , t his ex am ple should ser v e as a good gr ounding for fut ur e gr ow t h. Figu r e C- 1 . N e t w or k U se d f or Con f ig u r a t ion a n d Filt e r in g Sa m p le

An im por t ant elem ent t o w hich y ou should pay at t ent ion t hr oughout t hese ex am ples is t he v er y r igor ous filt er ing t hat has been applied t hr oughout t he net w or k . Filt er ing at t he r ight places ensur es secur it y for t he I SP, as w ell as a r eliably per for m ing net work.

328

Con figu r a t ion s The follow ing Cisco I OS Soft w ar e configur at ions w or k and hav e been t est ed in a lab env ir onm ent . They ar e based heav ily on k now n cur r ent configur at ions used in t he field t oday , w it h addit ions/ m odificat ions t o include as m any r ecom m endat ions from t he m ain t ext of t his book as is feasible. The configur at ions hav e been annot at ed using I OS Soft w ar e com m ent s ( w it h ! ) w her e m or e explanat ion is r equir ed. This should m ake it easier t o cut and past e configur at ions int o y our ow n t est env ir onm ent . But r e m em ber t he golden rule: “ Nev er cut and past e int o a liv e net w or k , and don’t im plem ent any configur at ion unt il y ou hav e ex t ensiv ely t est ed it in y our ow n env ir onm ent .” N OTE You can find a com panion w eb sit e for t his book w it h t he configur at ions in t his appendix at w w w . ispbook . com.

The I P addr esses and AS num ber s used ar e com plet ely fict it ious and, at t he t im e of w r it ing, st ill belong t o I ANA r eser v ed addr ess space. At som e t im e in t he fut ur e, t hese addr esses or addr ess spaces w ill becom e allocat ed t o an I SP and assigned t o an end cust om er . I t is v er y im por t ant t hat y ou do not use t hese addr esses in y our ow n back bone or announce t hem t o t he I nt er net . Addr ess space should be obt ained from your upst ream I SP or fr om one of t he r egional I nt er net r egist r ies.

I SP Addressing Pla n For r efer ence pur poses, t he follow ing addr essing plan has been used for t his I SP back bone. The addr ess block is 220.144.128.0/ 19. 220.144.159.192/ 26

Rout er loopbacks

220.144.159.128/ 26

I SP net w or k oper at ions cent er

220. 144. 159. 64/ 26

Point - t o- point links and VLANs

220.144.159.0/ 26

I SP ser v ices LAN

220.144.158.0/ 24

Cu st om er- host ing LAN

220.144.157.0/ 24

DI AL pool

: : 220.144.128.0 and upw ar ds t o be used for cust om er assignm ent s N OC H ost s 220.144.159.129 loghost , TACACS+ , FTP, TFTP 220.144.159.130 Net w or k m anagem ent w or k st at ion 220.144.159.131 Secur e Shell gat ew ay

329

220.144.159.132 RADI US server for DI AL 220.144.159.160 .160 t hr ough .175: Host r ange for t er m inal ser v er console por t a ccess 220.144.159.180 Prim ary DNS 220.144.159.189 Out - of- band m anagem ent t er m inal ser ver 220.144.159.190 NOC fir ew all r out er I SP Se r v ice s LAN 220.144.159.1

Secondar y DNS, NTP ser v er

220.144.159.2

Secondar y MX, caching DNS ser v er

220.144.159.3

Prim ary MX, FTP, WWW

220.144.159.4

PoP3, SMTP relay for DI AL only

220.144.159.5

Newsreading syst em for DI AL only

220. 144. 159. 59

Ser v ice– Rout er1

220. 144. 159. 60

Ser v ice– Rout er2

220. 144. 159. 61

HSRP gat ew ay 1

220. 144. 159. 62

HSRP gat ew ay 2

Bor de r Rout e r The bor der rout er is perhaps t he m ost im port ant rout er in an I SP’s backbone net w or k oper at ion. I t pr ov ides connect iv it y t o t he r est of t he I nt er net for t he I SP and also pr ot ect s t he I SP’s net w or k and cust om er s fr om t he r av ages of t he I nt er net . I n I SP w or k shop pr esent at ions, w e use t he analogy of t he fr ont door of your house or t he gar den gat e pr ot ect ing y ou fr om t he w or ld out t her e. The configur at ion of t he bor der r out er is cr it ical t o t he cor r ect and r eliable oper at ion of t he r est of t he I SP’s business. The configurat ion in Ex am ple C- 1 is sim ilar t o som e of t hose t hat ar e deploy ed on liv e net w or k s t oday . Ex a m ple C- 1 Bor d e r Rou t e r Con f ig u r a t ion Ex a m p le version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname border-router ! boot system flash c7200-k4p-mz.120-10.S2 boot system flash ! logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable

330

aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ ! enable secret shhhhhthisisasecret ! clock timezone GMT 0 ip subnet-zero ip cef no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ! ! Set up DNS – note that one secondary NS is hosted by my upstream ISP ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! SSH support ip ssh time-out 120 ip ssh authentication-retries 3 ! interface Loopback0 description Loopback interface on border-router ip address 220.144.159.192 255.255.255.255 no ip directed-broadcast ! interface FastEthernet0/0 description Ethernet to Core1 (x-over ethernet) ip address 220.144.159.65 255.255.255.252 no ip redirects no ip directed-broadcast no ip proxy-arp ip route-cache flow ! interface Serial1/0 description 256Kb HDLC link to Buzz Internet bandwidth 256 ip address 219.10.1.2 255.255.255.252 ip access-group 100 in ip access-group 101 out no ip redirects no ip directed-broadcast ip route-cache flow no fair-queue ! interface Serial1/1 description 512Kb HDLC link to Whoosh Internet bandwidth 512 ip address 219.50.10.2 255.255.255.252 ip access-group 100 in ip access-group 101 out

331

no no ip no

ip redirects ip directed-broadcast route-cache flow fair-queue

! interface FastEthernet2/0 description Ethernet to Core2 (x-over ethernet) ip address 220.144.159.69 255.255.255.252 no ip redirects no ip directed-broadcast no ip proxy-arp ip route-cache flow ! router ospf 100 network 219.50.10.0 0.0.0.3 area 0 network 219.10.1.0 0.0.0.3 area 0 network 220.144.159.64 0.0.0.7 area 0 network 220.144 159.192 0.0.0.0 area 0 passive-interface Serial1/0 passive-interface Serial1/1 passive-interface Loopback0 log-adjacency-changes ! router bgp 64511 no synchronization bgp log-neighbor-changes ! Use peer-groups – more efficient neighbor core-ibgp peer-group neighbor core-ibgp remote-as 64511 neighbor core-ibgp update-source Loopback0 neighbor core-ibgp password BGPsecretPW neighbor core-ibgp send-community ! Get full routing table from both upstreams – block RFC1918+ address space inbound ! only announce my address block outbound neighbor 219.10.1.2 remote-as 64400 neighbor 219.10.1.2 description Connection to Buzz Internet neighbor 219.10.1.2 prefix-list infilter in neighbor 219.10.1.2 prefix-list outfilter out neighbor 219.10.1.2 password BuzzSecretPW neighbor 219.50.10.1 remote-as 64500 neighbor 219.50.10.1 description Connection to Whoosh Internet neighbor 219.50.10.1 prefix-list infilter in neighbor 219.50.10.1 prefix-list outfilter out neighbor 219.50.10.1 password WhooshSecretPW ! iBGP with core routers – using route reflectors neighbor 220.144.159.193 peer-group core-ibgp neighbor 220.144.159.193 description Core1 neighbor 220.144.159.194 peer-group core-ibgp neighbor 220.144.159.194 description Core2 no auto-summary ! ip classless ip route 10.0.0.0 255.0.0.0 Null0 ip route 172.16.0.0 255.240.0.0 Null0 ip route 192.168.0.0 255.255.0.0 Null0 ip route 220.144.128.0 255.255.224.0 Null0

332

ip tacacs source-interface Loopback0 ip bgp-community new-format ! no logging console logging trap debugging logging source-interface Loopback0 logging 220.144.159.129 ! SNMP access-list access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log ! INBOUND access-list on external interfaces ! BLOCK THE MARTIANS access-list 100 deny ip 10.0.0.0 0.255.255.255 any access-list 100 deny ip 127.0.0.0 0.255.255.255 any access-list 100 deny ip 172.16.0.0 0.15.255.255 any access-list 100 deny ip 192.168.0.0 0.0.255.255 any access-list 100 deny ip 220.144.128.0 0.0.31.255 any access-list 100 deny ip any 0.0.0.255 255.255.255.0 access-list 100 deny ip any 0.0.0.0 255.255.255.0 ! ACCESS TO SERIAL PORTS access-list 100 permit icmp any host 219.10.1.2 access-list 100 permit icmp any host 219.50.10.2 access-list 100 permit icmp any 220.144.128.0 0.0.31.255 access-list 100 permit udp any host 219.10.1.2 eq ntp access-list 100 permit udp any host 219.50.10.2 eq ntp access-list 100 deny ip any host 219.10.1.2 log access-list 100 deny ip any host 219.50.10.2 log access-list 100 permit tcp any 220.144.128.0 0.0.31.255 established ! SSH access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq 22 access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq ftp access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq ftp-data ! For web access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq ident access-list 100 permit udp any 220.144.128.0 0.0.31.255 eq ntp access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq smtp access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq www access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq pop3 ! IMAP access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq 143 ! LDAP access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq 389 ! HTTPS access-list 100 permit tcp any 220.144.128.0 0.0.31.255 eq 443 ! NFS access-list 100 deny tcp any 220.144.128.0 0.0.31.255 eq 2049 log ! NFS access-list 100 deny udp any 220.144.128.0 0.0.31.255 eq 2049 log ! X access-list 100 deny tcp any 220.144.128.0 0.0.31.255 eq 6000 log access-list 100 permit tcp any 220.144.128.0 0.0.31.255 gt 1023 access-list 100 permit udp any 220.144.128.0 0.0.31.255 gt 1023 access-list 100 deny ip any any log ! OUTBOUND access-list on external interfaces ! My address block access-list 101 permit ip 220.144.128.0 0.0.31.255 any

333

! and external facing interfaces access-list 101 permit ip host 219.10.1.2 any access-list 101 permit ip host 219.50.10.2 any access-list 101 deny ip any any log ! VTY access-list access-list 198 permit ip 220.144.159.128 0.0.0.63 any ! NOC systems access-list 198 permit ip 220.144.159.192 0.0.0.63 any ! Router loopbacks access-list 198 deny ip any any log ! Industry convention is that acl 199 blocks everything access-list 199 deny ip any any log ! ! BGP INBOUND filters ip prefix-list infilter description Networks which shouldn't be accepted ip prefix-list infilter deny 0.0.0.0/8 le 32 ip prefix-list infilter deny 10.0.0.0/8 le 32 ip prefix-list infilter deny 127.0.0.0/8 le 32 ip prefix-list infilter deny 169.254.0.0/16 le 32 ip prefix-list infilter deny 172.16.0.0/12 le 32 ip prefix-list infilter deny 192.0.2.0/24 le 32 ip prefix-list infilter deny 192.168.0.0/16 le 32 ip prefix-list infilter deny 220.144.128.0/19 le 32 ip prefix-list infilter deny 224.0.0.0/3 le 32 ip prefix-list infilter deny 0.0.0.0/0 ge 25 ip prefix-list infilter permit 0.0.0.0/0 le 32 ! ! BGP OUTBOUND filters ip prefix-list outfilter description Networks which should be announced to upstreams ip prefix-list outfilter permit 220.144.128.0/19 ! tacacs-server host 220.144.159.129 tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo banner login ^ Authorized Access Only This system is the property of Galaxy Internet Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line aux 0 transport preferred none transport input none

334

transport output telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! ! Where Router core-dumps go exception protocol ftp exception dump 220.144.159.129 ! NTP configuration ntp authentication-key 1 md5 secretAlso ntp authenticate ntp trusted-key 1 ntp source Loopback0 ntp server 219.10.1.1 ntp peer 220.144.159.1 key 1 ntp server 219.50.10.1 end

Core Rout e r Tw o cor e r out er s w er e giv en in t he ex am ple in Figure C- 1. How ev er , only one configurat ion is given here —t he configur at ion for Cor e - r out er 2 is alm ost ident ical. Not ice t hat t he cor e r out er has a sim pler configur at ion t han t he r out er s at t he edge of t he I SP’s net w or k . Cor e r out er s t end not t o do pack et filt er ing or r out ing policy ; t he design goal is m or e r eliabilit y and a configur at ion t hat r equir es no change in t he shor t t o m edium t er m s. Ex am ple C- 2 assum es t hat t he cor e r out er is a Cisco 7206. Ex a m ple C- 2 Cor e Rou t e r Con f ig u r a t ion Ex a m p le version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname Core-router1 ! boot system flash slot0:c7200-k4p-mz.120-10.S2 boot system flash ! logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ enable secret shhhhhthisisasecret ! ip cef clock timezone GMT 0

335

ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ! ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! SSH support ip ssh time-out 120 ip ssh authentication-retries 3 ! interface Loopback0 description Loopback interface on Core-router1 ip address 220.144.159.193 255.255.255.255 no ip directed-broadcast ! interface FastEthernet0/0 description Ethernet to Border (x-over ethernet) ip address 220.144.159.66 255.255.255.252 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial1/0 description E1 HDLC link to Smalltown PoP bandwidth 2048 ip address 220.144.159.77 255.255.255.252 no ip redirects no ip directed-broadcast no fair-queue ! interface FastEthernet2/0 description Ethernet to Core2 (x-over ethernet) ip address 220.144.159.73 255.255.255.252 delay 9 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet3/0 description Core Ethernet (SW1 port 1) ip address 220.144.159.97 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet4/0 description Core Ethernet (SW2 port 1) ip address 220.144.159.113 255.255.255.240

336

no ip redirects no ip directed-broadcast no ip proxy-arp ! router ospf 100 network 220.144.159.64 0.0.0.3 area 0 network 220.144.159.72 0.0.0.3 area 0 network 220.144.159.76 0.0.0.3 area 0 network 220.144.159.96 0.0.0.15 area 0 network 220.144.159.112 0.0.0.15 area 0 network 220.144 159.193 0.0.0.0 area 0 passive-interface Loopback0 log-adjacency-changes ! router bgp 64511 network 220.144.128.0 mask 255.255.224.0 no synchronization bgp log-neighbor-changes ! Two peergroups, one for the CORE iBGP, ! the other for Route Reflector Cluster neighbor core-ibgp peer-group neighbor core-ibgp remote-as 64511 neighbor core-ibgp update-source Loopback0 neighbor core-ibgp password BGPsecretPW neighbor core-ibgp send-community neighbor rr-client peer-group neighbor rr-client remote-as 64511 neighbor rr-client update-source Loopback0 neighbor rr-client password BGPsecretPW neighbor rr-client send-community neighbor rr-client route-reflector-client neighbor 220.144.159.192 peer-group core-ibgp neighbor 220.144.159.192 description Border neighbor 220.144.159.194 peer-group core-ibgp neighbor 220.144.159.194 description Core2 neighbor 220.144.159.195 peer-group rr-client neighbor 220.144.159.195 description Gateway1 neighbor 220.144.159.196 peer-group rr-client neighbor 220.144.159.196 description Gateway2 neighbor 220.144.159.197 peer-group rr-client neighbor 220.144.159.197 description Service1 neighbor 220.144.159.198 peer-group rr-client neighbor 220.144.159.198 description Service2 neighbor 220.144.159.200 peer-group rr-client neighbor 220.144.159.200 description DIAL Access Router no auto-summary ! ip classless ip route 10.0.0.0 255.0.0.0 Null0 ip route 172.16.0.0 255.240.0.0 Null0 ip route 192.168.0.0 255.255.0.0 Null0 ip route 220.144.128.0 255.255.224.0 Null0 ip tacacs source-interface Loopback0 ip bgp-community new-format ! no logging console logging trap debugging

337

logging source-interface Loopback0 logging 220.144.159.129 access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log ! VTY access-list - NOC access-list 198 permit ip 220.144.159.128 0.0.0.63 any ! Router loopbacks access-list 198 permit ip 220.144.159.192 0.0.0.63 any access-list 198 deny ip any any log access-list 199 deny ip any any log ! tacacs-server host 220.144.159.129 tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo banner login ^ Authorized Access Only This system is the property of Galaxy Internet Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line aux 0 transport preferred none transport input none transport output telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! exception protocol ftp exception dump 220.144.159.129 ntp authentication-key 1 md5 secretAlso ntp authenticate ntp trusted-key 1 ntp source Loopback0 ntp server 220.144.159.192 key 1 ntp server 220.144.159.1 key 1 ntp peer 220.144.159.194 key 1 end

338

Aggr e ga t ion Rout e r The aggr egat ion or gat ew ay r out er s ar e used for connect ing fix ed- link cust om er s t o t he I SP’s backbone. As in pr ev ious sect ions, only one ex am ple w ill be giv en. The second aggr egat ion r out er w ill use a ver y sim ilar configur at ion. The aggr egat ion rout er in Ex am ple C- 3 is assum ed t o be a 3640. Ex a m ple C- 3 Ag g r e g a t ion Rou t e r Con f ig u r a t ion Ex a m p le version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname Gateway-router1 ! boot system flash logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ enable secret shhhhhthisisasecret ! ip cef clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! SSH support ip ssh time-out 120 ip ssh authentication-retries 3 ! partition flash 2 8 8 ! interface Loopback0 description Loopback interface on GW1 ip address 220.144.159.195 255.255.255.255 no ip directed-broadcast !

339

interface FastEthernet0/0 description Core Ethernet (SW1 port 3) ip address 220.144.159.99 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet0/1 description Core Ethernet (SW2 port 3) ip address 220.144.159.115 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial1/0 description HDLC 64K link to Cust1 CT1 bandwidth 64 ip verify unicast reverse-path ip unnumbered Loopback0 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial1/1 description 64K HDLC link to Cust2 CT2 bandwidth 64 ip unnumbered Loopback0 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial1/2 description 64K HDLC link to Cust 3 CT3 bandwidth 64 ip verify unicast reverse-path ip unnumbered Loopback0 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Serial1/3 description 64k HDLC link to Cust 4 CT4 bandwidth 64 ip verify unicast reverse-path ip unnumbered Loopback0 no ip redirects no ip directed-broadcast no ip proxy-arp ! router ospf 100 network 220.144.159.64 0.0.0.3 area 0 network 220.144.159.72 0.0.0.3 area 0 network 220.144.159.76 0.0.0.3 area 0 network 220.144.159.96 0.0.0.15 area 0 network 220.144.159.112 0.0.0.15 area 0 network 220.144 159.193 0.0.0.0 area 0 passive-interface Loopback0

340

log-adjacency-changes ! router bgp 64511 network 220.144.128.0 network 220.144.129.0 mask 255.255.255.224 network 220.144.129.32 mask 255.255.255.224 network 220.144.129.64 mask 255.255.255.192 no synchronization bgp log-neighbor-changes neighbor rr peer-group neighbor rr remote-as 64511 neighbor rr update-source Loopback0 neighbor rr password BGPsecretPW neighbor rr send-community neighbor 220.144.159.193 peer-group rr neighbor 220.144.159.193 description Core1 neighbor 220.144.159.194 peer-group rr neighbor 220.144.159.194 description Core2 no auto-summary ! ip classless ip route 10.0.0.0 255.0.0.0 Null0 ip route 172.16.0.0 255.240.0.0 Null0 ip route 192.168.0.0 255.255.0.0 Null0 ip route 220.144.128.0 255.255.224.0 Null0 ip route 220.144.128.0 255.255.255.0 Serial1/0 permanent ip route 220.144.129.0 255.255.255.224 Serial1/1 permanent ip route 220.144.129.32 255.255.255.224 Serial1/2 permanent ip route 220.144.129.64 255.255.255.192 Serial1/3 permanent ip tacacs source-interface Loopback0 ip bgp-community new-format ! no logging console logging trap debugging logging source-interface Loopback0 logging 220.144.159.129 access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log ! VTY access-list - NOC access-list 198 permit ip 220.144.159.128 0.0.0.63 any ! Router loopbacks access-list 198 permit ip 220.144.159.192 0.0.0.63 any access-list 198 deny ip any any log access-list 199 deny ip any any log ! tacacs-server host 220.144.159.129 tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo banner login ^ Authorized Access Only This system is the property of Galaxy Internet

341

Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line aux 0 transport preferred none transport input telnet transport output telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! exception protocol ftp exception dump 220.144.159.129 ntp authentication-key 1 md5 secretAlso ntp authenticate ntp trusted-key 1 ntp source Loopback0 ntp server 220.144.159.193 key 1 ntp server 220.144.159.194 key 1 ntp peer 220.144.159.196 key 1 ntp peer 220.144.159.197 key 1 ntp peer 220.144.159.198 key 1 end

Service Rou t e r The ser v ice r out er s ar e used for connect ing t he host ed ser v ices ( for ex am ple, cont ent pr ov ider sy st em s) and t he I SP’s ow n ser v ices ( m ail, new s, DNS) t o t he I SP’s back bone. Only one ex am ple is pr ov ided. The second aggr egat ion r out er w ill use a very sim i lar configur at ion. The r out er in Ex am ple C- 4 is assum ed t o be a 2620 w it h four Fast Et her net por t s. Ex a m ple C- 4 Se r v ice Rou t e r Con f ig u r a t ion Ex a m p le version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname Service-router1 !

342

boot system flash logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ enable secret shhhhhthisisasecret ! clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! SSH support ip ssh time-out 120 ip ssh authentication-retries 3 ! partition flash 2 8 8 ! interface Loopback0 description Loopback interface on SR1 ip address 220.144.159.197 255.255.255.255 no ip directed-broadcast ! interface FastEthernet0/0 description Core Ethernet (SW1 vlan1 port 4) ip address 220.144.159.101 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet0/1 description Core Ethernet (SW2 vlan2 port 13) ip address 220.144.159.117 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet1/0 description Hosted Server Ethernet (SW3 vlan1 port 1) ip address 220.144.158.1 255.255.255.0 ip access-group 140 in ip access-group 141 out standby 10 priority 150 standby 10 preempt standby 10 ip 220.144.158.253

343

standby 20 ip 220.144.158.254 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet1/1 description Galaxy ISP Server Ethernet (SW3 vlan2 port 8) ip address 220.144.159.59 255.255.255.192 ip access-group 150 in ip access-group 151 out standby 30 priority 150 standby 30 preempt standby 30 ip 220.144.159.61 standby 40 ip 220.144.159.62 no ip redirects no ip directed-broadcast no ip proxy-arp ! router ospf 100 network 220.144.159.96 0.0.0.15 area 0 network 220.144.159.112 0.0.0.15 area 0 network 220.144 159.197 0.0.0.0 area 0 network 220.144.159.0 0.0.0.63 area 0 passive-interface FastEthernet1/1 passive-interface Loopback0 log-adjacency-changes ! router bgp 64511 network 220.144.158.0 no synchronization bgp log-neighbor-changes neighbor rr peer-group neighbor rr remote-as 64511 neighbor rr update-source Loopback0 neighbor rr password BGPsecretPW neighbor rr send-community neighbor 220.144.159.193 peer-group rr neighbor 220.144.159.193 description Core1 neighbor 220.144.159.194 peer-group rr neighbor 220.144.159.194 description Core2 no auto-summary ! ip classless ip route 10.0.0.0 255.0.0.0 Null0 ip route 172.16.0.0 255.240.0.0 Null0 ip route 192.168.0.0 255.255.0.0 Null0 ip route 220.144.128.0 255.255.224.0 Null0 ip tacacs source-interface Loopback0 ip bgp-community new-format ! no logging console logging trap debugging logging source-interface Loopback0 logging 220.144.159.129 access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log

344

! WHAT GETS OUT access-list 140 permit ip 220.144.158.0 0.0.0.255 any access-list 140 deny ip any any log ! WHAT GETS IN access-list 141 permit ip host 220.144.158.1 host 220.144.158.2 access-list 141 permit ip host 220.144.158.2 host 220.144.158.1 access-list 141 permit ip 220.144.159.128 0.0.0.63 220.144.158.0 0.0.0.255 access-list 141 permit icmp any 220.144.158.0 0.0.0.255 access-list 141 permit tcp any 220.144.158.0 0.0.0.255 established access-list 141 permit tcp any 220.144.158.0 0.0.0.255 eq ident ! SSH access-list 141 permit tcp any 220.144.158.0 0.0.0.255 eq 22 access-list 141 permit tcp any 220.144.158.0 0.0.0.255 eq www access-list 141 permit udp any 220.144.158.0 0.0.0.255 gt 1023 access-list 141 deny ip any any log ! WHAT GETS OUT access-list 150 permit ip 220.144.159.0 0.0.0.63 any access-list 150 deny ip any any log ! WHAT GETS IN access-list 151 permit ip host 220.144.159.1 host 220.144.159.2 access-list 151 permit ip host 220.144.159.2 host 220.144.159.1 access-list 151 permit ip 220.144.159.128 0.0.0.63 220.144.159.0 0.0.0.63 access-list 151 permit icmp any 220.144.159.0 0.0.0.63 access-list 151 permit tcp any 220.144.159.0 0.0.0.63 established access-list 151 permit tcp any 220.144.159.0 0.0.0.63 eq ident access-list 151 permit udp any 220.144.159.0 0.0.0.63 gt 1023 ! 220.144.159.1 is 2ary DNS, NTP access-list 151 permit tcp any host 220.144.159.1 eq domain access-list 151 permit udp any host 220.144.159.1 eq domain access-list 151 permit udp any host 220.144.159.1 eq ntp ! 220.144.159.2 is 2ary MX, DNS cache access-list 151 permit tcp any host 220.144.159.2 eq smtp access-list 151 permit tcp any host 220.144.159.2 eq domain access-list 151 permit udp any host 220.144.159.2 eq domain ! 220.144.159.3 is 1ary MX, FTP & WWW access-list 151 permit tcp any host 220.144.159.3 eq smtp access-list 151 permit tcp any host 220.144.159.3 eq ftp access-list 151 permit tcp any host 220.144.159.3 eq ftp-data access-list 151 permit tcp any host 220.144.159.3 eq www ! 220.144.159.4 is PoP3 server and SMTP relay for DIAL only access-list 151 permit tcp any host 220.144.159.4 eq pop3 access-list 151 permit tcp 220.144.157.0 0.0.0.255 host 220.144.159.4 eq smtp ! 220.144.159.5 is News reader for DIAL only access-list 151 permit tcp 220.144.157.0 0.0.0.255 host 220.144.159.5 eq nntp access-list 151 deny ip any any log ! VTY access-list - NOC access-list 198 permit ip 220.144.159.128 0.0.0.63 any ! Router loopbacks access-list 198 permit ip 220.144.159.192 0.0.0.63 any access-list 198 deny ip any any log access-list 199 deny ip any any log ! tacacs-server host 220.144.159.129

345

tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo banner login ^ Authorized Access Only This system is the property of Galaxy Internet Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line aux 0 transport preferred none transport input telnet transport output telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! exception protocol ftp exception dump 220.144.159.129 ntp authentication-key 1 md5 secretAlso ntp authenticate ntp trusted-key 1 ntp source Loopback0 ntp server 220.144.159.193 key 1 ntp server 220.144.159.194 key 1 ntp peer 220.144.159.196 key 1 ntp peer 220.144.159.197 key 1 ntp peer 220.144.159.198 key 1 end

N OC Rou t e r The NOC r out er is used t o connect t he I SP’s essent ial ser v ices ( such as SYSLOG, TACACS+ , and pr im ar y DNS) and t he oper at ions engineer s’ w or k st at ions t o t he I SP’s backbone. The r out er in Ex am ple C- 5 is assum ed t o be a 2620 w it h t hr ee Fast Et her net por t s. Ex a m ple C- 5 N OC Rou t e r Con f ig u r a t ion Ex a m p le version 12.0

346

service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname NOC-router ! boot system flash logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ enable secret shhhhhthisisasecret ! clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! SSH support ip ssh time-out 120 ip ssh authentication-retries 3 ! partition flash 2 8 8 ! interface Loopback0 description Loopback interface on NOC ip address 220.144.159.199 255.255.255.255 no ip directed-broadcast ! interface FastEthernet0/0 description Core Ethernet (SW1 vlan1 port 6) ip address 220.144.159.103 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet0/1 description Core Ethernet (SW2 vlan2 port 15) ip address 220.144.159.119 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp

347

! interface FastEthernet1/0 description NOC Ethernet ip address 220.144.159.190 255.255.255.192 ip access-group 130 in ip access-group 131 out no ip redirects no ip directed-broadcast no ip proxy-arp ! router ospf 100 network 220.144.159.96 0.0.0.15 area 0 network 220.144.159.112 0.0.0.15 area 0 network 220.144 159.199 0.0.0.0 area 0 network 220.144.159.128 0.0.0.63 area 0 passive-interface FastEthernet1/0 passive-interface Loopback0 log-adjacency-changes ! ip classless ip route 10.0.0.0 255.0.0.0 Null0 ip route 172.16.0.0 255.240.0.0 Null0 ip route 192.168.0.0 255.255.0.0 Null0 ip route 220.144.128.0 255.255.224.0 Null0 ip tacacs source-interface Loopback0 ip bgp-community new-format ! no logging console logging trap debugging logging source-interface Loopback0 logging 220.144.159.129 access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log access-list 130 permit ip 220.144.159.128 0.0.0.63 any ! WHAT GETS OUT access-list 130 deny ip any any log access-list 131 permit icmp any 220.144.159.128 0.0.0.63 ! WHAT GETS IN access-list 131 permit tcp any 220.144.159.128 0.0.0.63 eq ident access-list 131 permit tcp any 220.144.159.128 0.0.0.63 established ! 220.144.159.129 is LOGHOST, TACACS+, FTP-dumps, TFTP access-list 131 permit udp 220.144.159.192 0.0.0.63 host 220.144.159.129 eq syslog access-list 131 permit udp 220.144.159.192 0.0.0.63 host 220.144.159.129 eq tftp access-list 131 permit tcp 220.144.159.192 0.0.0.63 host 220.144.159.129 eq ftp access-list 131 permit tcp 220.144.159.192 0.0.0.63 host 220.144.159.129 eq ftp-data access-list 131 permit tcp 220.144.159.192 0.0.0.63 host 220.144.159.129 eq tacacs ! 220.144.159.130 is NOC MGMT Workstation access-list 131 permit udp 220.144.159.192 0.0.0.63 host 220.144.159.130 eq snmp access-list 131 permit udp 220.144.159.192 0.0.0.63 host 220.144.159.130 eq snmptrap

348

! 220.144.159.131 is SSH gateway access-list 131 permit tcp any host 220.144.159.131 eq 22 ! 220.144.159.132 is DIAL Radius server (NT running Cisco Secure) access-list 131 permit udp host 220.144.159.200 host 220.144.159.132 eq 1645 access-list 131 permit udp host 220.144.159.200 host 220.144.159.132 eq 1646 ! 220.144.159.180 is Primary DNS access-list 131 permit udp any host 220.144.159.180 eq domain access-list 131 permit tcp any host 220.144.159.180 eq domain access-list 131 deny ip any any log ! VTY access-list - NOC access-list 198 permit ip 220.144.159.128 0.0.0.63 any access-list 198 permit ip 220.144.159.192 0.0.0.63 any ! Router loopbacks access-list 198 deny ip any any log access-list 199 deny ip any any log ! tacacs-server host 220.144.159.129 tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo banner login ^ Authorized Access Only This system is the property of Galaxy Internet Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line aux 0 transport preferred none transport input telnet transport output telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! exception protocol ftp exception dump 220.144.159.129 ntp authentication-key 1 md5 secretAlso ntp authenticate ntp trusted-key 1 ntp source Loopback0

349

ntp server 220.144.159.193 key 1 ntp server 220.144.159.194 key 1 end

Acce ss Se r ve r This is t he t y pical configur at ion of an access ser v er w it h a connect ed m odem bank . The rout er in Ex am ple C- 6 is assum ed t o be a 3640 w it h 64 asy nchr onous por t s. Ex a m ple C- 6 Acce ss Se r v e r Con f ig u r a t ion Ex a m p le version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname Access-router ! boot system flash logging buffered 16384 debugging aaa new-model aaa new-model aaa authentication login default group tacacs+ enable aaa authentication login radius-login group radius aaa authentication enable default group tacacs+ enable aaa authentication ppp default none aaa authentication ppp ppp-auth if-needed group radius aaa authorization network radius-auth group radius aaa accounting network default stop-only group radius enable secret shhhhhthisisasecret ! clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! SSH support ip ssh time-out 120 ip ssh authentication-retries 3 ! partition flash 2 8 8 !

350

interface Loopback0 description Loopback interface on Access-Router ip address 220.144.159.200 255.255.255.255 no ip directed-broadcast ! interface FastEthernet0/0 description Core Ethernet (SW1 port 6) ip address 220.144.159.104 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface FastEthernet0/1 description Core Ethernet (SW2 port 6) ip address 220.144.159.120 255.255.255.240 no ip redirects no ip directed-broadcast no ip proxy-arp ! interface Group-Async1 ip unnumbered Loopback0 no ip directed-broadcast encapsulation ppp ip tcp header-compression passive async mode interactive peer default ip address pool ppp-dialin no cdp enable ppp max-bad-auth 3 ppp reliable-link ppp authentication chap ms-chap pap ppp-auth ppp authorization radius-auth group-range 65 128 ! router ospf 100 log-adjacency-changes network 220.144.159.200 0.0.0.0 area 0 network 220.144.159.96 0.0.0.15 area 0 network 220.144.159.112 0.0.0.15 area 0 passive-interface Loopback0 ! router bgp 64511 network 220.144.157.0 mask 255.255.255.128 no synchronization bgp log-neighbor-changes neighbor rr peer-group neighbor rr remote-as 64511 neighbor rr update-source Loopback0 neighbor rr password BGPsecretPW neighbor rr send-community neighbor 220.144.159.193 peer-group rr neighbor 220.144.159.193 description Core1 neighbor 220.144.159.194 peer-group rr neighbor 220.144.159.194 description Core2 no auto-summary ! ip local pool ppp-dialin 220.144.157.1 220.144.157.64 !

351

ip classless ip route 10.0.0.0 255.0.0.0 Null0 ip route 172.16.0.0 255.240.0.0 Null0 ip route 192.168.0.0 255.255.0.0 Null0 ip route 220.144.128.0 255.255.224.0 Null0 ip route 220.144.157.0 255.255.255.128 Null0 ip tacacs source-interface Loopback0 no ip http server ip bgp-community new-format ! ip radius source-interface Loopback0 no logging console logging trap debugging logging facility local5 logging source-interface Loopback0 logging 220.144.159.129 access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log ! ! Access-list 155 is a dynamic access used to restrict e-mail only clients ! to the PoP3 and MAIL relay servers access-list 155 permit tcp any host 220.144.159.4 eq pop3 access-list 155 permit tcp any host 220.144.159.4 eq smtp access-list 155 deny ip any any ! ! Access-list 156 is a dynamic access list used to restrict ! clients with expired subscriptions to the billing server ! so they can renew their subscription – it is the default unless ! clients have paid up :-) access-list 156 permit tcp any host 220.144.129.36 eq www access-list 156 deny ip any any ! VTY access-list - NOC access-list 198 permit ip 220.144.159.128 0.0.0.63 any access-list 198 permit ip 220.144.159.192 0.0.0.63 any ! Router loopbacks access-list 198 deny ip any any log access-list 199 deny ip any any log ! dialer-list 1 protocol ip permit ! tacacs-server host 220.144.159.129 tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo radius-server host 220.144.159.132 auth-port 1645 acct-port 1646 radius-server key AlsoSecret radius-server vsa send accounting radius-server vsa send authentication banner login ^ Authorized Access Only This system is the property of Galaxy Internet

352

Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line 65 128 session-timeout 5 output exec-timeout 1 0 autoselect during-login autoselect ppp login authentication radius-login modem Dialin autocommand ppp length 25 transport input all escape-character NONE autohangup stopbits 1 flowcontrol hardware line aux 0 transport preferred none transport input telnet transport output telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! exception protocol ftp exception dump 220.144.159.129 ntp authentication-key 1 md5 secretAlso ntp authenticate ntp trusted-key 1 ntp source Loopback0 ntp server 220.144.159.193 key 1 ntp server 220.144.159.194 key 1 ntp peer 220.144.159.196 key 1 ntp peer 220.144.159.197 key 1 ntp peer 220.144.159.198 key 1 end

Ou t -of- Ba nd Console Server This is t he t y pical configur at ion of an access ser v er t hat has been configur ed as an out - of- band m anage m ent dev ice ( r efer t o Exam ple C- 7) . The r out er in t his case is a 2611 r out er w it h 32 asy nchr onous por t s t hat hav e been w ir ed t o t he consoles of t he equipm ent used in Figur e C- 1 and descr ibed in t he pr eceding configur at ions.

353

Ex a m ple C- 7 Out - o f- Ba n d Con sole Se r v e r Con f ig u r a t ion Ex a m p le version 12.1 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption ! hostname cs2611 ! logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ enable secret shhhhhthisisasecret ! clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip rcmd source-interface Loopback0 ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip ftp username cisco ip ftp password shhhhsecret no ip bootp server ip domain-name net.galaxy ip name-server 220.144.159.1 ip name-server 220.144.159.2 ip name-server 219.10.2.1 ! ! interface Ethernet0/0 description Galaxy Net Core LAN ip address 220.144.159.189 255.255.255.192 no ip route-cache no ip mroute-cache ! ip classless ip route 0.0.0.0 0.0.0.0 220.144.159.190 ip tacacs source-interface Ethernet0/0 ! ! map IP addresses to reverse telnet ports – easier for humans to deal with ! than trying to remember port numbers ip alias 220.144.159.160 2065 ip alias 220.144.159.161 2066 ip alias 220.144.159.162 2067 ip alias 220.144.159.163 2068 ip alias 220.144.159.164 2069 ip alias 220.144.159.165 2070

354

ip alias 220.144.159.166 2071 ip alias 220.144.159.167 2072 ip alias 220.144.159.168 2073 ip alias 220.144.159.169 2074 ip alias 220.144.159.170 2075 ip alias 220.144.159.171 2076 ip alias 220.144.159.172 2077 ip alias 220.144.159.173 2078 ip alias 220.144.159.174 2079 ip alias 220.144.159.175 2080 ip alias 220.144.159.176 2081 ip alias 220.144.159.177 2082 ip alias 220.144.159.178 2083 ip alias 220.144.159.179 2084 ! no ip http server ! no logging console logging trap debugging logging source-interface Loopback0 logging 220.144.159.129 access-list 1 permit 220.144.159.129 access-list 1 permit 220.144.159.130 access-list 1 deny any log ! VTY access-list - NOC access-list 198 permit ip 220.144.159.128 0.0.0.63 any access-list 198 permit ip 220.144.159.192 0.0.0.63 any ! Router loopbacks access-list 198 deny ip any any log access-list 199 deny ip any any log ! tacacs-server host 220.144.159.129 tacacs-server key SecretToo snmp-server community NotTelling RO 1 snmp-server location Somewhere snmp-server contact Network Operations Center snmp-server enable traps snmp snmp-server host 220.144.159.130 SecretToo banner login ^ Authorized Access Only This system is the property of Galaxy Internet Disconnect IMMEDIATELY if you are not an authorized user! Contact [email protected] +98 765 4321 for help. ^ ! line con 0 exec-timeout 3 0 transport preferred none transport input none transport output telnet line aux 0 transport preferred none transport input telnet transport output telnet

355

line 65 location Console of SW1 no exec transport preferred none transport input telnet line 66 location Console of SW2 no exec transport preferred none transport input telnet line 67 location Console of Border-router no exec transport preferred none transport input telnet line 68 location Console of Core1 no exec transport preferred none transport input telnet line 69 location Console of Core2 no exec transport preferred none transport input telnet line 70 location Console of Gateway1 no exec transport preferred none transport input telnet line 71 location Console of Gateway2 no exec transport preferred none transport input telnet line 72 location Console of Service1 no exec transport preferred none transport input telnet line 73 location Console of Service 2 no exec transport preferred none transport input telnet line 74 location Console of NOC router no exec transport preferred none transport input telnet line 75 location Console of SSH Gateway no exec transport preferred none transport input telnet line 76 location Console of LOGHOST/TACACS+/NOC Workstation

356

no exec transport preferred none transport input telnet line 77 location Console of Network Management Workstation no exec transport preferred none transport input telnet line 78 location Console of RADIUS server no exec transport preferred none transport input telnet line 79 location Console of Primary DNS no exec transport preferred none transport input telnet line 80 location Console of Secondary DNS no exec transport preferred none transport input telnet line 81 location Console of Secondary MX no exec transport preferred none transport input telnet line 82 location Console of Primary MX, ftp, www no exec transport preferred none transport input telnet line 83 location Console of PoP3 server no exec transport preferred none transport input telnet line 84 location Console of DIAL News no exec transport preferred none transport input telnet line 85 96 location Spare no exec transport preferred none transport input telnet line vty 0 4 access-class 198 in exec-timeout 0 0 transport preferred none transport input telnet ssh transport output telnet ! exception protocol ftp exception dump 220.144.159.129

357

ntp ntp ntp ntp ntp ntp end

authentication-key 1 md5 secretAlso authenticate trusted-key 1 source Loopback0 server 220.144.159.193 key 1 server 220.144.159.194 key 1

Su m m a r y This appendix offer ed a sim ple exam ple of t he configur at ions used in a m edium- size I SP. The pr eceding ex am ples ar e k now n t o w or k at t he t im e of t his w r it ing, and t hey should be useful in helping new I SP businesses get a foot on t he ladder t o success. OSPF has been used as t he I GP her e sim ply because t he candidat e sam ple net w or k im plem ent ed OSPF. I m plem ent ing t he net w or k using I S - I S or EI GRP w ould also be quit e feasible.

BGP Fla p D a m pin g Con figu r a t ion Recom m ended r out e flap- dam ping par am et er s for use by I SPs w er e com posed int o a docum ent by t he RI PE Rout ing Wor king Gr oup and ar e available at w w w .r ipe.net / docs/ r ipe- 229.ht m l . These values ar e used by m any Eur opean and U.S. I SPs, and t hey ar e based on t he oper at ional ex per ienced gained in t he indust r y . The configur at ion ex am ples ar e r epr oduced her e for conv enience—t he v alues hav e been updat ed t o include r ecent changes in t he locat ions of t he r oot nam eser v er s. The cur r ent addr ess list is docum ent ed at w w w .golden- net works.net —t his sit e should be consult ed pr ior t o im plem ent ing RI PE- 229 flap dam ping.

I P Access List Exam ple This is a configur at ion ex am ple using t he I P access list , sim ilar t o w hat is quot ed in t he RI PE- 229 docum ent . We don’t r ecom m end t his m et hod any longer—I P pr efix list s hav e long super seded access list s for pr efix filt er ing. See t he nex t sect ion for t he pr efix- list configur at ion. Not e t hat access- list 180 cov er ing t he r oot nam eser v er and access- list 184 covering t he global t op- level dom ain nam eser ver net w or ks have been updat ed w it h t he m ost r ecent v alues. I t is st r ongly r ecom m ended t hat y ou check t hese addr esses and net w or k pr efix es announced t o t he I nt er net befor e im plem ent ing t he filt ers. ( You can use t he UNI X com m and dig . n s t o find t he nam eser v er addr esses and t hen use sh ip b g p x .x .x .x on a r out er t o find t he size of t he pr efix being adver t ised in t he I nt er net r out ing t able.)

router bgp 65280 bgp dampening route-map RIPE229-flap-damp

358

! no flap dampening for special user defined networks defined in access-list 183 route-map RIPE229-flap-damp deny 10 match ip address 183 ! no flap dampening for root nameserver networks in access-list 180 route-map RIPE229-flap-damp deny 20 match ip address 180 ! no flap dampening for gtld nameserver networks in access-list 184 route-map RIPE229-flap-damp deny 30 match ip address 184 ! flap dampening for all the /24 and longer prefixes route-map RIPE229-flap-damp permit 30 match ip address 181 set dampening 30 820 3000 60 ! flap dampening for all /22 and /23 prefixes route-map RIPE229-flap-damp permit 40 match ip address 182 set dampening 15 750 3000 45 ! flap dampening for all remaining prefixes route-map RIPE229-flap-damp permit 50 set dampening 10 1500 3000 30 ! ! Access Lists for route flap dampening as per RIPE-229 definition ! with updated root server networks access-list 180 permit ip host 198.41.0.0 host 255.255.255.0 ! A J access-list 180 permit ip host 128.9.0.0 host 255.255.0.0 ! B access-list 180 permit ip host 192.33.4.0 host 255.255.255.0 ! C access-list 180 permit ip host 128.8.0.0 host 255.255.0.0 ! D access-list 180 permit ip host 192.203.230.0 host 255.255.255.0 ! E access-list 180 permit ip host 192.5.4.0 host 255.255.254.0 ! F access-list 180 permit ip host 192.112.36.0 host 255.255.255.0 ! G access-list 180 permit ip host 128.63.0.0 host 255.255.0.0 ! H access-list 180 permit ip host 192.36.148.0 host 255.255.255.0 ! I access-list 180 permit ip host 193.0.14.0 host 255.255.255.0 ! K access-list 180 permit ip host 198.32.64.0 host 255.255.255.0 ! L access-list 180 permit ip host 202.12.27.0 host 255.255.255.0 ! M access-list 180 deny ip any any ! match /24 prefixes access-list 181 permit ip any 255.255.255.0 0.0.0.255 access-list 181 deny ip any any ! match /22 and /23 prefixes access-list 182 permit ip any 255.255.252.0 0.0.3.255 access-list 182 deny ip any any ! match special prefixes access-list 183 permit ip host 169.223.0.0 host 255.255.0.0 access-list 183 deny ip any any ! updated gtld name server networks access-list 184 permit ip host 192.5.6.0 host 255.255.255.0 ! A access-list 184 permit ip host 192.33.14.0 host 255.255.255.0 ! B access-list 184 permit ip host 192.26.92.0 host 255.255.255.0 ! C access-list 184 permit ip host 192.31.80.0 host 255.255.255.0 ! D access-list 184 permit ip host 192.12.94.0 host 255.255.255.0 ! E access-list 184 permit ip host 192.35.51.0 host 255.255.255.0 ! F access-list 184 permit ip host 192.42.93.0 host 255.255.255.0 ! G access-list 184 permit ip host 192.54.112.0 host 255.255.255.0 ! H access-list 184 permit ip host 192.36.144.0 host 255.255.255.0 ! I

359

access-list access-list access-list access-list access-list !

184 184 184 184 184

permit ip host 210.132.96.0 host 255.255.224.0 permit ip host 213.177.192.0 host 255.255.248.0 permit ip host 192.41.162.0 host 255.255.255.0 permit ip host 202.153.112.0 host 255.255.240.0 deny ip any any

! ! ! !

J K L M

I P Prefix List Exam ple Pr efix list s also can be used and, indeed, should be used. The pr eceding ex am ple w as r ew r it t en using t he ip prefix - list com m ands av ailable in Cisco I OS Soft w ar e v er sions 11.1CC and 12.0 and lat er soft w ar e r eleases. This m ak es t he configur at ion m or e r eadable, if not m or e int uit ive. Again, r em em ber t o check t he r oot nam eser ver net w or k s for any changes befor e im plem ent ing t hese. I t is also w or t h check ing t he BGP rout ing t able t o ensur e t hat t hese net w or k s st ill ar e announced w it h t he following prefix lengt hs:

router bgp 65280 bgp dampening route-map RIPE229-flap-damp ! ip prefix-list my-nets description Networks we don't suppress ip prefix-list my-nets seq 5 permit 169.223.0.0/16 ! ip prefix-list suppress22 description Dampening of /22 and /23 prefixes ip prefix-list suppress22 seq 5 permit 0.0.0.0/0 ge 22 le 23 ! ip prefix-list suppress24 description Dampening of /24 and longer prefixes ip prefix-list suppress24 seq 5 permit 0.0.0.0/0 ge 24 ! ip prefix-list rootns description Root-nameserver networks ip prefix-list rootns permit 198.41.0.0/24 ! A & J ip prefix-list rootns permit 128.9.0.0/16 ! B ip prefix-list rootns permit 192.33.4.0/24 ! C ip prefix-list rootns permit 128.8.0.0/16 ! D ip prefix-list rootns permit 192.203.230.0/24 ! E ip prefix-list rootns permit 192.5.4.0/23 ! F ip prefix-list rootns permit 192.112.36.0/24 ! G ip prefix-list rootns permit 128.63.0.0/16 ! H ip prefix-list rootns permit 192.36.148.0/24 ! I ip prefix-list rootns permit 193.0.14.0/24 ! K ip prefix-list rootns permit 198.32.64.0/24 ! L ip prefix-list rootns permit 202.12.27.0/24 ! M ! ip prefix-list gtldns description Global Top Level Domain Servers ip prefix-list gtldns permit 192.5.6.0/24 ! A ip prefix-list gtldns permit 192.33.14.0/24 ! B ip prefix-list gtldns permit 192.26.92.0/24 ! C ip prefix-list gtldns permit 192.31.80.0/24 ! D ip prefix-list gtldns permit 192.12.94.0/24 ! E ip prefix-list gtldns permit 192.35.51.0/24 ! F ip prefix-list gtldns permit 192.42.93.0/24 ! G ip prefix-list gtldns permit 192.54.112.0/24 ! H ip prefix-list gtldns permit 192.36.144.0/24 ! I

360

ip prefix-list gtldns permit 210.132.96.0/19 ip prefix-list gtldns permit 213.177.192.0/21 ip prefix-list gtldns permit 192.41.162.0/24 ip prefix-list gtldns permit 202.153.112.0/20 ! route-map RIPE229-flap-damp deny 10 match ip address prefix-list my-nets ! route-map RIPE229-flap-damp deny 20 match ip address prefix-list rootns ! route-map RIPE229-flap-damp deny 30 match ip address prefix-list gtldns ! route-map RIPE229-flap-damp permit 40 match ip address prefix-list suppress24 set dampening 30 820 3000 60 ! route-map RIPE229-flap-damp permit 50 match ip address prefix-list suppress22 set dampening 15 750 3000 45 ! route-map RIPE229-flap-damp permit 60 set dampening 10 1500 3000 30 !

! ! ! !

J K L M

361

Appe ndix E. Tr a ffic Engine e r ing Tools As a follow- up on how t o t r ack w her e y our cust om er s ar e going on t he I nt er net , t his appendix pr ov ides a list of publicly av ailable t ools t hat can be used t o pull in st at ist ics fr om y our net w or k . Most I SPs do not use t hings like HP OpenView , Sun Net Manager , CiscoWor k s, or Spect r um t o m anage t heir net w or k s. These net w or k m anagem ent pack ages ar e gr eat for t he ent er pr ise LANs, but t hey do not hav e t he sim ple scaleable t ools needed for I SP net w or k s. I nst ead, I SPs pull t oget her differ ent , m ost ly public - dom ain, t ools and use UNI X scr ipt s t o gener at e char t s and r epor t s ( w it h GNUplot or RRDTool) for t r affic engineer ing and qualit y of ser v ice ( QoS) m anagem ent . Not e t hat one Cisco- specific solut ion is t o use Net Flow . Net Flow first w as m ade available on 75 x x, 72 x x, and 7000/ RSP r unning t he 11.1CC. The int r oduct ion of r elease 12.0 m eant t hat Net Flow becam e available on t he 3600 ser ies and higher plat for m s fr om 12.0( 1) , and becam e available on t he sm aller Cisco I OS Soft w ar e plat for m s fr om 12.0( 2) T. I t is also available in t he for m of Sam pled Net Flow on t he GSR. Ther e ar e a few w hit epaper s on t he Cisco w eb pages—sear ching using t he Cisco. com sear ch engine w ill r ev eal sev er al of t hose quit e r eadily .

I n t e r n e t Tr a ffic a n d N e t w or k En gin e e r in g Tools This sect ion list s j ust som e of t he m any I nt er net t r affic and net w or k engineer ing t ools av ailable. These ar e t he t ools t hat I SPs use t o m onit or t heir net w or k s, help m onit or t r affic flow s, and det er m ine m any business- cr it ical funct ions such as peer ing policies, QoS, and so on. St an Bar ber pr esent ed a t alk at t he Febr uar y 1998 NANOG t it led “ Monit or ing Your Net w or k w it h Fr eely Available St at ist ics Repor t ing Tools.” The slides for t his pr esent at ion ar e av ailable at ht t p: / / w w w .academ .com / nanog/ feb1998/ net t ools.ht m l and ar e r ecom m ended r eading for all I SPs w onder ing how t o m onit or t heir new infr ast r uct ur e.

CAI DA The Cooper at iv e Associat ion for I nt er net Dat a Analy sis ( CAI DA) has a v er y com pr ehensiv e w eb sit e list ing a lot of t ools and point er s. This CAI DA effor t is suppor t ed t hr ough sponsor ship pr ov ided by t he Unit ed St at es Nat ional Science Foundat ion ( NSF) , Cisco Syst em s, and ot her or ganizat ions. CAI DA: ht t p: / / w w w .caida.or g CAI DA I nt er net Tools Tax onom y : ht t p: / / w w w .caida.or g/ t ools/ t ax onom y .ht m l

Scion/ N et Sca rf This w as a pr oj ect by Mer it Net w or k , I nc., t o get a pict ur e of w hat is happening on t he I nt er net . Apar t fr om ver sions t hat r un on m ost UNI X plat for m s and t heir v ar iant s, t her e is a v er sion t hat r uns on Window s NT. The m ost r ecent v er sion of t he soft w ar e w as r eleased in June 1997. The pr oj ect has now ceased because funding

362

has r un out , but t he w eb sit e and soft w ar e ar e st ill available: ht t p: / / w w w . m er it . edu/ ~ net scar f/ .

N eTraM et / N et Flow M et NeTr aMet is one of t he or iginal and bet t er t ools for TCP/ I P flow analy sis. SingNet used NeTr aMet on an I nt el PC w it h BSD UNI X and a Digit al FDDI car d. The r esult s w er e dum ped ont o a sy st em t hat did all t he flow analy sis, and t he r esult s w er e post ed t o an int er nal w eb ser v er . Mor e r ecent ly , a capabilit y has been added t o analy ze Cisco Net Flow r ecor ds—Net Flow Met is par t of t his pack age now . ( See ht t p: / / w w w . auck land. ac. nz/ net / NeTr aMet / . )

cf low d One of t he m ost popular public - dom ain t ools being used for Net Flow analysis is cflow d. I t is host ed by CAI DA and can be found at ht t p: / / w w w .caida.or g/ t ools/ m easur em ent / cflow d/ . The k ey Cisco docum ent s on Net Flow ar e const ant ly being updat ed ( because Cisco is adding new feat ur es and funct ionalit y all t he t im e) . I t ’s best t o do a k ey w or d sear ch on Cisco. com t o find all t he docum ent at ion on Net Flow . Net Flow t ools ( flow dat a.h, fdr ecor der .c, fdplay back .c, and fdg.c) t hat w er e used t o build cflow d ar e locat ed on Cisco’s FTP sit e: ft p: / / ft peng.cisco.com / ft p/ Net Flow / fde/ README.

M RTG Mult i Rout er Traffic Grapher ( MRTG) is a Perl script - based pack age t o cr eat e gr aphics of int er face loading on t he r out er s. I t sav es y ou fr om hav ing t o cr eat e UNI X scr ipt s t o do t he sam e t hing. MRTG w orks on UNI X ( requiring t he m ore re cent v er sions of Per l 5) , and t her e is a v er sion for Window s NT/ 2000, t oo. The pack age also cont ains cont r ibut ed t ools enabling y ou t o m onit or CPU loads, disk space, t em per at ur e, and m any ot her funct ions t hat an I SP can use t o w at ch a net w or k . I ndeed, v ir t ually anyt hing can be m onit or ed and gr aphed w it h MRTG, w hich m akes it a ver y pow er ful and ex t r em ely popular t ool for I SPs. Ev er y t hing t o do w it h MRTG can be found at ht t p: / / w w w .m r t g.or g.

RRDTool RRDTool com es from t he aut hor of MRTG and is designed t o be a m ore pow erful and flexible syst em for gr aphing collect ed st at ist ics. I t is not m eant t o be a full r eplacem ent for MRTG, and fut ur e v er sions of MRTG ar e planned t o sit on t op of RRDTool. RRDTool is gaining popular it y w it h I SPs t hat w ant gr eat er v ar iet y and flex ibilit y in t he gr aphing t hat t hey can do w it h t heir MRTG capt ur ed dat a. I t can be found at it s hom e sit e, ht t p: / / w w w .r r dt ool.or g.

363

Linux N et w ork Managem e n t Tools This sit e pr ov ides a com pr ehensiv e list of m any of t he cur r ent I nt er net net w or k m anagem ent t ools. I t includes m any of t he t ools list ed in t his appendix—for r efer ence pur poses, consult h t t p: / / linas.or g/ linux / NMS.ht m l.

Vult ure SNMP Vult ure is a t ool t o do long- t er m SNMP dat a collect ion and analy sis of r out er s and ot her sim ilar devices. Vult ur e has a num ber of feat ur es t hat m ake it suit able for v ar ious t ask s: • • • • •

Per- int er face configur at ion. Differ ent dat a can be collect ed on each defined int er face. Tem plat e- based configur at ion. Differ ent sor t s of int er faces m ight r equir e r ecor ding differ ent infor m at ion; w hoev er hear d of collisions on a ser ial int er face? Configur able per- r out er com m unit y st r ings. Web- based gr aphical br ow sing of st or ed dat a. Built - in dat a- archival m echanism for st ale dat a.

Vult ur e is w r it t en in Per l ver sion 5 and uses t he CMU ver sion 2 libr ar ies t o do t he low- level SNMP access. The Vult ur e dist r ibut ion includes bot h t he CMU libr ar ie s and a sm all m odule t o connect t he libr ar ies t o Per l. The br ow ser int er face also r equir es t he gener ally av ailable Gnuplot and PBMPlus ut ilit ies t o gener at e gr aphical out put . See ht t p: / / w w w . v ix . com/ v ix / v ult ur e/ for m ore det ails.

N e t SN M P The CMU SNMP pr oj ect w as t ak en ov er by UCD, br iefly called UCD- SNMP and m ore r ecent ly r enam ed and m ov ed ov er t o Sour ceFor ge. The soft w ar e has been ex t ensiv ely m odified and updat ed and is now consider ed quit e a useable, ext ensible, and configur able sy st em . Ev en bet t er , it is fr eely av ailable at ht t p: / / net snm p. sour cefor ge. net / .

SysMon Sysm on is a net work- m onit or ing t ool designed by Jar ed Mauch t o pr ovide hi gh per for m ance and accur at e net w or k m onit or ing. Cur r ent ly suppor t ed pr ot ocols include SMTP, I MAP, HTTP, TCP, UDP, NNTP, and PI NG t est s. Mor e infor m at ion and t he lat est v er sion can be found at ht t p: / / w w w . sy sm on.org/ .

Tr e n o Tr eno is a t ool t o develop end- t o- end per for m ance infor m at ion. I t can be t r ied out fr om PSC fr om t his WWW for m int er face: ht t p: / / w w w .psc.edu/ ~ pscnoc/ t r eno.ht m l . For m ore inf or m at ion about Tr eno, go t o ht t p: / / w w w .psc.edu/ ~ pscnoc/ t r eno_info.ht m l.

364

Scot t y—Tcl Ext ensions for N et w ork Managem ent Applica t ion s Scot t y is t he nam e of a soft w ar e pack age t hat allow s implem ent at ion of sit e- specific net w or k m anagem ent soft w ar e using high- lev el, st r ing- based API s. The soft w ar e is based on t he Tool Com m and Language ( Tcl) , t he lat est sour ce for w hich can be found at ht t p: / / www.script ics.com / , and it is now par t of m ost Linux dist r ibut ions. Tcl sim plifies t he dev elopm ent of por t able net w or k m anagem ent scr ipt s. The Scot t y sour ce dist r ibut ion includes t w o m aj or com ponent s. The fir st is t he Tnm Tcl Ex t ension, w hich pr ov ides ac cess t o net w or k m anagem ent infor m at ion sour ces. The second com ponent is t he Tk ined net w or k edit or t hat pr ov ides a fr am ew or k for an ex t ensible net w or k m anagem ent sy st em . This t ool used t o be called Tk ined. For m or e infor m at ion, see ht t p: / / w w w hom e. cs. ut w ent e. nl/ ~ schoenw / scot t y / .

N e t Sa int Net Saint is a w eb- based net work- m onit or ing t ool used by sev er al I SPs. I t can m onit or ser v er s and t he net w or k and send e- m ail or pages if t here are problem s. I t ’s w r it t en in C and designed t o r un on Linux. See ht t p: / / net saint . sour cefor ge. net / for m or e infor m at ion.

Big Br ot he r Anot her useful t ool, sim ilar t o Net Saint , is Big Br ot her . Sour ce code is available for UNI X and Linux. Client s include NT, Novell, and Mac. Visit ht t p: / / www.bb4.com / for det ails. N OTE Wit h all of t his soft w are available, it is not ex pensiv e or difficult t o collect and analy ze dat a on y our net w or k . You can cr eat e y our ow n t ools and r un t hem on t he m any I nt el- based UNI X syst em s ( for exam ple, Linux, BSDI , and so on) .

Ot h e r Use fu l Tools t o M a n a ge You r N e t w or k A var iet y of ot her t ools ar e available t o help w it h t he configur at ion and m anagem ent of t he net w or k . Most I SPs ar e w ell aw ar e of t hese, but it is w or t h m ak ing a list of t he m ost useful ones t o assist new er ent r ant s t o t he indust r y in get t ing a good feel for w hat is current ly in use.

t ra cerout e The st andard UNI X t r a ce r ou t e com m and is pr obably t he m ost basic t ool av ailable for fault det ect ion and diagnosis in t he I nt er net t oday . The ut ilit y also has spaw ned sev er al enhancem ent s, such as pr t r acer out e fr om t he PRI DE pr oj ect and t he m or e adv anced t r acer out e t hat com es w it h Linux . Cisc o I OS Soft w ar e, of cour se, has it s ow n t r acer out e com m and, using t he r out ing t able in t he r out er t o pr ov ide det ailed

365

infor m at ion. The sour ce for t r acer out e can be found at ft p: / / ft p. r ipe.net / t ools/ t r acer out e.t ar .Z. For t hose w it hout access t o any of t hese, or if y ou r equir e a t r acer out e fr om anot her par t of t he I nt er net , an ex cellent public t r acer out e facilit y is locat ed at ht t p: / / w w w .t r acer out e.or g, a cat alog of t r acer out e “ look ing glasses” ar ound t he w or ld.

Look ing Gla sse s A look ing glass isn’t r eally a UNI X t ool as such; it ’s m or e a facilit y t hat has been m ade av ailable by a lar ge num ber of I SPs and ot her or ganizat ions ar ound t he globe. I t pr ov ides a w eb int er face t o user s, allow ing t hem t o check condit ions of t he I nt er net r out ing t able at t hat sit e. A com plet e list of look ing glass sit es can be found at ht t p: / / w w w .t r acer out e.org. The m ost w ell know n and one of t he fir st looking glasses w as t he one set up by Ed Ker n at Digex . I t can be found at ht t p: / / nit rous.digex.net . The sour ce code for Ed’s Looking Glass is also t here —it builds on any UNI X syst em . I SPs t hat w ant t o m ak e a look ing glass av ailable should do so w it h due consider at ion of any secur it y issues and, of cour se, loading on t he r out er . I deally , look ing glass– suppor t ing r out er s should do j ust t hat and should not be par t of any cr it ical infr ast r uct ur e.

w h ois Anot her st andar d UNI X com m and t hat is inv aluable for use on t he I nt er net is w hois. Many enhancem ent s hav e been m ade t o w hois it self by t he PRI DE pr oj ect in t he ear ly 1990s and t hen by t he RI PE NCC, t o opt im ize it t o suppor t look ups on I nt er net dat abases t hat w er e based on t he RI PE dat abase soft w ar e. I t is r ecom m ended t hat y ou r eplace t he shipping w hois client on UNI X sy st em s w it h t he RI PE v er sion, av ailable at ft p: / / ft p.r ipe.net / t ools/ r ipe- whois- t ools- 2.4.t ar .gz.

Gn u plot Gnuplot is a useful public - dom ain graphing t ool. I f configuring MRTG is t oo m uch, or if y ou need t o gr aph som et hing else quick ly , t his is pr obably t he w ay t o do it . See ht t p: / / w w w .gnuplot .or g/ or ht t p: / / sour cefor ge. net / pr oj ect s/ gnuplot / for det ails.

RTRM on—A Tool for Rout er Monit oring and M a nipula t i on The RTR sy st em cur r ent ly com es w it h t hr ee pr ogr am s, r t r m on, r t r pass, and r t r login. r t r m on ( for “ r out er m onit or ” ) is t he cor e of t he sy st em . I t uses pr edefined act ions t o log int o r out er s, issue com m ands, pr ocess t he out put , ar chiv e t he r esult s, and possibly m ail r epor t s. I t is designed t o pr ovide t he fr am ew or k for a var iet y of pot ent ial m onit or ing t ask s and t o be r eadily ex t ensible w it h new r epor t ing code if t he

366

built - in m et hods ar e insufficient for com plex analysis. r t r m on can even updat e r out er configurat ions, despit e it s m onit or m oniker. The r t r pass pr ogr am is m eant t o pr ovide an easy int er face t o a m or e secur e m et hod of st or ing passw or ds. Because r t r m on needs t o be capable of pr ov iding passw or ds t o r out er s t o log int o t hem and t o gain enable pr ivileges, r out er s m ust be accessible t o t he host s on w hich r t r m on is r unning. To r educe t he r isk s associat ed w it h t his, rt rpass m anages a PGP- encr ypt ed file for each r t r m on user t hat cont ains his ow n passw or d and his enable passw or d, if t he user has enable access. Passw or ds can be cont r olled on a per- rout er basis, if desired. rt rlogin is used t o log int o rout ers, t o aut om at ically log in using a usernam e and passw or d, t o get enable pr iv ileges, t o set t he t er m inal lengt h and w idt h t o t he size of y our w indow , and t o r un any com m ands t hat you have saved in your per sonal login file. The session t hen can be used int er act iv ely . See ht t p: / / w w w .v ix .com / v ix / r t r m on/ for m ore inform at ion.

RAToolSet / I RRToolSet The RAToolSet is a suit e of t ools t hat enable I SPs t o sim plify and st r eam line t he im plem ent at ion of BGP policies in t heir peer ings w it h ot her I SPs. The soft w ar e suit e r uns on m ost UNI X sy st em s and w as built by Cengiz Alaet t inoglu w hile w or k ing at I SI in California. The RAToolSet r equir es t he exist ence of a r out ing r egist r y fr om w hich t o der ive it s configur at ions—t his r out ing r egist r y can be par t of t he I nt er net Rout ing Regist r y ( I RR) , or it can be a pr iv at e r out ing r egist r y r un by t he I SP. The r egist r y cont ains policy descr ipt ions of r out es t hat appear in t he I nt er net , policy descr ipt ions of aut onom ous sy st em s used in t he I nt er net , and ot her v aluable infor m at ion. The t hr ee regional I nt ernet regist ries are part of t he I nt ernet Rout ing Regist ry —ot her m em bers of t he I RR include t he RADB ( r un by Mer it Net w or k ) and sev er al pr ov ider- run rout ing regist ries. The RAToolSet now is m aint ained by t he RI PE NCC and has been renam ed t he I RRToolSet t o r eflect it s funct ion in t oday ’s I nt er net . Mor e infor m at ion can be found on t he RI PE NCC’s w eb sit e at ht t p: / / w w w .r ipe.net / r ipencc/ pubser v ices/ db/ ir r t oolset / index .ht m l —t his includes infor m at ion about t he or igin and r equir em ent s t o use t he t ools et , as w ell as a m ailing list of user s and par t icipant s. The soft w ar e can be dow nloaded fr om ft p: / / ft p.r ipe.net / t ools/ I RRToolSet . Feat ur es t hat ar e par t of t he I RRToolSet include Rt Config ( a r out er configur at ion gener at or ) , AOE ( AS Obj ect Edit or ) , and ROE ( Rout e Obj ect Edit or ) . These t hr ee t ools ar e per haps t he m ost w idely used com ponent s of t he t oolset , and m any I SPs m ak e use of t hem t o sim plify and aut om at e t he gener at ion of eBGP configu rat ions.

Cisco’s M I Bs All t he Cisco SNMP MI Bs ar e publicly av ailable. I f y ou hav e com m er cial SNMP m anagem ent packet s or shar ew ar e - fr eew ar e pack et s, y ou m ight need t o gr ab t he MI B. Here is t he FTP sit e: ft p: / / ft p.cisco.com / pub/ m ibs/ .

367

Re pla ce m e nt Syslog D a e m ons Som e I SPs ar e not sat isfied w it h t he st andar d sy slog funct ion bundled w it h m ost UNI X sy st em s. Hence, t her e hav e been som e effor t s t o pr ov ide an enhanced capabilit y . Tw o ex am ples ar e giv en her e. sy slog- ng syslog- ng is a r eplacem ent for t he st andar d syslog found on m ost UNI X syst em s. I t is support ed on Linux, BSD, AI X, HP- UX, and Solar is. I t gives a m uch enhanced configur at ion schem e t hat enables t he filt er ing of m essages based not only on pr ior it y / facilit y pair s, but also on m essage cont ent . Regular ex pr essions also can be used t o dir ect log st r eam s t o differ ent dest inat ions. sy slog- ng also suppor t s for w ar ding logs on TCP, w it h hash- pr ot ect ed log files planned for a fut ur e r elease. See ht t p: / / w w w .balabit .hu/ en/ pr oduct s/ sy slog- ng/ for m ore inform at ion. M od u la r Sy slog ( m sy slog ) Unt il quit e r ecent ly , secur e sy slog ( ssy slog) w as av ailable for UNI X sy st em s. Designed t o r eplace t he sy slog daem on, ssy slog im plem ent s a cr y pt ogr aphic pr ot ocol called PEO- 1 t hat allow s t he r em ot e audit ing of syst em logs. Audit ing r em ains possible ev en if an int r uder gains super user pr iv ileges in t he sy st em . The pr ot ocol guar ant ees t hat t he infor m at ion logged befor e and dur ing t he int r usion pr ocess cannot be m odified w it hout t he audit or ( on a r em ot e, t r ust ed host ) not icing. How ev er , t he pr oj ect now has been super ceded by Modular Sy slog ( m sy slog) , w hich uses m any of t he feat ur es of ssyslog—m syslog can be obt ained from w w w . cor est . com.

Ov e r a ll I n t e r n e t St a t u s a n d Pe r for m a n ce Tools Many people ar e ask ing j ust how w ell t he I nt er net is per for m ing. Ev er since t he NSFnet w as decom m issioned, t her e has been no one place t o under st and t he per for m ance and t r affic pr ofiles on t he I nt er net . Yet people t r y ing t o figur e out how t o do t his. The follow ing list s ar e sit es, pr oj ect s, and soft w ar e at t em pt ing t o get a t r ue big pict ur e of w hat is happening on t he I nt er net . I SPs can elect t o j oin one or m or e of t hese pr ogr am s t o add m or e dat a t o t hese pr oj ect s.

N et St at This t ool pings v ar ious par t s of t he I nt er net fr om v ar ious locat ions on t he I nt er net , collect s t he dat a, and pr ov ides an av er age r esponse t im e on t he m aj or U.S. bac kbone. I t is based on ping. For m ore inform at ion, visit ht t p: / / w w w .net st at sys.com / .

W h a t Ot h e r I SPs Ar e D oin g Her e ar e j ust a few ex am ples of w hat I SPs fr om all ov er t he I nt er net ar e using t o m anage t heir net work. Randy Bush ( r an dy @psg. com) asked m aj or I SPs in t he Unit ed St at es on t he NANOG m ailing list w hat t hey used for t r affic analy sis. His sum m ar y

368

follow s. Not ice especially t he num ber of UNI X scr ipt - based t ools. Readers can find m ore inform at ion by looking at t he NANOG m ailing list archives at w w w .nanog.org.

We do SNMP polling every 15 m inut es at SESQUI NET on every line over which we have adm inist rat ive cont rol and over every peering point . We produce a daily report on errors and usage. We are get t ing ready t o swit ch t o Vult ure or Net Scarf ( or som e com bo) t o give us m ore int eract ive inform at ion. We perform m easurem ent of cert ain basic net work param et ers, such as usage ( bandwidt h used/ t ot al bandwidt h) and line error rat es on all of our noncust om er links. We perform CPU usage, m em ory usage, and environm ent al m onit oring of all our rout ers. We also perform t he line usage and error rat e on all cust om er lines. We m onit or all of our cust om ers’ rout ers unless t hey say ot herwise and not ify t hem of any problem s. Finally, we m onit or select point s t hroughout t he I nt ernet ( root nam e servers and so on) on a four- t im es- an- hour basis using pings. We accom plish t his m onit oring using t he following it em s: an inhouse built package t hat uses SNMP, t racerout e, and ping t o provide graphs and t abular st at ist ical inform at ion. We use Cablet ron’s Spect rum for a quick net work overview. We do SNMP MI B- I I st uff, plus t he cflow d st uff and som et hing we call m xd ( which m easures round- t rip t im es, packet loss and pot ent ial reason, and so on, from a whole bunch of different point s in our net work t o a bunch of ot her point s in our net work. We use it t o creat e delay m at rices, packet loss report s, and ot her report s. There are som e ot her t hings, but t hese are t he biggies. The m xd t hing was originally j ust sort of a t oy for neat report s, but in t he last year, it has becom e a crit ical t ool for m easuring delay variance for one of our VPDN cust om ers t hat does real t im e video st uff ( and is, t o som e ext ent , helping us figure out where we have delay j it t er and why; however, it ’s also raising m ore quest ions) . Because m ost of m y professional career has been in t he ent erprise world, I can offer you what we used t o measure availabilit y t o our m ail servers, Web servers, DNS servers, and so on at one of m y previous em ployers. We em ployed several applicat ion t est s, along wit h net work perform ance t est s. Our prim ary link was via UUnet , a burst able T1. We purchased an I SDN account from anot her local provider

369

who wasn’t direct ly connect ed t o UUnet . Probably a good ex am ple of a j oe- average- user out t here. Every five m inut es, we m easured round- t rip response t im es t o each of t he servers and gat eway rout er ( via ping) and record ed it . We also had applicat ion t est s, such as DNS lookups on our servers, t im ing sendm ail t est m ails t o a / dev/ null account and t im e t o ret rieve t he whole hom e page. This wasn’t m eant t o be a really great perform ance- m onit oring syst em ; it was act ually m eant t o 1. check how our availabilit y looked from a “ j oe user” perspect ive on t he Net ( grant ed, reachabilit y/ availabilit y wasn’t perfect because it was only one point in t he net ) , and 2. look at response t im e t rends/ applicat ion t rends t o see if our hardware/ soft ware was cut t ing it . We use a t raffic flow m onit oring syst em from Kaspia Syst em s ( w w w .k aspia.com) . The Kaspia product collect s all sort s of dat a from rout er port s and RMON probes, st ores t he dat a, and perform s various t rend analysis. We collect t raffic flow, rout er CPU usage, and rout er m em ory inform at ion plus various errors. Ther e is a dat a- reduct ion process t hat runs once a day and a very nift y Web int erface. The product is not cheap, but t he syst em definit ely fills a void here. Maybe I should organize a t alk on what we are doing wit h it for an upcom ing NANOG. As an old inst rum ent at ion engineer, I t hink t he basis of our use of t he t ool is pret t y solid. Plus, I act ually developed a m eans for calibrat ion of t he accuracy of t he flow dat a. I have not had t im e yet t o work out a validat ion for t he t rends, but I ’ll get t o it one of t hese decades. Also, t he Kaspia people will give you a 30- day t rial on t heir product at no charge. For nonint rusive st uff, we ke ep a log of all int erface st at us changes on our rout ers, and we pull five - m inut e byt e- count s inbound and out bound on each int erface, which we graph against port speed. Wat ching t he graphs for any sort of clipping of peaks gives a pret t y good indicat ion of problem s, and wat ching for shift s of t raffic bet ween port s on parallel pat hs does likewise. As for int rusive t est ing, we do a t hree- pack et m in-lengt h ping t o t he LAN- side port of each of our cust om ers’ rout ers once

370

each five m inut es, and we follow t hat up wit h addit ional at t em pt s if t hose t hree are lost . We log lat ency, and if we have t o follow up wit h a burst , we log loss rat e from t he burst . Pinging t hrough t o t he LAN port obviously let s us know when CPE rout ers konk out ; occasionally we see hung rout ers t hat st ill have operat ional WAN port s t alking t o us. Likewise, sim ply w at ching VC-st at e isn’t a reliable enough indicat or of t he st at us of t he rem ot e rout er. Plus, it t ells you if t he cust om er has kicked t he Et hernet t ransceiver off t heir equipm ent , for inst ance. I t wouldn’t m at t er t o you probably, but our dem arc is all t he way out at t he WAN port because we own and operat e our cust om ers’ CPE. I t hink a bit about what m ore we could be doing; flows analysis and what not . . .. I t ’s nice t o t hink about , and eve nt ually we’ll get around t o it , but program m er t im e is relat ively precious and ot her t hings have higher priorit y because t he current syst em works and t ends t o t ell us m ost of what we seem t o need t o know t o provide decent service. We place quit e a bit of em phasis on net work st at s. Current ly we have about t hree years of st at s online, and we are working on convert ing our in- house engine t o an rdbm s so we can m ore easily perform t rend analysis. Besides Kaspia, ot her com m ercial packages include t rendsnm p (www.deskt alk.com) and concord’s packages (www.concord.com) . Our in- house st uff is locat ed at ht t p: / / net op.cc.buffalo.edu/ , if you are curious about w hat w e do. We are Neandert hals right now—we use a hacked rcisco t o dat a t o nocol. We wat ch bandwidt h ( separat ely as well) on links—and also wat ch input errors and int erface t ransit ions nocol) —all done wit h Perl and expect -like rout ines, parsing int ’s” every few m inut es.

feed key ( for “ sho

Em ergency st uff goes t hrough nocol; bandwidt h sum m aries are m ailed t o int erest ed part ies overnight . We have running here now t he MRTG package t hat generat es som e fancy graphics, but , in m y opinion, t hese graphics are useless, and looking in det ail t o som e of t he report s, t hey are not accurat e. Several of our client s request t he raw dat a, but t his package only m aint ains raw dat a j ust t o generat e t he graphs. I n t he past we used t o have a kind of ASCI I report ( Vikas wrot e

371

som e of t he script s and program s) generat ed from inform at ion obt ained using t he old SNMP t ool set developed by nysernet , but I guess t hat nobody m aint ained t he config files and I believe t hat t he SNMP libra ry rout ines used aren’t working. We have been using t he MRTG package, which is basically a special SNMP agent t hat queries t he rout ers for st at s and t hen does som e nice graphing of t he dat a on t he Web. SNMP queries wit h a heavily m odified version of MRTG from t he nice guy in Germ any. I t works very nicely. We have recent ly inst alled Net Scarf 2.0, and are cont em plat ing m erging Net Scarf 3.0 wit h t he MRTG front end. I ’m researching whet her I can rewrit e St eve Corbat o’s fast poll program using t he fast snm p library from t he Net Scarf people. I t hink t his will allow fast poll t o scale bet t er. I ’ve successfully writ t en a quick C program t hat uses t he library t o collect t he required dat a for a rout er—now I ’ve j ust got t o m ake it so t hat we can m anage it easily ( in ot her words, aut ogenerat ed config files from our dat abases) . My goal is t o be able t o collect 1- t o 2- m inut e period dat a on all links t hat are great er t han 10 Mbps— 15 m inut es of dat a for everyt hing else. The t wo- m inut e collect ion period will allow t he bandwidt h t o scale up t o 280 Mbps before experiencing t wo count er rollovers wit hin a polling int erval. Hopefully t hat will hold us over unt il t he int erface count ers are available as Count er64 obj ect s wit h SNMPv2 ( if t hat ever happens) . What fast poll collect s now is ifI nOct et s, ifOut Oct et s, ifI nUcast Pkt s, ifOut Ucast Pkt s, ifI nErrors, and ifOut Discards. Rat her t han st oring t he raw count ers, it calculat es t he rat e by dividing t he delt a by t he period. Get t ing t he accurat e period is act ually t he hard part —I am having SNMP send m e t he upt im e of t he rout er in each query and using t hat t o calculat e t he int erval bet ween polls and t o det ect count er reset s due t o reboot s. The ot her t rick t o handle is t he fact t hat , alt hough I OS Soft ware updat es t he SNMP count ers for process- swit ched packet s as t hey are rout ed, it looks like t he count er for SSE swit ched packet s on C70X0 rout ers get updat ed only once every 10 seconds.

372

Su m m a r y As you can see, t here are m any w ays of m onit oring inform at ion from t he I SP net w or k. That m ost I SPs use t heir ow n cust om scr ipt s per haps show s t he lack of com m er cial soft w ar e av ailable in t his ar ea and t hat each I SP oper at ion has it s ow n unique condit ions and w ay s of going about t hings. One day a for w ar d- look ing or ganizat ion w ill pr oduce a com bined soft war e t ool w it h t he usabilit y of all t hese feat ur es and y et is light w eight enough t o oper at e scaleably and flex ibly in an I SP’s back bone net w or k . Ev en t hose I SPs t hat hav e im plem ent ed com m er cial packages seem t o r equir e a fair am ount of Per l scr ipt ing t o pr o vide cover age or suppor t for feat ur es or ser vices in t heir net w or k. This appendix nat ur ally list s only a snapshot of w hat is av ailable. You ar e encour aged t o check updat es and ask on t he v ar ious oper at or for um s w hat t he cur r ent t r ends and pr act ices ar e.

373

Appendix F. Ex am ple I SP Access Securit y M igrat ion Pla n This appendix giv es one ex am ple of how an I SP could m igr at e it s net w or k equipm ent ( r out er s, sw it ches, and NAS) fr om a st at e in w hich Telnet access is open t o t he out side w or ld t o t he point at w hich only specific aut hor ized w or k st at ions ar e allow ed access t o t he Telnet pr om pt . Unfor t unat ely , at t he t im e t his t ex t w as w r it t en, m ost I SPs w er e not t ak ing t hese sim ple pr ecaut ions t o help secur e t heir net w or k s. This sect ion is designed t o help t hose I SPs put in t he m inim um necessar y pr ecaut ions. This is a sim ple pr ocedur e t hat dr aw s a secur it y cir cle ar ound an I SP’s net w or k and t hen slow ly nar r ow s t he cir cle unt il j ust t he aut hor ized I P addr esses ar e included in t he VTY’s ACL. Use t his appendix in conj unct ion w it h t he secur it y t em plat e r ecom m ended in Appendix B, “ Cut - and- Past e Tem plat es.” I SPs st ar t ing w it h a new net w or k deploy m ent should be using t he t em plat es discussed ear lier—t o fix a problem highlight ed by r eading t his book , follow t he t hr ee- phase plan list ed in t his appendix.

Ph a se 1 —Close Off Acce ss t o Ev e r y on e Ou t side t h e CI D R Block The t hem e in Phase 1 is t o get t he ball r olling by lim it ing access t o j ust t hose I P addr esses inside t he I SP’s CI DR block . A st andar d ACL is cr eat ed t o per m it Telnet only fr om t he I P addr esses in t he CI DR block . This ACL is used w it h t he VTY’s a cce ss- cla ss com m and t o ensur e t hat t he sour ce I P addr ess of any Telnet pack e t com ing t o t he VTY por t m at ches t he ACL. Why j ust t he I SP’s CI DR block ? Fir st , it ’s an easy- t o- im plem ent t echnique for an I SP t hat has no filt er s. The I SP does not hav e t o w or r y about lock ing out st aff m em ber s w ho need access t o t he r out er fr om differ ent par t s of t he net w or k . Hence, it can be done w it h t he least w or r y t hat it w ill affect t he I SP’s oper at ions. Second, it lim it s t he t hr eat fr om t he ent ir e I nt er net t o j ust I P addr esses inside t he I SP’s CI DR block . Minim izing r isk is one of t he fundam ent al r equir em ent s of secur it y . Finally , because t he I SP is beginning w it h no pr ot ect ion, t his incr em ent al st ep ensur es t hat ev er y t hing is w or k ing befor e deepening t he secur it y configur at ions on t he net w or k . Figur e F- 1 is an exam ple of an I SP’s net w or k. The allocat ed CI DR blocks ar e 169.223.0.0/ 16 and 211.255.0.0/ 19 [ 1] . Ph ase 1 in v olv es creat ing an ACL t hat can be used w it h t he VTY’s a cce ss- cla ss com m and t o r est r ict Telnet t o I P addr esses w it h t he CI DR block. Figu r e F - 1 . I SP N e t w or k Ex a m ple

374

The configur at ion in Ex am ple F- 1 is used on all rout ers in t he net w ork. Not ice t hat t he AAA aut hent icat ion used is sim plist ic —it nor m ally w ould not be r ecom m en ded for an I SP back bone. Sim ilar configur at ions should be used on t he sw it ches in t he net w or k . Any st aff w or k st at ions/ ser v er s also should use appr opr iat e t ools t o lim it Telnet access t o t he w or k st at ions/ ser v er ’s r esour ces [ 2] . Ex a m ple F - 1 Sim p le V TY Con f ig u r a t ion t o Lim it Acce ss t o Ju st t h e I SP’s CI D R Blo ck aaa new-model aaa authentication login ISP local ! username Cisco1 password 7 11041811051B13 ! access-list 3 permit 211.255.0.0 0.0.31.255 access-list 3 permit 169.223.0.0 0.0.255.255 access-list 3 deny any ! line vty 0 4 access-class 3 in exec-timeout 5 0 transport preferred none transport input telnet login authentication ISP history size 256

Ph a se 2 —Add An t ispoofin g Filt e r s t o You r Pe e r s Lim it ing t he secur it y r isk t hr ough r est r ict ed Telnet access is only t he fir st st ep. This j ust st ops t he r est of t he w or ld fr om logging int o t he I SP’s r out er s —t hey st ill ar e

375

open t o your cust om er s and any ot her locat ion t hat is using addr ess space ( legit im at ely or ot herw ise) from I SP’s CI DR block. The nex t st ep is t o ensur e t hat par t ies out side t he I SP’s net w or k cannot spoof sour ce addr esses fr om t he I SP’s CI DR block . Sev er al for m s of sour ce addr ess spoofing and Telnet sequence num ber hij ack ing at t ack s can penet r at e t he VTY’s access class pr ot ect ions. To m inim ize t he r isk of t hese sor t s of at t ack s, an I SP can place ant ispoofing filt er s at t he edges of it s net w or k. As highlight ed in Chapt er 4, “ Secur it y ,” ant ispoofing filt er s ar e used t o ensur e t hat any addr ess com ing fr om t he I nt er net int o t he I SP’s net w or k does not cont ain a sour ce addr ess fr om t he I SP’s net w or k. For exam ple, if a pack et fr om t he I nt er net w it h a sour ce addr ess of 211.255.1.1 com es int o t he I SP in Figur e F- 2, t he ant ispoofing filt er w ould dr op t he pack et . Figu r e F - 2 . Apply in g An t ispoof in g Filt e r s

W ARN I N G Rem em ber , no one fr om t he gener al I nt er net should be sending y ou pack et s w it h a source address from your ow n net w ork!

Few I SPs im plem ent ant ispoofing packet filt er s. The key r eason given is possible per for m ance im pact . Yes, apply ing pack et filt er s t o any r out er m ight cause a per for m ance im pact , but , as has been dem onst r at ed in Chapt er 4, t ools such as Unicast RPF hav e m inim al CPU im pact at m ult im egabit - per- second speeds. An essent ial t enet of secur it y is balancing t he t r ade- offs. Sacr ificing som e per for m ance t o m inim ize t he secur it y r isk t o v aluable net w or k r esour ces [ 3] is a logical t r ade- off,

376

m ade less of a t rade- off t hese day s especially w it h t he new im pr ov em ent s in pack et per second ( PPS) t hat t he m or e recent Cisco I OS Soft w ar e offer s I SPs. [ 4]

W here t o Pla ce t he Ant ispoofing Pa ck et Filt ers Place ant ispoofing filt er s at t he edge of an I SP’s net w or k. This usually m eans t h e r out er ( s) t hat int er connect s w it h ot her I SPs. Rout er s Gat ew ay1 and Gat ew ay2 ar e exam ples in Figur e F- 2. Any at t em pt t o spoof fr om t he cor e of t he I nt er net ( in ot her w or ds, t he upst r eam I SP) or an I SP peer connect ion w ould be dr opped on t he inbound int er face. Exam ple F- 2 highlight s a t y pical ant ispoofing filt er . Not ice t hat t w o CI DR blocks are used in t he exam ple —169.223.0.0/ 16 and 211.255.0.0/ 19—t w o lines of sev en in t he ACL 111. Here is w hat t he ot her lines do: •

•

• • • • •

deny ip 127.0.0.0 0.255.255.255 —This is t he loopback addr ess for TCP w or k st at ions, PCs, and ser v er s. I t should not be t r ansm it t ed ov er t he I nt er net . I f it is, t hen it is eit her a br oken TCP st ack or an indicat ion t hat som eone is t r y ing t o br eak int o a r esour ce. deny ip 10.0.0.0 0.255.255.255— RFC 1918 pr iv at e addr ess space. Cisco adv ocat es RFC 1918 pr iv at e addr ess space use in ent erprise net works in conj unct ion w it h Net w or k Addr ess Tr anslat ion ( NAT) . Any pack et s w it h RFC 1918 addr esses in t heir sour ce eit her ar e fr om a br ok en NAT im plem ent at ion or ar e par t of a spoofing at t ack. deny ip 172.16.0.0 0.15.255.255 — RFC 1918 pr iv at e addr ess space. deny ip 192.168.0.0 0.0.255.255 — RFC 1918 pr iv at e addr ess space. deny ip 169.223.0.0 0.0.255.255 — One of t he exam ple I SP’s CI DR blocks. deny ip 211.255.0.0 0.0.31.255— The ot her CI DR block. perm it ip any any — Per m it nor m al pack et s.

Each line w it h a deny has a log opt ion t ur ned on. This includes any m at ches in t he int er nal log buffer and out put t o sy slog ( if t he r out er has it configur ed) . The log opt ion is not used on t he last perm it . Good packet s do not need t o be logged. Not e t hat enabling logging w ill t ake a m or e significant CPU hit because logged packet s need t o be sent t o t he pr ocess lev el in t he r out er t o be r ecor ded and sent t o t he loghost . Logging gener ally is enabled only w hen t he I SP w ant s t o m onit or pot ent ial or appar ent at t acks happening on it s net work. Ex a m ple F - 2 An t isp oof in g Con f ig u r a t ion Ex a m p le Router Gateway1 ! interface hssi 0/1 description 16Mbps link to our upstream provider bandwidth 16384 ip access-group 111 in no ip redirects no ip directed-broadcast no ip proxy-arp ! access-list 111 deny ip 127.0.0.0 0.255.255.255 any log access-list 111 deny ip 10.0.0.0 0.255.255.255 any log

377

access-list 111 deny ip 172.16.0.0 0.15.255.255 any log access-list 111 deny ip 192.168.0.0 0.0.255.255 any log access-list 111 deny ip 169.223.0.0 0.0.255.255 any log access-list 111 deny ip 211.255.0.0 0.0.31.255 any log access-list 111 permit ip any any Router Gateway2 ! interface serial 0/1 description Compressed 2Mbps bi-lateral peer to our neighboring country bandwidth 2048 ip access-group 111 in no ip redirects no ip directed-broadcast no ip proxy-arp ! access-list 111 deny ip 127.0.0.0 0.255.255.255 any log access-list 111 deny ip 10.0.0.0 0.255.255.255 any log access-list 111 deny ip 172.16.0.0 0.15.255.255 any log access-list 111 deny ip 192.168.0.0 0.0.255.255 any log access-list 111 deny ip 169.223.0.0 0.0.255.255 any log access-list 111 deny ip 211.255.0.0 0.0.31.255 any log access-list 111 permit ip any any

Ph a se Th r e e —Close Of f Ne t w or k Eq u ip m e n t t o Un a u t h or iz e d Acce ss When Phase 1 and Phase 2 ar e com plet ed, w or k can begin on Phase 3—closing off access t o ev er y one ex cept t he NOC st aff. The speed an I SP m ov es fr om t he fir st t w o phases t o t he t hir d phase is dependent on t he confidenc e of t he I SP’s engineer s. Som e m ight w ant t o m onit or t he oper at ional im pact of t he fir st t w o phases befor e incr em ent ing anot her layer of secur it y. Ot her s m ight bypass Phase 1 and im m ediat ely r est r ict access only t o t he NOC st aff. Eit her w ay w or k s. Each I SP needs t o dev elop plans t hat suit it s env ir onm ent . The m ost im por t ant t hing is t o im plem ent t he securit y m easures. Tw o basic st eps ar e necessar y t o close Telnet access t o j ust t he NOC st aff. Fir st , ident ify t he I P addr ess block for t he NOC’s net w or k. Figur e F- 3 shows t he NOC’s net w or k behind a fir ew all using I P block 211.255.1.0/ 24. Second, m odify t he ACLs for t he VTY’s access class w it h t he addr esses assigned t o t he NOC. Exam ple F- 3 highlight s t his m odificat ion. Figu r e F - 3 . Closin g Of f Acce ss t o Ev e r y on e Ex ce p t t h e N OC St a f f

378

Ex a m ple F - 3 ACLs w it h Te ln e t Acce ss Close d t o All b u t t h e N OC’s N e t w or k aaa new-model aaa authentication login ISP local ! username Cisco1 password 7 11041811051B13 ! access-list 3 permit 211.255.1.0 0.0.0.255 access-list 3 deny any ! line vty 0 4 access-class 3 in exec-timeout 5 0 transport preferred none transport input telnet login authentication ISP history size 256

Su m m a r y The t hr ee sim ple phases in t his appendix ar e all t hat ar e r equir ed t o conv er t a w ideopen, insecur e I SP backbone int o one w it h m oder at e secur it y. I t is quit e possible t o w r ap up t he net w or k ev en m or e secur ely , and y ou ar e encour aged t o consult t he m ain body of t he book, especially Chapt er 4, t o find r ecom m endat ions of w hat t o do nex t . The dow nsides of an insecur e back bone hav e been cov er ed alr eady in t he m ain t ex t . But it is w or t h r eit er at ing t hat if som e unaut hor ized per son can get a login pr om pt for an I SP’s net w or k equipm ent , t hat per son has a ver y good chance of br eaking int o

379

it and w r eaking havoc in t hat backbone. Wor se, t he unaut hor ized individual could plant “ t im e bom bs” t hat t he I SP is unaw ar e of unt il t hey t ak e effect . Securit y is a m andat or y r equir em ent for an I SP back bone—if it isn’t t her e at t he m om ent , y ou ar e st r ongly encour aged t o dr op ev er y t hing, ev en pause r eading t his book , and pr ot ect y our net w or k im m ediat ely . I t ’s nev er t oo lat e. As m or e I SPs t ight en up secur it y , t hose t hat pr ev iously r elied on obscur it y for t heir secur it y w ill becom e t ar get s, r esult ing in oper at ional m iser y and angr y or disappoint ed cust om er s. Because cust om er s ar e usually a business’s liv elihood, av oiding cust om er dissat isfact ion is usually a high pr ior it y .

En dn ot e s 1. These net w or k s do not r epr esent a r eal allocat ion. They w er e pulled fr om t he APNI C blocks as an exam ple, not t o por t r ay any r eal, live net w or k. 2. Tools such as TCP Wrapper are w ell know n and have a role in an I SP’s overall secur it y ar chit ect ur e. 3. Spoofing at t ack s ar e m or e lik ely t o t ar get w or k st at ion and ser v er r esour ces. These r esour ces lik ely w ould depend on t ools such as TCP Wr apper—t hese w r apper s also can be by passed by spoofing at t ack s. So placem ent of ant ispoofing filt er s pr ot ect s t he ent ir e net w or k, not j ust t he r out er s. 4. This includes dist r ibut ed sw it ching ( FI B) , Cisco Ex pr ess For w ar ding ( CEF) , Dist r ibut ed Net Flow , and ot her im pr ov em ent s in t he 11.1CC and 12.0S code t rains.

380

Glossa r y ACE access cont r ol ent r y ( single line in an ACL)

ACL access cont r ol list

AFRI N I C Afr ican Regional Net w or k I nfor m at ion Cent er

APN I C Asia Pacific Net w or k I nfor m at ion Cent er

APOPS Asia Pacific Operat ors

APRI COT Asia Pacific Rim I nt er net Confer ence on Oper at ional Technologies

ARI N Am erican Regist ry for I nt ernet Num bers

ARPAN ET Adv anced Resear ch Pr oj ect s Agency Net w or k

381

AS aut onom ous sy st em

ASI C applicat ion- specific int egr at ed cir cuit

ATM Asy nchr onous Tr ansfer Mode

BCP best cur r ent pr act ice

BGP Bor der Gat ew ay Pr ot oc ol

BOOTP Boot Pr ot ocol

CAR Com m it t ed Access Rat e

CEF Cisco Expr ess For w ar ding

CI D R

382

classless int er dom ain r out ing

CLI com m and- line int er face

CPU cent ral processing unit

d CEF Dist r ibut ed Cisco Ex pr ess For w ar ding

D H CP Dynam ic Host Configur at io n Pr ot ocol

D LL dynam ic link library

DNS Dom ain Nam e Sy st em

D oS denial of ser v ice

EGP ex t er ior gat ew ay pr ot ocol

383

EOF Eur opean Oper at or s For um

FI B Forw arding I nform at ion Base

FTP File Tr ansfer Pr ot ocol

GPS global posit ioning sy st em

H SRP Hot St andby Rout ing Pr ot ocol

I AN A I nt er net Assigned Num ber s Aut hor it y

I EPG I nt er net Engineer ing and Planning Gr oup

I ETF I nt er net Engineer ing Task For ce

I GP

384

int er ior gat ew ay pr ot ocol

I N ET I nt er net Societ y ’s annual confer ence

I nterNI C I nt er net Net w or k I nfor m at ion Cent er

I OS I nt er net Oper at ing Sy st em ( Cisco I OS)

I RR I nt ernet Rout ing Regist ry

I S- I S I nt er m ediat e Sy st em- t o- I nt er m ediat e Sy st em

I SP I nt er net ser v ice pr ov ider

LACN I C Lat in Am er ican and Car ibbean Net w or k I nfor m at ion Cent er

LI R Local I nt ernet Regist ry

385

M ED m ult iex it discr im inat or

MI B Managem ent I nfor m at ion Base

MI T m aint enance- induced t rouble

M PLS Mult ipr ot ocol Label Sw it ching

N AN OG Nort h Am erican Net w ork Operat ors Group

N AS net w or k access ser v er

N OC net w or k oper at ions cen t er

N SAP net w or k ser v ice access point

N SP

386

net w or k ser v ices pr ov ider

N TP Net w or k Tim e Pr ot ocol

ORF out bound r out e filt er

OSPF Open Shor t est Pat h Fir st

PGP pr et t y good pr iv acy

PoP point of pr esence

PoP3 Post Office Pr ot ocol v er sion 3

PPP Point - t o- Point Pr ot ocol

PPS pack et s per second

387

QoS qualit y of ser v ice

RAD B Rout ing Ar bit er Dat abase

RAM r an d om- access m em or y

RFC Request For Com m ent s ( I ETF docum ent )

RI B Rout ing I nform at ion Base

RI P Rout ing I nfor m at ion Pr ot ocol

RI PE Réseaux I P Eur opéens

RI PE N CC RI PE Net w or k Coor dinat ion Cent er

RI R

388

Regional I nt ernet Regist ry

RR r out e r eflect or

RPC r em ot e- procedure call

RPF r ever se pat h for w ar ding

SM TP Sim ple Mail Tr ansfer Pr ot ocol

SN M P Sim ple Net w or k Managem ent Pr ot ocol

SPD select iv e pack et discar d

STD I nt er net st andar d —st at us of I ETF docum ent

TAC Technical Assist ance Cent er

389

TCAM t ernary cont ent addressable m em ory

TCP Tr ansm ission Cont r ol Pr ot ocol

TFTP Tr iv ial File Tr ansfer Pr ot ocol

TLD t op- level dom ain

u RPF u n icast r ev er se pat h for w ar ding

VI P Ver sat ile I nt er face Pr ocessor ( found on 7500 fam ily)

V PN vir t ual pr ivat e net w or k

WWW Wor ld Wide Web

390

Technica l References a nd Recom m ended Rea ding This appendix list s t he t echnical r efer ences and fur t her r eading per t inent t o t his book . You ar e encour aged t o r ev iew t he r efer ences quot ed. The r efer ences fr om each sect ion of Cisco I SP Essent ials have been list ed separ at ely.

Soft w a r e a n d Rou t e r M a n a ge m e n t Cisco pr odu ct bu lle t in s ht t p: / / w w w .cisco.com / w ar p/ public/ cc/ gener al/ bullet in/ index .sht m l Cisco I OS Sof t w a r e Re le a se 1 2 .0 S n e w f e a t u r e s ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ iom j r e12/ pr odlit / 934_pb. ht m Cisco I OS Sof t w a r e Re le a se 1 2 .0 S or d e r in g p r oce d u r e s a n d p la t f or m h a r dw a r e su ppor t ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ iom j r e12/ pr odlit / 935_pb. ht m Cisco I OS Sof t w a r e r e le a se n ot e s f or Re le a se 1 2 .0 S ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ r elnot e/ 7000fam / r n 1 2 0 s. h t m Cisco I OS Sof t w a r e Re le a se 1 2 .0 S m ig r a t ion g u id e h t t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ iom j r e12/ pr odlit / 940_pb. ht m Cisco I OS Re le a se s ht t p: / / w w w .cisco.com / w ar p/ public/ 732/ Releases/ Ty p e s of Cisco I OS Sof t w a r e r e le a se s ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ pr odlit / 537_pp. ht m Cisco I OS Sof t w a r e r e le a se d e sig n a t ion s d e f in e d —“ So f t w a r e Lif e cy cle D e fin it ion s” ht t p: / / w w w .cisco.com / w ar p/ public/ 417/ 109.ht m l Sof t w a r e n a m in g con v e n t ion s f or Cisco I OS ht t p: / / w w w . cisco. com / w ar p/ cust om er / 432/ 7.ht m l Cisco I OS r e fe r e n ce gu ide

391

ht t p: / / w w w .cisco.com / w ar p/ public/ 620/ 1.ht m l Cisco I OS r oa d m a p ht t p: / / w w w .cisco.com / w ar p/ public/ 620/ r oadm ap.sht m l Cisco Re sou r ce M a n a g e r ht t p: / / w w w .cisco.com / w ar p/ public/ cc/ pd/ w r 2k / r sm n/ index .sht m l Pr iv a t e I ht t p: / / w w w .opensy st em s.com / index .asp Cr y st a l Re por t s ht t p: / / w w w .seagat esoft w ar e.com / cr y st alr epor t s/ N e t for e n sics ht t p: / / w w w .net forensics.com / N TP RFCs RFC 1128, RFC 1129, RFC 1165, and RFC 1305, all available at ht t p: / / w w w .iet f.or g/ r fc Cisco I OS Soft w a r e N TP a r ch it e ct u r e ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / iaabu/ cddm / cddm 111/ adguide/ nt p.ht m N e t w or k Tim e Pr ot ocol ( N TP) m a st e r clock f or t h e U n it e d St a t e s ht t p: / / t ycho.usno.navy.m il/ D a t u m I n c., Ba n com m Tim in g D iv ision ht t p: / / w w w .dat um .com / Tr u e Tim e , I n c. ht t p: / / w w w .t r uet im e. com Th e Tim e W e b Se r v e r ( Tim e Sy n c) , by D a v e M ills ht t p: / / w w w .eecis.udel.edu/ ~ nt p/ Coe t a n ia n Sy st e m s Tim e Sy n ch r on iz a t ion Se r v e r 1 0 0

392

ht t p: / / w w w . coet anian. com / t ss/ t ss1 0 0 . ht m U CD- SN M P p r oj e ct h om e p a g e ht t p: / / net - snm p. sour cefor ge. net / Cr e a t in g cor e d u m p s in Cisco I OS Sof t w a r e ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / it g_v 1/ t r 19aa. ht m

Ge n e r a l Fe a t u r e s Cisco I OS com m a n d - lin e in t e r f a ce ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 12cgcr / fun_r / ind ex . h t m CEF w h it e p a p e r ht t p: / / w w w . cisco. com / w ar p/ public/ cc/ pd/ iosw / ior e/ t ech/ cef_w p. ht m Cisco Ex p r e ss For w a r d in g ov e r v ie w ( 1 2 .0 d ocs) ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 12cgcr / sw it ch_c/ x cpr t 2 / x ccef. h t m Cisco Ex p r e ss For w a r d in g con f ig u r a t ion t a sk list ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 12cgcr / sw it ch_c/ x cpr t 2 / x ccefc. h t m Ph il H a r r is’s Pa ck e t M a ga z in e a r t icle on CEF ( m or e of a CEF FAQ) ht t p: / / w w w . cisco. com / w ar p/ public/ 784/ packet / oct 99/ r out er .ht m l

Se cu r it y I n cr e a sin g se cu r it y on I P n e t w or k s. An old but v er y im por t ant docum ent on som e of t he securit y essent ials in I P- based net w orks. ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / ics/ cs003. ht m Cisco’s I n t e r n e t se cu r it y a d v isor ie s. An online list of all of Cisco’s secur it y adv isor ies. I t includes t ut or ials and det ails on how t o pr ot ect y our self fr om som e of t he ugliness on t he I nt er net t oday . ht t p: / / w w w .cisco.com / w ar p/ cust om er / 707/ adv isor y .ht m l

393

Cisco’s I OS Sof t w a r e d ocu m e n t a t ion—“ 1 2 . 2 Se cu r it y Con f ig u r a t ion Gu id e . ” The docum ent at ion w it h I OS Soft w ar e Release 12.2 r elease r eor ganizes m any of t he secur it y feat ur es of I OS int o t heir ow n chapt er . Av ailable on t he Cisco Docum ent at ion CD or publicly online at Cisco. com: ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios122/ 122cgcr / fsecur _c /f Cisco I OS p a ssw or d- e n cr y p t ion t ip s h t t p: / / w w w .cisco.com / w ar p/ public/ 701/ 64.ht m l D e f in in g st r a t e g ie s t o p r ot e ct a g a in st U D P d ia g n ost ic p or t d e n ia l- of- se r v ice a t t a ck s ht t p: / / w w w .cisco.com / w ar p/ public/ 707/ 3.ht m l Cisco pu blic d om a in TACACS+ se r v e r f or U N I X ft p: / / ft p- eng.cisco.com / pub/ t acacs/ t ac_plus.F4.0.4.alpha.t ar .Z N e ig h b or in g r ou t e r a u t h e n t ica t ion : ov e r v ie w a n d g u id e lin e s ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios112/ 122cgcr / fsecur _c / fot her s/ scfr out r . ht m Cisco I OS Tu r b oACL d ocu m e n t a t io n ht t p: / / w w w .cisco.com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios120/ 120new ft / 120lim i t / 1 2 0 s/ 1 2 0 s6 / t u r boacl. h t m Com m it t e d Acce ss Ra t e ( CAR) ht t p: / / w w w .cisco.com / w ar p/ public/ 732/ Tech/ car / index .ht m l Con f ig u r in g Com m it t e d Acce ss Ra t e ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ pr oduct / soft w ar e/ ios122/ 122cgcr / fqos_c/ f qcpr t 1 / qcfcar . ht m RFC 1 8 1 2 . “ Re q u ir e m e n t s f or I P V e r sion 4 Rou t e r s.” F. Baker ( ed) . June 1995. ( St at us: Pr oposed st andar d.) Also see t he updat e, RFC 2644. ht t p: / / w w w .iet f.or g/ r fc/ r fc1812.t xt RFC 2 1 9 6 / FYI 8 . “ Sit e Se cu r it y H a n d b ook .” B. Fr aser . Sept em ber 1997. ( Obsolet es RFC 1244) ( St at us: I nfor m at ional.) One of t he m ost useful st ar t ing places for I nt er net secur it y . ht t p: / / w w w .iet f.or g/ r fc/ r fc2196.t xt

394

RFC 2 8 2 7 / BCP 3 8 . “ N e t w or k I n g r e ss Filt e r in g : D e f e a t in g D e n ia l- of- Se r v ice At t a ck s W h ich Em p loy I P Sou r ce Ad d r e ss Sp oof in g . ” P. Ferguson and D. Senie. May 2000. ( St at us: Best cur r ent pr act ice.) ht t p: / / w w w .iet f.or g/ r fc/ r fc2827.t xt RFC 2 3 5 0 / BCP 2 1 . “ Ex p e ct a t ion s f or Com p u t e r Se cu r it y I n cid e n t Re spon se .” N. Br ow nlee and E. Gut t m an, June 1998. ( St at us: Best cur r ent pr act ice. ) ht t p: / / w w w .iet f.or g/ r fc/ r fc2350.t xt RFC 2 6 4 4 / BCP 3 4 . “ Ch a n g in g t h e D e f a u lt f or D ir e ct e d Br oa d ca st s on Rou t e r s.” D. Senie. August 1999. ( Supplem ent t o RFC 1812.) ( St at us: Best cur r ent pr act ice. ) ht t p: / / w w w .iet f.or g/ r fc/ r fc2644.t xt RFC 3 0 1 3 / BCP 4 6 . “ Se cu r it y Ex p e ct a t ion s f or I n t e r n e t Se r v ice Pr o v id e r s. ” T. Killalea. Nov em ber 2000. ( St at us: Best cur r ent pr act ice.) ht t p: / / w w w .iet f.or g/ r fc/ r fc3013.t xt “ Se cu r it y Ch e ck list f or I n t e r n e t Se r v ice Pr ov id e r ( I SP) Con su m e r s.” d r a f t ie t f- gr ip- us e r- 0 2 .t x t . ( Now ex pir ed.) T. Hansen. June 1999. Not an I ETF w eb sit e any m ore, but you can st ill find it by searching various online ar chiv es. “ Sit e Se cu r it y H a n d b ook Ad d e n d u m f or I SPs.” d r a f t - ie t f- g r i p- ssh- a dd0 0 .t x t . ( Now ex pir ed.) T. Debeaupuis, August 1999. Not an I ETF w eb sit e any m ore, but you can st ill find it by searching various online ar chiv es. Cr a ig H u e g e n ’s Sm u r f Pa g e . A v er y useful r esour ce for I SPs t o lear n how t o pr ot ect t hem selv es fr om t he m any flav or s of denial- of- ser v ice at t ack s. ht t p: / / w w w . pent ics. net / D e n ia l- o f- Se r v ice At t a ck s I n f or m a t ion Pa g e s. By Paul Ferguson and Daniel Senie. Anot her v er y useful r esour ce for I SPs t o lear n how t o pr ot ect t hem selv es fr om t he m any flav or s of denial- of- ser v ice at t ack s. ht t p: / / w w w .denialinfo.com / Ja r e d M a u ch ’s Sm u r f Sw e e p Re su lt s Pa g e . ( j a r e d @p u ck .n e t h e r .n e t ) Jar ed has scanned lar ge sect ions of t he I nt er net looking for net w or ks t hat could be used as sm ur f am plifier s. This page list s his r esult s and pr ov ides a w ay t o check I P pr efix es and AS num ber s.

395

ht t p: / / puck .net her .net / sm ur f - check/

Rou t in g I n t e r n e t Rou t in g Ar ch it e ct u r e s, Se con d Ed it ion . Cisco Pr ess. ( ht t p: / / w w w . ciscopr ess. com) I SBN 1- 5 7 8 7 0- 0233- X. Aut hors: Sam Halabi w it h Danny McPher son. RFC 2 2 8 1 . “Cisco H ot St a n dby Rou t in g Pr ot ocol. ” Mar ch 1998 ( St at us: I nfor m at ional.) ht t p: / / w w w .iet f.or g/ r fc/ r fc2281.t xt “ U sin g t h e Bor d e r Ga t e w a y Pr ot ocol f or I n t e r d om a in Rou t in g .” Available on t he Cisco Docum ent at ion CD or publicly onl ine at Cisco. com: ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / ics/ icsbgp4. ht m “ I n t e r n e t w or k in g Te ch n olog y Ov e r v ie w .” Online w hit epaper s and t ut or ials on t he essent ials of r out ing and sw it ching. Av ailable on t he Cisco Docum ent at ion CD or publicly online at Cisco. com: ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / it o_doc/ index . ht m Te ch n olog y in f or m a t ion a n d w h it e p a p e r s. Key r efer ences and pr act ical int er net w or k ing ex am ples. Av ailable on t he Cisco Docum ent at ion CD or pub licly online at Cisco. com: ht t p: / / w w w . cisco. com / univ er cd/ cc/ t d/ doc/ cisint w k / index . ht m

Ot h e r Re fe r e n ce s a n d Re com m e n de d R e a din g Cisco I SP Esse n t ia ls o n lin e v e r sio n ht t p: / / w w w .cisco.com / public/ cons/ isp/ essent ials Cisco I SP Essent ials suppor t ing w eb sit e: ht t p: / / w w w . ispbook . com. Com put er Net w or ks , Thir d Edit ion , by Andr ew Tannenbaum ( I SBN: 0- 13349- 9 4 56) . I nt er connect ions: Br idges and Rout er s , Second Edit ion , by Radia Perlm an ( I SBN: 020163- 4 4 8- 1) . I nt er net w or king w it h TCP/ I P , Volum e 1: Pr inciples, Pr ot ocols, and Ar chit ect ur e, by Douglas Com er ( I SBN: 0- 1 3 2 1 6- 987- 8) . I P Rout ing Fundam ent als , by Mar k Spor t ack ( I SBN: 1- 57870- 0 7 1- x) .

396

I P Rout ing Prim er, by Rober t Wr ight ( I SBN: 1- 57870- 1 0 8- 2) . Rout ing in t he I nt ernet , by Chr ist ian Huit em a ( I SBN: 0- 13132- 19 2- 7) . OSPF Net work Design Solut ions, by Thom as M. Thom as ( I SBN: 1- 5 7 8 7 0- 046- 9) . I SP Survival Guide: St rat egies for Running a Com pet it ive I SP, by Geoff Hust on ( I SBN: 0- 4 7 1 3 1- 499- 4) . “ Docum ent ing Special Use I Pv4 Address Blocks That Have Been Regist ered w it h I ANA,” I nt er net dr aft by Bill Manning. ht t p: / / w w w .iet f.or g/ int er net - dr aft s/ dr aft- m anning- dsua- 07.t xt

397