data center fundamentals

Cisco Press800 East 96th StreetIndianapolis, IN 46240 USA

Cisco Press

Data Center Fundamentals

Mauricio Arregoces, CCIE No. 3285Maurizio Portolani

DataCenter.book Page i Wednesday, November 12, 2003 9:52 AM

ii

Data Center Fundamentals

Mauricio ArregocesMaurizio PortolaniCopyright 2004 Cisco Systems, Inc.Published by:Cisco Press800 East 96th StreetIndianapolis, IN 46240 USAAll rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without written permission from the publisher, except for the inclusion of brief quotations in a review.ISBN: 1-58705-023-4Library of Congress Cataloging-in-Publication Number: 2001086631Printed in the United States of America 1 2 3 4 5 6 7 8 9 0First Printing December 2003

Trademark Acknowledgments

All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Cisco Press or Cisco Systems, Inc., cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark.

Warning and Disclaimer

This book is designed to provide information about Data Center technologies. Every effort has been made to make this book as complete and as accurate as possible, but no warranty or tness is implied.The information is provided on an as is basis. The authors, Cisco Press, and Cisco Systems, Inc., shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book or from the use of the discs or programs that may accompany it.The opinions expressed in this book belong to the author and are not necessarily those of Cisco Systems, Inc.

Feedback Information

At Cisco Press, our goal is to create in-depth technical books of the highest quality and value. Each book is crafted with care and precision, undergoing rigorous development that involves the unique expertise of members from the professional technical community.Readers feedback is a natural continuation of this process. If you have any comments regarding how we could improve the quality of this book or otherwise alter it to better suit your needs, you can contact us through e-mail at [email protected]. Please make sure to include the book title and ISBN in your message.We greatly appreciate your assistance.

Corporate and Government Sales

Cisco Press offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact:

U.S. Corporate and Government Sales

1-800-382-3419 [email protected]

For sales outside of the U.S. please contact:

International Sales

1-317-581-3793 [email protected]

DataCenter.book Page ii Wednesday, November 12, 2003 9:52 AM

iii

. Publisher John WaitEditor-In-Chief John KaneCisco Representative Anthony WolfendenCisco Press Program Manager Nannette M. NobleProduction Manager Patrick KanouseDevelopment Editors Christopher Cleveland

Betsey HenkelsSenior Project Editor Sheri CainCopy Editors Krista Hansing, Kris SimmonsTechnical Editors Mario Baldi, Robert Batz, Mark Gallo, Ron Hromoko,

Fabio Maino, Scott Van de Houten, Stefano Testa, Brian WalckTeam Coordinator Tammi BarnettCover Designer Louisa AdairComposition Octal Publishing, Inc.Indexers Tim Wright, Eric SchroederProofreader Angela Rosio

fmatter.fm Page iii Wednesday, November 12, 2003 2:30 PM

iv

About the Authors

Mauricio Arregoces

, CCIE No. 3285, is the manager of the Cisco Enterprise Solutions Engineering team on Data Center designs. He has been in the networking industry for 17 years and has been involved in designing, implementing, and maintaining large-scale enterprise networks. Mauricio holds a B.S. degree in computer science from the Colombian School of Engineering and a M.S. degree in computer science from California State University at Northridge.

Maurizio Portolani

is a network design consultant at Cisco Systems Inc., involved in architecting and validating large-scale Data Center designs.Maurizio has led patents on advanced spanning-tree and load-balancing features, and is the author of several Cisco Data Center solution architectures that cover the Layer 2 and Layer 3 design, load balancing, security, DNS, SSL ofoading, and integration with application environments.Maurizio also works closely with various technology teams at Cisco to dene and validate new features that enhance Cisco product support for customer system level solutions. Maurizio has designed product features in areas such as spanning-tree, SSL, and HTTP persistence mainly for the Catalyst 6500 family.

DataCenter.book Page iv Wednesday, November 12, 2003 9:52 AM

v

About the Contributor

Martin Pueblas

, CCIE No. 2133, CISSP No. 40844, technical marketing engineer, Central Marketing Organiza-tion, Cisco Systems, Inc. Martin contributed to the content in the security-related chapters in this book (Chapters 5, 15, and 21).Martin is a network security expert with more than ten years of experience in the networking industry who obtained his CCIE certication in 1996 and recently achieved his CISSP. Martin joined Cisco in 1998 and since then has held a variety of technical positions. In 2000, Martin joined the Advanced Engineering Services team as a network design consultant, where he provided design and security consulting services to large corporations and service providers. During this period, Martin wrote a variety of technical documents, including design guides and white papers that dene the Cisco best practices for security and virtual private networks (VPNs). In late 2001, Martin began his current position as a technical marketing engineer for security and VPN technologies. As part of his current responsibilities, Martin is leading the development of a security architecture for service pro-viders. Before joining Cisco Systems, Martin worked for a Cisco Gold partner in South America, where he provided support, consulting, and training services to numerous customers and partners in Latin America.

About the Technical Reviewers

Mario Baldi

is associate professor on the tenure track at the Computer Engineering Department of Torino Poly-technic, Torino, Italy, and vice president for Protocol Architecture at Synchrodyne Networks, Inc., New York. He received his M.S. degree

summa cum laude

in electrical engineering in 1993 and his Ph.D. in computer and system engineering in 1998, both from Torino Polytechnic. He was assistant professor on the tenure track at Torino Poly-technic from 1997 to 2002. He joined Synchrodyne Networks, Inc., in November 1999.Mario has been a visiting researcher at the IBM T.J. Watson Research Center, Yorktown Heights, New York; at Columbia University, New York; and at the International Computer Science Institute (ICSI), Berkeley, California.As part of his extensive research activity at Torino Polytechnic, Mario has been leading various networking research projects involving universities and industrial partners, funded by the European Union, local government, and various companies, including telecommunications carriers, such as Infostrada and Telecom Italia, and research institutions, such as Telecom Italia Labs. Mario provides on a regular basis consultancy and training services, both directly to companies and through various training and network consultancy centers. Mario has co-authored more than 50 papers on various networking related topics and two books, one on internet-working and one on switched LANs.

Robert (Bob) Batz

is a technical leader in the Cisco Mobile Wireless group. Bob develops software for content networking services, primarily addressing the mobile wireless space. He has also developed and sustained load-balancer products, frequently working closely with the TAC support teams in Cisco to resolve customer issues and to assist with network designs. Bob has been at Cisco for eight years and has been working in the content network-ing area for four years.

Mark Gallo

is a technical manager with America Online, where he leads a group of engineers responsible for the design and deployment of the domestic corporate intranet. His network certications include CCNP and CCDP. He has led several engineering groups responsible for designing and implementing enterprise LANs and international IP networks. He has a B.S. in electrical engineering from the University of Pittsburgh. Mark resides in northern Vir-ginia with his wife, Betsy, and son, Paul.

DataCenter.book Page v Wednesday, November 12, 2003 9:52 AM

vi

Fabio Maino

is a senior security architect at the San Josebased Andiamo Systems, recently acquired by Cisco Systems. Fabio is one of the major contributors of the INCITS T11 Fibre Channel Security Protocols (FC-SP) working group, which is designing the security layer of the next-generation Fibre Channel architecture. Fabio is also an active contributor to the activities of the Internet Engineering Task Force (IETF) Simple Network Manage-ment Protocol version 3 (SNMPv3) working group, where he recently proposed an Advanced Encryption System (AES) extension for the USM security model.Fabio received an M.Sc. degree in electronic engineering and a Ph.D. in computer and system engineering from Torino Polytechnic, Torino, Italy, in 1994 and 1999, respectively. During his Ph.D. studies, he was a guest researcher at Hewlett-Packard, working on VerSecure; then, he researched public-key infrastructure in Torino and nally moved to San Jose. After joining Cisco Systems at the beginning of 2000, he moved to Andiamo Systems with the original group of engineers that founded the company.

Scott Van de Houten

, CCIE No. 1640, is a distinguished systems engineer for the Technical Operations group. He is currently a technical lead for the Enterprise Routing and Switching Technology leadership program. His respon-sibilities include developing customer requirements for the product teams and customer technical consulting. Scott has been with Cisco for 11 years and has worked as a network engineer for 17 years.

Stefano Testa

joined Cisco Systems, Inc., in 1998, as part of the Catalyst 6500 software development team. He moved to technical marketing in 2000, initially focusing on content switching and geographic load balancing. Then, Stefano expanded his role to cover security products, such as SSL and rewall modules. Stefano works closely with Cisco eld teams to help large customers design fully redundant, high-performance integrated Data Centers and content-aware solutions. He also works on a daily basis with multiple Cisco engineering teams on future software releases, network management, and platforms for L4-7 services.Cisco Press gratefully acknowledges the efforts of technical reviewers

Ron Hromoko

and

Brian Walck

; their con-tributions helped ensure the quality and accuracy of the text.

DataCenter.book Page vi Wednesday, November 12, 2003 9:52 AM

vii

Dedications

Mauricio Arregoces

:

To my wife Frances, whose support, patience, and encouragement got me through the writing of this book.To my daughter, Gabrielle, who lets me see the wonders of the world through her eyes.To my son, Julian, who constantly reminds me of the right priorities in life through his candid self.To my family, who gave me time and understanding during the many days, nights, and weekends I spent away from them working on this project.

Maurizio Portolani

:

I dedicate this book to Rosangela and Stefano for their continued support and understanding and to Margherita, Alda, and Leonardo for their example of hard work and strength that helped me during the long nights of work. I want to thank Giovanni, whose religious love for books always inspired me. A special thanks to all my friends who bore with my schedule during the last two years of hard work.

DataCenter.book Page vii Wednesday, November 12, 2003 9:52 AM

viii

Acknowledgments

Mauricio Arregoces and Maurizio Portolani

:

To John Kane, our executive editor, for his constant encouragement and support through the evolution of the book and for his understanding and willingness to look past our many schedule slips.To Chris Cleveland, our development editor, for his keen eye, great work, and always helpful attitude, without whom the book would certainly not be what it is.To Bob Batz, Brian Walck, Ron Hromoko, Scott Van deHouten, Stefano Testa, Mario Baldi, and Fabio Maino, whose feedback contributed to the quality of the book.Very special thanks to Fabio Maino for helping us on security topics. Fabio, your strength and optimism are beyond words.To Martin Pueblas, who contributed substance and knowledge on matters of cryptography and overall security.To Patrick Folstrom, who helped us understand the world of Domain Name System (DNS) and for his accurate review of the DNS-related topics.And to the Cisco Press team behind the scenes for supporting this project for making this book a reality.

Mauricio Arregoces:

To Maurizio, for his dedication, appetite for knowledge, and search for perfection. You kept me honest and focused; may you nd the perfect mountain.

Maurizio Portolani

:

I want to especially thank Mauricio for his vision, which made it possible to develop a book on such an interesting topic. Thank you for involving me in this project for the past two years, for the help and encouragement, and for the focus on quality.

DataCenter.book Page viii Wednesday, November 12, 2003 9:52 AM

ix

Contents at a Glance

Introduction xxxvi

Part I An Introduction to Server Farms 3

Chapter 1

Overview of Data Centers 5

Chapter 2

Server Architecture Overview 31

Chapter 3

Application Architectures Overview 71

Chapter 4

Data Center Design Overview 117

Chapter 5

Data Center Security Overview 159

Chapter 6

Server Load-Balancing Overview 205

Part II Server Farm Protocols 239

Chapter 7

IP, TCP, and UDP 241

Chapter 8

HTTP and Related Concepts 309

Chapter 9

SSL and TLS 369

Chapter 10

DNS Essentials and Site-Selection Considerations 397

Chapter 11

Streaming Protocols Overview 441

Part III Infrastructure Protocols 477

Chapter 12

Layer 2 Protocol Essentials 479

Chapter 13


Chapter 14

IBM Data Center Technology 569

DataCenter.book Page ix Wednesday, November 12, 2003 9:52 AM

x

Part IV Security and Server Load Balancing 595

Chapter 15

Security Protocols and Technologies 597

Chapter 16

Load-Balancing Modes and Predictors 653

Chapter 17

Server Health Management 689

Chapter 18

Session Tracking and Cookies 727

Chapter 19

Persistence Mechanisms on Load Balancers 753

Part V Data Center Design 799

Chapter 20

Designing the Data Center Infrastructure 801

Chapter 21

Integrating Security into the Infrastructure 865

Chapter 22

Performance Metrics of Data Center Devices 919

Part VI Appendixes 961

Appendix A

Character Sets 963

Appendix B

HTTP Header Fields 977

Appendix C

Video Encoding Mechanisms 987

Appendix D

Loopback Interface Configuration Procedures 995

Appendix E

Configuring Servers to Insert Cookies 1009

Appendix F

Client-Side and Server-Side Programming 1013

Index

1025

DataCenter.book Page x Wednesday, November 12, 2003 9:52 AM

xi

Table of Contents

Introduction xxxvi

Part I An Introduction to Server Farms 3

Chapter 1

Overview of Data Centers 5

Data Centers Defined 5Data Center Goals 6Data Center Facilities 7Roles of Data Centers in the Enterprise 7Roles of Data Centers in the Service Provider Environment 9

Application Architecture Models 9The Client/Server Model and Its Evolution 9The n-Tier Model 11Multitier Architecture Application Environment 12

Data Center Architecture 13Aggregation Layer 15Access Layer 16

Front-End Segment 16Application Segment 17Back-End Segment 18

Storage Layer 19Data Center Transport Layer 20

Data Center Services 22IP Infrastructure Services 23Application Services 24Security Services 25Storage Services 26Business Continuance Infrastructure Services 26

Summary 27

Chapter 2

Server Architecture Overview 31

Network Attachment 32Network Interface Cards 32Server Multihoming 33PCI and PCI-X Buses 34

DataCenter.book Page xi Wednesday, November 12, 2003 9:52 AM

xii

Client and Server Packet Processing 35User Mode and Kernel Mode 35Ethernet Driver 36Server TCP/IP Processing 37Sockets 39

TCP and Server Applications 41Segments, Datagrams, and Frames 41TCP Interactive Traffic Profile 43

Connection Setup 43Maximum Segment Size 44TCP Retransmission 44Delayed ACK 45Nagle Algorithm 46Connection Closure 46

TCP Bulk Transfer Traffic Profile 47TCP Windows 47ACK Scheme 48High-Speed Networks and the Window Scale Option 49

UDP and Server Applications 50

Server Availability 51Processes and Threads 51TCP and UDP Connections to a Failed Server 54

TCP Timeout 54SYN Retransmission 55

Status of HTTP Applications 55

Configuring a Web Server 56Configuring TCP and HTTP Parameters 57Server Processes 57Directories 58Virtual Hosting 58

Running Multiple Servers on the Same Machine 59Using Multiple IP Addresses on the Same Machine 59Using Multiple Layer 4 Ports on the Same Machine 60Using the HTTP Host Tag Header 61

Network Architecture Design Options 61Increasing Server Performance 62

Jumbo Frames 62Reverse Proxy Caching 63

Increasing Server Availability with Load Balancing 65Preventing Server Overload 65Monitoring TCP Connections 67

DataCenter.book Page xii Wednesday, November 12, 2003 9:52 AM

xiii

Summary 68

For Further Reading 69

Chapter 3

Application Architectures Overview 71

Taxonomy of Applications and Hosted Servers 72

Integration of Applications 75Enterprise Application Integration 75Network Design Implications of EAI 76

Multitier Applications 77Markup Languages: HTML and XML 79

HTML 80XML 82

User Agents 83Browsers 84Helpers and Plug-Ins 84Client-Side Programming 85

Web Servers 86Server-Side Programming 87

Web Programming Technologies Overview 88Case Study: Web-Client Interaction with Java Servlets 90

Middleware 91Components: EJBs and DCOM 93Network Trafc Patterns: RPC, RMI, ORPC, IIOP 93

Database Access 95

Network Architecture Considerations 97Load Balancing 97Clustering 99

Cluster Models 100Geographical Clustering 101

Security 104Using RMI and DCOM Through a Firewall 106IDS Signatures 107

Multitier Design Case Study 108High Availability 109Security 111

Summary 113


DataCenter.book Page xiii Wednesday, November 12, 2003 9:52 AM

xiv

Chapter 4

Data Center Design Overview 117

Types of Server Farms and Data Centers 119Internet Server Farms 120Intranet Server Farms 122Extranet Server Farms 124Internet Data Center 125Corporate Data Center 126

Data Center Topologies 126Generic Layer 3/Layer 2 Designs 126

The Need for Layer 2 at the Access Layer 130Alternate Layer 3/Layer 2 Designs 132Multiple-Tier Designs 133

Expanded Multitier Design 135Collapsed Multitier Design 137The Need for Layer 2 at the Access Layer 138

Fully Redundant Layer 2 and Layer 3 Designs 139The Need for Redundancy 139Layer 2 and Layer 3 in Access Layer 141

Layer 2, Loops, and Spanning Tree 142

Fully Redundant Layer 2 and Layer 3 Designs with Services 146Additional Services 146Service Deployment Options 147Design Considerations with Service Devices 148 Application Environment Trends 150

Application Architecture Trends 150Network Infrastructure Trends 152

Summary 157

Chapter 5

Data Center Security Overview 159

The Need for a Secure Data Center 159

Vulnerabilities and Common Attacks 160Threats 160Vulnerabilities 161

Exploitation of Out-of-Date Software 161Exploitation of Software Defaults 162

Common Attacks 162Scanning or Probing 162DoS 162Distributed Denial of Service 164Unauthorized Access 165

DataCenter.book Page xiv Wednesday, November 12, 2003 9:52 AM

xv

Eavesdropping 165Viruses and Worms 165Internet Infrastructure Attacks 166Trust Exploitation 166Session Hijacking 166Buffer Overow Attacks 167Layer 2 Attacks 167

Network Security Infrastructure 169ACLs 169

Standard and Extended Access Lists 169 Router ACLs and VLAN ACLs 170Dynamic ACLs (Lock and Key) 171Reexive ACLs 172

Firewalls 173Packet-Filtering Firewalls 174Proxy Firewalls 174Stateful Firewalls 175Hybrid Firewalls 176Common Firewall Limitations 178

IDSs 178Network-Based IDSs 179Host-Based IDSs 180Network-Based Versus Host-Based IDSs 180Anomaly-Based Versus Signature-Based IDS 181Signatures 181Typical IDS Response Actions 182

Layer 2 Security 183Port Security 183ARP Inspection 184Private VLANs 185802.1Q Tag All 187Private VLANs and Firewalls 187

Security Fundamentals 188Cryptography 188

Symmetric Encryption 190Asymmetric Encryption 191Cryptographic Hashing Algorithms 193Cryptographic HMACs 194Digital Signatures 195

Virtual Private Networks 196AAA 197

DataCenter.book Page xv Wednesday, November 12, 2003 9:52 AM

xvi

Data Center Security Framework 197Security Policies 198Security Life Cycle 198Secure Management Framework 200

Isolating the Management Infrastructure 200Encryption of Control Data 201Strong Authentication for Access Control 201

Incident Response and Attack Mitigation 202

Summary 202

Chapter 6

Server Load-Balancing Overview 205

Load Balancing Defined 205Load Balancing Functions 206DNS Round-Robin 207Server Load Balancing 209Cache Load Balancing 210Other Load-Balancing Applications 211

VPN/IPSec Load Balancing 211Firewall Load Balancing 212

Key Concepts of Load Balancing 213Load-Balancing Process 215Layer 4 Load Balancing 216Layer 5 Load Balancing 216Connection Tracking 218Connection Persistence 219Session Persistence 219

Session-Persistence Problems and Solutions 221Server Health 224

In-Band Server Health Tracking 224Out-of-Band Server Health Tracking 225

High Availability Considerations 225Redundancy Protocol 227

Active-Standby Load-Balancing Conguration 228Active-Active Load-Balancing Conguration 228

Connection and Session State Failover 231Stateless Failover 231Sticky Failover 231Stateful Failover 231

DataCenter.book Page xvi Wednesday, November 12, 2003 9:52 AM

xvii

Generic Load Balancer Architecture 232Generic Architecture Components 232

Critical Components of a Load Balancer 234

Summary 235

Part II Server Farm Protocols 239

Chapter 7

IP, TCP, and UDP 241

Layers and Protocols 241

IP 245IP Header 246

Version Field 247Header Length Field 248Type of Service Field 248Total Length Field 250Identier Field 250Flags Field 251Fragment Offset Field 251Time-To-Live Field 251Protocol Field 252Header Checksum Field 254Source Address and Destination Address Fields 255Options Field 255

IP Header Compression 256

TCP 256TCP Header 258Source Port and Destination Port Fields 259

Sequence Number Field 262Acknowledgement Number Field 263TCP Header Length Field 264TCP Control Flags 264Window Size Field 266Checksum Field 266Urgent Pointer Field 266Options Field 266

TCP Connection Overview 267Connection Establishment 268Connection Termination 272

DataCenter.book Page xvii Wednesday, November 12, 2003 9:52 AM

xviii

TCP Flow Control 276Timeout and Retransmission 276Sliding Windows 276Congestion Control 278Fast Retransmission and Fast Recovery 280Delayed ACK and Immediate ACK 280Nagle Algorithm 281

TCP Half Close 282MSS Option 283Path MTU Discovery Option 284

Issues with PMTUD 287TCP SACK Option 292Timestamp Option 294Window Scale Option 295PAWS 295TCP Header Compression 296

UDP 299UDP Header 299

Source Port Field 299Destination Port Field 300Length Field 300Checksum Field 300

UDP Transaction Overview 301UDP Header Compression 305

Summary 306

References 307

Chapter 8

HTTP and Related Concepts 309

Resources and Messages 309URIs 310

Relative or Partial URIs 311Absolute or Full URIs 312Rules for Naming Relative and Absolute URIs 314

URLs 315Relative and Absolute URLs 316URL Encoding 316URL Syntax for Specic Schemes 319

URNs 320URN Namespace 321

URIs, URLs, and URNs 322

DataCenter.book Page xviii Wednesday, November 12, 2003 9:52 AM

xix

MIME 323MIME and HTTP Entities 326Character Sets 326Media Types 327

HTTP Overview 328HTTP Operation 329

HTTP Version 330HTTP Message Format 332

Message Header 334Message Body 334

HTTP Connection Overview 335Persistent Connections and Pipelining 338HTTP Performance 340

Performance of HTTP/1.1 Versus HTTP/1.0 340HTTP Compression 342

HTTP General Header 344Cache-Control General Header Field 344Connection General Header Field 345Date General Header Field 346Pragma General Header Field 346Transfer-Encoding General Header Field 347

Request Header 347Request Header Methods 348

OPTIONS Request Header Method 348GET Request Header Method 349HEAD Request Header Method 349POST Request Header Method 349PUT Request Header Method 350DELETE Request Header Method 351TRACE Request Header Method 351CONNECT Request Header Method 351

Request URI 351Request Header Fields 352

Accept Request Header Field 353Accept-Charset Request Header Field 353Accept-Encoding Request Header Field 353Authorization Request Header Field 354Host Request Header Field 354If-Modied-Since Request Header Field 355Max-Forwards Request Header Field 355Range Request Header Field 355Referer Request Header Field 355User-Agent Request Header Field 356

DataCenter.book Page xix Wednesday, November 12, 2003 9:52 AM

xx

Response Header 356HTTP Status Codes 356

1

xx

Informational Status Codes 3572

xx

Success Status Codes 3583

xx

Redirection Status Codes 3594

xx

Client Error Status Codes 3605

xx

Server Error Status Codes 362Response Header Fields 362HTTP Authentication 364

Basic Authentication 364Message Digest Authentication 364

Entity Header 365

Summary 366

Chapter 9

SSL and TLS 369

SSL Overview 370

SSL Operations 371HTTPS 372SSL Session Negotiation 374SSL Data Exchange 378

Performance Implications of SSL 379Session Resumption 380SSL and Load Balancing 382SSL Performance Optimization 384

Authentication and Digital Certificates 385SSL Authentication Overview 385Public Key Infrastructure 388

SSL Ciphersuites 389

Analyzing SSL Traces 391

Summary 393


Chapter 10

DNS Essentials and Site-Selection Considerations 397

DNS Architecture 398FQDN 400Zones 400Resource Records 402

DataCenter.book Page xx Wednesday, November 12, 2003 9:52 AM

xxi

DNS Components 404DNS Resolver 405DNS Server 407DNS Proxy 409

DNS Forwarder 410Caching-Only Server 410

DNS Resolution Process 411Query Format 412Root Hint 413Referrals 414Recursive and Iterative Queries 417

Redundant Name Servers 418Master and Slave 418Zone Transfers 418

Transport Protocols 420

DNS Caching 420TTL 421Client Applications and Caching 422

Distribution of Multiple Records 423NS Records 423A Records 425Client Applications and Multiple Records 426

DNS Server Placement 426DNS Forwarder Placement 427Internal and External Namespace 428DNS Resolution in the Presence of Split Namespace and Forwarders 430

Site-Selection Considerations 430Site Selection Architecture 431Referrals to Site Selectors and Subdomains 433Proximity 435Site Selection and Caching 436Stickiness 437

Summary 438


Chapter 11

Streaming Protocols Overview 441

Download-and-Play, HTTP Streaming, and Real-Time Streaming 442

UDP Versus TCP 445

DataCenter.book Page xxi Wednesday, November 12, 2003 9:52 AM

xxii

Analog and Digital Video 447

Codecs 448Basic Encoding Mechanisms 448Main Encoding Formats 450

Packetization 453

Transport Formats 454RTP 454RTCP 457Example of an RTP Session 458QuickTime, Real, and Windows Media 460Trace Analysis of UDP, TCP, and HTTP Tunneling 461

Control Protocols 466RTSP 467Interleaving 470

Unicast, Multicast, and Stream Splitting 471

Streaming Products 473Codecs 473Wire Formats 474

Summary 475

Part III Infrastructure Protocols 477

Chapter 12


IEEE 802 479

Ethernet 481Frame Format 482Address Format 485Frame Size 487Fast Ethernet 489Gigabit Ethernet (IEEE 802.3z) 49110-Gigabit Ethernet (IEEE 802.3ae) 492Ethernet Physical Layers 493

Ethernet Physical Layers 494Fast Ethernet Physical Layers 494Gigabit Ethernet Physical Layers 49510-Gigabit Ethernet 495

Giant and Jumbo Frames 496

DataCenter.book Page xxii Wednesday, November 12, 2003 9:52 AM

xxiii

Ethernet Switching 498

Layer 2 Protocols 500

VLANs and Trunks 502Creating VLANs 504Creating Trunks 505

EtherChannels 507Creating a Channel 507

STP 508Bridge Identifier 510Port Roles and States (802.1w) 510Failure Detection (802.1w) 513Multiple VLANs 513

4096 VLANs 513Rapid PVST+ 514802.1s 516

Logical Ports 517Configuring Rapid PVST+ 518Configuring 802.1s 519Access Ports 520

Summary 521


Chapter 13


ARP Protocol and Tables 525

HSRP, VRRP, and GLBP 527HSRP 528

Active/Standby Election 529HSRP Groups 530Failure Detection 531Tracking 533

VRRP 533Master/Backup Election 534VRRP Groups 535Failure Detection 535

GLBP 536Active/Standby Election 537GLBP Groups 538Failure Detection 538Tracking 539Load Distribution 540

DataCenter.book Page xxiii Wednesday, November 12, 2003 9:52 AM

xxiv

OSPF 540OSPF Neighbor States 542OSPF Areas 543LSAs 544Failure Detection in OSPF 545Metric Tuning 547Redistribution 547Summarization and Filtering 550Default Advertisement 551

EIGRP 551Failure Detection 552Metric Tuning 553Redistribution 554Summarization and Filtering 555Default Advertisement 555

NAT 556NAT Support for Applications 559IOS NAT on Routers 561NAT on PIX Firewalls 563NAT on Load Balancers 565

Summary 567


Chapter 14 IBM Data Center Technology 569Mainframes 569

IBM Data Center Components 570

Mainframe Attachment Options 573Channel Attachments 573LAN Attachment Options 575IP Addressing 576

IBM Networking 577Subarea SNA 577APPN 579

SNA over TCP/IP 580DLSw 580SNAsw 581

Enterprise Extender 582Branch Extender 583DLUR/DLUS 583TN3270 584

DataCenter.book Page xxiv Wednesday, November 12, 2003 9:52 AM

xxv

Sysplex and Parallel Sysplex 585Geographically Dispersed Parallel Sysplex 589

IBM Data Centers Today 590

Summary 592

Part IV Security and Server Load Balancing 595

Chapter 15 Security Protocols and Technologies 597Cryptography 597

Symmetric Cryptography 598Digital Encryption Standard (DES) 598Triple DES (3DES) 600Advanced Encryption Standard (AES)Rijndael 601Rivests Ciphers 602IDEA 602

Asymmetric Cryptography 602RSA 603RSA Key Exchange 604Digital Signature Standard (DSS) 605Dife-Hellman 606

Hashing Algorithms 607MD2, MD4, and MD5 607SHA 608

Cipher Summary 608

U.S. Government and Cryptography 609NIST and FIPS 609Export-Grade Ciphers 611

PKI 612PKI Standards 614Digital Certificates 615

Generating Certicates 616Digital Certicate Format 617

Certificate Authorities 619Role of CAs During the Key Exchange 619CA Certicates 621CA Deployment Options 623Enrollment with an External CA 624Enrollment with an Enterprise CA and Use of the SCEP 624Revocation 625

DataCenter.book Page xxv Wednesday, November 12, 2003 9:52 AM

xxvi

Transport Security 626SSL and TLS 626

SSLv2, SSLv3, and TLS 1.0 627SSL and the TCP/IP Layers 627SSL Certicates 629SGC and Step-Up 630Choosing SSL Ciphersuites 632

IPSec 633IPSec and TCP/IP Layers 634IKE 637Choosing IPSec Security Parameters 638

SSL VPNs and IPSec VPNs 639

Authentication Protocols and Technologies 640Authentication Technologies 641

OTPs 641Challenge/Response: IEEE 802.1x EAP-MD5 642Digital Certicates: Client Authentication in SSL 642Kerberos 644

AAA Protocols 645TACACS+ 645RADIUS 646

Network Management Security 647SSH 647SNMPv3 649

Summary 649

Chapter 16 Load-Balancing Modes and Predictors 653Modes of Operation 653

Switching Concepts 654Bridging 654Routing 655

Dispatch Mode 657Directed or Server NAT Mode 660Client NAT 662Connection Spoofing 664

Connection Spoong Processing 664Connection Remapping 667

Direct Server Return 669Performance Implications 671

DataCenter.book Page xxvi Wednesday, November 12, 2003 9:52 AM

xxvii

Load-Balancing Algorithms 673Server Farm Load-Balancing Algorithms 673

Round-Robin Predictor 676Weighted Round-Robin Predictor 677Least Connections 678Weighted Least Connections Predictor 679Fastest Predictor 680Source IP Predictor 681Hash Address Predictor 681URL and Hash URL Predictors 681Maximum Connections 682

Cache Farm Load-Balancing Algorithms 683Domain and Domain Hash 685

Summary 686

Chapter 17 Server Health Management 689Load-Balancing Terminology 690

Server Management 690Graceful Shutdown 691Slowstart 693Max Connections and Min Connections 694

Server Management Interface 696XML 696SNMP 697

OID and MIBs 697CISCO-SLB-MIB 698RMON 699TRAPs 700

Server Failure Detection 700Server Monitoring Using Probes 700Server Monitoring Using SNMP 701

Probe Types 702In-Band Health Checks 703

Connection Reassign and Server Markdown 704Server Recovery (auto-unfail) 705HTTP Return Code Checks 706

Out-of-Band Probes 707Dynamic Feedback Protocol 708Probes Comparison: Determining What to Use 709

DataCenter.book Page xxvii Wednesday, November 12, 2003 9:52 AM

xxviii

Out-of-Band Probes 710Layer 2 Probes: ARP 711Layer 3 Probes: ICMP 711Layer 4 Probes 711

TCP Probe 711UDP Probe 712

Application Layer Probes 713HTTP Probe 714SSL Probe 715DNS Probe 717FTP Probe 717SMTP Probe 718POP3 Probe 718IMAP4 Probe 718

Case Study: Server Health for Virtual Hosting 718

Case Study: HTTP and HTTPS 722

Summary 724

Chapter 18 Session Tracking and Cookies 727What a Session Is and Why It Matters 727

Cookies 728Session Cookies and Persistent Cookies 728Cookie Format 729How Browsers Handle Cookies 731

How Browsers Handle Cookie Attributes 731How Browsers Handle Multiple Cookies 733Where Cookies Are Stored 734

Netscape, RFC 2109, and RFC 2965 735

How Servers Track User Sessions 736Session Tracking Overview 736Session Tracking with Form Hidden Fields 737Session Tracking with URL Rewrite 738Session Tracking with Cookies 739Session Tracking Methods Combined 739Case Study: Apache mod_session 740Case Study: HTTP Sessions with Servlets 743

Session Persistence for Server Clusters 749

Summary 750


DataCenter.book Page xxviii Wednesday, November 12, 2003 9:52 AM

xxix

Chapter 19 Persistence Mechanisms on Load Balancers 753The Concept of Session Persistence 754

HTTP Session Persistence 754SSL Persistence 755Persistence with Protocols Using Multiple Ports 755

Persistence Considerations for Clients Using Proxy Servers 758Proxy Server Overview 758Clustered Proxy Servers 759Implications of Proxy Servers on Load Balancing 759

Persistence Using Session Sharing Servers 761

Session Persistence Mechanisms 761Predictors 761Sticky Methods 762Sticky Groups 764

Source IP Sticky 765Source IP Sticky Configuration 765Configuration for Mega Proxies 766Source IP Hash 768

Cookie Sticky 768Cookie Passive 768Cookie Match 771Cookie Active 774

URL Sticky 776URL Cookie 776URL Match 779URL Hash 780

HTTP Redirection Sticky 782

SSL Sticky 785SSL Sticky Configuration 786SSL Persistence Caveats 787

Case Study 789E-Commerce Applications 790

SSL Persistence and Servers Sharing Session Information 791Source IP Persistence 792HTTP Redirection Persistence 792SSL Ofoading and URL Cookie Persistence 794

Summary 797

DataCenter.book Page xxix Wednesday, November 12, 2003 9:52 AM

xxx

Part V Data Center Design 799

Chapter 20 Designing the Data Center Infrastructure 801Topology Overview 801

Switching Paths 806Cisco IOS Switching Paths 807Multilayer Switching (MLS) 809

Using VLANs to Virtualize the Physical Data Center Infrastructure 810VLAN Topologies 810SVIs 812Autostate 814

Link Redundancy and Load Distribution 815Scaling the Bandwidth with EtherChannels 815Traffic Distribution on Layer 2 Links 818Traffic Distribution on Layer 3 Links 819Dual-Attached Servers 821

Spanning-Tree Considerations 822Choosing the Spanning-Tree Algorithm 823Rapid Convergence 827

Fast Convergence in PVST+ 828Fast Convergence in Rapid PVST+ 829Fast Convergence in MST 831

Minimizing Topology Changes 831Loop Prevention: UDLD and Loopguard 832

Internal Redundancy Considerations 833Supervisor Redundancy 834NSF 835

Layer 2 Data Center Design 837VLAN Configuration 837Access Ports 839Trunk Configuration 840Spanning-Tree Topology 841Layer 2 Configuration Summary 843

Layer 3 Data Center Design 845Routing Between Core and Aggregation Routers 846Default Gateway Redundancy: HSRP, VRRP, and GLBP 849

Matching Layer 3 and Layer 2 Topologies 850To Preempt or Not to Preempt? 851Timer Tuning 851

DataCenter.book Page xxx Wednesday, November 12, 2003 9:52 AM

xxxi

Using OSPF in Data Center Design 852OSPF Topology 852Area Assignment and Summarization 853Stub Areas 854Advertising the Local Subnets 854OSPF Metric Tuning 856Convergence Time 856OSPF Conguration Summary 857

Using EIGRP in Data Center Design 858EIGRP Topology 858Default Routes 859Summarization 860EIGRP Conguration Summary 860

Layer 3 Configuration Summary 861

Summary 862

Chapter 21 Integrating Security into the Infrastructure 865Defining Security Zones 865

Internet Edge 869Deploying Antispoofing Filtering 870Using uRPF 872Using ACLs 873Implementing Traffic Rate Limiting 874Securing Routing Protocols 875

Route Filters 876Neighbor Router Authentication 876

Deploying Stateful Firewalls 878Implementing Intrusion Detection 879Internet Edge Design 882

Campus Core 884

Intranet Server Farms 885Deploying Stateful Firewalls 887Applying Packet Filters 889Deploying Intrusion Detection 891

Network-Based Intrusion Detection 891Host-Based Intrusion Detection 893

Enabling Other Security Features 894Port Security 894ARP Inspection 895Private VLANs 895VLAN Tagging 896

DataCenter.book Page xxxi Wednesday, November 12, 2003 9:52 AM

xxxii

Server-Farm Design Alternatives 896Collapsed Server-Farm Design 897Expanded Server-Farm Design 900Redundant Firewall Designs 904

Active-Standby Firewall Environments 904MAC Address 905Election Process 905Failure Detection 906Stateful Failover 906

Active-Active (Clusters) 906Management Network 908

Management Isolation 908Encryption 910Strong Authentication 911Secure Management Design 914

Summary 916

Chapter 22 Performance Metrics of Data Center Devices 919Traffic Patterns Overview 919

Internet Traffic Patterns 920Intranet Traffic Patterns 923Traffic Patterns in the Data Center 924

Short-Lived Connections 925Long-Lived Connections 929CPS, CC, and PPS 932

Performance Metrics Overview 934Performance Metrics 934Multilayer Switch Metrics 936

Throughput 936Frame and Packet Loss 937Latency 937

Firewall Metrics 938

Load Balancer and SSL Offloader Metrics 939Load Balancer Performance Metrics 939

Trafc Patterns and Load Balancers 939CPS 942CC 943Throughput (PPS) 944Latency 944Response Time 945

DataCenter.book Page xxxii Wednesday, November 12, 2003 9:52 AM

xxxiii

SSL Offloaders Performance Metrics 946CPS 948CC 948PPS 949Latency 949Response Time 949

Testing Performance Metrics 950Testing Tools 951

Software Testing Tools 951Hardware Testing Tools 952

Staging the Testbed 953Testing Environment 954Selecting Your Data Mix 956Running the Tests 956

Summary 957

Part VI Appendixes 961

Appendix A Character Sets 963

Appendix B HTTP Header Fields 977

Appendix C Video Encoding Mechanisms 987

Appendix D Loopback Interface Configuration Procedures 995

Appendix E Configuring Servers to Insert Cookies 1009

Appendix F Client-Side and Server-Side Programming 1013

Index 1025

DataCenter.book Page xxxiii Wednesday, November 12, 2003 9:52 AM

xxxiv

Icons Used in This BookCisco uses the following standard icons to represent different networking devices.

You will encounter several of these icons within this book.

Cisco Works WorkstationPC

Laptop Web Browser

Web Server

Route/Switch Processor

Hub Intrusion DetectionSystemCisco 7500

Series Router

AccessServer

CiscoSecureScanner Cisco

Directory ServerCisco

CallManagerLoad Balancer IP/TVBroadcast

Server

SwitchRouter FirewallsMultilayer Switch

File Server Printer

Phone

DWDM-CWDM Storage Subsystem

Multilayer Switchwith Load Balancer

SSL Offloader

Fax VPN Concentrator

Bridge

ATM Switch

ISDN/Frame Relay switch

Gateway

Network Cloud

Concentrator

Cache or Content Engine Tape Subsystem Fibre Channel

Switch

DataCenter.book Page xxxiv Wednesday, November 12, 2003 9:52 AM

xxxv

Command Syntax ConventionsThe conventions used to present command syntax in this book are the same conventions used in the Cisco IOS Command Reference. The Command Reference describes these conventions as follows:

Vertical bars (|) separate alternative, mutually exclusive elements. Square brackets [ ] indicate optional elements. Braces { } indicate a required choice. Braces within brackets [{ }] indicate a required choice within an optional element. Boldface indicates commands and keywords that are entered literally as shown. In actual conguration exam-

ples and output (not general command syntax), boldface indicates commands that the user manually enters (such as a show command).

Italics indicate arguments for which you supply actual values.

DataCenter.book Page xxxv Wednesday, November 12, 2003 9:52 AM

xxxvi

IntroductionData Centers are complex systems encompassing a wide variety of technologies that are constantly evolving. Designing and maintaining a Data Center network requires skills and knowledge that range from routing and switching to load balancing and security, including the essential knowledge of servers and applications.This books addresses both fundamental information such as the protocols used by switches and routers; the proto-cols used in application environments; the network technology used to build the Data Center infrastructure and secure, scale, and manage the application environments; and design best practices. We hope this book becomes your Data Center reference on protocols, technology, and design.

Motivation for Writing This BookWhile speaking to networkers abroad on the topic of server load balancing, we realized that we could only convey the benets of the technology by explaining application layer information and describing the larger design issues common in application environments.Often through discussions with customers, the subjects related to load balancing take a back seat as issues of inte-gration with the entire Data Center take the forefront. This book attempts to cover the breadth and depth of the Data Center IP network. The storage network and distributed Data Center topics will be the subjects of other books.Having designed campus and Data Center networks, and having developed and supported technologies that are often referred to as content networking (load balancing, Secure Socket Layer [SSL] ofoading, and DNS routing), we felt the need for a book that described these topics in a single place and focused on what is relevant to the Data Center. This area is what this book is about: it is an all-encompassing view of Data Centers from routing and switching technologies to application-aware technologies.

Who Should Read This BookThis book is intended for any person or organization seeking to understand Data Center networks: the fundamental protocols used by the applications and the network, the typical network technologies, and their design aspects. The book is meant to be both a reference on protocols and technology and a design and implementation guide for per-sonnel responsible for planning, designing, implementing, and operating Data Center networks.

Chapter OrganizationThis book has six parts. This book is designed to be read in order from the overview of the Data Center environ-ment, through the server farms and infrastructure protocols, to security and load-balancing concepts, before you reach the Data Center design chapters. This organization also allows you to go directly to the desired chapter if you already know the information in the previous chapters.Part I, An Introduction to Server Farms, includes chapters that contain an overview of the architecture of Data Centers, servers, and applications. This part also introduces the security and load-balancing technology:

Chapter 1, Overview of Data Centers, presents Data Center environments, the Data Center architecture, and services that are used as a guide to the rest of the book.

Chapter 2, Server Architecture Overview, explores the architecture of servers. This chapter covers topics such as how servers process TCP and User Datagram Protocol (UDP) trafc, how processes and threads are used, and server health.

DataCenter.book Page xxxvi Wednesday, November 12, 2003 9:52 AM

xxxvii

Chapter 3, Application Architectures Overview, explores the application environments and how applications are architected. This chapter includes discussions on the relation between the application architectures and the design of the Data Center, the n-tier model, HTML and XML, user-agent technologies, web server technologies, and clustering technologies. This chapter introduces application concepts that are developed in Chapter 18 and Chapter 19.

Chapter 4, Data Center Design Overview, discusses the types of server farms on Data Centers, generic and alternative Layer 2 and Layer 3 designs, multitier designs, high availability, Data Center services, and trends that might affect Data Center designs.

Chapter 5, Data Center Security Overview, discusses threats, vulnerabilities and common attacks, network security devices such as rewalls and intrusion detection systems (IDSs), and other fundamental security con-cepts such as cryptography; VPNs; and authentication, authorization and accounting (AAA).

Chapter 6, Server Load-Balancing Overview, discusses reasons for load balancing, fundamental load-bal-ancing concepts, high-availability considerations, and generic load-balancing architectures. The fundamental load-balancing concepts include Layer 4 and Layer 5 load balancing, session tracking, session persistence, and server health.

Part II, Server Farm Protocols, explores the fundamental protocols used in server farms: Chapter 7, IP, TCP, and UDP, explores the protocol headers details and their relevance to network design

issues.

Chapter 8, HTTP and Related Concepts, discusses key concepts such as Uniform Resource Identiers (URIs) and URLs, Multipurpose Internet Mail Extension (MIME) and its relation to HTTP entities, and HTTP header details. Chapter 8 provides additional information on the operation of HTTP, the different versions and their performance characteristics.

Chapter 9, SSL and TLS, discusses SSL operations with specic focus on SSL session establishment, cipher-suites, and SSL performance considerations. Chapter 15 provides additional information on the public-key infrastructure (PKI), certicates, and more security-related aspects of SSL.

Chapter 10, DNS Essentials and Site-Selection Considerations, explores how the DNS namespace is organized, the DNS components in the Internet, how the DNS resolution process works, DNS conguration options, DNS server placement in the network, and how to use DNS to distribute application requests to multiple Data Centers.

Chapter 11, Streaming Protocols Overview, discusses HTTP and real streaming, the use of TCP and UDP in streaming, analog and digital video, coders-decoders (codecs), packetization, the streaming transport formats, unicast, multicast and stream splitting, and encoding mechanisms.

Part III, Infrastructure Protocols, explores the fundamental Layer 2 and Layer 3 protocols as well as IBM Data Center technologies:

Chapter 12, Layer 2 Protocol Essentials, discusses Ethernet frame types; the difference between unicast, multicast, and broadcast frames; physical layer characteristics of Ethernet technologies; jumbo frames; trunks and channels; and a variety of spanning-tree concepts. Chapter 20 provides the design best practices applied to the concepts described in this chapter.

Chapter 13, Layer 3 Protocol Essentials, discusses the Address Resolution Protocol (ARP); gateway redun-dancy protocols such as Hot Standby Router Protocol (HSRP), VRRP and GLBP; and routing-protocol essentials for Open Shortest Path First (OSPF) and Enhanced Interior Gateway Routing Protocol (EIGRP). Chapter 20 pro-vides the design best practices applied to the concepts described in this chapter.

DataCenter.book Page xxxvii Wednesday, November 12, 2003 9:52 AM

xxxviii

Chapter 14, IBM Data Center Technology, discusses mainframe attachment options, IBM networking, Systems Network Architecture (SNA) switching, Sysplex, TN3270, and current IBM Data Center designs.

Part IV, Security and Server Load Balancing, explores the security protocols and technology, load-balancing operations, server health management, session tracking and cookies, and persistence mechanisms on load balancers:

Chapter 15, Security Protocols and Technologies, discusses cryptography, U.S. governmentrelated topics about cryptography, PKI, transport security protocols (SSL and IP Security [IPSec]), authentication protocols and technologies, and network management security. This chapter also complements Chapter 9 with regards to the security design aspects of SSL and introduces the concept of SSL VPNs.

Chapter 16, Load-Balancing Modes and Predictors, discusses the load-balancing modes of operation, server load-balancing algorithms, and cache farm load-balancing algorithms.

Chapter 17, Server Health Management, discusses server health management through load balancers, SNMP, server failure detection and checking, in-band and out-of-band probes, and case studies on server checking for web hosting and e-commerce applications.

Chapter 18, Session Tracking and Cookies, explores the concept of user sessions from an application point of view. This chapter explains nonpersistent cookies, cookies in general, how servers track user sessions, ses-sion persistence on clusters of servers, and the challenges of dealing with HTTP and HTTPS. Chapter 19 fur-ther expands the topic of session persistence in load-balancing deployments.

Chapter 19, Persistence Mechanisms on Load Balancers, explains session persistence in relation to load bal-ancing; discusses key persistence mechanisms, including source-IP sticky, cookie-URL sticky, HTTP redirec-tion sticky, and SSL sticky; and presents a case study using an e-commerce application. Chapter 19 is based on the applications introduced in Chapter 3 and Chapter 18.

Part V, Data Center Design, explores the details behind designing the Data Center infrastructure, the integration of security into the infrastructure design, and the performance of Data Center devices:

Chapter 20, Designing the Data Center Infrastructure, discusses router switching paths, essential Data Center design concepts, and the design best practices of the infrastructure by explaining the conguration of Layer 2 and Layer 3 features and protocols that are described in Chapter 12 and 13.

Chapter 21, Integrating Security into the Infrastructure, discusses the concept of security zones and how to design application security at the Internet Edge and at intranet server farms. This chapter explains alternative designs and how to implement secure management.

Chapter 22, Performance Metrics of Data Center Devices, discusses the Data Center trafc patterns and per-formance metrics of various Data Center devices, including proposed metrics for devices for which there are none and no standard methodology exists (such as load balancers and SSL ofoaders).

Part VI, Appendixes, is the nal part of this book: Appendix A, Character Sets, covers multiple character sets, including ASCII, the extended ASCII sets, and

the ISO-8859-1 set.

Appendix B, HTTP Header Fields, explains the details of HTTP header elds that were not described in Chapter 8.

Appendix C, Video Encoding Mechanisms, explains the removal of special and temporal redundancy in codecs with special focus on MPEG.

Appendix D, Loopback Interface Conguration Procedures, provides an explanation about conguring a machine with multiple IP addresses used as loopbacks for certain load-balancing modes of operation.

DataCenter.book Page xxxviii Wednesday, November 12, 2003 9:52 AM

xxxix

Appendix E, Conguring Servers to Insert Cookies, examines several alternatives for conguring cookie insertion on web servers.

Appendix F, Client-Side and Server-Side Programming, provides excerpts of client-side programs to help you understand the differences and similarities between JavaScripts, Java applets, and ActiveX controls. The section on server-side programming explains the differences between CGI, servlets, and Active Server Pages (ASP) in terms of operating-system implications (threads versus processes). This appendix explains the adoption of certain technologies in todays enterprise applications and the performance and availability implications.

DataCenter.book Page xxxix Wednesday, November 12, 2003 9:52 AM

DataCenter.book Page 2 Wednesday, November 12, 2003 9:52 AM

PA R T IAn Introduction to Server FarmsChapter 1 Overview of Data Centers

Chapter 2 Server Architecture Overview

Chapter 3 Application Architecture Overview

Chapter 4 Data Center Design Overview

Chapter 5 Data Center Security Overview

Chapter 6 Server Load-Balancing Overview


This chapter covers the following topics:

Overview of the role of a Data Center in the enterprise Overview of the evolution of application environments The blueprint of the Data Center architecture The services provided by the Data Center network


C H A P T E R 1

Overview of Data CentersThis chapter presents an overview of enterprise Data Center environments, current application environment trends, the Data Center network architecture, and the services provided by the architecture. The approach to develop the architecture of the Data Center network is typically an internal process based on the requirement of the enterprise. This chapter provides the design criteria used by the authors to dene the Data Center design best practices presented throughout the book.

Data Centers DefinedData Centers house critical computing resources in controlled environments and under cen-tralized management, which enable enterprises to operate around the clock or according to their business needs. These computing resources include mainframes; web and application servers; le and print servers; messaging servers; application software and the operating systems that run them; storage subsystems; and the network infrastructure, whether IP or storage-area network (SAN). Applications range from internal nancial and human resources to external e-commerce and business-to-business applications. Additionally, a number of servers support network operations and network-based applications. Network operation applications include Network Time Protocol (NTP); TN3270; FTP; Domain Name System (DNS); Dynamic Host Conguration Protocol (DHCP); Simple Network Management Protocol (SNMP); TFTP; Network File System (NFS); and network-based applications, including IP telephony, video streaming over IP, IP video conferencing, and so on.

According to a report from the Renewable Energy Policy Project on Energy Smart Data Centers, Data Centers are . . . an essential component of the infrastructure supporting the Internet and the digital commerce and electronic communication sector. Continued growth of these sectors requires a reliable infrastructure because . . . interruptions in digital services can have signicant economic consequences.

Virtually, every enterprise has one or more Data Centers. Some have evolved rapidly to accommodate various enterprise application environments using distinct operating systems and hardware platforms. The evolution has resulted in complex and disparate environments that are expensive to manage and maintain. In addition to the application environment, the


6 Chapter 1: Overview of Data Centers

supporting network infrastructure might not have changed fast enough to be exible in accommodating ongoing redundancy, scalability, security, and management requirements.

A Data Center network design lacking in any of these areas risks not being able to sustain the expected service level agreements (SLAs). Data Center downtime, service degradation, or the inability to roll new services implies that SLAs are not met, which leads to a loss of access to critical resources and a quantiable impact on normal business operation. The impact could be as simple as increased response time or as severe as loss of data.

Data Center GoalsThe benets provided by a Data Center include traditional business-oriented goals such as the support for business operations around the clock (resiliency), lowering the total cost of operation and the maintenance needed to sustain the business functions (total cost of ownership), and the rapid deployment of applications and consolidation of computing resources (exibility).These business goals generate a number of information technology (IT) initiatives, including the following:

Business continuance Increased security in the Data Center Application, server, and Data Center consolidation Integration of applications whether client/server and multitier (n-tier), or web

services-related applications Storage consolidation

These IT initiatives are a combination of the need to address short-term problems and establishing a long-term strategic direction, all of which require an architectural approach to avoid unnecessary instability if the Data Center network is not exible enough to accommodate future changes. The design criteria are

Availability Scalability Security Performance Manageability

These design criteria are applied to these distinct functional areas of a Data Center network:

Infrastructure servicesRouting, switching, and server-farm architecture Application servicesLoad balancing, Secure Socket Layer (SSL) ofoading, and

caching


Data Centers Defined 7

Security servicesPacket ltering and inspection, intrusion detection, and intrusion prevention

Storage servicesSAN architecture, Fibre Channel switching, backup, and archival Business continuanceSAN extension, site selection, and Data Center interconnectivity

The details of these services are discussed later in this chapter.

Data Center FacilitiesBecause Data Centers house critical computing resources, enterprises must make special arrangements with respect to both the facilities that house the equipment and the personnel required for a 24-by-7 operation. These facilities are likely to support a high concentration of server resources and network infrastructure. The demands posed by these resources, coupled with the business criticality of the applications, create the need to address the following areas:

Power capacity Cooling capacity Cabling Temperature and humidity controls Fire and smoke systems Physical security: restricted access and surveillance systems Rack space and raised oors

Discussing the facilities where the Data Center resides and the related planning functions is outside the scope of this book.

The sections that follow introduce the role of the Data Center in the enterprise network.

Roles of Data Centers in the EnterpriseFigure 1-1 presents the different building blocks used in the typical enterprise network and illustrates the location of the Data Center within that architecture.

The building blocks of this typical enterprise network include

Campus network Private WAN Remote access Internet server farm Extranet server farm Intranet server farm



Figure 1-1 Data Centers in the Enterprise

Data Centers typically house many components that support the infrastructure building blocks, such as the core switches of the campus network or the edge routers of the private WAN. Data Center designs can include any or all of the building blocks in Figure 1-1, including any or all server farm types. Each type of server farm can be a separate physical entity, depending on the business requirements of the enterprise. For example, a company might build a single Data Center and share all resources, such as servers, rewalls, routers, switches, and so on. Another company might require that the three server farms be physi-cally separated with no shared equipment. This book focuses on the details of architecting server farms in the context of a highly available and scalable Data Center. These server farms support a wide number of enterprise applications.

Enterprise applications typically focus on one of the following major business areas: Customer relationship management (CRM) Enterprise resource planning (ERP)

Internet

SP1

VPN & Private WAN

Campus

Remote Access

AAA

RPMSCore Switches

DMZInternet Server Farm

Extranet Server Farm

VPN

Data Center

Intranet Server Farm

SP2PSTN Partners


Application Architecture Models 9

Supply chain management (SCM) Sales force automation (SFA) Order processing E-commerce

Roles of Data Centers in the Service Provider EnvironmentData Centers in service provider (SP) environments, known as Internet Data Centers (IDCs), unlike in enterprise environments, are the source of revenue that supports collocated server farms for enterprise customers. The SP Data Center is a service-oriented environment built to house, or host, an enterprise customers application environment under tightly controlled SLAs for uptime and availability. Enterprises also build IDCs when the sole reason for the Data Center is to support Internet-facing applications.

The IDCs are separated from the SP internal Data Centers that support the internal business applications environments.

Whether built for internal facing or collocated applications, application environments fol-low specic application architectural models such as the classic client/server or the n-tier model.

Application Architecture ModelsApplication architectures are constantly evolving, adapting to new requirements, and using new technologies. The most pervasive models are the client/server and n-tier models that refer to how applications use the functional elements of communication exchange. The client/server model, in fact, has evolved to the n-tier model, which most enterprise software application vendors currently use in application architectures. This section introduces both models and the evolutionary steps from client/server to the n-tier model.

The Client/Server Model and Its EvolutionThe classic client/server model describes the communication between an application and a user through the use of a server and a client. The classic client/server model consists of the following:

A thick client that provides a graphical user interface (GUI) on top of an application or business logic where some processing occurs

A server where the remaining business logic resides Thick client is an expression referring to the complexity of the business logic (software) required on the client side and the necessary hardware to support it. A thick client is then a portion of the application code running at the clients computer that has the responsibility



of retrieving data from the server and presenting it to the client. The thick client code requires a fair amount of processing capacity and resources to run in addition to the management overhead caused by loading and maintaining it on the client base.

The server side is a single server running the presentation, application, and database code that uses multiple internal processes to communicate information across these distinct functions. The exchange of information between client and server is mostly data because the thick client performs local presentation functions so that the end user can interact with the application using a local user interface.

Client/server applications are still widely used, yet the client and server use proprietary interfaces and message formats that different applications cannot easily share. Part a of Figure 1-2 shows the client/server model.

Figure 1-2 Client/Server and n-Tier Application Interaction

The most fundamental changes to the thick client and single-server model started when web-based applications rst appeared. Web-based applications rely on more standard interfaces and message formats where applications are easier to share. HTML and HTTP provide a standard framework that allows generic clients such as web browsers to commu-nicate with generic applications as long as they use web servers for the presentation func-tion. HTML describes how the client should render the data; HTTP is the transport protocol used to carry HTML data. Netscape Communicator and Microsoft Internet Explorer are examples of clients (web browsers); Apache, Netscape Enterprise Server, and Microsoft Internet Information Server (IIS) are examples of web servers.The migration from the classic client/server to a web-based architecture implies the use of thin clients (web browsers), web servers, application servers, and database servers. The web browser interacts with web servers and application servers, and the web servers interact with application servers and database servers. These distinct functions supported by the servers are referred to as tiers, which, in addition to the client tier, refer to the n-tier model.

Application Server

Thick Client

Application GUI

Database Server

Web Server

ApplicationServer

DatabaseServer

Thick Client

Web Browser

a b


Application Architecture Models 11

The n-Tier ModelPart b of Figure 1-2 shows the n-tier model. Figure 1-2 presents the evolution from the classic client/server model to the n-tier model. The client/server model uses the thick client with its own business logic and GUI to interact with a server that provides the counterpart business logic and database functions on the same physical device. The n-tier model uses a thin client and a web browser to access the data in many different ways. The server side of the n-tier model is divided into distinct functional areas that include the web, application, and database servers.

The n-tier model relies on a standard web architecture where the web browser formats and presents the information received from the web server. The server side in the web architec-ture consists of multiple and distinct servers that are functionally separate. The n-tier model can be the client and a web server; or the client, the web server, and an application server; or the client, web, application, and database servers. This model is more scalable and man-ageable, and even though it is more complex than the classic client/server model, it enables application environments to evolve toward distributed computing environments.

The n-tier model marks a signicant step in the evolution of distributed computing from the classic client/server model. The n-tier model provides a mechanism to increase perfor-mance and maintainability of client/server applications while the control and management of application code is simplied.

Figure 1-3 introduces the n-tier model and maps each tier to a partial list of currently available technologies at each tier.

Figure 1-3 n-Tier Model

Notice that the client-facing servers provide the interface to access the business logic at the application tier. Although some applications provide a non-webbased front end, current trends indicate the process of web-transforming business applications is well underway.

CRM ERP SCM SFA OrderProcessingE-

Commerce

Apache IIS Netscape NCSA Other

Java, ASP, J2EE, Java Scripting, Application Code

Sybase Oracle SQL Server DB2

Hitachi EMC Compaq IBM

n-Tier

Application Areas

Web andOther Servers

Business Logic

Database Systems

Storage



This process implies that the front end relies on a web-based interface to face the users which interacts with a middle layer of applications that obtain data from the back-end systems.

These middle tier applications and the back-end database systems are distinct pieces of logic that perform specic functions. The logical separation of front-end application and back-end functions has enabled their physical separation. The implications are that the web and application servers, as well as application and database servers, no longer have to coexist in the same physical server. This separation increases the scalability of the services and eases the management of large-scale server farms. From a network perspective, these groups of servers performing distinct functions could also be physically separated into different network segments for security and manageability reasons.

Chapter 3, Application Architectures Overview, discusses the details on applications that follow the n-tier model and the implications on the design of the Data Center.

Multitier Architecture Application EnvironmentMultitier architectures refer to the Data Center server farms supporting applications that provide a logical and physical separation between various application functions, such as web, application, and database (n-tier model). The network architecture is then dictated by the requirements of applications in use and their specic availability, scalability, and secu-rity and management goals. For each server-side tier, there is a one-to-one mapping to a net-work segment that supports the specic application function and its requirements. Because the resulting network segments are closely aligned with the tiered applications, they are described in reference to the different application tiers.

Figure 1-4 presents the mapping from the n-tier model to the supporting network segments used in a multitier design.

Figure 1-4 Multitier Network Segments

The web server tier is mapped to the front-end segment, the business logic to the application segment, and the database tier to the back-end segment. Notice that all the segments supporting the server farm connect to access layer switches, which in a multitier architec-ture are different access switches supporting the various server functions.

Database Systems

Business Logic

Web and Other Servers

n-Tiers

Front End

Application

Back End

Network Segments

Access Layer


Data Center Architecture 13

The evolution of application architectures and departing from multitier application environ-ments still requires a network to support the interaction between the communicating entities. For example, a web service (dened as A web service is a software system designed to support interoperable machine-to-machine interaction over a network by the W3C web services architecture document) still refers to the network element. In this case, the network would be used for networked resources that support such interaction realiably. This layer of abstraction does not necesarily translate on to a layered network design as much as the capability of the network to support networked applications, resources, and their interaction.

The following section presents a high-level overview of the distinct network layers of the Data Center architecture.

Data Center ArchitectureThe enterprise Data Center architecture is inclusive of many functional areas, as presented earlier in Figure 1-1. The focus of this section is the architecture of a generic enterprise Data Center connected to the Internet and supporting an intranet server farm. Other types of server farms, explained in Chapter 4, Data Center Design Overview, follow the same architec-ture used for intranet server farms yet with different scalability, security, and management requirements. Figure 1-5 introduces the topology of the Data Center architecture.

Figure 1-5 Topology of an Enterprise Data Center Architecture

Primary Data Center

Server

Farms

Application

Front End

Back End

Data Center TransportFCStorage

Distributed Data Center

Campus Core

Aggregation

Internet Edge

Access

Service Provider A Service Provider B

Internet



Figure 1-5 shows a fully redundant enterprise Data Center supporting the following areas:

No single point of failureredundant components Redundant Data Centers

Although the focus of this book is the architecture of an IP network that supports server farms, we include explanations pertaining to how the server farms are connected to the rest of the enterprise network for the sake of clarity and thoroughness. The core connectivity functions supported by Data Centers are Internet Edge connectivity, campus connectivity, and server-farm connectivity, as presented in Figure 1-5.

The Internet Edge provides the connectivity from the enterprise to the Internet and its associated redundancy and security functions, such as the following:

Redundant connections to different service providers External and internal routing through exterior border gateway protocol (EBGP) and

interior border gateway protocol (IBGP) Edge security to control access from the Internet Control for access to the Internet from the enterprise clients

The campus core switches provide connectivity between the Internet Edge, the intranet server farms, the campus network, and the private WAN. The core switches physically connect to the devices that provide access to other major network areas, such as the private WAN edge routers, the server-farm aggregation switches, and campus distribution switches.

As depicted in Figure 1-6, the following are the network layers of the server farm:

Aggregation layer Access layer

Front-end segment Application segment Back-end segment

Storage layer Data Center transport layer

Some of these layers depend on the specic implementation of the n-tier model or the requirements for Data Center-to-Data-Center connectivity, which implies that they might not exist in every Data Center implementation. Although some of these layers might be optional in the Data Center architecture, they represent the trend in continuing to build highly available and scalable enterprise Data Centers. This trend specically applies to the storage and Data Center transport layers supporting storage consolidation, backup and archival consolidation, high-speed mirroring or clustering between remote server farms, and so on.



The sections that follow present the specic details of each layer.

Aggregation LayerThe aggregation layer is the aggregation point for devices that provide services to all server farms. These devices are multilayer switches, rewalls, load balancers, and other devices that typically support services across all servers. The multilayer switches are referred to as aggregation switches because of the aggregation function they perform. Service devices are shared by all server farms. Specic server farms are likely to span multiple access switches for redundancy, thus making the aggregation switches the logical connection point for service devices, instead of the access switches.

If connected to the front-end Layer 2 switches, these service devices might not offer optimal services by creating less than optimal trafc paths between them and servers connected to different front-end switches. Additionally, if the service devices are off of the aggregation switches, the trafc paths are deterministic and predictable and simpler to manage and maintain. Figure 1-6 shows the typical devices at the aggregation layer.

Figure 1-6 Aggregation and Access Layers

As depicted in Figure 1-6, the aggregation switches provide basic infrastructure services and connectivity for other service devices. The aggregation layer is analogous to the traditional distribution layer in the campus network in its Layer 3 and Layer 2 functionality.

The aggregation switches support the traditional switching of packets at Layer 3 and Layer 2 in addition to the protocols and features to support Layer 3 and Layer 2 connectivity. A more in-depth explanation on the specic services provided by the aggregation layer appears in the section, Data Center Services.

AggregationMultilayer Switches: L2 L3 FirewallsCachesLoad BalancersSSL Offloaders Intrusion Detection Systems

AccessLayer 2 SwitchesIDS & Host IDS Web and Client Facing Servers

Internet EdgePrivate WAN

Campus Core

Layer 3

Layer 2



Access LayerThe access layer provides Layer 2 connectivity and Layer 2 features to the server farm. Because in a multitier server farm, each server function could be located on different access switches on different segments, the following section explains the details of each segment.

Front-End SegmentThe front-end segment consists of Layer 2 switches, security devices or features, and the front-end server farms. See the section, Data Center Services for a detailed description of the features provided by the devices at this layer. The front-end segment is analogous to the traditional access layer of the hierarchical campus network design and provides the same functionality. The access switches are connected to the aggregation switches in the manner depicted in Figure 1-6. The front-end server farms typically include FTP, Telnet, TN3270 (mainframe terminals), Simple Mail Transfer Protocol (SMTP), web servers, DNS servers, and other business application servers, in addition to network-based application servers such as IP television (IPTV) broadcast servers and IP telephony call managers that are not placed at the aggregation layer because of port density or other design requirements.

The specic network features required in the front-end segment depend on the servers and their functions. For example, if a network supports video streaming over IP, it might require multicast, or if it supports Voice over IP (VoIP), quality of service (QoS) must be enabled. Layer 2 connectivity through VLANs is required between servers and load balancers or rewalls that segregate server farms.

The need for Layer 2 adjacency is the result of Network Address Translation (NAT) and other header rewrite functions performed by load balancers or rewalls on trafc destined to the server farm. The return trafc must be processed by the same device that performed the header rewrite operations.

Layer 2 connectivity is also required between servers that use clustering for high availabil-ity or require communicating on the same subnet. This requirement implies that multiple access switches supporting front-end servers can support the same set of VLANs to provide layer adjacency between them. Security features include Address Resolution Protocol (ARP) inspection, broadcast suppression, private VLANs, and others that are enabled to counteract Layer 2 attacks. Security devices include network-based intrusion detection systems (IDSs) and host-based IDSs to monitor and detect intruders and prevent vulnerabilities from being exploited. In general, the infrastructure components such as the Layer 2 switches provide intelligent network services that enable front-end servers to provide their functions.

Note that the front-end servers are typically taxed in their I/O and CPU capabilities. For I/O, this strain is a direct result of serving content to the end users; for CPU, it is the connection rate and the number of concurrent connections needed to be processed.



Scaling mechanisms for front-end servers typically include adding more servers with iden-tical content and then equally distributing the load they receive using load balancers. Load balancers distribute the load (or load balance) based on Layer 4 or Layer 5 information. Layer 4 is widely used for front-end servers to sustain a high connection rate without nec-essarily overwhelming the servers. See Chapter 22, Performance Metrics of Data Center Devices, to understand the performance of servers and load balancers under load.

Scaling mechanisms for web servers also include the use of SSL ofoaders and Reverse Proxy Caching (RPC). Refer to Chapter 9, SSL and TLS, for more information about the use of SSL and its performance implications.

Application SegmentThe application segment has the same network infrastructure components as the front-end segment and the application servers. The features required by the application segment are almost identical to those needed in the front-end segment, albeit with additional security. This segment relies strictly on Layer 2 connectivity, yet the additional security is a direct requirement of how much protection the application servers need because they have direct access to the database systems. Depending on the security policies, this segment uses rewalls between web and application servers, IDSs, and host IDSs. Like the front-end segment, the application segment infrastructure must support intelligent network services as a direct result of the functions provided by the application services.

Application servers run a portion of the software used by business applications and provide the communication logic between the front end and the back end, which is typically referred to as the middleware or business logic. Application servers translate user requests to commands that the back-end database systems understand. Increasing the security at this segment focuses on controlling the protocols used between the front-end servers and the application servers to avoid trust exploitation and attacks that exploit known application vulnerabilities. Figure 1-7 introduces the front-end, application, and back-end segments in a logical topology.

Note that the application servers are typically CPU-stressed because they need to support the business logic. Scaling mechanisms for application servers also include load balancers. Load balancers can select the right application server based on Layer 5 information.

Deep packet inspection on load balancers allows the partitioning of application server farms by content. Some server farms could be dedicated to selecting a server farm based on the scripting language (.cgi, .jsp, and so on). This arrangement allows application adminis-trators to control and manage the server behavior more efciently.



Figure 1-7 Access Layer Segments

Back-End SegmentThe back-end segment is the same as the previous two segments except that it supports the connectivity to database servers. The back-end segment features are almost identical to those at the application segment, yet the security considerations are more stringent and aim at protecting the data, critical or not.

The hardware supporting the database systems ranges from medium-sized servers to high-end servers, some with direct locally attached storage and others using disk arrays attached to a SAN. When the storage is separated, the database server is connected to both the Ethernet switch and the SAN. The connection to the SAN is through a Fibre Channel interface. Figure 1-8 presents the back-end segment in reference to the storage layer. Notice the connections from the database server to the back-end segment and storage layer.

Note that in other connectivity alternatives, the security requirements do not call for physical separation between the different server tiers. These alternatives are discussed in Chapter 4.

FirewallsLayer 2 SwitchesIntrusion Detection SystemsApplication Servers

FirewallsLayer 2 SwitchesIntrusion Detection SystemsDatabase Servers

Back End

Application

Layer 2 SwitchesIDS and Host IDS Web and Client Facing Servers

Front End

Access Layer

Layer 2

Layer 2

AggregationLayer



Storage LayerThe storage layer consists of the storage infrastructure such as Fibre Channel switches and routers that support small computer system interface (SCSI) over IP (iSCSI) or Fibre Channel over IP (FCIP). Storage network devices provide the connectivity to servers, storage devices such as disk subsystems, and tape subsystems.

NOTE SAN environments in Data Centers commonly use Fibre Channel to connect servers to the storage device and to transmit SCSI commands between them. Storage networks allow the transport of SCSI commands over the network. This trans

data center fundamentals

Documents