Pro MySQL MICHAEL KRUCKENBERG AND JAY PIPES 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page i

Pro MySQL Copyright 2005 by Michael Kruckenberg and Jay Pipes All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN (pbk): 1-59059-505-X Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editors: Jason Gilmore, Matthew Moodie Technical Reviewer: Chad Russell Editorial Board: Steve Anglin, Dan Appleman, Ewan Buckingham, Gary Cornell, Tony Davis, Jason Gilmore, Jonathan Hassell, Chris Mills, Dominic Shakeshaft, Jim Sumser Associate Publisher: Grace Wong Project Manager: Kylie Johnston Copy Edit Manager: Nicole LeClerc Copy Editors: Marilyn Smith, Susannah Pfalzer Assistant Production Director: Kari Brooks-Copony Production Editor: Linda Marousek Compositor, Artist, and Interior Designer: Diana Van Winkle, Van Winkle Design Group Proofreader: Patrick Vincent, Write Ideas Editorial Consulting Indexer: Ann Rogers Cover Designer: Kurt Krames Manufacturing Manager: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-ny@springer-sbm.com, or visit http://www.springeronline.com. For information on translations, please contact Apress directly at 2560 Ninth Street, Suite 219, Berkeley, CA 94710. Phone 510-549-5930, fax 510-549-5939, e-mail info@apress.com, or visit http://www.apress.com. The information in this book is distributed on an as is basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. The source code for this book is available to readers at http://www.apress.com in the Downloads section. 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page ii

Contents at a Glance Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii PART 1 Design and Development CHAPTER 1 Analyzing Business Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 CHAPTER 2 Index Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 CHAPTER 3 Transaction Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 CHAPTER 4 MySQL System Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 CHAPTER 5 Storage Engines and Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 CHAPTER 6 Benchmarking and Profiling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 CHAPTER 7 Essential SQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 CHAPTER 8 SQL Scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 CHAPTER 9 Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 CHAPTER 10 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 CHAPTER 11 Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 CHAPTER 12 Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 CHAPTER 13 Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 PART 2 Administration CHAPTER 14 MySQL Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 469 CHAPTER 15 User Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 CHAPTER 16 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533 CHAPTER 17 Backup and Restoration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 CHAPTER 18 Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585 CHAPTER 19 Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 CHAPTER 20 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645 CHAPTER 21 MySQL Data Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669 INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699 iii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page iii

505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page iv

Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii PART 1 Design and Development CHAPTER 1 Analyzing Business Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . 3 The Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Common Team Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Importance of Team Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 From Concept to Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Textual Object Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Modeling Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 A Database Blueprint (Baseline Model) . . . . . . . . . . . . . . . . . . . . . . . . 30 Database Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Surveying the Landscape. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Why Choose MySQL? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Your Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 On Hosting Companies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Commercial Web Software Development . . . . . . . . . . . . . . . . . . . . . . 35 On Controlled Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 CHAPTER 2 Index Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Data Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 The Hard Disk: Persistent Data Storage. . . . . . . . . . . . . . . . . . . . . . . . 40 Memory: Volatile Data Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Pages: Logical Data Representation. . . . . . . . . . . . . . . . . . . . . . . . . . . 42 How Indexes Affect Data Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Computational Complexity and the Big O Notation. . . . . . . . . . . . . 44 Data Retrieval Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Analysis of Index Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 v 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page v

Clustered vs. Non-Clustered Data and Index Organization . . . . . . . . . . . . 55 Index Layouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 The B-Tree Index Layout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 The R-Tree Index Layout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 The Hash Index Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 The FULLTEXT Index Layout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 General Index Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Clustering Key Selection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Query Structuring to Ensure Use of an Index . . . . . . . . . . . . . . . . . . . 66 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 CHAPTER 3 Transaction Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Transaction Processing Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Transaction Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 The ACID Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Ensuring Atomicity, Consistency, and Durability. . . . . . . . . . . . . . . . . . . . . . 76 The Transaction Wrapper and Demarcation . . . . . . . . . . . . . . . . . . . . 76 MySQLs Autocommit Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Checkpointing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Implementing Isolation and Concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Locking Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Isolation Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Locking and Isolation Levels in MySQL: Some Examples. . . . . . . . . 92 Multiversion Concurrency Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Identifying Your Transaction Control Requirements . . . . . . . . . . . . . . . . . . 102 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 CHAPTER 4 MySQL System Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 The MySQL Source Code and Documentation . . . . . . . . . . . . . . . . . . . . . . 106 The Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 The MySQL Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 TEXI and texi2html Viewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 MySQL Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 MySQL Server Subsystem Organization . . . . . . . . . . . . . . . . . . . . . . 112 Base Function Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 CONTENTSvi 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page vi

Process, Thread, and Resource Management . . . . . . . . . . . . . . . . . . . . . . 114 Thread-Based vs. Process-Based Design . . . . . . . . . . . . . . . . . . . . . 114 Implementation Through a Library of Related Functions . . . . . . . . 115 User Connection Threads and THD Objects. . . . . . . . . . . . . . . . . . . . 117 Storage Engine Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Key Classes and Files for Handlers. . . . . . . . . . . . . . . . . . . . . . . . . . . 118 The Handler API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Caching and Memory Management Subsystem . . . . . . . . . . . . . . . . . . . . 121 Record Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Key Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Table Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Hostname Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Privilege Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Other Caches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Network Management and Communication . . . . . . . . . . . . . . . . . . . . . . . . 128 Access and Grant Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Log Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Query Parsing, Optimization, and Execution. . . . . . . . . . . . . . . . . . . . . . . . 135 Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Execution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 The Query Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 A Typical Query Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 CHAPTER 5 Storage Engines and Data Types. . . . . . . . . . . . . . . . . . . . . . . . . . 153 Storage Engine Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 The MyISAM Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 MyISAM File and Directory Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 MyISAM Record Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 The .MYI File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 MyISAM Table-Level Locking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 MyISAM Index Choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 MyISAM Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 The InnoDB Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Enforcement of Foreign Key Relationships . . . . . . . . . . . . . . . . . . . . 164 InnoDB Row-Level Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 ACID-Compliant Multistatement Transaction Control. . . . . . . . . . . . 166 The InnoDB File and Directory Layout . . . . . . . . . . . . . . . . . . . . . . . . 166 CONTENTS vii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page vii

InnoDB Data Page Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Internal InnoDB Buffers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 InnoDB Doublewrite Buffer and Log Format . . . . . . . . . . . . . . . . . . . 170 The Checkpointing and Recovery Processes . . . . . . . . . . . . . . . . . . 172 Other Storage Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 The MERGE Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 The MEMORY Storage Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 The ARCHIVE Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 The CSV Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 The FEDERATED Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 The NDB Cluster Storage Engine. . . . . . . . . . . . . . . . . . . . . . . . . 178 Guidelines for Choosing a Storage Engine . . . . . . . . . . . . . . . . . . . . . . . . . 178 Data Type Choices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Numeric Data Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 String Data Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Temporal Data Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Spatial Data Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 SET and ENUM Data Considerations . . . . . . . . . . . . . . . . . . . . . . . . . 184 Boolean Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Some General Data Type Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . 185 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 CHAPTER 6 Benchmarking and Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 What Can Benchmarking Do for You? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Conducting Simple Performance Comparisons . . . . . . . . . . . . . . . . 191 Determining Load Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Testing an Applications Ability to Deal with Change . . . . . . . . . . . . 192 Finding Potential Problem Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 General Benchmarking Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Setting Real Performance Standards . . . . . . . . . . . . . . . . . . . . . . . . . 193 Being Proactive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Isolating Changed Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Using Real Data Sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Making Small Changes and Rerunning Benchmarks . . . . . . . . . . . 197 Turning Off Unnecessary Programs and the Query Cache . . . . . . . 197 Repeating Tests to Determine Averages . . . . . . . . . . . . . . . . . . . . . . 197 Saving Benchmark Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Benchmarking Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 MySQLs Benchmarking Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 MySQL Super Smack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 CONTENTSviii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page viii

MyBench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 ApacheBench (ab) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 httperf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 What Can Profiling Do for You? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 General Profiling Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Profiling Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 The SHOW FULL PROCESSLIST Command . . . . . . . . . . . . . . . . . . . . 219 The SHOW STATUS Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 The EXPLAIN Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 The Slow Query Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 The General Query Log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Mytop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 The Zend Advanced PHP Debugger Extension . . . . . . . . . . . . . . . . . 229 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 CHAPTER 7 Essential SQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 SQL Style. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Theta Style vs. ANSI Style. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Code Formatting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Specific and Consistent Coding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 MySQL Joins. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 The Inner Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 The Outer Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 The Cross Join (Cartesian Product). . . . . . . . . . . . . . . . . . . . . . . . . . . 253 The Union Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 The Natural Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 The USING Keyword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 EXPLAIN and Access Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 The const Access Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 The eq_ref Access Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 The ref Access Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 The ref_or_null Access Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 The index_merge Access Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 The unique_subquery Access Type . . . . . . . . . . . . . . . . . . . . . . . . . . 267 The index_subquery Access Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 The range Access Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 The index Access Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 The ALL Access Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 Join Hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 The STRAIGHT_JOIN Hint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 CONTENTS ix 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page ix

The USE INDEX and FORCE INDEX Hints . . . . . . . . . . . . . . . . . . . . . . 277 The IGNORE INDEX Hint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Subqueries and Derived Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Derived Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 CHAPTER 8 SQL Scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Handling OR Conditions Prior to MySQL 5.0. . . . . . . . . . . . . . . . . . . . . . . . 300 Dealing with Duplicate Entries and Orphaned Records . . . . . . . . . . . . . . 303 Identifying and Removing Duplicate Entries . . . . . . . . . . . . . . . . . . . 304 Identifying and Removing Orphaned Records . . . . . . . . . . . . . . . 307 Dealing with Hierarchical Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 Understanding the Nested Set Model. . . . . . . . . . . . . . . . . . . . . . . . . 313 Finding the Depth of a Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Finding All Nodes Under a Specific Parent. . . . . . . . . . . . . . . . . . . . . 317 Finding All Nodes Above a Specific Node. . . . . . . . . . . . . . . . . . . . . . 317 Summarizing Across the Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 Inserting a Node into the Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 Removing a Node from the Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 Retrieving Random Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 Calculating Distances with Geographic Coordinate Data . . . . . . . . . . . . . 328 Understanding the Distance Calculation Formula . . . . . . . . . . . . . . 329 Calculating the Distance Between Two Points . . . . . . . . . . . . . . . . . 331 Determining Zip Codes Within a Given Radius . . . . . . . . . . . . . . . . . 334 Generating Running Sums and Averages . . . . . . . . . . . . . . . . . . . . . . . . . . 344 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 CHAPTER 9 Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Stored Procedure Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 The Debate Over Using Stored Procedures. . . . . . . . . . . . . . . . . . . . 350 Other Considerations in Using Stored Procedures . . . . . . . . . . . . . . 352 Stored Procedures in MySQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 Building Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 The CREATE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 The Procedure Body . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 Using Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 Calling Procedures from the MySQL Client . . . . . . . . . . . . . . . . . . . . 366 Calling Procedures from PHP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 CONTENTSx 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page x

Managing Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 Viewing Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 Altering and Removing Stored Procedures . . . . . . . . . . . . . . . . . . . . 371 Editing Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 Stored Procedure Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 CHAPTER 10 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Database Function Uses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 Functions Compared with Other Database Tools . . . . . . . . . . . . . . . . . . . . 377 Stored Functions vs. Stored Procedures . . . . . . . . . . . . . . . . . . . . . . 377 Functions vs. Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 Functions vs. Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 Functions in MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 Creating Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 CREATE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Function Name. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Input Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Return Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 The Function Body. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 Using Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 Managing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 Viewing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 Changing and Removing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 401 Function Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 Performance of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 CHAPTER 11 Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Database Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Server and Client Cursors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 Cursor Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Data Behind the Cursor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Read and Write Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Cursors in MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Creating Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 DECLARE Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 OPEN Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 CONTENTS xi 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xi

FETCH Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 CLOSE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Using Cursors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 CHAPTER 12 Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Database View Uses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Views in MySQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 Creating Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 The CREATE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Algorithm Attributes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 View Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Column Names. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 The SELECT Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 Check Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Creating Updatable Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Defining Views of Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 Managing Views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436 Displaying Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436 Changing Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438 Removing Views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 View Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 Performance of Views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 Running Performance Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441 Using EXPLAIN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 CHAPTER 13 Triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 Database Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 The Debate Over Using Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 Trigger Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 Trigger Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Triggers in MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Creating MySQL Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 The CREATE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 Trigger Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 Activation Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454 Event for Activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454 CONTENTSxii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xii

Table to Activate the Trigger. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Trigger Body Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Using Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456 Managing Triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 Viewing Triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 Modifying and Removing Triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 Trigger Permissions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463 Trigger Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 PART 2 Administration CHAPTER 14 MySQL Installation and Configuration. . . . . . . . . . . . . . . . . . . . 469 Using an Existing Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470 Installing Prebuilt Binaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470 Supported Operating Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 MySQL Build Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472 Manual Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 Windows Installer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 RPMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 Mac OS X Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476 Building from Source or the Development Source Tree . . . . . . . . . . . . . . 477 Manually Installing the Source Tarball on Unix . . . . . . . . . . . . . . . . . 477 Installing from the Development Source Tree on Unix. . . . . . . . . . . 481 Starting and Stopping MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Unix Startup and Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Windows Startup and Shutdown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Mac OS X Startup and Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484 Performing Post-Installation Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484 Configuring MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 Location of Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 Configuration Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486 Configuration Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487 Upgrading MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 Uninstalling MySQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494 Running Multiple Database Servers on a Single Machine . . . . . . . . . . . . 495 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 CONTENTS xiii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xiii

CHAPTER 15 User Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 MySQL Privileges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 Granting and Revoking Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 Understanding Privilege Scope Levels . . . . . . . . . . . . . . . . . . . . . . . . 498 Granting All Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 Viewing User Privileges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 How MySQL Controls Access and Verifies Privileges. . . . . . . . . . . . . . . . . 510 How MySQL Authenticates Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 How MySQL Verifies User Privileges. . . . . . . . . . . . . . . . . . . . . . . . . . 513 The Purpose of the host Grant Table. . . . . . . . . . . . . . . . . . . . . . . . . . 515 Managing User Accounts from the Command Line. . . . . . . . . . . . . . . . . . 516 Adding User Accounts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516 Restricting User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517 Removing Accounts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518 Effecting Account Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518 Using the MySQL Administrator GUI Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . 519 Connecting to the Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520 Navigating User Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521 Adding a New User Account. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524 Viewing and Editing User Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . 526 Removing an Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528 Thinking in Terms of User Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 Practical Guidelines for User Administration. . . . . . . . . . . . . . . . . . . . . . . . 531 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531 CHAPTER 16 Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533 Understanding Security Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 Why Arent We Secure? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 Wheres the Security Threat?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 What Does It Mean to Be Secure? . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 Building a Security Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536 Using Security Incidents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 Keeping Good Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 Getting Started: A MySQL Security Quick List . . . . . . . . . . . . . . . . . . . . . . 540 Setting Up Database Security: An Example. . . . . . . . . . . . . . . . . . . . . . . . . 541 Locking Up the Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541 Locking Down Network Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 Controlling Access to Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 CONTENTSxiv 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xiv

Adding Remote Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543 Adding Servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544 Securing Your Whole System: Seven Major Areas . . . . . . . . . . . . . . . . . . . 545 Physical Access to the Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546 Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 Files, Directories, and Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 Network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548 User Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549 Application Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 Data Storage and Encryption. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 CHAPTER 17 Backup and Restoration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 Why Do We Create Backups? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 Creating a Backup and Restore Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 Developing Your Backup Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557 Developing Your Restore Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558 Backing Up a Database: A Quick Example . . . . . . . . . . . . . . . . . . . . . . . . . 559 Using MySQL Tools to Make and Restore Backups . . . . . . . . . . . . . . . . . . 560 Using mysqldump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 Using mysqlhotcopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569 Creating Backups of InnoDB Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 Using MySQL Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577 Using Binary Logs for Up-to-Date Tables . . . . . . . . . . . . . . . . . . . . . . . . . . 581 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583 CHAPTER 18 Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585 What Is Replication? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586 Synchronous vs. Asynchronous Replication . . . . . . . . . . . . . . . . . . . 587 One-Way vs. Merge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588 Why Replicate Data?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589 Geographic Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590 Limited Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590 Redundancy and Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590 Storage Engine and Index Optimization. . . . . . . . . . . . . . . . . . . . . . . 590 What Isnt Solved with Replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 CONTENTS xv 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xv

Planning for Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 How MySQL Implements Replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592 Binary Log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592 Replication Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 Relay Log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 info Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595 master.info File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595 Initial Replication Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597 Adjust Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597 Create Replication Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598 Schema and Data Snapshot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598 Start Replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 600 Configuration Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601 Core Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601 Other Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602 How Does MySQL Decide What to Replicate? . . . . . . . . . . . . . . . . . 605 Monitoring and Managing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606 SHOW MASTER STATUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606 SHOW SLAVE HOSTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606 SHOW SLAVE STATUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607 CHANGE MASTER. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609 START SLAVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610 STOP SLAVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610 RESET SLAVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611 Replication Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611 Hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612 Network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612 Database Load . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612 Record Size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612 Amount of Data and Types of Queries . . . . . . . . . . . . . . . . . . . . . . . . 612 Replication Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 Simple Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 Multiple Slaves. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 Daisy Chain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614 Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 CONTENTSxvi 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xvi

CHAPTER 19 Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 What Is Clustering?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618 MySQLs Cluster Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619 Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620 Node Arrangement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621 Calculating the Number of Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 624 Using the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625 Limitations of MySQL Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625 Installing MySQL Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 Initial Configuration and Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 Management Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 Storage Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629 SQL Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629 Check Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 630 Cluster Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634 Management Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634 Storage Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635 Management Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636 Configuration File Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636 Management Server Configuration File . . . . . . . . . . . . . . . . . . . . . . . 636 Storage Node Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639 SQL Node Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639 Managing the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640 Management Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640 Single User Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641 Backing Up and Restoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641 Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Cluster Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Node Log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644 CHAPTER 20 Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645 Troubleshooting Toolkit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646 Error Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646 General Query Log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 648 Slow Query Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650 MySQL Server Thread List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650 Operating System Process List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652 Monitoring Programs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652 CONTENTS xvii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xvii

Commonly Encountered Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652 Troubleshooting User, Connection, and Client Issues . . . . . . . . . . . 653 Troubleshooting Start, Restart, and Shutdown Issues. . . . . . . . . . . 661 Resolving Data Corruption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666 How to Report MySQL Bugs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667 Support Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668 CHAPTER 21 MySQL Data Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669 Benefits of a Standardized Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670 Adherence to Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670 Using SELECT to Retrieve Metadata. . . . . . . . . . . . . . . . . . . . . . . . . . 670 More Information than SHOW Commands. . . . . . . . . . . . . . . . . . . . . 671 The INFORMATION_SCHEMA Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671 INFORMATION_SCHEMA.SCHEMATA . . . . . . . . . . . . . . . . . . . . . . . . . 673 INFORMATION_SCHEMA.TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674 INFORMATION_SCHEMA.TABLE_CONSTRAINTS . . . . . . . . . . . . . . . 676 INFORMATION_SCHEMA.COLUMNS . . . . . . . . . . . . . . . . . . . . . . . . . . 678 INFORMATION_SCHEMA.KEY_COLUMN_USAGE . . . . . . . . . . . . . . . 680 INFORMATION_SCHEMA.STATISTICS . . . . . . . . . . . . . . . . . . . . . . . . . 682 INFORMATION_SCHEMA.ROUTINES . . . . . . . . . . . . . . . . . . . . . . . . . . 684 INFORMATION_SCHEMA.VIEWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686 INFORMATION_SCHEMA.CHARACTER_SETS . . . . . . . . . . . . . . . . . . 687 INFORMATION_SCHEMA.COLLATIONS. . . . . . . . . . . . . . . . . . . . . . . . 688 INFORMATION_SCHEMA.COLLATION_CHARACTER_ SET_APPLICABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689 INFORMATION_SCHEMA.SCHEMA_PRIVILEGES. . . . . . . . . . . . . . . . 690 INFORMATION_SCHEMA.USER_PRIVILEGES. . . . . . . . . . . . . . . . . . . 691 INFORMATION_SCHEMA.TABLE_PRIVILEGES . . . . . . . . . . . . . . . . . . 692 INFORMATION_SCHEMA.COLUMN_PRIVILEGES. . . . . . . . . . . . . . . . 693 Usage Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694 Example 1: Gathering Selectivity Numbers on Indexes. . . . . . . . . . 694 Example 2: Summarizing Table Sizes by Engine . . . . . . . . . . . . . . . 697 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 698 INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699 CONTENTSxviii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xviii

xix Foreword Iget a lot of requests to review booksat least one a month. Ive made a point of telling pub- lishers that I will be happy to try, but then explain to them that I complete only every other request. I stay fairly busy working on MySQL and directing its development. If the book puts me to sleep in the first chapter, I know I will never make it through. Even on the best of occa- sions, I normally read only perhaps three-quarters of the book before I need to get back to coding. One reason I rarely finish books that I am reviewing is that I dont believe that technical books need to be dry. I think that any topic can be made interesting. I read very few technical books from cover to cover for this reason. To write a good technical book that someone will enjoy does not mean that you have to be funny. (I once read a JDBC book that made mention of different rock lyrics every other paragraph in an attempt to be hip; I obviously never com- pleted it.) I could never finish a book that covered the basics of MySQL and rehashed the material of the dozens of other MySQL books written. Now, writing a foreword for a book is even harder than reviewing a book for me. Not only do I have to read the book, but I know that I am so late in the review process that I cant point out factual errors. Lets start right here by giving some credit to the authors of this book. Point- ing out any errors in this book would just be me being nitpicky. This book covers the 5.0 release of the MySQL Server, and getting a book right on 5.0 is pretty hard, since we are at a code freeze for release right now; 5.0 is done and we are getting ready to ship. Software being what it is, that doesnt mean a few changes dont occur. Getting a book this close is an incredi- ble feat, and nothing you read here will lead you astray. MySQL use is continuing to grow. I first started hacking on MySQL in graduate school while I worked on medical projects. The database I was using at the time ran out of steam, and I needed something that was fast enough to store the data I was mining. I had MySQL up and running the same morning I found it, and had data being logged into it by that afternoon. Now, years later, Im one of the few dozen programmers who work on it full time. Success was not only due to the fact that the database was easy to install and free, but was because it was well- written software. MySQLs install base is about six million today, with forty thousand downloads a day occurring from our web site alone. This means, in the world of the Internet and Open Source, that we probably dramatically underestimate our usage. Two linked observations occur to me nowadays about MySQLs place in the software ecosystem: its spreading fast, and not enough people know it deeply enough to truly be called MySQL experts. Once a month in Seattle, I go to my favorite bar and sit down with MySQL users and dis- cuss whatever they want to talk about. Some months we discuss obscure issues like B-tree fragmentation, and other months we talk about the latest and greatest in RAID controller cards. We never discuss beer, honest. There are many large companies in Seattle, from classical com- panies like actuaries to more modern ones like online merchants. MySQL users run the gamut. They realize that MySQL plays some part in their diverse IT infrastructure, and their DBAs and CTOs show up to talk about what they are doing and how they are making use of MySQL. We 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xix

also get a few people each meeting who are trying to learn about MySQL and are looking for resources to do so. Even with more people learning about MySQL every day, there are simply not enough MySQL experts to go around. There are employers and projects who need more experts. As enterprise growth with MySQL has increased, Ive seen an increase in the number of questions concerning how MySQL works. How does it store its indexes? How can I benchmark my appli- cation with my database? What is the best storage engine for my application? These questions and similar ones really push what MySQL experts need to know. If you are a DBA today, you really need to know more about how your database runs and how you integrate it into your environment. There have been a number of poorly written books on MySQL over the years. Reading this book brings a smile to my face, since I can see that some great writers are now putting their efforts into changing this. This bodes well for people on all sides of the database world. Brian Aker Director of Architecture, MySQL AB FOREWORDxx 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xx

About the Authors MICHAEL KRUCKENBERG started his career with web technologies more than ten years ago. His first major undertaking was bringing a small mail-order company online (using MySQL). After hopping around a bit during the 1990s Internet boom and spending time in the Internet startup world, Mike put his feet down at his current gig, senior programmer at Tufts University. He is now the technical manager for the Apache/Perl/MySQL-driven Tufts University Sciences Knowledgebase (TUSK), a content repository for faculty and students. Mike likes to spend his elusive free time with his wife and kids on New England adventures, has a healthy addiction to music, and likes to dabble in the arts (photography, video, and literature). You can read the latest about Mike at http:// mike.kruckenberg.com and can always get in touch with him at mike@kruckenberg.com. For the past ten years, JAY PIPES has worked with all kinds of companies, large and small, to identify the value of the information they collect and help them build software that best addresses the needs of their businesses. From e-commerce to work-order management systems, Jay has been involved in projects with both Microsoft and open-source technologies. Passionate about programming and all things technical, Jay now runs his own consulting business, based in Columbus, Ohio. When not being both- ered by his two cats, two dogs, and a constantly ringing phone, he can be found, headphones pulled across his ears, happily coding away at home. He welcomes any suggestions regarding this book (or any other subject, for that matter), and you can read about Jays current adventures at http://jpipes.com. You can always contact him at jay@jpipes.com. xxi 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxi

505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxii

xxiii About the Technical Reviewer CHAD RUSSELL is currently a software engineer/project manager for CDI IT Solutions, specifi- cally with LiquidMedium web-based software. He is also the founder of Russell Information Technologies, Inc. (RIT), an open-source enterprise software startup. Chad currently resides in Jacksonville, Florida, with his wife Kim. Chad has worked on numerous enterprise-level projects over the past six years, primarily developing and integrating PHP and MySQL web- based applications. He is currently busy with RIT and providing IT consulting. Chad is very active in his church, where he has been a member for 23 years. His hobbies include music (playing bass guitar), writing, fishing, programming, and finding new business opportunities. His key to success: Matthew 6:33. 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxiii

505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxiv

Acknowledgments Igrew up in a tiny town in North Dakota, where it was not unusual to see tumbleweeds roll through the middle of town, and movies always came six months later than the posters said they would. Despite being a bit behind the times, the towns two-year college and library both prioritized acquisition of a few of the early personal computers, and made them available to reckless kids who would otherwise be putting coins on the train tracks or riding dirt bikes through the alleys looking for trouble. It was in that library and the iconic Old Main building of the technical college that I first got a taste for the power of writing instructions and letting the computer do the work. Its easy to trace my computing addiction back to those early years. Thanks to the folks in small-town North Dakota for putting at my fingertips the tools that would eventually lead to a career in technology and my having the confidence to agree to tackle this book. I would like to thank all the friends, family members, employees, employers, and co-workers who supported my interest in learning new technologies. More specifically, Id like to thank my older brother, Pete, who is a mentor extraordinaire and a source of career and personal inspiration. Our history of conversations is a big part of where I am and where Im going. I also owe a lot to my father, Marlyn, who has always been an example of working hard and not letting sleep take precedence over opportunity. Thanks for letting Pete and me convince you that having a TRS-80 at home would revitalize your in-home business. This book wouldnt be much without Jay, who wrote about everything I couldnt (or didnt want to), including taking the book into the depths of the MySQL code. Your writing, and sug- gestions for my writing, made a huge difference in the direction and outcome of the book. I must also thank the Apress team, who are all superstars and have an incredible (although its the only one Ive seen) process for helping authors get their thoughts into print.Working with Jason, Kylie, Chad, Matt, Marilyn, Linda, and Susannah has been a great experience. Thanks for your contributions to the book, which wouldnt be nearly as good without your efforts. To my kids, Johanna and Ezra, thanks for learning to get breakfast yourself so I could sleep a few more minutes in the morning after countless late nights. You have both been patient while dad spends evenings and weekends hunkered over the laptop. And last, to Heidi, you have been more supportive than I could have imagined. Thanks for pulling double-duty during this longer-than-expected process, and for making it easy for me to feel good about spending yet another evening writing. Michael Kruckenberg xxv 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxv

I once sat through a technical interview a number of years ago for a position I eventually received. My interviewer, Ron Kyle, drilled me on a battery of programming questions, and I answered each to the best of my knowledge. After hearing my answers to the technical ques- tions, Ron turned to me and asked with a sly smile on his mug, In the United States Senate, what is the purpose of cloture? I remember turning a bit pale-faced at the time. Ron had noticed on my rsum that I had been a political science major in university. It just so happened that Ron had held a politics and military history teaching position at West Point while in the United States Army. He had decided to test whether I actually remembered anything from my classes in college. I dont remember whether I answered correctly (cloture is a method by which a filibuster can be ended through a 60% vote of senators), but Rons question made a lasting impression on me. The technical acumen of a person is not the entire measure of ones ability to contribute to a project. A team of people devoted to a project brings a wide range of skills, passions, knowledge, and differences to the table; it is the mark of a good team to recognize and cele- brate those differences. Though the members of the remarkable team of which Ron and I were each a part have long since gone our separate ways, I look back fondly at the work we accomplished and the extraordinary mixture of talent and diversity manifested there. So, Ron, Id like to thank you for showing me the human side of technology and reminding me that our differences provide our work with strength and flexibility. I would also like to thank the brilliant and patient team at Apress. First and foremost, Jason Gilmore, our editor, and the Apress team, thank you for the opportunity to write about a sub- ject which so thrills me, and secondly for providing a consistent and supportive foundation on which to create both Mikes and my first major text. Kylie Johnston, our project manager, thank you for your unending patience in dealing with our often sidetracked schedule; your efforts are much appreciated. Chad Russell, our excellent technical reviewer, thank you for your diligence in examining our code and scripts with as much attention as you paid to your own book. Marilyn Smith, your attention to detail during our copyedit stage saved me unknown embarrassment for my flagrant violations of English grammar rules. And finally, Linda Marousek, thank you for your work during the production stages of the book. To my co-author, Mike, I know that both of us have been through a roller coaster in this, our first book. I thank you for your wonderful input and your technical and writing skills. I couldnt have asked for a more complementary author in this endeavor. I feel our long hours and many conversations have created something greater than the sum of its two parts. Thank you ever so much. Last, but far from least, thank you to Julie for persevering through countless nights of me working until 4 a.m. scrambling to complete a chapter; youve been my biggest and most unsung supporter. Jay Pipes ACKNOWLEDGMENTSxxvi 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxvi

Introduction Data, in all varieties, continues to be the cornerstone from which modern organizations and businesses derive their information capital. Whether putting items in an online shopping cart for checkout or tracking terabytes of purchase history to find shopping trends, having a database to store and retrieve data has become a necessity. From the individual to the small nonprofit organization, the small business to the large corporation, retrieving stored informa- tion at the right time and in the right format is helping people make more informed business decisions. The demand for larger amounts of data, stored in more user-friendly formats and using faster database tools, continues to grow. About MySQL and MySQL AB While the demand for tools to store, retrieve, and mine data has grown, MySQL AB has been hard at work building the database to handle these ever-increasing needs.When initially released, MySQL focused on speed and simplicity, intentionally sacrificing some of the more advanced features of proprietary relational database management systems for better perform- ance. For many years, MySQL focused on performance and stability, but as it rose in the market, the database fell under criticism for its lack of advanced features, including things like foreign keys, row-level locking, stored procedures, views, triggers, and clustering. As MySQL has matured, these features have been added to the database server, after ensuring that such functionality does not come at the cost of performance and ease of use. The combination of simplicity, speed, and a full feature set makes MySQL a compelling option when considering how to meet the data demands of your application or organization. We believe MySQL can play a central part in helping you better meet your data storage and retrieval needs. We are excited to be writing about MySQL, because its an exciting move- ment as well as exciting software, which continues to demonstrate its value to a wide array of organizations. We are both passionate about MySQL as a company and MySQL as a database platform. MySQL AB has set the bar for how open-source technology companies can and should embrace the community of developers who have long been locked into the traditional patent-based software paradigm. The MySQL database server platform is a shining example of what can happen when truly brilliant minds are brought together under the umbrella of a company that values diversity, community input, and the contributions of the world community. As weve written this book, its painfully clear that MySQL is constantly moving forward. Performance enhancement, stability, and new features continue to take the source code and interface to the next level. While this can make writing a book a challenge, it is a sign of the health of the MySQL community and the active development happening at MySQL AB. As were writing, there is a lot of talk starting about MySQL 5.1 and things like multimaster repli- cation, foreign keys for all storage engines, new backup mechanisms, and more. We look forward to the future of MySQL and the changes it will require to the chapters in this book. xxvii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxvii

As you read this book, going from general subjects like determining business require- ments all the way into the gritty details of subjects like clustering, we hope you will find your interaction with the database as rewarding and exciting as weve found our work with MySQL. Who This Book Is For This book is for the individual who has used MySQL at some level and is interested in explor- ing advanced MySQL topics. Youll find coverage of techniques and technologies that arent included in introductory or general-usage MySQL books. If you have a passion for learning new technologies, this book is for you. We assume that readers have worked with relational databases, are familiar with general database concepts, have some experience with MySQL, have written database queries, and have used command-line tools. We do not cover basic SQL, but we do explore how to use joins, subqueries, and derived tables to retrieve information efficiently. If you arent familiar with SQL, general database concepts, or MySQL, you might find Beginning MySQL Database Design and Optimization: From Novice to Professional, by John Stephens and Chad Russell (Apress, 2004), a helpful place to start. How This Book Is Structured The first part of the book covers issues in design and development of your MySQL applications. The second part of the book examines topics in the database administration and maintenance arena. Here are brief summaries of the contents of each chapter: Chapter 1, Analyzing Business Requirements: Before getting into the nitty-gritty of MySQL, well first take a look at the software development process in general, as well as specifics that apply to almost all database-driven applications. From the unique perspective of each development team role, well graze over topics such as requirements gathering, use cases, and putting together a project plan. Next, well dive into a discussion on object and data modeling, covering UML and entity-relationship diagramming.Well look at the strength of MySQL and its versions. Finally, well talk a bit about your development environment in general and the target audience of your projects software. Chapter 2, Index Concepts: Here, well examine how MySQLs different indexes work. Well start with a detailed discussion on index concepts, breaking down how they help the database quickly locate information in the database. Youll learn how the database server manages disk and memory-based data access, and how indexes and other data structures aid in this retrieval process. From B-tree indexing to hash-based lookups, well thoroughly explore how indexing algorithms work with varying table sizes and data types. Chapter 3, Transaction Processing: In this chapter, well start with a detailed discussion of transaction theory. Well go over what the ACID test is and how transactions serve to pro- tect the integrity of your data. After the theory, well jump right into some examples where transactions can be used in common business scenarios, and youll work through exer- cises designed to highlight how transactions are used. Well then dig into how the MySQL database server enforces transactions and what happens in the event of a failure when transactions are not used. Well end with a look at how locking, logging, and checkpointing INTRODUCTIONxxviii 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxviii

come into play with transactions in general. This will provide a foundation on transaction processing concepts that well build on with our internal examination of the InnoDB stor- age engine in Chapter 5. Chapter 4, MySQL System Architecture: Things start to get really interesting in Chapter 4, where well present an in-depth examination of the MySQL system internalssomething you wont find in other texts. Youll learn how the various subsystems interrelate inside the server architecture and how to find what you need in the developers documentation. Well even walk you through the code execution of a simple SELECT statement. Chapter 5, Storage Engines and Data Types: Your choice of storage engines in MySQL largely depends on the type of application you will run on the database and the data types you intend to store in your tables. In Chapter 5, well take a look at each of MySQLs stor- age engines, their various abilities, and where you will want to employ them. Well focus predominantly on the InnoDB and MyISAM engines, detailing the record and index for- mats of each. After looking into the storage engines, well examine MySQLs data types and how best to use each of them. Well discuss the differences between seemingly common data types and explain which data types to use for common storage requirements and typical data elements. Chapter 6, Benchmarking and Profiling: Profiling your database system is an essential part of the ongoing development process. In this chapter, youll learn a number of tech- niques to monitor the performance of your database queries. Youll learn how to identify key bottlenecks in the system, both in your SQL code and on the hardware in general. In the section on benchmarking, youll work on using open-source tools to determine the limits of your application on your deployment hardware. Well have you benchmarking real-world scenarios, walking you through setting up a benchmarking framework and running profiles on your own hardware to give you a feel for how to profile and bench- mark your own code. Chapter 7, Essential SQL: Your SQL code is the workhorse of your database application. Learning how to write effective queries is a critical component of designing high- performance systems, and well use this chapter to both review the SQL language in general and also provide some insight into the difference in set-based programming versus procedural or object-oriented programming. Well focus on the fundamentals here, and ask you to reexamine what you already know about SQL. Most of the chapter will be centered around how to retrieve the information you need efficiently and effectively through the various joins that MySQL provides. Extensive coverage of the subqueries and derived tables will lead up to the more advanced SQL topics covered in Chapter 8. Chapter 8, SQL Scenarios: The fundamentals you learned in Chapter 7 will provide the foundation for this chapter, which contains a collection of database development and administrative scenarios. Here, you will learn how to push the limits of the SQL language and explore alternate ways to accomplish complex database applications tasks. Well step through exercises that show how slight variations in similar SQL statements can have per- formance impacts for your applications. Among other topics, youll learn how to manage tree structures in SQL using set-based techniques; how to handle duplicate, orphaned, and random data in MySQL; and how to perform distance and radius calculations on GIS data using only your MySQL database server. INTRODUCTION xxix 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxix

Chapter 9, Stored Procedures: This chapter will show you how to harness the power of stor- ing groups of SQL statements and logic using MySQL and stored procedures compliant with the SQL:2003 standard. Well look at concepts surrounding stored procedure tech- nology and how stored procedures fit into the process of designing and building your application. Then well delve into the details of MySQLs implementation and how to build stored procedures using the SQL:2003 syntax. Chapter 10, Functions: Following on the tails of stored procedures are stored functions. Stored functions allow you to encapsulate sets of SQL statements into a single function call, which can be used in any SQL statement. Unlike previous versions of MySQL, where you were required to write C code, with MySQL 5.0, stored functions are created dynami- cally in the database with SQL statements. This chapter will look at some of the differences between stored functions, stored procedures, triggers, and views. Then well cover the details of MySQLs function implementation, using examples to demonstrate creating and managing functions in your database. Finally, well look at the performance implica- tions of using stored functions. Chapter 11, Cursors: Long supported in other commercial database servers, cursors have now found their way into MySQL. Well start this chapter with an examination of how cur- sors have been implemented in MySQL, and then move on to examples of how they are used. Then well take a look at the pros and cons of using cursors, and situations where they can be used effectively. Chapter 12,Views: This chapter details MySQLs implementation of views, which provide a mechanism to join columns from multiple tables into one virtual table, or limit access to certain columns and/or rows of a single table. Well review how views can help or hinder your use of the database, and go through the syntax to create and manage MySQLs views. Youll also learn how using views affects the performance of queries, and how to manage permissions for views. Chapter 13, Triggers: With MySQL 5.0 comes support for triggers. The ability to trigger actions based on events in a table can provide a great deal of relief to application pro- grammers and database programmers, but the power must be used wisely. This chapter will step through how triggers are created and used in MySQL, giving examples of various situations where triggers are particularly useful. Chapter 14, MySQL Installation and Configuration: MySQL can be installed, configured, and put to use in a matter of minutes, or the process can consume days. It all depends on whether you use a prebuilt installation or want to dig into the fine details of building and configuring the database on your own. This chapter will review the available release for- mats and how to get up and running with each one. Once youre up and running with the database, well go through some post-installation steps and configuration options you may consider before using the database. Before we leave the chapter, we will go through the procedures for upgrading and uninstalling a MySQL installation. Chapter 15, User Administration: Proper administration of user accounts and permissions is an important skill to have under your belt. This chapter will go through the simple process of creating users and granting permissions, from the most limited to global. As we go over granting permissions, well talk about some of the implications in providing user accounts and how they impact the security of your data and database. INTRODUCTIONxxx 505x_Ch00_FINAL.qxd 6/27/05 3:21 PM Page xxx

Chapter 16, Security: Protecting your data, and the server it lives on, is a critical piece of administering a database. One of the first steps in securing the database is to review your data and determine the risk associated with the information in your database. In this chapter, well start with a discussion and examples of setting up a security policy. Next, well present a quick list of critical MySQL security issues you should address immedi- ately. Well then go through an example of implementing a security plan. Finally, well look into seven major areas that deserve attention when attempting to secure your database. Chapter 17, Backup and Restoration: In this chapter, well discuss the reasons for backing up data, how to make sure your backup mechanism matches the expectations for restor- ing data, and what to consider when creating your backup and recovery plan. Well then look at all the major tools for backing up and restoring MySQL data, including specific examples of the syntax used to create point-in-time snapshots of your data and restore your database with those snapshots. Chapter 18, Replication: The MySQL database server software includes everything to per- form near real-time replication to one or more servers. Replication can help with a wide variety of database demands, but it doesnt solve every problem with database load. This chapter will go into detail on a variety of scenarios where database replication will and will not help your database or application. We will also cover the commands to establish and manage both the master and slave database servers, and configuration options to tweak the behavior of your replication setup. Chapter 19, Cluster: MySQL Cluster is the combination of MySQL and an in-memory stor- age engine, NDB Cluster, which spreads data redundantly across storage nodes. Using clustering provides scalability, high availability, and high-performance interactions with the database, but isnt always the right answer. In this chapter, we will give an overview of cluster concepts, and then go through MySQLs implementation of clustering technology. Well review commands for setting up and maintaining a cluster environment. Chapter 20, Troubleshooting: In this chapter, well take a look at some of the most commonly experienced problems with MySQL and provide pointers on how to fix or find solutions as problems arise. For example, youll find tips on identifying and repairing corrupt tables, determining why mysqld wont start on your system, and ident