disc

Notes on Discrete MathematicsCS 202: Fall 2013

James Aspnes

2013-11-04 17:31

Contents

Table of contents i

List of figures xiv

List of tables xv

List of algorithms xvi

Preface xvii

Syllabus xviii

Resources xxii

Lecture schedule xxiv

1 Introduction 11.1 So why do I need to learn all this nasty mathematics? . . . . 11.2 But isnt math hard? . . . . . . . . . . . . . . . . . . . . . . . 21.3 Thinking about math with your heart . . . . . . . . . . . . . 31.4 What you should know about math . . . . . . . . . . . . . . . 3

1.4.1 Foundations and logic . . . . . . . . . . . . . . . . . . 31.4.2 Basic mathematics on the real numbers . . . . . . . . 41.4.3 Fundamental mathematical objects . . . . . . . . . . . 41.4.4 Modular arithmetic and polynomials . . . . . . . . . . 51.4.5 Linear algebra . . . . . . . . . . . . . . . . . . . . . . 61.4.6 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.7 Counting . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.8 Probability . . . . . . . . . . . . . . . . . . . . . . . . 71.4.9 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

i

CONTENTS ii

2 Mathematical logic 92.1 The basic picture . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Axioms, models, and inference rules . . . . . . . . . . 92.1.2 Consistency . . . . . . . . . . . . . . . . . . . . . . . . 102.1.3 What can go wrong . . . . . . . . . . . . . . . . . . . 102.1.4 The language of logic . . . . . . . . . . . . . . . . . . 112.1.5 Standard axiom systems and models . . . . . . . . . . 11

2.2 Propositional logic . . . . . . . . . . . . . . . . . . . . . . . . 122.2.1 Operations on propositions . . . . . . . . . . . . . . . 13

2.2.1.1 Precedence . . . . . . . . . . . . . . . . . . . 152.2.2 Truth tables . . . . . . . . . . . . . . . . . . . . . . . . 162.2.3 Tautologies and logical equivalence . . . . . . . . . . . 17

2.2.3.1 Inverses, converses, and contrapositives . . . 192.2.3.2 Equivalences involving true and false . . . . 21

Example . . . . . . . . . . . . . . . . . . . . . . 222.2.4 Normal forms . . . . . . . . . . . . . . . . . . . . . . . 23

2.3 Predicate logic . . . . . . . . . . . . . . . . . . . . . . . . . . 242.3.1 Variables and predicates . . . . . . . . . . . . . . . . . 252.3.2 Quantifiers . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3.2.1 Universal quantifier . . . . . . . . . . . . . . 262.3.2.2 Existential quantifier . . . . . . . . . . . . . 262.3.2.3 Negation and quantifiers . . . . . . . . . . . 272.3.2.4 Restricting the scope of a quantifier . . . . . 272.3.2.5 Nested quantifiers . . . . . . . . . . . . . . . 282.3.2.6 Examples . . . . . . . . . . . . . . . . . . . . 30

2.3.3 Functions . . . . . . . . . . . . . . . . . . . . . . . . . 312.3.4 Equality . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.3.4.1 Uniqueness . . . . . . . . . . . . . . . . . . . 322.3.5 Models . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.3.5.1 Examples . . . . . . . . . . . . . . . . . . . . 332.4 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.4.1 Inference Rules . . . . . . . . . . . . . . . . . . . . . . 352.4.2 Proofs, implication, and natural deduction . . . . . . . 37

2.4.2.1 The Deduction Theorem . . . . . . . . . . . 37Caveat: . . . . . . . . . . . . . . . . . . . . . . 38

2.5 Natural deduction . . . . . . . . . . . . . . . . . . . . . . . . 382.5.1 Inference rules for equality . . . . . . . . . . . . . . . 392.5.2 Inference rules for quantified statements . . . . . . . . 39

2.6 Proof techniques . . . . . . . . . . . . . . . . . . . . . . . . . 42

CONTENTS iii

3 Set theory 473.1 Naive set theory . . . . . . . . . . . . . . . . . . . . . . . . . 473.2 Operations on sets . . . . . . . . . . . . . . . . . . . . . . . . 493.3 Proving things about sets . . . . . . . . . . . . . . . . . . . . 503.4 Axiomatic set theory . . . . . . . . . . . . . . . . . . . . . . . 523.5 Cartesian products, relations, and functions . . . . . . . . . . 53

3.5.1 Examples of functions . . . . . . . . . . . . . . . . . . 553.5.2 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . 553.5.3 Functions of more (or less) than one argument . . . . 563.5.4 Composition of functions . . . . . . . . . . . . . . . . 563.5.5 Functions with special properties . . . . . . . . . . . . 56

3.5.5.1 Surjections . . . . . . . . . . . . . . . . . . . 573.5.5.2 Injections . . . . . . . . . . . . . . . . . . . . 573.5.5.3 Bijections . . . . . . . . . . . . . . . . . . . . 573.5.5.4 Bijections and counting . . . . . . . . . . . . 57

3.6 Constructing the universe . . . . . . . . . . . . . . . . . . . . 583.7 Sizes and arithmetic . . . . . . . . . . . . . . . . . . . . . . . 60

3.7.1 Infinite sets . . . . . . . . . . . . . . . . . . . . . . . . 603.7.2 Countable sets . . . . . . . . . . . . . . . . . . . . . . 623.7.3 Uncountable sets . . . . . . . . . . . . . . . . . . . . . 62

3.8 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4 The real numbers 644.1 Field axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.1.1 Axioms for addition . . . . . . . . . . . . . . . . . . . 654.1.2 Axioms for multiplication . . . . . . . . . . . . . . . . 664.1.3 Axioms relating multiplication and addition . . . . . . 684.1.4 Other algebras satisfying the field axioms . . . . . . . 69

4.2 Order axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . 704.3 Least upper bounds . . . . . . . . . . . . . . . . . . . . . . . 714.4 Whats missing: algebraic closure . . . . . . . . . . . . . . . . 734.5 Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.6 Connection between the reals and other standard algebras . . 744.7 Extracting information from reals . . . . . . . . . . . . . . . . 76

5 Induction and recursion 775.1 Simple induction . . . . . . . . . . . . . . . . . . . . . . . . . 775.2 Alternative base cases . . . . . . . . . . . . . . . . . . . . . . 795.3 Recursive definitions work . . . . . . . . . . . . . . . . . . . . 805.4 Other ways to think about induction . . . . . . . . . . . . . . 80

CONTENTS iv

5.5 Strong induction . . . . . . . . . . . . . . . . . . . . . . . . . 815.5.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.6 Recursively-defined structures . . . . . . . . . . . . . . . . . . 835.6.1 Functions on recursive structures . . . . . . . . . . . . 845.6.2 Recursive definitions and induction . . . . . . . . . . . 845.6.3 Structural induction . . . . . . . . . . . . . . . . . . . 85

6 Summation notation 866.1 Summations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6.1.1 Formal definition . . . . . . . . . . . . . . . . . . . . . 876.1.2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.1.3 Summation identities . . . . . . . . . . . . . . . . . . . 876.1.4 Choosing and replacing index variables . . . . . . . . . 896.1.5 Sums over given index sets . . . . . . . . . . . . . . . 896.1.6 Sums without explicit bounds . . . . . . . . . . . . . . 916.1.7 Infinite sums . . . . . . . . . . . . . . . . . . . . . . . 916.1.8 Double sums . . . . . . . . . . . . . . . . . . . . . . . 91

6.2 Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926.3 Other big operators . . . . . . . . . . . . . . . . . . . . . . . 926.4 Closed forms . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

6.4.1 Some standard sums . . . . . . . . . . . . . . . . . . . 946.4.2 Guess but verify . . . . . . . . . . . . . . . . . . . . . 956.4.3 Ansatzes . . . . . . . . . . . . . . . . . . . . . . . . . . 966.4.4 Strategies for asymptotic estimates . . . . . . . . . . . 97

6.4.4.1 Pull out constant factors . . . . . . . . . . . 976.4.4.2 Bound using a known sum . . . . . . . . . . 97

Geometric series . . . . . . . . . . . . . . . . . 97Constant series . . . . . . . . . . . . . . . . . . 98Arithmetic series . . . . . . . . . . . . . . . . . 98Harmonic series . . . . . . . . . . . . . . . . . . 98

6.4.4.3 Bound part of the sum . . . . . . . . . . . . 986.4.4.4 Integrate . . . . . . . . . . . . . . . . . . . . 986.4.4.5 Grouping terms . . . . . . . . . . . . . . . . 986.4.4.6 Oddities . . . . . . . . . . . . . . . . . . . . . 996.4.4.7 Final notes . . . . . . . . . . . . . . . . . . . 99

7 Asymptotic notation 1007.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1007.2 Motivating the definitions . . . . . . . . . . . . . . . . . . . . 1007.3 Proving asymptotic bounds . . . . . . . . . . . . . . . . . . . 101

CONTENTS v

7.4 Asymptotic notation hints . . . . . . . . . . . . . . . . . . . . 1027.4.1 Remember the difference between big-O, big-, and

big- . . . . . . . . . . . . . . . . . . . . . . . . . . . 1027.4.2 Simplify your asymptotic terms as much as possible . 1037.4.3 Remember the limit trick . . . . . . . . . . . . . . . . 103

7.5 Variations in notation . . . . . . . . . . . . . . . . . . . . . . 1037.5.1 Absolute values . . . . . . . . . . . . . . . . . . . . . . 1047.5.2 Abusing the equals sign . . . . . . . . . . . . . . . . . 104

8 Number theory 1068.1 Divisibility and division . . . . . . . . . . . . . . . . . . . . . 1068.2 Greatest common divisors . . . . . . . . . . . . . . . . . . . . 108

8.2.1 The Euclidean algorithm for computing gcd(m,n) . . 1088.2.2 The extended Euclidean algorithm . . . . . . . . . . . 109

8.2.2.1 Example . . . . . . . . . . . . . . . . . . . . 1098.2.2.2 Applications . . . . . . . . . . . . . . . . . . 109

8.3 The Fundamental Theorem of Arithmetic . . . . . . . . . . . 1128.3.1 Applications . . . . . . . . . . . . . . . . . . . . . . . 112

8.4 Modular arithmetic and residue classes . . . . . . . . . . . . . 1138.4.1 Arithmetic on residue classes . . . . . . . . . . . . . . 1148.4.2 Division in Zm . . . . . . . . . . . . . . . . . . . . . . 1158.4.3 The Chinese Remainder Theorem . . . . . . . . . . . . 1168.4.4 The size of Zm and Eulers Theorem . . . . . . . . . . 119

8.5 RSA encryption . . . . . . . . . . . . . . . . . . . . . . . . . . 120

9 Relations 1229.1 Representing relations . . . . . . . . . . . . . . . . . . . . . . 122

9.1.1 Directed graphs . . . . . . . . . . . . . . . . . . . . . . 1229.1.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 123

9.2 Operations on relations . . . . . . . . . . . . . . . . . . . . . 1249.2.1 Composition . . . . . . . . . . . . . . . . . . . . . . . 1249.2.2 Inverses . . . . . . . . . . . . . . . . . . . . . . . . . . 124

9.3 Classifying relations . . . . . . . . . . . . . . . . . . . . . . . 1259.4 Equivalence relations . . . . . . . . . . . . . . . . . . . . . . . 125

9.4.1 Why we like equivalence relations . . . . . . . . . . . . 1279.5 Partial orders . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

9.5.1 Drawing partial orders . . . . . . . . . . . . . . . . . . 1299.5.2 Comparability . . . . . . . . . . . . . . . . . . . . . . 1309.5.3 Lattices . . . . . . . . . . . . . . . . . . . . . . . . . . 1319.5.4 Minimal and maximal elements . . . . . . . . . . . . . 131

CONTENTS vi

9.5.5 Total orders . . . . . . . . . . . . . . . . . . . . . . . . 1329.5.5.1 Topological sort . . . . . . . . . . . . . . . . 132

9.5.6 Well orders . . . . . . . . . . . . . . . . . . . . . . . . 1349.6 Closures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

9.6.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 138

10 Graphs 13910.1 Types of graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 140

10.1.1 Directed graphs . . . . . . . . . . . . . . . . . . . . . . 14010.1.2 Undirected graphs . . . . . . . . . . . . . . . . . . . . 14010.1.3 Hypergraphs . . . . . . . . . . . . . . . . . . . . . . . 141

10.2 Examples of graphs . . . . . . . . . . . . . . . . . . . . . . . . 14210.3 Local structure of graphs . . . . . . . . . . . . . . . . . . . . 14310.4 Some standard graphs . . . . . . . . . . . . . . . . . . . . . . 14410.5 Operations on graphs . . . . . . . . . . . . . . . . . . . . . . 148

10.5.1 Set-like operations . . . . . . . . . . . . . . . . . . . . 14810.5.2 Minors . . . . . . . . . . . . . . . . . . . . . . . . . . . 14910.5.3 Functions . . . . . . . . . . . . . . . . . . . . . . . . . 150

10.6 Paths and connectivity . . . . . . . . . . . . . . . . . . . . . . 15110.7 Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15310.8 Proving things about graphs . . . . . . . . . . . . . . . . . . . 154

10.8.1 Paths and simple paths . . . . . . . . . . . . . . . . . 15410.8.2 The Handshaking Lemma . . . . . . . . . . . . . . . . 15410.8.3 Characterizations of trees . . . . . . . . . . . . . . . . 15510.8.4 Spanning trees . . . . . . . . . . . . . . . . . . . . . . 15810.8.5 Eulerian cycles . . . . . . . . . . . . . . . . . . . . . . 159

11 How to count 16011.1 What counting is . . . . . . . . . . . . . . . . . . . . . . . . . 160

11.1.1 Countable and uncountable sets . . . . . . . . . . . . 16011.2 Basic counting techniques . . . . . . . . . . . . . . . . . . . . 162

11.2.1 Reducing to a previously-solved case . . . . . . . . . . 16211.2.2 Showing |A| |B| and |B| |A| . . . . . . . . . . . . 16211.2.3 Sum rule . . . . . . . . . . . . . . . . . . . . . . . . . 163

11.2.3.1 For infinite sets . . . . . . . . . . . . . . . . 16411.2.3.2 The Pigeonhole Principle . . . . . . . . . . . 164

11.2.4 Inclusion-exclusion (with two sets) . . . . . . . . . . . 16411.2.4.1 For infinite sets . . . . . . . . . . . . . . . . 16511.2.4.2 Combinatorial proof . . . . . . . . . . . . . . 165

11.2.5 Product rule . . . . . . . . . . . . . . . . . . . . . . . 166

CONTENTS vii

11.2.5.1 Examples . . . . . . . . . . . . . . . . . . . . 16611.2.5.2 For infinite sets . . . . . . . . . . . . . . . . 167

11.2.6 Exponent rule . . . . . . . . . . . . . . . . . . . . . . . 16811.2.7 Counting the same thing in two different ways . . . . 168

11.3 Applying the rules . . . . . . . . . . . . . . . . . . . . . . . . 17011.4 An elaborate counting problem . . . . . . . . . . . . . . . . . 17211.5 Binomial coefficients . . . . . . . . . . . . . . . . . . . . . . . 175

11.5.1 Recursive definition . . . . . . . . . . . . . . . . . . . 17511.5.1.1 Pascals identity: algebraic proof . . . . . . . 176

11.5.2 Vandermondes identity . . . . . . . . . . . . . . . . . 17711.5.2.1 Combinatorial proof . . . . . . . . . . . . . . 17811.5.2.2 Algebraic proof . . . . . . . . . . . . . . . . . 178

11.5.3 Sums of binomial coefficients . . . . . . . . . . . . . . 17911.5.4 The general inclusion-exclusion formula . . . . . . . . 17911.5.5 Negative binomial coefficients . . . . . . . . . . . . . . 18011.5.6 Fractional binomial coefficients . . . . . . . . . . . . . 181

11.6 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . 18111.7 Generating functions . . . . . . . . . . . . . . . . . . . . . . . 181

11.7.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . 18111.7.1.1 A simple example . . . . . . . . . . . . . . . 18211.7.1.2 Why this works . . . . . . . . . . . . . . . . 18311.7.1.3 Formal definition . . . . . . . . . . . . . . . . 184

11.7.2 Some standard generating functions . . . . . . . . . . 18611.7.3 More operations on formal power series and generating

functions . . . . . . . . . . . . . . . . . . . . . . . . . 18611.7.4 Counting with generating functions . . . . . . . . . . . 187

11.7.4.1 Disjoint union . . . . . . . . . . . . . . . . . 18711.7.4.2 Cartesian product . . . . . . . . . . . . . . . 18811.7.4.3 Repetition . . . . . . . . . . . . . . . . . . . 188

Example: (0|11) . . . . . . . . . . . . . . . . . 188Example: sequences of positive integers . . . . 188

11.7.4.4 Pointing . . . . . . . . . . . . . . . . . . . . 19011.7.4.5 Substitution . . . . . . . . . . . . . . . . . . 190

Example: bit-strings with primes . . . . . . . . 191Example: (0|11)* again . . . . . . . . . . . . . 191

11.7.5 Generating functions and recurrences . . . . . . . . . 19111.7.5.1 Example: A Fibonacci-like recurrence . . . . 192

11.7.6 Recovering coefficients from generating functions . . . 19211.7.6.1 Partial fraction expansion and Heavisides

cover-up method . . . . . . . . . . . . . . . . 194

CONTENTS viii

Example: A simple recurrence . . . . . . . . . . 194Example: Coughing cows . . . . . . . . . . . . 195Example: A messy recurrence . . . . . . . . . . 196

11.7.6.2 Partial fraction expansion with repeated roots198Solving for the PFE directly . . . . . . . . . . . 198Solving for the PFE using the extended cover-

up method . . . . . . . . . . . . . . 20011.7.7 Asymptotic estimates . . . . . . . . . . . . . . . . . . 20111.7.8 Recovering the sum of all coefficients . . . . . . . . . . 202

11.7.8.1 Example . . . . . . . . . . . . . . . . . . . . 20211.7.9 A recursive generating function . . . . . . . . . . . . . 20311.7.10Summary of operations on generating functions . . . . 20611.7.11Variants . . . . . . . . . . . . . . . . . . . . . . . . . . 20611.7.12Further reading . . . . . . . . . . . . . . . . . . . . . . 206

11.8 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . 207

12 Probability theory 20812.1 History and interpretation . . . . . . . . . . . . . . . . . . . . 20812.2 Probability axioms . . . . . . . . . . . . . . . . . . . . . . . . 209

12.2.1 The Kolmogorov axioms . . . . . . . . . . . . . . . . . 21012.2.2 Examples of probability spaces . . . . . . . . . . . . . 210

12.3 Probability as counting . . . . . . . . . . . . . . . . . . . . . 21112.3.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 211

12.4 Independence and the intersection of two events . . . . . . . . 21212.4.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 213

12.5 Union of two events . . . . . . . . . . . . . . . . . . . . . . . 21312.5.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 214

12.6 Conditional probability . . . . . . . . . . . . . . . . . . . . . 21412.6.1 Conditional probabilities and intersections of non-independent

events . . . . . . . . . . . . . . . . . . . . . . . . . . . 21512.6.2 The law of total probability . . . . . . . . . . . . . . . 21612.6.3 Bayess formula . . . . . . . . . . . . . . . . . . . . . . 216

12.7 Random variables . . . . . . . . . . . . . . . . . . . . . . . . . 21712.8 The distribution of a random variable . . . . . . . . . . . . . 217

12.8.1 Some standard distributions . . . . . . . . . . . . . . . 21812.8.2 Joint distributions . . . . . . . . . . . . . . . . . . . . 219

12.8.2.1 Examples . . . . . . . . . . . . . . . . . . . . 21912.8.3 Independence of random variables . . . . . . . . . . . 219

12.8.3.1 Examples . . . . . . . . . . . . . . . . . . . . 22012.8.3.2 Independence of many random variables . . . 220

CONTENTS ix

12.9 The expectation of a random variable . . . . . . . . . . . . . 22112.9.1 Variables without expectations . . . . . . . . . . . . . 22212.9.2 Expectation of a sum . . . . . . . . . . . . . . . . . . 222

Example . . . . . . . . . . . . . . . . . . . . . . 22312.9.3 Expectation of a product . . . . . . . . . . . . . . . . 22312.9.4 Conditional expectation . . . . . . . . . . . . . . . . . 224

12.9.4.1 Examples . . . . . . . . . . . . . . . . . . . . 225The Treasure on Mt Everest . . . . . . . . . . . 225Waiting times . . . . . . . . . . . . . . . . . . . 225Rewarding success and punishing failure . . . . 225

12.9.4.2 Conditioning on a random variable . . . . . . 22612.9.5 Markovs inequality . . . . . . . . . . . . . . . . . . . 228

12.9.5.1 Example . . . . . . . . . . . . . . . . . . . . 22812.9.5.2 Conditional Markovs inequality . . . . . . . 229

12.10The variance of a random variable . . . . . . . . . . . . . . . 22912.10.1Multiplication by constants . . . . . . . . . . . . . . . 23012.10.2The variance of a sum . . . . . . . . . . . . . . . . . . 23012.10.3Chebyshevs inequality . . . . . . . . . . . . . . . . . . 231

12.10.3.1 Application: showing that a random variableis close to its expectation . . . . . . . . . . . 232

12.10.3.2 Application: lower bounds on random vari-ables . . . . . . . . . . . . . . . . . . . . . . 232

12.11Probability generating functions . . . . . . . . . . . . . . . . 23312.11.1Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . 23312.11.2Expectation and variance . . . . . . . . . . . . . . . . 234

12.12Summary: effects of operations on expectation and varianceof random variables . . . . . . . . . . . . . . . . . . . . . . . . 235

12.13The general case . . . . . . . . . . . . . . . . . . . . . . . . . 23512.13.1Densities . . . . . . . . . . . . . . . . . . . . . . . . . 23712.13.2 Independence . . . . . . . . . . . . . . . . . . . . . . . 23712.13.3Expectation . . . . . . . . . . . . . . . . . . . . . . . . 238

13 Linear algebra 23913.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

13.1.1 Interpretation . . . . . . . . . . . . . . . . . . . . . . . 24013.1.2 Operations on matrices . . . . . . . . . . . . . . . . . 241

13.1.2.1 Transpose of a matrix . . . . . . . . . . . . . 24113.1.2.2 Sum of two matrices . . . . . . . . . . . . . . 24113.1.2.3 Product of two matrices . . . . . . . . . . . . 24213.1.2.4 The inverse of a matrix . . . . . . . . . . . . 243

CONTENTS x

Example . . . . . . . . . . . . . . . . . . . . . . 24413.1.2.5 Scalar multiplication . . . . . . . . . . . . . . 245

13.1.3 Matrix identities . . . . . . . . . . . . . . . . . . . . . 24513.2 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

13.2.1 Geometric interpretation . . . . . . . . . . . . . . . . . 24713.2.2 Sums of vectors . . . . . . . . . . . . . . . . . . . . . . 24813.2.3 Length . . . . . . . . . . . . . . . . . . . . . . . . . . . 24913.2.4 Dot products and orthogonality . . . . . . . . . . . . . 249

13.3 Linear combinations and subspaces . . . . . . . . . . . . . . . 25113.3.1 Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

13.4 Linear transformations . . . . . . . . . . . . . . . . . . . . . . 25313.4.1 Composition . . . . . . . . . . . . . . . . . . . . . . . 25313.4.2 Role of rows and columns of M in the product Mx . . 25413.4.3 Geometric interpretation . . . . . . . . . . . . . . . . . 25513.4.4 Rank and inverses . . . . . . . . . . . . . . . . . . . . 25713.4.5 Projections . . . . . . . . . . . . . . . . . . . . . . . . 258

13.5 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . 261

14 Finite fields 26214.1 A magic trick . . . . . . . . . . . . . . . . . . . . . . . . . . . 26214.2 Fields and rings . . . . . . . . . . . . . . . . . . . . . . . . . . 26314.3 Polynomials over a field . . . . . . . . . . . . . . . . . . . . . 265

Division algorithm for polynomials . . . . . . . 26514.4 Algebraic field extensions . . . . . . . . . . . . . . . . . . . . 26614.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

14.5.1 Linear-feedback shift registers . . . . . . . . . . . . . . 26814.5.2 Checksums . . . . . . . . . . . . . . . . . . . . . . . . 26914.5.3 Cryptography . . . . . . . . . . . . . . . . . . . . . . . 270

A Assignments 271A.1 Assignment 1: due Thursday, 2013-09-12, at 5:00 pm (Huamin)271

A.1.1 Tautologies . . . . . . . . . . . . . . . . . . . . . . . . 272A.1.2 Positively equivalent . . . . . . . . . . . . . . . . . . . 273A.1.3 A theory of leadership . . . . . . . . . . . . . . . . . . 274

A.2 Assignment 2: due Thursday, 2013-09-19, at 5:00 pm (Xiao) . 275A.2.1 Subsets . . . . . . . . . . . . . . . . . . . . . . . . . . 275A.2.2 A distributive law . . . . . . . . . . . . . . . . . . . . 276A.2.3 Exponents . . . . . . . . . . . . . . . . . . . . . . . . . 276

A.3 Assignment 3: due Thursday, 2013-09-26, at 5:00 pm (Mike) . 277A.3.1 Surjections . . . . . . . . . . . . . . . . . . . . . . . . 277

CONTENTS xi

A.3.2 Proving an axiom the hard way . . . . . . . . . . . . . 278A.3.3 Squares and bigger squares . . . . . . . . . . . . . . . 279

A.4 Assignment 4: due Thursday, 2013-10-03, at 5:00 pm (Huamin)279A.4.1 A fast-growing function . . . . . . . . . . . . . . . . . 279A.4.2 A slow-growing set . . . . . . . . . . . . . . . . . . . . 280A.4.3 Double factorials . . . . . . . . . . . . . . . . . . . . . 281

A.5 Assignment 5: due Thursday, 2013-10-10, at 5:00 pm (Xiao) . 283A.5.1 A bouncy function . . . . . . . . . . . . . . . . . . . . 283A.5.2 Least common multiples of greatest common divisors . 284A.5.3 Adding and subtracting . . . . . . . . . . . . . . . . . 284

A.6 Assignment 6: due Thursday, 2013-10-31, at 5:00 pm (Mike) 285A.6.1 Factorials mod n . . . . . . . . . . . . . . . . . . . . . 285A.6.2 Indivisible and divisible . . . . . . . . . . . . . . . . . 285A.6.3 Equivalence relations . . . . . . . . . . . . . . . . . . . 286

A.7 Assignment 7: due Thursday, 2013-11-07, at 5:00 pm (Mike) 287A.7.1 Flipping lattices with a function . . . . . . . . . . . . 287A.7.2 Splitting graphs with a mountain . . . . . . . . . . . . 287A.7.3 Drawing stars with modular arithmetic . . . . . . . . 287

A.8 Assignment 8: due Thursday, 2013-11-14, at 5:00 pm (Huamin). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

A.9 Assignment 9: due Thursday, 2013-11-21, at 5:00 pm (Xiao) 288

B Exams 289B.1 CS202 Exam 1, October 17th, 2013 . . . . . . . . . . . . . . . 289

B.1.1 A tautology (20 points) . . . . . . . . . . . . . . . . . 289B.1.2 A system of equations (20 points) . . . . . . . . . . . 289B.1.3 A sum of products (20 points) . . . . . . . . . . . . . 290B.1.4 A subset problem (20 points) . . . . . . . . . . . . . . 291

C Midterm exams from previous semesters 292C.1 Midterm Exam, October 12th, 2005 . . . . . . . . . . . . . . 292

C.1.1 A recurrence (20 points) . . . . . . . . . . . . . . . . . 292C.1.2 An induction proof (20 points) . . . . . . . . . . . . . 293C.1.3 Some binomial coefficients (20 points) . . . . . . . . . 294C.1.4 A probability problem (20 points) . . . . . . . . . . . 294

C.2 Midterm Exam, October 24th, 2007 . . . . . . . . . . . . . . 295C.2.1 Dueling recurrences (20 points) . . . . . . . . . . . . . 295C.2.2 Seating arrangements (20 points) . . . . . . . . . . . . 295C.2.3 Non-attacking rooks (20 points) . . . . . . . . . . . . . 296C.2.4 Subsets (20 points) . . . . . . . . . . . . . . . . . . . . 297

CONTENTS xii

C.3 Midterm Exam, October 24th, 2008 . . . . . . . . . . . . . . 297C.3.1 Some sums (20 points) . . . . . . . . . . . . . . . . . . 297C.3.2 Nested ranks (20 points) . . . . . . . . . . . . . . . . . 297C.3.3 Nested sets (20 points) . . . . . . . . . . . . . . . . . . 298C.3.4 An efficient grading method (20 points) . . . . . . . . 299

C.4 Midterm exam, October 21st, 2010 . . . . . . . . . . . . . . . 299C.4.1 A partial order (20 points) . . . . . . . . . . . . . . . 300C.4.2 Big exponents (20 points) . . . . . . . . . . . . . . . . 300C.4.3 At the playground (20 points) . . . . . . . . . . . . . . 300C.4.4 Gauss strikes back (20 points) . . . . . . . . . . . . . . 301

D Final exams from previous semesters 302D.1 CS202 Final Exam, December 15th, 2004 . . . . . . . . . . . 302

D.1.1 A multiplicative game (20 points) . . . . . . . . . . . 302D.1.2 An equivalence in space (20 points) . . . . . . . . . . . 304D.1.3 A very big fraction (20 points) . . . . . . . . . . . . . 304D.1.4 A pair of odd vertices (20 points) . . . . . . . . . . . . 305D.1.5 How many magmas? (20 points) . . . . . . . . . . . . 305D.1.6 A powerful relationship (20 points) . . . . . . . . . . . 305D.1.7 A group of archaeologists (20 points) . . . . . . . . . . 306

D.2 CS202 Final Exam, December 16th, 2005 . . . . . . . . . . . 306D.2.1 Order (20 points) . . . . . . . . . . . . . . . . . . . . . 307D.2.2 Count the subgroups (20 points) . . . . . . . . . . . . 307D.2.3 Two exits (20 points) . . . . . . . . . . . . . . . . . . . 307D.2.4 Victory (20 points) . . . . . . . . . . . . . . . . . . . . 308D.2.5 An aggressive aquarium (20 points) . . . . . . . . . . 308D.2.6 A subspace of matrices (20 points) . . . . . . . . . . . 309

D.3 CS202 Final Exam, December 20th, 2007 . . . . . . . . . . . 310D.3.1 A coin-flipping problem (20 points) . . . . . . . . . . . 310D.3.2 An ordered group (20 points) . . . . . . . . . . . . . . 311D.3.3 Weighty vectors (20 points) . . . . . . . . . . . . . . . 311D.3.4 A dialectical problem (20 points) . . . . . . . . . . . . 312D.3.5 A predictable pseudorandom generator (20 points) . . 313D.3.6 At the robot factory (20 points) . . . . . . . . . . . . 314

D.4 CS202 Final Exam, December 19th, 2008 . . . . . . . . . . . 315D.4.1 Some logical sets (20 points) . . . . . . . . . . . . . . 315D.4.2 Modularity (20 points) . . . . . . . . . . . . . . . . . . 315D.4.3 Coin flipping (20 points) . . . . . . . . . . . . . . . . . 316D.4.4 A transitive graph (20 points) . . . . . . . . . . . . . . 316D.4.5 A possible matrix identity (20 points) . . . . . . . . . 316

CONTENTS xiii

D.5 CS202 Final Exam, December 14th, 2010 . . . . . . . . . . . 317D.5.1 Backwards and forwards (20 points) . . . . . . . . . . 317D.5.2 Linear transformations (20 points) . . . . . . . . . . . 318D.5.3 Flipping coins (20 points) . . . . . . . . . . . . . . . . 319D.5.4 Subtracting dice (20 points) . . . . . . . . . . . . . . . 320D.5.5 Scanning an array (20 points) . . . . . . . . . . . . . . 321

E How to write mathematics 322E.1 By hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322E.2 LATEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322E.3 Microsoft Word equation editor . . . . . . . . . . . . . . . . . 323E.4 ASCII and/or Unicode art . . . . . . . . . . . . . . . . . . . 323

F Tools from calculus 325F.1 Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325F.2 Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327F.3 Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328

G Other topics not covered in detail 331

H The natural numbers 332H.1 The Peano axioms . . . . . . . . . . . . . . . . . . . . . . . . 332H.2 A model of the Peano axioms . . . . . . . . . . . . . . . . . . 334H.3 A simple proof . . . . . . . . . . . . . . . . . . . . . . . . . . 335H.4 Defining addition . . . . . . . . . . . . . . . . . . . . . . . . . 337

H.4.1 Other useful properties of addition . . . . . . . . . . . 339H.5 A scary induction proof involving even numbers . . . . . . . . 339H.6 Defining more operations . . . . . . . . . . . . . . . . . . . . 341

Bibliography 342

Index 344

List of Figures

8.1 Trace of extended Euclidean algorithm . . . . . . . . . . . . . 1108.2 Python code for extended Euclidean algorithm . . . . . . . . 111

9.1 Factors of 12 partially ordered by divisibility . . . . . . . . . 1309.2 Maximal and minimal elements . . . . . . . . . . . . . . . . . 1329.3 Reflexive, symmetric, and transitive closures . . . . . . . . . . 137

10.1 Complete graphs K1 through K10 . . . . . . . . . . . . . . . . 14510.2 Cycle graphs C3 through C11 . . . . . . . . . . . . . . . . . . 14610.3 Path graphs P0 through P4 . . . . . . . . . . . . . . . . . . . 14610.4 Complete bipartite graph K3,4 . . . . . . . . . . . . . . . . . 14710.5 star graphs K1,3 through K1,8 . . . . . . . . . . . . . . . . . . 14710.6 Two presentations of the cube graph Q3 . . . . . . . . . . . . 148

A.1 Examples of Sm,k for Problem A.7.3 . . . . . . . . . . . . . . 288

xiv

List of Tables

2.1 Compound propositions . . . . . . . . . . . . . . . . . . . . . 152.2 Common logical equivalences . . . . . . . . . . . . . . . . . . 202.3 Absorption laws . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4 Natural deduction: introduction and elimination rules . . . . 402.5 Proof techniques . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.1 Features of various standard algebras . . . . . . . . . . . . . . 76

8.1 Addition and multiplication tables for Z3 . . . . . . . . . . . 1158.2 Multiplication table for Z9 . . . . . . . . . . . . . . . . . . . . 116

F.1 Table of derivatives . . . . . . . . . . . . . . . . . . . . . . . . 328F.2 Table of integrals . . . . . . . . . . . . . . . . . . . . . . . . . 329

xv

List of Algorithms

xvi

Preface

These are notes for the Fall 2013 semester version of the Yale course CPSC202a, Mathematical Tools for Computer Science. This document also in-corporates the lecture schedule and assignments, as well as some sampleassignments from previous semesters. Because this is a work in progress, itwill be updated frequently over the course of the semester.

Notes from some previous versions of the course can be found at http://pine.cs.yale.edu/pinewiki/CS202/.

xvii

Syllabus

DescriptionIntroduction to formal methods for reasoning and to mathematical tech-niques basic to computer science. Topics include propositional logic, discretemathematics, and linear algebra. Emphasis on applications to computer sci-ence: recurrences, sorting, graph traversal, Gaussian elimination.

Meeting timesTuesday and Thursday 13:0014:15 in LC 317.

On-line course informationThe lecture schedule, course notes, and all assignments can be found in asingle gigantic PDF file at http://cs.yale.edu/homes/aspnes/classes/202/notes.pdf. You should probably bookmark this file, as it will be up-dated frequently.

Office hours and other information can be found in the course calendarat Google Calendar.

StaffIf you cant make the open office hours listed in the course calendar, youmay be able to schedule appointments at other times by email.

InstructorJames Aspnes. Office: AKW 401. Email: [email protected].

xviii

SYLLABUS xix

Teaching fellows Jerrod Ankenman. Office: AKW 114. Email: Huamin Li. Office: AKW 015. Email: [email protected]. Mike Marmar. Office: AKW 313. Email: [email protected].

Undergraduate course grader Xiao Shi. Email: [email protected].

Peer tutors Iulia Tamas. Email: [email protected]. Michael Tan. Email: [email protected]. Michael Zhao. Email: [email protected].

TextbookThe textbook for the class is Kevin Ferland, Discrete Mathematics, CengageLearning, 2008. ISBN 0618415386.

Course requirementsNine weekly homework assignments and two exams held at the regular lec-ture time. The exams will count for approximately three homework assign-ments each.

Use of outside helpStudents are free to discuss homework problems and course material witheach other, and to consult with the instructor or a TA. Solutions handed in,however, should be the students own work. If a student benefits substan-tially from hints or solutions received from fellow students or from outsidesources, then the student should hand in their solution but acknowledgethe outside sources, and we will apportion credit accordingly. Using outsideresources in solving a problem is acceptable but plagiarism is not.

SYLLABUS xx

Clarifications for homework assignmentsFrom time to time, ambiguities and errors may creep into homework assign-ments. Questions about the interpretation of homework assignments shouldbe sent to the instructor at [email protected]. Clarifications willappear in an updated version of the assignment.

Late assignmentsLate assignments will not be accepted without a Deans Excuse.

TopicsThe course will cover the minimal topics in mathematics that you will needto survive the Computer Science major. We assume that coming in to thecourse you will already have a thorough grounding in high school algebra,but may or may not have taken any other math classes. By the end of thecourse, you should:

Understand definitions and proofs, including quantifiers and induction. Understand basic set theory and set-theoretic notation. Be comfortable with manipulating commonly-used functions like ex-ponentials and logarithms.

Know how to count (not as easy as you think). Understand asymptotic notation. Know how to solve recurrences. Understand basic probability. Have a passing familiarity with standard mathematical concepts thatshow up in computer science, including graphs, algebraic structures(e.g., groups, rings, and fields), linear algebra and matrices, and basicnumber theory.

Because CS202 is only a one-semester course, coverage of most topicswill necessarily be rather sketchy. If you expect to do further work in thetheoretical end of computer science or in math-intensive fields like graphics,

SYLLABUS xxi

vision, neural networks, robotics, or scientific computation, you should planto take further courses in mathematics (a serious linear algebra course isparticularly recommended). One of the goals of CS202 is to provide youwith the mathematical maturity you will need to profit from such courses.

Resources

In addition to the textbook and these notes, you may find many other re-sources useful.

Reserved books at Bass Library Kevin Ferland. Discrete Mathematics. Cengage Learning, 2008. ISBN978-0618415380. [Fer08]

Kenneth H. Rosen. Discrete Mathematics and Its Applications, Sev-enth Edition, McGraw-Hill, 2012. ISBN 978-0072899054. QA39.3 R672012 (LC). [Ros12]

Norman L. Biggs. Discrete Mathematics, second edition. Oxford Uni-versity Press, 2002. ISBN 0198507178. QA76.9 M35 B54 2002. [Big02]

Ronald L. Graham, Donald E. Knuth, and Oren Patashnik. ConcreteMathematics: A Foundation for Computer Science. Addison-Wesley,1989. QA39.2 G733X 1989. [GKP94]

George Polya. How to Solve It: A New Aspect of Mathematical Method.Princeton University Press, 2004. QA11 .P66 2004 (LC). [Pol04]

Daniel Solow. How to Read and Do Proofs: An Introduction to Mathe-matical Thought Processes. Wiley, 2005. QA9.54 .S65X 2005 (LC) [Sol05]

Internet resourcesPlanetMath http://planetmath.org

Wolfram MathWorld http://mathworld.wolfram.com

WikiPedia http://en.wikipedia.org

xxii

RESOURCES xxiii

Google http://www.google.com

Lecture schedule

As always, the future is uncertain, so you should take parts of the schedulethat havent happened yet with a grain of salt. Readings refer to chaptersor sections in the course notes, except for those specified as Ferland, whichrefer to the course textbook [Fer08].

See also the course calendar, which lists lecture times, office hours forthe instructor and TA, and assignment due dates.

2013-08-29 Overview of the course. Start of mathematical logic: basicprinciples and propositional logic. Boolean operators, truth tables,tautologies, and logical equivalence. Readings: Chapter 1, 2.12.2;Ferland 1.1.

2013-09-03 More proposition logic: simplifying expressions using stan-dard equivalences. Predicate logic: constants, predicates, variablesand quantifiers. Nested quantifiers at the playground and the subtlebut important difference between x : y : likes(y, x) and y : x :likes(y, x). Readings: 2.3 through 2.3.2; Ferland 1.3.

2013-09-05 More predicate logic: Function symbols and equality. Mod-els in predicate logic. Start of proofs and inference rules. Readings:2.3.32.5.2; Ferland 1.5.

2013-09-10 Start of sets: set notation, basic operations on sets, examplesof set-theory axioms. Readings: 3.13.4; Ferland 1.2 and 1.4.

2013-09-12 More set theory: Cartesian products, relations, and functions.Sequences as functions. Bijections and counting. Readings: 3.53.7.1. Ferland 5.1 through Example 5.5, 5.3 through Example 5.37,5.4 through Example 5.47.

2013-09-17 More on writing proofs about relations between sets. Count-able vs uncountable sets. Readings: 3.3, 3.7.2, and 3.7.3; Ferland2.22.3.

xxiv

LECTURE SCHEDULE xxv

2013-09-19 The real numbers and their properties. Readings: Chapter 4;Ferland Appendix B.

2013-09-24 Induction and recursion. Readings: 5.15.5; Ferland 4.1,4.3, and 4.5.

2013-09-26 Summation notation. Readings: 6.16.3; Ferland 4.2 and 4.4.

2013-10-01 Closed-form solutions to summations. Asymptotic notation.Readings: 6.4, Chapter 7; Ferland 10.4.

2013-10-03 Number theory: divisibility and greatest common divisors.Readings: 8.18.2.1; Ferland 3.13.2.

2013-10-08 Modular arithmetic. The extended Euclidean algorithm andinverses mod m. Readings: 8.2.2 and 8.4; Ferland 3.3 and 3.6.

2013-10-10 Various number theorems: The Fundamental Theorem of Arith-metic, the Chinese Remainder Theorem, Eulers Theorem. RSA en-cryption. Readings: 8.3 and 8.5.

2013-10-15 Start of relations: relations digraphs, and matrices; equiva-lence relations. Readings: Chapter 9 through 9.4; Ferland 5.1, sec-ond part of 5.2.

2013-10-17 Exam 1. The first exam was given at the usual class time inSCL 110. It was a closed-book exam covering all material discussedup to this point. Sample solutions are in Appendix B.1.

2013-10-22 Partial orders. Readings: 9.5 though 9.5.3; Ferland first partof 5.2.

2013-10-29 More on relations and digraphs. Minimal and maximal ele-ments and topological sort. A brief digression on well-ordered sets andtransfinite induction. Reflexive, symmetric, and transitive closures.Strongly-connected components and pre-orders. Readings: 9.5.49.6.

2013-10-31 More graph theory. Readings: Chapter 10; Ferland 8.1, 8.3,9.19.3, and 10.1.

2013-11-05 [[[ To be announced. ]]][[[ combinatorics ]]]

2013-11-07 [[[ To be announced. ]]]

LECTURE SCHEDULE xxvi

2013-11-12 [[[ To be announced. ]]][[[ probability ]]]

2013-11-14 [[[ To be announced. ]]]

2013-11-19 [[[ To be announced. ]]][[[ linear algebra ]]]

2013-11-21 [[[ To be announced. ]]]

2013-12-03 [[[ To be announced. ]]][[[ maybe polynomials and finitefields if there is time ]]]

2013-12-05 Exam 2. The second exam will be given at the usual classtime in SPL 59. It will be a closed-book exam covering all materialdiscussed during the semester. See Appendix D for examples of examsfrom previous semesters.

Chapter 1

Introduction

This is a course on discrete mathematics as used in Computer Science. Itsonly a one-semester course, so there are a lot of topics that it doesnt coveror doesnt cover in much depth. But the hope is that this will give you afoundation of skills that you can build on as you need to, and particularlyto give you a bit of mathematical maturitythe basic understanding ofwhat mathematics is and how mathematical definitions and proofs work.

1.1 So why do I need to learn all this nasty math-ematics?

Why you should know about mathematics, if you are interested in ComputerScience: or, more specifically, why you should take CS202 or a comparablecourse: Computation is something that you cant see and cant touch, and yet(thanks to the efforts of generations of hardware engineers) it obeysstrict, well-defined rules with astonishing accuracy over long periodsof time.

Computations are too big for you to comprehend all at once. Imagineprinting out an execution trace that showed every operation a typical$500 desktop computer executed in one (1) second. If you could readone operation per second, for eight hours every day, you would dieof old age before you got halfway through. Now imagine letting thecomputer run overnight.

So in order to understand computations, we need a language that allowsus to reason about things we cant see and cant touch, that are too big for us

1

CHAPTER 1. INTRODUCTION 2

to understand, but that nonetheless follow strict, simple, well-defined rules.Wed like our reasoning to be consistent: any two people using the languageshould (barring errors) obtain the same conclusions from the same infor-mation. Computer scientists are good at inventing languages, so we couldinvent a new one for this particular purpose, but we dont have to: the exactsame problem has been vexing philosophers, theologians, and mathemati-cians for much longer than computers have been around, and theyve had alot of time to think about how to make such a language work. Philosophersand theologians are still working on the consistency part, but mathemati-cians (mostly) got it in the early 20th-century. Because the first virtue of acomputer scientist is laziness, we are going to steal their code.

1.2 But isnt math hard?Yes and no. The human brain is not really designed to do formal mathe-matical reasoning, which is why most mathematics was invented in the lastfew centuries and why even apparently simple things like learning how tocount or add require years of training, usually done at an early age so thepain will be forgotten later. But mathematical reasoning is very close tolegal reasoning, which we do seem to be very good at.1

There is very little structural difference between the two sentences:

1. If x is in S, then x+ 1 is in S.

2. If x is of royal blood, then xs child is of royal blood.

But because the first is about boring numbers and the second is aboutfascinating social relationships and rules, most people have a much easiertime deducing that to show somebody is royal we need to start with someknown royal and follow a chain of descendants than they have deducing thatto show that some number is in the set S we need to start with some knownelement of S and show that repeatedly adding 1 gets us to the number wewant. And yet to a logician these are the same processes of reasoning.

So why is statement (1) trickier to think about than statement (1)? Partof the difference is familiaritywe are all taught from an early age what itmeans to be somebodys child, to take on a particular social role, etc. Formathematical concepts, this familiarity comes with exposure and practice,just as with learning any other language. But part of the difference is that

1For a description of some classic experiments that demonstrate this, see http://en.wikipedia.org/wiki/Wason_selection_task.


we humans are wired to understand and appreciate social and legal rules:we are very good at figuring out the implications of a (hypothetical) rulethat says that any contract to sell a good to a consumer for $100 or morecan be canceled by the consumer within 72 hours of signing it provided thegood has not yet been delivered, but we are not so good at figuring out theimplications of a rule that says that a number is composite if and only if itis the product of two integer factors neither of which is 1. Its a lot easier toimagine having to cancel a contract to buy swampland in Florida that yousigned last night while drunk than having to prove that 82 is composite. Butagain: there is nothing more natural about contracts than about numbers,and if anything the conditions for our contract to be breakable are morecomplicated than the conditions for a number to be composite.

1.3 Thinking about math with your heartThere are two things you need to be able to do to get good at mathematics(the creative kind that involves writing proofs, not the mechanical kindthat involves grinding out answers according to formulas). One of them isto learn the language: to attain what mathematicians call mathematicalmaturity. Youll do that in CS202, if you pay attention. But the other is tolearn how to activate the parts of your brain that are good at mathematical-style reasoning when you do maththe parts evolved to detect when theother primates in your band of hunter-gatherers are cheating.

To do this it helps to get a little angry, and imagine that finishing a proofor unraveling a definition is the only thing that will stop your worst enemyfrom taking some valuable prize that you deserve. (If you dont have a worstenemy, there is always the universal quantifier.) But whatever motivationyou choose, you need to be fully engaged in what you are doing. Your brainis smart enough to know when you dont care about something, and if youdont believe that thinking about math is important, it will think aboutsomething else.

1.4 What you should know about math1.4.1 Foundations and logicWhy: This is the assembly language of mathematicsthe stuff at the bottomthat everything else complies to.

Propositional logic.


Predicate logic. Axioms, theories, and models. Proofs. Induction and recursion.

1.4.2 Basic mathematics on the real numbersWhy: You need to be able to understand, write, and prove equations andinequalities involving real numbers.

Standard functions and their properties: addition, multiplication, ex-ponentiation, logarithms.

More specialized functions that come up in algorithm analysis: floor,ceiling, max, min.

Techniques for proving inequalities, including: General inequality axioms (transitivity, anti-symmetry, etc.) Inequality axioms for R (i.e., how < interacts with addition, mul-

tiplication, etc.) Techniques involving derivatives (assumes calculus): Finding local extrema of f by solving for f (x) = 0. Using f to distinguish local minima from local maxima. Using f (x) g(x) in [a, b] and f(a) g(a) or f(b) g(b)

to show f(x) g(x) in [a, b]. Special subsets of the real number: rationals, integers, natural num-bers.

1.4.3 Fundamental mathematical objectsWhy: These are the mathematical equivalent of data structures, the waythat more complex objects are represented.

Set theory. Naive set theory. Predicates vs sets.


Set operations. Set comprehension. Russells paradox and axiomatic set theory.

Functions. Functions as sets. Injections, surjections, and bijections. Cardinality. Finite vs infinite sets. Sequences.

Relations. Equivalence relations, equivalence classes, and quotients. Orders.

The basic number tower. Countable universes: N,Z,Q. (Can be represented in a com-

puter.) Uncountable universes: R,C. (Can only be approximated in a

computer.)

Other algebras. The string monoid. Z/m and Z/p. Polynomials over various rings and fields.

1.4.4 Modular arithmetic and polynomialsWhy: Basis of modern cryptography.

Arithmetic in Z/m. Primes and divisibility. Euclids algorithm and inverses. The Chinese Remainder Theorem.


Fermats Little Theorem and Eulers Theorem. RSA encryption. Galois fields and applications.

1.4.5 Linear algebraWhy: Shows up everywhere.

Vectors and matrices. Matrix operations and matrix algebra. Geometric interpretations. Inverse matrices and Gaussian elimination.

1.4.6 GraphsWhy: Good for modeling interactions. Basic tool for algorithm design.

Definitions: graphs, digraphs, multigraphs, etc. Paths, connected components, and strongly-connected components. Special kinds of graphs: paths, cycles, trees, cliques, bipartite graphs. Subgraphs, induced subgraphs, minors.

1.4.7 CountingWhy: Basic tool for knowing how much resources your program is going toconsume.

Basic combinatorial counting: sums, products, exponents, differences,and quotients.

Combinatorial functions. Factorials. Binomial coefficients. The 12-fold way.

Advanced counting techniques.


Inclusion-exclusion. Recurrences. Generating functions.

1.4.8 ProbabilityWhy: Cant understand randomized algorithms or average-case analysiswithout it. Handy if you go to Vegas.

Discrete probability spaces. Events. Independence. Random variables. Expectation and variance. Probabilistic inequalities.

Markovs inequality. Chebyshevs inequality. Chernoff bounds.

Stochastic processes. Markov chains. Martingales. Branching processes.

1.4.9 ToolsWhy: Basic computational stuff that comes up, but doesnt fit in any of thebroad categories above. These topics will probably end up being mixed inwith the topics above.

Things you may have forgotten about exponents and logarithms. Inequalities and approximations. and notation.


How to differentiate and integrate simple functions. Computing or approximating the value of a sum. Asymptotic notation.

Chapter 2

Mathematical logic

Mathematical logic is the discipline that mathematicians invented in the latenineteenth and early twentieth centuries so they could stop talking nonsense.Its the most powerful tool we have for reasoning about things that we cantreally comprehend, which makes it a perfect tool for Computer Science.

2.1 The basic pictureReality Model Theoryherds of sheeppiles of rocks N = {0, 1, 2, . . .} x : y : y = x+ 1tally marks

We want to model something we see in reality with something we can fitin our heads. Ideally we drop most of the features of the real thing that wedont care about and keep the parts that we do care about. But there is asecond problem: if our model is very big (and the natural numbers are veryvery big), how do we know what we can say about them?

2.1.1 Axioms, models, and inference rulesOne approach is to true to come up with a list of axioms that are true state-ments about the model and a list of inference rules that let us derive newtrue statements from the axioms. The axioms and inference rules togethergenerate a theory that consists of all statements that can be constructedfrom the axioms by applying the inference rules. The rules of the gameare that we cant claim that some statement is true unless its a theorem:something we can derive as part of the theory.

9

CHAPTER 2. MATHEMATICAL LOGIC 10

Simple example: All fish are green (axiom). George Washington is afish (axiom). From all X are Y and Z is X, we can derive Z is Y (inference rule). Thus George Washington is green (theorem). Since wecant do anything else with our two axioms and one inference rule, thesethree statements together form our entire theory about George Washington,fish, greenness, etc.

Theories are attempts to describe models. A model is typically a collec-tion of objects and relations between them. For a given theory, there may bemany models that are consistent with it: for example, a model that includesboth green fishy George Washington and MC 900-foot Abraham Lincoln isconsistent with the theory above, because the theory doesnt say anythingabout Abraham Lincoln.

2.1.2 ConsistencyA theory is consistent if it cant prove both P and not-P for any P . Con-sistency is incredibly important, since all the logics people actually use canprove anything starting from P and not-P .

2.1.3 What can go wrongIf we throw in too many axioms, you can get an inconsistency: All fish aregreen; all sharks are not green; all sharks are fish; George Washington is ashark gets us into trouble pretty fast.

If we dont throw in enough axioms, we underconstrain the model. Forexample, the Peano axioms for the natural numbers (see example below) say(among other things) that there is a number 0 and that any number x hasa successor S(x) (think of S(x) as x + 1). If we stop there, we might havea model that contains only 0, with S(0) = 0. If we add in 0 6= S(x) for anyx, then we can get stuck at S(0) = 1 = S(1). If we add yet another axiomthat says S(x) = S(y) if and only if x = y, then we get all the ordinarynatural numbers 0, S(0) = 1, S(1) = 2, etc., but we could also get someextras: say 0, S(0) = 1, S(1) = 0. Characterizing the correct naturalnumbers historically took a lot of work to get right, even though we all knowwhat we mean when we talk about them. The situation is of course worsewhen we are dealing with objects that we dont really understand; here themost we can hope for is to try out some axioms and see if anything strangehappens.

Better yet is to use some canned axioms somebody else has alreadydebugged for us. In this respect the core of mathematics acts like a system


libraryits a collection of useful structures and objects that are known towork, and (if we are lucky) may even do exactly what we expect.

2.1.4 The language of logicThe basis of mathematical logic is propositional logic, which was essen-tially invented by Aristotle. Here the model is a collection of statementsthat are either true or false. There is no ability to refer to actual things;though we might include the statement George Washington is a fish, fromthe point of view of propositional logic that is an indivisible atomic chunkof truth or falsehood that says nothing in particular about George Wash-ington or fish. If we treat it as an axiom we can prove the truth of morecomplicated statements like George Washington is a fish or 2+2=5 (truesince the first part is true), but we cant really deduce much else. Still, thisis a starting point.

If we want to talk about things and their properties, we must upgradeto predicate logic. Predicate logic adds both constants (stand-ins forobjects in the model like George Washington) and predicates (stand-insfor properties like is a fish). It also lets use quantify over variables andmake universal statements like For all x, if x is a fish then x is green. Asa bonus, we usually get functions (f(x) = the number of books GeorgeWashington owns about x) and equality (George Washington = 12 im-plies George Washington + 5 = 17). This is enough machinery to defineand do pretty much all of modern mathematics.

We will discuss both of these logics in more detail below.

2.1.5 Standard axiom systems and modelsRather than define our own axiom systems and models from scratch, it helpsto use ones that already have a track record of consistency and usefulness.Almost all mathematics fits in one of the following models:

The natural numbers N. These are defined using the Peano axioms,and if all you want to do is count, add, and multiply, you dont needmuch else. (If you want to subtract, things get messy.)

The integers Z. Like the naturals, only now we can subtract. Divisionis still a problem.

The rational numbers Q. Now we can divide. But what about 2? The real numbers R. Now we have 2. But what about (1)?


The complex numbers C. Now we are pretty much done. But what ifwe want to talk about more than one complex number at a time?

The universe of sets. These are defined using the axioms of set theory,and produce a rich collection of sets that include, among other things,structures equivalent to the natural numbers, the real numbers, col-lections of same, sets so big that we cant even begin to imagine whatthey look like, and even bigger sets so big that we cant use the usualaccepted system of axioms to prove whether they exist or not. Fortu-nately, in computer science we can mostly stop with finite sets, whichmakes life less confusing.

Various alternatives to set theory, like lambda calculus1, category the-ory2 or second-order arithmetic3. We wont talk about these much,since they generally dont let you do anything you cant do alreadywith sets.

In practice, the usual way to do things is to start with sets and then defineeverything else in terms of sets: e.g., 0 is the empty set, 1 is a particularset with 1 element, 2 a set with 2 elements, etc., and from here we work ourway up to the fancier numbers. The idea is that if we trust our axioms forsets to be consistent, then the things we construct on top of them shouldalso be consistent, although if we are not careful in our definitions they maynot be exactly the things we think they are.

2.2 Propositional logicPropositional logic is the simplest form of logic. Here the only statementsthat are considered are propositions, which contain no variables. Becausepropositions contain no variables, they are either always true or always false.

Examples of propositions:

2 + 2 = 4. (Always true). 2 + 2 = 5. (Always false).Examples of non-propositions:

x+ 2 = 4. (May be true, may not be true; it depends on the value ofx.)

1http://en.wikipedia.org/wiki/lambda_calculus2http://en.wikipedia.org/wiki/category_theory3http://en.wikipedia.org/wiki/second-order_arithmetic


x 0 = 0. (Always true, but its still not a proposition because of thevariable.)

x 0 = 1. (Always false, but not a proposition because of the variable.)As the last two examples show, it is not enough for a statement to be

always true or always falsewhether a statement is a proposition or not isa structural property. But if a statement doesnt contain any variables (orother undefined terms), it is a proposition, and as a side-effect of being aproposition its always true or always false.

2.2.1 Operations on propositionsPropositions by themselves are pretty boring. So boring, in fact, that logi-cians quickly stop talking about specific propositions and instead haul outplaceholder names like p, q, or r. But we can build slightly more interest-ing propositions by combining propositions together using various logicalconnectives, such as:

Negation The negation of p is written as p, or sometimes p or p. Ithas the property that it is false when p is true, and true when p isfalse.

Or The or of two propositions p and q is written as p q, and is true aslong as at least one, or possibly both, of p and q is true.4 This is notalways the same as what or means in English; in English, or oftenis used for exclusive or which is not true if both p and q are true. Forexample, if someone says You will give me all your money or I willstab you with this table knife, you would be justifiably upset if youturn over all your money and still get stabbed. But a logician wouldnot be at all surprised, because the standard or in propositional logicis an inclusive or that allows for both outcomes.

Exclusive or If you want to exclude the possibility that both p and q aretrue, you can use exclusive or instead. This is written as p q, andis true precisely when exactly one of p or q is true. Exclusive or isnot used in classical logic much, but is important for many computingapplications, since it corresponds to addition modulo 2 (see 8.4) and

4The symbol is a stylized V, intended to represent the Latin word vel, meaningor. (Thanks to Noel McDermott for remembering this.) Much of this notation is actu-ally pretty recent (early 20th century): see http://jeff560.tripod.com/set.html for asummary of earliest uses of each symbol.


has nice reversibility properties (e.g. p (p q) always has the sametruth-value as q).

And The and of p and q is written as p q, and is true only when both pand q are true.5 This is pretty much the same as in English, where Ilike to eat ice cream and I own a private Caribbean island is not atrue statement when made by most people even though most peoplelike to eat ice cream. The only complication in translating Englishexpressions into logical ands is that logicians cant tell the differencebetween and and but: the statement 2 + 2 = 4 but 3 + 3 = 6becomes simply (2 + 2 = 4) (3 + 3 = 6).

Implication This is the most important connective for proofs. An impli-cation represents an if. . . then claim. If p implies q, then we writep q or p q, depending on our typographic convention and theavailability of arrow symbols in our favorite font. In English, p qis usually rendered as If p, then q, as in If you step on your ownhead, it will hurt. The meaning of p q is that q is true wheneverp is true, and the proposition p q is true provided (a) p is false (inwhich case all bets are off), or (b) q is true.In fact, the only way for p q to be false is for p to be true but qto be false. Because of this, p q can be rewritten as p q. So,for example, the statements If 2 + 2 = 5, then Im the Pope, If Imthe Pope, then 2 + 2 = 4, and If 2 + 2 = 4, then 3 + 3 = 6, areall true, provided the if/then is interpreted as implication. NormalEnglish usage does not always match this pattern; instead, if/then innormal speech is often interpreted as the much stronger biconditional(see below).

Biconditional Suppose that p q and q p, so that either both p andq are true or both p and q are false. In this case, we write p qor p q, and say that p holds if and only if q holds. The truthof p q is still just a function of the truth or falsehood of p and q;though there doesnt need to be any connection between the two sidesof the statement, 2 + 2 = 5 if and only if I am the Pope is a truestatement (provided it is not uttered by the Pope). The only way forp q to be false is for one side to be true and one side to be false.

The result of applying any of these operations is called a compoundproposition.

5The symbol is a stylized A, short for the latin word atque, meaning and also.


NOT p p p,pp AND q p qp XOR q p qp OR q p q

p implies q p q p q, p qp if and only if q p q p q

Table 2.1: Compound propositions. The rightmost column gives alternateforms. Precedence goes from strongest for to weakest for (but see2.2.1.1 for a discussion of variation in conventions for this).

Table 2.1 shows what all of this looks like when typeset nicely. Note thatin some cases there is more than one way to write a compound expression.Which you choose is a matter of personal preference, but you should try tobe consistent.

2.2.1.1 Precedence

The short version: for the purposes of this course, we will use the ordering inTable 2.1, which corresponds roughly to precedence in C-like programminglanguages. But see caveats below. Remember always that there is no shamein putting in a few extra parentheses if it makes a formula more clear.

Examples: (p q r s t) is interpreted as ((((p) (q r)) s) t). Both OR and AND are associative, so (p q r) is the same as((p q) r) and as (p (q r)), and similarly (p q r) is the same as((p q) r) and as (p (q r)).

Note that this convention is not universal: many mathematicians giveAND and OR equal precedence, so that the meaning of p q r is ambigu-ous without parentheses. There are good arguments for either convention.Making AND have higher precedence than OR is analogous to giving mul-tiplication higher precedence than addition, and makes sense visually whenAND is written multiplicatively (as in pq qr for (p q) (q r). Mak-ing them have the same precedence emphasizes the symmetry between thetwo operations, which well see more about later when we talk about DeMorgans laws in 2.2.3. But as with anything else in mathematics, eitherconvention can be adopted, as long as you are clear about what you aredoing and it doesnt cause annoyance to the particular community you arewriting for.

There does not seem to be a standard convention for the precedence ofXOR, since logicians dont use it much. There are plausible arguments for


putting XOR in between AND and OR, but its probably safest just to useparentheses.

Implication is not associative, although the convention is that it bindsto the right, so that a b c is read as a (b c); except fortype theorists and Haskell programmers, few people ever remember this,so it is usually safest to put in the parentheses. I personally have no ideawhat p q r means, so any expression like this should be written withparentheses as either (p q) r or p (q r).

2.2.2 Truth tablesTo define logical operations formally, we give a truth table. This gives, forany combination of truth values (true or false, which as computer scientistswe often write as 1 or 0) of the inputs, the truth value of the output. In thisusage, truth tables are to logic what addition and multiplication tables areto arithmetic.

Here is a truth table for negation:

p p0 11 0

And here is a truth table for the rest of the logical operators:

p q p q p q p q p q p q0 0 0 0 0 1 10 1 1 1 0 1 01 0 1 1 0 0 01 1 1 0 1 1 1

See also [Fer08, 1.1], [Ros12, 1.11.2], or [Big02, 3.13.3].We can think of each row of a truth table as a model for propositional

logic, since the only things we can describe in propositional logic are whetherparticular propositions are true or not. Constructing a truth table corre-sponds to generating all possible models.

This can be useful if we want to figure out when a particular propositionis true. Proving a proposition using a truth table is a simple version ofmodel checking: we enumerate all possible models of a given collectionof simple propositions, and see if what we want to prove holds in all models.This works for propositional logic because the list of models is just the listof possible combinations of truth values for all the simple propositions P ,


Q, etc. We can check that each truth table we construct works by checkingthat the truth values each column (corresponding to some subexpression ofthe thing we are trying to prove) follow from the truth values in previouscolumns according to the rules established by the truth table defining theappropriate logical operation.

For predicate logic, model checking becomes more complicated, becausea typical system of axioms is likely to have infinitely many models, many ofwhich are likely to be infinitely large. There we will need to rely much moreon proofs constructed by applying inference rules.

2.2.3 Tautologies and logical equivalenceA compound proposition that is true no matter what the truth-values of thepropositions it contains is called a tautology. For example, p p, p p,and (p p) are all tautologies, as can be verified by constructing truthtables. If a compound proposition is always false, its a contradiction. Thenegation of a tautology is a contradiction and vice versa.

The most useful class of tautologies are logical equivalences. This is atautology of the form X Y , where X and Y are compound propositions.In this case, X and Y are said to be logically equivalent and we cansubstitute one for the other in more complex propositions. We write X Yif X and Y are logically equivalent.

The nice thing about logical equivalence is that is does the same thing forBoolean formulas that equality does for algebraic formulas: if we know (forexample), that pp is equivalent to 1, and q 1 is equivalent to q, we cangrind qpp q1 1 without having to do anything particularly clever.(We will need cleverness later when we prove things where the consequentisnt logically equivalent to the premise.)

To prove a logical equivalence, one either constructs a truth table to showthat X Y is a tautology, or transforms X to Y using previously-knownlogical equivalences.

Some examples:

p p 0: Construct a truth tablep p p p 00 1 0 01 0 0 0

and observe that the last two columns are always equal.


p p p: Use the truth tablep p p0 01 1

p q p q: Again construct a truth tablep q p q p q0 0 1 10 1 1 11 0 0 01 1 1 1

(p q) pq: (one of De Morgans laws; the other is (p q) p q).

p q p q (p q) p q p q0 0 0 1 1 1 10 1 1 0 1 0 01 0 1 0 0 1 01 1 1 0 0 0 0

p (q r) (p q) (p r) (one of the distributive laws; the other isp (q r) (p q) (p r)).

p q r q r p (q r) p q p r (p q) (p r)0 0 0 0 0 0 0 00 0 1 0 0 1 0 00 1 0 0 0 0 1 00 1 1 1 1 1 1 11 0 0 0 1 1 1 11 0 1 0 1 1 1 11 1 0 0 1 1 1 11 1 1 1 1 1 1 1

(p r) (q r) (p q) r. Now things are getting messy,so building a full truth table may take awhile. But we have take ashortcut by using logical equivalences that weve already proved (plus


associativity of ):

(p r) (q r) (p r) (q r) [Using p q p q twice] p q r r [Associativity and commutativity of ] p q r [p p p] (p q) r [De Morgans law] (p q) r. [p q p q]

This last equivalence is a little surprising. It shows, for example, thatif somebody says It is either the case that if you study you will graduatefrom Yale with distinction, or that if you join the right secret society youwill graduate from Yale with distinction, then this statement (assumingwe treat the or as ) is logically equivalent to If you study and join theright secret society, then you will graduate from Yale with distinction. It iseasy to get tangled up in trying to parse the first of these two propositions;translating to logical notation and simplifying using logical equivalence is agood way to simplify it.

Over the years, logicians have given names to many logical equivalences.Some of the more useful ones are summarized in Table 2.2. More complicatedequivalences can often be derived from these. If that doesnt work (and youdont have too many variables to work with), you can always try writing outa truth table.

2.2.3.1 Inverses, converses, and contrapositives

The contrapositive of p q is q p; it is logically equivalent to theoriginal implication. For example, the contrapositive of If I am BarackObama then I am a Democrat is If I am not a Democrat then I am notBarack Obama. A proof by contraposition demonstrates that p impliesq by assuming q and then proving p; it is similar but not identical to anindirect proof, which assumes p and derives a contradiction.

The inverse of p q is p q. So the inverse of If you take CPSC202, you will surely die is If you do not take CPSC 202, you will not surelydie. There is often no connection between the truth of an implication andthe truth of its inverse: If I am Barack Obama then I am a Democratdoes not have the same truth-value as If I am not Barack Obama then Iam not a Democrat, at least according to current polling numbers.

The converse of p q is q p. E.g. the converse of If I am BarackObama then I am a Democrat is If I am a Democrat then I am Barack


p p Double negation(p q) p q De Morgans law(p q) p q De Morgans law

p q q p Commutativity of ANDp q q p Commutativity of OR

p (q r) p (q r) Associativity of ANDp (q r) p (q r) Associativity of ORp (q r) (p q) (p r) AND distributes over ORp (q r) (p q) (p r) OR distributes over AND

p q p q Equivalence of implication and ORp q q p Contrapositionp q (p q) (q p) Expansion of if and only ifp q p q Inverse of if and only fp q q p Commutativity of if and only if

Table 2.2: Common logical equivalences (see also [Fer08, Theorem 1.1])


Obama. The converse of a statement is always logically equivalent to theinverse. Often in proving a biconditional (e.g., I am Barack Obama if andonly if I am a Democrat), one proceeds by proving first the implication inone direction and then either the inverse or the converse (which are logicallyequivalent).

2.2.3.2 Equivalences involving true and false

Any tautology is equivalent to true; any contradiction is equivalent to false.Two important cases of this are the law of the excluded middle

P P 1

and its dual, the law of non-contradiction

P P 0.

The law of the excluded middle is what allows us to do case analysis, wherewe prove that some proposition Q holds by showing first that P implies Qand then that P also implies Q.6

6Though we will use the law of the excluded middle, it has always been a little bitcontroversial, because it is non-constructive: it tells you that one of P or P is true,but it doesnt tell you which.

There is a variant of classical logic called intuitionistic logic where the law of theexcluded middle does not hold. The reason for the controversy is that the law of theexcluded middle is non-constructive: it tells you that one of P or P is true, but itdoesnt tell you which.

Though intuititionistic logic might seem like a curiosity, there is a deep connection be-tween computer programs and proofs in intuitionistic logic, known as the Curry-Howardisomorphism. The idea is that you get intuitionistic logic if you interpret P as an object of type P ; P Q as a function that takes a P as an argument and returns a Q; P Q as an object that contains both a P and a Q (like a struct in C); P Q as an object that contains either a P or a Q (like a union in C); and P as P , a function that given a P produces a special error value that

cant otherwise be generated.With this interpretation, many theorems of classical logic continue to hold. For example,

modus ponens says(P (P Q)) Q.

Seen through the Curry-Howard isomorphism, this means that there is a function that,given a P and a function that generates a Q from a P , generates a Q. For example, thefollowing Scheme function:


P 0 0 P 0 PP 1 P P 1 1P 0 P P 0 PP 1 P P 1 PP 0 P 0 P 1P 1 1 1 P P

Table 2.3: Absorption laws. The first four are the most important. Notethat , , , and are all commutative, so reversed variants also work.

One strategy for simplifying logical expressions is to try to apply knownequivalences to generate sub-expressions that reduce to true or false viathe law of the excluded middle or the law of non-contradiction. These canthen be absorbed into nearby terms using various absorption laws, shown inTable 2.3.

Example Lets show that (P (P Q)) Q is a tautology. (Thisjustifies the inference rule modus ponens, defined below.) Working from the

(define (modus-ponens p p-implies q) (p-implies-q p))Similarly, in a sufficiently sophisticated programming language we can show P P ,

since this expands to P ((P ) ), and we can write a function that takes a Pas its argument and returns a function that takes a P function and feeds the P to it:

(define (double-negation p) (lambda (p-implies-fail)(p-implies-fail p)))

But we cant generally show P P , since there is no way to take a function of type(P ) and extract an actual example of a P from it. Nor can we expect to showP P , since this would require exhibiting either a P or a function that takes a P andproduces an error, and for any particular type P we may not be able to do either.For normal mathematical proofs, we wont bother with this, and will just assume PP

always holds.


inside out, we can compute

(P (P Q)) Q (P (P Q)) Q expand ((P P ) (P Q)) Q distribute over (0 (P Q)) Q non-contradiction (P Q) Q absorption (P Q) Q expand (P Q) Q De Morgans law P (Q Q) associativity P 1 excluded middle 1 absorption

In this derivation, weve labeled each step with the equivalence we used.Most of the time we would not be this verbose.

2.2.4 Normal formsA compound proposition is in conjuctive normal form (CNF for short)if it is obtained by ANDing together ORs of one or more variables or theirnegations (an OR of one variable is just the variable itself). So for exampleP , (P Q) R, (P Q) (Q R) (P ), and (P Q) (P R) (P Q S T U) are in CNF, but (P Q) (P R) (P Q),(P Q) (P R) (P Q), and (P (Q R)) (P R) (P Q)are not. Using the equivalence P Q P Q, De Morgans laws, andthe distributive law, it is possible to rewrite any compound proposition inCNF.

CNF formulas are particularly useful because they support resolution(see 2.4.1). Using the tautology (P Q) (P R) Q R, we canconstruct proofs from CNF formulas by looking for occurrences of somesimple proposition and its negation and resolving them, which generates anew clause we can add to the list. For example, we can compute

(P Q) (P R) (P Q) (Q R)`(P Q) (P R) (P Q) (Q R) Q`(P Q) (P R) (P Q) (Q R) Q R`(P Q) (P R) (P Q) (Q R) Q R P`P.


This style of proof is called a resolution proof. Because of its simplicityit is particularly well-suited for mechanical theorem provers. Such proofscan also encode traditional proofs based on modus ponens: the inferenceP (P Q) ` Q can be rewritten as resolution by expanding to getP (P Q) ` Q.

Similarly, a compound proposition is in disjunctive normal form(DNF) if it consists of an OR of ANDs, e.g. (P Q) (P R) (P Q).Just as any compound proposition can be transformed into CNF, it cansimilarly be transformed into DNF.

Note that conjunctive normal forms are not unique; for example, P Qand (P Q) (P Q) (P Q) are both in conjunctive normal form andare logically equivalent to each other. So while CNF can be handy as a wayof reducing the hairiness of a formula (by eliminating nested parenthesesor negation of non-variables, for example), it doesnt necessarily let us seeimmediately if two formulas are really the same.

2.3 Predicate logicUsing only propositional logic, we can express a simple version of a famousargument:

Socrates is a man. If Socrates is a man, then Socrates is mortal. Therefore, Socrates is mortal.This is an application of the inference rule calledmodus ponens, which

says that from p and p q you can deduce q. The first two statements areaxioms (meaning we are given them as true without proof), and the last isthe conclusion of the argument.

What if we encounter Socratess infinitely more logical cousin Spocrates?Wed like to argue

Spocrates is a man. If Spocrates is a man, then Spocrates is mortal. Therefore, Spocrates is mortal.

Unfortunately, the second step depends on knowing that humanity im-plies mortality for everybody, not just Socrates. If we are unlucky in our


choice of axioms, we may not know this. What we would like is a generalway to say that humanity implies mortality for everybody, but with justpropositional logic, we cant write this fact down.

2.3.1 Variables and predicatesThe solution is to extend our language to allow formulas that involve vari-ables. So we might let x, y, z, etc. stand for any element of our universe ofdiscourse or domainessentially whatever things we happen to be talkingabout at the moment. We can now write statements like:

x is human. x is the parent of y. x+ 2 = x2.These are not propositions because they have variables in them. Instead,

they are predicates; statements whose truth-value depends on what con-crete object takes the place of the variable. Predicates are often abbreviatedby single capital letters followed by a list of arguments, the variables thatappear in the predicate, e.g.:

H(x) = x is human. P (x, y) = x is the parent of y. Q(x) = x + 2 = x2.We can also fill in specific values for the variables, e.g. H(Spocrates) =

Spocrates is human. If we fill in specific values for all the variables, wehave a proposition again, and can talk about that proposition being true(e.g. H(2) and H(1) are true) or false (H(0) is false).

In first-order logic, which is what we will be using in this course, vari-ables always refer to things and never to predicates: any predicate symbolis effectively a constant. There are higher-order logics that allow variablesto refer to predicates, but most mathematics accomplishes the same thingby representing predicates with sets (see Chapter 3).

2.3.2 QuantifiersWhat we really want is to be able to say when H or P or Q is true for manydifferent values of their arguments. This means we have to be able to talk


about the truth or falsehood of statements that include variables. To do this,we bind the variables using quantifiers, which state whether the claim weare making applies to all values of the variable (universal quantification),or whether it may only apply to some (existential quantification).

2.3.2.1 Universal quantifier

The universal quantifier (pronounced for all) says that a statementmust be true for all values of a variable within some universe of allowedvalues (which is often implicit). For example, all humans are mortal couldbe written x : Human(x) Mortal(x) and if x is positive then x + 1 ispositive could be written x : x > 0 x+ 1 > 0.

If you want to make the universe explicit, use set membership notation,e.g. x Z : x > 0 x + 1 > 0. This is logically equivalent to writingx : x Z (x > 0 x + 1 > 0) or to writing x : (x Z x > 0) x+ 1 > 0, but the short form makes it more clear that the intent of x Zis to restrict the range of x.7

The statement x : P (x) is equivalent to a very large AND; for example,x N : P (x) could be rewritten (if you had an infinite amount of paper)as P (0) P (1) P (2) P (3) . . . . Normal first-order logic doesnt allowinfinite expressions like this, but it may help in visualizing what x : P (x)actually means. Another way of thinking about it is to imagine that x issupplied by some adversary and you are responsible for showing that P (x)is true; in this sense, the universal quantifier chooses the worst case valueof x.

2.3.2.2 Existential quantifier

The existential quantifier (pronounced there exists) says that a state-ment must be true for at least one value of the variable. So some human ismortal becomes x : Human(x)Mortal(x). Note that we use AND ratherthan implication here; the statement x : Human(x) Mortal(x) makesthe much weaker claim that there is some thing x, such that if x is human,then x is mortal, which is true in any universe that contains an immortalpurple penguinsince it isnt human, Human(penguin) Mortal(penguin)is true.

As with , can be limited to an explicit universe with set membershipnotation, e.g., x Z : x = x2. This is equivalent to writing x : x Z x = x2.

7Programmers will recognize this as a form of syntactic sugar.


The formula x : P (x) is equivalent to a very large OR, so that x N :P (x) could be rewritten as P (0)P (1)P (2)P (3) . . . . Again, you cantgenerally write an expression like this if there are infinitely many terms, butit gets the idea across.

2.3.2.3 Negation and quantifiers

The following equivalences hold:

x : P (x) x : P (x).x : P (x) x : P (x).These are essentially the quantifier version of De Morgans laws: the first

says that if you want to show that not all humans are mortal, its equivalentto finding some human that is not mortal. The second says that to showthat no human is mortal, you have to show that all humans are not mortal.

2.3.2.4 Restricting the scope of a quantifier

Sometimes we want to limit the universe over which we quantify to somerestricted set, e.g., all positive integers or all baseball teams. Weve previ-ously seen how to do this using set-membership notation, but can also dothis for more general predicates either explicitly using implication:

x : x > 0 x 1 0or in abbreviated form by including the restriction in the quantifier expres-sion itself:

x > 0 : x 1 0.Similarly

x : x > 0 x2 = 81can be written as

x > 0 : x2 = 81.Note that constraints on get expressed using AND rather than impli-

cation.The use of set membership notation to restrict a quantifier is a special

case of this. Suppose we want to say that 79 is not a perfect square, by whichwe mean that there is no integer whose square is 79. If we are otherwisetalking about real numbers (two of which happen to be square roots of 79),we can exclude the numbers we dont want by writing

x Z : x2 = 79


which is interpreted as

x : (x Z x2 = 79)

or, equivalentlyx : x Z x2 6= 79.

Here Z = {. . . ,2,1, 0, 1, 2, . . .} is the standard set of integers.For more uses of , see Chapter 3.

2.3.2.5 Nested quantifiers

It is possible to nest quantifiers, meaning that the statement bound by aquantifier itself contains quantifiers. For example, the statement there isno largest prime number could be written as

x : (Prime(x) y : y > x Prime(y))

i.e., there does not exist an x that is prime and any y greater than x is notprime. Or in a shorter (though not strictly equivalent) form:

xy : y > x Prime(y)

which we can read as for any x there is a bigger y that is prime.To read a statement like this, treat it as a game between the player

and the player. Because the comes first in this statement, the for-allplayer gets to pick any x it likes. The exists player then picks a y to makethe rest of the statement true. The statement as a whole is true if the player always wins the game. So if you are tr

disc

Documents

field axioms

order axioms

composition of functions

simple induction

strong induction

structural induction

inference rules92

inference rules352