Lecture 6: Signal Processing III

EEN 112: Introduction to Electrical and Computer Engineering

Professor Eric Rozier, 2/25/13

PIGEONS AND HOLES

Pigeonholes

The Pigeonhole Principle

• First formalized by Johann Dirichlet in 1834– Schubfachprinzip (drawer principle)

• Given n items, which must be put into m pigeonholes, with n > m, at least one pigeon hole must contain more than one item.

The Pigeonhole Principle

• Seems simple, right? But has some non-obvious consequences.

• A typical person has aroung 150,000 hairs. – A reasonable assumption is that no one has more

than 1,000,000 hairs.– All people have between 0 and 1,000,000 hairs.– There are 5,564,635 people in Miami– Consequences?

The Pigeonhole Principle

• The Birthday Paradox

• How likely is it that two people in our class share the same birthday?

• How would we know?

The Pigeonhole Principle

• How many “holes” do we have that can be filled?

• Each person is equally likely to inhabit any one hole.

Birthday Probabilities

Birthday Probability

• Imagine everyone has a deck of cards with 365 possible values. We each draw independently.

• Let’s think about the likelyhood…

Pigeons and Holes

• We have “pigeons” in signal processing, and “holes” we want to put them into.

Pigeons and Holes

• In a N-bit system, how many holes do we have?

Pigeons and Holes

• Think of the bits as labels we put on the holes, and k as the decimal number equivalent. Our classification rule gives us a way to know what hole to put each pigeon into… and we have a LOT of pigeons…

Labeling our Pigeonholes

• We can label our pigeon holes with decimal integers– This is what k is in our equation

• But why use decimals? What are decimals?

Numeral Systems

• In mathematics, we talk about the base of a numeral system. Decimals are a base-10 numeral system.– Why?

Numeral Systems

• Decimal uses 10 numerals– 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

– Once we exhaust the numerals, we add a more significant digit

– 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20

– 100, 101, 102, 103, 104, 105, 106, 107, 108, 109

Numeral Systems

• What base is binary? Why?

Numeral Systems

• Binary enumeration– 0, 1– 10, 11– 100, 101– 110, 111

There are 10 types of people in this world.

Those who can count in binary and those who can’t!

Numeral Systems

• We can pick any base we want, even large than base-10!– Hexadecimal, base-16– 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F– (Actually a very useful system in ECE…)

Numeral SystemsHexidecimal Binary Decimal

0 0000 0

1 0001 1

2 0010 2

3 0011 3

4 0100 4

5 0101 5

6 0110 6

7 0111 7

8 1000 8

9 1001 9

A 1010 10

B 1011 11

C 1100 12

D 1101 13

E 1110 14

F 1111 15

3-bits worth of Pigeonholes

Decimal number (k) Binary number

0 0

1 1

2 10

3 11

4 100

5 101

6 110

7 111

Classification Rule

• Let’s say we have one pigeon for every real number between 0 and 1.

• How many pigeons?– Actually we have more than simply an infinite

number of pigeons…– We have uncountably infinite pigeons

Thinking about infinity

• Let’s say we had a number of pigeonholes equal to the cardinality of the set of natural numbers (0, 1, 2, …). How many do we have?

• Let’s say we have a number of pigeons equal to the cardinality of the set of integers (…, -2, -1, 0, 1, 2, …)

• Do we have a hole for each pigeon?

Thinking about infinity

• Let’s say we had a number of pigeonholes equal to the cardinality of the set of natural numbers (0, 1, 2, …). How many do we have?

• Let’s say we have a number of pigeons equal to the cardinality of the set of real numbers (…, -1, …, -0.333333, …, 0, …, 1, …, 2.9756, …)

• Do we have a hole for each pigeon?

Ordinal Numbers

Thinking about Infinity

• Countably infinite

•Uncountably infinite - c

Quantization

• Classification and Reconstruction

Types of Functions

• Functions can be classified by how the elements of the domain and codomain relate

• F: X -> Y

Types of functions

• Injective (one-to-one)– Preserves distinctiveness

Types of functions

• Surjective (onto)– Every element

Types of functions

• Bijection (both)– Injective and surjective

Quantization

• Quantization is surjective