david m keil, framingham state university csci 460 theory of

1. Sets, languages, and models David Keil Theory of computing 6/12

David Keil Theory of Computing 12/12 1

1. Sets, languages,

and models 1. Countable sets and formal languages

2. Uncountable sets

3. Streams and coinduction

4. Circuit and lookup computation

David M Keil, Framingham State University

CSCI 460 Theory of Computing

Inquiry

• Are the sets of natural numbers and

strings of comparable size?

• Are all infinite sets the same size?

• Can we define infinite sets of infinitely

large data streams?



Objectives


1a. Use the notation of formal language

theory*

1b. Describe the logic-circuit model of

computation*

1c. Prove that a set is countable

1d. Prove that a set is uncountable

1e. Explain or write a coinductive definition

How Topic 1 relates to computing

• We show that some problems are

algorithmically unsolvable

• We show a diagonal proof method that is

used again in topic 4

• Computation deals with finite, infinite, and

uncountable objects or sets

• Computations are operations on sets of

strings (languages)



• How big is a set?

• What does countability have to do with

computing?

• How many of the following exist?

– natural numbers

– rational numbers

– real numbers

– functions

– programs

– sets David Keil Theory of Computing 12/12 5

1. Countable sets and formal languages

Countability • The natural numbers are countable, i.e.,

they can be arranged in a particular order

• A bijection is a one-to-one correspondence,

i.e., a relation between two sets that is both

surjective and injective

• A set is countable iff there exists a bijection

between it and the natural numbers (ℕ)

• Countable sets tend to be sets of finite

objects, defined inductively



Peano’s axioms: definition by induction

1. 0 is a natural number (0 N)

2. Every natural number n has a unique

successor, n, also a natural number

(n N) n N

3. All natural numbers follow (1) or (2)

(n N) n = 0 (m N) n = m

• Significance: These axioms, or assumptions, provide a formal logical basis to work with counting numbers. Note: (2) is recursive

• Computation is a formal way to manipulate numbers and objects representable by them.

7 David Keil Theory of Computing 12/12

Theorem: ℕ is infinite Proof by contradiction:

1. Suppose ℕ is finite

2. Then let n be the largest element of ℕ

3. Consider the number (n + 1). It cannot be in ℕ,

because it is larger than ℕ’s largest element

4. But by definition of ℕ, any successor of an

element of ℕ is in ℕ.

5. Hence by the contradiction of (3) and (4), we

must reject (1).

6. Hence ℕ is infinite.



Strings • Alphabet: a finite set of symbols, e.g.,

{0,1},

{0, 1, …9}, {a, b, c, …, z}

• A string is a sequence of symbols over a

finite alphabet

• Notation:

– xR is the string x reversed

– n0(x) is the number of zeroes in string x


The set of strings over an alphabet

• * is the set of strings over finite alphabet

• Recursive definition:

– Base: * ( = = null string)

– Recursive: a s * sa *

– Restriction: only objects defined as above

are strings

• In set notation,

* = {} { ax | a , x *}



A bijection ℕ {0,1}*


Str(n) =

if n = 0

Str(n/2) + ‘0’ if Odd(n) n > 0

Str(n/2) + ‘1’ otherwise

Num(s) =

0 if s =

2 Num(s[1 .. |s| - 1]) +

s[|s|] + 1 otherwise

• This bijection is the core of a proof that

{0,1}* is countable

Cardinalities of sets • Informally, their sizes

• For infinite sets, we say that two sets have

the same cardinality iff there is a bijection

(1-to-1 correspondence) between their

elements

• Example: Card (EVEN) = Card (ℕ)

Proof: EVEN0 = 0, EVEN1 = 2, etc.

• Example: Card (ℚ) = Card (ℕ)

Proof: Zig-zag through a table of rationals




Formal languages • Language: a set of strings over an alphabet

• Languages may be enumerated:

, 0, 1, 00, 01, 10, 11, …

• * is the set of all strings over

• For any language L over , L *

• Inductively, * = {xy | x y *}

Defining languages

• 0n1n is the language of strings with n zeroes

followed by n ones

• Let alphabet be {0,1}, let be the null

string

• Then 0 = {}, 1 = {(0), (1)}

2 = {00, 01, 10, 11}

k = the set of strings of length k

* = Uk N k



Operations on languages

• Concatenation (L1 L2 ): strings that are a

string in L1 followed by one in L2

• L1 L2 = {xy | x L1 y L2}

• Union (selection, L1 L2, L1 | L2):

strings either in L1 or L2

L1 L2 = {x | x L1 x L2}

• Intersection (L1 L2) strings that are in

both L1 and L2


Kleene star languages

• Kleene star (Iteration, L*): strings that are

sequences of strings in L

L* = {xy | x L y L*}

• Alternative definition:

– L0 = {}, L1 = L, L2 = LL, L3 = LLL, etc.

– LL ={xx | x L}

– L* = Uk N Lk

– Example: If L = {0, 00} then

LL = {00, 000, 0000}



Countability of strings • To help prove equivalence of number-based

and string-based models of computation, we

show mappings of any string to a natural

number and any natural number to a string

• Enumeration according to bit representation:

N {0,1}* 0

1 0

2 1

3 00

4 01


Languages and countability

• Computation operates on strings;

instances of models of computation

accept sets of strings

• A language may be represented as an

infinite bit sequence, with a 1 bit in a

certain bit location denoting a string’s

membership in the language:

{0, 01, 11, 100} = 10101100…



Countable set: the rational numbers

• The rationals may

be enumerated as

at right, where the

numerator is the

row number and

the denominator is

the column number

• Reference: “The set of rational numbers is countable,”

(http://www.homeschoolmath.net/teaching/rational-

numbers-countable.php)


2. Uncountable sets

• Are all infinite sets the same size?

• Does infinity matter in the real

finite world?



Uncountability

• We show that some sets, such as the

real numbers, are not countable

• We prove this by showing that no

bijection between such a set and the set

of natural numbers can exist

• This proof and its method imply limits

on the power of algorithmic

computation


The set of real numbers • Intuitively, the real numbers express analog or

continuous quantities; they give all the

possible weights or distances.

• We may consider the size of the set ℝ, of reals,

as corresponding to all the points on a line

segment, line, plane, space, or space/time

• Another definition: The reals are all the

numbers that can be expressed using an

infinite sequence of digits after the decimal or

binary point



Theorem: |ℝ| > |ℕ| Proof (Cantor) :

1.Suppose ℝ were countable

2.Then the interval (0, 1] could be enumerated as:

r1 = 0 . b1,1 b1,2 b1,3 …

r2 = 0 . b2,1 b2,2 b2,3 …

… where bm,n {0,1}

3.Now, consider the real number s =

0. b1,1 b2,2 b3,3 …

That is, for all n, bit n is bn,n (bitwise negation

of the diagonal of r)


Cantor’s proof (cont’d)

4. Now, since for every i, the ith bit of s is

different from the ith bit of ri, so s differs

from every element of r, so there is no

element of sequence r that matches s

5. Hence r does not contain s, which is

clearly a real number

6. Hence we have a contradiction and must

reject the supposition

7. Hence ℝ is not countable



Cantor’s proof, illustrated Supposed enumeration of ℝ:

r1 = (0, 0, 0, 0, 0, 0, 0, ...)

r2 = (1, 1, 1, 1, 1, 1, 1, ...)

r3 = (0, 1, 0, 1, 0, 1, 0, ...)

r4 = (1, 0, 1, 0, 1, 0, 1, ...)

r5 = (1, 1, 0, 1, 0, 1, 1, ...)

r6 = (0, 0, 1, 1, 0, 1, 1, ...)

r7 = (1, 0, 0, 0, 1, 0, 0, ...)

... David Keil Theory of Computing 12/12 25

Real number that differs

with every element in

enumeration in at least

one bit:

s = (1, 0, 1, 1, 1, 0, 1, ...)

Summary of Cantor’s proof

• “The proof shows that no matter how a list

of real numbers were arranged, a real

number could still be defined that would not

be in the list”

T. DiRienzo, J. Bartlett, 2/09

• A diagonal proof is both by construction and

by contradiction



The predicates are uncountable • Start with the set of functions

{ f : {0} {0,1}}

• This is a set of two functions:

{{(0,1)}, {(0,0)}}

• The set { f : {0, 1} {0,1}} has four elements,

{ f : {0, 1, 2} {0,1}} has eight, etc.

• There are 2|ℕ| predicates f : ℕ {0,1}

• Question: Is this the same as, or more than, the

cardinality of the natural numbers?

• Compare with the cardinality of reals


Can all predicates on ℕ be

computed in Java? 1. Enumerate all Java methods that take an integer

parameter and return 0 or 1, as J1, J2, J3, … in a

bitwise ordering

2. Consider the predicate f : ℕ {0,1} s.t. f (n) = 1

if Jn(n) = 0 or Jn(n) hangs, f (n) = 0 if Jn(n) = 1

3. The predicate f differs in its behavior from each

element of our enumeration of Java methods

4. Hence f is a predicate not computed by any Java

method



Results

• ℕ is countable (cardinality 0, aleph-null)

• ℝ is uncountable (cardinality 1)

• The set of Java methods is countable using

their bit representations

• The predicates f : ℕ {0,1} are uncountable

• There are 20 = 1 predicates on ℕ

• The sets of natural numbers are uncountable

(see handout)


3. Streams and coinduction


• Can we define infinite sets of infinitely

large data streams?

• How can infinite I/O be described

mathematically?

• What is induction without a base case?


Streams

• A reactive system may receive an infinite

stream of inputs and may emit the

corresponding infinite stream of outputs

• Sets of streams express interactive behavior

• Coinduction defines sets of infinite objects

as induction defines sets of finite ones


ProgramEnvironmentinputs

Interaction stream

outputs

Inductively defined sets • (i) 0 is a natural number;

(ii) Every n ℕ has a unique successor, n; (iii) i and ii are the only ways to obtain a

natural number (Peano)

• Set of expressions using numerals, +, and ( ):

expression numeral |

numeral + expression | ( expression )

• The definition of such a set may use itself

• But such a set does not contain itself




Inductive and coinductive definitions of languages

• * = {} U {ax | a , x *}

(note base case )

• “L is a language over alphabet *”

means that L *

• A stream is an infinite string

• A stream language is a set of streams

• Example, defined coinductively:

= {ax | a , x }

(note lack of base case)

Well-founded sets • (B. Russell) Let the village hair stylist be

the person who cuts the hair of everyone

who doesn’t cut own hair

• Question: Who cuts the hair-stylist’s hair?

• Similar Questions: Is there a set of all sets?

If so, does it contain itself ? Is there a set of

all sets that don't contain themselves?

• In classical (well founded) set theory, it is

meaningless to say a set belongs or doesn't

belong to itself (Foundation Axiom)




Another diagonal proof • Theorem: The notion “set of all sets” leads

to a contradiction

• Proof: Consider = { A : A A } (sets not members of themselves)

• ( ) ( ) , ( ) ( ) , contradictions

• Note role of “Spoiler” instance

• We say that , are not well founded (NWF)

Streams and non-well-founded sets

• A non-well-founded set may contain itself,

directly or indirectly

• Example: A = { B, C }; B = { A, D }

• Every stream of characters is a character,

followed by a stream of characters

• Streams contain streams, which contain

streams, which contain streams, ...

• Ex.: The set of all bit streams {0,1} consists

of any infinite sequence that consists of 0 or 1

followed by a stream



Streams and coinduction

• Induction defines countable sets of finite

objects: * = {} { ax | a , x *}

• Coinduction

– is a dual of induction (recursion), lacking a base case

– is used to define sets of infinite objects

• The set of streams over an alphabet, express-

ing ongoing processes such as interaction:

= { ax | a , x }


Least and greatest fixed points • A fixed point x of function f is a value x

(which may be a set), for which f (x) = x

• Whereas induction is characterized by

minimality conditions (least fixed points),

coinduction has a maximality condition

(greatest fixed point)

• Minimality example: + is the smallest set that

fits the spec {ax | a , x *}

• Maximality example: is the largest set that

fits the spec {ax | a , x }



• What is a function on a finite set?

• What is a property of a finite set?

• How can a function on a finite domain be

defined?

• What’s a way to compute such a function?

• What’s a simple model of simple

computation?


4. Circuit and lookup computation

Problems and languages • A computational problem with a Boolean

(yes/no) solution is called a decision problem

• Computing devices that solve them are called

acceptors or recognizers

• A decision problem corresponds to a

language: all the strings that get a “yes”

decision

• Other problems, with string solutions, are

called transduction problems



Decision and transduction • A decision problem requires yes/no responses

to input

• Hence it is also known as a language-

recognition problem

• A transduction computation produces a string

as output

• Hence its solution is the computation of a

function

• We speak of decidable languages (problems)

or computable functions


Some language-decision problems

• Some languages:

– 1 | 0 {0, 1}

– (1 | 0) 1 {11, 01}

– 1* {, 1, 11, …}

– 1*0 {0, 10, 110, 1110, …}

– 0(1|0)* {0, 00, 01, 000, 001,

010, 011, …}

• Q: How do we build a simple machine to

accept strings in one of these languages?



No-loop computation • The simplest type of computation uses no

loops, but maps from finite domains

• Examples: lookup table; logic circuit

• Other equivalent models:

– Finite computation trees

– Flowcharts with bounded input, no loops

– Formulas in propositional logic

– Finite languages L k for any or k


Logic circuits • A model for the computation

of functions f : {0,1}m {0,1}n

where m is the arity of f

(number of inputs to circuit)

and n is the number of outputs

• Proposition: for any such function f, a logic

circuit that computes f may be constructed from a

finite number of , , or gates

• A logic circuit computes a function on strings of

bounded size, i.e., functions with a finite domain



Lookup tables


• A function on a finite domain may be

represented and effectively computed

using a lookup table

a b a xor b

0 0 0 0 1 1 1 0 1 1 1 0

Square function XOR operator

Labeled transition systems • Definition: a digraph, where vertices denote

state, labeled edges denote transitions

• Example (right): an input string

of bits is read from the

top, following edges

• A loopless transition

system can accept

any finite language of strings that leave the

system in an accepting state



Transducers and accepters • Theorem:Any computation of a function

f : {0, 1}* {0, 1}n can be done by n accepters

• Proof:

1. for i n, let Li = {x | bit i of f(x) is 1}

2. Construct accepters M1 .. i for L1 .. i as follows.

3. Mi decides the ith bit of f (x) for any string x.

4. Let Pi be the predicate computed by Mi

5. So f (x) = Concati m(Pi(x))

6. Hence f (x) is computable by M1..n


Barwise-Moss. Vicious Circles, 1996

S. Epp. Discrete Mathematics with

Applications. Brooks/Cole, 2011.

Finsler, 1920s (NWF sets)

Goldin-Wegner (at www.engr.uconn.edu/~dqg)

Peano, 1889 (induction)

J. Savage.


References

david m keil, framingham state university csci 460 theory of

Documents