cdt314 faber formal languages, automata and models of computation lecture 1 - intro

1

CDT314

FABER

Formal Languages, Automata and Models of Computation

Lecture 1 - Intro

School of Innovation, Design and Engineering Mälardalen University

2011

2

Content

Adminstrivia

Mathematical Preliminaries

Countable Sets (Uppräkneliga mängder)

Uncountable sets (Överuppräkneliga mängder)

3

Lecturer & Examiner

Gordana Dodig-Crnkovic

Lessons: Leo HatvaniAssistant: Svetlana Girs

4

http://www.idt.mdh.se/kurser/cd5560/11_11/

visit home page regularly!

Course Home Page

5

How Much Work?

20 hours a week for this type of course (norm)

4 hours lectures

2 hours exercises

14 hours own work a week!

6

Why Theory of Computation?

1. A real computer can be modelled by a mathematical object: a theoretical computer.

2. A formal language is a set of strings, and can represent a computational problem.

3. A formal language can be described in many different ways that ultimately prove to be identical.

4. Simulation: the relative power of computing models can be based on the ease with which one model can simulate another.

7

5. Robustness of a general computational model.

6. The Church-Turing thesis: "everything algorithmically computable is computable by a Turing machine."

7. Study of non-determinism: languages can be described by the existence or no-nexistence of computational paths.

8. Understanding unsolvability: for some computational problems there is no corresponding algorithm that will unerringly solve them.

Why Theory of Computation?

8

Practical Applications

1. Efficient compilation of computer languages

2. String search

3. Investigation of the limits of computation,recognizing difficult/unsolvable problems

4. Applications to other areas:– circuit verification– economics and game theory (finite automata as

strategy models in decision-making); – theoretical biology (L-systems as models of

organism growth) – computer graphics (L-systems) – linguistics (modelling by grammars)

9

History

• Euclid's attempt to axiomatize geometry

(Archimedes realized, during his own efforts to define the area of a planar figure, that Euclid's attempt had failed and that additional postulates were needed. )

• Leibniz's (1646 - 1716) dream reasoning as calculus - "Characteristica Universalis" aiming at:

"a general method in which all truths of the reason would be reduced to a kind of calculation. At the same time this would be a sort of universal language or script, but infinitely different from all those projected hitherto; for the symbols and even the words in it would direct reason; and errors, except those of fact, would be mere mistakes in calculation.“

• de Morgan, Boole, Frege, Russell, Whitehead:

Mathematics as a branch of symbolic logic!

10

1900 Hilberts program for axiomatization of mathematics, redefined "proof" to become a completely rigorous notion, totally different from the psycho/sociological "A proof is something that convinces other mathematicians.“ He confirms the prediction Leibniz made, that "the symbols would direct reason"

1880 -1936 first programming languages

1931 Gödels incompleteness theorems

1936 Turing maschine (showed to be equivalent with recursive functions). Commonly accepted: TM as ultimate computer

1950 automata

1956 language/automata hierarchy

11

Every mathematical truth expressed in a formal language is consisting of

• a fixed alphabet of admissible symbols, and

• explicit rules of syntax for combining those symbols into meaningful words and sentences

Gödel's two incompleteness theorems of mathematical logic show limitations of all but the most trivial axiomatic systems. The theorems are widely interpreted as showing that Hilbert's program to find a complete and consistent set of axioms for all of mathematics is impossible, thus giving a negative answer to Hilbert's second problem.

Formalization of Mathematics

The main goal of Hilbert's program was to provide secure foundations for all mathematics. In particular this should include:

A formalization of all mathematics; in other words all mathematical statements should be written in a precise formal language, and manipulated according to well defined rules.

Completeness: a proof that all true mathematical statements can be proved in the formalism.

Consistency: a proof that no contradiction can be obtained in the formalism of mathematics. This consistency proof should preferably use only "finitistic" reasoning about finite mathematical objects.

Decidability: there should be an algorithm for deciding the truth or falsity of any mathematical statement.

12

Statement of Hilbert’s program

Gödel showed that most of the goals of Hilbert's program were impossible to achieve. His second incompleteness theorem stated that any consistent theory powerful enough to encode addition and multiplication of integers cannot prove its own consistency. This wipes out most of Hilbert's program as follows:

It is not possible to formalize all of mathematics, as any attempt at such a formalism will omit some true mathematical statements.

The most interesting mathematical theories are not complete.

A theory such as Peano arithmetic cannot even prove its own consistency.

13

Statement of Hilbert’s program

14

Turing used a Universal Turing machine (UTM) to prove an incompleteness theorem even more powerful than Gödel’s because it destroyed not one but two of Hilbert's dreams:

1. Finding a finite list of axioms from which all

mathematical truths can be deduced

2. Solving the entscheidungsproblem, ("decision

problem“) by producing a "fully automatic procedure"

for deciding whether a given proposition (sentence) is

true or false.

Turing’s Contribution

15

Mathematical Preliminaries

16

• Sets

• Functions

• Relations

• Proof Techniques

• Languages, Alphabets and Strings

• Strings & String Operations

• Languages & Language Operations

17

}3,2,1{AA set is a collection of elements

SETS

},,,{ airplanebicyclebustrainB

We write

A1

Bship

18

Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k }

S = { 2, 4, 6, … }

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

finite set

infinite set

19

A = { 1, 2, 3, 4, 5 }

Universal Set: All possible elements

U = { 1 , … , 10 }

1 2 3

4 5

A

U

6

7

8

910

20

Set Operations

A = { 1, 2, 3 } B = { 2, 3, 4, 5}

• Union

A U B = { 1, 2, 3, 4, 5 }

• Intersection

A B = { 2, 3 }

• Difference

A - B = { 1 }

B - A = { 4, 5 }

U

A B

A-B

21

• Complement

Universal set = {1, …, 7}

A = { 1, 2, 3 } A = { 4, 5, 6, 7}

12

3

4

5

6

7

AA

A = A

22

{ even integers } = { odd integers }

02

4

6

1

3

5

7

even

odd

Integers

23

DeMorgan’s Laws

A U B = A BU

A B = A U B

U

24

Empty, Null Set:

= { }

S U = S

S =

S - = S

- S =

U = Universal Set

25

Subset

A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }

A B

U

Proper Subset: A B

U

A

B

26

Disjoint Sets

A = { 1, 2, 3 } B = { 5, 6}

A B = U

A B

27

Set Cardinality

For finite sets

A = { 2, 5, 7 }

|A| = 3

28

Powersets

A powerset is a set of sets

Powerset of S = the set of all the subsets of S

S = { a, b, c }

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )

29

Cartesian Product

A = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

A X B X … X Z

30

PROOF TECHNIQUES

• Proof by construction

• Proof by induction

• Proof by contradiction

31

Proof by Construction

We define a graph to be k-regular

if every node in the graph has degree k.

Theorem. For each even number n > 2 there exists

3-regular graph with n nodes.

1

2

4

3

0

5

1 2

0

3n = 4 n = 6

32

Construct a graph G = (V, E) with n > 2 nodes.

V= { 0, 1, …, n-1 }

E = { {i, i+1} for 0 i n-2} {{n-1,0}} (*)

{{i, i+n/2 for 0 i n/2 –1} (**)

The nodes of this graph can be written consecutively around the circle.

(*) edges between adjacent pairs of nodes

(**) edges between nodes on opposite sides

Proof by Construction

END OF PROOF

33

Inductive Proof

We have statements P1, P2, P3, …

If we know

• for some k that P1, P2, …, Pk are true

• for any n k that

P1, P2, …, Pn imply Pn+1

Then

Every Pi is true

34

Proof by Induction

• Inductive basis

Find P1, P2, …, Pk which are true

• Inductive hypothesis

Let’s assume P1, P2, …, Pn are true,

for any n k

• Inductive step

Show that Pn+1 is true

35

Example

Theorem A binary tree of height n

has at most 2n leaves.

Proof

let L(i) be the number of leaves at level i

L(0) = 1

L(3) = 8

36

We want to show: L(i) 2i

• Inductive basis

L(0) = 1 (the root node)

• Inductive hypothesis

Let’s assume L(i) 2i for all i = 0, 1, …, n

• Induction step

we need to show that L(n + 1) 2n+1

37

Induction Step

hypothesis: L(n) 2n

Level

n

n+1

38

hypothesis: L(n) 2n

Level

n

n+1

L(n+1) 2 * L(n) 2 * 2n = 2n+1

Induction Step

END OF PROOF

39

Proof by induction: Cardinality of a power set

Let S be a finite set with N elements. Then the powerset of S (that is the set of all subsets of S ) contains 2^N elements. In other words, S has 2^N subsets. This statement can be proved by induction. It's true for N=0,1,2,3 as can be shown by examination.

The notation 2^N means 2 to the power N, i.e., the product of N factors all of which equal 2. 2^0 is defined to be 1

40

For the induction step suppose that the statement is true for a set with N-1 elements, and let S be a set with N elements. Remove on element x from S to obtain a set T with N-1 elements. There are two types of subsets of S: those that contain x and those that do not contain x. The latter are subsets of T, of which there are 2^ N-1.

Every subset P of S that does contain x can be obtained from a subset Q of T by adding x. The set Q is simply the set P with x removed. Clearly there is a unique and distinct set Q for each set P and every subset Q of T gives rise to a unique and distinct subset P of S. There are thus also 2^ (N-1) subsets of S that contain x, for a total of 2^ (N-1) + 2^ (N-1) = 2^ N subsets of S.

The size of a finite power set

41

Inductionsbevis: Potensmängdens kardinalitet

Påstående: En mängd med n element har 2n delmängder

Kontroll

• Tomma mängden {} (med noll element) har bara en delmängd: {}.

• Mängden {a} (med ett element) har två delmängder: {} och {a}

42

Påstående: En mängd med n element har 2n delmängder

Kontroll (forts.)

• Mängden {a, b} (med två element) har fyra delmängder: {}, {a}, {b} och {a,b}

• Mängden {a, b, c} (med tre element) har åtta delmängder:

{}, {a}, {b}, {c} och {a,b}, {a,c}, {b,c}, {a,b,c}

Påstående stämmer så här långt.

43

Bassteg

Enklaste fallet är en mängd med noll element (det finns bara en sådan), som har 20 = 1 delmängder.

44

Induktionssteg

Antag att påståendet gäller för alla mängder med k element, dvs antag att varje mängd med k element har 2k delmängder.

Visa att påståendet i så fall också gäller för alla mängder med k+1 element, dvs visa att varje mängd med k+1 element har 2k+1 delmängder.

45

Vi betraktar en godtycklig mängd med k+1 element. Delmängderna till mängden kan delas upp i två sorter:

Delmängder som inte innehåller element nr k+1: En sådan delmängd är en delmängd till mängden med de k första elementen, och delmängder till en mängd med k element finns det (enligt antagandet) 2k stycken.

46

Delmängder som innehåller element nr k+1: En sådan delmängd kan man skapa genom att ta en delmängd som inte innehåller element nr k+1 och lägga till detta element. Eftersom det finns 2k delmängder utan element nr k+1 kan man även skapa 2k

delmängder med detta element.

Totalt har man 2k + 2k = 2. 2k= 2k+1 delmängder till den betraktade mängden.

END OF PROOF

(Exempel från boken: Diskret matematik och diskreta modeller, K Eriksson, H. Gavel)

47

Proof by Contradiction

We want to prove that a statement P is true

• we assume that P is false

• then we arrive at a conclusion that contradicts our assumptions

• therefore, statement P must be true

48

Example

Theorem is not rational

Proof

Assume by contradiction that it is rational

= n/m

n and m have no common factors

We will show that this is impossible

2

2

49

Therefore, n2 is evenn is even

n = 2 k

2 m2 = 4k2 m2 = 2k2m is even

m = 2 p

Thus, m and n have common factor 2

Contradiction!

= n/m 2 m2 = n2 2

END OF PROOF

50

Countable Sets

51

Infinite sets are either

Countable or Uncountable

52

Countable set

There is a one to one correspondence

between elements of the set

and natural numbers

53

We started with the natural numbers, then• add infinitely many negative whole numbers to get the integers, • then add infinitely many rational fractions to get the rationals, • then added infinitely many irrational fractions to get the reals.

Each infinite addition seem to increase cardinality: |N| < |Z| < |Q| < |R|

But is this true? NO!

54

Example

Integers: ,2,2,1,1,0

The set of integers is countable

Correspondence:

Natural numbers: ,4,3,2,1,0

oddnnevennnnf 2/)1(;2/)( {

55

ExampleThe set of rational numbers

is countable

Positive

Rational numbers:,

87

,43

,21

56

Naive Idea

Rational numbers: ,31

,21

,11

Natural numbers:

Correspondence:

,3,2,1

Doesn’t work!

we will never count

numbers with nominator 2:,

32

,22

,12

57

Better Approach

11

21

31

41

12

22

32

13

23

14

...

...

...

...

Rows: constant numerator (täljare)

Columns: constant denominator

58

11

21

31

41

12

22

32

13

23

14

...

...

...

...

59

We proved:

the set of rational numbers is countable

by describing an enumeration procedure

60

Definition

An enumeration procedure for is an

algorithm that generates

all strings of one by one

Let be a set of strings S

S

S

61

A set is countable if there is an

enumeration procedure for it

Observation

62

Example

The set of all finite strings

is countable

},,{ cba

We will describe the enumeration procedure

Proof

63

Naive procedure:

Produce the strings in lexicographic order:

aaaaaa

......Doesn’t work!

Strings starting with will never be produced b

aaaa

64

Better procedure

1. Produce all strings of length 1

2. Produce all strings of length 2

3. Produce all strings of length 3

4. Produce all strings of length 4

..........

Proper Order

65

Produce strings in

Proper Order

aaabacbabbbccacbcc

aaaaabaac......

length 2

length 3

length 1abc

66

Theorem

The set of all finite strings is countable

Proof

Find an enumeration procedure

for the set of finite strings

Any finite string can be encoded

with a binary string of 0’s and 1’s

67

Produce strings in Proper Order

length 2

length 3

length 10

1

00

01

10

11

000

001

….

0

1

2

3

4

5

6

7

….

String = program Natural number

68

PROGRAM = STRING (syntactic way)

PROGRAM = FUNCTION (semantic way)

PROGRAMstring string

PROGRAMnatural number

n

natural number

n

69

Uncountable Sets

70

A set is uncountable if it is not countable

Definition

71

Theorem

The set of all infinite strings is uncountable

We assume we have

an enumeration procedure

for the set of infinite strings

Proof (by contradiction)

72

Infinite string Encoding

0w

1w

2w

...

...

...

...

00b

10b

20b

01b

11b

21b

02b

12b

22b

=

=

=

Cantor’s diagonal argument

... ... ... ...

73

Cantor’s diagonal argument

We can construct a new string that is missing in our enumeration!

w

The set of all infinite strings is uncountable!

Conclusion

74

There are some integer functions that

that cannot be described by finite strings (programs/algorithms).

Conclusion

An infinite string can be seen as FUNCTION (n:th output is n:th bit in the string)

75

Theorem

Let be an infinite countable set

The powerset of is uncountable S2 S

S

Example of uncountable infinite sets

76

Proof

Since is countable, we can write S

},,,{ 321 sssS

77

Elements of the powerset have the form:

},{ 31 ss

},,,{ 10975 ssss

……

78

We encode each element of the power set

with a binary string of 0’s and 1’s

1s 2s 3s 4s

1 0 0 0}{ 1s

Powerset

element

Encoding

0 1 1 0},{ 32 ss

1 0 1 1},,{ 431 sss

...

...

...

...

79

Let’s assume (by contradiction)

that the powerset is countable.

we can enumerate

the elements of the powerset

Then:

80

1 0 0 0 0

1 1 0 0 0

1 1 0 1 0

1 1 0 0 1

Powerset

elementEncoding

1p

2p

3p

4p

...

...

...

...

...

81

Take the powerset element

whose bits are the complements

in the diagonal

82

1 0 0 0 0

1 1 0 0 0

1 1 0 1 0

1 1 0 0 1

New element: 0011

(binary complement of diagonal)

...

...

...

...

1p

2p

3p

4p

83

The new element must be some

of the powerset ip

However, that’s impossible:

the i-th bit of must be

the complement of itself

from definition of

Contradiction!

ip

ip

84

Since we have a contradiction:

The powerset of is uncountable S2 S

END OF PROOF

85

Example Alphabet : },{ ba

The set of all finite strings:

},,,,,,,,,{},{ * aabaaabbbaabaababaS

infinite and countable

uncountable infinite

}},,,}{,{},{},{{2 aababaabaaS 1L 2L 3L 4L

The powerset of contains all languages:S

An Application: Languages

86

Finite strings (algorithms): countable

Languages (power set of strings): uncountable

There are infinitely many more languages

than finite strings.

87

There are some languages

that cannot be described by finite strings (algorithms).

Conclusion

88

Cardinality - Kardinaltal

Kardinaltal är mått på storleken av mängder. Kardinaltalet för en ändlig mängd är helt enkelt antalet element i mängden.

Två mängder är lika mäktiga om man kan para ihop elementen i den ena mängden med elementen i den andra på ett uttömmande sätt, dvs det finns en bijektion mellan dem.

Numbers

89

http://teachers.henrico.k12.va.us/math/ms/c30708/pics/1_4realnumbers.jpg

90

Cardinality - Kardinaltal

Kardinaltal kan generaliseras till oändliga mängder. Till exempel är mängden av positiva heltal och mängden av heltal lika mäktiga (har samma kardinaltal).

Däremot kan man inte para ihop alla reella tal med heltalen på detta sätt. Mängden av reella tal har större mäktighet än mängden av heltal.

91

Cardinality - Kardinaltal

Man kan införa kardinaltal på ett sådant sätt att två mängder har samma kardinaltal om och endast om de har samma mäktighet. T ex kallas kardinaltalet som hör till de hela talen för 0 (alef 0, alef är den första bokstaven i det hebreiska alfabetet).

Dessa oändliga kardinaltal kallas transfinita kardinaltal.

92

Georg Cantor utvecklade i slutet av 1800-talet matematikens logiska grund, mängdläran.

Den enklaste, "minsta", oändligheten kallade han 0.

Det är den uppräkningsbara oändliga mängdens (exempelvis mängden av alla heltal) kardinaltalet.

Kardinaltalet av mängden punkter på en linje, och även punkterna på ett plan och i en kropp, kallade Cantor 1.

Fanns det större oändligheter?

Mer om oändligheter…More about Infinty

93

Ja! Cantor kunde visa att antalet funktioner på en linje var ”ännu oändligare” än punkterna på linjen,

och han kallade den mängden 2.

Cantor fann att det gick att räkna med kardinaltalen precis som med vanliga tal, men räknereglerna blev något enahanda..

0 + 1= 0 0 + 0 = 0 0 · 0 = 0.

94

Men vid exponering hände det något:

0 0 (0 upphöjt till 0) = 1.

Mer generellt visade det sig att

2 n (2 upphöjt till n) = n+1

Det innebar att det fanns oändligt många oändligheter, den ena mäktigare än den andra!

95

Men var det verkligen säkert att det inte fanns någon oändlighet mellan den uppräkningsbara och punkterna på linjen? Cantor försökte bevisa den så kallade kontinuumhypotesen.

Cantor: two different infinities 0 and 1 http://www.ii.com/math/ch/#cardinals

Continuum Hypothesis: 0 < 1 = 2 0

Se även:http://www.nyteknik.se/pub/ipsart.asp?art_id=26484