'

&

$

%

Chapter 1: Languages and Computation

1.1 Programming Problems and Computation

The aim of the text is to give you a feel and fluency with some of thefundamental abstract models (in the theory of computation) andtheir properties as well as to develop your ability with prooftechniques.

Theory, acting as an early warning system, provides a science of theimpossible–what shouldn’t be attempted because it cannot be done.However, a good theory should also provide guidelines for thepossible.

1

'

&

$

%

We find that the theory of computation does indeed do this at manydifferent levels. At coarsest level it partitions problems into threeclasses:

1. the impossible problems (undecidable);

2. the possible-with-unlimited-resources-butimpossible-with-limited-resources problems (intractable); and

3. the possible-with-limited-resources problems (tractable).

At a finer level the theory also considers problems which require, forexample, time linearly proportional to their size.

2

'

&

$

%

Theory also faces and deals with questions as fundamental anddiverse as

• Program Existence

• Program Specification

• Expressive Power

• Program Termination

• Program Correctness

• Program Construction

• Program Semantics

• Program Efficiency

• Program Implementation

• Machine Comparison

3

'

&

$

%

Given a programming problem it is easier to convince someone thatthere is a program which solves the problem (if one exists) than toconvince someone there is no program for the problem (if onedoesn’t). This because in the former case we need only outline asolution informally, but in the latter case we need to show that overall possible programs none solves the given problem.

4

'

&

$

%

Basic conditions that a description of a program and a machine toexecute it must satisfy (algorithm and computing agent):

1. It (an algorithm) consists of a finite set of instructions.

2. there is a computing agent that can carry out the instructions ofthe algorithm.

3. The computing agent must be able to store, retrieve, and makesteps in a computation.

4. The computing agent must carry out the instructions in discretesteps.

5. The computing agent must carry out the instructionsdeterministically or mechanistically.

5

'

&

$

%

The above five conditions capture the essence of programs andcomputers by way of algorithms and computing agents. However,there are a number of questions that should still be answered:

1. Should an algorithm halt or terminate in a finite number of stepson each input?

2. Should the size of a set of instructions be restricted?

3. Should the size and number of the initial values or input data berestricted?

4. Should the number of computation steps or execution time berestricted?

5. Should the memory of the computing agent be restricted?

6. Should the ability of the computing agent be restricted?

6

'

&

$

%

1.2 Alphabets, Word, and Languages

• A finite underlying set is called an alphabet.

• A finite sequence of the elements of an alphabet is called a word.

• A set of finite sequences (words) is called a language.

1.2.1 Alphabets and Words

An alphabet is a finite, nonempty set of elements, the elements of thealphabet are usually called symbols or letters.

A word over an alphabet Σ is a finite sequence of symbols from Σ.

7

'

&

$

%

The empty word is the empty sequence and is denoted by λ.

The universal set or universal language of all words over an alphabetΣ is denoted by Σ∗. Σ∗ is always an infinite set and is enumerable.

The length of a word x is denoted by |x| and is simply the number ofsymbols in the given word.

The a-length of x is denoted by |x|a and is the number of times a

occurs in x.

The catenation of x with y, denoted by xy, is the word obtained byappending the word y to x.

Given a word x over an alphabet Σ we define x0 to be λ and for alli ≥ 1, xi = xxi−1. We say that xi is ith the power of x.

8

'

&

$

%

Given two words x and y over an alphabet Σ, we say that x equals y,writen x = y, if they have the same length and the same symbols atthe same positions.

We say x is a prefix of y if there is a word z such that xz = y. Ifx 6= λ and x 6= y, then x is a proper prefix.

We say x is a suffix of y if there is a word z such that zx = y. Ifx 6= λ and x 6= y, then x is a proper suffix.

We say that x is a subword of y if there are words w and z withwxz = y. If x 6= λ and x 6= y, then x is a proper subword.

9

'

&

$

%

The reversal, or mirror image, of x as an . . . a1 if x = a1 . . . an, whereai is in Σ, 1 ≤ i ≤ n. We denote the reversal of x by xR.

For all words x over some alphabet Σ, xR is definied recursively as

1. x, if x = λ;

2. yRa, if x = ay for some a in Σ and y in Σ∗.

10

'

&

$

%

1.2.2 Languages and Operations

The universal language Σ∗ given by an alphabet Σ is an infinte setbut it is worth emphasizing that every word in Σ∗ is finite, that is, ithas finite length.

Given an alphabet Σ, then Σ∗ consists solely of words obtained fromthe following items:

• λ is in Σ∗;

• if x is in Σ∗, then ax is in Σ∗, for all a in Σ.

A language L over an alphabet Σ is a set of words over Σ, that isL ⊆ Σ∗.

11

'

&

$

%

∅ is a language over every alphabet, as is {λ} since λ is a word overevery alphabet.

Notice that ∅ 6= {λ} since ∅ contains no words but {λ} contains asingle word, the empty word. We call {λ} the empty word set todistinguisk it from ∅, the empty set.

We denote by alph(x), the set of symbols in the word x. Thisdefinition implies alph(λ) = ∅ and alph(abba) = {a, b}. If L ⊆ Σ∗,then alph(L) ⊆ Σ.

12

'

&

$

%

Since languages are sets the boolean operations can be applied tothem.

Given two languages L1 ⊆ Σ∗1 and L2 ⊆ Σ∗2, their catenation(product), denoted by L1L2, is defined by

{x1x2 : x1 is in L1 and x2 is in L2} .

Product is similar to but different from cartestan product.

Product is similar to but different from cartestan product. Forexample, let L1 = {a, ab}, and L2 = {b, bb}, thenL1L2 = {ab, abb, abbb} contains only three words, whereasL1 × L2 = {(a, b), (a, bb), (ab, b), (ab, bb)} contains four pairs of words.

13

'

&

$

%

Catenation satisfies a number of properties:

• A(BC) = (AB)C = ABC

• {λ}A = A{λ} = A

• ∅A = A∅ = ∅• A(B ∪ C) = AB ∪AC and (B ∪ C)A = BA ∪ CA

but in general,A(B ∩ C) 6= AB ∩AC and (B ∩ C)A 6= BA ∩ CA

Given a language L ⊆ Σ∗ and a nonnegative integer i, define the ithpower of L, denoted by Li, by

Li =

{λ} if i = 0

LLi−1 if i ≥ 1

14

'

&

$

%

We define the star or closure of L, denoted by L∗, by

L∗ =∞⋃

i=0

Li

The plus or positive closure of L, denoted by L+, is defined by

L+ =∞⋃

i=1

Li

15

'

&

$

%

Star and plus satisfy a number of properties:

• L+ = LL∗ = L∗L

• ∅+ = ∅ but ∅∗ = {λ}• {λ}+ = {λ} and {λ}∗ = {λ}• AA+ = A+A, (A+)+ = A+, (A+)∗ = A∗

(A∗)+ = A∗, (A∗)∗ = A∗

• A ⊆ B implies A+ ⊆ B+, A ⊆ B implies A∗ ⊆ B∗

Let L be an arbitary language over some alphabet; then the reversal,of L is defined as

LR = {xR : x is in L}

16

'

&

$

%

1.2.3 Defining Languages

• To list the words in the language.

• To define languages inductively.

• A generator defines a language by giving a scheme for generatingwords in the language.

• A recognizer defines a language by providing an algorithm orpartial algorithm to recognize words in the language.

• Aparser is a hybrid consisting of a generator and a recognizer.

A finite combination of languages over Σ using the operations ofcatenation, complement, intersection, plus, reversal, star, and unionis called a language expression over Σ. Each language expression E

over Σ specifies a language L(E) ⊆ Σ∗.

17

'

&

$

%

How many languages over Σ can be defined by language expressionsover Σ?

Theorem 1.2.2 For all alphabets Σ, Σ∗ is enumerable.

Each language expression over Σ is itself a word over some alphabet∆, where ∆ = Σ ∪ {, ,−,∪,∩, ∗, +, R, {, }, (, )}. Let EΣ denote the setof all language expressions over Σ, EΣ is itself a language, sinceEΣ ⊆ ∆∗.

Corollay 1.2.3 EΣ is enumerable.

Let LΣ be the set of languages over Σ given by language expressionsover Σ. LΣ is more usually called a family of languages.

Corollary 1.2.4 LΣ is enumerable.

Corollary 1.2.5 LΣ ⊂ 2Σ∗ .

18

'

&

$

%

1.3 Languages, Problems, and Computations

Given a descision problem II, it specifies a functionfII : A1 ×A2 × . . .×Am → {false, true}.Assume that each Ai can be represented as a set of words Li oversome alphabet Σi. This representation must be functional from Li toAi, that is to each representative in Li, there must correspond aunique element in Ai. We call such a representation an encoding.

19

'

&

$

%

We are now able to represent or encode A1 ×A2 × . . .×Am byL1{:}L2{:} . . . {:}Lm where : is a new symbol not appearing in any Σi

- the separation symbol. An instance determined by (a1, a2, . . . , am)is represented by r(a1) : r(a2) : . . . : r(am), where r(ai) is arepresentative corresponding to ai; in other words it is a word.

L(r, II) =

x1 : x2 : . . . : xm

∥∥∥∥∥∥∥∥∥∥∥∥∥∥∥∥∥

(a1, a2, . . . , am)

determines a yes

instance of II

and xi is a

representation of

ai, 1 ≤ i ≤ m

20

'

&

$

%

• For an instance (a1, a2, . . . , am) of a decision problem II whichhas a representation r, fII(a1, a2, . . . , am) = true iffx1 : x2 : . . . : xm is in L(r, II), where xi is a representation of ai

under r, 1 ≤ i ≤ m.

Hence we have replaced the evaluation of a function by a membershiptest; a computational problem has been replaced with alanguage-theoretical problem.

Finite specification of languages

What can be expressed or specified?(What functions can be computed?)

21