::ICS 804::Theory of

Computation

- Ibrahim Otieno -

iotieno@uonbi.ac.ke

+254-0722-429297

SCI/ICT Building Rm. G15

Course OutlineCourse Outline

Mathematical Preliminaries Turing Machines Recursion Theory Markov Algorithms Register Machines Regular Languages and finite-state

automata Aspects of Computability

Last Week: Recursion Last Week: Recursion TheoryTheory

The primitive recursive functions The partial recursive functions The class of partial recursive

functions = class of Turing-computable functions

Course OutlineCourse Outline

Mathematical Preliminaries Turing Machines

◦ Additional Varieties of Turing Machines Recursion Theory Markov Algorithms Register Machines Regular Languages and finite-state

automata Aspects of Computability

Markov AlgorithmsMarkov Algorithms Sequential computation and Markov

Algorithms Language Acceptors/Recognizers Number-theoretic functions Labeled Markov algorithms Markov-Computable functions and

partial recursive functions Efficiency

Sequential computation and Markov Algorithms

Three Computational Paradigms Three Computational Paradigms (recap)(recap)Number-theoretic function

computation (functions from N to N)Language acceptance (recognition)Transduction (transformation of an

input into an appropriate output)

Markov AlgorithmsMarkov Algorithms• So far: main focus on function

computation and language acceptance

• A.A. Markov in the 1950’s formalized string-rewriting systems

• Focus on transductionBUT: paradigm interreducibility

An Algorithm SchemeAn Algorithm SchemeLet S be alphabet ={a, b, c, d}.

By a Markov algorithm scheme or schema we shall mean a finite sequence of productions or rewrite rules.

As a first example, consider the following two-member sequence of productions: 

(i) a c(ii) b

An Algorithm SchemeAn Algorithm Scheme

Apply production sequence to words over e.g. w: baba

Apply rules:• find leftmost occurrence of the symbol on the left-hand side and substitute it with the right-hand side• rule (i) transforms word w into w1

w: babaw1: bcba

(i) a c

(ii) b

An Algorithm SchemeAn Algorithm Scheme

Apply production sequence to words over e.g. w: baba

Apply rules:• find leftmost occurrence of the symbol on the left-hand side and substitute it with the right-hand side• rule (i) transforms word w1 into w2

w1: bcbaw2: bcbc

(i) a c

(ii) b

An Algorithm SchemeAn Algorithm Scheme

Apply production sequence to words over . e.g. w: baba

Apply rules:• (i) can not be applied, so now we investigate the applicability of (ii)• rule (ii) transforms word w2 into w3

w2: bcbcw3: cbc

(i) a c

(ii) b

An Algorithm SchemeAn Algorithm Scheme

Apply production sequence to words over . e.g. w: baba

Apply Rules:• (i) can not be applied, so now we investigate the applicability of (ii)• rule (ii) transforms word w3 into w4

w3: cbcw4: cc

No more production rules can be applied to w4 so substitution process ceases

(i) a c

(ii) b

An Algorithm SchemeAn Algorithm Scheme

abab * ccabcd * ccd

bbbb * *

(i) a c

(ii) b

TerminologyTerminologythe substitution process that results when some Markov algorithm schema is applied to some input word = the Markov algorithm corresponding to that Markov algorithm scheme

S: a Markov algorithm scheme (production or re-write rules)AS: corresponding Markov algorithm

(substitution process)

Another ExampleAnother Example

Given input alphabet ={a,b,c,d}.

Next, consider the following three-member sequence of productions.

(i) a c(ii) bc cb(iii) b .cd (terminal production)

Exercise: baba, baaa

Work AlphabetWork Alphabet• Sometimes temporary markers (cf.

additional symbols in Turing Machine Tape Alphabet) are indispensable

: work alphabet containing the input alphabet and temporary markers like #,$,%,*

• Extra symbols typically do not occur in the input or output

Appends Appends ababLet input alphabet ={a, b}. Let work alphabet be {#}.

We see that the Markov algorithm scheme consisting of the four-member production sequence  

(i) #a a#(ii) #b b#(iii) # .ab(iv) #

 has the effect of appending string ab to any word over SExercise: ba, aba

Markov SchemaMarkov Schema Ordered sequence of production

rules: Markov Schema Each production is of the form

: non-terminal productionor .: terminal production

Deterministic in nature

Markov AlgorithmMarkov Algorithm

Process type: apply productions of the corresponding Markov algorithm scheme until no production is applicable or a terminal production has been applied

A Formal DefinitionA Formal DefinitionMarkov algorithm schema S is any triple

, , nonempty input alphabet finite work alphabet with finite, ordered sequence of productions

of form or of form . and (possibly empty) words over

w S w’ or w w’ (zero or more steps)

w *S w’ or w * w’ (upon halting)

A Formal DefinitionA Formal DefinitionAbbreviations in schema

e.g. ={a,b}#a a##b b#we can summarize this as follows:# # (for )

Language Language Acceptors/RecognizersAcceptors/Recognizers

Markov AlgorithmsMarkov Algorithms

We can implement each of our three computational paradigms: paradigm interreducibility

Language TransductionLanguage acceptance

(recognition)Function computation

Language AcceptanceLanguage AcceptanceEX4-2-1.MKV in Deus ex MachinaInput alphabet ={a, b} Work alphabet = {@, %, $, 1}Six productions/one of them terminalTransforms all and only words in

language {anbm|n 0, m 1} to word 1

DefinitionDefinitionLet S be a Markov algorithm schema

with input alphabet and work alphabet with 1 and 1. Then S accepts word w if w*1

1: accepting 1

If Markov algorithm Schema S accepts word w, then Markov algorithm AS, corresponding to S, accepts word w as well

Markov-Acceptable Markov-Acceptable LanguageLanguageSchema S accepts language L if S

accepts all and only words in L. A language that is accepted by

some Markov algorithm is said to be a Markov-acceptable language.

ExampleExampleLanguage {anbm|n 0, m 1} is

acceptedWhat happens when we take input

word aabbba

Language RecognitionLanguage Recognition Input alphabet S and work alphabet such that

both 0, 1 \ S recognizes language L over S transforms w L into 1, that is, w * 1

(accepting 1)S transforms w L into 0, that is, w * 0

(rejecting 0)Language recognized by some Markov algorithm

S is said to be Markov-Recognizable

Example:EX4-2-2.MKV recognizes language L={(ab)n|n0}

Resources (Time)Resources (Time)

DefinitiontimeS(n) = the maximum number of steps in any terminating computation of S for input of length n

Resources (Space)Resources (Space)Computation of a Markov algorithm

= sequence of computation wordsDefinition

spaceS(n) = the maximum length of any computation word in any terminating computation of S for input of length n

Resource AnalysisResource AnalysisEX4-2-3.MKV in Deus ex Machina accepts

language {w*|na(w) = nb(w)} with = {a, b}

timeS(n) = (n/2)2 + n + 2 for even n so O(n2)

◦words of uneven length can be disregarded for time/space analyses

◦Most costly words to accept are an/2bn/2 and bn/2an/2 These are accepted after (n/2)2+n+2 steps

spaceS(n) = n + 1 so O(n)

ComplexityComplexityGiven Turing machine M accepting L,

there exists a Markov algorithm AS that accepts L in O(timeM(n)) steps

Given Markov algorithm AS accepting L, there exists a Turing machine M that accepts L in O([timeAS

(n)]4)

steps

Number-theoretic Number-theoretic functionsfunctions

Function ComputationFunction ComputationNatural number represented by

n+1 1sPair <2,3> represented by string

111*1111, arguments separated by asterisk

ExampleExample

Schema S for unary number-theoretic function:

One terminal production 1 .11

What does this Markov Schema compute?

ExampleExample

Schema S for unary number-theoretic function:

One terminal production 1 .11

Computes unary successor function

Formal DefinitionFormal DefinitionS computes unary partial number-

theoretic function fS applied to input 1n+1 yields output 1f(n)

+ 1

If S applied to input 1n+1, where function f is not defined for n, then either the computation never terminates or its output is not of the form of an unbroken series of 1s

ExerciseExercise

Schema S for unary number-theoretic function:

One production 11 1 Which initial function does this

schema compute?

ExerciseExerciseSchema S for unary number-theoretic

function:One terminal production 11 .1 Which initial function does this schema

compute?unary constant-0 functionC1

0(n) = 0 for all n

!! All constant functions and projection functions are Markov-computable

ExerciseExerciseSchema S for binary number-theoretic

function:$1 1$$* %%1 %% . $

Which number theoretic function does this schema compute?

ExerciseExerciseSchema S for binary number-theoretic

function:$1 1$$* %%1 %% . $

Which number theoretic function does this schema compute?

p21

ExerciseExercise

Show that the sg function is Markov-computable

sg(n) = 1 if n = 00 otherwise

Hint: reduce the problem of n>0 to whether n=1

ExerciseExercise

Show that the sg function is Markov-computable

sg(n) = 1 if n = 00 otherwise

111 1111 .11 .11

Labeled Markov Labeled Markov AlgorithmsAlgorithms

LabeledLabeled• Adds concept of branching and production

labels• “gotos”• Easier to design and comprehend

• Production labels: L1, L2 ,L3 …

• Schema S: Li: ;Lj

replace with branch off execution to production rule Lj

Labeled Markov Algorithm That AcceptsLabeled Markov Algorithm That Accepts{{ww||nnaa((ww) = ) = nnbb((ww) = ) = nncc((ww)})}

L1: a ; L2

; L4

L2: b ; L3

; L5

L3: c ; L1

; L5

L4: b ; L5

c ; L5

1 ; L5L5: .

Equivalence ResultEquivalence ResultLet AS be a labeled Markov

algorithm with input alphabet . Then there exists a standard Markov algorithm AS´ with input alphabet that is computationally equivalent to AS

Markov-Computable Markov-Computable functions and partial functions and partial recursive functionsrecursive functions

Equivalence ResultEquivalence Result

Class of Markov-computable functions is identical to the class of ◦partial recursive functions◦Turing-computable functions

Function f is Markov-computable iff f is Turing-computable

ProofProofStart Turing Machine: open EX4-5-

1.TM and EX4-5-1.TT to see a Turing machine that simulates a Markov algorithm

Start Markov Algorithm and open EX4-5-2.MKV to see a Markov algorithm that simulates a Turing machine

EfficiencyEfficiencyAny language that is Markov-

acceptable is also Turing acceptable

Feasibility (recap)Feasibility (recap)Cobham–Edmonds thesis regarding

computational feasibility: the problem of determining whether a given string is a member of a given language L is feasible if and only if L is in P

Complexity class P: class of all languages accepted in polynomial time by some (single-tape) Turing Machine

EfficiencyEfficiencyAny language that is Markov-

acceptable is also Turing acceptableMarkov Algorithms often easier to

implement easier to determine computational feasibility of accepting some given language L