functionswithinfinitedomains,automata,andregularex- âŠfunctions with infinite domains, automata,...
TRANSCRIPT
-
Figure 6.1: Once you know how to multiply multi-digit numbers, you can do so for every number ðof digits, but if you had to describe multiplicationusing Boolean circuits or NAND-CIRC programs,you would need a different program/circuit for everylength ð of the input.
6Functions with Infinite domains, Automata, and Regular ex-pressions
âAn algorithm is a finite answer to an infinite number of questions.â, At-tributed to Stephen Kleene.
The model of Boolean circuits (or equivalently, the NAND-CIRCprogramming language) has one very significant drawback: a Booleancircuit can only compute a finite function ð . In particular, since everygate has two inputs, a size ð circuit can compute on an input of lengthat most 2ð . Thus this model does not capture our intuitive notion of analgorithm as a single recipe to compute a potentially infinite function.For example, the standard elementary school multiplication algorithmis a single algorithm that multiplies numbers of all lengths. However,we cannot express this algorithm as a single circuit, but rather need adifferent circuit (or equivalently, a NAND-CIRC program) for everyinput length (see Fig. 6.1).
In this chapter, we extend our definition of computational tasks toconsider functions with the unbounded domain of {0, 1}â. We focuson the question of defining what tasks to compute, mostly leavingthe question of how to compute them to later chapters, where we willsee Turing machines and other computational models for computingon unbounded inputs. However, we will see one example of a sim-ple restricted model of computation - deterministic finite automata(DFAs).
This chapter: A non-mathy overviewIn this chapter, we discuss functions that take as input stringsof arbitrary length. We will often focus on the special caseof Boolean functions, where the output is a single bit. Theseare still infinite functions since their inputs have unbounded
Compiled on 3.16.2021 13:56
Learning Objectives:⢠Define functions on unbounded length inputs,
that cannot be described by a finite size tableof inputs and outputs.
⢠Equivalence with the task of decidingmembership in a language.
⢠Deterministic finite automatons (optional): Asimple example for a model for unboundedcomputation.
⢠Equivalence with regular expressions.
-
218 introduction to theoretical computer science
Figure 6.2: The NAND circuit and NAND-CIRCprogram for computing the XOR of 5 bits. Note howthe circuit for XOR5 merely repeats four times thecircuit to compute the XOR of 2 bits.
length and hence such a function cannot be computed by anysingle Boolean circuit.In the second half of this chapter, we discuss finite automata,a computational model that can compute unbounded lengthfunctions. Finite automata are not as powerful as Python orother general-purpose programming languages but can serveas an introduction to these more general models. We alsoshow a beautiful result - the functions computable by finiteautomata are precisely the ones that correspond to regularexpressions. However, the reader can also feel free to skipautomata and go straight to our discussion of Turing machinesin Chapter 7.
6.1 FUNCTIONS WITH INPUTS OF UNBOUNDED LENGTH
Up until now, we considered the computational task of mappingsome string of length ð into a string of length ð. However, in gen-eral, computational tasks can involve inputs of unbounded length.For example, the following Python function computes the functionXOR ⶠ{0, 1}â â {0, 1}, where XOR(ð¥) equals 1 iff the number of 1âsin ð¥ is odd. (In other words, XOR(ð¥) = â|ð¥|â1ð=0 ð¥ð mod 2 for everyð¥ â {0, 1}â.) As simple as it is, the XOR function cannot be com-puted by a Boolean circuit. Rather, for every ð, we can compute XORð(the restriction of XOR to {0, 1}ð) using a different circuit (e.g., seeFig. 6.2).
def XOR(X):
'''Takes list X of 0's and 1's
Outputs 1 if the number of 1's is odd and outputs 0
otherwise'''âª
result = 0
for i in range(len(X)):
result = (result + X[i]) % 2
return result
Previously in this book, we studied the computation of finite func-tions ð ⶠ{0, 1}ð â {0, 1}ð. Such a function ð can always be describedby listing all the 2ð values it takes on inputs ð¥ â {0, 1}ð. In this chap-ter, we consider functions such as XOR that take inputs of unboundedsize. While we can describe XOR using a finite number of symbols(in fact, we just did so above), it takes infinitely many possible in-puts, and so we cannot just write down all of its values. The same istrue for many other functions capturing important computationaltasks, including addition, multiplication, sorting, finding paths in
-
functions with infinite domains, automata, and regular expressions 219
graphs, fitting curves to points, and so on. To contrast with the fi-nite case, we will sometimes call a function ð¹ ⶠ{0, 1}â â {0, 1} (orð¹ ⶠ{0, 1}â â {0, 1}â) infinite. However, this does not mean that ð¹takes as input strings of infinite length! It just means that ð¹ can takeas input a string of that can be arbitrarily long, and so we cannot sim-ply write down a table of all the outputs of ð¹ on different inputs.
Big Idea 8 A function ð¹ ⶠ{0, 1}â â {0, 1}â specifies the computa-tional task mapping an input ð¥ â {0, 1}â into the output ð¹(ð¥).
As we have seen before, restricting attention to functions that usebinary strings as inputs and outputs does not detract from our gener-ality, since other objects, including numbers, lists, matrices, images,videos, and more, can be encoded as binary strings.
As before, it is essential to differentiate between specification andimplementation. For example, consider the following function:
TWINP(ð¥) =â§{âš{â©
1 âðââ s.t.ð, ð + 2 are primes and ð > |ð¥|0 otherwise
(6.1)
This is a mathematically well-defined function. For every ð¥,TWINP(ð¥) has a unique value which is either 0 or 1. However, atthe moment, no one knows of a Python program that computes thisfunction. The Twin prime conjecture posits that for every ð thereexists ð > ð such that both ð and ð + 2 are primes. If this conjectureis true, then ð is easy to compute indeed - the program def T(x):return 1 will do the trick. However, mathematicians have triedunsuccessfully to prove this conjecture since 1849. That said, whetheror not we know how to implement the function TWINP, the definitionabove provides its specification.
6.1.1 Varying inputs and outputsMany of the functions that interest us take more than one input. Forexample, the function
MULT(ð¥, ðŠ) = ð¥ â ðŠ (6.2)takes the binary representation of a pair of integers ð¥, ðŠ â â, and
outputs the binary representation of their product ð¥â ðŠ. However, sincewe can represent a pair of strings as a single string, we will considerfunctions such as MULT as mapping {0, 1}â to {0, 1}â. We will typi-cally not be concerned with low-level details such as the precise wayto represent a pair of integers as a string, since virtually all choices willbe equivalent for our purposes.
https://en.wikipedia.org/wiki/Twin_prime
-
220 introduction to theoretical computer science
Another example of a function we want to compute is
PALINDROME(ð¥) =â§{âš{â©
1 âðâ[|ð¥|]ð¥ð = ð¥|ð¥|âð0 otherwise
(6.3)
PALINDROME has a single bit as output. Functions with a singlebit of output are known as Boolean functions. Boolean functions arecentral to the theory of computation, and we will discuss them oftenin this book. Note that even though Boolean functions have a singlebit of output, their input can be of arbitrary length. Thus they are stillinfinite functions that cannot be described via a finite table of values.
âBooleanizingâ functions. Sometimes it might be convenient to ob-tain a Boolean variant for a non-Boolean function. For example, thefollowing is a Boolean variant of MULT.
BMULT(ð¥, ðŠ, ð) =â§{âš{â©
ðð¡â bit of ð¥ â ðŠ ð < |ð¥ â ðŠ|0 otherwise
(6.4)
If we can compute BMULT via any programming language such asPython, C, Java, etc., we can compute MULT as well, and vice versa.
Solved Exercise 6.1 â Booleanizing general functions. Show that for everyfunction ð¹ ⶠ{0, 1}â â {0, 1}â, there exists a Boolean function BF â¶{0, 1}â â {0, 1} such that a Python program to compute BF can betransformed into a program to compute ð¹ and vice versa.
â
Solution:
For every ð¹ ⶠ{0, 1}â â {0, 1}â, we can define
BF(ð¥, ð, ð) =â§{{âš{{â©
ð¹(ð¥)ð ð < |ð¹(ð¥)|, ð = 01 ð < |ð¹(ð¥)|, ð = 10 ð ⥠|ð¥|
(6.5)
to be the function that on input ð¥ â {0, 1}â, ð â â, ð â {0, 1} out-puts the ðð¡â bit of ð¹(ð¥) if ð = 0 and ð < |ð¥|. If ð = 1 then BF(ð¥, ð, ð)outputs 1 iff ð < |ð¹(ð¥)| and hence this allows to compute the lengthof ð¹(ð¥).
Computing BF from ð¹ is straightforward. For the other direc-tion, given a Python function BF that computes BF, we can computeð¹ as follows:
def F(x):
res = []
i = 0
while BF(x,i,1):
-
functions with infinite domains, automata, and regular expressions 221
res.apppend(BF(x,i,0))
i += 1
return res
â
6.1.2 Formal LanguagesFor every Boolean function ð¹ ⶠ{0, 1}â â {0, 1}, we can define the setð¿ð¹ = {ð¥|ð¹(ð¥) = 1} of strings on which ð¹ outputs 1. Such sets areknown as languages. This name is rooted in formal language theory aspursued by linguists such as Noam Chomsky. A formal language is asubset ð¿ â {0, 1}â (or more generally ð¿ â Σâ for some finite alphabetΣ). The membership or decision problem for a language ð¿, is the task ofdetermining, given ð¥ â {0, 1}â, whether or not ð¥ â ð¿. If we can com-pute the function ð¹ , then we can decide membership in the languageð¿ð¹ and vice versa. Hence, many texts such as [Sip97] refer to the taskof computing a Boolean function as âdeciding a languageâ. In thisbook, we mostly describe computational tasks using the function nota-tion, which is easier to generalize to computation with more than onebit of output. However, since the language terminology is so popularin the literature, we will sometimes mention it.
6.1.3 Restrictions of functionsIf ð¹ ⶠ{0, 1}â â {0, 1} is a Boolean function and ð â â then the re-striction of ð¹ to inputs of length ð, denoted as ð¹ð, is the finite functionð ⶠ{0, 1}ð â {0, 1} such that ð(ð¥) = ð¹(ð¥) for every ð¥ â {0, 1}ð. Thatis, ð¹ð is the finite function that is only defined on inputs in {0, 1}ð, butagrees with ð¹ on those inputs. Since ð¹ð is a finite function, it can becomputed by a Boolean circuit, implying the following theorem:
Theorem 6.1 â Circuit collection for every infinite function. Let ð¹ ⶠ{0, 1}â â{0, 1}. Then there is a collection {ð¶ð}ðâ{1,2,âŠ} of circuits such thatfor every ð > 0, ð¶ð computes the restriction ð¹ð of ð¹ to inputs oflength ð.
Proof. This is an immediate corollary of the universality of Booleancircuits. Indeed, since ð¹ð maps {0, 1}ð to {0, 1}, Theorem 4.15 impliesthat there exists a Boolean circuit ð¶ð to compute it. In fact, the size ofthis circuit is at most ð â 2ð/ð gates for some constant ð †10.
â
In particular, Theorem 6.1 implies that there exists such a circuitcollection {ð¶ð} even for the TWINP function we described before,even though we do not know of any program to compute it. Indeed,this is not that surprising: for every particular ð â â, TWINPð is eitherthe constant zero function or the constant one function, both of which
-
222 introduction to theoretical computer science
can be computed by very simple Boolean circuits. Hence a collectionof circuits {ð¶ð} that computes TWINP certainly exists. The difficultyin computing TWINP using Python or any other programming lan-guage arises from the fact that we do not know for each particular ðwhat is the circuit ð¶ð in this collection.
6.2 DETERMINISTIC FINITE AUTOMATA (OPTIONAL)
All our computational models so far - Boolean circuits and straight-line programs - were only applicable for finite functions.
In Chapter 7, we will present Turing machines, which are the centralmodels of computation for unbounded input length functions. How-ever, in this section we present the more basic model of deterministicfinite automata (DFA). Automata can serve as a good stepping-stone forTuring machines, though they will not be used much in later parts ofthis book, and so the reader can feel free to skip ahead to Chapter 7.DFAs turn out to be equivalent in power to regular expressions: a pow-erful mechanism to specify patterns, which is widely used in practice.Our treatment of automata is relatively brief. There are plenty of re-sources that help you get more comfortable with DFAs. In particular,Chapter 1 of Sipserâs book [Sip97] contains an excellent exposition ofthis material. There are also many websites with online simulators forautomata, as well as translators from regular expressions to automataand vice versa (see for example here and here).
At a high level, an algorithm is a recipe for computing an outputfrom an input via a combination of the following steps:
1. Read a bit from the input2. Update the state (working memory)3. Stop and produce an output
For example, recall the Python program that computes the XORfunction:
def XOR(X):
'''Takes list X of 0's and 1's
Outputs 1 if the number of 1's is odd and outputs 0
otherwise'''âª
result = 0
for i in range(len(X)):
result = (result + X[i]) % 2
return result
In each step, this program reads a single bit X[i] and updates itsstate result based on that bit (flipping result if X[i] is 1 and keep-ing it the same otherwise). When it is done transversing the input,
http://ivanzuzak.info/noam/webapps/fsm2regex/https://cyberzhg.github.io/toolbox/nfa2dfa
-
functions with infinite domains, automata, and regular expressions 223
Figure 6.3: A deterministic finite automaton thatcomputes the XOR function. It has two states 0 and 1,and when it observes ð it transitions from ð£ to ð£ â ð.
the program outputs result. In computer science, such a program iscalled a single-pass constant-memory algorithm since it makes a singlepass over the input and its working memory is finite. (Indeed, in thiscase, result can either be 0 or 1.) Such an algorithm is also known asa Deterministic Finite Automaton or DFA (another name for DFAs is afinite state machine). We can think of such an algorithm as a âmachineâthat can be in one of ð¶ states, for some constant ð¶. The machine startsin some initial state and then reads its input ð¥ â {0, 1}â one bit at atime. Whenever the machine reads a bit ð â {0, 1}, it transitions into anew state based on ð and its prior state. The output of the machine isbased on the final state. Every single-pass constant-memory algorithmcorresponds to such a machine. If an algorithm uses ð bits of mem-ory, then the contents of its memory can be represented as a stringof length ð. Therefore such an algorithm can be in one of at most 2ðstates at any point in the execution.
We can specify a DFA of ð¶ states by a list of ð¶ â 2 rules. Each rulewill be of the form âIf the DFA is in state ð£ and the bit read from theinput is ð then the new state is ð£â²â. At the end of the computation,we will also have a rule of the form âIf the final state is one of thefollowing ⊠then output 1, otherwise output 0â. For example, thePython program above can be represented by a two-state automatonfor computing XOR of the following form:
⢠Initialize in the state 0.⢠For every state ð â {0, 1} and input bit ð read, if ð = 1 then change
to state 1 â ð , otherwise stay in state ð .⢠At the end output 1 iff ð = 1.
We can also describe a ð¶-state DFA as a labeled graph of ð¶ vertices.For every state ð and bit ð, we add a directed edge labeled with ðbetween ð and the state ð â² such that if the DFA is at state ð and reads ðthen it transitions to ð â². (If the state stays the same then this edge willbe a self-loop; similarly, if ð transitions to ð â² in both the case ð = 0 andð = 1 then the graph will contain two parallel edges.) We also labelthe set ð® of states on which the automaton will output 1 at the end ofthe computation. This set is known as the set of accepting states. SeeFig. 6.3 for the graphical representation of the XOR automaton.
Formally, a DFA is specified by (1) the table of the ð¶ â 2 rules, whichcan be represented as a transition function ð that maps a state ð â [ð¶]and bit ð â {0, 1} to the state ð â² â [ð¶] which the DFA will transition tofrom state ð on input ð and (2) the set ð® of accepting states. This leadsto the following definition.
-
224 introduction to theoretical computer science
Definition 6.2 â Deterministic Finite Automaton. A deterministic finiteautomaton (DFA) with ð¶ states over {0, 1} is a pair (ð , ð®) withð ⶠ[ð¶] à {0, 1} â [ð¶] and ð® â [ð¶]. The finite function ð is knownas the transition function of the DFA. The set ð® is known as the set ofaccepting states.
Let ð¹ ⶠ{0, 1}â â {0, 1} be a Boolean function with the infinitedomain {0, 1}â. We say that (ð , ð®) computes a function ð¹ ⶠ{0, 1}â â{0, 1} if for every ð â â and ð¥ â {0, 1}ð, if we define ð 0 = 0 andð ð+1 = ð (ð ð, ð¥ð) for every ð â [ð], then
ð ð â ð® â ð¹(ð¥) = 1 (6.6)
PMake sure not to confuse the transition function ofan automaton (ð in Definition 6.2), which is a finitefunction specifying the table of ârulesâ which it fol-lows, with the function the automaton computes (ð¹ inDefinition 6.2) which is an infinite function.
RRemark 6.3 â Definitions in other texts. Deterministicfinite automata can be defined in several equivalentways. In particular Sipser [Sip97] defines a DFA as afive-tuple (ð, Σ, ð¿, ð0, ð¹ ) where ð is the set of states,Σ is the alphabet, ð¿ is the transition function, ð0 isthe initial state, and ð¹ is the set of accepting states.In this book the set of states is always of the formð = {0, ⊠, ð¶ â 1} and the initial state is always ð0 = 0,but this makes no difference to the computationalpower of these models. Also, we restrict our attentionto the case that the alphabet Σ is equal to {0, 1}.
Solved Exercise 6.2 â DFA for (010)â. Prove that there is a DFA that com-putes the following function ð¹ :
ð¹(ð¥) =â§{âš{â©
1 3 divides |ð¥| and âðâ[|ð¥|/3]ð¥ðð¥ð+1ð¥ð+2 = 0100 otherwise
(6.7)
â
Solution:
When asked to construct a deterministic finite automaton, it isoften useful to start by constructing a single-pass constant-memory
-
functions with infinite domains, automata, and regular expressions 225
Figure 6.4: A DFA that outputs 1 only on inputsð¥ â {0, 1}â that are a concatenation of zero or morecopies of 010. The state 0 is both the starting stateand the only accepting state. The table denotes thetransition function of ð , which maps the current stateand symbol read to the new symbol.
algorithm using a more general formalism (for example, usingpseudocode or a Python program). Once we have such an algo-rithm, we can mechanically translate it into a DFA. Here is a simplePython program for computing ð¹ :
def F(X):
'''Return 1 iff X is a concatenation of zero/more
copies of [0,1,0]'''âª
if len(X) % 3 != 0:
return False
ultimate = 0
penultimate = 1
antepenultimate = 0
for idx, b in enumerate(X):
antepenultimate = penultimate
penultimate = ultimate
ultimate = b
if idx % 3 == 2 and ((antepenultimate,
penultimate, ultimate) != (0,1,0)):âª
return False
return True
Since we keep three Boolean variables, the working memory canbe in one of 23 = 8 configurations, and so the program above canbe directly translated into an 8 state DFA. While this is not neededto solve the question, by examining the resulting DFA, we can seethat we can merge some states and obtain a 4 state automaton, de-scribed in Fig. 6.4. See also Fig. 6.5, which depicts the execution ofthis DFA on a particular input.
â
6.2.1 Anatomy of an automaton (finite vs. unbounded)Now that we are considering computational tasks with unboundedinput sizes, it is crucial to distinguish between the components of ouralgorithm that have fixed length and the components that grow withthe input size. For the case of DFAs these are the following:
Constant size components: Given a DFA ðŽ, the following quantities arefixed independent of the input size:
⢠The number of states ð¶ in ðŽ.
⢠The transition function ð (which has 2ð¶ inputs, and so can be speci-fied by a table of 2ð¶ rows, each entry in which is a number in [ð¶]).
⢠The set ð® â [ð¶] of accepting states. This set can be described by astring in {0, 1}ð¶ specifiying which states are in ð® and which are not.
-
226 introduction to theoretical computer science
Together the above means that we can fully describe an automatonusing finitely many symbols. This is a property we require out of anynotion of âalgorithmâ: we should be able to write down a completespecification of how it produces an output from an input.
Components of unbounded size: The following quantities relating to aDFA are not bounded by any constant. We stress that these are stillfinite for any given input.
⢠The length of the input ð¥ â {0, 1}â that the DFA is provided. Theinput length is always finite, but not a priori bounded.
⢠The number of steps that the DFA takes can grow with the length ofthe input. Indeed, a DFA makes a single pass on the input and so ittakes precisely |ð¥| steps on an input ð¥ â {0, 1}â.
Figure 6.5: Execution of the DFA of Fig. 6.4. Thenumber of states and the transition function size arebounded, but the input can be arbitrarily long. Ifthe DFA is at state ð and observes the value ð then itmoves to the state ð (ð , ð). At the end of the executionthe DFA accepts iff the final state is in ð®.
6.2.2 DFA-computable functionsWe say that a function ð¹ ⶠ{0, 1}â â {0, 1} is DFA computable if thereexists some DFA that computes ð¹ . In Chapter 4 we saw that everyfinite function is computable by some Boolean circuit. Thus, at thispoint, you might expect that every infinite function is computable bysome DFA. However, this is very much not the case. We will soon seesome simple examples of infinite functions that are not computable byDFAs, but for starters, let us prove that such functions exist.
Theorem 6.4 â DFA-computable functions are countable. Let DFACOMP bethe set of all Boolean functions ð¹ ⶠ{0, 1}â â {0, 1} such that thereexists a DFA computing ð¹ . Then DFACOMP is countable.
Proof Idea:
-
functions with infinite domains, automata, and regular expressions 227
Every DFA can be described by a finite length string, which yieldsan onto map from {0, 1}â to DFACOMP: namely, the function thatmaps a string describing an automaton ðŽ to the function that it com-putes.
â
Proof of Theorem 6.4. Every DFA can be described by a finite string,representing the transition function ð and the set of accepting states,and every DFA ðŽ computes some function ð¹ ⶠ{0, 1}â â {0, 1}. Thuswe can define the following function ðð¡ð·ð¶ ⶠ{0, 1}â â DFACOMP:
ðð¡ð·ð¶(ð) =â§{âš{â©
ð¹ ð represents automaton ðŽ and ð¹ is the function ðŽ computesONE otherwise
(6.8)where ONE ⶠ{0, 1}â â {0, 1} is the constant function that outputs1 on all inputs (and is a member of DFACOMP). Since by definition,every function ð¹ in DFACOMP is computable by some automaton,ðð¡ð·ð¶ is an onto function from {0, 1}â to DFACOMP, which meansthat DFACOMP is countable (see Section 2.4.2).
â
Since the set of all Boolean functions is uncountable, we get thefollowing corollary:
Theorem 6.5 â Existence of DFA-uncomputable functions. There exists aBoolean function ð¹ ⶠ{0, 1}â â {0, 1} that is not computable by anyDFA.
Proof. If every Boolean function ð¹ is computable by some DFA, thenDFACOMP equals the set ALL of all Boolean functions, but by Theo-rem 2.12, the latter set is uncountable, contradicting Theorem 6.4.
â
6.3 REGULAR EXPRESSIONS
Searching for a piece of text is a common task in computing. At itsheart, the search problem is quite simple. We have a collection ð ={ð¥0, ⊠, ð¥ð} of strings (e.g., files on a hard-drive, or student records ina database), and the user wants to find out the subset of all the ð¥ â ðthat are matched by some pattern (e.g., all files whose names end withthe string .txt). In full generality, we can allow the user to specify thepattern by specifying a (computable) function ð¹ ⶠ{0, 1}â â {0, 1},where ð¹(ð¥) = 1 corresponds to the pattern matching ð¥. That is, theuser provides a program ð in a programming language such as Python,and the system returns all ð¥ â ð such that ð(ð¥) = 1. For example,
-
228 introduction to theoretical computer science
one could search for all text files that contain the string importantdocument or perhaps (letting ð correspond to a neural-network basedclassifier) all images that contain a cat. However, we donât want oursystem to get into an infinite loop just trying to evaluate the programð ! For this reason, typical systems for searching files or databases donot allow users to specify the patterns using full-fledged programminglanguages. Rather, such systems use restricted computational models thaton the one hand are rich enough to capture many of the queries neededin practice (e.g., all filenames ending with .txt, or all phone numbersof the form (617) xxx-xxxx), but on the other hand are restrictedenough so that queries can be evaluated very efficiently on huge filesand in particular cannot result in an infinite loop.
One of the most popular such computational models is regularexpressions. If you ever used an advanced text editor, a command-lineshell, or have done any kind of manipulation of text files, then youhave probably come across regular expressions.
A regular expression over some alphabet Σ is obtained by combin-ing elements of Σ with the operation of concatenation, as well as |(corresponding to or) and â (corresponding to repetition zero ormore times). (Common implementations of regular expressions inprogramming languages and shells typically include some extra oper-ations on top of | and â, but these operations can be implemented asâsyntactic sugarâ using the operators | and â.) For example, the fol-lowing regular expression over the alphabet {0, 1} corresponds to theset of all strings ð¥ â {0, 1}â where every digit is repeated at least twice:
(00(0â)|11(1â))â . (6.9)The following regular expression over the alphabet {ð, ⊠, ð§, 0, ⊠, 9}
corresponds to the set of all strings that consist of a sequence of oneor more of the letters ð-ð followed by a sequence of one or more digits(without a leading zero):
(ð|ð|ð|ð)(ð|ð|ð|ð)â(1|2|3|4|5|6|7|8|9)(0|1|2|3|4|5|6|7|8|9)â . (6.10)
Formally, regular expressions are defined by the following recursivedefinition:
Definition 6.6 â Regular expression. A regular expression ð over an al-phabet Σ is a string over Σ ⪠{(, ), |, â,â , ""} that has one of thefollowing forms:
1. ð = ð where ð â Σ
2. ð = (ðâ²|ðâ³) where ðâ², ðâ³ are regular expressions.
https://goo.gl/2vTAFUhttps://goo.gl/2vTAFU
-
functions with infinite domains, automata, and regular expressions 229
3. ð = (ðâ²)(ðâ³) where ðâ², ðâ³ are regular expressions. (We oftendrop the parentheses when there is no danger of confusion andso write this as ðâ² ðâ³.)
4. ð = (ðâ²)â where ðâ² is a regular expression.
Finally we also allow the following âedge casesâ: ð = â andð = "". These are the regular expressions corresponding to accept-ing no strings, and accepting only the empty string respectively.
We will drop parentheses when they can be inferred from thecontext. We also use the convention that OR and concatenation areleft-associative, and we give highest precedence to â, then concate-nation, and then OR. Thus for example we write 00â|11 instead of((0)(0â))|((1)(1)).
Every regular expression ð corresponds to a function Ίð ⶠΣâ â{0, 1} where Ίð(ð¥) = 1 if ð¥ matches the regular expression. For exam-ple, if ð = (00|11)â then Ίð(110011) = 1 but Ίð(101) = 0 (can you seewhy?).
PThe formal definition of Ίð is one of those definitionsthat is more cumbersome to write than to grasp. Thusit might be easier for you first to work out the defini-tion on your own, and then check that it matches whattis written below.
Definition 6.7 â Matching a regular expression. Let ð be a regular expres-sion over the alphabet Σ. The function Ίð ⶠΣâ â {0, 1} is definedas follows:
1. If ð = ð then Ίð(ð¥) = 1 iff ð¥ = ð.
2. If ð = (ðâ²|ðâ³) then Ίð(ð¥) = Ίðâ²(ð¥)âšÎŠðâ³(ð¥) where âš is the OR op-erator.
3. If ð = (ðâ²)(ðâ³) then Ίð(ð¥) = 1 iff there is some ð¥â², ð¥â³ â Σâ suchthat ð¥ is the concatenation of ð¥â² and ð¥â³ and Ίðâ²(ð¥â²) = Ίðâ³(ð¥â³) =1.
4. If ð = (ðâ²)â then Ίð(ð¥) = 1 iff there is some ð â â and someð¥0, ⊠, ð¥ðâ1 â Σâ such that ð¥ is the concatenation ð¥0 ⯠ð¥ðâ1 andΊðâ²(ð¥ð) = 1 for every ð â [ð].
5. Finally, for the edge cases Ίâ is the constant zero function, andΊ"" is the function that only outputs 1 on the empty string "".
-
230 introduction to theoretical computer science
We say that a regular expression ð over Σ matches a string ð¥ â Σâif Ίð(ð¥) = 1.
PThe definitions above are not inherently difficult butare a bit cumbersome. So you should pause here andgo over it again until you understand why it corre-sponds to our intuitive notion of regular expressions.This is important not just for understanding regularexpressions themselves (which are used time andagain in a great many applications) but also for get-ting better at understanding recursive definitions ingeneral.
A Boolean function is called âregularâ if it outputs 1 on preciselythe set of strings that are matched by some regular expression. That is,
Definition 6.8 â Regular functions / languages. Let Σ be a finite set andð¹ ⶠΣâ â {0, 1} be a Boolean function. We say that ð¹ is regular ifð¹ = Ίð for some regular expression ð.
Similarly, for every formal language ð¿ â Σâ, we say that ð¿ is reg-ular if and only if there is a regular expression ð such that ð¥ â ð¿ iffð matches ð¥.
â Example 6.9 â A regular function. Let Σ = {ð, ð, ð, ð, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9}and ð¹ ⶠΣâ â {0, 1} be the function such that ð¹(ð¥) outputs 1 iffð¥ consists of one or more of the letters ð-ð followed by a sequenceof one or more digits (without a leading zero). Then ð¹ is a regularfunction, since ð¹ = Ίð where
ð = (ð|ð|ð|ð)(ð|ð|ð|ð)â(0|1|2|3|4|5|6|7|8|9)(0|1|2|3|4|5|6|7|8|9)â(6.11)
is the expression we saw in (6.10).If we wanted to verify, for example, that Ίð(ððð12078) = 1,
we can do so by noticing that the expression (ð|ð|ð|ð) matchesthe string ð, (ð|ð|ð|ð)â matches ðð, (0|1|2|3|4|5|6|7|8|9) matches thestring 1, and the expression (0|1|2|3|4|5|6|7|8|9)â matches the string2078. Each one of those boils down to a simpler expression. For ex-ample, the expression (ð|ð|ð|ð)â matches the string ðð because bothof the one-character strings ð and ð are matched by the expressionð|ð|ð|ð.
Regular expression can be defined over any finite alphabet Σ, butas usual, we will mostly focus our attention on the binary case, where
-
functions with infinite domains, automata, and regular expressions 231
Σ = {0, 1}. Most (if not all) of the theoretical and practical generalinsights about regular expressions can be gleaned from studying thebinary case.
6.3.1 Algorithms for matching regular expressionsRegular expressions would not be very useful for search if we couldnot evaluate, given a regular expression ð, whether a string ð¥ ismatched by ð. Luckily, there is an algorithm to do so. Specifically,there is an algorithm (think âPython programâ though later wewill formalize the notion of algorithms using Turing machines) thaton input a regular expression ð over the alphabet {0, 1} and a stringð¥ â {0, 1}â, outputs 1 iff ð matches ð¥ (i.e., outputs Ίð(ð¥)).
Indeed, Definition 6.7 actually specifies a recursive algorithm forcomputing Ίð. Specifically, each one of our operations -concatenation,OR, and star- can be thought of as reducing the task of testing whetheran expression ð matches a string ð¥ to testing whether some sub-expressions of ð match substrings of ð¥. Since these sub-expressionsare always shorter than the original expression, this yields a recursivealgorithm for checking if ð matches ð¥, which will eventually terminateat the base cases of the expressions that correspond to a single symbolor the empty string.
-
232 introduction to theoretical computer science
Algorithm 6.10 â Regular expression matching.
Input: Regular expression ð over Σâ, ð¥ â ΣâOutput: Ίð(ð¥)1: procedure Match(ð,ð¥)2: if ð = â then return 0 ;3: if ð¥ = "" then return MatchEmpty(()ð) ;4: if ð â Σ then return 1 iff ð¥ = ð ;5: if ð = (ðâ²|ðâ³) then return Match(ðâ², ð¥) or Match(ðâ³, ð¥)
;6: if ð = (ðâ²)(ðâ³) then7: for ð â [|ð¥| + 1] do8: if Match(ðâ², ð¥0 ⯠ð¥ðâ1) and Match(ðâ³, ð¥ð ⯠ð¥|ð¥|â1)
then return 1 ;9: end for
10: end if11: if ð = (ðâ²)â then12: if ðâ² = "" then return Match("", ð¥) ;13: # ("")â is the same as ""14: for ð â [|ð¥|] do15: # ð¥0 ⯠ð¥ðâ1 is shorter than ð¥16: if Match(ð, ð¥0 ⯠ð¥ðâ1) and Match(ðâ², ð¥ð ⯠ð¥|ð¥|â1)
then return 1 ;17: end for18: end if19: return 020: end procedure
We assume above that we have a procedure MatchEmpty thaton input a regular expression ð outputs 1 if and only if ð matches theempty string "".
The key observation is that in our recursive definition of regular ex-pressions, whenever ð is made up of one or two expressions ðâ², ðâ³ thenthese two regular expressions are smaller than ð. Eventually (whenthey have size 1) then they must correspond to the non-recursivecase of a single alphabet symbol. Correspondingly, the recursive callsmade in Algorithm 6.10 always correspond to a shorter expression or(in the case of an expression of the form (ðâ²)â) a shorter input string.Thus, we can prove the correctness of Algorithm 6.10 on inputs of theform (ð, ð¥) by induction over min{|ð|, |ð¥|}. The base case is when ei-ther ð¥ = "" or ð is a single alphabet symbol, "" or â . In the case theexpression is of the forrm ð = (ðâ²|ðâ³) or ð = (ðâ²)(ðâ³), we make recur-sive calls with the shorter expressions ðâ², ðâ³. In the case the expressionis of the form ð = (ðâ²)â, we make recursive calls with either a shorter
-
functions with infinite domains, automata, and regular expressions 233
string ð¥ and the same expression, or with the shorter expression ðâ²and a string ð¥â² that is equal in length or shorter than ð¥.Solved Exercise 6.3 â Match the empty string. Give an algorithm that oninput a regular expression ð, outputs 1 if and only if Ίð("") = 1.
â
Solution:
We can obtain such a recursive algorithm by using the followingobservations:
1. An expression of the form "" or (ðâ²)â always matches the emptystring.
2. An expression of the form ð, where ð â Σ is an alphabet sym-bol, never matches the empty string.
3. The regular expression â does not match the empty string.
4. An expression of the form ðâ²|ðâ³ matches the empty string if andonly if one of ðâ² or ðâ³ matches it.
5. An expression of the form (ðâ²)(ðâ³) matches the empty string ifand only if both ðâ² and ðâ³ match it.
Given the above observations, we see that the following algo-rithm will check if ð matches the empty string:
procedure{MatchEmpty}{ð} lIf {ð = â } return 0 lendif lIf{ð = ""} return 1 lendif lIf {ð = â or ð â Σ} return 0 lendif lIf{ð = (ðâ²|ðâ³)} return ððð¡ðâðžððð¡ðŠ(ðâ²) or ððð¡ðâðžððð¡ðŠ(ðâ³) lendifLIf {ð = (ðâ²)(ðâ²)} return ððð¡ðâðžððð¡ðŠ(ðâ²) or ððð¡ðâðžððð¡ðŠ(ðâ³)lendif lIf {ð = (ðâ²)â} return 1 lendif endprocedure
â
6.4 EFFICIENT MATCHING OF REGULAR EXPRESSIONS (OP-TIONAL)
Algorithm 6.10 is not very efficient. For example, given an expressioninvolving concatenation or the âstarâ operation and a string of lengthð, it can make ð recursive calls, and hence it can be shown that in theworst case Algorithm 6.10 can take time exponential in the length ofthe input string ð¥. Fortunately, it turns out that there is a much moreefficient algorithm that can match regular expressions in linear (i.e.,ð(ð)) time. Since we have not yet covered the topics of time and spacecomplexity, we describe this algorithm in high level terms, withoutmaking the computational model precise. Rather we will use thecolloquial notion of ð(ð) running time as used in introduction to
-
234 introduction to theoretical computer science
programming courses and whiteboard coding interviews. We will seea formal definition of time complexity in Chapter 13.
Theorem 6.11 â Matching regular expressions in linear time. Let ð be aregular expression. Then there is an ð(ð) time algorithm thatcomputes Ίð.
The implicit constant in the ð(ð) term of Theorem 6.11 depends onthe expression ð. Thus, another way to state Theorem 6.11 is that forevery expression ð, there is some constant ð and an algorithm ðŽ thatcomputes Ίð on ð-bit inputs using at most ð â ð steps. This makes sensesince in practice we often want to compute Ίð(ð¥) for a small regularexpression ð and a large document ð¥. Theorem 6.11 tells us that wecan do so with running time that scales linearly with the size of thedocument, even if it has (potentially) worse dependence on the size ofthe regular expression.
We prove Theorem 6.11 by obtaining more efficient recursive al-gorithm, that determines whether ð matches a string ð¥ â {0, 1}ð byreducing this task to determining whether a related expression ðâ²matches ð¥0, ⊠, ð¥ðâ2. This will result in an expression for the runningtime of the form ð (ð) = ð (ð â 1) + ð(1) which solves to ð (ð) = ð(ð).
Restrictions of regular expressions. The central definition for the algo-rithm behind Theorem 6.11 is the notion of a restriction of a regularexpression. The idea is that for every regular expression ð and symbolð in its alphabet, it is possible to define a regular expression ð[ð] suchthat ð[ð] matches a string ð¥ if and only if ð matches the string ð¥ð. Forexample, if ð is the regular expression 01|(01) â (01) (i.e., one or moreoccurrences of 01) then ð[1] is equal to 0|(01) â 0 and ð[0] will be â .(Can you see why?)
Algorithm 6.12 computes the resriction ð[ð] given a regular ex-pression ð and an alphabet symbol ð. It always terminates, since therecursive calls it makes are always on expressions smaller than theinput expression. Its correctness can be proven by induction on thelength of the regular expression ð, with the base cases being when ð is"", â , or a single alphabet symbol ð .
-
functions with infinite domains, automata, and regular expressions 235
Algorithm 6.12 â Restricting regular expression.
Input: Regular expression ð over Σ, symbol ð â ΣOutput: Regular expression ðâ² = ð[ð] such that Ίðâ²(ð¥) =
Ίð(ð¥ð) for every ð¥ â Σâ1: procedure Restrict(ð,ð)2: if ð = "" or ð = â then return â ;3: if ð = ð for ð â Σ then return "" if ð = ð and return
â otherwise ;4: if ð = (ðâ²|ðâ³) then return (Restrict(ðâ², ð)|Restrict(ðâ³, ð))
;5: if ð = (ðâ²)â then return (ðâ²)â(Restrict(ðâ², ð)) ;6: if ð = (ðâ²)(ðâ³) and Ίðâ³("") = 0 then return
(ðâ²)(Restrict(ðâ³, ð)) ;7: if ð = (ðâ²)(ðâ³) and Ίðâ³("") = 1 then return
(ðâ²)(Restrict(ðâ³, ð) | Restrict(ðâ², ð)) ;8: end procedure
Using this notion of restriction, we can define the following recur-sive algorithm for regular expression matching:
Algorithm 6.13 â Regular expression matching in linear time.
Input: Regular expression ð over Σâ, ð¥ â Σð where ð â âOutput: Ίð(ð¥)1: procedure FMatch(ð,ð¥)2: if ð¥ = "" then return MatchEmpty(()ð) ;3: Let ðâ² â Restrict(ð, ð¥ðâ2)4: return FMatch(ðâ², ð¥0 ⯠ð¥ðâ1)5: end procedure
By the definition of a restriction, for every ð â Σ and ð¥â² â Σâ,the expression ð matches ð¥â²ð if and only if ð[ð] matches ð¥â². Hence forevery ð and ð¥ â Σð, Ίð[ð¥ðâ1](ð¥0 ⯠ð¥ðâ2) = Ίð(ð¥) and Algorithm 6.13does return the correct answer. The only remaining task is to analyzeits running time. Note that Algorithm 6.13 uses the MatchEmptyprocedure of Solved Exercise 6.3 in the base case that ð¥ = "". However,this is OK since this procedureâs running time depends only on ð andis independent of the length of the original input.
For simplicity, let us restrict our attention to the case that the al-phabet Σ is equal to {0, 1}. Define ð¶(â) to be the maximum numberof operations that Algorithm 6.12 takes when given as input a regularexpression ð over {0, 1} of at most â symbols. The value ð¶(â) can beshown to be polynomial in â, though this is not important for this the-orem, since we only care about the dependence of the time to compute
-
236 introduction to theoretical computer science
Ίð(ð¥) on the length of ð¥ and not about the dependence of this time onthe length of ð.
Algorithm 6.13 is a recursive algorithm that input an expressionð and a string ð¥ â {0, 1}ð, does computation of at most ð¶(|ð|) stepsand then calls itself with input some expression ðâ² and a string ð¥â² oflength ð â 1. It will terminate after ð steps when it reaches a string oflength 0. So, the running time ð (ð, ð) that it takes for Algorithm 6.13to compute Ίð for inputs of length ð satisfies the recursive equation:
ð (ð, ð) = max{ð (ð[0], ð â 1), ð (ð[1], ð â 1)} + ð¶(|ð|) (6.12)
(In the base case ð = 0, ð (ð, 0) is equal to some constant dependingonly on ð.) To get some intuition for the expression Eq. (6.12), let usopen up the recursion for one level, writing ð (ð, ð) as
ð (ð, ð) = max{ð (ð[0][0], ð â 2) + ð¶(|ð[0]|),ð (ð[0][1], ð â 2) + ð¶(|ð[0]|),ð (ð[1][0], ð â 2) + ð¶(|ð[1]|),ð (ð[1][1], ð â 2) + ð¶(|ð[1]|)} + ð¶(|ð|) .
(6.13)
Continuing this way, we can see that ð (ð, ð) †ð â ð¶(ð¿) + ð(1)where ð¿ is the largest length of any expression ðâ² that we encounteralong the way. Therefore, the following claim suffices to show thatAlgorithm 6.13 runs in ð(ð) time:
Claim: Let ð be a regular expression over {0, 1}, then there is a num-ber ð¿(ð) â â, such that for every sequence of symbols ðŒ0, ⊠, ðŒðâ1, ifwe define ðâ² = ð[ðŒ0][ðŒ1] ⯠[ðŒðâ1] (i.e., restricting ð to ðŒ0, and then ðŒ1and so on and so forth), then |ðâ²| †ð¿(ð).
Proof of claim: For a regular expression ð over {0, 1} and ðŒ â {0, 1}ð,we denote by ð[ðŒ] the expression ð[ðŒ0][ðŒ1] ⯠[ðŒðâ1] obtained by restrict-ing ð to ðŒ0 and then to ðŒ1 and so on. We let ð(ð) = {ð[ðŒ]|ðŒ â {0, 1}â}.We will prove the claim by showing that for every ð, the set ð(ð) is fi-nite, and hence so is the number ð¿(ð) which is the maximum length ofðâ² for ðâ² â ð(ð).We prove this by induction on the structure of ð. If ð is a symbol, theempty string, or the empty set, then this is straightforward to showas the most expressions ð(ð) can contain are the expression itself, "",and â . Otherwise we split to the two cases (i) ð = ðâ²â and (ii) ð =ðâ²ðâ³, where ðâ², ðâ³ are smaller expressions (and hence by the inductionhypothesis ð(ðâ²) and ð(ðâ³) are finite). In the case (i), if ð = (ðâ²)â thenð[ðŒ] is either equal to (ðâ²)âðâ²[ðŒ] or it is simply the empty set if ðâ²[ðŒ] = â .Since ðâ²[ðŒ] is in the set ð(ðâ²), the number of distinct expressions inð(ð) is at most |ð(ðâ²)| + 1. In the case (ii), if ð = ðâ²ðâ³ then all therestrictions of ð to strings ðŒ will either have the form ðâ²ðâ³[ðŒ] or the formðâ²ðâ³[ðŒ]|ðâ²[ðŒâ²] where ðŒâ² is some string such that ðŒ = ðŒâ²ðŒâ³ and ðâ³[ðŒâ³]
-
functions with infinite domains, automata, and regular expressions 237
matches the empty string. Since ðâ³[ðŒ] â ð(ðâ³) and ðâ²[ðŒâ²] â ð(ðâ²), thenumber of the possible distinct expressions of the form ð[ðŒ] is at most|ð(ðâ³)| + |ð(ðâ³)| â |ð(ðâ²)|. This completes the proof of the claim.
The bottom line is that while running Algorithm 6.13 on a regularexpression ð, all the expressions we ever encounter are in the finite setð(ð), no matter how large the input ð¥ is, and so the running time ofAlgorithm 6.13 satisfies the equation ð (ð) = ð (ð â 1) + ð¶â² for someconstant ð¶â² depending on ð. This solves to ð(ð) where the implicitconstant in the O notation can (and will) depend on ð but crucially,not on the length of the input ð¥.
6.4.1 Matching regular expressions using DFAsTheorem 6.11 is already quite impressive, but we can do even better.Specifically, no matter how long the string ð¥ is, we can compute Ίð(ð¥)by maintaining only a constant amount of memory and moreovermaking a single pass over ð¥. That is, the algorithm will scan the inputð¥ once from start to finish, and then determine whether or not ð¥ ismatched by the expression ð. This is important in the common caseof trying to match a short regular expression over a huge file or docu-ment that might not even fit in our computerâs memory. Of course, aswe have seen before, a single-pass constant-memory algorithm is sim-ply a deterministic finite automaton. As we will see in Theorem 6.16, afunction can be computed by regular expression if and only if it can becomputed by a DFA. We start with showing the âonly ifâ direction:
Theorem 6.14 â DFA for regular expression matching. Let ð be a regularexpression. Then there is an algorithm that on input ð¥ â {0, 1}âcomputes Ίð(ð¥) while making a single pass over ð¥ and maintaininga constant amount of memory.
Proof Idea:
The single-pass constant-memory for checking if a string matchesa regular expression is presented in Algorithm 6.15. The idea is toreplace the recursive algorithm of Algorithm 6.13 with a dynamic pro-gram, using the technique of memoization. If you havenât taken yet analgorithms course, you might not know these techniques. This is OK;while this more efficient algorithm is crucial for the many practicalapplications of regular expressions, it is not of great importance forthis book.
â
https://goo.gl/kgLdX1https://goo.gl/kgLdX1https://en.wikipedia.org/wiki/Memoization
-
238 introduction to theoretical computer science
Algorithm 6.15 â Regular expression matching by a DFA.
Input: Regular expression ð over Σâ, ð¥ â Σð where ð â âOutput: Ίð(ð¥)1: procedure DFAMatch(ð,ð¥)2: Let ð â ð(ð) be the set {ð[ðŒ]|ðŒ â {0, 1}â} as defined
in the proof of [reglintimethm]().ref.3: for ðâ² â ð do4: Let ð£ðâ² â 1 if Ίðâ²("") = 1 and ð£ðâ² â 0 otherwise5: end for6: for ð â [ð] do7: Let ððð ð¡ðâ² â ð£ðâ² for all ðâ² â ð8: Let ð£ðâ² â ððð ð¡ðâ²[ð¥ð] for all ðâ² â ð9: end for
10: return ð£ð11: end procedure
Proof of Theorem 6.14. Algorithm 6.15 checks if a given string ð¥ â Σâis matched by the regular expression ð. For every regular expres-sion ð, this algorithm has a constant number 2|ð(ð)| Boolean vari-ables (ð£ðâ² , ððð ð¡ðâ² for ðâ² â ð(ð)), and it makes a single pass overthe input string. Hence it corresponds to a DFA. We prove its cor-rectness by induction on the length ð of the input. Specifically, wewill argue that before reading the ð-th bit of ð¥, the variable ð£ðâ² isequal to Ίðâ²(ð¥0 ⯠ð¥ðâ1) for every ðâ² â ð(ð). In the case ð = 0 thisholds since we initialize ð£ðâ² = Ίðâ²("") for all ðâ² â ð(ð). For ð > 0this holds by induction since the inductive hypothesis implies thatððð ð¡â²ð = Ίðâ²(ð¥0 ⯠ð¥ðâ2) for all ðâ² â ð(ð) and by the definition of the setð(ðâ²), for every ðâ² â ð(ð) and ð¥ðâ1 â Σ, ðâ³ = ðâ²[ð¥ðâ1] is in ð(ð) andΊðâ²(ð¥0 ⯠ð¥ðâ1) = Ίðâ³(ð¥0 ⯠ð¥ð).
â
6.4.2 Equivalence of regular expressions and automataRecall that a Boolean function ð¹ ⶠ{0, 1}â â {0, 1} is defined to beregular if it is equal to Ίð for some regular expression ð. (Equivalently,a language ð¿ â {0, 1}â is defined to be regular if there is a regularexpression ð such that ð matches ð¥ iff ð¥ â ð¿.) The following theorem isthe central result of automata theory:
Theorem 6.16 â DFA and regular expression equivalency. Let ð¹ ⶠ{0, 1}â â{0, 1}. Then ð¹ is regular if and only if there exists a DFA (ð , ð®) thatcomputes ð¹ .
Proof Idea:
-
functions with infinite domains, automata, and regular expressions 239
Figure 6.6: A deterministic finite automaton thatcomputes the function Ί(01)â .
Figure 6.7: Given a DFA of ð¶ states, for every ð£, ð€ â[ð¶] and number ð¡ â {0, ⊠, ð¶} we define the functionð¹ ð¡ð£,ð€ ⶠ{0, 1}â â {0, 1} to output one on inputð¥ â {0, 1}â if and only if when the DFA is initializedin the state ð£ and is given the input ð¥, it will reach thestate ð€ while going only through the intermediatestates {0, ⊠, ð¡ â 1}.
One direction follows from Theorem 6.14, which shows that forevery regular expression ð, the function Ίð can be computed by a DFA(see for example Fig. 6.6). For the other direction, we show that givena DFA (ð , ð®) for every ð£, ð€ â [ð¶] we can find a regular expression thatwould match ð¥ â {0, 1}â if and only if the DFA starting in state ð£, willend up in state ð€ after reading ð¥.
â
Proof of Theorem 6.16. Since Theorem 6.14 proves the âonly ifâ direc-tion, we only need to show the âifâ direction. Let ðŽ = (ð , ð®) be a DFAwith ð¶ states that computes the function ð¹ . We need to show that ð¹ isregular.
For every ð£, ð€ â [ð¶], we let ð¹ð£,ð€ ⶠ{0, 1}â â {0, 1} be the functionthat maps ð¥ â {0, 1}â to 1 if and only if the DFA ðŽ, starting at thestate ð£, will reach the state ð€ if it reads the input ð¥. We will prove thatð¹ð£,ð€ is regular for every ð£, ð€. This will prove the theorem, since byDefinition 6.2, ð¹(ð¥) is equal to the OR of ð¹0,ð€(ð¥) for every ð€ â ð®.Hence if we have a regular expression for every function of the formð¹ð£,ð€ then (using the | operation), we can obtain a regular expressionfor ð¹ as well.
To give regular expressions for the functions ð¹ð£,ð€, we start bydefining the following functions ð¹ ð¡ð£,ð€: for every ð£, ð€ â [ð¶] and0 †ð¡ †ð¶, ð¹ ð¡ð£,ð€(ð¥) = 1 if and only if starting from ð£ and observ-ing ð¥, the automata reaches ð€ with all intermediate states being in the set[ð¡] = {0, ⊠, ð¡ â 1} (see Fig. 6.7). That is, while ð£, ð€ themselves mightbe outside [ð¡], ð¹ ð¡ð£,ð€(ð¥) = 1 if and only if throughout the execution ofthe automaton on the input ð¥ (when initiated at ð£) it never enters anyof the states outside [ð¡] and still ends up at ð€. If ð¡ = 0 then [ð¡] is theempty set, and hence ð¹ 0ð£,ð€(ð¥) = 1 if and only if the automaton reachesð€ from ð£ directly on ð¥, without any intermediate state. If ð¡ = ð¶ thenall states are in [ð¡], and hence ð¹ ð¡ð£,ð€ = ð¹ð£,ð€.
We will prove the theorem by induction on ð¡, showing that ð¹ ð¡ð£,ð€ isregular for every ð£, ð€ and ð¡. For the base case of ð¡ = 0, ð¹ 0ð£,ð€ is regularfor every ð£, ð€ since it can be described as one of the expressions "", â ,0, 1 or 0|1. Specifically, if ð£ = ð€ then ð¹ 0ð£,ð€(ð¥) = 1 if and only if ð¥ isthe empty string. If ð£ â ð€ then ð¹ 0ð£,ð€(ð¥) = 1 if and only if ð¥ consistsof a single symbol ð â {0, 1} and ð (ð£, ð) = ð€. Therefore in this caseð¹ 0ð£,ð€ corresponds to one of the four regular expressions 0|1, 0, 1 or â ,depending on whether ðŽ transitions to ð€ from ð£ when it reads either 0or 1, only one of these symbols, or neither.
Inductive step: Now that weâve seen the base case, let us prove thegeneral case by induction. Assume, via the induction hypothesis, thatfor every ð£â², ð€â² â [ð¶], we have a regular expression ð ð¡ð£,ð€ that computesð¹ ð¡ð£â²,ð€â² . We need to prove that ð¹ ð¡+1ð£,ð€ is regular for every ð£, ð€. If the
-
240 introduction to theoretical computer science
automaton arrives from ð£ to ð€ using the intermediate states [ð¡ + 1],then it visits the ð¡-th state zero or more times. If the path labeled by ð¥causes the automaton to get from ð£ to ð€ without visiting the ð¡-th stateat all, then ð¥ is matched by the regular expression ð ð¡ð£,ð€. If the pathlabeled by ð¥ causes the automaton to get from ð£ to ð€ while visiting theð¡-th state ð > 0 times then we can think of this path as:⢠First travel from ð£ to ð¡ using only intermediate states in [ð¡ â 1].
⢠Then go from ð¡ back to itself ð â 1 times using only intermediatestates in [ð¡ â 1]
⢠Then go from ð¡ to ð€ using only intermediate states in [ð¡ â 1].Therefore in this case the string ð¥ is matched by the regular expres-
sion ð ð¡ð£,ð¡(ð ð¡ð¡,ð¡)âð ð¡ð¡,ð€. (See also Fig. 6.8.)Therefore we can compute ð¹ ð¡+1ð£,ð€ using the regular expression
ð ð¡ð£,ð€ | ð ð¡ð£,ð¡(ð ð¡ð¡,ð¡)âð ð¡ð¡,ð€ . (6.14)This completes the proof of the inductive step and hence of the theo-rem.
â
Figure 6.8: If we have regular expressions ð ð¡ð£â²,ð€â²corresponding to ð¹ ð¡ð£â²,ð€â² for every ð£â², ð€â² â [ð¶], we canobtain a regular expression ð ð¡+1ð£,ð€ corresponding toð¹ ð¡+1ð£,ð€ . The key observation is that a path from ð£ to ð€using {0, ⊠, ð¡} either does not touch ð¡ at all, in whichcase it is captured by the expression ð ð¡ð£,ð€, or it goesfrom ð£ to ð¡, comes back to ð¡ zero or more times, andthen goes from ð¡ to ð€, in which case it is captured bythe expression ð ð¡ð£,ð¡(ð ð¡ð¡,ð¡)âð ð¡ð¡,ð€.
6.4.3 Closure properties of regular expressionsIf ð¹ and ðº are regular functions computed by the expressions ð and ðrespectively, then the expression ð|ð computes the function ð» = ð¹ âš ðºdefined as ð»(ð¥) = ð¹(ð¥) âš ðº(ð¥). Another way to say this is that the setof regular functions is closed under the OR operation. That is, if ð¹ and ðºare regular then so is ð¹ âš ðº. An important corollary of Theorem 6.16is that this set is also closed under the NOT operation:
-
functions with infinite domains, automata, and regular expressions 241
Lemma 6.17 â Regular expressions closed under complement. If ð¹ ⶠ{0, 1}â â{0, 1} is regular then so is the function ð¹ , where ð¹(ð¥) = 1 â ð¹(ð¥) forevery ð¥ â {0, 1}â.
Proof. If ð¹ is regular then by Theorem 6.11 it can be computed by aDFA ðŽ = (ð , ð) with some ð¶ states. But then the DFA ðŽ = (ð , [ð¶]⧵ð)which does the same computation but where flips the set of acceptedstates will compute ð¹ . By Theorem 6.16 this implies that ð¹ is regularas well.
â
Since ð ⧠ð = ð âš ð, Lemma 6.17 implies that the set of regularfunctions is closed under the AND operation as well. Moreover, sinceOR, NOT and AND are a universal basis, this set is also closed un-der NAND, XOR, and any other finite function. That is, we have thefollowing corollary:
Theorem 6.18 â Closure of regular expressions. Let ð ⶠ{0, 1}ð â {0, 1} beany finite Boolean function, and let ð¹0, ⊠, ð¹ðâ1 ⶠ{0, 1}â â {0, 1} beregular functions. Then the function ðº(ð¥) = ð(ð¹0(ð¥), ð¹1(ð¥), ⊠, ð¹ðâ1(ð¥))is regular.
Proof. This is a direct consequence of the closure of regular functionsunder OR and NOT (and hence AND), combined with Theorem 4.13,that states that every ð can be computed by a Boolean circuit (which issimply a combination of the AND, OR, and NOT operations).
â
6.5 LIMITATIONS OF REGULAR EXPRESSIONS AND THE PUMPINGLEMMA
The efficiency of regular expression matching makes them very useful.This is why operating systems and text editors often restrict theirsearch interface to regular expressions and do not allow searching byspecifying an arbitrary function. However, this efficiency comes ata cost. As we have seen, regular expressions cannot compute everyfunction. In fact, there are some very simple (and useful!) functionsthat they cannot compute. Here is one example:
Lemma 6.19 â Matching parentheses. Let Σ = {âš, â©} and MATCHPAREN â¶Î£â â {0, 1} be the function that given a string of parentheses, out-puts 1 if and only if every opening parenthesis is matched by a corre-sponding closed one. Then there is no regular expression over Σ thatcomputes MATCHPAREN.
Lemma 6.19 is a consequence of the following result, which isknown as the pumping lemma:
-
242 introduction to theoretical computer science
Theorem 6.20 â Pumping Lemma. Let ð be a regular expression oversome alphabet Σ. Then there is some number ð0 such that for ev-ery ð€ â Σâ with |ð€| > ð0 and Ίð(ð€) = 1, we can write ð€ = ð¥ðŠð§ forstrings ð¥, ðŠ, ð§ â Σâ satisfying the following conditions:
1. |ðŠ| ⥠1.
2. |ð¥ðŠ| †ð0.
3. Ίð(ð¥ðŠðð§) = 1 for every ð â â.
Figure 6.9: To prove the âpumping lemmaâ we lookat a word ð€ that is much larger than the regularexpression ð that matches it. In such a case, part ofð€ must be matched by some sub-expression of theform (ðâ²)â, since this is the only operator that allowsmatching words longer than the expression. If welook at the âleftmostâ such sub-expression and defineðŠð to be the string that is matched by it, we obtain thepartition needed for the pumping lemma.
Proof Idea:
The idea behind the proof the following. Let ð0 be twice the num-ber of symbols that are used in the expression ð, then the only waythat there is some ð€ with |ð€| > ð0 and Ίð(ð€) = 1 is that ð containsthe â (i.e. star) operator and that there is a non-empty substring ðŠ ofð€ that was matched by (ðâ²)â for some sub-expression ðâ² of ð. We cannow repeat ðŠ any number of times and still get a matching string. Seealso Fig. 6.9.
â
PThe pumping lemma is a bit cumbersome to state,but one way to remember it is that it simply says thefollowing: âif a string matching a regular expression islong enough, one of its substrings must be matched usingthe â operatorâ.
-
functions with infinite domains, automata, and regular expressions 243
Proof of Theorem 6.20. To prove the lemma formally, we use inductionon the length of the expression. Like all induction proofs, this willbe somewhat lengthy, but at the end of the day it directly follows theintuition above that somewhere we must have used the star operation.Reading this proof, and in particular understanding how the formalproof below corresponds to the intuitive idea above, is a very goodway to get more comfortable with inductive proofs of this form.
Our inductive hypothesis is that for an ð length expression, ð0 =2ð satisfies the conditions of the lemma. The base case is when theexpression is a single symbol ð â Σ or that the expression is â or"". In all these cases the conditions of the lemma are satisfied simplybecause there ð0 = 2 and there is no string ð¥ of length larger than ð0that is matched by the expression.
We now prove the inductive step. Let ð be a regular expressionwith ð > 1 symbols. We set ð0 = 2ð and let ð€ â Σâ be a stringsatisfying |ð€| > ð0. Since ð has more than one symbol, it has one ofthe forms (a) ðâ²|ðâ³, (b), (ðâ²)(ðâ³), or (c) (ðâ²)â where in all these casesthe subexpressions ðâ² and ðâ³ have fewer symbols than ð and hencesatisfy the induction hypothesis.
In the case (a), every string ð€ matched by ð must be matched byeither ðâ² or ðâ³. If ðâ² matches ð€ then, since |ð€| > 2|ðâ²|, by the inductionhypothesis there exist ð¥, ðŠ, ð§ with |ðŠ| ⥠1 and |ð¥ðŠ| †2|ðâ²| < ð0 suchthat ðâ² (and therefore also ð = ðâ²|ðâ³) matches ð¥ðŠðð§ for every ð. Thesame arguments works in the case that ðâ³ matches ð€.
In the case (b), if ð€ is matched by (ðâ²)(ðâ³) then we can write ð€ =ð€â²ð€â³ where ðâ² matches ð€â² and ðâ³ matches ð€â³. We split to subcases. If|ð€â²| > 2|ðâ²| then by the induction hypothesis there exist ð¥, ðŠ, ð§â² with|ðŠ| †1, |ð¥ðŠ| †2|ðâ²| < ð0 such that ð€â² = ð¥ðŠð§â² and ðâ² matches ð¥ðŠðð§â²for every ð â â. This completes the proof since if we set ð§ = ð§â²ð€â³then we see that ð€ = ð€â²ð€â³ = ð¥ðŠð§ and ð = (ðâ²)(ðâ³) matches ð¥ðŠðð§ forevery ð â â. Otherwise, if |ð€â²| †2|ðâ²| then since |ð€| = |ð€â²| + |ð€â³| >ð0 = 2(|ðâ²| + |ðâ³|), it must be that |ð€â³| > 2|ðâ³|. Hence by the inductionhypothesis there exist ð¥â², ðŠ, ð§ such that |ðŠ| ⥠1, |ð¥â²ðŠ| †2|ðâ³| and ðâ³matches ð¥â²ðŠðð§ for every ð â â. But now if we set ð¥ = ð€â²ð¥â² we see that|ð¥ðŠ| †|ð€â²| + |ð¥â²ðŠ| †2|ðâ²| + 2|ðâ³| = ð0 and on the other hand theexpression ð = (ðâ²)(ðâ³) matches ð¥ðŠðð§ = ð€â²ð¥â²ðŠðð§ for every ð â â.
In case (c), if ð€ is matched by (ðâ²)â then ð€ = ð€0 ⯠ð€ð¡ where forevery ð â [ð¡], ð€ð is a nonempty string matched by ðâ². If |ð€0| > 2|ðâ²|,then we can use the same approach as in the concatenation case above.Otherwise, we simply note that if ð¥ is the empty string, ðŠ = ð€0, andð§ = ð€1 ⯠ð€ð¡ then |ð¥ðŠ| †ð0 and ð¥ðŠðð§ is matched by (ðâ²)â for everyð â â.
â
-
244 introduction to theoretical computer science
RRemark 6.21 â Recursive definitions and inductiveproofs. When an object is recursively defined (as in thecase of regular expressions) then it is natural to proveproperties of such objects by induction. That is, if wewant to prove that all objects of this type have prop-erty ð , then it is natural to use an inductive step thatsays that if ðâ², ðâ³, ð⎠etc have property ð then so is anobject ð that is obtained by composing them.
Using the pumping lemma, we can easily prove Lemma 6.19 (i.e.,the non-regularity of the âmatching parenthesisâ function):
Proof of Lemma 6.19. Suppose, towards the sake of contradiction, thatthere is an expression ð such that Ίð = MATCHPAREN. Let ð0 bethe number obtained from Theorem 6.20 and let ð€ = âšð0â©ð0 (i.e.,ð0 left parenthesis followed by ð0 right parenthesis). Then we seethat if we write ð€ = ð¥ðŠð§ as in Lemma 6.19, the condition |ð¥ðŠ| †ð0implies that ðŠ consists solely of left parenthesis. Hence the stringð¥ðŠ2ð§ will contain more left parenthesis than right parenthesis. HenceMATCHPAREN(ð¥ðŠ2ð§) = 0 but by the pumping lemma Ίð(ð¥ðŠ2ð§) = 1,contradicting our assumption that Ίð = MATCHPAREN.
â
The pumping lemma is a very useful tool to show that certain func-tions are not computable by a regular expression. However, it is not anâif and only ifâ condition for regularity: there are non-regular func-tions that still satisfy the pumping lemma conditions. To understandthe pumping lemma, it is crucial to follow the order of quantifiers inTheorem 6.20. In particular, the number ð0 in the statement of Theo-rem 6.20 depends on the regular expression (in the proof we chose ð0to be twice the number of symbols in the expression). So, if we wantto use the pumping lemma to rule out the existence of a regular ex-pression ð computing some function ð¹ , we need to be able to choosean appropriate input ð€ â {0, 1}â that can be arbitrarily large andsatisfies ð¹(ð€) = 1. This makes sense if you think about the intuitionbehind the pumping lemma: we need ð€ to be large enough as to forcethe use of the star operator.
Solved Exercise 6.4 â Palindromes is not regular. Prove that the followingfunction over the alphabet {0, 1, ; } is not regular: PAL(ð€) = 1 if andonly if ð€ = ð¢; ð¢ð where ð¢ â {0, 1}â and ð¢ð denotes ð¢ âreversedâ:the string ð¢|ð¢|â1 ⯠ð¢0. (The Palindrome function is most often definedwithout an explicit separator character ;, but the version with such aseparator is a bit cleaner, and so we use it here. This does not make
-
functions with infinite domains, automata, and regular expressions 245
Figure 6.10: A cartoon of a proof using the pumping lemma that a function ð¹ is not regular. The pumping lemma states that if ð¹ is regular then thereexists a number ð0 such that for every large enough ð€ with ð¹(ð€) = 1, there exists a partition of ð€ to ð€ = ð¥ðŠð§ satisfying certain conditions suchthat for every ð â â, ð¹(ð¥ðŠðð§) = 1. You can imagine a pumping-lemma based proof as a game between you and the adversary. Every there existsquantifier corresponds to an object you are free to choose on your own (and base your choice on previously chosen objects). Every for every quantifiercorresponds to an object the adversary can choose arbitrarily (and again based on prior choices) as long as it satisfies the conditions. A valid proofcorresponds to a strategy by which no matter what the adversary does, you can win the game by obtaining a contradiction which would be a choiceof ð that would result in ð¹(ð¥ðŠðð§) = 0, hence violating the conclusion of the pumping lemma.
-
246 introduction to theoretical computer science
much difference, as one can easily encode the separator as a specialbinary string instead.)
â
Solution:
We use the pumping lemma. Suppose toward the sake of con-tradiction that there is a regular expression ð computing PAL,and let ð0 be the number obtained by the pumping lemma (The-orem 6.20). Consider the string ð€ = 0ð0 ; 0ð0 . Since the reverseof the all zero string is the all zero string, PAL(ð€) = 1. Now, bythe pumping lemma, if PAL is computed by ð, then we can writeð€ = ð¥ðŠð§ such that |ð¥ðŠ| †ð0, |ðŠ| ⥠1 and PAL(ð¥ðŠðð§) = 1 forevery ð â â. In particular, it must hold that PAL(ð¥ð§) = 1, but thisis a contradiction, since ð¥ð§ = 0ð0â|ðŠ|; 0ð0 and so its two parts arenot of the same length and in particular are not the reverse of oneanother.
â
For yet another example of a pumping-lemma based proof, seeFig. 6.10 which illustrates a cartoon of the proof of the non-regularityof the function ð¹ ⶠ{0, 1}â â {0, 1} which is defined as ð¹(ð¥) = 1 iffð¥ = 0ð1ð for some ð â â (i.e., ð¥ consists of a string of consecutivezeroes, followed by a string of consecutive ones of the same length).
6.6 ANSWERING SEMANTIC QUESTIONS ABOUT REGULAR EX-PRESSIONS
Regular expressions have applications beyond search. For example,regular expressions are often used to define tokens (such as what is avalid variable identifier, or keyword) in the design of parsers, compilersand interpreters for programming languages. Regular expressionshave other applications too: for example, in recent years, the worldof networking moved from fixed topologies to âsoftware definednetworksâ. Such networks are routed by programmable switchesthat can implement policies such as âif packet is secured by SSL thenforward it to A, otherwise forward it to Bâ. To represent such policieswe need a language that is on one hand sufficiently expressive tocapture the policies we want to implement, but on the other handsufficiently restrictive so that we can quickly execute them at networkspeed and also be able to answer questions such as âcan C see thepackets moved from A to B?â. The NetKAT network programminglanguage uses a variant of regular expressions to achieve preciselythat. For this application, it is important that we are not merely ableto answer whether an expression ð matches a string ð¥ but also answersemantic questions about regular expressions such as âdo expressions
https://goo.gl/oeJNuwhttps://goo.gl/oeJNuw
-
functions with infinite domains, automata, and regular expressions 247
ð and ðâ² compute the same function?â and âdoes there exist a string ð¥that is matched by the expression ð?â. The following theorem showsthat we can answer the latter question:
Theorem 6.22 â Emptiness of regular languages is computable. There is analgorithm that given a regular expression ð, outputs 1 if and only ifΊð is the constant zero function.
Proof Idea:
The idea is that we can directly observe this from the structureof the expression. The only way a regular expression ð computesthe constant zero function is if ð has the form â or is obtained byconcatenating â with other expressions.
â
Proof of Theorem 6.22. Define a regular expression to be âemptyâ if itcomputes the constant zero function. Given a regular expression ð, wecan determine if ð is empty using the following rules:
⢠If ð has the form ð or "" then it is not empty.
⢠If ð is not empty then ð|ðâ² is not empty for every ðâ².
⢠If ð is not empty then ðâ is not empty.
⢠If ð and ðâ² are both not empty then ð ðâ² is not empty.
⢠â is empty.
Using these rules, it is straightforward to come up with a recursivealgorithm to determine emptiness.
â
Using Theorem 6.22, we can obtain an algorithm that determineswhether or not two regular expressions ð and ðâ² are equivalent, in thesense that they compute the same function.
Theorem 6.23 â Equivalence of regular expressions is computable. LetREGEQ ⶠ{0, 1}â â {0, 1} be the function that on input (a stringrepresenting) a pair of regular expressions ð, ðâ², REGEQ(ð, ðâ²) = 1if and only if Ίð = Ίðâ² . Then there is an algorithm that computesREGEQ.
Proof Idea:
The idea is to show that given a pair of regular expressions ð andðâ² we can find an expression ðâ³ such that Ίðâ³(ð¥) = 1 if and only ifΊð(ð¥) â Ίðâ²(ð¥). Therefore Ίðâ³ is the constant zero function if and only
-
248 introduction to theoretical computer science
if ð and ðâ² are equivalent, and thus we can test for emptiness of ðâ³ todetermine equivalence of ð and ðâ².
â
Proof of Theorem 6.23. We will prove Theorem 6.23 from Theorem 6.22.(The two theorems are in fact equivalent: it is easy to prove Theo-rem 6.22 from Theorem 6.23, since checking for emptiness is the sameas checking equivalence with the expression â .) Given two regu-lar expressions ð and ðâ², we will compute an expression ðâ³ such thatΊðâ³(ð¥) = 1 if and only if Ίð(ð¥) â Ίðâ²(ð¥). One can see that ð is equiva-lent to ðâ² if and only if ðâ³ is empty.
We start with the observation that for every bit ð, ð â {0, 1}, ð â ð ifand only if
(ð ⧠ð) âš (ð ⧠ð) . (6.15)Hence we need to construct ðâ³ such that for every ð¥,
Ίðâ³(ð¥) = (Ίð(ð¥) ⧠Ίðâ²(ð¥)) âš (Ίð(ð¥) ⧠Ίðâ²(ð¥)) . (6.16)To construct the expression ðâ³, we will show how given any pair of
expressions ð and ðâ², we can construct expressions ð ⧠ðâ² and ð thatcompute the functions Ίð ⧠Ίðâ² and Ίð respectively. (Computing theexpression for ð âš ðâ² is straightforward using the | operation of regularexpressions.)
Specifically, by Lemma 6.17, regular functions are closed undernegation, which means that for every regular expression ð, there is anexpression ð such that Ίð(ð¥) = 1 â Ίð(ð¥) for every ð¥ â {0, 1}â. Now,for every two expressions ð and ðâ², the expression
ð ⧠ðâ² = (ð|ðâ²) (6.17)
computes the AND of the two expressions. Given these two transfor-mations, we see that for every regular expressions ð and ðâ² we can finda regular expression ðâ³ satisfying (6.16) such that ðâ³ is empty if andonly if ð and ðâ² are equivalent.
â
â Chapter Recap
⢠We model computational tasks on arbitrarily largeinputs using infinite functions ð¹ ⶠ{0, 1}â â {0, 1}â.
⢠Such functions take an arbitrarily long (but stillfinite!) string as input, and cannot be described bya finite table of inputs and outputs.
⢠A function with a single bit of output is known asa Boolean function, and the task of computing it isequivalent to deciding a language ð¿ â {0, 1}â.
-
functions with infinite domains, automata, and regular expressions 249
⢠Deterministic finite automata (DFAs) are one simplemodel for computing (infinite) Boolean functions.
⢠There are some functions that cannot be computedby DFAs.
⢠The set of functions computable by DFAs is thesame as the set of languages that can be recognizedby regular expressions.
6.7 EXERCISES
Exercise 6.1 â Closure properties of regular functions. Suppose that ð¹, ðº â¶{0, 1}â â {0, 1} are regular. For each one of the following defini-tions of the function ð» , either prove that ð» is always regular or give acounterexample for regular ð¹, ðº that would make ð» not regular.
1. ð»(ð¥) = ð¹(ð¥) âš ðº(ð¥).
2. ð»(ð¥) = ð¹(ð¥) ⧠ðº(ð¥)
3. ð»(ð¥) = NAND(ð¹(ð¥), ðº(ð¥)).
4. ð»(ð¥) = ð¹(ð¥ð ) where ð¥ð is the reverse of ð¥: ð¥ð = ð¥ðâ1ð¥ðâ2 ⯠ð¥ð forð = |ð¥|.
5. ð»(ð¥) =â§{âš{â©
1 ð¥ = ð¢ð£ s.t. ð¹(ð¢) = ðº(ð£) = 10 otherwise
6. ð»(ð¥) =â§{âš{â©
1 ð¥ = ð¢ð¢ s.t. ð¹(ð¢) = ðº(ð¢) = 10 otherwise
7. ð»(ð¥) =â§{âš{â©
1 ð¥ = ð¢ð¢ð s.t. ð¹(ð¢) = ðº(ð¢) = 10 otherwise
â
Exercise 6.2 One among the following two functions that map {0, 1}âto {0, 1} can be computed by a regular expression, and the other onecannot. For the one that can be computed by a regular expression,write the expression that does it. For the one that cannot, prove thatthis cannot be done using the pumping lemma.
⢠ð¹(ð¥) = 1 if 4 divides â|ð¥|â1ð=0 ð¥ð and ð¹(ð¥) = 0 otherwise.
⢠ðº(ð¥) = 1 if and only if â|ð¥|â1ð=0 ð¥ð ⥠|ð¥|/4 and ðº(ð¥) = 0 otherwise.
â
Exercise 6.3 â Non-regularity. 1. Prove that the following function ð¹ â¶{0, 1}â â {0, 1} is not regular. For every ð¥ â {0, 1}â, ð¹(ð¥) = 1 iff ð¥ isof the form ð¥ = 13ð for some ð > 0.
-
250 introduction to theoretical computer science
2. Prove that the following function ð¹ ⶠ{0, 1}â â {0, 1} is not regular.For every ð¥ â {0, 1}â, ð¹(ð¥) = 1 iff âð ð¥ð = 3ð for some ð > 0.
â
6.8 BIBLIOGRAPHICAL NOTES
The relation of regular expressions with finite automata is a beautifultopic, on which we only touch upon in this text. It is covered moreextensively in [Sip97; HMU14; Koz97]. These texts also discuss top-ics such as non-deterministic finite automata (NFA) and the relationbetween context-free grammars and pushdown automata.
The automaton of Fig. 6.4 was generated using the FSM simulatorof Ivan Zuzak and Vedrana Jankovic. Our proof of Theorem 6.11 isclosely related to the Myhill-Nerode Theorem. One direction of theMyhill-Nerode theorem can be stated as saying that if ð is a regularexpression then there is at most a finite number of strings ð§0, ⊠, ð§ðâ1such that Ίð[ð§ð] â Ίð[ð§ð] for every 0 †ð â ð < ð.
http://ivanzuzak.info/noam/webapps/fsm_simulator/https://goo.gl/mnKVMP