Download - theory of computation lecture 01
![Page 1: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/1.jpg)
Theory of Computation
Mahmoud Ali Ahmed (PhD) Dean
Faculty of Mathematical Sciences
U. of K.
email: [email protected]
![Page 2: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/2.jpg)
What is Computation?Computation is a general term for any type of information
processing that can be represented as an algorithm
precisely (mathematically).
Examples:• Adding two numbers in our brains, on a piece of paper or using a calculator.
• Converting a decimal number to its binary presentation or vise versa.• Finding the greatest common divisors of two numbers.
• …
![Page 3: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/3.jpg)
What is Theory of Computation
• A very fundamental and traditional branch
of Theory of Computation seeks:
1. A more tangible definition for the intuitive
notion of algorithm which results in a more
concrete definition for computation.
2. Finding the boundaries (limitations) of
computation.
![Page 4: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/4.jpg)
"Theoretical Computer Science (TCS) studies the
inherent powers and limitations of computation,
that is broadly defined to include both current and
future, man-made and naturally arising
computing phenomena."
Why Computation
![Page 5: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/5.jpg)
What can be computed with very limited memory?
Such memory restricted devices are all around us
Many applications within CS ranging from compiler construction
and text search to computer challenges in video games
![Page 6: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/6.jpg)
Computability
What can be computed at all?
and any future “computers”
There are fundamental problems that cannot be solved.
![Page 7: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/7.jpg)
ComplexityWhat can be computed efficiently?
Understanding the fundamental limits of computation
including the limits of and any future “computers”
Some problems seems to require more resources to solve than others,
for example coming up with a correct proof for a mathematical
statement seems to take more time than verifying the correctness of
a proof
![Page 8: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/8.jpg)
Theory of Computation tries to answer the
following questions :
o What are the fundamental capabilities and
limitations of computers?
o Which problems can be solved by computers
and which ones cannot?
o What makes some problems computationally
hard and others easy?
![Page 9: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/9.jpg)
Based on this discussion, the course provides an
introduction to the theory of computation.
Traditionally, the study of theory of computation
comprises three central areas:
o Automata.
o Computability, and
o Complexity.
![Page 10: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/10.jpg)
Objective of this CourseThe aim of this course is to introduce several apparently different
formalizations of the informal notion of algorithm; to show that
they are equivalent; and to use them to demonstrate that there are
uncomputable functions and algorithmically undecidable
problems, and enables the student to be aware about:
• Automata theory (Dealing with solving simple decision
problems (solvable and unsolvable problems)),
• Computability (Computable problems: Turing machine), and
• Complexity (dealing with solvable problems: Time Complexity
(P and NP, and NP-complete) and Space Complexity
(PSPACE, NL)).
![Page 11: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/11.jpg)
Course Outlines and Planning:
Historical Perspective
Review: Sets, Mathematical Notations, Graphs and Algorithms.
Machines: Register Machine, Automat, Turing Machine
Complexity Classes.
Course Outlines
![Page 12: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/12.jpg)
Course Planning:Week No. Topics
1 Orientations, Review and Introduction
2,3,4,5,6,
7,8,9
Machines Models: Register Machine and
Automata theory:- (solvability and decidability)
10,11,12,
13
Computability Model :Turing machine
14 Complexity theory: Complexity Classes
Final Exam
![Page 13: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/13.jpg)
Student Evaluation
The student will be evaluated (Assessed) based on the
following:
Course work
Final exam
30%
70%
![Page 14: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/14.jpg)
References- Michael Sipser, Introduction to the Theory of Computation, PWS
Publishing Company, 1997.
- Mikhail J. Atallah and Mariana Blanton (Eds.), Algorithms and
Theory of Computation Handbook: General Concepts and Techniques,
CRC Press, New York, 2009 (2nd Edition).
Thomas Cormen, Charles Leiserson, Ronald Rivest, and Cliff Stein,
Introduction to Algorithms, McGraw Hill Publishing Company and
MIT Press, 2009 (3rd Edition).
Peter Linz, An Introduction to Formal Languages and Automata,
Jones and Bartlett Publishers, 2006.
- Other open sources (Internet)
![Page 15: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/15.jpg)
Historical PerspectivesEuclid (350 B.C.) : ElementsLeonhard Euler (1707-1783) : Graph theory
Alan Turing (1912-1954) : Computability
Alonzo Church (1903-1995) : Lambda-calculusJohn von Neumann (1903-1957) : Stored program
Claude Shannon (1916-2001) : Information theory
Noam Chomsky (1928-) : Formal languages
John Backus (1924-) : Functional programming
Edsger Dijkstra (1930-2002) : Structured programming
E N D
![Page 16: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/16.jpg)
Review: Sets and Mathematical Notations
DefinitionsSET: It is an unordered collection of elements.
A= {1, 2, 3}
C={x|x is positive integer greater than or equal to x2}
B ={hi, there}Example:
D = {3, 3.1, 3.14, 3.141, 3.1415, 3.14159, …}
A, B and C are finite sets while D is an infinite.
![Page 17: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/17.jpg)
Set construction| or means “such that”
Example:E = {k | 0<k<4}F = {k | k is a perfect square}
Set membership
means “belong to” or “ not belong to”,7 {p | p prime}q {0, 2, 4, 6,...}
Example:
![Page 18: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/18.jpg)
Sets can contain other sets
Example:G = {2, {5}}
H = {{{0}}} {0} 0 S = {1, 2, 3, {1}, {{2}}}
Common SetsNaturals: N = {1, 2, 3, 4, ...}
Integers: Z = {..,-2, -1, 0, 1, 2,..}Rationals: Q = {a/b| a,b Z, b 0}Reals: = {x | x a real #}
![Page 19: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/19.jpg)
Empty set: Ø = {}z+ = non-negative integers
= non-positive real
MultisetsE N D
a set with repeated elements allowed, (i.e., each element has “multiplier”).
Example H = {0, 1, 2, 2, 2, 5, 5}
![Page 20: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/20.jpg)
sequencesDefinition: it an ordered list of elements
(0, 1, 2, 5) “4-tuple”Example:
(1,2) “2-tuple”
Universal & Existential Quantification
“for all”12 xxxExample:
![Page 21: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/21.jpg)
“there exists”x| 2 xxExample:
Combinations:
yxyx
Boolean Operations
E N D
"" AND"" OR
"" NOT
"" XOR
NAND
NOR
![Page 22: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/22.jpg)
Logical Implication"" implies
331 yxyx Example:
"" if and only if (iff)
"" equivalent
)]()[()( ABBABA or
ABBABA
![Page 23: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/23.jpg)
Example:)(),max(),min( yxyxyx
))()( PQQP
![Page 24: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/24.jpg)
Subsets"" Subset notation
)( TxSxTS "" Proper Subset
))()(( TSTSTS ))()(( TSSTTS
SS SSS
![Page 25: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/25.jpg)
Union
Example:
Example:
TxSxxTS Intersection
TxSxxTS
![Page 26: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/26.jpg)
Set difference: S - T
Example: TxSxxTS
Example:
TSTS
TxSxxTS
Symmetric difference: S T
![Page 27: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/27.jpg)
Universal set: U (everything)
Set Complement SorS
Disjoint sets:
Example
SUSxxS Example:
TSTSTS
SSSS
![Page 28: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/28.jpg)
Set Identities• Commutative Law:Example
STTS STTS
• Associative Law:
VTSVTS )()(VTSVTS )()(
Example
![Page 29: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/29.jpg)
Example:
• Associative Law:
)()()( VSTSVTS )()()( VSTSVTS
Example:• Absorption Law:
STSS )(STSS )(
![Page 30: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/30.jpg)
E N D
• DeMorgan's Laws:Example
TSTS )(TSTS )(
![Page 31: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/31.jpg)
Proof Types• Construction• Contradiction
• Induction
• Counter-example
![Page 32: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/32.jpg)
GraphsDefinition: Used to represent and model a problem with special kind of relationship such as:
• Common relationships• Communication networks
• Dependency constraints
• Reachability information• + many more practical applications!
![Page 33: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/33.jpg)
Definition:Graph : is the combination of a set of vertices , and a set of edges VVE V
),( EVG
Pictorially: nodes & lines
Undirected GraphsDefinition: it the graph whose edges have no direction.
![Page 34: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/34.jpg)
a
b
c e
d
V={a, b, c, d, e}
E={(c, a), (c, b), (c, d), (c, e), (a, b), (b, d), (d, e)}
Example
![Page 35: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/35.jpg)
Directed GraphsDefinition: it the graph whose edges have direction.
Example
a
b
c e
d
V={a, b, c, d, e}
E={(a, b), (a, c), (b, c), (b, d), (d, c), (d, e), (c, e)}
![Page 36: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/36.jpg)
Graph TerminologyGraph VVE ),,( EVG node vertexedge arc
a
b
c e
d
f
For any pair of vertices (u, v) V are said to be neighbors in G (u, v) or (v, u) is an edge of G.
a & b are neighbors. a & e are not neighbors.
![Page 37: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/37.jpg)
Undirected Node Degree
Degree in undirected graphs:Degree of any undirected node (vertex) of the graphs is denoted by deg(v):
deg(v) = # of adjacent (incident) edges to vertex v in G
Example
deg(c)=4 deg(f)=0
![Page 38: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/38.jpg)
directed Node Degree
Degree in directed graphs:
For any directed node (vertex) of the graph G there are two types of degree.
In-deg(v): = # of incoming edges
Out-deg(v): = # of outgoing edges
![Page 39: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/39.jpg)
Example
a
b
c e
d
f
in-deg(c)=3 out-deg(c)=1
in-deg(f)=0 out-deg(f)=0
![Page 40: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/40.jpg)
Complete Graph
Graph : is said to be complete if it contains all edges, i. e., vuVVvuE ,
),( EVKn
a
b
c e
d
f
![Page 41: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/41.jpg)
Transition GraphDefinition: a transition graph is the graph that can be defined by vertices representing states (s) and directed lines from each state (sk) to itself (loop) or to another state (sj)
S1 S2
![Page 42: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/42.jpg)
Graph Representation
a
b
c
d
Adjacency list:
1: (a) b d 2: (a) c d
![Page 43: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/43.jpg)
Adjacency matrix:
a b c da 0 1 1 0b 1 0 0 1c 1 0 0 0d 0 1 0 0
![Page 44: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/44.jpg)
AlgorithmAn algorithm is a set of unambiguous computational
procedures or steps that produce an output given an input.
Implementing an algorithm means transforming the set of
steps specified in the algorithm into a computer program.
Algorithms can be used to search, sort, and select data.
Data structures such as stacks, queues, arrays, and
trees can all have algorithms applied to them.
![Page 45: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/45.jpg)
• The study and analysis of algorithms is
important as algorithms play a large role in
solving real world problems particularly in
business and other sectors where computing
has had a large impact.
Why it is important to study & analyze an algorithms?
![Page 46: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/46.jpg)
• Different algorithms that accomplish the
same task can vary in their efficiency.
• This may not be important when used on a
small scale but when applied to large-scale
problems, efficiency becomes ever more
important.
![Page 47: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/47.jpg)
• Algorithm analysis involves measuring how
long an algorithm will take to reach a result and
terminate.
What is an algorithm analysis?
• This may be measured by comparison with
other algorithms or it can be measured by
determining of what "order" the algorithm is.
![Page 48: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/48.jpg)
Algorithm Complexity
Complexity is measured in the form of "Big-O"
notation. For example, "O(1)" means "order 1"
and describes programs where statements are
simply executed one after the other.
These programs/methods are described as "constant time" methods .
![Page 49: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/49.jpg)
Where loops are involved in the program,
this gives the program the order of "O(N)"
and these programs are known as having
"linear time".
Quadratic time methods involve loops being
nested within another loop and is given the
notation "O(N^2)".
![Page 50: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/50.jpg)
By doubling the input of programs with
methods of complexity O(N^2), the number
of steps needed to execute the code to
completion will be multiplied by four.
![Page 51: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/51.jpg)
Based on the complexity (the time needed to execute
the algorithm), algorithms can be classified into
several classes such as
• Linear Time Algorithms
• Exponential Time Algorithms
• Open Time (never complete) Algorithms
![Page 52: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/52.jpg)
Running Time Running time can be estimated in a more general
manner by using Pseudocode to represent the
algorithm as a set of fundamental operations
which can then be counted.
Pseudocode gives a high-level description of an
algorithm without the ambiguity associated with
plain text but also without the need to know the
syntax of a particular programming language.
![Page 53: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/53.jpg)
Pseudocode contains all of the following
features:
Conventional loops: If... then... else...
While... do... Repeat... until... For... do...
![Page 54: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/54.jpg)
Method declaration: Method name: Algorithm methodName(args) Input description: Input... Output description: Output...
Method calls: var.methodName(args) Return values: return... Expressions:
Assignment: < -- Equality comparison: ==
![Page 55: theory of computation lecture 01](https://reader036.vdocument.in/reader036/viewer/2022062401/58ec90af1a28abeb4f8b463d/html5/thumbnails/55.jpg)
In a pseudo code description of an algorithm, each line can be viewed as a certain number of operations which can be counted and added up to find a total value (order of algorithm) for the algorithm. Some examples are shown below: • varNumber <-- X[1] (2 operations - 1 array value retrieval, 1 assignment).
• For a <-- 1 to n do... (n operations - any operations within this loop will then be multiplied by n also as they will be carried out n times).• return varNumber (1 operations).