design and analysis of algorithms...
TRANSCRIPT
![Page 1: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/1.jpg)
Design and Analysis ofAlgorithms (I)
Introduction
Guoqiang Li
School of Software, Shanghai Jiao Tong University
![Page 2: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/2.jpg)
Instructor and Teaching Assistants
• Guoqiang LI
• Homepage: http://basics.sjtu.edu.cn/∼liguoqiang• Course page:
http://basics.sjtu.edu.cn/∼liguoqiang/teaching/X037506/• Canvas: https://oc.sjtu.edu.cn/courses/21120• Email: li.g (AT) outlook (DOT) com• Office: Rm. 1212, Building of Software• Phone: 3420-4167
• TA:
• Qizhe YANG: yangqizhe1994 (AT) hotmail (DOT) com• Yanfei YANG: danielyang1996 (AT) 163 (DOT) com
• Office hour: Wed. 14:00-17:00 @ SEIEE 3-327 & SoftwareBuilding 34XX
![Page 3: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/3.jpg)
Instructor and Teaching Assistants
• Guoqiang LI• Homepage: http://basics.sjtu.edu.cn/∼liguoqiang• Course page:
http://basics.sjtu.edu.cn/∼liguoqiang/teaching/X037506/• Canvas: https://oc.sjtu.edu.cn/courses/21120• Email: li.g (AT) outlook (DOT) com• Office: Rm. 1212, Building of Software• Phone: 3420-4167
• TA:
• Qizhe YANG: yangqizhe1994 (AT) hotmail (DOT) com• Yanfei YANG: danielyang1996 (AT) 163 (DOT) com
• Office hour: Wed. 14:00-17:00 @ SEIEE 3-327 & SoftwareBuilding 34XX
![Page 4: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/4.jpg)
Instructor and Teaching Assistants
• Guoqiang LI• Homepage: http://basics.sjtu.edu.cn/∼liguoqiang• Course page:
http://basics.sjtu.edu.cn/∼liguoqiang/teaching/X037506/• Canvas: https://oc.sjtu.edu.cn/courses/21120• Email: li.g (AT) outlook (DOT) com• Office: Rm. 1212, Building of Software• Phone: 3420-4167
• TA:• Qizhe YANG: yangqizhe1994 (AT) hotmail (DOT) com• Yanfei YANG: danielyang1996 (AT) 163 (DOT) com
• Office hour: Wed. 14:00-17:00 @ SEIEE 3-327 & SoftwareBuilding 34XX
![Page 5: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/5.jpg)
Instructor and Teaching Assistants
• Guoqiang LI• Homepage: http://basics.sjtu.edu.cn/∼liguoqiang• Course page:
http://basics.sjtu.edu.cn/∼liguoqiang/teaching/X037506/• Canvas: https://oc.sjtu.edu.cn/courses/21120• Email: li.g (AT) outlook (DOT) com• Office: Rm. 1212, Building of Software• Phone: 3420-4167
• TA:• Qizhe YANG: yangqizhe1994 (AT) hotmail (DOT) com• Yanfei YANG: danielyang1996 (AT) 163 (DOT) com
• Office hour: Wed. 14:00-17:00 @ SEIEE 3-327 & SoftwareBuilding 34XX
![Page 6: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/6.jpg)
Notification
Students who take this lecture are assumed to have a solid backgroundof algorithms.
Theory of Algorithms
Algorithms
![Page 7: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/7.jpg)
Notification
Students who take this lecture are assumed to have a solid backgroundof algorithms.
Theory of Algorithms
Algorithms
![Page 8: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/8.jpg)
Notification
Students who take this lecture are assumed to have a solid backgroundof algorithms.
Theory of Algorithms
Algorithms
![Page 9: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/9.jpg)
Algorithm Design
Basic methodologies:• Algorithms on Lists, Trees and Graphs• Divide and Conquer
• Master Theorem
• Recursion
Advanced topics:• Dynamic Programming• Greedy Algorithm• Linear Programming• Approximation Algorithm• Randomized Algorithm• Massive Dataset Algorithms• …
![Page 10: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/10.jpg)
Algorithm DesignBasic methodologies:• Algorithms on Lists, Trees and Graphs
• Divide and Conquer• Master Theorem
• Recursion
Advanced topics:• Dynamic Programming• Greedy Algorithm• Linear Programming• Approximation Algorithm• Randomized Algorithm• Massive Dataset Algorithms• …
![Page 11: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/11.jpg)
Algorithm DesignBasic methodologies:• Algorithms on Lists, Trees and Graphs• Divide and Conquer
• Master Theorem
• Recursion
Advanced topics:• Dynamic Programming• Greedy Algorithm• Linear Programming• Approximation Algorithm• Randomized Algorithm• Massive Dataset Algorithms• …
![Page 12: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/12.jpg)
Algorithm DesignBasic methodologies:• Algorithms on Lists, Trees and Graphs• Divide and Conquer
• Master Theorem
• Recursion
Advanced topics:• Dynamic Programming• Greedy Algorithm• Linear Programming• Approximation Algorithm• Randomized Algorithm• Massive Dataset Algorithms• …
![Page 13: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/13.jpg)
Algorithm DesignBasic methodologies:• Algorithms on Lists, Trees and Graphs• Divide and Conquer
• Master Theorem
• Recursion
Advanced topics:• Dynamic Programming• Greedy Algorithm• Linear Programming• Approximation Algorithm• Randomized Algorithm• Massive Dataset Algorithms• …
![Page 14: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/14.jpg)
Algorithm Analysis
• Big-O Notation• Advanced Methodology:
• Probability Analysis• Amortized Analysis• Competition Analysis
![Page 15: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/15.jpg)
Algorithm Analysis
• Big-O Notation
• Advanced Methodology:
• Probability Analysis• Amortized Analysis• Competition Analysis
![Page 16: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/16.jpg)
Algorithm Analysis
• Big-O Notation• Advanced Methodology:
• Probability Analysis• Amortized Analysis• Competition Analysis
![Page 17: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/17.jpg)
Algorithm Analysis
• Big-O Notation• Advanced Methodology:
• Probability Analysis
• Amortized Analysis• Competition Analysis
![Page 18: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/18.jpg)
Algorithm Analysis
• Big-O Notation• Advanced Methodology:
• Probability Analysis• Amortized Analysis
• Competition Analysis
![Page 19: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/19.jpg)
Algorithm Analysis
• Big-O Notation• Advanced Methodology:
• Probability Analysis• Amortized Analysis• Competition Analysis
![Page 20: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/20.jpg)
Standard Algorithms
• Sorting• Searching & Hashing• Strongly connected components• Finding shortest paths in graphs• Minimum spanning trees in graphs• Matchings in bipartite graphs• Maximum flows in networks
![Page 21: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/21.jpg)
Standard Algorithms
• Sorting
• Searching & Hashing• Strongly connected components• Finding shortest paths in graphs• Minimum spanning trees in graphs• Matchings in bipartite graphs• Maximum flows in networks
![Page 22: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/22.jpg)
Standard Algorithms
• Sorting• Searching & Hashing
• Strongly connected components• Finding shortest paths in graphs• Minimum spanning trees in graphs• Matchings in bipartite graphs• Maximum flows in networks
![Page 23: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/23.jpg)
Standard Algorithms
• Sorting• Searching & Hashing• Strongly connected components
• Finding shortest paths in graphs• Minimum spanning trees in graphs• Matchings in bipartite graphs• Maximum flows in networks
![Page 24: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/24.jpg)
Standard Algorithms
• Sorting• Searching & Hashing• Strongly connected components• Finding shortest paths in graphs
• Minimum spanning trees in graphs• Matchings in bipartite graphs• Maximum flows in networks
![Page 25: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/25.jpg)
Standard Algorithms
• Sorting• Searching & Hashing• Strongly connected components• Finding shortest paths in graphs• Minimum spanning trees in graphs
• Matchings in bipartite graphs• Maximum flows in networks
![Page 26: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/26.jpg)
Standard Algorithms
• Sorting• Searching & Hashing• Strongly connected components• Finding shortest paths in graphs• Minimum spanning trees in graphs• Matchings in bipartite graphs
• Maximum flows in networks
![Page 27: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/27.jpg)
Standard Algorithms
• Sorting• Searching & Hashing• Strongly connected components• Finding shortest paths in graphs• Minimum spanning trees in graphs• Matchings in bipartite graphs• Maximum flows in networks
![Page 28: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/28.jpg)
Data Structure
• Link lists• Trees, graphs• Kripke structure, automata• Priority queue• Disjoint set
![Page 29: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/29.jpg)
Data Structure
• Link lists• Trees, graphs
• Kripke structure, automata• Priority queue• Disjoint set
![Page 30: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/30.jpg)
Data Structure
• Link lists• Trees, graphs• Kripke structure, automata
• Priority queue• Disjoint set
![Page 31: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/31.jpg)
Data Structure
• Link lists• Trees, graphs• Kripke structure, automata• Priority queue
• Disjoint set
![Page 32: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/32.jpg)
Data Structure
• Link lists• Trees, graphs• Kripke structure, automata• Priority queue• Disjoint set
![Page 33: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/33.jpg)
Reference Book
• Algorithms• Sanjoy Dasgupta• San Diego Christos Papadimitriou• Umesh Vazirani• McGraw-Hill, 2007.
![Page 34: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/34.jpg)
Reference Book
• Algorithm Design• Jon Kleinberg• Éva Tardos• Addison-Wesley, 2005.
![Page 35: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/35.jpg)
Reference Book
• Introduction to Algorithms• Thomas H. Cormen• Charles E. Leiserson• Ronald L. Rivest• Clifford Stein• The MIT Press (3rd edition), 2009.
![Page 36: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/36.jpg)
Reference Book
• Approximation Algorithms• Vijay V. Vazirani• Springer-Verlag, 2004
![Page 37: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/37.jpg)
Reference Book
• The Design of Approximation Algorithms• David P. Williamson• David B. Shmoys• Cambridge University Press, 2011.
![Page 38: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/38.jpg)
Reference Book
• Randomized Algorithms• Rajeev Motwani,• Prabhakar Raghavan• Cambridge University Press, 1995.
![Page 39: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/39.jpg)
Scoring Policy
• 10% Attendance.• 20% Homework.• 70% Final exam.
![Page 40: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/40.jpg)
Scoring Policy
• 10% Attendance.• 20% Homework.
• Five assignments.
• Each one is 4pts.• Work out individually.• Each assignment will be evaluated by A, B, C, D, F (Excellent(4),
Good(4), Fair(3), Delay(2), Fail(0))
• 70% Final exam.
![Page 41: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/41.jpg)
Scoring Policy
• 10% Attendance.• 20% Homework.
• Five assignments.• Each one is 4pts.
• Work out individually.• Each assignment will be evaluated by A, B, C, D, F (Excellent(4),
Good(4), Fair(3), Delay(2), Fail(0))
• 70% Final exam.
![Page 42: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/42.jpg)
Scoring Policy
• 10% Attendance.• 20% Homework.
• Five assignments.• Each one is 4pts.• Work out individually.
• Each assignment will be evaluated by A, B, C, D, F (Excellent(4),Good(4), Fair(3), Delay(2), Fail(0))
• 70% Final exam.
![Page 43: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/43.jpg)
Scoring Policy
• 10% Attendance.• 20% Homework.
• Five assignments.• Each one is 4pts.• Work out individually.• Each assignment will be evaluated by A, B, C, D, F (Excellent(4),
Good(4), Fair(3), Delay(2), Fail(0))
• 70% Final exam.
![Page 44: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/44.jpg)
Any Questions?
![Page 45: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/45.jpg)
Typical Algorithms
![Page 46: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/46.jpg)
Sorting
![Page 47: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/47.jpg)
Sorting
Input: A sequence of n numbers 〈a1, a2, . . . , an〉.Output: A permutation (reordering) 〈a′
1, a′2, . . . , a′
n〉 of the input sequencesuch that a′
1 ≤ a′2 ≤ . . . ≤ a′
n
![Page 48: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/48.jpg)
Various Sorts
• Insert Sort• Bubble Sort
• Heap Sort• Quick Sort• Merge Sort
![Page 49: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/49.jpg)
Various Sorts
• Insert Sort
• Bubble Sort
• Heap Sort• Quick Sort• Merge Sort
![Page 50: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/50.jpg)
Various Sorts
• Insert Sort• Bubble Sort
• Heap Sort• Quick Sort• Merge Sort
![Page 51: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/51.jpg)
Various Sorts
• Insert Sort• Bubble Sort
• Heap Sort
• Quick Sort• Merge Sort
![Page 52: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/52.jpg)
Various Sorts
• Insert Sort• Bubble Sort
• Heap Sort• Quick Sort
• Merge Sort
![Page 53: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/53.jpg)
Various Sorts
• Insert Sort• Bubble Sort
• Heap Sort• Quick Sort• Merge Sort
![Page 54: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/54.jpg)
Merge Sort
![Page 55: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/55.jpg)
The Algorithm
MERGESORT(a[1 . . . n])An array of numbers a[1 . . . n]
if n > 1 thenreturn(MERGE(MERGESORT(a[1 . . . bn/2c]),MERGESORT(a[bn/2c . . . , n]))) else return(a);
end
MERGE(x[1 . . . k], y[1 . . . l])
if k = 0 then return y[1 . . . l];if l = 0 then return x[1 . . . k];if x[1] ≤ y[1] then
return( x[1]MERGE(x[2 . . . k], y[1 . . . l]));else return( y[1]MERGE(x[1 . . . k], y[2 . . . l]));
end
![Page 56: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/56.jpg)
The Algorithm
MERGESORT(a[1 . . . n])An array of numbers a[1 . . . n]
if n > 1 thenreturn(MERGE(MERGESORT(a[1 . . . bn/2c]),MERGESORT(a[bn/2c . . . , n]))) else return(a);
end
MERGE(x[1 . . . k], y[1 . . . l])
if k = 0 then return y[1 . . . l];if l = 0 then return x[1 . . . k];if x[1] ≤ y[1] then
return( x[1]MERGE(x[2 . . . k], y[1 . . . l]));else return( y[1]MERGE(x[1 . . . k], y[2 . . . l]));
end
![Page 57: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/57.jpg)
An Iterative Version
ITERTIVE-MERGESORT(a[1 . . . n])An array of numbers a[1 . . . n];
Q = [ ]empty queue;for i = 1 to n do
Inject(Q, [a]);endwhile |Q| > 1 do
Inject (Q,MERGE (Eject (Q),Eject (Q)));endreturn(Eject (Q));
![Page 58: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/58.jpg)
The Time Analysis
• The recurrence relation:T(n) = 2T(n/2) + O(n)
• By Master Theorem:T(n) = O(n log n)
![Page 59: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/59.jpg)
The Time Analysis
• The recurrence relation:T(n) = 2T(n/2) + O(n)
• By Master Theorem:T(n) = O(n log n)
![Page 60: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/60.jpg)
The Time Analysis
• The recurrence relation:T(n) = 2T(n/2) + O(n)
• By Master Theorem:T(n) = O(n log n)
![Page 61: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/61.jpg)
Master Theorem
If T(n) = aT(dn/be) + O(nd) for some constants a > 0, b > 1 and d ≥ 0,then
T(n) =
O(nd) if d > logb aO(nd log n) if d = logb aO(nlogb a) if d < logb a
![Page 62: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/62.jpg)
Can we do better?
![Page 63: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/63.jpg)
Sorting
3 2 1
Yes
a2 < a3?
a1 < a2?
a1 < a3?
a2 < a3? a1 < a3?
2 3 1
2 1 3
3 1 2 1 3 2
1 2 3
No
• A sorting algorithm can be depicted as a decision tree.
• The depth of the tree - the number of comparisons on the longestpath from root to leaf, is the worst-case time complexity of thealgorithm.• Assume n elements. Each of its leaves is labeled by a
permutation of 1, 2, . . . , n.
![Page 64: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/64.jpg)
Sorting
3 2 1
Yes
a2 < a3?
a1 < a2?
a1 < a3?
a2 < a3? a1 < a3?
2 3 1
2 1 3
3 1 2 1 3 2
1 2 3
No
• A sorting algorithm can be depicted as a decision tree.• The depth of the tree - the number of comparisons on the longest
path from root to leaf, is the worst-case time complexity of thealgorithm.
• Assume n elements. Each of its leaves is labeled by apermutation of 1, 2, . . . , n.
![Page 65: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/65.jpg)
Sorting
3 2 1
Yes
a2 < a3?
a1 < a2?
a1 < a3?
a2 < a3? a1 < a3?
2 3 1
2 1 3
3 1 2 1 3 2
1 2 3
No
• A sorting algorithm can be depicted as a decision tree.• The depth of the tree - the number of comparisons on the longest
path from root to leaf, is the worst-case time complexity of thealgorithm.• Assume n elements. Each of its leaves is labeled by a
permutation of 1, 2, . . . , n.
![Page 66: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/66.jpg)
Sorting
3 2 1
Yes
a2 < a3?
a1 < a2?
a1 < a3?
a2 < a3? a1 < a3?
2 3 1
2 1 3
3 1 2 1 3 2
1 2 3
No
• Every permutation must appear as the label of a leaf.
• This is a binary tree with n! leaves.• So ,the depth of the tree - and the complexity of the algorithm -
must be at least
log(n!) ≈ log(√π(2n + 1/3) · nn · e−n) = Ω(n log n)
![Page 67: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/67.jpg)
Sorting
3 2 1
Yes
a2 < a3?
a1 < a2?
a1 < a3?
a2 < a3? a1 < a3?
2 3 1
2 1 3
3 1 2 1 3 2
1 2 3
No
• Every permutation must appear as the label of a leaf.• This is a binary tree with n! leaves.
• So ,the depth of the tree - and the complexity of the algorithm -must be at least
log(n!) ≈ log(√π(2n + 1/3) · nn · e−n) = Ω(n log n)
![Page 68: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/68.jpg)
Sorting
3 2 1
Yes
a2 < a3?
a1 < a2?
a1 < a3?
a2 < a3? a1 < a3?
2 3 1
2 1 3
3 1 2 1 3 2
1 2 3
No
• Every permutation must appear as the label of a leaf.• This is a binary tree with n! leaves.• So ,the depth of the tree - and the complexity of the algorithm -
must be at least
log(n!) ≈ log(√π(2n + 1/3) · nn · e−n) = Ω(n log n)
![Page 69: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/69.jpg)
Shortest Path in Graph
![Page 70: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/70.jpg)
Lengths on Edges
• BFS treats all edges as having the samelength.
• In reality the unit lengths always do notexist.• Every edge e ∈ E with a length le.• If e = (u, v), we will sometimes also
writel(u, v) or luv
FranciscoSan
LosAngeles
Bakersfield
Sacramento
Reno
LasVegas
409
290
95
271
133
445
291112
275
![Page 71: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/71.jpg)
Lengths on Edges
• BFS treats all edges as having the samelength.• In reality the unit lengths always do not
exist.
• Every edge e ∈ E with a length le.• If e = (u, v), we will sometimes also
writel(u, v) or luv
FranciscoSan
LosAngeles
Bakersfield
Sacramento
Reno
LasVegas
409
290
95
271
133
445
291112
275
![Page 72: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/72.jpg)
Lengths on Edges
• BFS treats all edges as having the samelength.• In reality the unit lengths always do not
exist.• Every edge e ∈ E with a length le.
• If e = (u, v), we will sometimes alsowrite
l(u, v) or luv
FranciscoSan
LosAngeles
Bakersfield
Sacramento
Reno
LasVegas
409
290
95
271
133
445
291112
275
![Page 73: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/73.jpg)
Lengths on Edges
• BFS treats all edges as having the samelength.• In reality the unit lengths always do not
exist.• Every edge e ∈ E with a length le.• If e = (u, v), we will sometimes also
writel(u, v) or luv
FranciscoSan
LosAngeles
Bakersfield
Sacramento
Reno
LasVegas
409
290
95
271
133
445
291112
275
![Page 74: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/74.jpg)
An Adaption of Breadth-First Search
BFS finds shortest paths in any graph whose edges have unit length.
Q: Can we adapt it to a more general graph G = (V ,E) whose edgelengths le are positive integers?
A simple trick: For any edge e = (u, v) of E, replace it by le edges oflength 1, by adding le − 1 dummy nodes between u and v. It mighttake time
O(|V |+∑e∈E
le)
It is bad in case we have edges with high length.
![Page 75: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/75.jpg)
An Adaption of Breadth-First Search
BFS finds shortest paths in any graph whose edges have unit length.
Q: Can we adapt it to a more general graph G = (V ,E) whose edgelengths le are positive integers?
A simple trick: For any edge e = (u, v) of E, replace it by le edges oflength 1, by adding le − 1 dummy nodes between u and v. It mighttake time
O(|V |+∑e∈E
le)
It is bad in case we have edges with high length.
![Page 76: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/76.jpg)
An Adaption of Breadth-First Search
BFS finds shortest paths in any graph whose edges have unit length.
Q: Can we adapt it to a more general graph G = (V ,E) whose edgelengths le are positive integers?
A simple trick: For any edge e = (u, v) of E, replace it by le edges oflength 1, by adding le − 1 dummy nodes between u and v. It mighttake time
O(|V |+∑e∈E
le)
It is bad in case we have edges with high length.
![Page 77: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/77.jpg)
An Adaption of Breadth-First Search
BFS finds shortest paths in any graph whose edges have unit length.
Q: Can we adapt it to a more general graph G = (V ,E) whose edgelengths le are positive integers?
A simple trick: For any edge e = (u, v) of E, replace it by le edges oflength 1, by adding le − 1 dummy nodes between u and v. It mighttake time
O(|V |+∑e∈E
le)
It is bad in case we have edges with high length.
![Page 78: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/78.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.• Repeat until there are no more alarms:• Say the next alarm goes off at time T , for node
u. Then:
• The distance from s to u is T .• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 79: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/79.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.
• Repeat until there are no more alarms:• Say the next alarm goes off at time T , for node
u. Then:
• The distance from s to u is T .• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 80: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/80.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.• Repeat until there are no more alarms:
• Say the next alarm goes off at time T , for nodeu. Then:
• The distance from s to u is T .• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 81: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/81.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.• Repeat until there are no more alarms:• Say the next alarm goes off at time T , for node
u. Then:
• The distance from s to u is T .• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 82: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/82.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.• Repeat until there are no more alarms:• Say the next alarm goes off at time T , for node
u. Then:• The distance from s to u is T .
• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 83: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/83.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.• Repeat until there are no more alarms:• Say the next alarm goes off at time T , for node
u. Then:• The distance from s to u is T .• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 84: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/84.jpg)
Alarm Clocks
• Set an alarm clock for node s at time 0.• Repeat until there are no more alarms:• Say the next alarm goes off at time T , for node
u. Then:• The distance from s to u is T .• For each neighbor v of u in G:
• If there is no alarm yet for v, set one for timeT + l(u, v).
• If v’s alarm is set for later than T + l(u, v),then reset it to this earlier time.
G: 2 0 01 0 0 5 0G′:
![Page 85: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/85.jpg)
An Example
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D:∞
B: 4 E:∞
C: 2
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
4
2
4
1
3
5
2
1 3A: 0 D: 6B: 3 E: 7C: 2
B
C
D
E
A
4
1 3
2
1
5
2
3
4 A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
2
1 3
2
![Page 86: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/86.jpg)
An Example
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D:∞
B: 4 E:∞
C: 2
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
4
2
4
1
3
5
2
1 3A: 0 D: 6B: 3 E: 7C: 2
B
C
D
E
A
4
1 3
2
1
5
2
3
4 A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
2
1 3
2
![Page 87: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/87.jpg)
An Example
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D:∞
B: 4 E:∞
C: 2
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
4
2
4
1
3
5
2
1 3A: 0 D: 6B: 3 E: 7C: 2
B
C
D
E
A
4
1 3
2
1
5
2
3
4 A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
2
1 3
2
![Page 88: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/88.jpg)
An Example
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D:∞
B: 4 E:∞
C: 2
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
4
2
4
1
3
5
2
1 3A: 0 D: 6B: 3 E: 7C: 2
B
C
D
E
A
4
1 3
2
1
5
2
3
4 A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
2
1 3
2
![Page 89: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/89.jpg)
An Example
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D:∞
B: 4 E:∞
C: 2
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
4
2
4
1
3
5
2
1 3A: 0 D: 6B: 3 E: 7C: 2
B
C
D
E
A
4
1 3
2
1
5
2
3
4 A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
2
1 3
2
![Page 90: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/90.jpg)
An Example
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D:∞
B: 4 E:∞
C: 2
B
C
D
E
A
4
1 3
2
4
1
3
5
2
A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
4
2
4
1
3
5
2
1 3A: 0 D: 6B: 3 E: 7C: 2
B
C
D
E
A
4
1 3
2
1
5
2
3
4 A: 0 D: 5B: 3 E: 6C: 2
B
C
D
E
A
2
1 3
2
![Page 91: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/91.jpg)
Dijkstra’s Shortest-Path Algorithm
DIJKSTRA(G, l, s)input : Graph G = (V ,E), directed or undirected; positive edge length
le | e ∈ E; Vertex s ∈ Voutput: For all vertices u reachable from s, dist(u) is the set to the distance from s to
u
for all u ∈ V dodist(u) =∞;prev(u) = nil;
enddist(s) = 0;H =makequeue(V)\\ using dist-values as keys;while H is not empty do
u=deletemin(H);for all edge (u, v) ∈ E do
if dist(v) > dist(u) + l(u, v) thendist(v) = dist(u) + l(u, v); prev(v) = u;decreasekey (H,v);
endend
end
![Page 92: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/92.jpg)
Priority Queue
Priority queue is a data structure usually implemented by heap.
• Insert: Add a new element to the set.• Decrease-key: Accommodate the decrease in key value of a
particular element.• Delete-min: Return the element with the smallest key, and
remove it from the set.• Make-queue: Build a priority queue out of the given elements,
with the given key values. (In many implementations, this issignificantly faster than inserting the elements one by one.)
The first two let us set alarms, and the third tells us which alarm isnext to go off.
![Page 93: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/93.jpg)
Priority Queue
Priority queue is a data structure usually implemented by heap.• Insert: Add a new element to the set.
• Decrease-key: Accommodate the decrease in key value of aparticular element.• Delete-min: Return the element with the smallest key, and
remove it from the set.• Make-queue: Build a priority queue out of the given elements,
with the given key values. (In many implementations, this issignificantly faster than inserting the elements one by one.)
The first two let us set alarms, and the third tells us which alarm isnext to go off.
![Page 94: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/94.jpg)
Priority Queue
Priority queue is a data structure usually implemented by heap.• Insert: Add a new element to the set.• Decrease-key: Accommodate the decrease in key value of a
particular element.
• Delete-min: Return the element with the smallest key, andremove it from the set.• Make-queue: Build a priority queue out of the given elements,
with the given key values. (In many implementations, this issignificantly faster than inserting the elements one by one.)
The first two let us set alarms, and the third tells us which alarm isnext to go off.
![Page 95: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/95.jpg)
Priority Queue
Priority queue is a data structure usually implemented by heap.• Insert: Add a new element to the set.• Decrease-key: Accommodate the decrease in key value of a
particular element.• Delete-min: Return the element with the smallest key, and
remove it from the set.
• Make-queue: Build a priority queue out of the given elements,with the given key values. (In many implementations, this issignificantly faster than inserting the elements one by one.)
The first two let us set alarms, and the third tells us which alarm isnext to go off.
![Page 96: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/96.jpg)
Priority Queue
Priority queue is a data structure usually implemented by heap.• Insert: Add a new element to the set.• Decrease-key: Accommodate the decrease in key value of a
particular element.• Delete-min: Return the element with the smallest key, and
remove it from the set.• Make-queue: Build a priority queue out of the given elements,
with the given key values. (In many implementations, this issignificantly faster than inserting the elements one by one.)
The first two let us set alarms, and the third tells us which alarm isnext to go off.
![Page 97: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/97.jpg)
Priority Queue
Priority queue is a data structure usually implemented by heap.• Insert: Add a new element to the set.• Decrease-key: Accommodate the decrease in key value of a
particular element.• Delete-min: Return the element with the smallest key, and
remove it from the set.• Make-queue: Build a priority queue out of the given elements,
with the given key values. (In many implementations, this issignificantly faster than inserting the elements one by one.)
The first two let us set alarms, and the third tells us which alarm isnext to go off.
![Page 98: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/98.jpg)
Running Time
Since makequeue takes at most as long as |V | insert operations, weget a total of |V | deletemin and |V |+ |E|insert/decreasekey operations.
![Page 99: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/99.jpg)
Which Heap is Best
Implementation deletemin insert/decreasekey|V |×deletemin+(|V | +|E|)× insert
Array O(|V |) O(1)O(|V |2)
Binary heap O(log |V |) O(log |V |)O((|V |+ |E|) log |V |)
d-ary heap O( d log |V|log d ) O( log |V|
log d )O( (d|V|+|E|) log |V|
log d )
Fibonacci heap O(log |V |) O(1) (amortized)O(|V | log |V |+ |E|)
![Page 100: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/100.jpg)
Which heap is Best
A naive array implementation gives a respectable time complexity ofO(|V |2), whereas with a binary heap we get O((|V |+ |E|) log |V |).Which is preferable?
This depends on whether the graph is sparse or dense.• |E| is less than |V |2. If it is Ω(|V |2), then clearly the array
implementation is the faster.• On the other hand, the binary heap becomes preferable as soon as|E| dips below |V |2/ log |V |.• The d-ary heap is a generalization of the binary heap and leads to
a running time that is a function of d. The optimal choice isd ≈ |E|/|V |;
![Page 101: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/101.jpg)
Which heap is Best
A naive array implementation gives a respectable time complexity ofO(|V |2), whereas with a binary heap we get O((|V |+ |E|) log |V |).Which is preferable?
This depends on whether the graph is sparse or dense.
• |E| is less than |V |2. If it is Ω(|V |2), then clearly the arrayimplementation is the faster.• On the other hand, the binary heap becomes preferable as soon as|E| dips below |V |2/ log |V |.• The d-ary heap is a generalization of the binary heap and leads to
a running time that is a function of d. The optimal choice isd ≈ |E|/|V |;
![Page 102: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/102.jpg)
Which heap is Best
A naive array implementation gives a respectable time complexity ofO(|V |2), whereas with a binary heap we get O((|V |+ |E|) log |V |).Which is preferable?
This depends on whether the graph is sparse or dense.• |E| is less than |V |2. If it is Ω(|V |2), then clearly the array
implementation is the faster.• On the other hand, the binary heap becomes preferable as soon as|E| dips below |V |2/ log |V |.• The d-ary heap is a generalization of the binary heap and leads to
a running time that is a function of d. The optimal choice isd ≈ |E|/|V |;
![Page 103: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/103.jpg)
Edit Distance
![Page 104: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/104.jpg)
The Problem
When a spell checker encounters a possible misspelling, it looks in itsdictionary for other words that are close by.
• Q: What is the appropriate notion of closeness in this case?
A natural measure of the distance between two strings is the extent towhich they can be aligned, or matched up.
Technically, an alignment is simply a way of writing the strings oneabove the other.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 105: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/105.jpg)
The Problem
When a spell checker encounters a possible misspelling, it looks in itsdictionary for other words that are close by.• Q: What is the appropriate notion of closeness in this case?
A natural measure of the distance between two strings is the extent towhich they can be aligned, or matched up.
Technically, an alignment is simply a way of writing the strings oneabove the other.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 106: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/106.jpg)
The Problem
When a spell checker encounters a possible misspelling, it looks in itsdictionary for other words that are close by.• Q: What is the appropriate notion of closeness in this case?
A natural measure of the distance between two strings is the extent towhich they can be aligned, or matched up.
Technically, an alignment is simply a way of writing the strings oneabove the other.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 107: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/107.jpg)
The Problem
When a spell checker encounters a possible misspelling, it looks in itsdictionary for other words that are close by.• Q: What is the appropriate notion of closeness in this case?
A natural measure of the distance between two strings is the extent towhich they can be aligned, or matched up.
Technically, an alignment is simply a way of writing the strings oneabove the other.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 108: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/108.jpg)
The Problem
The cost of an alignment is the number of columns in which the lettersdiffer.
The edit distance between two strings is the cost of their best possiblealignment.
Edit distance is so named because it can also be thought of as theminimum number of edits.
Insertions, deletions, and substitutions of characters that needed totransform the first string into the second.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 109: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/109.jpg)
The Problem
The cost of an alignment is the number of columns in which the lettersdiffer.
The edit distance between two strings is the cost of their best possiblealignment.
Edit distance is so named because it can also be thought of as theminimum number of edits.
Insertions, deletions, and substitutions of characters that needed totransform the first string into the second.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 110: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/110.jpg)
The Problem
The cost of an alignment is the number of columns in which the lettersdiffer.
The edit distance between two strings is the cost of their best possiblealignment.
Edit distance is so named because it can also be thought of as theminimum number of edits.
Insertions, deletions, and substitutions of characters that needed totransform the first string into the second.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 111: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/111.jpg)
The Problem
The cost of an alignment is the number of columns in which the lettersdiffer.
The edit distance between two strings is the cost of their best possiblealignment.
Edit distance is so named because it can also be thought of as theminimum number of edits.
Insertions, deletions, and substitutions of characters that needed totransform the first string into the second.
S − N O W Y
S U N N − Y
Cost: 3
− S N O W − Y
S U N − − N Y
Cost: 5
![Page 112: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/112.jpg)
An Example
Edit distance between EXPONENTIAL and POLYNOMIAL,subproblem E(4, 3) corresponds to the prefixes EXPO and POL. Therightmost column of their best alignment must be one of the following:
O
−
or−
Lor
O
L
Thus, E(4, 3) = min1 + E(3, 3), 1 + E(4, 2); 1 + E(3, 2).(a)
i
j − 1 j
i− 1
m GOAL
n
(b)
P O L Y N O M I A L
0 1 2 3 4 5 6 7 8 9 10
E 1 1 2 3 4 5 6 7 8 9 10
X 2 2 2 3 4 5 6 7 8 9 10
P 3 2 3 3 4 5 6 7 8 9 10
O 4 3 2 3 4 5 5 6 7 8 9
N 5 4 3 3 4 4 5 6 7 8 9
E 6 5 4 4 4 5 5 6 7 8 9
N 7 6 5 5 5 4 5 6 7 8 9
T 8 7 6 6 6 5 5 6 7 8 9
I 9 8 7 7 7 6 6 6 6 7 8
A 10 9 8 8 8 7 7 7 7 6 7
L 11 10 9 8 9 8 8 8 8 7 6
![Page 113: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/113.jpg)
A Dynamic Programming Solution
When solving a problem by dynamic programming, the most crucialquestion is,
What are the subproblems?
Our goal is to find the edit distance between two stringsx[1, . . . ,m] and y[1, . . . , n] .
For every i, j with 1 ≤ i ≤ m and 1 ≤ j ≤ n, let• E(i, j): the edit distance between some prefix of the first string,
x[1, . . . , i], and some prefix of the second, y[1, . . . , j].
E(i, j) = min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1),where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.
![Page 114: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/114.jpg)
A Dynamic Programming Solution
When solving a problem by dynamic programming, the most crucialquestion is, What are the subproblems?
Our goal is to find the edit distance between two stringsx[1, . . . ,m] and y[1, . . . , n] .
For every i, j with 1 ≤ i ≤ m and 1 ≤ j ≤ n, let• E(i, j): the edit distance between some prefix of the first string,
x[1, . . . , i], and some prefix of the second, y[1, . . . , j].
E(i, j) = min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1),where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.
![Page 115: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/115.jpg)
A Dynamic Programming Solution
When solving a problem by dynamic programming, the most crucialquestion is, What are the subproblems?
Our goal is to find the edit distance between two stringsx[1, . . . ,m] and y[1, . . . , n] .
For every i, j with 1 ≤ i ≤ m and 1 ≤ j ≤ n, let• E(i, j): the edit distance between some prefix of the first string,
x[1, . . . , i], and some prefix of the second, y[1, . . . , j].
E(i, j) = min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1),where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.
![Page 116: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/116.jpg)
A Dynamic Programming Solution
When solving a problem by dynamic programming, the most crucialquestion is, What are the subproblems?
Our goal is to find the edit distance between two stringsx[1, . . . ,m] and y[1, . . . , n] .
For every i, j with 1 ≤ i ≤ m and 1 ≤ j ≤ n, let
• E(i, j): the edit distance between some prefix of the first string,x[1, . . . , i], and some prefix of the second, y[1, . . . , j].
E(i, j) = min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1),where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.
![Page 117: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/117.jpg)
A Dynamic Programming Solution
When solving a problem by dynamic programming, the most crucialquestion is, What are the subproblems?
Our goal is to find the edit distance between two stringsx[1, . . . ,m] and y[1, . . . , n] .
For every i, j with 1 ≤ i ≤ m and 1 ≤ j ≤ n, let• E(i, j): the edit distance between some prefix of the first string,
x[1, . . . , i], and some prefix of the second, y[1, . . . , j].
E(i, j) = min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1),where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.
![Page 118: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/118.jpg)
A Dynamic Programming Solution
When solving a problem by dynamic programming, the most crucialquestion is, What are the subproblems?
Our goal is to find the edit distance between two stringsx[1, . . . ,m] and y[1, . . . , n] .
For every i, j with 1 ≤ i ≤ m and 1 ≤ j ≤ n, let• E(i, j): the edit distance between some prefix of the first string,
x[1, . . . , i], and some prefix of the second, y[1, . . . , j].
E(i, j) = min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1),where diff(i, j) is defined to be 0 if x[i] = y[j] and 1 otherwise.
![Page 119: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/119.jpg)
The Algorithm
for i = 0 to m doE(i, 0) = i;
endfor j = 1 to n do
E(0, j) = j;endfor i = 1 to m do
for j = 1 to m doE(i, j) =min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1);
endendreturn(E(m, n));
The over running time is O(m · n).
![Page 120: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/120.jpg)
The Algorithm
for i = 0 to m doE(i, 0) = i;
endfor j = 1 to n do
E(0, j) = j;endfor i = 1 to m do
for j = 1 to m doE(i, j) =min1+E(i−1, j), 1+E(i, j−1),diff(i, j)+E(i−1, j−1);
endendreturn(E(m, n));
The over running time is O(m · n).
![Page 121: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/121.jpg)
Then, What is An Algorithm?
![Page 122: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/122.jpg)
What Is An Algorithm
An algorithm is a procedure that consists of• a finite set of instructions which,• given an input from some set of possible inputs,• enables us to obtain an output through a systematic execution of
the instructions• that terminates in a finite number of steps.
![Page 123: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/123.jpg)
Not Enough, Maybe
![Page 124: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/124.jpg)
What Is An Algorithm
In these problems we are searching for a solution (path, tree, matching,etc.) from among an exponential population of possibilities.
All these problems could in principle be solved in exponential time bychecking through all candidate solutions, one by one.
The quest for algorithms is about finding clever ways to bypass thisprocess of exhaustive search, using clues from the input in order todramatically narrow down the search space.
![Page 125: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/125.jpg)
What Is An Algorithm
In these problems we are searching for a solution (path, tree, matching,etc.) from among an exponential population of possibilities.
All these problems could in principle be solved in exponential time bychecking through all candidate solutions, one by one.
The quest for algorithms is about finding clever ways to bypass thisprocess of exhaustive search, using clues from the input in order todramatically narrow down the search space.
![Page 126: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/126.jpg)
What Is An Algorithm
In these problems we are searching for a solution (path, tree, matching,etc.) from among an exponential population of possibilities.
All these problems could in principle be solved in exponential time bychecking through all candidate solutions, one by one.
The quest for algorithms is about finding clever ways to bypass thisprocess of exhaustive search, using clues from the input in order todramatically narrow down the search space.
![Page 127: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/127.jpg)
Is That ALL?
![Page 128: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/128.jpg)
Lecture Agenda
• More on Graph Algorithms• NP Problem• Linear Programming• Network Flows• Coping with NP Completeness• Approximation Algorithms• Randomized Algorithms• Bigdata Algorithms
![Page 129: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/129.jpg)
Referred Materials
• [DPV07] Algorithms• [KT05] Algorithm Design• [CLRS09] Introduction to Algorithms• [Vaz04] Approximation Algorithms• [WS11] The Design of Approximation Algorithms• [MR95] Randomized Algorithms
• Content of this lecture comes from section 2.3, 4.4, 4.5 and 6.3 in[DPV07].• Suggest to read Chapter 15 of [CLRS09] and Chapter 6 in
[DPV07].
![Page 130: Design and Analysis of Algorithms (I)basics.sjtu.edu.cn/~liguoqiang/teaching/X037506/lectures/... · 2020-02-20 · Design and Analysis of Algorithms (I) Introduction Guoqiang Li](https://reader033.vdocument.in/reader033/viewer/2022041914/5e68da1542d6907423006e12/html5/thumbnails/130.jpg)
Referred Materials
• [DPV07] Algorithms• [KT05] Algorithm Design• [CLRS09] Introduction to Algorithms• [Vaz04] Approximation Algorithms• [WS11] The Design of Approximation Algorithms• [MR95] Randomized Algorithms
• Content of this lecture comes from section 2.3, 4.4, 4.5 and 6.3 in[DPV07].• Suggest to read Chapter 15 of [CLRS09] and Chapter 6 in
[DPV07].