csc446: pattren recognition (ln1)
TRANSCRIPT
![Page 1: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/1.jpg)
Slide - 1
CSC446 : Pattern Recognition
Prof. Dr. Mostafa G. M. Mostafa Faculty of Computer & Information Sciences
Computer Science Department
AIN SHAMS UNIVERSITY
Lecture Note 1:
Course Organization &
Chapter 1: Introduction to PRS
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 2: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/2.jpg)
Slide - 2
CSC446: Patt Recog - Course Outline
• Introduction to PRS
• Mathematical Foundations
• Supervised Learning
• Bayesian Decision Theory
• Maximum Likelihood Estimation
• Non-Parametric Methods
• Linear Discriminant Functions & NN
• Unsupervised Learning
• K-mean Clustering
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 3: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/3.jpg)
Slide - 3
• Text Book:
– Duda, Hart, and Stork. “Pattern
Classification”, 2nd ed. Wiley, 2001.
• Reference Book:
– C. M. Bishop. “Pattern Recognition & Machine
Learning”. Springer, 2007.
– A. Webb. “Statistical Pattern Recognition”. Arnold,
1999.
• Lab book: Handout materials + “Matlab
Getting Started” and “Building GUI” tutorials.
CSC446 : Course Organization & Guidelines
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 4: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/4.jpg)
Slide - 4
Prerequisites:
– CSW150: Structured Programming.
– SCC223 : Probability & Statistics
– SCC332 : Numerical Methods
– CSC343 : Artificial Intelligence
CSC446 : Course Organization & Guidelines
Refresh your Information
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 5: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/5.jpg)
Slide - 5
Grading:
• Midterm Exam (10 points)
• Assignments, Quizzes (10 points)
• Final Project, Lab test (15 points)
• Final Exam (65 points)
CSC446 : Course Organization & Guidelines
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 6: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/6.jpg)
Slide - 6
CSC446 : Course Organization & Guidelines
Lecture Protocol:
Feel free to interrupt and ask ME.
DON’T ask/talk to your colleagues.
Programming and homework assignments
•Late answers are given 50% of the mark.
Slides are available in pdf format.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 7: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/7.jpg)
Slide - 7
CSC446 : Course Organization & Guidelines
How to pass this course?
– You will learn a lot during this course, but you
will have to work hard to pass it!
– Don’t accumulate …
– Do it yourself …
– Ask for help …
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 8: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/8.jpg)
Slide - 8
CSC446 : Course Organization & Guidelines
Warning:
– Working policy: You are encouraged to collaborate
in study groups. But submitting a copy or slightly
changed others’ solutions or codes is Cheating.
– Cheating will be punished severely
• Assignments: All get 0
• Midterm or Final: you will get Fail
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 9: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/9.jpg)
Slide - 9
CSC446 : Course Organization & Guidelines
Resources (Most important):
Societies:
IAPR: http://www.iapr.org/
Journals:
PAMI: http://www.computer.org/tpami/
PR: http://www.elsevier.com/locate/pr
PRL: http://www.elsevier.com/locate/prl
Web Sites:
PRInfo: http://www.ph.tn.tudelft.nl/PRInfo/
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 10: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/10.jpg)
Slide - 10
Chapter 1:
Introduction to Pattern
Recognition
CSC446 : Pattern Recognition
(Read all sections in Chapter 1)
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 11: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/11.jpg)
Slide - 11
• Objectives of Pattern Recognition Systems
• Applications of Pattern Recognition Systems
• What is a Pattern Recognition System?
• An intuitive Example
• Components of Pattern Recognition Systems
• The Design Cycle
• Learning and Adaptation
– Supervised, Unsupervised, and Reinforcement Learning.
Intro Pattern Recognition - Outline
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 12: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/12.jpg)
Slide - 12
PRS Objective:
• Building a machine that can learn and
recognize patterns as human,
• Having such a machine is immensely useful
to mankind.
Pattern Recognition System
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 13: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/13.jpg)
Slide - 13
Pattern Recognition Systems
What is a Pattern?
What is a Pattern Recognition
System?
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 14: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/14.jpg)
Slide - 14
What is a Pattern?
• A pattern is a set of instances which share
some regularities, and are similar to each
other in the set.
• A pattern should occur repeatedly.
• A pattern is observable, sometimes partially,
by some sensors with noise and distortions.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 15: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/15.jpg)
Slide - 15
Examples of Patterns
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 16: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/16.jpg)
Slide - 16
Examples of Patterns
Speech Signal
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 17: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/17.jpg)
Slide - 17
What is Pattern Recognition?
• Definition: “The act of taking in raw data
and taking an action based on the category
of the pattern found in the data.”
an object
Decision raw data
Pattern
Recognition
System (Cylinder)
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 18: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/18.jpg)
Slide - 18
Applications of PRS
Applications: Robotics
Photo by courtesy of US Department of Energy .
Robot Manny is developed
at Battelle's Pacific
Northwest Laboratories in
Richand, Washington. It
took 12 researchers 3
years (1986-1989) and $2
million to develop this
robot. Manny was built
for the U.S. Army in the
late 1980s to test protective
clothing.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 19: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/19.jpg)
Slide - 19
Applications of PRS
Applications: In Military
A remote-controlled
bomb disposal robot
in action.
Photo by courtesy of US Airforce .
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 20: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/20.jpg)
Slide - 20
Applications of PRS
• OCR: Handwritten/printed optical characters recognition.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 21: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/21.jpg)
Slide - 21
Applications of PRS –Dictation machines, Voice Command, HCI
HCI,
Archiving Dictation
Voice
Command
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 22: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/22.jpg)
Slide - 22
Applications of PRS •Investigation: Lie detector,
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 23: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/23.jpg)
Slide - 23
Applications of PRS
•Other Applications:
– Manufacturing:
• Defect detection in chip manufacturing
• Fruit/vegetable recognition
– Biometrics: voice, iris, fingerprint, face, gait
recognition
– Medical diagnosis
–Smell recognition (e-nose, sensor networks)
–Bioinformatics: classification of DNA sequences.
–Security: intrusion detection
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 24: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/24.jpg)
Slide - 24
Machine Learning
Concepts
Readings: Chapter 1 in Bishop’s PRML
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 25: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/25.jpg)
Slide - 25
Machine Learning
• Learning Process:
–Learner (a computer program; an agent) processes data D
representing past experiences and tries to either develop an
appropriate response to future data, or describe the seen
data in some meaningful way.
• Example:
– Learner sees a set of patient records with corresponding
diagnoses. It can either try to :
– predict the presence of a disease for future patients.
– describe the dependencies between diseases, symptoms.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 26: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/26.jpg)
Slide - 26
Types of Machine Learning • Supervised learning
– Learning mapping between input x and desired output y
– Teacher presents some samples of pairs (x, y)
• Unsupervised learning
– Learning relations between data components
– No teacher signal.
• Reinforcement learning
– Learning mapping between input x and desired output y
– Teacher gives a Critic signal (reinforcement) of
how good the response was.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 27: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/27.jpg)
Slide - 27
Supervised Learning
• Data: A set of n examples
𝑿 = {𝒙𝟏, 𝒙𝟐, …, 𝒙𝒏} & 𝑻 = {𝒕𝟏, 𝒕𝟐, …, 𝒕𝒏}
x is input vector, and t is desired output (given by a teacher).
• Objective: Learn the mapping 𝑭: 𝑿 → 𝒀
That is to find: 𝒚𝒊 ≈ 𝒇(𝒙𝒊) for all 𝒊 = 𝟏, … , 𝒏
• Two types of problems:
Regression: X discrete or continuous Y is continuous
Classification: X discrete or continuous Y is discrete
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 28: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/28.jpg)
Slide - 28
Supervised Learning Examples
• Regression: Y continuous
Debt/equity
Earnings company stock price
Future product orders
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 29: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/29.jpg)
Slide - 29
Supervised Learning Examples
• Classification: Y discrete
{ a, b, c, …, x, y, z}
X is a vector/sequence of values
{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 30: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/30.jpg)
Slide - 30
Unsupervised Learning
• Data: A set of n examples
𝑿 = {𝒙𝟏, 𝒙𝟐, …, 𝒙𝒏}
desired output NOT GIVEN (no teacher).
• Objective:
– Learn the relation between data components
• Two types of problems:
Clustering: Group “similar” examples together
Density Estimation : Model probabilistically the samples
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 31: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/31.jpg)
Slide - 31
Unsupervised Learning Examples
• Clustering: Group “similar” examples together
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 32: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/32.jpg)
Slide - 32
Unsupervised Learning Examples
• Density Estimation: Find probability density p(x)
Model used : Mixture of Gaussians
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 33: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/33.jpg)
Slide - 33
Reinforcement Learning
• We want to learn: 𝑭: 𝑿 → 𝒀
• We only see samples of x but not y
• Instead of getting y we get a feedback
(reinforcement) from a critic about how good our
output was.
• Example:
– Real time strategic (RTS) games.
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq
![Page 34: Csc446: Pattren Recognition (LN1)](https://reader031.vdocument.in/reader031/viewer/2022020214/589bff801a28ab4f598b49c1/html5/thumbnails/34.jpg)
Slide - 34
Next Time
Introduction to Pattern
Recognition
ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq