sensing and perception - brooklyn collegekammet/lectured.pdfjoel kammet brooklyn college cuny d.7...
TRANSCRIPT
![Page 1: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/1.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.1
Unit D
Sensing and Perception
Exploring RoboticsSpring, 2013
Joel KammetBrooklyn College
CUNY
![Page 2: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/2.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.2
Why does a robot need sensors?
• the environment is complex• the environment is dynamic
Sensors enable the robot to learn about currentconditions in its environment.
![Page 3: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/3.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact
• obstacles• texture of terrain• distance to objects, size of objects• object recognition• robot’s own orientation (heading, tilt)• odometry• location
![Page 4: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/4.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles
• texture of terrain• distance to objects, size of objects• object recognition• robot’s own orientation (heading, tilt)• odometry• location
![Page 5: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/5.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles• texture of terrain
• distance to objects, size of objects• object recognition• robot’s own orientation (heading, tilt)• odometry• location
![Page 6: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/6.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles• texture of terrain• distance to objects, size of objects
• object recognition• robot’s own orientation (heading, tilt)• odometry• location
![Page 7: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/7.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles• texture of terrain• distance to objects, size of objects• object recognition
• robot’s own orientation (heading, tilt)• odometry• location
![Page 8: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/8.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles• texture of terrain• distance to objects, size of objects• object recognition• robot’s own orientation (heading, tilt)
• odometry• location
![Page 9: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/9.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles• texture of terrain• distance to objects, size of objects• object recognition• robot’s own orientation (heading, tilt)• odometry
• location
![Page 10: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/10.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.3
What kind of information does a robot need?
Examples
• contact• obstacles• texture of terrain• distance to objects, size of objects• object recognition• robot’s own orientation (heading, tilt)• odometry• location
![Page 11: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/11.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.4
Sensor categories
• Proprioceptive sensors provide information about therobot’s own body, especially the positions of its partsin relation to each other:
• angles of its joints• positions of it servo motors• direction “head” is facing relative to the body• is it standing, sitting, lying on its back?
• Interoceptive sensors provide information about therobot’s internal condition
• battery condition• internal temperature
• Exteroceptive sensors provide information about theexternal environment.
• presence/location of other objects in its surroundings• sound and other broadcast signals (e.g.
communication)
![Page 12: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/12.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.5
Sensor categories
• Passive sensors measure signals provided by theenvironment
• Active sensors emit their own signals and measuretheir interaction with the environment
![Page 13: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/13.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.6
Sensors
Examples
• switches (electrical current: on or off)• light sensors (photocells: resistance low when
illuminated)• passive: ambient light• active: infrared
• cameras• sonar (emit sound, measure reflected signal)• radar (emit radio waves, measure reflected signal)• laser (emit coherent light, measure reflected signal)• accelerometers (measure acceleration and
orientation)• compasses (measure orientation wrt magnetic field)• altimeters (measure air pressure)
![Page 14: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/14.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.7
Noise and Uncertainty
In the physical world,every measurement involves estimation.
Sources of uncertainty
• measurement errors• sensor noise• sensor resolution• effector/actuator noise• incomplete knowledge about environment• environmental conditions (hidden or partially
observable state)
![Page 15: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/15.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.8
State
• a description of a system in terms of parameters orattributes
• a set of values of the variables in a mathematicalmodel
• a unique configuration (snapshot) of the informationin a program or machine
Do sensors provide state?
![Page 16: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/16.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.9
Sensing vs. Perception
Perception requires:• sensor data• signal processing• computation
Perception:
understanding of sensory information
![Page 17: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/17.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.10
Complexity of Perception
Reconstructing the world is too hard.
Ways to reduce computational cost:
• action-oriented perception (“need to know”)• expectation-based perception (use knowledge about
environment)• task-driven attention• perceptual classes (Goldilocks)
![Page 18: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/18.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.11
Vision
What does vision entail?
• babies do it• insects do it
What’s so hard about vision?
![Page 19: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/19.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.12
Human Eye
About 126 million light-sensitive cells in the retina.
![Page 20: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/20.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.13
Robot Eye (Camera)
A few hundred thousand (or a few million) photocells inthe camera.
![Page 21: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/21.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.14
Camera Images
A camera image is made up of pixels.
![Page 22: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/22.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.15
RGB Color Chart
![Page 23: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/23.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.16
RGB Color Space
![Page 24: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/24.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.17
An Aibo’s-eye view of the field
![Page 25: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/25.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.18
Image representation in the robot
Image segmentation
• thresholding• scan for values within a range• ...
![Page 26: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/26.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.19
Loops and Jumps
Conditional:
Unconditional:
![Page 27: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/27.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.20
Conditional Branch: Touch Sensor Fork
Not so useful:
Much better:
![Page 28: Sensing and Perception - Brooklyn Collegekammet/lectureD.pdfJoel Kammet Brooklyn College CUNY D.7 Noise and Uncertainty In the physical world, every measurement involves estimation](https://reader033.vdocument.in/reader033/viewer/2022052004/6017898272457a7c5343ad7c/html5/thumbnails/28.jpg)
Sensing and Perception
SensingSensors
Uncertainty
State
Perception
Vision
RoboLabLoops and Jumps
Conditional Branches
Joel KammetBrooklyn CollegeCUNY
D.21
Conditional Branch: Light Sensor Fork
Not so useful:
Much better: