natural computation ii virginia de sa desa at cogscidesa/oldpublic_html/118b/lec2out.pdf · be the...
TRANSCRIPT
1Cogsci 118B
Natural Computation II
Virginia de Sadesa at cogsci
2Probability Mass function
Discrete random variables have probability mass functions P associated with them
∑a
P (a) = 1
for all possible outcomes a
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 17
Probability Mass functionProbability Mass function
0
0.5
1
1 2 3 4 5 6a
P (a )
A discrete random variable D has an associatedprobability mass function (pmf) P, defined over the setof outcomes U: P(D = a) = P(a), .
Example:die roll U = {1, 2, 3, 4, 5, 6}What is the pmf of thisrandom variable?
Ua!
" " ###a a
aPaDP 1)()(
Property:
uniform distribution
[figure from J. Triesch]
3Continuous probability density
For continuous random variables it does not make sense to talk about theprobability of a particular value (which is equal to 0)
Instead we talk about probability density
p(x) is a probability density over a continuous variable
Pr(x ∈ [a, b]) =∫ b
a
p(x)dx
p(x) = lim∆x→0
P (x < X ≤ x + ∆x)∆x
e.g. probability density of heights of females
∫ ∞
−∞p(x)dx = 1
4Expected Value
Expected value or mean
E(X) =∑
xP (x)
E(X) =∫
xp(x)dx
E(f(X)) =∫
f(x)p(x)dx
5Variance
V ar(X) = E(X − E(X))2 =∑
P (X)(X − E(X))2
V ar(X) =∫
(x− E(x))2p(x)dx
Note we can prove that
V ar(X) = E(X − E(X))2 = E(X2)− (E(X))2
6Cumulative distribution function
p(x) = lim∆x→0
P (x < X ≤ x + ∆x)∆x
cdf
Φ(x) = P (x < X) =∫ ∞
−∞p(x′)dx′
Examples on board
• 0 ≤ Φ(x) ≤ 1
• cdf is monotonically non-decreasing
• probability of X being between A and B = P (A ≤ X ≤ B) = Φ(B)− Φ(A) =∫ B
Ap(x)dx
• pdf is the derivative of the cdf (cdf is the integral of the pdf)
7Gaussian Distribution
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 7
8Gaussian Distribution
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 8
9The Normal Density
���� ��� ��� ���
�
� � �
�������������������� "!�# $%'& (*),+.-0/1321 46578 9�:<;=8>:<?3@A7CB;=DE:<FHG83I�?3;=83J5?383BC7�K:<IL;=B5M�KF3N0OPQRBSH83?3IL:<;=@<:0837�?>K@0;T:<7M�@U VXWZY,[�\^]C_a`b<cdcfegCh,ijlkmenpon<b�q�grs3enptu3cfs3v=u3wxs>u3giEeb<cay�b<z3xn|{~}3�L�����C��� �C�~�d�l���=��X����3���� �=���A���,�� �<���m <¡3 �~¢���£ � � ¡3���<¤ ��� ��¥<� ��¦p��§ ¡ ��T¨ �a©�ª<«>«3¬<=®°¯�±3ª<²f²f³3´Cµ<ª<«3³3¶®.·�¸�¹º¼»<½=¾3¿<ÀÁÂÃÅÄÆÆCÇEȼÉLÊÌËÍÎEÏ6Ð3Ñ3Ò<ÉpÓÕÔ<ËCÎÖf×Ø3ÎÙ�Ú Pattern Densities are commonly modeled byNormal Densities for several reasons
• Central Limit Theorem: sum of a large number of independent random variablesis normally distributed
• It’s analytically tractable!
• It’s been well studied
• It has the maximum entropy of all distributions with a given mean and variance
10Univariate normal density
p(x) =1√2πσ
e−(x−µ)2
2σ2
has mean = µ
variance = σ2
has roughly 95% of its area within 2 standard deviations on either side of themean (this is relevant for t-tests).
11Little Quiz
What is ∫ ∞
−∞
1√2π
e−(x−5)2
2 =
What is ∫ ∞
−∞e−(x−5)2
2 =
What is ∫ 5
−∞e−(x−5)2
2 =
What is ∫ ∞
−∞xe
−(x−5)2
2 =
12Multivariate Gaussians
13Central Limit Theorem
LetSn = X1 + X2 + ... + Xn
be the sum of n independent random variables with the same distribution (IID)with finite non-zero variance σ2 and mean µ
Then Sn is approximately normally distributed with mean E(Sn) = nµ andvariance V ar(Sn) = σ2n
limN−>∞
P(Sn − nµ
σ√
n≤ x) = Φ(x)
where Φ(x) is the probability that a standard normal variable is less than x
15Signal Detection Theory
Consider the case of a Doctor trying to determine whether a biopsy is cancerousor not.
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 15
Signal Detection TheorySignal Detection Theory
Example setting: consider radiologist trying to detect tumor in X-ray
The parts of the X-ray picture will be represented by certain activitypatterns in the radiologist’s brain, upon which the radiologist makesthe decision “tumor” or “no tumor”.
For now consider the case where the doctor is looking at a one-dimensionalmeasurement (and for the next few slides assume that the data is distributednormally for both the cancerous and non-cancerous case but with different means(same variance))
16Signal Detection Theory
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 17
Let’s consider the simple case that radiologist decides “tumor” if thefiring rate (internal response) is above a certain threshold (criterion):
Probabilities related to hit, miss,false alarm, correct rejection:
ratehit : )yes|*( xxp !
detection missed : )yes|*( xxp "
alarm false : )no|*( xxp !
rejectioncorrect : )no|*( xxp "
define: hit rate, false alarm rate, miss rate, correct rejection rate
17Discriminability
We may want a measure of how separable the two classes are independent of ourdecision threshold
d′ =|µ2 − µ1|
σ
intrinsic measure of discriminability (independent of decision threshold)
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 18
Depending on the overlap of the distributions and on the criterion,hits and false alarms will be more or less likely.
Define discriminability d’:
!"" 21'
#$d
Note: d’ does not depend on criterion c*, measures inherent difficulty of task.But what c* should we choose? (see blackboard)
18Changing the criterion (cutoff)
Changing the criterion will change the number of hits and false alarms
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 18
Depending on the overlap of the distributions and on the criterion,hits and false alarms will be more or less likely.
Define discriminability d’:
!"" 21'
#$d
Note: d’ does not depend on criterion c*, measures inherent difficulty of task.But what c* should we choose? (see blackboard)
19Signal Detection Theory
��
��� �
��� ��� � � �� ���
��������� ������� ��!#"%$%! &�'%(*)#+%,.-/+10 )#+�243#-/+%36587%9/+:+%;:9/</3#9/(*+%->=@?%'%=#2A9.)#2B?%(*9>249/+%3#C63#7%9.?%(*;%D%-/D�)#=#)#3#0E 9/+%24)#3F0HG#;%(I-/+J)#+%3#9/(*+%->=K24)#,>+%-/=K)F26+%;%(MLJ-/=#CK3F7%-/3K)#2ACONQP R@S T@UWVYX[Z�\#]_^4`/a�b>cAd%e�f�g/hjiFf%gHg>k/i#g/l*h%m>no4p#q h%m/n p#osr l*g o g/h%i#tui#f%gwv%g>h o4p i#x p#o_yQz {@| }�~>�j���������>�>�I���A�u�u�1�.���/�/�#�4�#�%���#�%�M�/�4�%�%�#�w���B�s #¡#¡¢%£/¤#£/¥*¦ #§ £H¤#¨�£H©%¥*ª%«�¬/« #¡# ¤#sª%®K¬s¨ ¤K¯F¤#¨%£H© #§%° ¬/¥M£/¬H± § ¢%£/¥I¤#¨%£�²�³s´/µ%¶M·>¸/¹Kº>»%¼�·>¸_½�¾À¿KÁ/Â%ÃJÄ%ÅKÁÅ#Á/Æ#Ç4È8Á>Æ#Á/É*ÊÌË#Í#Î%ÈÐÏ%Æ#Á/Ñ�Ò8Á/É*È/Á8Ó%Â�Ã%È/É�Í#Î%ÈÕÔ@ÖY×/Ø%Ù*Ú�Û/Ü1Ý/Þ%ß�Ú�ÛQà�áãâ#ä�åçæ*è�éJê1ëIì#í�î%ï/æMðòñBä1ósô%ð%ï/õ�öO÷/ø#÷/æù äKúsï/æ*ø#õKï/û%ðjósïçü/ì#ðjýsäKþ/ø#è�æ*ÿ/õ��������������� �������������������������������� "!�#�$�%'&(*)�+�+�,.-�/1032�4�5.687�9�:�/<;= 2�5�>�?�@�5BA�C
20ROC curve
A plot of hit rate vs false alarm rate
Plot pink area over black area (from previous slide) as the threshold is moved
��� ������� ���� �� �
�� ����� ������
�� �!
�� �#"
�$� �!%�$� �!&
'
'($)+*-, ./)+*-)10�2
3!4 5
687 9;:=<?>A@CB�@CDCB E�FHGJI-KMLMKMN O�K�IQPCRCKMI-G�S�N�FCTALVUCGMIWGMLMS�KMIXN�Y$S�N�LJZ�[]\_^a`bLMcCI-O�KMd/S�UCKAGMeCYfLMNgYfYfGJNgYhS�UiKRCI-PCeCGMeCN�j�N�SgklPCm�mgGMj�YfKlGMjgGMIXn_d�o/p qsrut�v�w xsy{zb|f}$~��M�C�;���C�l�C�X�i���C�M���l�������C�l�C�-�C�C�M�C�����������C���C����~�/� �s���8�M� �s�{�8���$�8���X i¡£¢�¤C¥�¡l¥M¦M§f¨C�X¥�©ª¤C«�¢8¦M¬C©;�¦M®�§f¥�¦�®�¦M�X¡£�W¦M¢�¥M§�¯�¤C¥M�X¥�°M C�-�X¥M§f±C C¬C©i«�¬C²l¢g ³�´/µ�¶l·Vµ�¸�¹iºC¹�»C¼l½�¶C¾l¿fÀCÁiÂöl½M¿aÄ�ÀiÅÃÆXÅM¾l¾iÁCÄ�Ç�ÈiÂ=ÅÃÉ�½M¶l¾CÅM¾iÊCÉMÅÃÄ�ÀC½MÄ�Ë�Ì#ÍÏÎCÐiÑÓÒXÔCÕlÖi×]Ø�ÙVÚCÛMÒXÜlÝhÐÞÃß ÜCÛMàâá#ãMä�ãMÒ/åMÐâæÃÛMÒXä�àâÛMçCÜ Þ Û�èMØ�ÜAéÃÐâêMä�ÔCÒXëMàaì�í�î�î�ïMðXñAòôó�íMõ$õfö�÷CøMíMî�ögùCñ?úâûôüCýÿþ����������� ����������������������������! #"�����$&%�'���(�)
ROC curves are commonly used by people studying sensory perception to measure
21the discriminability between different stimuli
Why don’t we use classification error?
22Operating Characteristic
Generalizes ROC to when the two categories are not Gaussian
�
��� ��� ��� ��
��
�
�
��� ����������������� �
� !" !#$! %& '(
)�* +
, -�. / 01-�. -�2 3
465 798;:�<>=@?�=@A@? BDCFE>G�H�C@H�I�E�JLK@MNH�IOE�P�QDC@GSR�TNE�IOE�R�PDH�IUQ�V�P�Q�R>R�W@IUX�H�YLPDT@H>E�Z@V�R�QDV�V�E>Q�V[P�T@H>M@IUK@ZNE�Z@Q�J�Q�P�\K@]^]�E�J�V_H>E`J�E�IUabYNced fhgjilk�m nhoqperts_u^v�w@xzy�{@|~}N�Ux@��w@v�y�|~����y�{N|~�@�U}@�@v��@�����Dy��~}@�^{@��y�uN�e� �h�j����� �h��������������D�����@���� O¡`��¢�£9¤@¢` U¢�¥6¦@§N¢� O¡�����¨N©«ª�¤@¡� �¡�ª���¢� O��������ª«ª��@ U¬�¢���¡� O¢«©�¢�¨@¢� �¡�����F¨@¦@�6��`®b®b¢��� O��ª�¥6¡����¤@¦N¯;¨°¡��±��¤N¢b O��©�¤@���±²� U¦@®q³±´±��ª�¤@¡� O£°µ«�±¶;�@£@¡�¥�·^¢���¢� ¹¸��±ºj¡` U��¥±¡�¨@£»¶j¡�¬���£»¼j�±½���¦N U¾�¥À¿�Á`Â�Â�Ã�ÄUÅÆ�Ç Á�È�È�É�ÊNË�Á�Â�É�Ì@Å�Í@ιÏ@ÐLÑ�ÒUÓ�Ô�Õ@ÖF×ØÚÙ@Û@Û@ÜqÝNÞeßáà@â@ãbäæå�ç�è`Þjéëê�à@ã@ì_í@î�ã@ï�ð