3d from photographs: structure from motion · 2020-04-28 · 3d from photographs: structure from...

3D from Photographs: Structure From Motion

Dr Francesco Banterle francesco.banterle@isti.cnr.it

Note: in these slides the optical center is placed back to simplify drawing and understanding.

3D model

3D from PhotographsAutomatic

Matching of Images

Camera Calibration

Dense Matching

Photographs

Surface Reconstruction

3D model

3D from PhotographsAutomatic

Matching of Images

Camera Calibration

Dense Matching

Photographs

Surface Reconstruction

Camera Pose Calibration

Camera Pose Calibration• We have seen methods for estimating the intrinsic

matrix K, and the extrinsic matrix G = [R | t] using a calibration pattern:

• DLT

• Zhang’s algorithm

• How do we get the camera’s pose, G, without patterns?

Camera Pose Calibration

• Let’s assume that:

• We have K for each photograph.

• We have matches between images.

A Two-Camera Example

A Two-Camera ExampleM

m1

m2

C1

C2

Camera 1

Camera 2

A Two-Camera Example: Epipolar Geometry

M

m1

m2

C1

C2

Camera 1

Camera 2

C1C2 is called baseline

A Two-Camera Example• Let’s assume that we have K1 and K2.

• Let’s assume that G1 is set in the origin and aligned with the reference frame. This means:

Note that we only need to estimate R and t !

G1 = [I|0] ! P1 = K1 ·G1

P2 = K2 · [R|t]

A Two-Camera Example

• To simplify, let’s remove K matrices:

• To points as well:

P 01 = K�1

1 · P1 = [I|0]P 02 = K�1

2 · P2 = [R|t]

m1 = K�11 ·m1

m2 = K�12 ·m2

A Two-Camera Example

• To simplify, let’s remove K matrices:

• To points as well:

P 01 = K�1

1 · P1 = [I|0]P 02 = K�1

2 · P2 = [R|t]

Normalized coordinates

m1 = K�11 ·m1

m2 = K�12 ·m2

A Two-Camera Example• Given the Longuet-Higgins equation, we know that:

• E (a 3×3 matrix) is called the essential matrix and it is defined as:

E = [t]⇥ ·R [t]⇥ =

2

40 �t3 t2t3 0 �t1�t2 t1 0

3

5

m>2 · E · m1 = 0

• If we do not have K1 and K2 and apply the Longuet-Higgins equation, we obtain:

• where F (a 3×3 matrix) is called the fundamental matrix. This matrix is linked to E as:

The Fundamental Matrix

F = K�>2 · E ·K�1

1 E = K>2 · F ·K1

<latexit sha1_base64="+NNPcjatjbkg5pujiKL5LtdcjHU=">AAACBXicbVBNSwMxEM36WevXqkcRgkXwVHaLoB6EoihCLxWsLXTrkk3TNjS7WZJZoSw9efGvePGg4tX/4M1/Y9ruQVsfDPN4b4ZkXhALrsFxvq25+YXFpeXcSn51bX1j097avtMyUZTVqBRSNQKimeARqwEHwRqxYiQMBKsH/YuRX39gSnMZ3cIgZq2QdCPe4ZSAkXx77xKf4YpfuvdAxtijbQn4atIqvuvbBafojIFniZuRAspQ9e0vry1pErIIqCBaN10nhlZKFHAq2DDvJZrFhPZJlzUNjUjIdCsdnzHEB0Zp445UpiLAY/X3RkpCrQdhYCZDAj097Y3E/7xmAp2TVsqjOAEW0clDnURgkHiUCW5zxSiIgSGEKm7+immPKELBJJc3IbjTJ8+SWql4WnRujgrl8yyNHNpF++gQuegYldE1qqIaougRPaNX9GY9WS/Wu/UxGZ2zsp0d9AfW5w+Qi5bY</latexit><latexit sha1_base64="+NNPcjatjbkg5pujiKL5LtdcjHU=">AAACBXicbVBNSwMxEM36WevXqkcRgkXwVHaLoB6EoihCLxWsLXTrkk3TNjS7WZJZoSw9efGvePGg4tX/4M1/Y9ruQVsfDPN4b4ZkXhALrsFxvq25+YXFpeXcSn51bX1j097avtMyUZTVqBRSNQKimeARqwEHwRqxYiQMBKsH/YuRX39gSnMZ3cIgZq2QdCPe4ZSAkXx77xKf4YpfuvdAxtijbQn4atIqvuvbBafojIFniZuRAspQ9e0vry1pErIIqCBaN10nhlZKFHAq2DDvJZrFhPZJlzUNjUjIdCsdnzHEB0Zp445UpiLAY/X3RkpCrQdhYCZDAj097Y3E/7xmAp2TVsqjOAEW0clDnURgkHiUCW5zxSiIgSGEKm7+immPKELBJJc3IbjTJ8+SWql4WnRujgrl8yyNHNpF++gQuegYldE1qqIaougRPaNX9GY9WS/Wu/UxGZ2zsp0d9AfW5w+Qi5bY</latexit><latexit sha1_base64="+NNPcjatjbkg5pujiKL5LtdcjHU=">AAACBXicbVBNSwMxEM36WevXqkcRgkXwVHaLoB6EoihCLxWsLXTrkk3TNjS7WZJZoSw9efGvePGg4tX/4M1/Y9ruQVsfDPN4b4ZkXhALrsFxvq25+YXFpeXcSn51bX1j097avtMyUZTVqBRSNQKimeARqwEHwRqxYiQMBKsH/YuRX39gSnMZ3cIgZq2QdCPe4ZSAkXx77xKf4YpfuvdAxtijbQn4atIqvuvbBafojIFniZuRAspQ9e0vry1pErIIqCBaN10nhlZKFHAq2DDvJZrFhPZJlzUNjUjIdCsdnzHEB0Zp445UpiLAY/X3RkpCrQdhYCZDAj097Y3E/7xmAp2TVsqjOAEW0clDnURgkHiUCW5zxSiIgSGEKm7+immPKELBJJc3IbjTJ8+SWql4WnRujgrl8yyNHNpF++gQuegYldE1qqIaougRPaNX9GY9WS/Wu/UxGZ2zsp0d9AfW5w+Qi5bY</latexit>

$<latexit sha1_base64="KThm7hMzkajF0dkF81AiBW1e9Ro=">AAAB93icbVBNS8NAEN3Ur1o/GvXoZbEInkoignorevFYwdhCG8pmu2mXbnbD7kSpob/EiwcVr/4Vb/4bt20O2vpg4PHeDDPzolRwA5737ZRWVtfWN8qbla3tnd2qu7d/b1SmKQuoEkq3I2KY4JIFwEGwdqoZSSLBWtHoeuq3Hpg2XMk7GKcsTMhA8phTAlbqudWuYDFoPhgC0Vo99tyaV/dmwMvEL0gNFWj23K9uX9EsYRKoIMZ0fC+FMCcaOBVsUulmhqWEjsiAdSyVJGEmzGeHT/CxVfo4VtqWBDxTf0/kJDFmnES2MyEwNIveVPzP62QQX4Q5l2kGTNL5ojgTGBSepoD7XDMKYmwJoZrbWzEdEk0o2KwqNgR/8eVlEpzWL+ve7VmtcVWkUUaH6AidIB+dowa6QU0UIIoy9Ixe0Zvz5Lw4787HvLXkFDMH6A+czx/SVpNk</latexit><latexit sha1_base64="KThm7hMzkajF0dkF81AiBW1e9Ro=">AAAB93icbVBNS8NAEN3Ur1o/GvXoZbEInkoignorevFYwdhCG8pmu2mXbnbD7kSpob/EiwcVr/4Vb/4bt20O2vpg4PHeDDPzolRwA5737ZRWVtfWN8qbla3tnd2qu7d/b1SmKQuoEkq3I2KY4JIFwEGwdqoZSSLBWtHoeuq3Hpg2XMk7GKcsTMhA8phTAlbqudWuYDFoPhgC0Vo99tyaV/dmwMvEL0gNFWj23K9uX9EsYRKoIMZ0fC+FMCcaOBVsUulmhqWEjsiAdSyVJGEmzGeHT/CxVfo4VtqWBDxTf0/kJDFmnES2MyEwNIveVPzP62QQX4Q5l2kGTNL5ojgTGBSepoD7XDMKYmwJoZrbWzEdEk0o2KwqNgR/8eVlEpzWL+ve7VmtcVWkUUaH6AidIB+dowa6QU0UIIoy9Ixe0Zvz5Lw4787HvLXkFDMH6A+czx/SVpNk</latexit><latexit sha1_base64="KThm7hMzkajF0dkF81AiBW1e9Ro=">AAAB93icbVBNS8NAEN3Ur1o/GvXoZbEInkoignorevFYwdhCG8pmu2mXbnbD7kSpob/EiwcVr/4Vb/4bt20O2vpg4PHeDDPzolRwA5737ZRWVtfWN8qbla3tnd2qu7d/b1SmKQuoEkq3I2KY4JIFwEGwdqoZSSLBWtHoeuq3Hpg2XMk7GKcsTMhA8phTAlbqudWuYDFoPhgC0Vo99tyaV/dmwMvEL0gNFWj23K9uX9EsYRKoIMZ0fC+FMCcaOBVsUulmhqWEjsiAdSyVJGEmzGeHT/CxVfo4VtqWBDxTf0/kJDFmnES2MyEwNIveVPzP62QQX4Q5l2kGTNL5ojgTGBSepoD7XDMKYmwJoZrbWzEdEk0o2KwqNgR/8eVlEpzWL+ve7VmtcVWkUUaH6AidIB+dowa6QU0UIIoy9Ixe0Zvz5Lw4787HvLXkFDMH6A+czx/SVpNk</latexit>

m>2 · F ·m1 = 0

<latexit sha1_base64="nVwmfvKdT5MuK4SdfZSl0uZeBcc=">AAACGHicbVBNS8NAEN3Ur1q/qh69LBbBU0mKoB6EoiAeK1hbaGLZbDft0k027E6EEvo3vPhXvHhQ8dqb/8ZNm0NtfbDs470ZZub5seAabPvHKqysrq1vFDdLW9s7u3vl/YNHLRNFWZNKIVXbJ5oJHrEmcBCsHStGQl+wlj+8yfzWM1Oay+gBRjHzQtKPeMApASN1y7YbEhj4QRqOu7UnF2SMXdqTgG/zf8538BW2u+WKXbWnwMvEyUkF5Wh0yxO3J2kSsgioIFp3HDsGLyUKOBVsXHITzWJCh6TPOoZGJGTaS6eXjfGJUXo4kMq8CPBUne9ISaj1KPRNZbanXvQy8T+vk0Bw4aU8ihNgEZ0NChKBQeIsJtzjilEQI0MIVdzsiumAKELBhFkyITiLJy+TZq16WbXvzyr16zyNIjpCx+gUOegc1dEdaqAmougFvaEP9Gm9Wu/Wl/U9Ky1Yec8h+gNr8gthnJ9x</latexit><latexit sha1_base64="nVwmfvKdT5MuK4SdfZSl0uZeBcc=">AAACGHicbVBNS8NAEN3Ur1q/qh69LBbBU0mKoB6EoiAeK1hbaGLZbDft0k027E6EEvo3vPhXvHhQ8dqb/8ZNm0NtfbDs470ZZub5seAabPvHKqysrq1vFDdLW9s7u3vl/YNHLRNFWZNKIVXbJ5oJHrEmcBCsHStGQl+wlj+8yfzWM1Oay+gBRjHzQtKPeMApASN1y7YbEhj4QRqOu7UnF2SMXdqTgG/zf8538BW2u+WKXbWnwMvEyUkF5Wh0yxO3J2kSsgioIFp3HDsGLyUKOBVsXHITzWJCh6TPOoZGJGTaS6eXjfGJUXo4kMq8CPBUne9ISaj1KPRNZbanXvQy8T+vk0Bw4aU8ihNgEZ0NChKBQeIsJtzjilEQI0MIVdzsiumAKELBhFkyITiLJy+TZq16WbXvzyr16zyNIjpCx+gUOegc1dEdaqAmougFvaEP9Gm9Wu/Wl/U9Ky1Yec8h+gNr8gthnJ9x</latexit><latexit sha1_base64="nVwmfvKdT5MuK4SdfZSl0uZeBcc=">AAACGHicbVBNS8NAEN3Ur1q/qh69LBbBU0mKoB6EoiAeK1hbaGLZbDft0k027E6EEvo3vPhXvHhQ8dqb/8ZNm0NtfbDs470ZZub5seAabPvHKqysrq1vFDdLW9s7u3vl/YNHLRNFWZNKIVXbJ5oJHrEmcBCsHStGQl+wlj+8yfzWM1Oay+gBRjHzQtKPeMApASN1y7YbEhj4QRqOu7UnF2SMXdqTgG/zf8538BW2u+WKXbWnwMvEyUkF5Wh0yxO3J2kSsgioIFp3HDsGLyUKOBVsXHITzWJCh6TPOoZGJGTaS6eXjfGJUXo4kMq8CPBUne9ISaj1KPRNZbanXvQy8T+vk0Bw4aU8ihNgEZ0NChKBQeIsJtzjilEQI0MIVdzsiumAKELBhFkyITiLJy+TZq16WbXvzyr16zyNIjpCx+gUOegc1dEdaqAmougFvaEP9Gm9Wu/Wl/U9Ky1Yec8h+gNr8gthnJ9x</latexit>

• F connects the two cameras “geometry” together.

• If we have a point m1 of I1 and we multiply it by F, we obtain a line equation in where its match, m2, lies in I2:

The Fundamental Matrix

l = F ·m1 $ (l1x+ l2y + l3) = 0<latexit sha1_base64="7fjjx60iUWDPR/7BxvUq2wD4Rg4=">AAACL3icbVBNS8NAFNz4bf2KevSyWARFKIkK6kEQBfVYwVqhKWGz3bSLm2zYfVFL6E/y4k/Ri4KKV/+FmzaCVgcWhpk3vH0TJIJrcJwXa2R0bHxicmq6NDM7N79gLy5dapkqympUCqmuAqKZ4DGrAQfBrhLFSBQIVg+uj3O/fsOU5jK+gG7CmhFpxzzklICRfPvUiwh0gjATPXyAT7BHWxLwtxj1fBd7goWgeLsDRCl5i9eF795tCn+rizex8Lc3TNDx7bJTcfrAf4lbkDIqUPXtR68laRqxGKggWjdcJ4FmRhRwKliv5KWaJYRekzZrGBqTiOlm1j+4h9eM0sKhVObFgPvqz0RGIq27UWAm80v0sJeL/3mNFMK9ZsbjJAUW08GiMBUYJM7bwy2uGAXRNYRQxc1fMe0QRSiYjkumBHf45L+ktlXZrzjnO+XDo6KNKbSCVtE6ctEuOkRnqIpqiKJ79IRe0Zv1YD1b79bHYHTEKjLL6Beszy8hCKdO</latexit><latexit sha1_base64="7fjjx60iUWDPR/7BxvUq2wD4Rg4=">AAACL3icbVBNS8NAFNz4bf2KevSyWARFKIkK6kEQBfVYwVqhKWGz3bSLm2zYfVFL6E/y4k/Ri4KKV/+FmzaCVgcWhpk3vH0TJIJrcJwXa2R0bHxicmq6NDM7N79gLy5dapkqympUCqmuAqKZ4DGrAQfBrhLFSBQIVg+uj3O/fsOU5jK+gG7CmhFpxzzklICRfPvUiwh0gjATPXyAT7BHWxLwtxj1fBd7goWgeLsDRCl5i9eF795tCn+rizex8Lc3TNDx7bJTcfrAf4lbkDIqUPXtR68laRqxGKggWjdcJ4FmRhRwKliv5KWaJYRekzZrGBqTiOlm1j+4h9eM0sKhVObFgPvqz0RGIq27UWAm80v0sJeL/3mNFMK9ZsbjJAUW08GiMBUYJM7bwy2uGAXRNYRQxc1fMe0QRSiYjkumBHf45L+ktlXZrzjnO+XDo6KNKbSCVtE6ctEuOkRnqIpqiKJ79IRe0Zv1YD1b79bHYHTEKjLL6Beszy8hCKdO</latexit><latexit sha1_base64="7fjjx60iUWDPR/7BxvUq2wD4Rg4=">AAACL3icbVBNS8NAFNz4bf2KevSyWARFKIkK6kEQBfVYwVqhKWGz3bSLm2zYfVFL6E/y4k/Ri4KKV/+FmzaCVgcWhpk3vH0TJIJrcJwXa2R0bHxicmq6NDM7N79gLy5dapkqympUCqmuAqKZ4DGrAQfBrhLFSBQIVg+uj3O/fsOU5jK+gG7CmhFpxzzklICRfPvUiwh0gjATPXyAT7BHWxLwtxj1fBd7goWgeLsDRCl5i9eF795tCn+rizex8Lc3TNDx7bJTcfrAf4lbkDIqUPXtR68laRqxGKggWjdcJ4FmRhRwKliv5KWaJYRekzZrGBqTiOlm1j+4h9eM0sKhVObFgPvqz0RGIq27UWAm80v0sJeL/3mNFMK9ZsbjJAUW08GiMBUYJM7bwy2uGAXRNYRQxc1fMe0QRSiYjkumBHf45L+ktlXZrzjnO+XDo6KNKbSCVtE6ctEuOkRnqIpqiKJ79IRe0Zv1YD1b79bHYHTEKjLL6Beszy8hCKdO</latexit>

The Fundamental Matrix

I2I1

m2

m1

F ·m1 = 0<latexit sha1_base64="df+ueZ9iy5T6iYr61cKNZLzHLvo=">AAACAnicbVBNS8NAEN3Ur1q/ot70slgETyURQT0IRUE8VjC20ISw2WzapbvZsLsRSih48a948aDi1V/hzX/jts1BWx8MPN6bYWZelDGqtON8W5WFxaXllepqbW19Y3PL3t65VyKXmHhYMCE7EVKE0ZR4mmpGOpkkiEeMtKPB1dhvPxCpqEjv9DAjAUe9lCYUI22k0N6D19DHsdDQ50j3o6Tgo9CFFxA6oV13Gs4EcJ64JamDEq3Q/vJjgXNOUo0ZUqrrOpkOCiQ1xYyMan6uSIbwAPVI19AUcaKCYvLDCB4aJYaJkKZSDSfq74kCcaWGPDKd40PVrDcW//O6uU7OgoKmWa5JiqeLkpxBLeA4EBhTSbBmQ0MQltTcCnEfSYS1ia1mQnBnX54n3nHjvOHcntSbl2UaVbAPDsARcMEpaIIb0AIewOARPINX8GY9WS/Wu/Uxba1Y5cwu+APr8weTNpW+</latexit><latexit sha1_base64="df+ueZ9iy5T6iYr61cKNZLzHLvo=">AAACAnicbVBNS8NAEN3Ur1q/ot70slgETyURQT0IRUE8VjC20ISw2WzapbvZsLsRSih48a948aDi1V/hzX/jts1BWx8MPN6bYWZelDGqtON8W5WFxaXllepqbW19Y3PL3t65VyKXmHhYMCE7EVKE0ZR4mmpGOpkkiEeMtKPB1dhvPxCpqEjv9DAjAUe9lCYUI22k0N6D19DHsdDQ50j3o6Tgo9CFFxA6oV13Gs4EcJ64JamDEq3Q/vJjgXNOUo0ZUqrrOpkOCiQ1xYyMan6uSIbwAPVI19AUcaKCYvLDCB4aJYaJkKZSDSfq74kCcaWGPDKd40PVrDcW//O6uU7OgoKmWa5JiqeLkpxBLeA4EBhTSbBmQ0MQltTcCnEfSYS1ia1mQnBnX54n3nHjvOHcntSbl2UaVbAPDsARcMEpaIIb0AIewOARPINX8GY9WS/Wu/Uxba1Y5cwu+APr8weTNpW+</latexit><latexit sha1_base64="df+ueZ9iy5T6iYr61cKNZLzHLvo=">AAACAnicbVBNS8NAEN3Ur1q/ot70slgETyURQT0IRUE8VjC20ISw2WzapbvZsLsRSih48a948aDi1V/hzX/jts1BWx8MPN6bYWZelDGqtON8W5WFxaXllepqbW19Y3PL3t65VyKXmHhYMCE7EVKE0ZR4mmpGOpkkiEeMtKPB1dhvPxCpqEjv9DAjAUe9lCYUI22k0N6D19DHsdDQ50j3o6Tgo9CFFxA6oV13Gs4EcJ64JamDEq3Q/vJjgXNOUo0ZUqrrOpkOCiQ1xYyMan6uSIbwAPVI19AUcaKCYvLDCB4aJYaJkKZSDSfq74kCcaWGPDKd40PVrDcW//O6uU7OgoKmWa5JiqeLkpxBLeA4EBhTSbBmQ0MQltTcCnEfSYS1ia1mQnBnX54n3nHjvOHcntSbl2UaVbAPDsARcMEpaIIb0AIewOARPINX8GY9WS/Wu/Uxba1Y5cwu+APr8weTNpW+</latexit>

• Note that:

• Therefore, If we have a point m2 of I2 and we multiply it by F T, we obtain a line equation in where its match, m1, lies in I1:

The Fundamental Matrix

m>2 · F ·m1 = 0

m>1 · F> ·m2 = 0

<latexit sha1_base64="Zl/LEIAhHeXLtIBq6dAY2KMtuQ8=">AAACU3icbVHPS8MwGE3r1DmdTj16CQ7B02iHoB6EoSAeJzg3WGtJs3QLS5uSfBVG2R+pB8G/xIsHs62H/fCDkJf3vUe+vISp4Boc59uyt0rbO7vlvcr+QfXwqHZ88qplpijrUCmk6oVEM8ET1gEOgvVSxUgcCtYNxw+zfvedKc1l8gKTlPkxGSY84pSAoYLa2IsJjMIoj6dB880DmWLs0YEE/FjsSwIX32HseF5lmVs1LU4bzqZxOkGt7jSceeFN4BagjopqB7VPbyBpFrMEqCBa910nBT8nCjgVbFrxMs1SQsdkyPoGJiRm2s/noUzxhWEGOJLKrATwnF125CTWehKHRjmbU6/3ZuR/vX4G0Y2f8yTNgCV0cVGUCQwSzxLGA64YBTExgFDFzayYjogiFMw/VEwI7vqTN0Gn2bhtOM9X9dZ9kUYZnaFzdIlcdI1a6Am1UQdR9IF+LGRZ1pf1a9t2aSG1rcJzilbKrv4BS8axBg==</latexit><latexit sha1_base64="Zl/LEIAhHeXLtIBq6dAY2KMtuQ8=">AAACU3icbVHPS8MwGE3r1DmdTj16CQ7B02iHoB6EoSAeJzg3WGtJs3QLS5uSfBVG2R+pB8G/xIsHs62H/fCDkJf3vUe+vISp4Boc59uyt0rbO7vlvcr+QfXwqHZ88qplpijrUCmk6oVEM8ET1gEOgvVSxUgcCtYNxw+zfvedKc1l8gKTlPkxGSY84pSAoYLa2IsJjMIoj6dB880DmWLs0YEE/FjsSwIX32HseF5lmVs1LU4bzqZxOkGt7jSceeFN4BagjopqB7VPbyBpFrMEqCBa910nBT8nCjgVbFrxMs1SQsdkyPoGJiRm2s/noUzxhWEGOJLKrATwnF125CTWehKHRjmbU6/3ZuR/vX4G0Y2f8yTNgCV0cVGUCQwSzxLGA64YBTExgFDFzayYjogiFMw/VEwI7vqTN0Gn2bhtOM9X9dZ9kUYZnaFzdIlcdI1a6Am1UQdR9IF+LGRZ1pf1a9t2aSG1rcJzilbKrv4BS8axBg==</latexit><latexit sha1_base64="Zl/LEIAhHeXLtIBq6dAY2KMtuQ8=">AAACU3icbVHPS8MwGE3r1DmdTj16CQ7B02iHoB6EoSAeJzg3WGtJs3QLS5uSfBVG2R+pB8G/xIsHs62H/fCDkJf3vUe+vISp4Boc59uyt0rbO7vlvcr+QfXwqHZ88qplpijrUCmk6oVEM8ET1gEOgvVSxUgcCtYNxw+zfvedKc1l8gKTlPkxGSY84pSAoYLa2IsJjMIoj6dB880DmWLs0YEE/FjsSwIX32HseF5lmVs1LU4bzqZxOkGt7jSceeFN4BagjopqB7VPbyBpFrMEqCBa910nBT8nCjgVbFrxMs1SQsdkyPoGJiRm2s/noUzxhWEGOJLKrATwnF125CTWehKHRjmbU6/3ZuR/vX4G0Y2f8yTNgCV0cVGUCQwSzxLGA64YBTExgFDFzayYjogiFMw/VEwI7vqTN0Gn2bhtOM9X9dZ9kUYZnaFzdIlcdI1a6Am1UQdR9IF+LGRZ1pf1a9t2aSG1rcJzilbKrv4BS8axBg==</latexit>

l = F> ·m2 $ (l1x+ l2y + l3) = 0<latexit sha1_base64="3xH5LConlrgZZHGPqVwnrj3nEzk=">AAACNHicbVDLSsNAFJ3Ud31VXboZLEKlUJIqqAtBFMSFCwWrhaaGyWTSDk4yYeZGLaE/5cb/cKULFypu/QanNYKvAwOHc+7lzjl+IrgG2360CiOjY+MTk1PF6ZnZufnSwuKZlqmirEGlkKrpE80Ej1kDOAjWTBQjkS/YuX+5P/DPr5jSXMan0EtYOyKdmIecEjCSVzpyIwJdP8xEH+/ggwsXZIJdGkjAX07U9+rYFSwExTtdIErJa1wRnnNTFV69h6tYeOtrZtv2SmW7Zg+B/xInJ2WU49gr3buBpGnEYqCCaN1y7ATaGVHAqWD9optqlhB6STqsZWhMIqbb2TB1H68aJcChVObFgIfq942MRFr3It9MDpLo395A/M9rpRButTMeJymwmH4eClOBQeJBhTjgilEQPUMIVdz8FdMuUYSCKbpoSnB+R/5LGvXads0+2Sjv7uVtTKJltIIqyEGbaBcdomPUQBTdogf0jF6sO+vJerXePkcLVr6zhH7Aev8Aif+pjg==</latexit><latexit sha1_base64="3xH5LConlrgZZHGPqVwnrj3nEzk=">AAACNHicbVDLSsNAFJ3Ud31VXboZLEKlUJIqqAtBFMSFCwWrhaaGyWTSDk4yYeZGLaE/5cb/cKULFypu/QanNYKvAwOHc+7lzjl+IrgG2360CiOjY+MTk1PF6ZnZufnSwuKZlqmirEGlkKrpE80Ej1kDOAjWTBQjkS/YuX+5P/DPr5jSXMan0EtYOyKdmIecEjCSVzpyIwJdP8xEH+/ggwsXZIJdGkjAX07U9+rYFSwExTtdIErJa1wRnnNTFV69h6tYeOtrZtv2SmW7Zg+B/xInJ2WU49gr3buBpGnEYqCCaN1y7ATaGVHAqWD9optqlhB6STqsZWhMIqbb2TB1H68aJcChVObFgIfq942MRFr3It9MDpLo395A/M9rpRButTMeJymwmH4eClOBQeJBhTjgilEQPUMIVdz8FdMuUYSCKbpoSnB+R/5LGvXads0+2Sjv7uVtTKJltIIqyEGbaBcdomPUQBTdogf0jF6sO+vJerXePkcLVr6zhH7Aev8Aif+pjg==</latexit><latexit sha1_base64="3xH5LConlrgZZHGPqVwnrj3nEzk=">AAACNHicbVDLSsNAFJ3Ud31VXboZLEKlUJIqqAtBFMSFCwWrhaaGyWTSDk4yYeZGLaE/5cb/cKULFypu/QanNYKvAwOHc+7lzjl+IrgG2360CiOjY+MTk1PF6ZnZufnSwuKZlqmirEGlkKrpE80Ej1kDOAjWTBQjkS/YuX+5P/DPr5jSXMan0EtYOyKdmIecEjCSVzpyIwJdP8xEH+/ggwsXZIJdGkjAX07U9+rYFSwExTtdIErJa1wRnnNTFV69h6tYeOtrZtv2SmW7Zg+B/xInJ2WU49gr3buBpGnEYqCCaN1y7ATaGVHAqWD9optqlhB6STqsZWhMIqbb2TB1H68aJcChVObFgIfq942MRFr3It9MDpLo395A/M9rpRButTMeJymwmH4eClOBQeJBhTjgilEQPUMIVdz8FdMuUYSCKbpoSnB+R/5LGvXads0+2Sjv7uVtTKJltIIqyEGbaBcdomPUQBTdogf0jF6sO+vJerXePkcLVr6zhH7Aev8Aif+pjg==</latexit>

The Fundamental Matrix

I2I1

m2

m1

F> ·m2 = 0<latexit sha1_base64="91/9qS3UFAB1EJUhF7qN1KD5cJ8=">AAACBXicbVBNS8NAEN3Ur1q/oh5FWCyCp5IWQT0IRUE8VrC20MSw2W7apZts2J0IJfTkxb/ixYOKV/+DN/+Nm7YHbX0w8Hhvhpl5QSK4Bsf5tgoLi0vLK8XV0tr6xuaWvb1zp2WqKGtSKaRqB0QzwWPWBA6CtRPFSBQI1goGl7nfemBKcxnfwjBhXkR6MQ85JWAk396/undBJtilXQnYjQj0gzCLRn4Nn2PHt8tOxRkDz5PqlJTRFA3f/nK7kqYRi4EKonWn6iTgZUQBp4KNSm6qWULogPRYx9CYREx72fiNET40SheHUpmKAY/V3xMZibQeRoHpzO/Us14u/ud1UghPvYzHSQosppNFYSowSJxngrtcMQpiaAihiptbMe0TRSiY5EomhOrsy/OkWaucVZyb43L9YppGEe2hA3SEqugE1dE1aqAmougRPaNX9GY9WS/Wu/UxaS1Y05ld9AfW5w/m05eq</latexit><latexit sha1_base64="91/9qS3UFAB1EJUhF7qN1KD5cJ8=">AAACBXicbVBNS8NAEN3Ur1q/oh5FWCyCp5IWQT0IRUE8VrC20MSw2W7apZts2J0IJfTkxb/ixYOKV/+DN/+Nm7YHbX0w8Hhvhpl5QSK4Bsf5tgoLi0vLK8XV0tr6xuaWvb1zp2WqKGtSKaRqB0QzwWPWBA6CtRPFSBQI1goGl7nfemBKcxnfwjBhXkR6MQ85JWAk396/undBJtilXQnYjQj0gzCLRn4Nn2PHt8tOxRkDz5PqlJTRFA3f/nK7kqYRi4EKonWn6iTgZUQBp4KNSm6qWULogPRYx9CYREx72fiNET40SheHUpmKAY/V3xMZibQeRoHpzO/Us14u/ud1UghPvYzHSQosppNFYSowSJxngrtcMQpiaAihiptbMe0TRSiY5EomhOrsy/OkWaucVZyb43L9YppGEe2hA3SEqugE1dE1aqAmougRPaNX9GY9WS/Wu/UxaS1Y05ld9AfW5w/m05eq</latexit><latexit sha1_base64="91/9qS3UFAB1EJUhF7qN1KD5cJ8=">AAACBXicbVBNS8NAEN3Ur1q/oh5FWCyCp5IWQT0IRUE8VrC20MSw2W7apZts2J0IJfTkxb/ixYOKV/+DN/+Nm7YHbX0w8Hhvhpl5QSK4Bsf5tgoLi0vLK8XV0tr6xuaWvb1zp2WqKGtSKaRqB0QzwWPWBA6CtRPFSBQI1goGl7nfemBKcxnfwjBhXkR6MQ85JWAk396/undBJtilXQnYjQj0gzCLRn4Nn2PHt8tOxRkDz5PqlJTRFA3f/nK7kqYRi4EKonWn6iTgZUQBp4KNSm6qWULogPRYx9CYREx72fiNET40SheHUpmKAY/V3xMZibQeRoHpzO/Us14u/ud1UghPvYzHSQosppNFYSowSJxngrtcMQpiaAihiptbMe0TRSiY5EomhOrsy/OkWaucVZyb43L9YppGEe2hA3SEqugE1dE1aqAmougRPaNX9GY9WS/Wu/UxaS1Y05ld9AfW5w/m05eq</latexit>

The Essential Matrix: 8-points algorithm

• From:

• We can define a linear system as:

• Given enough matches we can solve the system using the SVD. How many do we need? 8 is the minimum, as usual the more the better!

• This method is called 8-points algorithm, and it can be applied for computing F as well (we use pixel coordinates instead of normalized coordinates).

A · b = 0

A =

2

664

(p11)

> ⌦ (p12)

>

(p21)

> ⌦ (p22)

>

. . .(pn

1 )> ⌦ (pn

2 )>

3

775 b = vec(E)

m>2 · E · m1 = 0

A =

2

64(m1

1)> ⌦ (m1

2)>

...(mn

1 )> ⌦ (mn

2 )>

3

75 b = vec(E)

The Essential Matrix• The Kronecker product is defined as

• where A is m×n matrix, and B is a r×s matrix.

A⌦B =

2

6664

a1,1 ·B . . . a1,n ·Ba2,1 ·B . . . a2,n ·B

... . . ....

am,1 ·B . . . am,n ·B

3

7775

The Essential Matrix: Practice

• Typically, we compute E indirectly by computing F and then converting it into E :

• Why?

• The main reason is that we avoid to multiply K times all coordinates of points:

• computational efficiency.

• reducing accumulation of numerical errors.

E = K>2 · F ·K1

<latexit sha1_base64="+NNPcjatjbkg5pujiKL5LtdcjHU=">AAACBXicbVBNSwMxEM36WevXqkcRgkXwVHaLoB6EoihCLxWsLXTrkk3TNjS7WZJZoSw9efGvePGg4tX/4M1/Y9ruQVsfDPN4b4ZkXhALrsFxvq25+YXFpeXcSn51bX1j097avtMyUZTVqBRSNQKimeARqwEHwRqxYiQMBKsH/YuRX39gSnMZ3cIgZq2QdCPe4ZSAkXx77xKf4YpfuvdAxtijbQn4atIqvuvbBafojIFniZuRAspQ9e0vry1pErIIqCBaN10nhlZKFHAq2DDvJZrFhPZJlzUNjUjIdCsdnzHEB0Zp445UpiLAY/X3RkpCrQdhYCZDAj097Y3E/7xmAp2TVsqjOAEW0clDnURgkHiUCW5zxSiIgSGEKm7+immPKELBJJc3IbjTJ8+SWql4WnRujgrl8yyNHNpF++gQuegYldE1qqIaougRPaNX9GY9WS/Wu/UxGZ2zsp0d9AfW5w+Qi5bY</latexit><latexit sha1_base64="+NNPcjatjbkg5pujiKL5LtdcjHU=">AAACBXicbVBNSwMxEM36WevXqkcRgkXwVHaLoB6EoihCLxWsLXTrkk3TNjS7WZJZoSw9efGvePGg4tX/4M1/Y9ruQVsfDPN4b4ZkXhALrsFxvq25+YXFpeXcSn51bX1j097avtMyUZTVqBRSNQKimeARqwEHwRqxYiQMBKsH/YuRX39gSnMZ3cIgZq2QdCPe4ZSAkXx77xKf4YpfuvdAxtijbQn4atIqvuvbBafojIFniZuRAspQ9e0vry1pErIIqCBaN10nhlZKFHAq2DDvJZrFhPZJlzUNjUjIdCsdnzHEB0Zp445UpiLAY/X3RkpCrQdhYCZDAj097Y3E/7xmAp2TVsqjOAEW0clDnURgkHiUCW5zxSiIgSGEKm7+immPKELBJJc3IbjTJ8+SWql4WnRujgrl8yyNHNpF++gQuegYldE1qqIaougRPaNX9GY9WS/Wu/UxGZ2zsp0d9AfW5w+Qi5bY</latexit><latexit sha1_base64="+NNPcjatjbkg5pujiKL5LtdcjHU=">AAACBXicbVBNSwMxEM36WevXqkcRgkXwVHaLoB6EoihCLxWsLXTrkk3TNjS7WZJZoSw9efGvePGg4tX/4M1/Y9ruQVsfDPN4b4ZkXhALrsFxvq25+YXFpeXcSn51bX1j097avtMyUZTVqBRSNQKimeARqwEHwRqxYiQMBKsH/YuRX39gSnMZ3cIgZq2QdCPe4ZSAkXx77xKf4YpfuvdAxtijbQn4atIqvuvbBafojIFniZuRAspQ9e0vry1pErIIqCBaN10nhlZKFHAq2DDvJZrFhPZJlzUNjUjIdCsdnzHEB0Zp445UpiLAY/X3RkpCrQdhYCZDAj097Y3E/7xmAp2TVsqjOAEW0clDnURgkHiUCW5zxSiIgSGEKm7+immPKELBJJc3IbjTJ8+SWql4WnRujgrl8yyNHNpF++gQuegYldE1qqIaougRPaNX9GY9WS/Wu/UxGZ2zsp0d9AfW5w+Qi5bY</latexit>

The Essential Matrix: Practice

• If we solve the linear system using pixel coordinates, we may have numerical instabilities because pixel coordinates vary in [1, w] and in [1, h] (where h and w are, respectively, the height and width of the image).

• For reducing instabilities, we need to bound values in the range .

• Given the input n points mi, we compute:

u =1

n

nX

i=1

ui v =1

n

nX

i=1

vi mi =

2

4ui

vi1

3

5

s =1

np2

nX

i=1

p(vi � u)2 + (ui � v)2

[�p2,p2]

<latexit sha1_base64="tVqBtoomFSPBWvMEUqf4amPNCjM=">AAAB/XicbZDNSsNAFIUn9a/Wv6i4cjNYBBdakiKou6IblxWMLaShTKbTduhkEmduhBIKvoobFypufQ93vo3TNoK2Hhj4OPde7p0TJoJrcJwvq7CwuLS8Ulwtra1vbG7Z2zt3Ok4VZR6NRayaIdFMcMk84CBYM1GMRKFgjXBwNa43HpjSPJa3MExYEJGe5F1OCRirbe/5Jy19ryCrjo7xDwVtu+xUnInwPLg5lFGuetv+bHVimkZMAhVEa991EggyooBTwUalVqpZQuiA9JhvUJKI6SCbnD/Ch8bp4G6szJOAJ+7viYxEWg+j0HRGBPp6tjY2/6v5KXTPg4zLJAUm6XRRNxUYYjzOAne4YhTE0AChiptbMe0TRSiYxEomBHf2y/PgVSsXFefmtFy7zNMoon10gI6Qi85QDV2jOvIQRRl6Qi/o1Xq0nq03633aWrDymV30R9bHNxi6lSE=</latexit><latexit sha1_base64="tVqBtoomFSPBWvMEUqf4amPNCjM=">AAAB/XicbZDNSsNAFIUn9a/Wv6i4cjNYBBdakiKou6IblxWMLaShTKbTduhkEmduhBIKvoobFypufQ93vo3TNoK2Hhj4OPde7p0TJoJrcJwvq7CwuLS8Ulwtra1vbG7Z2zt3Ok4VZR6NRayaIdFMcMk84CBYM1GMRKFgjXBwNa43HpjSPJa3MExYEJGe5F1OCRirbe/5Jy19ryCrjo7xDwVtu+xUnInwPLg5lFGuetv+bHVimkZMAhVEa991EggyooBTwUalVqpZQuiA9JhvUJKI6SCbnD/Ch8bp4G6szJOAJ+7viYxEWg+j0HRGBPp6tjY2/6v5KXTPg4zLJAUm6XRRNxUYYjzOAne4YhTE0AChiptbMe0TRSiYxEomBHf2y/PgVSsXFefmtFy7zNMoon10gI6Qi85QDV2jOvIQRRl6Qi/o1Xq0nq03633aWrDymV30R9bHNxi6lSE=</latexit><latexit sha1_base64="tVqBtoomFSPBWvMEUqf4amPNCjM=">AAAB/XicbZDNSsNAFIUn9a/Wv6i4cjNYBBdakiKou6IblxWMLaShTKbTduhkEmduhBIKvoobFypufQ93vo3TNoK2Hhj4OPde7p0TJoJrcJwvq7CwuLS8Ulwtra1vbG7Z2zt3Ok4VZR6NRayaIdFMcMk84CBYM1GMRKFgjXBwNa43HpjSPJa3MExYEJGe5F1OCRirbe/5Jy19ryCrjo7xDwVtu+xUnInwPLg5lFGuetv+bHVimkZMAhVEa991EggyooBTwUalVqpZQuiA9JhvUJKI6SCbnD/Ch8bp4G6szJOAJ+7viYxEWg+j0HRGBPp6tjY2/6v5KXTPg4zLJAUm6XRRNxUYYjzOAne4YhTE0AChiptbMe0TRSiYxEomBHf2y/PgVSsXFefmtFy7zNMoon10gI6Qi85QDV2jOvIQRRl6Qi/o1Xq0nq03633aWrDymV30R9bHNxi6lSE=</latexit>

The Essential Matrix: Practice

• Finally, we shift and scale all n points using the following:

• We can now solve the linear system without numerical instabilities (or reducing them).

• Note that this operation, shift and scale, needs to be done for estimating a homography as well.

ui =ui � u

s

vi =vi � v

s

Non-Linear Optimization• As seen before, we need to refine E using a geometric

error, note that we compute E indirectly so we minimize F:

• where d𝜋 is the distance point-line ( is a line), and n is the number of matched points.

• Again we can solve it with the Nelder-Mead method (fminsearch in MATLAB).

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

Non-Linear Optimization• As seen before, we need to refine E using a geometric

error, note that we compute E indirectly so we minimize F:

• where d𝜋 is the distance point-line ( is a line), and n is the number of matched points.

• Again we can solve it with the Nelder-Mead method (fminsearch in MATLAB).

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

This is a lineargmin

E

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

Non-Linear Optimization• As seen before, we need to refine E using a geometric

error, note that we compute E indirectly so we minimize F:

• where d𝜋 is the distance point-line ( is a line), and n is the number of matched points.

• Again we can solve it with the Nelder-Mead method (fminsearch in MATLAB).

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

This is a line This is a lineargmin

E

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

Non-Linear Optimization: First Term of the Energy

I1

mi1

i

I2

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

F ·m1 = 0<latexit sha1_base64="df+ueZ9iy5T6iYr61cKNZLzHLvo=">AAACAnicbVBNS8NAEN3Ur1q/ot70slgETyURQT0IRUE8VjC20ISw2WzapbvZsLsRSih48a948aDi1V/hzX/jts1BWx8MPN6bYWZelDGqtON8W5WFxaXllepqbW19Y3PL3t65VyKXmHhYMCE7EVKE0ZR4mmpGOpkkiEeMtKPB1dhvPxCpqEjv9DAjAUe9lCYUI22k0N6D19DHsdDQ50j3o6Tgo9CFFxA6oV13Gs4EcJ64JamDEq3Q/vJjgXNOUo0ZUqrrOpkOCiQ1xYyMan6uSIbwAPVI19AUcaKCYvLDCB4aJYaJkKZSDSfq74kCcaWGPDKd40PVrDcW//O6uU7OgoKmWa5JiqeLkpxBLeA4EBhTSbBmQ0MQltTcCnEfSYS1ia1mQnBnX54n3nHjvOHcntSbl2UaVbAPDsARcMEpaIIb0AIewOARPINX8GY9WS/Wu/Uxba1Y5cwu+APr8weTNpW+</latexit><latexit sha1_base64="df+ueZ9iy5T6iYr61cKNZLzHLvo=">AAACAnicbVBNS8NAEN3Ur1q/ot70slgETyURQT0IRUE8VjC20ISw2WzapbvZsLsRSih48a948aDi1V/hzX/jts1BWx8MPN6bYWZelDGqtON8W5WFxaXllepqbW19Y3PL3t65VyKXmHhYMCE7EVKE0ZR4mmpGOpkkiEeMtKPB1dhvPxCpqEjv9DAjAUe9lCYUI22k0N6D19DHsdDQ50j3o6Tgo9CFFxA6oV13Gs4EcJ64JamDEq3Q/vJjgXNOUo0ZUqrrOpkOCiQ1xYyMan6uSIbwAPVI19AUcaKCYvLDCB4aJYaJkKZSDSfq74kCcaWGPDKd40PVrDcW//O6uU7OgoKmWa5JiqeLkpxBLeA4EBhTSbBmQ0MQltTcCnEfSYS1ia1mQnBnX54n3nHjvOHcntSbl2UaVbAPDsARcMEpaIIb0AIewOARPINX8GY9WS/Wu/Uxba1Y5cwu+APr8weTNpW+</latexit><latexit sha1_base64="df+ueZ9iy5T6iYr61cKNZLzHLvo=">AAACAnicbVBNS8NAEN3Ur1q/ot70slgETyURQT0IRUE8VjC20ISw2WzapbvZsLsRSih48a948aDi1V/hzX/jts1BWx8MPN6bYWZelDGqtON8W5WFxaXllepqbW19Y3PL3t65VyKXmHhYMCE7EVKE0ZR4mmpGOpkkiEeMtKPB1dhvPxCpqEjv9DAjAUe9lCYUI22k0N6D19DHsdDQ50j3o6Tgo9CFFxA6oV13Gs4EcJ64JamDEq3Q/vJjgXNOUo0ZUqrrOpkOCiQ1xYyMan6uSIbwAPVI19AUcaKCYvLDCB4aJYaJkKZSDSfq74kCcaWGPDKd40PVrDcW//O6uU7OgoKmWa5JiqeLkpxBLeA4EBhTSbBmQ0MQltTcCnEfSYS1ia1mQnBnX54n3nHjvOHcntSbl2UaVbAPDsARcMEpaIIb0AIewOARPINX8GY9WS/Wu/Uxba1Y5cwu+APr8weTNpW+</latexit>

mi2

Non-Linear Optimization: Second Term of the Energy

argminE

nX

i=1

d⇡(F ·mi1,m

i2)

2 + d⇡(F> ·mi

2,mi1)

2

I1

mi2

mi1

F> ·m2 = 0<latexit sha1_base64="91/9qS3UFAB1EJUhF7qN1KD5cJ8=">AAACBXicbVBNS8NAEN3Ur1q/oh5FWCyCp5IWQT0IRUE8VrC20MSw2W7apZts2J0IJfTkxb/ixYOKV/+DN/+Nm7YHbX0w8Hhvhpl5QSK4Bsf5tgoLi0vLK8XV0tr6xuaWvb1zp2WqKGtSKaRqB0QzwWPWBA6CtRPFSBQI1goGl7nfemBKcxnfwjBhXkR6MQ85JWAk396/undBJtilXQnYjQj0gzCLRn4Nn2PHt8tOxRkDz5PqlJTRFA3f/nK7kqYRi4EKonWn6iTgZUQBp4KNSm6qWULogPRYx9CYREx72fiNET40SheHUpmKAY/V3xMZibQeRoHpzO/Us14u/ud1UghPvYzHSQosppNFYSowSJxngrtcMQpiaAihiptbMe0TRSiY5EomhOrsy/OkWaucVZyb43L9YppGEe2hA3SEqugE1dE1aqAmougRPaNX9GY9WS/Wu/UxaS1Y05ld9AfW5w/m05eq</latexit><latexit sha1_base64="91/9qS3UFAB1EJUhF7qN1KD5cJ8=">AAACBXicbVBNS8NAEN3Ur1q/oh5FWCyCp5IWQT0IRUE8VrC20MSw2W7apZts2J0IJfTkxb/ixYOKV/+DN/+Nm7YHbX0w8Hhvhpl5QSK4Bsf5tgoLi0vLK8XV0tr6xuaWvb1zp2WqKGtSKaRqB0QzwWPWBA6CtRPFSBQI1goGl7nfemBKcxnfwjBhXkR6MQ85JWAk396/undBJtilXQnYjQj0gzCLRn4Nn2PHt8tOxRkDz5PqlJTRFA3f/nK7kqYRi4EKonWn6iTgZUQBp4KNSm6qWULogPRYx9CYREx72fiNET40SheHUpmKAY/V3xMZibQeRoHpzO/Us14u/ud1UghPvYzHSQosppNFYSowSJxngrtcMQpiaAihiptbMe0TRSiY5EomhOrsy/OkWaucVZyb43L9YppGEe2hA3SEqugE1dE1aqAmougRPaNX9GY9WS/Wu/UxaS1Y05ld9AfW5w/m05eq</latexit><latexit sha1_base64="91/9qS3UFAB1EJUhF7qN1KD5cJ8=">AAACBXicbVBNS8NAEN3Ur1q/oh5FWCyCp5IWQT0IRUE8VrC20MSw2W7apZts2J0IJfTkxb/ixYOKV/+DN/+Nm7YHbX0w8Hhvhpl5QSK4Bsf5tgoLi0vLK8XV0tr6xuaWvb1zp2WqKGtSKaRqB0QzwWPWBA6CtRPFSBQI1goGl7nfemBKcxnfwjBhXkR6MQ85JWAk396/undBJtilXQnYjQj0gzCLRn4Nn2PHt8tOxRkDz5PqlJTRFA3f/nK7kqYRi4EKonWn6iTgZUQBp4KNSm6qWULogPRYx9CYREx72fiNET40SheHUpmKAY/V3xMZibQeRoHpzO/Us14u/ud1UghPvYzHSQosppNFYSowSJxngrtcMQpiaAihiptbMe0TRSiY5EomhOrsy/OkWaucVZyb43L9YppGEe2hA3SEqugE1dE1aqAmougRPaNX9GY9WS/Wu/UxaS1Y05ld9AfW5w/m05eq</latexit>

i

I2

Now we have E, and so what?

E Factorization• Once we have estimated E, we would like estimate

R and t to get the pose of the camera:

• As you may notice we have:

• [t]× = S is an anti-symmetric matrix.

• R is orthogonal matrix.

E = [t]⇥ ·R

E Factorization• Given a m×n matrix A, its SVD decomposition is defined

as:

• where:

• U is an m×m orthogonal matrix.

• Σ is a diagonal m×n matrix.

• V* is the conjugate transpose of an orthogonal matrix.

SVD(A) = U · ⌃ · V ⇤

E Factorization• Theorem: A 3×3 matrix is an essential matrix if and only

if two singular values are equal and the third is zero.

• This means that:

• Note that:

SVD(E) = U · diag(1, 1, 0) · V >

diag(1, 1, 0) = W · Z

W =

2

40 �1 01 0 00 0 1

3

5 Z =

2

40 1 0�1 0 00 0 0

3

5

E Factorization

• Lemma: Given R a rotation matrix, and U and V two orthogonal matrices, we have that:

• Note that is still a rotation matrix!

R0 = det(U · V >) · U ·R ·R>

R0 = det(U · V >) · U ·R ·R>

E Factorization

• Given that:

• We can have four possible factorizations of E such that E = S . R:

SVD(E) = U · diag(1, 1, 0) · V >

S = U · (±Z) · U>

R = U ·W · V > or R = U ·W> · V >

S = U · (±Z) · U>

R = U ·W · V > or R = U ·W> · V >

E Factorization: The Four Cases

C1 C2 C1 C2

E Factorization: The Four Cases

C1 C2 C1 C2

Both cameras see the point. It is

in front of them

E Factorization: The Four Cases

C1 C2 C1 C2

Both cameras don’t see the point. It is behind them

Both cameras see the point. It is

in front of them

E Factorization: The Four Cases

C1

C2 C1

C2

E Factorization: The Four Cases

C1

C2 C1

C2

Left camera doesn’t see the point.

It is behind them

E Factorization: The Four Cases

C1

C2 C1

C2

Left camera doesn’t see the point.

It is behind them

Left camera doesn’t see the point.

It is behind them

Which is the correct configuration?

C1 C2

Why?

The red point is seen by both cameras!

How do we find it?

We need to compute all cases. Then, we determine the case in

which the majority of 3D points are in front of both cameras!

Triangulation

Triangulation

• Input: n matched 2D feature points in the two images and their P matrices (P1, and P2).

• Output: n 3D points.

Triangulation: Pure Translational Motion Case

X

Z M

m1 m2

f

b

M =

2

664

xyz1

3

775

mi =

2

4ui

vi1

3

5

Triangulation

• We first fix the frame of reference to one of the two cameras. Then, we know that:

• So, we can obtain:

(fz = �u1

xfz = � u2

x�b

z =b · f

u2 � u1

Triangulation: Pure Rotational Motion Case

X

Z M

m1 m2

M =

2

664

xyz1

3

775

mi =

2

4ui

vi1

3

5

There is no displacement —> The same lines for intersection —> no 3D

Triangulation: The General Case

M

m1

m2

C1

C2

Camera 1

Camera 2

Similar to DLT but different!

Triangulation: Eigen Method

P =

2

4p>1

p>2

p>3

3

5

8<

:u = p>

1 ·Mp>

3 ·M

v = p>2 ·M

p>3 ·M

Triangulation: Eigen Method

P =

2

4p>1

p>2

p>3

3

5

8<

:u = p>

1 ·Mp>

3 ·M

v = p>2 ·M

p>3 ·M

known!

Triangulation: Eigen Method

P =

2

4p>1

p>2

p>3

3

5

8<

:u = p>

1 ·Mp>

3 ·M

v = p>2 ·M

p>3 ·M

known! unknown!

• This leads to:

• Given that:

Triangulation: Eigen Method

((p1 � u · p1)> ·M = 0

(p2 � v · p1)> ·M = 0

Pi =

2

4(pi

1)>

(pi2)

>

(pi3)

>

3

5 mi =

2

4ui

vi1

3

5

Triangulation: Eigen Method• We obtain:

• For l cameras, this leads to:

2

664

(p11 � u1 · p1

3)>

(p12 � v1 · p1

3)>

(p21 � u2 · p2

3)>

(p22 � v2 · p2

3)>

3

775 ·M = 0

2

666664

(p11 � u1 · p1

3)>

(p12 � u1 · p1

3)>

...(pl

1 � u1 · pl3)

>

(pl2 � u1 · pl

3)>

3

777775·M = 0

Triangulation: Eigen Method• Again, we solve this linear system using SVD; i.e., the kernel of V:

• Again, we minimized an algebraic error without a geometric meaning!

• Again, we use this initial solution for a non-linear method that minimizes a geometric error. Here for each 3D points M, we minimize:

• where l is the number of cameras. [uj, vj, 1] is corner in the l-th camera (i.e., it is “the photograph” of M).

argminM

lX

j=1

✓uj �

(pj1)

> ·M(pj

3)> ·M

◆2

+

✓vj �

(pj2)

> ·M(pj

3)> ·M

◆2

<latexit sha1_base64="tgbyoZl2uYjfu4GCSF3+3G0c254=">AAAC5HicrVJNaxsxENVuv1L3y22OvYiYgkOp2XULbQ+FkFxyCaQQNwHLXrSydiNHH4s0GzDL3nvJIS255kf11v+SQ7W2W1KnuXVA8HjzNG80o7SQwkEU/QzCO3fv3X+w9rD16PGTp8/az198caa0jA+YkcYepdRxKTQfgADJjwrLqUolP0xPdpr84Sm3Thh9ALOCjxTNtcgEo+CppH1FqM2JEjqpiKJwnGbVXl0TV6qkmn6K67HELZKKPJfdVplM8RtMMktZ1f2tLurxNIk3xwRMQdjEAL5WZ0X29hbZwsFujvuv/5id3mrW/19mraTdiXrRPPBNEC9BBy1jP2n/IBPDSsU1MEmdG8ZRAaOKWhBMcl+6dLyg7ITmfOihpoq7UTXfUo1feWaCM2P90YDn7PUbFVXOzVTqlU23bjXXkP/KDUvIPowqoYsSuGYLo6yUGAxuVo4nwnIGcuYBZVb4XjE7pn6w4D9GM4R49ck3waDf+9iLPr/rbG0vp7GGXqIN1EUxeo+20C7aRwPEgjT4GpwH38I8PAu/hxcLaRgs76yjvyK8/AVs9epM</latexit><latexit sha1_base64="tgbyoZl2uYjfu4GCSF3+3G0c254=">AAAC5HicrVJNaxsxENVuv1L3y22OvYiYgkOp2XULbQ+FkFxyCaQQNwHLXrSydiNHH4s0GzDL3nvJIS255kf11v+SQ7W2W1KnuXVA8HjzNG80o7SQwkEU/QzCO3fv3X+w9rD16PGTp8/az198caa0jA+YkcYepdRxKTQfgADJjwrLqUolP0xPdpr84Sm3Thh9ALOCjxTNtcgEo+CppH1FqM2JEjqpiKJwnGbVXl0TV6qkmn6K67HELZKKPJfdVplM8RtMMktZ1f2tLurxNIk3xwRMQdjEAL5WZ0X29hbZwsFujvuv/5id3mrW/19mraTdiXrRPPBNEC9BBy1jP2n/IBPDSsU1MEmdG8ZRAaOKWhBMcl+6dLyg7ITmfOihpoq7UTXfUo1feWaCM2P90YDn7PUbFVXOzVTqlU23bjXXkP/KDUvIPowqoYsSuGYLo6yUGAxuVo4nwnIGcuYBZVb4XjE7pn6w4D9GM4R49ck3waDf+9iLPr/rbG0vp7GGXqIN1EUxeo+20C7aRwPEgjT4GpwH38I8PAu/hxcLaRgs76yjvyK8/AVs9epM</latexit><latexit sha1_base64="tgbyoZl2uYjfu4GCSF3+3G0c254=">AAAC5HicrVJNaxsxENVuv1L3y22OvYiYgkOp2XULbQ+FkFxyCaQQNwHLXrSydiNHH4s0GzDL3nvJIS255kf11v+SQ7W2W1KnuXVA8HjzNG80o7SQwkEU/QzCO3fv3X+w9rD16PGTp8/az198caa0jA+YkcYepdRxKTQfgADJjwrLqUolP0xPdpr84Sm3Thh9ALOCjxTNtcgEo+CppH1FqM2JEjqpiKJwnGbVXl0TV6qkmn6K67HELZKKPJfdVplM8RtMMktZ1f2tLurxNIk3xwRMQdjEAL5WZ0X29hbZwsFujvuv/5id3mrW/19mraTdiXrRPPBNEC9BBy1jP2n/IBPDSsU1MEmdG8ZRAaOKWhBMcl+6dLyg7ITmfOihpoq7UTXfUo1feWaCM2P90YDn7PUbFVXOzVTqlU23bjXXkP/KDUvIPowqoYsSuGYLo6yUGAxuVo4nwnIGcuYBZVb4XjE7pn6w4D9GM4R49ck3waDf+9iLPr/rbG0vp7GGXqIN1EUxeo+20C7aRwPEgjT4GpwH38I8PAu/hxcLaRgs76yjvyK8/AVs9epM</latexit>

Triangulation: Eigen Method Example

C1

C2

Camera 1

Camera 2

Projected M(triangulated)Corner mi

M (triangulated)

Triangulation: Eigen Method Example

C1

C2

Camera 1

Camera 2

M (real)

Projected M(triangulated)Corner mi

M (triangulated)

Triangulation: Eigen Method Example

C1

C2

Camera 1

Camera 2

NOTE: with the Geometric minimization, we force the green point to be the same as the red one.

M (real)

Projected M(triangulated)Corner mi

M (triangulated)

Structure from

Motion

Structure from Motion

• Input: n matched points (corners computed with the Harris algorithm) between two images, and K1 and K2 for the two cameras.

• Output: n 3D points, and G1 and G2 (i.e., for each camera).

Structure from Motion

• The algorithm is:

• Estimation of E.

• Factorization of E to obtain G.

• Triangulation of the n matched points using P1 and P2.

So far we have only used only a two cameras!

• We compute G for different views using the previous algorithm.

• We use a reference view for computing the different G matrices. For example, we can use the first image.

Structure from Motion: Multi-View

P1

P2P3

Structure from Motion: Multi-View Example

P1

P2 ReferenceP3

Structure from Motion: Multi-View Example

Structure from Motion: Multi-View Example

P1

P2P3 Reference

We compute G12

P1

P2P3 Reference

We compute G13

Structure from Motion: Multi-View Example

Hang on, was it a good reference the one before?

Hang on, what can possibly go wrong?

We are accumulating error, and we will drift

from the solution!

Structure from Motion: Bundle Adjustment

• To avoid error accumulation, we minimize in a non-linear way at the same time both poses estimation and 3D points generation:

• where l is the number of cameras, and n is the number of points.

P = K · [R|t]<latexit sha1_base64="OCW9Ob3Efg7x1KXDx4xLktLUPUo=">AAACAXicbVBNS8NAEN3Ur1q/qp7Ey2IRPJVUBPUgFL0IXqpYW0hC2Ww27dLNJuxOhBKLF/+KFw8qXv0X3vw3btoetPXBwOO9GWbm+YngGmz72yrMzS8sLhWXSyura+sb5c2tOx2nirImjUWs2j7RTHDJmsBBsHaiGIl8wVp+/yL3W/dMaR7LWxgkzItIV/KQUwJG6pR3GvgMX2GXBjE4Nw9uRKDnhxkMvU65YlftEfAsqU1IBU3Q6JS/3CCmacQkUEG0dmp2Al5GFHAq2LDkppolhPZJlzmGShIx7WWjF4Z43ygBDmNlSgIeqb8nMhJpPYh805mfqKe9XPzPc1IIT7yMyyQFJul4UZgKDDHO88ABV4yCGBhCqOLmVkx7RBEKJrWSCaE2/fIsaR5WT6v29VGlfj5Jo4h20R46QDV0jOroEjVQE1H0iJ7RK3qznqwX6936GLcWrMnMNvoD6/MHhuyWdg==</latexit><latexit sha1_base64="OCW9Ob3Efg7x1KXDx4xLktLUPUo=">AAACAXicbVBNS8NAEN3Ur1q/qp7Ey2IRPJVUBPUgFL0IXqpYW0hC2Ww27dLNJuxOhBKLF/+KFw8qXv0X3vw3btoetPXBwOO9GWbm+YngGmz72yrMzS8sLhWXSyura+sb5c2tOx2nirImjUWs2j7RTHDJmsBBsHaiGIl8wVp+/yL3W/dMaR7LWxgkzItIV/KQUwJG6pR3GvgMX2GXBjE4Nw9uRKDnhxkMvU65YlftEfAsqU1IBU3Q6JS/3CCmacQkUEG0dmp2Al5GFHAq2LDkppolhPZJlzmGShIx7WWjF4Z43ygBDmNlSgIeqb8nMhJpPYh805mfqKe9XPzPc1IIT7yMyyQFJul4UZgKDDHO88ABV4yCGBhCqOLmVkx7RBEKJrWSCaE2/fIsaR5WT6v29VGlfj5Jo4h20R46QDV0jOroEjVQE1H0iJ7RK3qznqwX6936GLcWrMnMNvoD6/MHhuyWdg==</latexit><latexit sha1_base64="OCW9Ob3Efg7x1KXDx4xLktLUPUo=">AAACAXicbVBNS8NAEN3Ur1q/qp7Ey2IRPJVUBPUgFL0IXqpYW0hC2Ww27dLNJuxOhBKLF/+KFw8qXv0X3vw3btoetPXBwOO9GWbm+YngGmz72yrMzS8sLhWXSyura+sb5c2tOx2nirImjUWs2j7RTHDJmsBBsHaiGIl8wVp+/yL3W/dMaR7LWxgkzItIV/KQUwJG6pR3GvgMX2GXBjE4Nw9uRKDnhxkMvU65YlftEfAsqU1IBU3Q6JS/3CCmacQkUEG0dmp2Al5GFHAq2LDkppolhPZJlzmGShIx7WWjF4Z43ygBDmNlSgIeqb8nMhJpPYh805mfqKe9XPzPc1IIT7yMyyQFJul4UZgKDDHO88ABV4yCGBhCqOLmVkx7RBEKJrWSCaE2/fIsaR5WT6v29VGlfj5Jo4h20R46QDV0jOroEjVQE1H0iJ7RK3qznqwX6936GLcWrMnMNvoD6/MHhuyWdg==</latexit>

arg minRi,ti,Mi

lX

j=1

nX

i=1

✓uij �

(pj1)

> ·Mi

(pj3)

> ·Mi

◆2

+

✓vij �

(pj2)

> ·Mi

(pj3)

> ·Mi

◆2

<latexit sha1_base64="iv8ND/8gWYWdlskkT7a0HK7d4R0=">AAADFXicrVLLbtNAFB2bVwmvtF2yGREhpaJEdkBqu0CqYMMGqSBCK8WxNZ6M3UnnYc1cV4osf0U3/AobFoDYIrHjbxgnBrUpsOJKI51zX+fOnUkLwS0EwQ/Pv3L12vUbazc7t27fuXuvu77xzurSUDaiWmhzlBLLBFdsBBwEOyoMIzIV7DA9edHED0+ZsVyrtzAv2ESSXPGMUwLOlax72xExeSS5Sqo3Md/GkSRwnGYV1OfZK8fqTmRLmVSzZ2EdC9wy3jDlWMrzXPQ7ZTKLOX6Mo8wQWvV/NSjqeJaEW3EEuojoVMPF1iuJT/6auNQxW/Hw0W/J039IDv+nZCfp9oJBsDB8GYQt6KHWDpLu92iqaSmZAiqIteMwKGBSEQOcCuZal5YVhJ6QnI0dVEQyO6kWz1rjh84zxZk27ijAC+/5iopIa+cydZnNvHY11jj/FBuXkO1OKq6KEpiiS6GsFBg0bv4InnLDKIi5A4Qa7mbF9Ji45YL7Sc0SwtUrXwaj4WBvELx+2tt/3m5jDd1HD1AfhWgH7aOX6ACNEPXOvA/eJ++z/97/6H/xvy5Tfa+t2UQXzP/2E0lZ/Dk=</latexit><latexit sha1_base64="iv8ND/8gWYWdlskkT7a0HK7d4R0=">AAADFXicrVLLbtNAFB2bVwmvtF2yGREhpaJEdkBqu0CqYMMGqSBCK8WxNZ6M3UnnYc1cV4osf0U3/AobFoDYIrHjbxgnBrUpsOJKI51zX+fOnUkLwS0EwQ/Pv3L12vUbazc7t27fuXuvu77xzurSUDaiWmhzlBLLBFdsBBwEOyoMIzIV7DA9edHED0+ZsVyrtzAv2ESSXPGMUwLOlax72xExeSS5Sqo3Md/GkSRwnGYV1OfZK8fqTmRLmVSzZ2EdC9wy3jDlWMrzXPQ7ZTKLOX6Mo8wQWvV/NSjqeJaEW3EEuojoVMPF1iuJT/6auNQxW/Hw0W/J039IDv+nZCfp9oJBsDB8GYQt6KHWDpLu92iqaSmZAiqIteMwKGBSEQOcCuZal5YVhJ6QnI0dVEQyO6kWz1rjh84zxZk27ijAC+/5iopIa+cydZnNvHY11jj/FBuXkO1OKq6KEpiiS6GsFBg0bv4InnLDKIi5A4Qa7mbF9Ji45YL7Sc0SwtUrXwaj4WBvELx+2tt/3m5jDd1HD1AfhWgH7aOX6ACNEPXOvA/eJ++z/97/6H/xvy5Tfa+t2UQXzP/2E0lZ/Dk=</latexit><latexit sha1_base64="iv8ND/8gWYWdlskkT7a0HK7d4R0=">AAADFXicrVLLbtNAFB2bVwmvtF2yGREhpaJEdkBqu0CqYMMGqSBCK8WxNZ6M3UnnYc1cV4osf0U3/AobFoDYIrHjbxgnBrUpsOJKI51zX+fOnUkLwS0EwQ/Pv3L12vUbazc7t27fuXuvu77xzurSUDaiWmhzlBLLBFdsBBwEOyoMIzIV7DA9edHED0+ZsVyrtzAv2ESSXPGMUwLOlax72xExeSS5Sqo3Md/GkSRwnGYV1OfZK8fqTmRLmVSzZ2EdC9wy3jDlWMrzXPQ7ZTKLOX6Mo8wQWvV/NSjqeJaEW3EEuojoVMPF1iuJT/6auNQxW/Hw0W/J039IDv+nZCfp9oJBsDB8GYQt6KHWDpLu92iqaSmZAiqIteMwKGBSEQOcCuZal5YVhJ6QnI0dVEQyO6kWz1rjh84zxZk27ijAC+/5iopIa+cydZnNvHY11jj/FBuXkO1OKq6KEpiiS6GsFBg0bv4InnLDKIi5A4Qa7mbF9Ji45YL7Sc0SwtUrXwaj4WBvELx+2tt/3m5jDd1HD1AfhWgH7aOX6ACNEPXOvA/eJ++z/97/6H/xvy5Tfa+t2UQXzP/2E0lZ/Dk=</latexit>

Structure from Motion: Bundle Adjustment

• Typically, the method is difficult to minimize as a whole thing. This is because there are many parameters to minimize.

• A two-step approach:

• First, minimize (or viceversa) all extrinsic parameters (G) without modifying the 3D points.

• Then, minimize (or viceversa) 3D points coordinates without modifying G.

Structure from Motion: Example

Structure from Motion: Example

Structure from Motion: Example

Structure From Motion: Multi-View

• To obtain something of interesting:

• we need to feed into the system hundreds of images.

• we need to manage thousands of features (corners)!

• Even the two-step approach would struggle a bit.

Structure from Motion: Multi-View

• To make the problem computational tractable, we can notice this:

12 4

3 5

6 8

7 9

A B C D

1,2,3,4,5,6,7,8, and 9 are points A,B,C, and D are cameras the point is seen in a camera

Structure from Motion: Multi-View

• The idea is to divide the scene into clusters.

• For each cluster we compute SfM.

• We combine all 3D reconstructions and camera poses together.

Structure from Motion: Multi-View

12 4

3 5

6 8

7 9

A B C D

Cluster A Cluster B

Structure from Motion: Conclusions

• Advantages:

• It requires only photographs/videos: cheap and fast.

• We can recover color information from photographs!

• Disadvantages:

• The output model may be skewed; it is hard to keep two things going at the same time (3D points and cameras’ poses).

• We do not have the scale of the 3D scene; we need some reference measures.

Structure from Motion: Scale

• This scale factor is due to the fact we do not know the real scale.

• if we took a photograph of:

• a large object far away from the camera.

• a small object near the camera. •

Structure from Motion: Scale ExampleDLT: Direct Linear Transform

Case 1 Case 2

One more thing…

We have some wrong matches!

RANSAC• Random sample consensus (RANSAC) is an

iterative method for estimating the parameters of a model in a robust way.

• The main idea is to get a subset of the set of samples and to estimate the model with this subset:

• We estimate the model using the best subset of samples!

RANSAC

• Input: a set of n samples S, and a model 𝜋.

• Output: parameters, P, for the model 𝜋., and the subset of S that was used to compute P.

RANSAC• e = +∞ and Sb = ∅

• For each iteration:

• Si ⊂ S where Si is random.

• Estimate Pi for 𝜋 using Si

• Compute the error ei for Pi

• if ei < e then

• e = ei and Sb = Si

RANSAC: Example𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 0 e = +∞

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 1 e = +∞

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 1 e = +∞

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 1 e = +∞

ei = 10

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 2 e = 10

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 2 e = 10

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 2 e = 10

𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 2 e = 10

ei = 1𝜋: a straight line

Si and Sb

RANSAC: Example

Iteration 2 e = 1

𝜋: a straight line

Si and Sb

and we continue for n iterations…

how many?

RANSAC: Iterations

• n has to be large; i.e., we need to have at least one subset containing only inliers Sinliers:

• We are interested for P = 1.

P (|Si| = c) = 1�✓1�

✓1� |Soutliers|

|S|

◆c◆n

Si ✓ Sinliers

RANSAC• When do we need to use it?

• Estimation of the fundamental/essential matrix.

• Estimation of a homography in the general case.

• When we do not need it for:

• DLT and Zhang’s algorithm:

• corners are extracted in an accurate way using a calibration pattern!

RANSAC: Fundamental Matrix Estimation• The algorithm is modified a bit:

• We count the inliers of each set given a threshold:

• terr takes into account this constraint:

• If we have a set with more inliers of the previous one it is accepted.

• We compute the F using only the inliers!

m>2 · F ·m1 = 0

m>1 · F> ·m2 = 0

<latexit sha1_base64="Zl/LEIAhHeXLtIBq6dAY2KMtuQ8=">AAACU3icbVHPS8MwGE3r1DmdTj16CQ7B02iHoB6EoSAeJzg3WGtJs3QLS5uSfBVG2R+pB8G/xIsHs62H/fCDkJf3vUe+vISp4Boc59uyt0rbO7vlvcr+QfXwqHZ88qplpijrUCmk6oVEM8ET1gEOgvVSxUgcCtYNxw+zfvedKc1l8gKTlPkxGSY84pSAoYLa2IsJjMIoj6dB880DmWLs0YEE/FjsSwIX32HseF5lmVs1LU4bzqZxOkGt7jSceeFN4BagjopqB7VPbyBpFrMEqCBa910nBT8nCjgVbFrxMs1SQsdkyPoGJiRm2s/noUzxhWEGOJLKrATwnF125CTWehKHRjmbU6/3ZuR/vX4G0Y2f8yTNgCV0cVGUCQwSzxLGA64YBTExgFDFzayYjogiFMw/VEwI7vqTN0Gn2bhtOM9X9dZ9kUYZnaFzdIlcdI1a6Am1UQdR9IF+LGRZ1pf1a9t2aSG1rcJzilbKrv4BS8axBg==</latexit><latexit sha1_base64="Zl/LEIAhHeXLtIBq6dAY2KMtuQ8=">AAACU3icbVHPS8MwGE3r1DmdTj16CQ7B02iHoB6EoSAeJzg3WGtJs3QLS5uSfBVG2R+pB8G/xIsHs62H/fCDkJf3vUe+vISp4Boc59uyt0rbO7vlvcr+QfXwqHZ88qplpijrUCmk6oVEM8ET1gEOgvVSxUgcCtYNxw+zfvedKc1l8gKTlPkxGSY84pSAoYLa2IsJjMIoj6dB880DmWLs0YEE/FjsSwIX32HseF5lmVs1LU4bzqZxOkGt7jSceeFN4BagjopqB7VPbyBpFrMEqCBa910nBT8nCjgVbFrxMs1SQsdkyPoGJiRm2s/noUzxhWEGOJLKrATwnF125CTWehKHRjmbU6/3ZuR/vX4G0Y2f8yTNgCV0cVGUCQwSzxLGA64YBTExgFDFzayYjogiFMw/VEwI7vqTN0Gn2bhtOM9X9dZ9kUYZnaFzdIlcdI1a6Am1UQdR9IF+LGRZ1pf1a9t2aSG1rcJzilbKrv4BS8axBg==</latexit><latexit sha1_base64="Zl/LEIAhHeXLtIBq6dAY2KMtuQ8=">AAACU3icbVHPS8MwGE3r1DmdTj16CQ7B02iHoB6EoSAeJzg3WGtJs3QLS5uSfBVG2R+pB8G/xIsHs62H/fCDkJf3vUe+vISp4Boc59uyt0rbO7vlvcr+QfXwqHZ88qplpijrUCmk6oVEM8ET1gEOgvVSxUgcCtYNxw+zfvedKc1l8gKTlPkxGSY84pSAoYLa2IsJjMIoj6dB880DmWLs0YEE/FjsSwIX32HseF5lmVs1LU4bzqZxOkGt7jSceeFN4BagjopqB7VPbyBpFrMEqCBa910nBT8nCjgVbFrxMs1SQsdkyPoGJiRm2s/noUzxhWEGOJLKrATwnF125CTWehKHRjmbU6/3ZuR/vX4G0Y2f8yTNgCV0cVGUCQwSzxLGA64YBTExgFDFzayYjogiFMw/VEwI7vqTN0Gn2bhtOM9X9dZ9kUYZnaFzdIlcdI1a6Am1UQdR9IF+LGRZ1pf1a9t2aSG1rcJzilbKrv4BS8axBg==</latexit>

that’s all folks!