![Page 1: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/1.jpg)
1
SurroundSense: Mobile Phone Localization via Ambience Fingerprinting
Ionut Constandache
Co-authors: Martin Azizyan and Romit Roy Choudhury
![Page 2: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/2.jpg)
2
Context
Pervasive wireless connectivity+
Localization technology=
Location-based applications
![Page 3: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/3.jpg)
3
Location-Based Applications (LBAs)
For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks
iPhone AppStore: 3000 LBAs, Android: 500 LBAs
![Page 4: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/4.jpg)
4
Location-Based Applications (LBAs)
For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks
iPhone AppStore: 3000 LBAs, Android: 500 LBAs
Location expresses context of user Facilitates content delivery
![Page 5: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/5.jpg)
5
Location is an IP addressLocation is an IP addressAs if for content delivery
![Page 6: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/6.jpg)
6
Thinking about Localizationfrom an application perspective…
![Page 7: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/7.jpg)
7
Emerging location based apps need place of user, not physical location
Starbucks, RadioShack, Museum, Library
Latitude, Longitude
![Page 8: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/8.jpg)
8
Emerging location based apps need place of user, not physical location
Starbucks, RadioShack, Museum, Library
Latitude, Longitude
We call this Logical Localization …We call this Logical Localization …
![Page 9: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/9.jpg)
9
Can we convert from Physical to Logical Localization?
![Page 10: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/10.jpg)
10
Can we convert from Physical to Logical Localization?
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
![Page 11: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/11.jpg)
11
Widely-deployable localization technologies have errors in the range of several meters
Widely-deployable localization technologies have errors in the range of several meters
Can we convert from Physical to Logical Localization?
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
![Page 12: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/12.jpg)
12
Several meters of error is inadequate to logically localize a phone
Physical LocationError
![Page 13: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/13.jpg)
13
Several meters of error is inadequate to logically localize a phone
RadioShackStarbucks
Physical LocationError
The dividing-wall problem The dividing-wall problem
![Page 14: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/14.jpg)
14
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 15: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/15.jpg)
15
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 16: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/16.jpg)
16
It is possible to localize phones by sensing the ambience
Hypothesis
such as sound, light, color, movement, WiFi …
![Page 17: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/17.jpg)
17
Sensing over multiple dimensions extracts more information from the ambience
Each dimension may not be unique, but put together, they may provide a
unique fingerprint
![Page 18: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/18.jpg)
18
SurroundSense
Multi-dimensional fingerprint Based on ambient sound/light/color/movement/WiFi
Starbucks
Wall
RadioShack
![Page 19: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/19.jpg)
19
BB AACC DDEE
Should Ambiences be Unique Worldwide?
FFGG
HHJJ
II
LLMMNN
OO
PPQQ
RR
KK
![Page 20: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/20.jpg)
20
Should Ambiences be Unique Worldwide?
BB AACC DDEE
FFGG
HHJJ
II
KKLL
MMNNOO
PPQQ
RR
GSM provides macro location (strip mall) SurroundSense refines to Starbucks
![Page 21: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/21.jpg)
21
++
Ambience Fingerprinting
Test Fingerprint
Sound
Acc.
Color/Light
WiFi
LogicalLocation
Matching
FingerprintDatabase
==
Candidate Fingerprints
GSM Macro Location
SurroundSense Architecture
![Page 22: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/22.jpg)
22
Fingerprints
Sound:(via phone microphone)
Color:(via phone camera)
Amplitude Values-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
N
orm
aliz
ed
C
ount
0.14
0.12
0.1
0.08
0.06
0.04 0.02
0
Acoustic fingerprint (amplitude distribution)
Color and light fingerprints on HSL space
Lightn
ess
1
0.5
0
Hue
0
0.5
1 00.2
0.4
0.6
0.8
1
Saturation
![Page 23: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/23.jpg)
23
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
![Page 24: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/24.jpg)
24
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
Queuing
![Page 25: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/25.jpg)
25
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Queuing Seated
Moving
![Page 26: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/26.jpg)
26
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product browsing
Moving
![Page 27: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/27.jpg)
27
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product browsing
Short walks between product browsing
Moving
![Page 28: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/28.jpg)
28
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more
Moving
![Page 29: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/29.jpg)
29
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more Quicker stops
Moving
![Page 30: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/30.jpg)
30
Fingerprints
Movement: (via phone accelerometer)
WiFi: (via phone wireless card)
Cafeteria Clothes Store Grocery Store
Static
ƒ(overheard WiFi APs)
ƒ(overheard WiFi APs)
Moving
![Page 31: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/31.jpg)
31
Discussion
Time varying ambience Collect ambience fingerprints over different time
windows
What if phones are in pockets? Use sound/WiFi/movement Opportunistically take pictures
Fingerprint Database War-sensing
![Page 32: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/32.jpg)
32
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 33: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/33.jpg)
33
Evaluation Methodology
51 business locations 46 in Durham, NC 5 in India
Data collected by 4 people 12 tests per location
Mimicked customer behavior
![Page 34: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/34.jpg)
34
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
![Page 35: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/35.jpg)
35
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
Multidimensional sensing
![Page 36: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/36.jpg)
36
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Fault tolerance
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
![Page 37: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/37.jpg)
37
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
Sparse WiFi APs
![Page 38: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/38.jpg)
38
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
No WiFi APs
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster
![Page 39: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/39.jpg)
39
Evaluation: Per-Scheme Accuracy
Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS
Accuracy 70%
74% 76% 87%
![Page 40: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/40.jpg)
40
Evaluation: User Experience
Random Person Accuracy
Average Accuracy (%)0 10 20 30 40 50 60 70 80 90 100
1 0.9 0.8 0.7 0.6 0.5
CD
F
0.4 0.3 0.2 0.1 0
WiFISnd-Acc-WiFiSnd-Acc-Clr-LtSurroundSense
![Page 41: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/41.jpg)
41
Economics forces nearby businesses to be different
Not profitable to have 3 coffee shopswith same lighting, music, color, layout, etc.
SurroundSense exploits this ambience diversity
Why does it work?
The Intuition:
![Page 42: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/42.jpg)
42
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 43: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/43.jpg)
43
Limitations and Future Work
Energy-Efficiency
Localization in Real Time
Non-business locations
![Page 44: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/44.jpg)
44
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy draw
Localization in Real Time
Non-business locations
![Page 45: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/45.jpg)
45
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy draw
Localization in Real Time User’s movement requires time to converge
Non-business locations
![Page 46: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/46.jpg)
46
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy draw
Localization in Real Time User’s movement requires time to converge
Non-business locations Ambiences may be less diverse
![Page 47: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/47.jpg)
47
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 48: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/48.jpg)
48
SurroundSense
Today’s technologies cannot provide logical localization
Ambience contains information for logical localization
Mobile Phones can harness the ambience through sensors
Evaluation results: 51 business locations, 87% accuracy
SurroundSense can scale to any part of the world
![Page 49: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache](https://reader035.vdocument.in/reader035/viewer/2022070409/5681449d550346895db14e4c/html5/thumbnails/49.jpg)
49
Questions?
Thank You!
Visit the SyNRG research group @http://synrg.ee.duke.edu/