6.s062: mobile and sensor computing
TRANSCRIPT
![Page 1: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/1.jpg)
6.S062: Mobile and Sensor Computing
Lecture4:Device-FreeLocaliza3on
![Page 2: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/2.jpg)
So Far: Device-based Localization
![Page 3: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/3.jpg)
This Lecture: Using radio signals to track humans without any sensors on
their bodies
![Page 4: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/4.jpg)
This Lecture: Using radio signals to track humans without any sensors on their bodies
Operates through occlusions
• Location • Vital Signs • Gestures
![Page 5: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/5.jpg)
Example: WiTrack
![Page 6: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/6.jpg)
Device in another room
Device
![Page 7: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/7.jpg)
ApplicationsSmartHomes
EnergySaving Gaming&VirtualReality
![Page 8: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/8.jpg)
Measuring Distances
Rx
Tx
Distance = Reflection time x speed of light
![Page 9: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/9.jpg)
Measuring Reflection Time
Time
Txpulse Rxpulse
Option1:Transmitshortpulseandlistenforecho
ReflectionTime
![Page 10: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/10.jpg)
Why?
Measuring Reflection Time
Time
Txpulse Rxpulse
Option1:Transmitshortpulseandlistenforecho
Capturingthepulseneedssub-nanosecondsampling
SignalSamples
ReflectionTime
![Page 11: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/11.jpg)
Why was this not a problem for Cricket?
Capturingthepulseneedssub-nanosecondsamplingWhy?
Multi-GHz samplers are expensive, have high
noise, and create large I/O problem
![Page 12: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/12.jpg)
Time
Freq
uenc
yTransmitted
t
FMCW: Measure time by measuring frequency
![Page 13: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/13.jpg)
![Page 14: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/14.jpg)
Time
Freq
uenc
yTransmitted
t t+ΔT
Received
FMCW: Measure time by measuring frequency
Reflection Time
How do we measure ΔF?
ΔF
ΔF slope=
![Page 15: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/15.jpg)
Measuring ΔF
MixerTransmittedReceived
Signal whose frequency is ΔF
FFT
Power
ΔF
• Subtracting frequencies is easy (e.g., removing carrier in WiFi)
• Done using a mixer (low-power; cheap)
![Page 16: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/16.jpg)
![Page 17: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/17.jpg)
Measuring ΔF
MixerTransmittedReceived
Signal whose frequency is ΔF
FFT
Power
ΔF
ΔF ➔Reflection Time ➔ Distance
• Subtracting frequencies is easy (e.g., removing carrier in WiFi)
• Done using a mixer (low-power; cheap)
![Page 18: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/18.jpg)
Challenge: Multipath➔ Many Reflections
Rx
Tx
Distance
Refle
ctio
n Po
wer Multi-paths mask person
![Page 19: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/19.jpg)
Static objects don’t move ➔ Eliminate by subtracting consecutive measurements
Distance
Power
Distance
Power @timet+30ms
@ time t
-=
Multi-path
Multi-path
2meters
Power
Distance
Why 2 peaks when we only have one moving person?
![Page 20: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/20.jpg)
Challenge: Dynamic Multipath
Rx
Tx
Distance
Power
DynamicMulti-path
MovingPerson
The direct reflection arrives before dynamic multipath!
![Page 21: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/21.jpg)
Mapping Distance to Location
Person can be anywhere on an ellipse whose foci are (Tx,Rx)
By adding another antenna and intersecting the ellipses, we can localize the person
TxRx
d
Rx’
![Page 22: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/22.jpg)
From Location to tracking
![Page 23: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/23.jpg)
Challenge: Dynamic Multipath
Rx
Tx
Distance
Power
DynamicMulti-path
MovingPerson
The direct reflection arrives before dynamic multipath!Fails for multiple people in the environment, and we need a more comprehensive solution
![Page 24: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/24.jpg)
How can we deal with multi-path reflections when there are multiple
persons in the environment?
![Page 25: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/25.jpg)
Idea: Person is consistent across different vantage points while multi-path is different
from different vantage points
![Page 26: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/26.jpg)
Combining across Multiple Vantage PointsExperiment: Two users walking
Setup Single Vantage Point
Mathematically: each round-trip distance can be mapped to an ellipse whose foci are the transmitter and the receiver
![Page 27: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/27.jpg)
Experiment: Two users walkingSetup Two Vantage Points
Combining across Multiple Vantage Points
![Page 28: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/28.jpg)
Experiment: Two users walkingSetup 16 Vantage Points
Localize the two users
Combining across Multiple Vantage Points
![Page 29: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/29.jpg)
Multi-User LocalizationExperiment: Four persons walkingSetup All Vantage Points
four persons
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0 1 2 3 4 5 6 7 8
Dis
tanc
e (m
eter
s)
first person other people or noise?
![Page 30: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/30.jpg)
Multi-User LocalizationExperiment: Four persons walkingSetup All Vantage Points
four persons
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0 1 2 3 4 5 6 7 8
Dis
tanc
e (m
eter
s)
first person other people or noise?
Near-Far Problem: Nearby persons have more power than distance reflectors and can mask them
![Page 31: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/31.jpg)
Successive Silhouette Cancellation: a new algorithm that localizes multiple
persons in the scene by addressing the near-far problem
![Page 32: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/32.jpg)
Decode human location
Goal: Recover human reflections
Model human and reconstruct
reflection patterns
Subtract
Successive Silhouette Cancellation: a new algorithm that localizes multiple persons in
the scene by addressing the near-far problem
![Page 33: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/33.jpg)
First localize the user with the strongest reflection
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0 1 2 3 4 5 6 7 8
Dis
tanc
e (m
eter
s)
Person 1 at (-0.5, 1.3)
![Page 34: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/34.jpg)
Cancel the impact of the person’s whole body
![Page 35: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/35.jpg)
After reconstructing and cancelling the first user’s reflections
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0 1 2 3 4 5 6 7 8
Dis
tanc
e (m
eter
s) Person 2 at (0.5, 2)
![Page 36: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/36.jpg)
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0 1 2 3 4 5 6 7 8
Dis
tanc
e (m
eter
s)Iteratively localize the remaining users in
the scene
Person 3 at (0.8, 2.7)
![Page 37: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/37.jpg)
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0 1 2 3 4 5 6 7 8
Dis
tanc
e (m
eter
s)Iteratively localize the remaining users in
the scene
Person 4 at (1, 4)
![Page 38: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/38.jpg)
How can we localize static users?
![Page 39: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/39.jpg)
Dealing with multi-path when there is one moving user
Rx
Tx
We eliminated direct table reflections by subtracting consecutive measurements
Needs User to Move
![Page 40: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/40.jpg)
Dealing with multi-path when there is one moving user
Rx
Tx
STATIC
We eliminated direct table reflections by subtracting consecutive measurements
Needs User to Move
![Page 41: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/41.jpg)
Exploit breathing motion for localize static users
• Breathing and walking happen at different time scales –A user that is pacing moves at 1m/s –When you breathe, chest moves by few mm/s
• Cannot use the same subtraction window to eliminate multi-path
![Page 42: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/42.jpg)
User Walking at 1m/s
30ms subtraction window 3s subtraction window
Localize the person
Person appears in two locations
![Page 43: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/43.jpg)
User Sitting Still (Breathing)
Cannot localizeLocalize the
person
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0
1
2
3
4
5
6
7
8
Dis
tanc
e (m
eter
s)
30ms subtraction window 3s subtraction window
![Page 44: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/44.jpg)
User Sitting Still (Breathing)
CannotlocalizeLocalizetheperson
-4 -3 -2 -1 0 1 2 3 4Distance (meters)
0
1
2
3
4
5
6
7
8
Dis
tanc
e (m
eter
s)
30ms subtraction window 3s subtraction window
Use multi-resolution subtraction window to eliminate multi-path while being able to localize
both static and moving users
![Page 45: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/45.jpg)
![Page 46: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/46.jpg)
Localize the two users
Centimeter-scale localization without requiring the user to carry a wireless device
![Page 47: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/47.jpg)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
Localize the two users
People are points
Want a silhouette
![Page 48: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/48.jpg)
Approach: Combine antenna arrays with FMCW to get 3D image • 2D Antenna array gives 2 angles • FMCW gives depth (1D)
2D array
1D
1D
![Page 49: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/49.jpg)
49
Output of 3D RF Scan
Blobsofreflectionpower
Challenge: We only obtain blobs in space
![Page 50: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/50.jpg)
Cannot Capture Reflection
Challenge: We only obtain blob in spaceAt frequencies that traverse walls, human body parts are specular (pure mirror)
RF Scanning
Setup
At every point in time, we get reflections from only a subset of body parts.
![Page 51: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/51.jpg)
Solution Idea: Exploit Human Motion and Aggregate over Time
RF Scanning Setup
![Page 52: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/52.jpg)
Solution Idea: Exploit Human Motion and Aggregate over Time
RF Scanning Setup
Previous Location
New Location
Combine the various snapshots
![Page 53: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/53.jpg)
3m 2.5m 2m
Chest (Largest Convex Reflector)
Use it as a pivot: for motion compensation and segmentation
Human Walks toward Sensor
![Page 54: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/54.jpg)
3m 2.5m 2m
Chest (Largest Convex Reflector)
Use it as a pivot: for motion compensation and segmentationCombine the various snapshots
Right arm
Left arm
Head
Lower TorsoFeet
Human Walks toward Sensor
![Page 55: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/55.jpg)
Human Walks toward Sensor
![Page 56: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/56.jpg)
SampleCapturedFiguresthroughWalls
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2y-
axis
(met
ers)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
Sample Captured Figures through Walls
![Page 57: 6.S062: Mobile and Sensor Computing](https://reader033.vdocument.in/reader033/viewer/2022053005/6290ba724a02d00fd06d35b3/html5/thumbnails/57.jpg)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2y-
axis
(met
ers)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
SampleCapturedFiguresthroughWalls
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2y-
axis
(met
ers)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
-0.2 0 0.2 0.4 0.6 0.8x-axis (meters)
0
0.5
1
1.5
2
y-ax
is (m
eter
s)
Sample Captured Figures through WallsThrough-wall classification accuracy of 90% among 13 users