machine learning approaches for wireless spectrum and energy …chintha/pdf/thesis/ppt_keyu.pdf ·...
TRANSCRIPT
![Page 1: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/1.jpg)
PhD Final Oral Defense
Machine Learning Approaches for Wireless Spectrum andEnergy Intelligence
Keyu Wu
Department of Electrical and Computer EngineeringUniversity of Alberta, Edmonton, Alberta T6G 1H9, Canada
September, 2018
![Page 2: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/2.jpg)
Outlines:
1 Background and Motivations
2 Sensing-Probing-Transmitting Control of EH CR
3 Selective Transmission for EH Sensors
4 CSS under Spectrum Heterogeneity
5 Conclusion and Future Research
![Page 3: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/3.jpg)
Outline
1 Background and Motivations
2 Sensing-Probing-Transmitting Control of EH CR
3 Selective Transmission for EH Sensors
4 CSS under Spectrum Heterogeneity
5 Conclusion and Future Research
![Page 4: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/4.jpg)
Spectrum and Energy Consideration of WirelessCommunications
With the increase of data volume, service types and devices, wirelesscommunication face
Spectrum scarcity
Limited spectrum for wireless applications(33% for all commercial applications in 225 to 3700 MHz);Spectrum reallocation is expensive and slow(70 MHz band costs 19.8 billion dollars).
Energy issue
Huge energy consumption without careful design(communication industry may use 51% of global electricity in 2030);Difficult for powering massive amount of IoT devices.
1 / 20
![Page 5: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/5.jpg)
Cognitive Radio and Energy Harvesting
Cognitive radioEnergy harvesting
2 / 20
![Page 6: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/6.jpg)
ML for wireless spectrum & energy intelligence
Machine learning, a data-driven methodology, is promising for handlingrelevant spectrum and energy management problems.
With ML as a primary tool, three research contributions are made
Joint sensing-probing-transmitting control for EH CR
Optimal transmission for an EH sensor with data priority consideration
Cooperative spectrum sensing under spectrum heterogeneity
3 / 20
![Page 7: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/7.jpg)
Outline
1 Background and Motivations
2 Sensing-Probing-Transmitting Control of EH CR
3 Selective Transmission for EH Sensors
4 CSS under Spectrum Heterogeneity
5 Conclusion and Future Research
![Page 8: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/8.jpg)
Sensing-probing-transmitting
Harvest: energy package arrives each time slot (uncontrollable)
Sense: measure channel output to detect and track PU activity
Probe: estimate CSI via pilot sequence
Transmit: based on CSI, adapt transmission power and send data
4 / 20
![Page 9: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/9.jpg)
Sensing-probing-transmitting
Harvest: energy package arrives each time slot (uncontrollable)
Sense: measure channel output to detect and track PU activity
Probe: estimate CSI via pilot sequence
Transmit: based on CSI, adapt transmission power and send data
4 / 20
![Page 10: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/10.jpg)
Sensing-probing-transmitting
Harvest: energy package arrives each time slot (uncontrollable)
Sense: measure channel output to detect and track PU activity
Probe: estimate CSI via pilot sequence
Transmit: based on CSI, adapt transmission power and send data
4 / 20
![Page 11: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/11.jpg)
Sensing-probing-transmitting
Harvest: energy package arrives each time slot (uncontrollable)
Sense: measure channel output to detect and track PU activity
Probe: estimate CSI via pilot sequence
Transmit: based on CSI, adapt transmission power and send data
4 / 20
![Page 12: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/12.jpg)
Sensing-probing-transmitting
Harvest: energy package arrives each time slot (uncontrollable)
Sense: measure channel output to detect and track PU activity
Probe: estimate CSI via pilot sequence
Transmit: based on CSI, adapt transmission power and send data
Problem
Based on energy status, PU activity and CSI, the node needs to decidewhether or not to sense and probe, and how much power for transmission,in order to maximize long-term throughput.
4 / 20
![Page 13: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/13.jpg)
Two-stage MDP for long-term optimization
Goal: solving a policy π∗ that maximizes expected throughput
π∗ = arg maxπ
{E
[ ∞∑t=0
γtr(st , π(st))
]}
5 / 20
![Page 14: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/14.jpg)
After-state simplification
s
r
a
s'
Currentstate
After-state Nextstate
π∗(s) = arg maxa∈A(s)
{r(s, a) + γE[V ∗(s ′)|s, a]
}= arg max {immed. reward + expected furture value}= arg max
a∈A(s){r(s, a) + J∗( %(s, a)︸ ︷︷ ︸
after-state β
)}
6 / 20
![Page 15: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/15.jpg)
Solve J∗ with RL
Exactly solving J∗ requires the pdfs of EH and fading processes, which canbe hard to obtain.
7 / 20
![Page 16: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/16.jpg)
Solve J∗ with RL
Exactly solving J∗ requires the pdfs of EH and fading processes, which canbe hard to obtain.
We consider to (approximately) learn J∗ with RL algorithm withoutdistribution information.
EH/fadingsample RL
7 / 20
![Page 17: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/17.jpg)
Learned policies
(a) Sensing-probing sub-policy (b) Transmitting sub-policy
aSP : ‘00’, no sense; ‘10’, sense but no probe; ‘11’, sense and probeEnergy for: sense, 1; probe, 2; transmit, {no tx, 3, 4, 5, 6}.
8 / 20
![Page 18: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/18.jpg)
Outline
1 Background and Motivations
2 Sensing-Probing-Transmitting Control of EH CR
3 Selective Transmission for EH Sensors
4 CSS under Spectrum Heterogeneity
5 Conclusion and Future Research
![Page 19: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/19.jpg)
Selective transmission for EH sensor
Incorporating data-centric consideration
packets associated different priorities
drop low priority packet to save energy
9 / 20
![Page 20: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/20.jpg)
Selective transmission for EH sensor
Problem
Based on EH, energy status, CSI and packet priority, the node needs todecide whether or not to send each packet, in order to maximize the totalpriority values of sent packets.
9 / 20
![Page 21: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/21.jpg)
MDP formulation with after-state
10 / 20
![Page 22: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/22.jpg)
Structural results of J∗ and π∗
Theorem
After-state value function J∗(p) is differentiable and non-decreasing.
Theorem
The optimal policy π∗ has the following structure
π∗([b, h, d ]) =
{1 if b ≥ h and d ≥ J∗(b)− J∗(b − h),
0 otherwise.
11 / 20
![Page 23: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/23.jpg)
Learn J∗ with monotone neural network
FittedValue
Iteration
z-1Data set
FMNN
new MNN
current MNN
ifCnvg.
12 / 20
![Page 24: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/24.jpg)
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 0.2 0.4 0.6 0.8 1
0.5
1
1.5
2
2.5
3
13 / 20
![Page 25: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/25.jpg)
Learning efficiency
5e2 1e3 1e4 1e5 1e6
Consumed data samples
0.06
0.08
0.1
0.12
0.14
0.16
0.18
Aver
aged
rew
ard
s
Online-DIS
FNN
FMNN
5 10
105
0.16
0.18
Learning curve
14 / 20
![Page 26: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/26.jpg)
Outline
1 Background and Motivations
2 Sensing-Probing-Transmitting Control of EH CR
3 Selective Transmission for EH Sensors
4 CSS under Spectrum Heterogeneity
5 Conclusion and Future Research
![Page 27: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/27.jpg)
CSS under spectrum heterogeneity
Spectrum heterogeneity
SUs at different spatial locations mayexperience different spectrumstatuses.
15 / 20
![Page 28: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/28.jpg)
CSS under spectrum heterogeneity
Spectrum heterogeneity
SUs at different spatial locations mayexperience different spectrumstatuses.
Problem
Under spectrum heterogeneity, how to exploit neighbor information to fuseSU observations for improving sensing performance.
15 / 20
![Page 29: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/29.jpg)
Fuse data via MAP-MRF
MAP-MRF CSS framework
Compute maximum a posterior estimation
xMAP = arg maxx
{ΦX (x)
N∏i=1
γxi fY |X (yi | xi )
},
weight γ > 0 introduces tradeoff
Existing works in references [99–102] fuse data via solving marginaldistributions.Compared with [99–102], the proposed MAP-MRF can be solved moreflexibly and efficiently.
16 / 20
![Page 30: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/30.jpg)
Three CSS algorithms based on MAP-MRF
Via graph cut theory, GC-CSS algorithm solves xMAP exactly; complexityorder: O(N · |E|2).
17 / 20
![Page 31: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/31.jpg)
Three CSS algorithms based on MAP-MRF
Via dual decomposition theory, DD-CSS estimates xMAP distributedly (atcluster-level); complexity: O(T · Nl · |El |2).
17 / 20
![Page 32: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/32.jpg)
Three CSS algorithms based on MAP-MRF
Distributed network: clusters with size 1, DD1-CSS becomes fullydistributedly message passing algorithm; guaranteed for solving xMAP ;complexity order O(T · |N (i)|3).
17 / 20
![Page 33: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/33.jpg)
Three CSS algorithms based on MAP-MRF
Existing algorithms (based on belief propagation) only work in distributedsetting; complexity: O(T · |N (i)| · 2|N (i)|).
17 / 20
![Page 34: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/34.jpg)
Performance comparison
0 0.1 0.2 0.3 0.40.5
0.6
0.7
0.8
0.9
1
GC-CSS
DD-CSS
DD1-CSS
BP-CSS
Ind-SS
ROC for various algorithms.18 / 20
![Page 35: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/35.jpg)
Outline
1 Background and Motivations
2 Sensing-Probing-Transmitting Control of EH CR
3 Selective Transmission for EH Sensors
4 CSS under Spectrum Heterogeneity
5 Conclusion and Future Research
![Page 36: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/36.jpg)
Conclusion
In EH CR, the joint optimization of sensing, probing and transmittingis modeled as a two-stage MDP, whose structure is exploited forafter-state simplification.
In EH WSNs, the optimal selective transmission policy is investigated,which is proved to be threshold-based and derived by training amonotone neural network.
CSS under spectrum heterogeneity is formulated via MAP-MRF,which can be effectively solved by graph cut theory and dualdecomposition theory with polynomial complexity.
19 / 20
![Page 37: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/37.jpg)
Future research
Optimal sensing-probing policy without primary user model
Multi-link selective transmission for energy-harvesting sensors
Learn MRF model from data
20 / 20
![Page 38: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum](https://reader036.vdocument.in/reader036/viewer/2022071610/6149b87e12c9616cbc68f1fb/html5/thumbnails/38.jpg)
Thank you!