measurements to inform policy makers and end consumers · indispensible to all key stake‐holders...
TRANSCRIPT
![Page 1: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/1.jpg)
Ahmed ElmokashfiAIMs 201703/03/2017
Measurements to inform policy makers and end consumers
![Page 2: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/2.jpg)
Measurement nodes spread across Norway
About 100 active stationary measurement nodes and seven nodes on regional trains
Multi‐homed to all major Norwegian MBB operators
Operational since July 2013
![Page 3: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/3.jpg)
NNE measurement nodes are single board computers that run Linux
Measurement nodes
Measurement servers Management and data repository
Visualization
![Page 4: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/4.jpg)
A brief history of NorNet
2012 2017
Robusthet i norske mobilnett Tilstandsrapport 2014, CRNA
CRNA Centre for Resilient Networks & Applications
2013 2014 2015 2016
![Page 5: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/5.jpg)
A few ways to engage policy makers and the public
Periodic measurements reports and on‐demand incidents reports – credibility?
Give expert opinion – You need to convince them to approach you
Take initiative to inform ongoing debates – You needto follow
![Page 6: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/6.jpg)
A framework for measuring reliability at multiple levels
Performance stability
Data plane stability
Connection stability
Coverage
Performance stability
Data plane stability
Connection stability
Coverage
Good service
Can send data
Able to connect
Communication possible
Good service
Can send data
Able to connect
Communication possible
HTTP downloads
Packet loss, large events
Failure rate, downtime
Available technology
HTTP downloads
Packet loss, large events
Failure rate, downtime
Available technology
Aspect of reliability User experience Metrics
Increased im
portan
ce
Baltrunas et al. ACM IMC 2014
![Page 7: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/7.jpg)
Permanent dedicated measurement are important to evaluating performance over
timeMed
ian Packet Loss %
![Page 8: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/8.jpg)
The spread of NNE nodes helps identifying local, regional and global outages
Baltrunas et al. IEEE Infocom 2015, Zhou et al. NetSci 2016
Reporting on outages gives consumers more details than operators postmortem brief press releases
![Page 9: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/9.jpg)
The government has recognized the role of measurements in it’s 10‐year plan
![Page 10: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/10.jpg)
Use data to inform policy decisions• Coverage on trains is bad, so shall
we invest in signal repeaters?
Nodes on trains continuously measure experienced coverage (4G, 3G, 2G, No coverage)
Geo‐tag all measurements and map them to 1kmx1km grids
Measure typical coverage (most experienced) and optimistic (the best technology seen > 10% of the time
![Page 11: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/11.jpg)
Typicalcoverage
Technologyseen themost
Local trainstretches
![Page 12: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/12.jpg)
Optimisticcoverage
Best technologyseen at least
10%of the time
Local trainstretches
![Page 13: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/13.jpg)
Rogue cell phones surveillance devices in Oslo?
![Page 14: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/14.jpg)
Take initiative to inform current debates
The pitfalls of using speedtest results by operators in their marketing campaigns.
![Page 15: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/15.jpg)
Measure download speed in connection with web and streaming performance
2) Web test
• Load the first page of the two most popular websites in Norway
• Dash streaming a 30‐seconds long video that is divided into 15 chunks with bit rate from 100kb/s to 20Mb/s
![Page 16: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/16.jpg)
More meaningful metrics are needed
1
10
100
Speed Mb/s
Node
Speed varies widely although the nodes are stationary!
0.1
1
10
1 10 100 1000
Median video bit rate Mbit/s
Speed Mbit/s
And that happens over a very short time scale.
![Page 17: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/17.jpg)
More meaningful metrics are needed
1
10
100
Speed Mb/s
Node
Speed varies widely although the nodes are stationary!
0.1
1
10
1 10 100 1000
Median video bit rate Mbit/s
Speed Mbit/s
And that happens over a very short time scale.
![Page 18: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/18.jpg)
Dedicated end‐to‐end measurements are indispensible to all key stake‐holders
• Informing policy can happen in different ways including: ‐ Publishing periodic reports‐ Reporting on incidents ‐ Take initiative to contribute to ongoing debates
• The needed input does not necessarily map to exciting research problems
• This is a whole different kind of impact • There is however still a number of open difficult
questions – This is likely to get more exciting
![Page 19: Measurements to inform policy makers and end consumers · indispensible to all key stake‐holders • Informing policy can happen in different ways including: ‐Publishing periodic](https://reader033.vdocument.in/reader033/viewer/2022042114/5e910195d4392b4c1d2a92f5/html5/thumbnails/19.jpg)
Some interesting questions that can have policy implications
• Vulnerability of digital infrastructures to attacks and large scale failures
• Digital sovereignty. What if we put up a digital border?
• Measure new technologies like VoLTE, NB‐IoT
• Congestion in cellular networks
• Access security in cellular networks