cable-tec expo 2018...© 2018 scte•isbe and ncta. all rights reserved. | scte.org• isbe.org 1...
TRANSCRIPT
![Page 1: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/1.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 1
C A B L E - T E C E X P O ® 2 0 1 8
![Page 2: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/2.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 2
D I G I T I Z I N G T H E C U S T O M E R E X P E R I E N C E
Andrew SundelinDirector, Product ManagementGuavus
Embracing Service Delivery Changes with Machine Learning
![Page 3: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/3.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 3
Agenda
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
The Change Imperative
Techniques Operationalization
• Change-Driven Segmentation
• Change Scoring
• Example
![Page 4: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/4.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 4
The Change Imperative
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
“I feel the need for speed”
• More services than ever before• More devices than ever before• More complex network than ever before
• More changes than ever before
• Subscribers expect change at internet speed &the business is pushing to meet that expectation
![Page 5: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/5.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 5
“There are many different types of change and all need to be taken into account to be able to establish potential root causes for any given incident. Among the most important types of change are code changes, data changes, workload changes and infrastructure topology changes. While not all performance incidents can be traced to such changes, Gartner estimates that approximately 85% of all performance incidents can be so traced.”*
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
Will CappelliResearch VP, Gartner
* “Causal Analysis Makes Availability and Performance Data Actionable”,
Gartner Report, 7 Oct 2015
Embrace Change – Don’t Fear It
Key is systematically identifying subscriber-related changes & evaluating them
![Page 6: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/6.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 6
Change-Driven Segmentation
Divide subscribers into subscriber segments with identical changes
Example subscriber-related attributes
• CPE & Aggregation Device Attributes
• Make/model software version of any CPE; config file
• Make/model software version of CMTS/CCAP/DPoE System, Remote Phy, etc.
• Topology Attributes
• Fiber node (node split, fiber deep), Aggregation Device split/change
• Video controller, email server, etc. – don’t need full connectivity just subscriber
association
• Billing/Service Attributes
• Billing system, speed tier, All IP, etc. even changes intended to be transparent
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 7: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/7.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 7
Change Time Variation
Align like changes relative to the time of the change itself
• Treat the date of the change as t0• Evaluate based on scoring before and after t0
• e.g. t0 +/-1 day, +/- 2 days,+/- 7 days, etc. much like looking at new sub tenancy
• Some changes may yield obvious & dramatic impacts• Other change impacts may be more subtle
• Sample size (i.e. size of the changed subscriber segment) may need to be/grow large enough to identify statistically significant before/after deviations
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 8: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/8.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 8
Scoring ChangesDirect Measures
Ultimate goal generally to evaluate impact on subscriber QoE• Technical Support Call Rate (Agent Handled) • Customer Reported Trouble Ticket Rate• Scheduled Truck Roll Rate
Downside of these care-drive direct measures is customer impacted• Scoring based on predictive measures of QoE preferable
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 9: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/9.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 9
Scoring Changes
Predictive/Indirect Measures – The Legacy Method
Lots of service- & network-layer telemetry data available
But at what level do these measures start to impact the delivered service?
• e.g. corrected codewords – clearly indicate a problem but at what level does it impact
QoE?
Old way of determining “right” level of service/network telemetry impacting subs
Convene a bunch of RF experts, choose a region, gather a lot of data and “do a study”
• Output of the study – thresholds for SNR, CER, etc. for subscriber “in spec”
Limitations
• Generally performed infrequently, on a subset of the network
• Expensive & time consuming – doesn’t keep up with changes to user behavior & apps
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 10: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/10.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 10
Scoring Changes
Predictive/Indirect Measures – The Machine Learning Way
Key to determining when telemetry/other data is predictive of QoE impacts – Machine Learning
• Automatic feature selection used to determine which service/network measures are most predictive of direct QoE measures
• Historic service-/network-telemetry data for these measures are utilized to train the model
• Current service-/network-telemetry measures (aka “features”) input to model to evaluate how predictive data is of poor QoE measures
• The output of the model is the “score”• Time-series telemetry data before & after the change yields a time series of scores
• Trend of before/after scores can be compared or representative score calculated
• Model automatically retrained periodically to keep up with changes in user behavior & apps
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 11: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/11.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 11
Too many changes to manually review trends
• Need a summary score for different time periods
• Automatically identify changes which negatively impact QoE
Big data, machine learning & streaming analytics are key• Lots of service & network attributes on per sub basis
critical for change-driven segmentation• Scoring relies on direct and predictive measures
of QoE requiring historic & current data• Machine learning critical for getting ahead of calls &
ensuring current QoE sensitivityDoing this at scale is non-trivial
Operationalization
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 12: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/12.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 12
Example Before/After Scores on Change-Driven Segments
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
![Page 13: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/13.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 13
E m br a c i ng S e r v i c e D e l i v e r y C ha nge s w i t h M a c h i ne Le a r n i ng
Customers demanding change at faster pace
Operators can embrace change with
right tools in placeNeed for analytics
• Change-Driven Subscriber Segmentation
• Machine-Learning Driven Change Scoring
• When you change things stuff breaks
• No lab can fully replicate production
• Big data, machine learning & streaming analytics are key to realizing this vision
• Not quite rocket science, but very difficult to do at scale
![Page 14: CABLE-TEC EXPO 2018...© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org• isbe.org 1 CABLE-TEC EXPO® 2018](https://reader034.vdocument.in/reader034/viewer/2022050103/5f41b57bc4f716312b58583e/html5/thumbnails/14.jpg)
© 2018 SCTE•ISBE and NCTA. All rights reserved. | scte.org • isbe.org 14
Andrew [email protected]
Thank You!