virtual and augmented reality – how do they affect … · figure 1 - gartner 2015 hype curve...
TRANSCRIPT
VIRTUALANDAUGMENTEDREALITY–HOWDOTHEYAFFECTTHECURRENTSERVICEDELIVERYANDHOMEANDNETWORKARCHITECTURES?
CHARLESCHEEVERS,MARKBUGAJSKI,AJAYLUTHRA,SEANMCCARTHY,PAULMORONEY,KEVINWIRICK
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 2
TABLEOFCONTENTSINTRODUCTION.............................................................................................3DESCRIPTIONOFTHEVIRTUALREALITYSOLUTIONANDAREASOFIMPORTANCEFORTHESERVICEPROVIDER...................................................5NewCameraTechnologies.............................................................................................7
NewStorageTechnologies.............................................................................................7
ContentStitching,Encoding,CompressionandMapping..............................................8
TheDeliveryNetwork...................................................................................................10
TheFOVHeadsetSceneGeneratorakaPCGameConsoleortheFutureVRSTB........10
TheVRHeadMountedDevice(HMD)..........................................................................11
OPPORTUNITIESFORTHECABLEINDUSTRYANDVIRTUALREALITYSOLUTIONS..................................................................................................14VRandARContentDelivery.........................................................................................15
AvidVRUserswillbeMoreLikelytoSubscribetoGbpsBroadbandServices.........15
VRTurboDownloadMode.......................................................................................15
VRFileorPre-emptiveSceneCaching......................................................................16
MulticastCarouselsforPopularVRTitleDownload.................................................16
LiveVRUnicastSolutions.........................................................................................17
HybridMulticastSolutionforLiveVREfficiencies....................................................22
InHomeConnectivitytoHMDandVRServer/RenderingDevice................................22
SymbioticWi-FiDeliveryandQoS/QoEforVirtualReality.......................................22
60GHzRoutingAddedtoServiceProvidersGatewayorInRoomDeviceStrategy.26
Pre-emptiveCachingOpportunityforNextRenderedVRScene.............................28
MergingtheVRandTVExperience–HMDand/orTVand/orPC/Tablet................29
CONCLUSION...............................................................................................30ABBREVIATIONS...........................................................................................32RELATEDREADINGS.....................................................................................34REFERENCES................................................................................................35
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 3
INTRODUCTIONAswemoveforwardwithnewformsofimmersiveentertainment,aswellasinteractivitywithcloudservicesandeverydaytasks,oneofthekeytechnologiesemergingisthebendingofrealityasoureyeandbrainperceivesittocreateanothernewlevelofentertainmentandinteractionwithoureverydaysurroundings.Thistechnologycomesinanumberofforms,butwetypicallygroupthetechnologiesanddirectionsintotwomaincategories:
(i) VirtualReality(VR)–Thedigitalcreationofafabricatedimmersiveworld,typicallyviaaheadsettechnology,thatgeneratesallthephotonsthattheeyesees.
(ii) AugmentedReality(AR)–Thedigitalcreationofafabricatedsetofobjectsthatcanbeinterspersedwithrealworldelements,usuallythroughaheadsetthatoverlaystheobjectsonthelens,astheuseralsoviewstheirrealsurroundings.
Athirdvarianthasbeendefinedbyseveralnewplayerstothemarketas:
(iii) MixedReality–Thisisafusionofbothvirtualrealityandaugmentedreality.
Fromthe2015publishedGartnerHypeCyclecurve[1]–wecanseethatvirtualrealityisataveryinterestingstageofitslife.VRisabouttoenterthe‘SlopeofEnlightenment’fromthe‘TroughofDisillusionment’–afterbeingaroundinoneshapeorotherfor20+years.However,withtheimprovementintechnologyandemergenceofanewVRHeadMountedDevice(HMD),VRispotentiallyreadytobecomeasignificantserviceinearlyadopterhouseholdsin2016andinto2017.
Augmentedrealityisalittlefurtherbehindvirtualreality;it’spotentiallyabouttomaketheturnupwardstothe‘SlopeofEnlightenmentPhase’asweleavethecurrenteraofexperimentationandnowsignificantVentureCapitalinvestmentandmoveintothefirstrealproductsandcontentforVR,AR,andMRandthenewexperiencescreatedmayinfluenceeventuallymoreaspectsofaperson’sdailyactivities.TheimplementationsofARhavetheadditionaltechnicalchallengeofusingcamerasandsensorstomapandsenseroomsurfacesandobjectstooverlaytheaugmentationelements,aswellastrackingthesixdegreesoffreedomfromtheeye–tryingtocreateablendedexperienceofwhatisreallyinviewandwhatisoverlaidsothattheeyefindsithardtodiscernthedifference.ThispaperwillfocusonVRmorethanARgivenVR’spresentrelevanceforuseingaming,adultcontentviewing,andthepotentialviewingoflivesportingevents.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 4
Thestickinessofthetechnologyhastobeproven,yetthereisagrowingbeliefthatwearenowonthepathtomakingbothvirtualandaugmentedrealityusefulinourlives.Inparticular,therehasbeengoodprogresswithtrialingofprototypesofVirtualRealitydevices.Thishasstimulatedmuchinterestindrivingtheconceptstoproductsandthisyearanumberofseriousplayersinthegamingandentertainmentareasarebringingtheirproductstomarket,aswellasAugmentedandVirtualRealityVentureCapitalinvestmentrisingtolevelswherethereislikelytobesignificantproductandcontentbreakthroughsoverthenext2-5years.
Ifthephysicsofthetechnologycanmakethebrainbelievewhattheeyeistakinginfromtheartificialdeviceandworld–andallowtheusertostayinthisworldformorethan15minutes–thenthetechnologymaystickandevolvetobeausefulpartofourworkandentertainmentlives.WhiletheinitialsetofdeviceslaunchingthisyearwillnodoubtfallalittleshortofalltheparametersrequiredtomakeVRandARwork,itisexpectedthatthesedeviceswillalsoservetofuelenoughacceptancetodrivefurthergenerationsofdeviceanddelivery.Thisinturnwillforcetheacceptanceofthetechnologythroughhigherresolution,higherframerates,lowerlatencyforpitch,yaw,androllderivedscenegeneration,non-blurringphotonsintheHeadMountedDevicescreen,lighterHMD,andnon-tethereduseofthetechnology.
Whatdoesthisnewtechnologicalexperiencemeanforthecableandconnectivityindustry?Thispaperwillattempttodefinethetechnologyandthecause&effectopportunityofVRandARonthedeliverynetworkandthehomenetwork.ThispaperwillfocusonthebandwidthrequiredtomakethetechnologysuccessfulandonsomeoftheideasthatwillmakeVRandARmoresymbioticintheserviceprovidernetwork.Italsoproposessomewaystheserviceprovidercanworkwithgameconsoleandheadsetmanufacturerstoprovidethebestchanceforthistechnologytosucceed.LiveVRandcloudtoheadsetARsolutionswillrequirethecontentdeliverycompany,theserviceprovider,andtheheadsetmanufacturerstoallworktogetherandunderstandtheend-to-endsolutionmetricsandqualityofexperience.Thisissomethingweexploremoreofinthispaper.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 5
Figure1-Gartner2015HypeCurve
DESCRIPTIONOFTHEVIRTUALREALITYSOLUTIONANDAREASOFIMPORTANCEFORTHESERVICEPROVIDERVirtualrealityimmersivemultimedia,orcomputer-simulatedreality,isacomputertechnologythatreplicatesanenvironment,realorimagined,andsimulatesauser'sphysicalpresenceandenvironmentinawaythatallowstheusertointeractwithit.VRtypicallyrequiresaheadsetthatcontainsascreenandlenstoimmersetheuserintothesimulatedenvironment.ItcurrentlyneedsadedicatedPCorgameconsoletogeneratethe3DsimulatedgraphicsandvideofortheheadsetalthoughtherehasbeenstreamingcontentavailableforsometimeandcompaniesnowexperimentingwithliveVRtransmission.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 6
Figure2-ConceptualViewofWhatMostPeopleThinkofVRExperience
OnekeyelementofthispaperistoexplorehowVRaffectstheserviceproviderbothfromtheobviousadditionalbandwidthoflargefilesbeingsenttoconsolesandPCsandalsoifliveVRcangenerateopportunitiesfornewsolutionslikemulticastVRorsourcetoheadsetallowingnewcaching,pre-emption,andstitching.ThemigrationofheadsetsfromHDMI/USBtetheredconnectionstoawirelessconnectionisalsosomethingthatmayofferanopportunityforserviceproviderstoaddinsupportforheadsetsaspartofthewirelessdevicessupportedfromtheirgatewayandextenders.Thismaybedoneprimarilyinthe802.11axramporpotentiallyusing60GHztechnology–whichoffersahugeopportunitytoservicetheheadsetwith20GbpsofwirelessconnectionwithnewIEEE802.11ayunderwayasthenextgenerationtocurrent802.11ad.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 7
Figure3-OverallVRAcquisitionandDeliverySystem
Thevirtualrealityend-to-endsolutionchainisdefinedbelowinFigure3.Itcomprises:
NewCameraTechnologiesCamerasthatcancapturethesceneinstereoscopic360degreeformatsandatthehighestresolutionandframespersecondpossiblearebeingdeveloped.Mostdeploymentstodayuseaconcaveviewwithmulti-lenscameras.VRvideotodaycanbe180degreesor360degrees.For360degreesystemsthefieldofviewratiotothecapturedvideoisabout20%with180degreesystemitscloserto33%offeringhigherresolutions.
NewStorageTechnologiesStoragetechnologiesfocusedonsolidstatedevicestoallowthehighuncompressedvideotoberecordedandreadyforcompressionprocess.ForliveVRitmayrequirethateverythingfromstadiumstooutsidebroadcasttruckshavebothnewstoragesolutionsforsubstantiallymorevideocontent,andhigherspeednetworkconnectionstootherdatacentersthatalsosupportprocessingtherawvideofeedsforeitherunicastfoveatedvideodistributionorpotentiallymulticaststreamingofallcamerastootherlocationsforVRsceneprocessingtotheindividualusersheadset.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 8
Figure4-FoveatedRenderingofVRScene
ContentStitching,Encoding,CompressionandMappingForprocessingfoveatedscenesfortheliveVRstreamingofnon-livefilebasedVRexperiences,theindividualcamerastreamsareencodedandformattedtoallowtheheadsetattachedPCorgamingdevicetocreatetherequiredscenesincludingtheassociated3Daudiowheresupported.LiveVRalsorequiresthestitchingofthecamerafeedstopresenttheviewwheretheenduserislooking–calledtheRegionofInterest(ROI).Stitchinginreal-time,eitherinthecloudoronthegameconsole/PC,istypicallydimensionedatthemomentfor4-61080pstreamsintoonevideostreamofupto2160p608-bitresolution.
ROIMappingisanotherareaofavidresearchwithmappingthe180or360degreeviewstoequirectangularprojectionorpolygons.Forexample,somecompaniesarelookingatpyramidalmappingclaiming80%improvementsinbandwidth.Thisinventionismakingitmoredifficulttoensureinteroperability.WithITU,MPEG,andDVBalsogettinginvolvedwithVRstandards,itwillbeinterestingtoseewhereinnovationandstandardsintersect.EncodingtriesalsotofollowtheROIandthemappingalgorithmstoimprovereal-timespeedsandtherelevanceoftheoutputVRstream.
ManybelievethatforVRtobesuccessful,theentireviewableareaneedstohavemorethan16kx12koreven32kx24kpixels.ConsideranexamplecaseofonehighqualityVRsystemwithaviewingrangeof150x120degrees.Itmayneedmorethan16,000x12,000pixelstocoverthatrange.InthedeliveryarchitecturewheretheentireVRsceneissenttoaclient,assumingnon-stereoscopic(2D)capture,10bitsperpixeland60fps,thescenewillhave16,000x12,000x60x30(>300Gbps)bitspersecofraw(uncompressed)bitrate.Evenifitisassumedthatacompressionratioofabout300:1is
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 9
used(thisisofaboutthesameorderasusedindigitalTVdistribution),itgivesabout>1Gbpscompressedbitrate(farhigherthanwhatisgettingusedtodayinlowqualityVR).
Inreality,inthatapproachthatratemaybeevensignificantlyhigherasaquickresponsewithlowlatencyrequirementmayprohibitthathigh(~300)ofacompressionratio.Onotherhand,inaverylowendsystemasdonetodayinmanydemonstrations,theentireVRsceneislimitedto4kandthebitratesusedareofthesameorderasincurrent4kTV(10to20Mbps).However,itiswidelybelievedthatthecurrentversionsofVRdeliveryapproachesandplatformsareveryinadequatetoprovidethemuchneededqualityofexperienceforittobesuccessful.
Therearesomestudieswhichshowthatmanypeoplefeeluncomfortablebeyond15minofviewingVRscenes.Onekeyconcernis“simulatorsickness”whichincludesvertigo,discomfort,headaches,andmotion-sicknesslikesymptoms.SimulatorsicknessisfairlycommonwiththecurrentgenerationofVRcontentandgear.However,itisquitepossiblethatsimulatorsicknessissueswillgoawaywithtimeasthecontentanddisplayindustrieslearnthecraftofcreatinggoodVR.Itisknownthatakeyfactorinavoidingsimulatorsicknessforlongerduration(>15min)viewingistohaveverygoodvisualquality.
Thisleadstorequiringsignificantlyhigherspatialresolutionandhigherframeratesincomparisontowhatisdonetodayinmanydemonstrations.47,000pixelsperdegreeisbeingproposedbysomeastheminimumrequiredpixels/degree.Inaddition,tominimizethesimulatorsickness,thedelayinresponsetotheheadmovementneedstobeverysmall.Itisnotclearexactlyhowsmallthatneedstobe,butitisbelievedbyseveralexpertsthatitneedstolessthanabout30milliseconds(ms)whichisfarlessthanthe100sofmsinmanydemonstrationsshowntoday.Thedelaymayevenneedtobesignificantlysmallerthan30ms,butitishardtoquantifythisnumberduetothelackofcomprehensiveformalstudiesonthistopic.This30mstimeissomethingthatcanpotentiallybeaccomplishedfromthenetworkwithfiberandevenDOCSISlatencybeingabletosupport.However,itassumesthatnotonlythenetwork,butalsopotentiallystitchingandevenWi-FiandTCPlatencyissueshavetobereallyunderstoodandtuned.
Sincethemajorityofviewersarenowusedtoseeinghighresolutionvideo,toavoidsimulatorsicknesshighvisualqualityisnecessary.Hence,thebitratesrequiredtostreamVRvideo,inthedeliveryarchitecturewhereentireVRsceneisdownloaded,willbeseveralhundredsofmegabitspersecondorevengigabitspersecond.ThiswilllikelyrequirealternativedeliveryarchitectureswhereentireVRscenesarenotdownloaded.Therearevariousoptionsinthatregardbeingdiscussed,buttheyareinveryembryonicstage–typicallyfocusingonpre-emptionofheadmovementortrackingusermotion.Discussionsincludeconceptslikestreamingaselectedpartofthescenedependinguponwhatistheviewingangleorhavingavariableresolutionofthescenedependingupontheviewingangleandhybriddeliverynetworkarchitectureswheresomepartofthesceneiscachedlocallyandothersarestreamedasneeded.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 10
TheMPEGgrouphasalsoalreadystartedtheworkitemofcarryingVRstreamsinMPEGDASH.ThegroupisexploringthetopicofwhatkindofVRrelatedsignalingneedtobeincluded.MPEGhasalsoinitiatedMPEGOmnidirectionalMediaApplicationFormat(OMAF)standardizationforstorageanddelivery.
TheDeliveryNetworkWhethercontentisliveornot–networkswillexperienceanincreaseintrafficasVRheadsetsareadoptedandused.Thiswillhaveaneffectonthecore,edgedevices,thenetworkitself,andperhapseventhegatewaysinthehome.(Thatlasttopicwillbediscussedfurtherinthispaper.)VRbroadcastmightusetransportstreamdeliverywhichwillallowaregularset-topbox(STB)todisplayVRonaTV.Today,thisispossiblewiththeDVBUHD-1specification.UnicastofVRisexpectedtouseDASHISOBMFFforequirectangularandtiledtransmission.Therehasbeensomediscussionofhavingsomethinglike30camerastreamswith5ABRprofiles–creatingasmanyas150feedsorfilestosupport–particularlyforstreamingovertheunreliableinternet.Onnetsolutionscouldleveragethequalityofservice(QoS)tohaveasingleprofileandleverageDOCSISQoSmechanisms.Thisprofilecouldbeaffectedwhentheheadsetdeviceisun-tetheredwhileusingWi-FiorotherWirelesssolutions.Thiscouldloweravailableairtimewhenthevideosceneistransmitted.
TheFOVHeadsetSceneGeneratorakaPCGameConsoleortheFutureVRSTBThefieldofview(FOV)devicecanprocessallthevideocameraanglesandproducethe110degreelowlatency,pitch,andyawcorrectedvideofortheheadset.Ithastostitcheachcameraviewtogetherfordisplayontheheadsetunder30msandcloserto15ms.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 11
Figure5-CubeandPyramidalMappingof360DegreeScenes
TheVRHeadMountedDevice(HMD)TwoofthemaincomplaintsagainstheadmounteddevicesforVRarethattheyareheavyandwired.Variouscompaniesareworkingonaddressingtheseissues.Thereareanumberofissuestosolveincluding:
1. Whatwirelesstechnologywillsupportthebitratesrequired?DoesthePC/consoledotherenderingortheHMD?
2. Whatwillbethehighspeedvideowirelesstransmissionsolution?TheJPEGcommittee’smostrecentefforttowardsdevelopinganewstandardforthis,JPEGXS,hasthisapplication(amongotherslikemobiledeviceorset-topboxtoTVwirelessconnection,etc.)inthecrosshairswherelowcost,lowpower,andlowdelaycompressioncanhelptowardsachievingwirelessheadmounteddisplay.SomeexperimentsarealsobeingconductedtoshowtheVRvideoonaTVscreen.Theydemonstrationscurrentlylookmorelikemulti-viewTVorfree-viewTV.ThemuchlargernumberofdeployedTVsthanthenumberofdeployedVRheadsetsandlackofviewingsicknessprobleminTVviewingisencouragingsometopushfartherinthisdirection.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 12
Figure6-LargeVRFileDownloadExperience
ThecurrentVRexperiencecanbeprobablyclassifiedastwo-fold:
(i) Adownloadoflargefile(s)tothegameconsoleorPC(ii) StreamingvideofromsiteslikeYouTube;theseareoftenencodedwithABR
profilestoallowforuseovermobileandconstrainedwirelessnetworksincludingWi-Fi,typicallyat720Pandsometimes1080presolutions.
AsyoucanseefromthenumbersinFigure6,theexperienceofatypicalusertoday:
• AtthesmallersideofVRexperience–6GB–itcantakeover8minutestodownloadthefile–evenon100Mbpsservices
• Ifweassumeareasonablelevelof2017VRexperienceat100GBofcontentwithhighqualityvideoaswellaslocalimagegeneration–theexperienceat100Mbpswouldbeverypoorhavingtowaitalmost2.5hoursbeforeenjoyingtheVRexperiencegeneratedfromthelocalsystem.Evenat1Gbpsofnetworkbandwidth–thefilewouldstilltakeoveralmost15minutestodownload–soeventhen,it’stoomuchtimetowait
• LiveVRandstreamingthecontentat100Mbps–ifusingpre-emptionofmovementandfoveatedrendering–canstillonlycopewithmoderatesub4KresolutionsandwemaybemovingtosolutionswhereGbpsspeedscanreallyonlygivethattruly‘presence’basedexperienceandasolutionthatallowsimmersionformorethan15minuteswithoutnausea
o Itisalsouncleariflivesportswilltranslatewelltoconsumers.Therehasbeengreatprogressincameracapturetechnology,butitseemsthatthe
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 13
ultimatesportsexperiencemaybethecombinationofimmersiveactionfrommultiplecameralocations(includinginterpolatingwithintheactionitself)andwhatthedirectorshootsin2D.
Forexample,watchingatennisgamefromthebestseatatthecenterline–lookingleftandrighttotracktheballandthencuttingtothecloseupofthelosingplayershowingdisappointment–wouldbeacombinationofVRandinserting2DcloseupsintotheHMD.Football,soccer,oranysportbecomesanopportunityforpromoterstosellprimeseatstoanynumberoffans.Theopportunitymayalsotranslatetoplays,concertsandanyotherformsofentertainment
Figure7-BestSeatintheHouseDeliveredtoYour(and1Million)OtherSofas
o Theotherexperienceconsideredisusingtheflat2DTVtoallowtheusertopanaroundthegameandcontroltheviewusingsomehandheldgyrometerorhandgestures.Itremainstobeseenwhetherthisoffersauserexperiencethatwillcaptureenoughoftheviewingmarket
• TherearealsoattemptstodevelopparallelVRexperiencesinadditiontotheregular2DTVviewing.ThisparalleluseofVRtoprovideengagementsnippetsof15minutesorless,alsoneedstobeprovenwiththeconsumer.ThegoalistogiveviewersadditionalimmersionintheoriginalshowbycapturingelementsinVRandthenallowingtheviewertoimmersethemselvesintokeyareasandevents.Ineachsnippet,theviewercouldpotentiallydiscoveradditionalfeatures,plotpoints,storyaspects,orevengaintheabilitytovisitwherethecharacters
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 14
havebeen.Thiscouldleadtoanincreaseinviewerloyaltyandpresentopportunitiesforcontests,aswellas,perhapsevenmeetingtheactors“live”viaVR.
Figure8-End-to-EndLiveVR:KeyElements
Figure8highlightssomeofthekeychallengesofend-to-endliveVR–inparticular,astheresolutionincreasesandproviding180or360stereoscopicviews.Atthemoment,thesolutionsareunicasttotheVRapplicationwhichisloadedonsomethinglikeasmartphoneusingaVRheadset,oraPC/gameconsoleconnectedtoatetheredHMD.ThereareanumberofcompaniesemergingtryingtospecializeinthetransmissionofliveVRandthistechniqueandtechnologyissettoimproveasthe“LiveExperience”isexploredforvideo,surroundaudio,andtactile/hapticexperiences.
OPPORTUNITIESFORTHECABLEINDUSTRYANDVIRTUALREALITYSOLUTIONSIt’stimetoreviewthepotentialopportunitiesforvirtualrealityandaugmentedrealityapplicationsintheMSOprovidedhomeuserexperience.Theassumptionhereisthatthesolutionsproposedwillbemadetoworkandtheissuesoftheexperienceandqualitywillimprove–particularly,withsymbioticrelationshipsbetweencontentcreators,VRsolutionproviders,andtheserviceproviders.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 15
Thisopportunitypotentiallybreaksoutintothefollowingareas:
(i) VRandARContentDeliverya. ImprovementsindownloadspeedsforlargeVRfilesb. ImprovementsindeliveryandlatencyforliveVRc. UnicastandMulticastVR–isthereopportunitythere
(ii) InHomeConnectivitytoHMDandVRserver(PC/console)
a. SymbioticWi-FideliveryandQoS/QoEforvirtualrealityb. 60GHzroutingaddedtoserviceprovidersgatewayorinroomdevice
strategyc. Pre-emptivecachingopportunityfornextrenderedVRscened. MergingtheVRandTVexperience–HMDand/orTVand/orPC/tablet
VRandARContentDeliveryAsshownabove,thefilesizesforVRfilesarelargerthanhaseverseenbeforeforfullHDvideomoviesorthelatest2Dor3Dgames.Assuch,ifVRorARissuccessful,manymultipleversionsoflargefileswillbedownloaded.Theexperienceat100Mbpswaitingalmost2.5hourstodownloadwilldriveVRgamerstoservicesthatimprovetheexperience.
AvidVRUserswillbeMoreLikelytoSubscribetoGbpsBroadbandServicesAnobviouscorrelationtoVRandHSDservicesisthatanavidVRuserwilllikelypayforthelowestlatencyandhighbandwidthtierstoimprovetheoveralltimetooptimizedownloadofVRfilesandfutureliveVRexperiencetobestpossiblelevels.ThisshouldprovetobeinterestinginthefuturetocorrelateVRandARusageagainsttheHSDtierlevelsofferedandinthefuturepotentialwirelessrouter/spectrumcollaborationbetweentheServiceProviderandtheHMDorGameConsoleproviders.MappingQoSfromthenetworktotheendphotonwillcertainlyhavethebenefitoftuningtheoverallVRexperienceforlowestpossiblelatencyandbestpossibleenjoyment.
VRTurboDownloadModeForthosesubscribersnotonGbpsspeedswhowanttheirfilesdownloadedquicker–anMSOcouldofferaspecialadditionalfeeservicetoallowadditionalburstspeedsforthedownloadofthefile.We’llcallitTurboModeforVRinstantaneousgratification.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 16
Figure9-VRTurboModeofDOCSISQoS
VRFileorPre-emptiveSceneCachingWithlargemultiplefiledownloadstherehasalwaysbeenthepotentialtocachethefileslocallyratherthanpullthemmultipletimesfromthesourcethroughanexpensiveCDN.WithVR,thesolutionlooksevenmoreattractivetocacheVRfileslocallyintheserviceprovidernetwork.
Figure10-LocalCDNcachingofVRfiles
MulticastCarouselsforPopularVRTitleDownloadConsidercreatingascopeforthereleaseofnewVRtitles–toalsoofferamulticastwindowonthenetworkforhalfaday–toallowuserstodownloadevenmoreefficiently.Thisseemstobeworthlookingat,especiallyifthegamingmarketforVRhasthefirstdayandweekreleaserampasthecurrenthighprofilegameslaunches.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 17
Figure11-MulticastVRFileCarousel
LiveVRUnicastSolutionsLiveVRisextremelydependentonlowlatencysolutionandarchitecture.CantheDOCSISnetworkbetunedtosupportalowlatencypathfromHMDtrackingtoanewVRscenesenttotheheadset<20-15msthroughthenetworkandoverwiredorwirelessconnection?Theanswerisstillunclear.It’sprobably,“Yes,”forlowresolutionandframerates,butaswemovetohigherresolutionsandframeratesthenitwillcertainlyhavetotracklowernodesubscribercountsandintheeraoffullduplexDOCSISandfiberPONgrowthto64orlesshomes.Inthemeantime,for720p-1080pliveVRtransmissions-DOCSISQoSmechanismswillhavetobeusedtoensurethislowlatency.DOCSISbesteffortsolutionsareoftentheonlyconfigurationthatanoperatorsetsupoutsidesomespecifictreatmentofvoicecalls.
LetusstepbackandtakealookattheelementsrequiredforVRtowork–tocreateanimmersiveand‘present’experiencewhilereducingthelikelihoodofsimulatorsickness:
• Resolutiontosatisfythefoveaandperipheryoftheretinaarebeingresearchedo Thehigh-acuitycenterofvisioncanresolveuptoabout60pixelsper
visualdegree(equivalenttoa1080pHDdisplayspanningabout30degreesofone’svisualfield,whichisthetypicalHDTVviewingcondition).Thefieldofviewforeachofoureyesis~170degreeshorizontallyand~150degreesvertically.Thecentralbinocularoverlapofoureyes’fieldofvisionis~120degrees,andthetotalmonocularfieldofvisionis~210degrees.ThecurrentgenerationofHMDspresentsa~100x100degreeviewtoeacheye.Peopletypicallyuseonlytheireyestolookaroundthecentral~40degreesofthevisualfield,andusecombinedheadandeyemotionstolookaroundlargerareas[2]
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 18
o Thus,itisprobablysafetoestimatethata~2kx2kdisplay(40x40degreeat60pixelsperdegree)wouldbejustsufficientforbinocularhigh-acuityviewingandeye-onlymovements.(Ifthesamepixeldensitywereusedtocovertheentire~210x150degreefieldofvision,a12kx9kdisplaywouldbeneeded,thoughtheencodedimagecouldhavelowereffectiveresolutionoutsidethecentral40x40degreeregion.)RecentworkindicatesthatadynamicFOVreducessimulatorsickness[3]
o Currentlymostofthedevicesuse1280x720pdisplaystopresenta~100x100degreeviewtoeacheye.Thus,theresolutionattheleveloftheretinais7to13pixelsperdegreeatbest–whichiswellbelowthe60pixelsperdegreeneedtomakepixelsessentiallydisappear
• Lowpixelpersistenceo Toavoidblurringorstreakingwitheyemotionrequiresthatthepixel
persist(remainslit)fornomorethan3ms.ThisismorerelevantforVRHMD,becauseoftherelationoftheeyemovementtothedistancefromtheVRHMD.Thelongerpixelspersistthefarthertheysmearacrosstheretinaandtheblurrierthescenebecomes
o Unfortunately,thereisarequirementtodecreasethepixelpersisttimewithhigherresolutionswitha1kx1ksupporting3msandhigherpixeldensitiesrequiringshorterpersistence
• Refreshrateataleveltomaketheeyecomfortableo Correlatedwithpixelpersistence,thisisnecessarytoavoidflickerwhen
persistenceislowo Togainpresence95MHzworks,butitpresentssignificantchallengeto
maintainhighrefreshratesforstereorenderingwhenresolutionsincrease
• TrackingofHMDandeyeataccuratelevelso Hastosupportaccuratetrackingtoallowimagetranslationtox,y,zo Millimeterorlessprecisionand¼degreeaccuraterotationisrequired
• Opticallensandcalibrationo Oneortwolensespereyetotryandmaximizefocusonretinaofthe
projectedimages.ItisalsohardertoachievemultiplelenseswiththerestrictionsofweightandindustrialdesignandtheviewingdistanceoftheHMDscreen
o Thisstillleavesaproblemwithfocallength,distortion,andaberrationso Aprocessthataccuratelycharacterizesthelensesandcorrectingthe
renderedimageisrequiredinthesolution• Lowlatency
o Wehavebeenusing15msasthebenchmarkthroughoutthisdocument,buttheresultsarereasonableat20ms.Thismeansthevirtualimageneedstobeintherightplaceattherighttime.Thislatencymeasurement
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 19
isafunctionfromthetimetheheadandHMDmovetothetimethelastphotonisemitted(resultinginpaintingthatnewimageoverthemotion)
• Positionaltrackingo AsyoumovearoundinVRscenes,thereneedstobeawaytoreportyour
locationtothescenegeneratoraccurately.Thereareanumberofschemesbeingconsidered–someincludepositioningmarkersaroundtheVRgamingareasuchasmarkingorallowingpositiontobetriangulated.TheseschemesmayuseeitherRFlocationorcamerarecognitionofthelocation.Inthefuture,betterdepthcamerasmaybeabletoworkwithlessaidspre-located
Figure12-CriticalParametersofImmersivePresencebasedVR
LetusnowlookattheDOCSISlatencycapabilities.Aswediscussed,VRwillprompttheuseofQoSpoliciestoDOCSISeithertoimprovedownloadspeedandexperience;moreimportantly,ifwearetosendliveVRtotheconsoleandattachedheadset.AsyoucanseefromFigure13,thelatencysensitiveloopistheconnectionfromthePC/console/VRSTBtotheHMD.Thereislatencyinencoding,mappingthefoveatedvideostream,andtheinternetadds30-50mswithadditionaltimeaddedbyDOCSIS(couldbeaslowas2msandasaverageat15ms).Additionally,wirelesstransmissiontime,aswellasthecomputingaframeofvideotoHDMI,couldincreasedelayby16-32ms.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 20
Figure13-LatencyFactorsfromVRContenttoHMDDevice
Figure13suggeststhattheentire180degreefieldofviewneedstobedeliveredforlocalrendering;whichrequiresusinglotsofbandwidth,aswellasleveragingthepre-emptivecachingofpredictivenextscene(left/rightoftennisgame).Figure14and15illustratetheseprojectedbandwidths.
Figure14-BandwidthforcurrentandfuturestereoscopicVRprojections
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 21
Figure15-180x90DegreeFOVSDtoUHDVRbandwidth
UsingDOCSISandunsolicitedgrantserviceflowsetup,thesebandwidthsarefeasible.Fortheminimumlatenciesforasingleframe(17msfor60fps)immersioncanbeachievedandthereisscopetoincreasetheburstprofiletoyield2xto10x,ormore,ofcachedsceneslocally.Thiswillimprovethelatencyattheexpenseofnetworkbandwidthanddecreasedstatisticalmultiplexingtolargersubscribergroups.xPONfibersolutionswiththeirsmallPONservicegroupsshouldalsobeabletoachieveroundtriplatency.
CompaniesspecializinginliveVRhavebeenworkingtoreducethebitratesforuseincurrentaccessnetworks;particularlyoverLTEandotherconstrainednetworks.Fromcapturesof6K@80fpstheycanencodedownto4Mbpsandstillclaimsomequalityofimmersiveandpresenceexperience.Thesetechnologiesfocusonfoveatedrenderingandpredictionofthemotionoftheheadoreye.Foveatedrendering,asexplainedpreviously,understandstheareathattheuserislookingatasitrelatestothefocusareaorfoveaoftheretina–producingahighresolutionandhighframerateversionofthisareaofthescenewhilereducingtheperipheryoftheviewtolowerbitrates.Thesedifferentialcompressionandtransmissiontechniquesallowdifferentportionsofthe360tobetransmittedatvaryingframeratesandresolutions.
RequiringauniqueviewperconsumerreducesthebandwidthtotheHMD,butwillbebandwidthintensiveiftheunicastcountofuniqueviewsincreases.Ofcourse,therewillbecommonviewareasaspeoplefocusontheactionof,forexamplealivesportsgame.Thereshouldbealargestatisticaloverlapforencoding,mapping,andprocessingofthevideo.However,thehighertheresolution,framerate,andwiderfieldofviewcoupled
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 22
withthe<20mslatencyrequirementtopresentthefirstnewframeofthesceneseemstorequirethattheentire180degree(potentially110degree)FOVneedstobedeliveredtogetherforlocalrenderingvsstitchedfromthePCorVRset-top.Thisimpliesbandwidthsreachingalmost600Mbpsfor4kp60stereoscopicVR.Thisalsoassumesreasonableimprovementsincompressionandmapping.Weexpecttheindustrytodriveevenmoreinnovationtoconstantlyimprovethisbandwidth/latency/qualityequation.Timewilltellifthecurrentlowerresolutionexperienceswillworkoriftheywillneedimprovementtobecomearealstickyuserapplication.
HybridMulticastSolutionforLiveVREfficienciesWhileitseemsmostofthefocusonmakingliveVRviableistoworkthroughfoveatedrenderingandpre-emptivetrackingofthenextfocuspointsoftheeye.Theremaybescopetolookatmulticastingforhighviewershipevents.Whileeachusercanhavetheirownspecificvisualexperience–thereislikelytobeacommonclusterofheadmovementindirectionsofplayandaction.ThistrackinganalysisofthefeedbackfromtheHMDscouldgeneratemulticastflowsofthecompressedscenestogetthemcachedattheedgeorattherenderingdevice(gateway,PC,gameconsole,set-top,etc.).Audiocuesandothervideopointsofinterestcouldalsobecandidatestomulticastthesescenesforstitchingandrenderinglocally.
InHomeConnectivitytoHMDandVRServer/RenderingDevice
SymbioticWi-FiDeliveryandQoS/QoEforVirtualRealityThehigherperformanceHMDsusewiredHDMIand/orUSB3.0connections.Notesomeprototypesofdeviceshavewithupto5cables!ThedecodingofthesceneistypicallydoneonaPCorgameconsoleandthensenttoHMDuncompressedfordirectrenderingoverHDMI.Lowerperformingsolutions,likesmartphonebasedVR,canoperatethescenedecodinglocallyonthesmartphoneitself.However,thisislimitedincapabilityforhigherresolution/framerateandscreenlatencyandit’sevenmorelimitedonthebandwidthfromthescenestothephoneoverWi-FiandLTEor3G.ThesesolutionsusuallysupportWi-Fionlyformoredeterministicperformance.Theyalsorelyondownloadedvideo,particularlyfor360andstereoscopicviewing,andalsoplayVRvideosdirectlyfromYouTubeatlowerresolution;typicallysupportingABRprofilestodealwithlatencyandcongestionissues.
ForsmartphonebasedVRsolutionsandsimilarwirelesssolutionsthereareanumberofopportunitiesforsymbioticandimprovedexperiencesthataserviceprovidercanexploretohelpuntethertheHMD.Thisisamajortechnologicalchallengeandonethatdoesnothaveasimplesolutionpath.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 23
Figure16-Wi-FiandVRHMD-VideoDecodingLatencyPreventsItsUse
ThecurrentapproachhasaPCrenderthevirtualworld,andalsouseWi-FitostreamtherenderedimagestotheHMDusingWi-Fidirect,Steamlink,castingsolutions,etc.Theproblemwiththisis,again,latency.Inthisscenario,videostreamingcangetdowntoa1msinlatencyforcompressedMPEGvideo,butanuncompressedvideostreamof1440P90Hzvideoisalmost8Gbpsofdata.AgoodWi-Fiaverageinahouseholdiscloserto100Mbpstoday,eventhoughweaspiretoGbpsinthefuture!UncompressedscenesoverWi-Fidonotworkandcompressionisrequired.Becausethecompressionanddecompressionprocesswilladdanywhereupto80msoflatencydependingonsolution–itfallswayoutsidethesimulatorsicknesswindowof<20msdesired.
Aswehaveexploredthroughoutthispaper,morebandwidthcansolvetheproblem.Fromservertorenderer,futurewirelesstechnologieslike5Gandmillimeterwaveareaimingfor10GbpsandTerabitnumbers.Thisisnotyetpractical.ThereareafewtechnologiessuchasLi-Fiwhichmayfindagoodnichetotransmit10Gbpsatlowlatencyespeciallyinthedownstreamdirection.OnecouldenvisionabankofLEDsontheserver/PC/STB,andusingthesurfaceoftheHMDastheopticalreceiveraperture,itcouldbeanapplicationthatworks.
Anotherapproach,whichishugelycomplex,istoacceptthelatencyandtrytocorrecttheoldimagetothenewheadposition.Thisstillproducessomedelay,butavoidsthenauseacreatedbyheadmotionlag.Thereisatermforthiscalledasynchronoustimewarp;whereasmallgraphicsprocessingunit(GPU)ontheHMDwarpstheoldimageintothenewperspectiveallowingafluidupdatedespitethedelay.Theproblemwiththisisthatittransferssimulatorsicknesstowhatiscalled‘positionaljudder’whereasyoumoveyourheadside-to-sideordoasimplerotationofthehead–youwillsee
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 24
judderonobjectsintheimagethatareclosetoyou.IncreasingGPUcorrection,youcanimplementamorecomprehensivewarpingwhichdoesbothtranslationandorientationcorrectiontotheoriginalrenderedframe.Thisneedstoincludethedepthandobjectdisplacementwhichinitselfcreatesartifactsatobjectedgeswithgapsandspaces.EvenfurtherGPUlogiccouldbeappliedtofillthesegapsinterpolatingwhattoputthere.
Thereareafewtechniquesbeingconsideredtosendmoreinformationtotryandfillthesegaps.Theseinclude
• Renderthenearandfarfieldobjectsseparatelyandsendseparatestreamsforboth
• Renderalightfieldwhichcanbere-projectedbylargeheadoffsetswithoutdis-occlusion.Howmuchthiscanminimizeartifactsisstillaresearchproject
• Usepredictionandmotionmodelstopredicthowtominimizedis-occlusionerrors
Figure17-TryingtoCreatePredictiveSolutionsforLowerLatency
RenderingpushedasclosetotheHMDaspossibleminimizeslatency.UltimatelyhavingtheHMDrenderitselfpotentiallyofferslowestlatencyandallowswirelesstransmission.SmartphonebasedVRworksthisway,butitdoesnothavethehorsepowerofmulti-processorgameconsolesorPCstodealwithhigherresolutionVRscenes.
Thetechnologyisimprovingandproductslikesomeoftheaugmentedrealitydevicesinprototypestageatthemomentalsodoestherenderinglocally.ThesesmartphonelevelGPUsrunattheirlimitformediocreVRandgenerateheat.InvestmentisgoingintocompaniesthatarealsotryingtodevelopHMDspecificGPUsforVRbyspecifically
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 25
targetingtheimageprocessingandrenderingrequiredfortheVRexperienceandtryingtokeeptheheatdown.
TheremaybetwoelementstotheHMDwheretheGPUsarecontainedsomewhereelseonthebody(onthewaistorinyourpocket)andthedisplayoftheHMDonlyneedstobeonyourhead.Thiscouldalsobewired.ManyofthecurrentVRgamerstodayactuallyuseabackpackfortheirPCwithbatteryandcrampthecableontheirbodytoallowfreedomofmovementforgameslikefirstpersonshooters.
OneimmediatewirelesstechnologythatcouldhaverealusesforVRis60GHz.SometimescalledWiGigandstandardizedas802.11ad.ThistechnologywasoriginallyconceivedwithoneusecasetobewirelessHDMI–toreplacetheHDMIHDcable.Ithasbeencommercializedandcanrunat5-7Gbpsinthecurrent802.11adgeneration.Itoffersthisperformancewithlineofsightandsufferssharpdropoffinperformancewhenobscuredbyobjectsincludingthehumanbodyandhead.Thefutureversionsofthestandardtodefine802.11aywilldrivethebandwidthevenfurtherto25Gbps.The802.11ad/aystandardsallowforalowlatencypotentialatunder20ms.However,onechallengecanbereformattingthevideotoresendtoawiredHDMIinput.
Figure18-60GHzCouldbetheKeytoUntetheredVRHMDConnectionTechnology
ApplyingQoStagstoVRvideosessionscanbedoneeitherdirectlyfromthenetworksideorbyrecognizingthevideoflowsand/orthesinkdevicetypes.Gamersarealwayskeenlyinterestedinthelowestlatencyandalsothebestuserexperience.TheymightpaymoretoapplyQoEimprovementstotheirexperiences.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 26
Wealreadydiscussedthiswithfiledownloadtimes,forWi-FibasedservicesMSOscouldalsoapplyairtimepolicesforVRtoincreasetheQoSanddecreasethelatency.ThiswillbeakeypartofaserviceprovideradditiontoVRdeliveryandarelationshipopportunitywithVRcontentexperienceprovidersbothliveandfilebased.
UsinglocalSSDbasedcachedevicesinthegateway(GW)thereisthepotentialtocachepre-emptivescenesforlocalstitchingandrendering.Again,theSDread/writetechnologydoesintroducelatency,butthetechnologytodayissub2msandimproving.Figure19showsatrivialhomewirelessarchitecturehighlightingtheseideas;hereWANbasedDSCPtagsforVRflowscouldbeusedtosetuppriorityqueuesinWi-Fiand60GHzairtimemanagers.802.11axmayprovetobeastrongimprovementforcompressedvideoVRtransmissionasitoffersmuchbetterdeterministicperformanceandallowstheVRrenderingdevicetobescheduledoverWi-Fi.Italsoimprovestheoverallbandwidthcapabilitieswith160MHzchannelsandincreasedclientWi-Fiantenna.Assuggestedbefore,60GHzoffersabetterscopeforuncompressedvideotransmissiontotheHMDtodecreaselatency.AssumingtheGPUperformanceoftheHMDstilllackstheabilitytorenderuncompressedHDMIover60GHzinashortertimeframe.
Figure19-PotentialFutureVRcapableGW/RouterforWirelessHMDConnection
60GHzRoutingAddedtoServiceProvidersGatewayorInRoomDeviceStrategyAswenoteinFigure19,60GHztransmissionstotheHMDseemtobesomethingtoexploreforanuntetheredVRexperience.Thishasapotentialtoaffecttheoverallhomearchitecturewiththeuseof60GHzbeingapotentialinclusionintheserviceprovidersdevices–forbothVRinclusion–andforinroom5Gbps+privateareanetworkcapacity.Initial60GHzdeploymentsforVRwillprobablybededicateddirectlinkfromVR
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 27
renderingdevicetotheheadsetandwillbecapableofsendingbothuncompressedandcompressedscenestotheHMD.However,as60GHzdevicesbecomemoreprevalent(seeFigure20)therewillbearequirementtoadd60GHzroutingtotheGWandtheextender/Hub.Thiswillallowfornon-directservertoasingleclientdeviceandtosharethe60GHzISMradiobandsofspectrumbetweenVRandotherapplications.
Figure20-60GHzAdditionstotheHome:PotentialforInRoom&IntraRoomNetworking
Thisevolutiontohigherbandwidth,wirelessconnectionsinthehomeforun-tetheredVRmaydrivethehomearchitecturetosomethinglikethatillustratedinFigure21.Inthisscenario,everyroomhasahighspeedwirelessmicronodethatmaybeuncompressingorcompressingvideotoenduserdevices.AstheHMDitselftrackstechnologyimprovementsinGPUsforpowerandenergy,theremaybemorecompressedvideousage.However,thedelaysinperformingthedecompressionoftheVRsceneneedothertechniquestodealwithlatencyissues.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 28
Figure21-Future10Gbps+WirelessHomewithVRHMDCapability
Pre-emptiveCachingOpportunityforNextRenderedVRSceneWiththemassivebandwidthsrequiredtosendafull360degreestereoscopic4Kp6090HzscenefastenoughtotheHMDtonotincursimulatorsickness,itisclearthatintelligentalgorithmsmustbeusedtominimizethehighresolution,fieldofviewareaofthescene.Figure22illustratessomeoftheitemsthatmaybeused.Therewillbe
• Trackingheadandeyemovement,andkeepingtheimagescominginpre-emptively,inthedirectionofthetrackingontheheadset
• Usingvisualandaudiocues.Intheexamplebelow,thespeakerswilltriggerviewingfocusaswillthechangestonewslides/projections.Sportingeventswillbesimilarwithtriggerssuchasaplayer’srunningdirection,strikeoftheball,etc.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 29
Figure22-Cachingthemostlikelynextfocusareatoimprovelatency
ThisoffersthepotentialfortheGWand/ortheVRrenderingdevice(PCorconsoleorVRSTB)tocachethenextscenestoeitherprocessthemorsendthemuncompressedtotheHMD.
MergingtheVRandTVExperience–HMDand/orTVand/orPC/TabletThereisalsotheopportunitytomergetheVRexperiencewiththeTVexperience.Thereareafewcombinationsofthisthathavebeendiscussed.Theseare
(i) MovingtheTVexperiencetoaVRHMDexperience:ThemovieorTVshowiscapturedinVRandtheusercanexperiencemoreimmersionintheactionorscene.SomescenescouldbeVRandsomecouldberegular2D.Thisplacesnewcostsonthegenerationofthemoviecontent(fullnotpartialmoviesetswouldhavetobemadeandadministered)ordrivestheindustrytoallgreenscreenswithcomputeradditions.Thiswouldoccurlaterwithaugmentedreality.
(ii) CombiningtheTVexperiencewithanout-of-bandVRexperience:SomecontentproducersmaycreateaseparateparallelVRimmersiveexperiencethatcanbeinvestigatedlaterafter(orbefore)watchingtheregular2DshowontheTV.ThisadditionalVRsessionmaygivetheviewermoreinsightstokeyscenesintheprogramandwouldallowthemtodiscoveradditionalelementsofaplotorexperiencebeinginascene.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 30
(iii) Lookingatnon-stereoscopic110or360degreeFOVscenesontheTV:Usinganattachedhandhelddevice(gyrometer)tonavigatearoundthescene(likeinFigure22)andhavetheimagedisplayonthestatic,always-in-front-of-you,TVdevice.Thismaybeusefulforexploringlocationslikethoseinnews,discovery,ornatureprograms.Formostfiction-basedmovieproductions,thedirectorusuallywantstheusertostaywithinacertainframeviewaspartofthestorytelling.However,theremaybenewtakesondirectingVRmoviesthatgiveeachvieweradifferentperspectiveonwhatishappeningandfromwhereintheirown360degreeFOV.
Figure23-UsingtheHMDtoWatchallTVor“MadewithVR”Programs
CONCLUSIONWehavetowaittoseeifVR,AR,andMRwillbecomethenextversionofourentertainmentexperienceandhaveotherfarreachingapplications,likereplacingallthescreenswecarrywithuswithoneheadset/glasses/contactlensprojection.Imagineanentertainmentexperiencewhereyoufeelasthoughyouareinthescene–whetherit’samovie,TVshow,orexploringsomeforeignland.Nowimagineaworkingenvironmentwhereyoudon’tneedyourPCortabletanymore.Bothareprojectionsonyourretinacoupledwitheitherahapticfeedbacksolutionfortypingintheairorusingyoureyetracking,voice,orthoughtstoinitiateactions.
Inthenearrealistictimeframe,VR,AR,andMRlooktohavegreatpromiseforgaming,adultcontent,andsportsviewing.Aswehaveshowninthisgeneraltutorialpaper–thekeyelementsofthesheersizeofthescenesneededtocreateimmersionandthelow
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 31
latencyforthesescenestobedisplayed–therequirementsarenotyetfullyavailablewiththecurrentnetwork,encoding,videoprocessing,GPU,Wi-Fi,HDMI,screentechnologyetc.However,wearetakingbabystepsandworkingonlowerresolutionsolutions,aswellasinitiallytoleratingtetheredconnectionstotheHMD.
WehaveshownthattherewillbearolefortheserviceprovidertoembraceVRandtoaddvaluetotheVRchainaroundimprovedQoSoffileandlivedelivery.WehavedefinedasymbioticwirelesshomesolutionthatdoesnotcompetewiththeVRdevicecompaniestryingtocarveouttheirownspectrum(toensurequalityandlowlatency)attheexpenseofsharedspectrumproblemsinthehome.
Figure24-Ournewvirtualworkandexperiencesawaitus
Thereisalotmoretounderstandanddeveloptoperfectthistechnology,yetwearewellonthewaytotheStarTrekholodeckexperience.IfyouwatchthemanyVRvideosonYouTube,aswellassomeoftheconceptvideosfromMagicLeap,youcanseethatthereisalotofworkalreadydonetodetermineourimmersiveandpresencebasedfuture.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 32
ABBREVIATIONS2D TwodimensionalABR AdaptivebitrateAR Augmentedrealitybps BitspersecondCDN ContentdatanetworkDASH DynamicAdaptiveStreamingonHTTPISOBMFF ISObasemediafileformatDOCSIS DataOverCableSystemInterfaceSpecificationFOV FieldofviewGB GigabyteGbps BillionsofbitspersecondGHz GigahertzGPU GraphicsprocessingunitGW GatewayHD HighdefinitionHDMI High-DefinitionMultimediaInterfaceHDTV HighdefinitiontelevisionHFC Hybridfiber-coaxHMD HeadmounteddisplayHz HertzISBE InternationalSocietyofBroadbandExpertsISMradioband Industrial,scientific,andmedical(ISM)radiobandJPEG JointPhotographicExpertsGroupLi-Fi LightfidelityLTE Long-termevolutionMbps MegabitspersecondMHz MegahertzMPEG MovingPictureExpertsGroupMPEGOMAF MPEGOmnidirectionalMediaApplicationFormatms MillisecondPON PassiveopticalnetworkQoS QualityofserviceSCTE SocietyofCableTelecommunicationsEngineersSD StandarddefinitionSTB Set-topboxTCP TransmissioncontrolprotocolTV TelevisionUSB Universalserialbus
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 33
VR VirtualrealityWiGig WirelessGigabitAlliance
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 34
RELATEDREADINGS• TheEdgeResourceCenter:LeveragingNFVandSDNforHighAvailability/High
PerformanceNetworkFunctions–ThispapercoverstheconceptofanEdgeResourceCenter,asmall-scalevirtualmachineinterface(VMI)thatisco-locatedwiththeCCAPdeviceinacablehead-end,andsupportshighavailabilityandhighcapacitySDN/NFVresources.
• ServiceandManagementOrchestrationinDistributedHighSpeedDataNetworks–Thispaperdescribesacloud-basedorchestratorthatleveragesSDNprincipalsandworksinconjunctionwithpurposebuiltapplicationsinhigh-speeddatanetworks.Thisorchestrationcapabilitycanunburdentheexistingnetworkelementsinthecollectionandmanagementofnetworkdata,andhelpcreatevalueaddedservicesfornetworkconfiguration,networkoptimizationandnetworkplanning.
• AdvancedQoEMonitoringTechniquesforaNewGenerationofTrafficTypesCarriedbyDOCSIS–thispaperexploresfutureimprovementstoDOCSISQoEmonitoringtools,addressingmetricsbasedontraffictypes,SLAs,activesubscribercounts,CMTSscheduling,requestedloadsandsamplingrates.
MEETONEOFOUREXPERTS:CharlesCheeversCharlesCheevers,ChiefTechnicalOfficer,CustomerPremisesEquipment,isresponsibleforthetwotofiveyeartechnologyvisionofARRISCPEBusinessandtheevolutionofhomegateways,set-topboxes,anduserinteractionwithalldigitalservices.HeisresponsiblefordefiningarchitecturesforCPEdevicesandthecloudtogroundsolutionsandevolvingcustomerdigitalliferequirements.Previously,CharlesservedasChiefTechnologyOfficerofEuropeatARRISGroup,Inc.andservedasVicePresidentofProductLineManagement,MultimediaandwasalsoresponsibleforARRISEdgeQAMbusiness.SincejoiningARRISin2003,CharleshasdevelopedDVB,DOCSISheadend,andCPEsolutionsandproducts.HehasalsofocusedonconvergentsolutionsforthenetworkandhomethroughallIPdelivery.HeisarecipientoftheCableandSatelliteEuro50awardforhiscontributionstocabletechnology.
Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 35
REFERENCES(1) GartnerHypeCycle;Gartner,Inc.onlineathttp://www.gartner.com/technology/research/methodologies/hype-cycle.jsp
(2) “HowIndependentareHDR,WCG,andHFRinHumanVisualPerceptionandtheCreativeProcess”;SeanT.McCarthy;SMPTEMotionImagingJournal125(4):24-33;April2016
(3) “VRmotionsicknessalleviatedwithfieldofviewalterations”;JackLoughran;TheInstitutionofEngineeringandTechnology;June2016;onlineathttp://eandt.theiet.org/news/2016/jun/vr-motion-sickness-alleviated.cfm