low latency rendering for mobile augmented reality w. pasman, a. van der schaaf, r.l. lagendijk,...
Post on 19-Dec-2015
223 views
TRANSCRIPT
Low latency rendering for mobile augmented reality
W. Pasman, A. van der Schaaf, R.L. Lagendijk, F.W. Jansen
Delft University of TechnologyUbiCom – http://www.ubicom.tudelft.nl
System sketch
Way finding
Maintenance, assistance
How to do this?
1. System overview
2. Why low latency3. Low-latency rendering4. Low-latency tracking
5. Status of project
1. System overview
InternetBasestationBackboneMobile
unitBase
stationMass
storageComputepower
Mobile unit off-loads processes to backbone
Camera serves 2 purposesViewing through camera of another user Tracking the position of the user
Mobile unit hardware17GHz, IR
TransmissionCompressAnalysisvideo outTrackingDecom-press
Render
2. Latency requirements
Accurate positioning requires low latencyMost demanding situation: head rotation
Alignment error depends on lag and rotational speed
See-throughdisplayReal objectMisalignedvirtual object
Humans can detect <8ms
But hardly any literature on relation with task.
Estimated for mentioned tasks:Max. error 0.5˚ @ 50˚/s latency required ≤ 10 ms
VR: several sources of latency
Source of latency typical3D Rendering 30-200 msTracking 15-40 msDisplay @50Hz 20 ms
Fastest reported: 17 (33)ms latency
Our goal: 10 msRendering + Display 8msTracking 2ms
Std. Voodoo 3D game card
Render just ahead of raster beam
4 partitionsgive latency5-10 ms
20msDisplaying here
Rendering hereTo be cleared and rendered
3. Low-latency rendering
~ hundred texture-mapped polygons Simplify virtual world
In backbone, using LOD polygon reduction polygon-to-texture techniques
Feasible: relatively small #virtual objects
ApplicationdatabasesNetworkresourcesComputestationsimplified sceneBasestation
Latency-layered system
complexvirtualobjects
simplevirtualobjects
displaylistDisplay>200HzHeadsetBackbone~10Hz<1HzPositionfrom inertial tracker
Position from camera and prediction
Usermovement~10ms~100ms~1s
PrototypeKURT Realtime Linux to• schedule renderer at 200Hz• in sync with displayLatency measured: 4.2-8.8ms (6.35 avg)
TrackerSimilar latency-layered techniques as rendering, using• inertial tracker in low-latency loop• camera feature tracker in outer loop
Prototypeoff-the-shelf componentsNot yet wireless (tracker & power cables)
Rendering system• “VRML2” scene graphs• changed animation:
no 200Hz scene graph traversal-> compile @10Hz to display list @ 200Hz-> tags in displaylist for last-minute updates
5. Status of project
Tracking• Proto: Intersense IS600 (inertial+acoustic)• Working on optical tracking with artificial feducials• Later: natural feducials
Future• Fast & simple hardware for rendering, combining well with video transmission• Task - latency relation • Applications• Incremental scene simplification
ConclusionAccurate alignment for ARLatency-layered system architectureFirst-ever <10ms rendering (no prediction)
www.ubicom.tudelft.nl