360° video without stitching
TRANSCRIPT
360° Video without stitching
Anne Jan BrouwerNoProtocol
Creative Coding Amsterdam 111020 – 05 - 2016 IceMobile
2
360° Video
● Recently everywhere
– YouTube– Facebook
● Recording video
– Multi camera solutions– Parabola– Extreme fisheye
● Usually requires post-processing
3
360° Still
360p.co.uk
4
360° Still
● Image aspect ratio of 2:1
● Wrapped around a sphere
5
360° Still
● Camera inside of the sphere
● That's it . .
6
Three.js
7
Three.js
● 18 lines to get that panorama working
● Works with any 2:1 panorama image
● Needs gyroscope and mouse look
● And of-course . .
● Video!
8
Three.js
9
Three.js
https://annejan.com/media/360/sitting
https://annejan.com/media/360/standing
● Demo time
● Code: https://github.com/annejan/360-Player
10
Hardware
● 360fly.com
● 360x240°
● Extreme fisheye
● No stitching
● Deadzone
11
Stitching . . well . . processing
● Multi-camera setups require stitching
● Fisheye lens needs unwrapping
● Raw image
12
Processed video
● Getting
https://annejan.com/media/360/roof-processed.mp4
● Going from
https://annejan.com/media/360/roof.mp4
● But that is a lot of effort . .
● Specialized software
13
Three.js
14
Vertex shader
● Sets up the texture mappers “normal”
● Is just some more plumbing
● Resides in the html, or a separate file
● Shader language
15
Normal
● What is a normal?
● A vector
16
Fragment shader
● 11 lines to do the “conversion”
● Let's take a look inside that main()
● See that if statement?
17
Extreme fisheye
● 360x240°
● Deadzone
● Demo?
https://annejan.com/media/360/
18
What more
● Streaming
● Streaming is hard
● Reverse engineered the 360fly app-cameracommunication
● RTSP
●
● Some other time
19
Check it out . .
https://github.com/annejan/360-Player
https://annejan.com/media/360/
● Questions?
@annejanbrouwer