klas skogmar, lund institute of technology real-time video effects using programmable graphics cards...
TRANSCRIPT
Klas Skogmar, Lund Institute of Technology
Real-time Video Effects Using Programmable
Graphics Cards
Master of Science Thesis
Klas Skogmar
Klas Skogmar, Lund Institute of Technology
Introduction
• Graphics cards have much computing power but are only used by 3D applications
• Video and image editing programs often needs to perform per pixel operations that are ideal for graphics cards
Klas Skogmar, Lund Institute of Technology
Problems
• How can graphics cards be used for displaying and altering video?
• What kinds of effects are suitable for a 3D environment?
• Is it possible to speed up effects by using the graphics card?
• How are the transfer rates affecting the system?• How much disc space is required and how does
this affect the system?
Klas Skogmar, Lund Institute of Technology
Programmable graphics cards
• Graphics cards can do massive parallel calculations
• 3D developers required more flexibility
• 3D card producers introduced “Vertex shaders” and “Pixel shaders”
• Makes it possible to program graphics cards in a similar way to processors
Klas Skogmar, Lund Institute of Technology
Effects on graphics cards
• Color correction
• Masking
• Color keying
• Compositing
• Transitions
• Painting
• 3D effects
Klas Skogmar, Lund Institute of Technology
High resolution challenges
HDTV standard 1080p requires throughput of 156 Mb/sec uncompressed. Current broadcasts takes approximately 30 Mb/sec uncompressed
Challenges:• Disc space• Transfer speed• Other problems (e.g. dynamic range)
Klas Skogmar, Lund Institute of Technology
Working practice
• Real-time• Expensive • Scanning film• Many formats• High resolution• Proxies and AOI• Specialized hardware
Klas Skogmar, Lund Institute of Technology
Available technologies
• Dedicated hardware• OpenGL (Imaging subset)• DirectX (Direct3D and
DirectShow)• Quicktime• ATI’s and Nvidia’s
Klas Skogmar, Lund Institute of Technology
DirectShow
• Microsoft’s media development environment• Built up of filters – each with its own task• Filters are assembled in a filter graph• Is supported by many video and image formats
Klas Skogmar, Lund Institute of Technology
My program
• Built using MFC• Uses a DirectShow filter• Shows how DirectShow can be used
Klas Skogmar, Lund Institute of Technology
Performance
• My program was slow• ATI has developed a
program that is fast (uses VMR-9)
• Serious Magic’s Texture Download Benchmark
• Matt Craighead’s program
Klas Skogmar, Lund Institute of Technology
Performance
Results:
• ATI’s program shows the potential – adding effects do not give visible frame drops
• The other programs shows the transfer speed from the graphics card to the processor – it is not enough, but driver updates enhances the performance
Klas Skogmar, Lund Institute of Technology
Conclusions
• It is already a technique that can be used!
• Drivers has to be enhanced to allow the modifications to be written back to memory
• Graphics cards are more flexible than dedicated hardware
• Effects have to be rewritten
• Many effects are more suitable to do using graphics cards
Klas Skogmar, Lund Institute of Technology
Conclusions
• The transfer to and from disc can be a problem
• There is a need for modules (e.g. DirectShow filters), that makes it easy to use programmable graphics cards