design in motion: video production workflow
TRANSCRIPT
Design in Motion:Video Production WorkflowDES05Ben WaggonerTechnical Evangelist,Core Media Processing TechnologiesMicrosoft Corporation
Agenda
Video and audio in WPF/EBest practices for processing video and audioBest practices for Windows Media encoding for WPF/EUsing the Expression Media Encoder
Video and audio in WPF/E
It's just Windows Media!Huge library of existing content already compatibleExisting URLs from Windows Media Services still workAlready supported by many hosting companiesLots of tools available for different markets
Video codec;Windows Media Video 7/8/9 and 9 Advanced Profile
Audio codec:Windows Media Audio "Standard"
Not Pro, Voice, or Lossless in this version
HD Video Playback
video
FLV v. WMV
What does WMV give content creators?
Higher quality video and audio Fewer bits needed for same quality
Much faster encoding3x faster for optimum quality on 4-core workstation15x faster with Tarari card for hardware acceleration
Much better TCO for video hostingMany more streams off a 2K3 Server than Flash serverCheaper per GB charges from hosting providers
Mature live streaming/encoding tools
Best Practices for Video Processing
CaptureGetting the content in higher quality
PreprocessingOptimizing the content for the codec itself
Best Practices: Capture
Capture native bitstream or uncompressedDV, DVCPRO HD as native bitstreamOther formats as uncompressedDon’t use DV bridge for analog-digital conversion
DV codec throws out a lot of image data, especially color
Access earliest generation dub as possibleMost VHS tapes didn’t get posted on VHS!Ripping from DVD also sub-optimal
Best to grab digital export from NLE
Preprocessing
Everything we do to the video and audio between the source and the codecThe biggest single factor in good-looking WPF/E video
The best the codec can do is give back what you gave it
For challenging projets, I spend ~80% of my keyboard-and-mouse time on preprocessing
Once the codec settings are nailed, you're good to go.
Unprocessed frame
Same Frame Preprocessed
Unprocessed @ 800 Kbps
Preprocessed @ 800 Kbps
Deinterlacing and Inverse Telecine
WPF/E uses progressive videoFor 60i/50i, deinterlace to progressiveFor video telecined from film, use inverse telecine to restore 24pIf it’s progressive, leave it alone!<example slide>
Crop
Video monitors don’t go to edgesComputers doCrop edge blankingAsymmetric okay to cut out noiseFor low rez, crop to safe areaCrop letterboxing as wellIf going to 16x16 resolution, can crop to final aspect ratioFor mixed video/computer playback, matte out noise in blanking
The Safe Area
Image adjustment
Brightness and ContrastComputers have broader range — use itBlacks should look identical to RGB=0,0,0 in browser
Noise ReductionMake image easier for compressorRemove grainSmooth sharp edges
Levels example
Levels in Action
After The Settings
Best Practices: Audio Normalization
Make loudest point of audio as loud as system beep-3 dB / 90%Not compressor/limiter - no change in dynamic range
Best Practices: Encoding
Progressive versus StreamingData RatesFrame SizeFrame RateAudio SettingsAdvanced Registry Key settings
Progressive v. Streaming
Two legitimate modes of delivering web videoNeed to know which one will be used before encoding
Progressive should use 2-pass VBRStreaming should use 2-pass CBR
Progressive Download
Off a web serverE.g., IIS in Windows 2003 Server
Most efficient with Variable Bit Rate (VBR) encodingRandom access in part of stream already downloadedGreat for movie trailers
Streaming
Files fromWindows Media ServicesIncluded in Windows 2003 Server
Needs Constant Bit Rate (CBR) encodingOffers random access throughout streamReduces need for local caching (big for mobile devices)Great for movies
Data Rates
We measure rates in power-of-ten numbersK=1000, not 1024M=1000000, not 1048576
StreamingNeed more sustained bandwidth to user more than data rate
ProgressiveA slower connection means user waits longer for playbackBut can use high bitrates for short clips
Frame Size
WPF/E wants square pixel videoSo need to convert to aspect ratioExample
DV is 720x4804:3 content becomes 640:48016:9 content becomes 832x480
Depends somewhat on data rate200-500 Kbps: 320x240 or 432x2401-2 Mbps: 640x480 or 832x240
Encoding modes
1-pass versus 2-pass1-pass reads the source in and encodes out at the same time2-pass does analysis pass first, calculating optimal bit-budget2-pass is always better if you've got the time
And it’s not 2x slower – the analysis pass runs quite a bit faster
CBR versus VBRConstant Bitrate
Optimum for streaming – quality varies, bitrate constant
Variable BitrateOptimum for progressive download – varies bitrate, quality constant
Frame Rates
Use frame rate that’s an integer division of source59.94 to: 59.94, 29.97, 15, 10, 7.5, 6, 550 to: 50, 25, 12.5, 10, 524 to: 24, 12, 8, 6
For anything over 200 Kbps, can use full frame rate of source
Expression Media Encoder
New compression tool targeting .WMV for WPF/EBunded with Expression Media ProductAccessible to both designers and video professionalsOffline and Live Encoding
But there's no secret sauce – you're welcome to use any WMV encoding tool with WPF/E
Using Expression Media Encoder
demo
Advanced Compression Settings
The codec has a bunch of registry key setting to further optimize encodingI fear regedit – you can use “WMV9 PowerToy”
Lots more options than I’ll discuss todayLink to full documentation at end
Tools will be integrating these controls later this year
WMV9 PowerToy
http://citizeninsomniac.com/WMV/#WMV9PowerToy
Number of Threads
Current codec can use up to four threads for encoding
Big speed improvement with multiple cores
Auto picks number of threads availableSees HyperThreading (Pentium 4) as 2 processors
But HT is less of a gain with compression than additional core
Want to have at least 64 pixels high per thread<128 height: 1-thread<256 height (so 320x240): 2-threads256+ height: 4-threads
Motion Vector Search Range
Motion Estimation copies moving elements between frames
Much more efficient to encode “motion vector” than second copy of element
Using a larger Motion Search rangeProvides better efficiency with faster motionSlows encodingCan reduce efficiency with slower motion
MV Range options
Available Options:Macroblock-adaptive128 pixels horizontally, 64 pixels vertically (default)256 pixels horizontally, 128 pixels vertically1024 pixels horizontally, 256 pixels vertically2056 pixels horizontally, 512 pixels vertically
Each progressive option slowest encoding by ~2xMacroblock-adaptive slightly faster than 1024x256
Use Macroblock Adaptive for non-realtime encoding
MV Range – Reference Frame
MV Range – Search Frame
Chroma Search
Finds where color changes but B&W channel doesn’tQuality improvements 5-20% (most with colorful content)
Crowd shots, motion graphics helped in particular
Modes:Macroblock-adaptive with nearest integer chroma (fast, good)Macroblock-adaptive with true chroma (speed/quality mix)Luma only (default)Luma with nearest-integer chromaLuma with true chroma (best, slowest)
B-Frames
Number of bi-directional (B-Frames) between I-Frames and P-FramesB-Frames based off previous and next frameB-Frames typically fewer bits for same qualityNo frames are based on B-frame
Don’t insert I-frame for flashes if they land on a B-frame
So dynamically insert B-frames when there are flashes!Big win for gunfights and discos
1 B-frame optimal for most web encoding
B-Frames Example
Loop Filter
Smooths out hard edge blocking artifacts“In-loop” - used as prediction for future frames
Only blurs content where an artifact would appearMuch less noticeable at higher resolutionsSoftness better if alternative is artifacts
Increased CPU decoder requirements somewhat
Overlap Filter
Softens edges where motion vectors overlapSoftens image somewhatBut reduces artifacts
RecommendationOn if you see artifactsOtherwise Off
Also increases decoder complexity some
Median Filter
Denoises image before calculating motion vectorsKeeps grain/noise from “swirling”Helps low-bitrate encodesBut can leave “trails”
RecommendationOnly use with grainy source, and QA carefully
Denoise filter
Simple noise reduction filterHelps when the source is noisyNot as good as offline noise reducersRecommendation
Good for realtime encoding in low lightBut better to not have noisy source…
Noise Edge Removal
Blanks Line 21, other noise around edge of frameHandy for real-time encoding of analog sourcesOnly use if source has edge noise
Lookahead
Specifies up to 16 frames forward for analysisOnly needed in 1-pass encoding
Used to detect scene changes, flash framesBig improvement for 1-pass CBR encodesCan dynamically insert B-frames with flashes
Default is Off (Does not look ahead)Adds to latency for live encoding
~½ second for Lookahead=15 @ 29.97 fps
Slightly slows encoding as Lookahead increases
DQuant
“Differential Quantization”Different macroblocks are compressed different amounts
Compress gradients less, to reduce banding
RecommendationI and P frames
Using on B-frames wastes bits that will be discarded
Lots of levels available“Regular” good default with most content
Perceptual Option
Like DQuant, optimizes bits based on imageUsed in addition to DQuantAdaptive Dead Zone 1 good default for film content
Experiment carefully with other kinds of sources
Optimum Settings: On demand
Highest quality encoding and reasonable performance
Up to 4x slower than default settings
Encoder complexity = 4/80 (second slowest)Motion Search Level = macroblock adaptive true chromaMotion Search Range = macroblock adaptiveMotion Match Method = Adaptive SAD/HadamardDQuant = RegularPerceptual Option = Adaptive Dead Zone 1 for filmForce LoopFilter = OnNumBFrames = 1At least 96 pixels high per thread (so 2 max for 320x240)
Optimum Settings: Webcasting
Optimized for reliable real-time encodingRemember dual-duals are almost 4x faster than single core
Motion Search Range = Biggest that doesn’t drop framesMotion Search Level = Luma + integer chromaNumBFrames = 1Force LoopFilter = 1 (on) Dquant Option = RegularPerceptual Option = Adaptive Dead Zone 1 for filmLookahead = 16 (Max – most important single option)
Designing Windows Applications with Expression Blend
2-day instructor-led training in over 20 US cities
Tools and Technologies UsedExpression Blend and Expression Design
NET Framework 3.0, Windows Presentation Foundation, XAML
Visual Studio 2005
Target AudienceUser interface designers with advanced experience with Flash or Director. Understanding of Visual Studio 2005 or programming skills is not required.
PrerequisitesDoes not require prior experience using Expression Blend or Windows Presentation Foundation.
We recommend:
Building an understanding of .NET Framework 3.0
Download Expression Blend http://www.microsoft.com/expression
Get inspired! www.seewindowsvista.com
Learn More: http://tinyurl.com/3y6jg7
Q&A and Contact
My blog:http://on10.net/blogs/benwagg/
My class at Stanfordwww.digitalmediaacademy.org/courses/video-compression-training.html
Please fill out your eval
evaluation
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.