the image interchange framework...
TRANSCRIPT
The Image Interchange Framework Demystified:
What it is, and why it matters
Andy Maltz Director Science and Technology Council Academy of Motion Picture Arts and Sciences
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
From urbandictionary.com
Special Snowflake:
“A product created specifically for a customer or client who cannot or will not conform their
requirements or needs to the general specifications causing said vendor to customize uniqueness to an
established product thereby creating more work, time, energy, and effort to be expended on the vendor’s
employees' part…”
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
The Last Snowflake Workflow
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Agenda
• What’s changing • Why the Academy is supporting IIF • IIF overview, core technical features • Demo room pointers • Your call to action
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
This is a period of great change and instability
• Film is going away • Digital motion picture cameras are getting better, and
generating LOTS of data • Cinema may not stay the highest “quality” for long:
– Brighter displays in the home – Higher framerates coming – 4K, UHDTV (wider color gamut than HDTV)
• Low cost technology lets anyone do nearly anything they want
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Problems in Motion Picture Mastering
• It’s a mess out there: – Many image file formats and implementations – Color management is slathered in “secret sauces” that make inter-facility
interchange difficult – Snowflake Workflows: good for HPA, bad for filmmakers
• 10-bit Cineon isn’t good enough anymore – Insufficient precision and dynamic range – Based on an end-of-life technology (film) – Extending to 16 bits won’t fix the other problems
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
The Problems - 2
• There is no defined “Digital Source Master” – And therefore no digital archiving standard
• The traditional sources of industry leadership are absent
– 81% of a movie’s total revenue is from non-theatrical distribution – A well-designed, system-wide solution won’t happen on its own.
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Why the Academy?
• Mission since 1927: To Advance the Art and Sciences of Motion Pictures
• Image quality is important and shouldn’t be so difficult to achieve
• The Academy is a neutral forum – Represents the interests of the creators of motion pictures – Takes on problems that are not commercially feasible – Historical role in development of industry standards
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Academy-led standards
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Other Academy efforts: Digital Preservation
� � � � � ��� � �� � � � � � �� � � � � � � �� � � �� �� � � � � � �
� � � � � � �� � � � �� � � ���� � � � � � � � � � � � �� � � �� �
� � �� � � � � �� �� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Just published (free copies outside)
�THE SCIENCE AND TECHNOLOGY COUNCIL
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Back to the matter at hand…
• Next-generation cinema is impossible without a next-generation imaging architecture
• This is complicated, and it won’t happen all by itself
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
The Image Interchange Framework (IIF-ACES)
• An architecture for high fidelity motion picture imaging to support numerous workflows – Digital Intermediate – Visual Effects – Animation – Restoration – Film and/or digital capture – Scene- and output-referred methodologies
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Key IIF components
• Standardized image encoding specifications: – Colorimetric: 16 bit half-float, wide gamut, HDR – Densitometric: 16 bit integer, fully specified metrics
• Standardized transforms and methodologies: – Image input (digital and film) – Rendering and display
• Standardized file formats – “Constrained” OpenEXR – Extended, fully specified DPX
• Straightforward adoption path
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
A Brief History of IIF
• 2004: – Siggraph BOF: film scans are inconsistent, there needs to be better “image interchange” – Academy File Format Project initiated
• 2005: – problem study, requirements document written (42 pages long) – Conclusion: color management is the toughest nut to crack – Ed Giorgianni commissioned to write Digital Cinema color management whitepaper
• 2006: – Project Committee begins designing, viewing images – Follow-on whitepaper by Giorgianni – Academy hires Senior Imaging Engineer
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
A Brief History of IIF – 2
• 2007: – Draft image encoding specifications published for comment and test – Core transforms prototyped – Proof-of-concept work launched – Color Transform Language (CTL) published as SMPTE RDD-15
• 2008: – IIF introduced at HPA, Call for Participation issued – Color corrector implementations begin – Container and metadata work begins – IIF papers at SMPTE Fall Conference
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
A Brief History of IIF – 3
• 2009: – Working Groups continue working – Academy Metadata Symposium – IIF introduced at IBC Technical Sessions – Draft encoding specifications finalized
• 2010: – Transform development and testing continues – SMPTE standards work initiated
• 2011: – Industry trials begin – First “Designed With IIF In Mind” products appear at NAB”
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
A Brief History of IIF – 4
• 2011 (cont’d): – Key transforms updated with results from trials – Television discovers IIF: at least a dozen shows (pilots & series)
– IIF at NAB – Justified panel – Academy hires two more imaging engineers and builds state-of-the art imaging lab – First 3 SMPTE standards elevated to Draft Publication
Approved <TBD>
Warning
This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication.
Contents Page
Foreword ............................................................................................................................................................ 2!
Intellectual Property............................................................................................................................................ 2!
Introduction......................................................................................................................................................... 3!
1! Scope ........................................................................................................................................................... 5!
2! Conformance Notation ................................................................................................................................. 5!
3! Normative References.................................................................................................................................. 5!
4! Terms and Definitions .................................................................................................................................. 6!
5! Academy Color Encoding Specification (ACES) .......................................................................................... 8!
Annex A (Normative) ACES Reference Input Capture Device spectral sensitivities ........................................ 12!
Annex B (Informative) Characteristics of ACES RGB color encoding .............................................................. 19!
Annex C (Informative) ACES RICD Camera Flare ........................................................................................... 21!
Annex D (Informative) ACES RGB values of common stimuli as produced by the ACES RICD ..................... 22!
Annex E (Informative) Bibliography .................................................................................................................. 25!
Page 1 of 20 pages
SMPTE ST 2065-1:2011 SMPTE STANDARD
Academy Color Encoding Specification (ACES)
Approved <TBD>
Warning
This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication.
Contents Page
Foreword ............................................................................................................................................................ 2!
Intellectual Property............................................................................................................................................ 2!
Introduction......................................................................................................................................................... 3!
1! Scope ........................................................................................................................................................... 4!
2! Conformance Notation ................................................................................................................................. 4!
3! Normative References.................................................................................................................................. 4!
4! Terms and Definitions .................................................................................................................................. 5!
5! Academy Printing Density ............................................................................................................................ 6!
Annex A (Normative) Spectral Responsivities !APD ........................................................................................... 9!
Annex B (Informative) Influx Spectrum SAPD..................................................................................................... 12!
Annex C (Informative) Bibliography.................................................................................................................. 15!
Page 1 of 20 pages
SMPTE ST 2065-2:2011 SMPTE STANDARD
Academy Printing Density (APD) —Spectral Responsivities, Reference Measurement Device, and Spectral Calculation
Approved <TBD>
Warning
This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication.
Contents Page
Foreword ............................................................................................................................................................ 2!
Intellectual Property............................................................................................................................................ 2!
Introduction......................................................................................................................................................... 3!
1! Scope ........................................................................................................................................................... 4!
2! Conformance Notation ................................................................................................................................. 4!
3! Normative References.................................................................................................................................. 4!
4! Terms and Definitions .................................................................................................................................. 5!
5! Academy Density Exchange Encoding ........................................................................................................ 6!
Annex A (Informative) Bibliography .................................................................................................................... 9!
Page 1 of 20 pages
SMPTE ST 2065-3:2011 SMPTE STANDARD
Academy Density Exchange Encoding (ADX) — Encoding Academy Printing Density (APD) Values
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
A Brief History of IIF – 5
• 2012 (so far): – Engineering nearing completion on key transforms – Next round of industry trials in preparation – Collaborative software development site (GitHub)
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Industry-wide collaborative development
• More than 50 leading practitioners, engineers and scientists • Leading equipment manufacturers • Major post-production, VFX and animation facilities • Major technical contributions from:
– Adobe, Arri, Autodesk, Dolby, FilmLight, Fujifilm, ILM, Kodak, Pixar, Sony Electronics and top color science and imaging experts
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
The IIF Eye Chart (Idealized System Block Diagram)
ADX: Academy Density eXchange ACES: Academy Color Encoding Specification
RRT: Reference Rendering Transform RDT: Reference Display Transform IDT: Input Device Transform
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Motion picture workflow: “legacy”
digital color
correction
release print film emulation
hd mon
rec 709
xform
on-set timing session
digital acquisition
source cam lut
Full-range raw digital acquisition
dist master
color correction session
digital color
correction
release print film emulation
DCDM xform
cam lut
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Motion picture workflow - IIF
ACES RRT hd mon ODT
on-set timing session
digital acquisition
source IDT
Full-range raw digital acquisition
dist master
color correction session
ACES RRT RDT IDT
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
ACES: Key features
• Enhanced Precision • Wide Color Gamut • High Dynamic Range • IMF-ready
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Precision: why 10 bits isn’t enough
Steps visible
Steps not visible
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Exposure latitude of film negative exceeds 10 bit Cineon - that’s why ACES is 16 bit half-float
Video Conversion Range Video Conversion
Beyond Video Conversion Range
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Wide gamut: ACES covers all visible colors
ACES color primaries in perspective
CIE x CIE y Red 0.73470 0.26530
Green 0.00000 1.00000 Blue 0.00010 -0.07700
ACES color primaries in perspective
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Color gamut is also increased with greater dynamic range
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
What the colorists say
“It’s All There”
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
What a famous Hollywood director says
“I want my whole cow. Not just the steak you serve me.”
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Why we render
• Film and digital sensors capture scene exposures (proportional to # of photons hitting the imaging plane)
• Unprocessed scene exposures don’t look good on any display medium
• “Rendering” applies an image processing algorithm to scene exposure values to generate a “pleasing” image on a target display device or medium
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Scene exposure vs. “output referred”
42
Accurate Reproduction of Scene Colorimetry Image Rendered for Pleasing Reproduction
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Good rendering is hard to do
• Film manufacturers spent 70 years working on it • Digital Intermediate facilities spent 15+ years on it (and they all do it
differently) • The IIF Project Committee spent 5 years working on a “baseline
renderer”: the Reference Rendering Transform, or RRT
46
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
RRT Facts
• Designed to go several stops beyond film and today’s digital displays (>1,000,000:1)
• Designed to make the colorist’s job easier: a better starting point
• Designed to not get in the way of the creative vision • Mathematically sound
®2012 A.M.P.A.S.
Print Film Emulation (PFE)
Digital Cinema Projector
Print Film Projector
Video Display
Dolby PRM
Lab HDR Display
IIF Theoretical Display
®2012 A.M.P.A.S.
RRT Release Candidate (February, 2012)
Digital Cinema Projector
Print Film Projector
Video Display Dolby
PRM
Lab HDR Display
IIF Theoretical Display
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
IIF Myths
• “It’s not baked yet” – First standards are done – Field trials taught us a lot, and they resulted in significant
improvements – Continued comprehensive, industry-wide testing is crucial
• “It forces the ‘Film Look’ ” – Rendering is required – All “pleasing images” have some things in common – The RRT does not go to all the places PFEs go
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
IIF Myths - 2
• “IIF will never work in production” – Every trial project found some issues, and they all were resolved – Issue: smaller, under-resourced VFX facilities may not be good early
adopters • “IIF will never work for everything”
– How do you prove a negative? – We think it will – Even if we’re wrong, huge benefits for the projects it does work for
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
IIF Myths - 3
• “The Academy is dictating that you use IIF” - Early SMPTE proponents include:
• ASC • Paramount Pictures • Sony Pictures • Pixar • Several manufacturers
• Visit the Demo Room
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
The cost of waiting
• SMPTE standards are publishing now – Very difficult to change after publication – The time to ask for changes is now
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
What’s next
• RRT release candidate: March 1, 2012 • Next round of industry trials • Complete file container and base metadata
standardization • Support plan development
®2012 A.M.P.A.S.
The Image Interchange Framework • HPA Tech Retreat February 2012
Demo Room: IIF in Real Life
• Dolby, Image Systems: High dynamic range display driven by IIF-enabled color corrector
• FilmLight: Baselight with IIF support, matching multiple digital motion picture cameras
• Fujifilm: IIF-based on-set color management tool • Image Essence: IIF software implementation