imf mechanisms for carrying metadata by marquise technologies
TRANSCRIPT
EBU Production Technology Seminar, January 2017
Dan Tatut, VP Business Strategy & Development
IMF Mechanisms for carryingMetadataCurrent Practices
IMF mechanisms for carrying metadata
• CPL structure• Intrinsic & Extrinsic metadata• Metadata in track files• Metadata processing via OPLs (Output Profile Lists)• About timecode• Conclusion / Topics for debate
OVERVIEW
IMF mechanisms for carrying metadata
IMF mechanisms for carrying metadata
• <Annotation> user-defined text (comments about/description of the composition)• <Creator> software/device that created the composition (usually not user-
editable)• <ContentOriginator> who created the composition (e.g. studio name)• <ContentTitle> title of the composition, often using a “naming convention”
• <ContentKind> defined in ST2067-3 or custom value (scope attribute should be
used)• <ContentVersionList> a list of ID/label pairs, references to external metadata• <LocaleList> a list of <Locale> structures for language(s), region(s) and content
maturity rating(s)
What is available in the “Prologue section” ?
• <ContentVersionList> is a list of <ContentVersion> elements• Each <ContentVersion> is a pair of <Id> and <LabelText> elements• Each Id/LabelText pair can be anything, e.g.:
IMF mechanisms for carrying metadata
External metadata through “<ContentVersionList>”
OWNER FORMAT
Disney <Id>tag:disney.com,2015:wmls-id:7000001</Id><LabelText>In & Out (US English)</LabelText>
Warner Bros <Id>tag:warnerbros.com/series/mpmNumber,2004582</Id><LabelText>series(GILLIGAN'S ISLAND - YR02 65/66)</LabelText>
EIDR <Id>urn:eidr:10.5240:4FEC-7003-F150-AD59-9D70-S</Id><LabelText>This is an EIDR reference</LabelText>
ISAN <Id>urn:isan:aef0-1000-6721-022-x-0000-9034-1</Id><LabelText>This is an ISAN reference</LabelText>
• <EssenceDescriptor> elemets are part of <EssenceDescriptorList>
• Each <EssenceDescriptor> exposes in the CPL asset metadata using SMPTE
RegXML• Some assets can carry user defined metadata, e.g. MCA in the audio tracks
IMF mechanisms for carrying metadata
A word on “<EssenceDescriptor>” elements
MCA Metadata
...<MCATitle>The Great Escape</MCATitle><MCATitleVersion>this is version 2</MCATitleVersion><MCATitleSubVersion>actually version 2.1b</MCATitleSubVersion><MCAEpisode>of the first episode<MCAEpisode>...
• <ExtensionProperties> is the weapon of choice for adding static metadata to
the CPL• One unique instance of any custom-defined element• Must be identified by their specific name spaces• Can contain any sub-structure (as defined by their XML schema)• Can be ignored by QC systems (unless they understand the schema)• Can be stored internally or externally to the CPL (linking is extension specific)• <ExtensionProperties> outside of the timeline, not well suited for time-based
metadata
IMF mechanisms for carrying metadata
Adding metadata through“<ExtensionProperties>”
• CPL-embedded EBU extension for linking EBUCore documents as presented at PTS 2016
IMF mechanisms for carrying metadata
Examples using “<ExtensionProperties>”<ExtensionProperties> <ebuext:EBUExtension xmlns:ebuext="http://tech.ebu.ch/schemas/XXXX"> <ebuext:EBUCoreList> <ebuext:EBUCore> <Id>urn:uuid:7a5656b3-64b0-4dc7-960b-5a819c4669e5</Id> <LabelText>France TV</LabelText> </ebuext:EBUCore> <ebuext:EBUCore> <Id>urn:uuid:2945c709-8f76-491e-8aaf-bccb97bf99d8</Id> <LabelText>UK DPP</LabelText> </ebuext:EBUCore> </ebuext:EBUCoreList> </ebuext:EBUExtension></ExtensionProperties>
IMF mechanisms for carrying metadata
Examples using “<ExtensionProperties>”<ExtensionProperties> <mt:ExtensionProperties xmlns:mt="http://www.marquise-tech.com/imf/2016/cpl-extension-properties"> <mt:DM_AS_11_UKDPP_Framework> <mt:UKDPP_Audio_Loudness_Standard>Loudness_None</mt:UKDPP_Audio_Loudness_Standard> <mt:UKDPP_Textless_Elements_Exist>False</mt:UKDPP_Textless_Elements_Exist> <mt:UKDPP_Picture_Ratio>16/9</mt:UKDPP_Picture_Ratio> <mt:UKDPP_Tertiary_Audio_Language>zxx</mt:UKDPP_Tertiary_Audio_Language> <mt:UKDPP_Audio_Comments>Mixed to PPM 6</mt:UKDPP_Audio_Comments> <mt:UKDPP_PSE_Pass>PSE_Not_tested</mt:UKDPP_PSE_Pass> ... <mt:UKDPP_Genre>Rock/Pop</mt:UKDPP_Genre> ... <mt:UKDPP_Originator>BBC S&PP DMS</mt:UKDPP_Originator> <mt:UKDPP_Product_Placement>False</mt:UKDPP_Product_Placement> ... </mt:DM_AS_11_UKDPP_Framework> </mtextprop:ExtensionProperties></ExtensionProperties>
IMF mechanisms for carrying metadata
Examples using “<ExtensionProperties>”<ExtensionProperties> <imf:ExtensionProperties xmlns:imf="http://dalet.com/api/imf/2015-12/imp-metadata-extension"> <WB:WBExtensionProperties xmlns:WB="http://warnerbros.com/schemas/imf/2016/extensionProperties"> <Series xmlns="http://warnerbros.com/schemas/imf/2016/extensionProperties"> <Name>GILLIGAN'S ISLAND - YR02 65/66</Name> <MpmNumber>2004582</MpmNumber> ... </Series> <Title xmlns="http://warnerbros.com/schemas/imf/2016/extensionProperties"> <MpmNumber>4034033</MpmNumber> ... <EpisodeNumber>001056</EpisodeNumber> <EpisodeSequenceNumber/> ... </Title> ... </WB:WBExtensionProperties> </imf:ExtensionProperties></ExtensionProperties>
• IMF allows metadata extension at the timeline level. This is done through virtual
tracks• Can be intrinsic or extrinsic to the CPL• Can be time varying or constant, contiuous or discrete.• Extensions are identified by their XML namespace/schema• Can be ignored by QC systems that do not understand them• Must follow the timeline rules (tracks, segments, sequence, etc.)
IMF mechanisms for carrying metadata
Adding timeline metadata
M1 & M2
A1
Soundfield 5.1
Metadata
IMF mechanisms for carrying metadata
SEGMENT 1 SEGMENT 2 SEGMENT 3
V1Sequence
Sequence
Sequence
IMF mechanisms for carrying metadata
Examples of intrinsic timeline metadata: markers<MarkerSequence>... <ResourceList> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" si:type="cpl:MarkerResourceType"> <Id>urn:uuid:ce398822-cd01-4eee-b725-965bc78ea3e1</cpl:Id> <Annotation/> <EditRate>24000 1001</EditRate> <IntrinsicDuration>0</IntrinsicDuration> <EntryPoint>0</EntryPoint> <SourceDuration>0</SourceDuration> <Marker> <Annotation>SMPTE_BARS</Annotation> <Label scope="http://vubiquity.com/dete/imf/2016/markerLabel">SMPTE_BARS</Label> <Offset>0</Offset> </Marker> </Resource> </ResourceList></MarkerSequence>
IMF mechanisms for carrying metadata
Examples of intrinsic timeline metadata: RDD6<rdd6:RDD6Sequence xmlns:rdd6="http://bbc.co.uk/imf/rdd6/2017">... <ResourceList> <Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" si:type="rdd6:RDD6ResourceType"> <Id>urn:uuid:ce366622-cd01-4e2e-1115-7622c48ea3e1</Id> <EditRate>25 1</EditRate> <IntrinsicDuration>0</IntrinsicDuration> <EntryPoint>0</EntryPoint> <SourceDuration>0</SourceDuration> <RDD6> <FirstSubFrame>... </FirstSubFrame> <SecondSubFrame>... </SecondSubFrame> </RDD6> </Resource> </ResourceList></rdd6:RDD6Sequence>
IMF mechanisms for carrying metadata
Examples of extrinsic timeline metadata: TT commentary<cc:CommentarySequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016"> <Id>urn:uuid:0ea040c7-418f-4b3d-b27e-8b4b3da9bdfa</Id> <TrackId>urn:uuid:b13a9521-df5b-403a-8634-cacf14094835</TrackId> <ResourceList> <Resource xsi:type="TrackFileResourceType"> <Id>urn:uuid:53c5dd91-9b31-4a89-a63b-b2d8d7112627</Id> <EditRate>24000 1001</EditRate> <IntrinsicDuration>4208</IntrinsicDuration> <SourceEncoding>urn:uuid:3ef4dbcf-d4e1-a432-b4b0-38da2d96442f</SourceEncoding> <TrackFileId>urn:uuid:3ef4dbcf-d4e1-a432-b4b0-38da2d96442f</TrackFileId> <Hash>T8pvcuXA+GVQM/fc0DMlgoMwOSE=</Hash> </Resource> </ResourceList></cc:CommentarySequence>
• <SourceEncoding> references an <EssenceDescriptor> defined in the <EssenceDescriptorList>
• The actual “external metadata” is referenced by the <TrackFileId> element
IMF mechanisms for carrying metadata
Examples of extrinsic timeline metadata: ancillary data<cc:AncillaryDataSequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016"> <Id>urn:uuid:cc64ab4c-d7bf-4453-85a6-40f372bc4f6c</Id> <TrackId>urn:uuid:cf4a166f-3110-4512-a3d7-8193102c0458</TrackId> <ResourceList> <Resource xsi:type="TrackFileResourceType"> <Id>urn:uuid:0739ffce-918b-496c-b2c7-f3d2ef06f3f2</Id> <EditRate>24000 1001</EditRate> <IntrinsicDuration>719</IntrinsicDuration> <SourceEncoding>urn:uuid:0d28fd36-8485-490e-a06d-b2bdcebc8fbf</SourceEncoding> <TrackFileId>urn:uuid:c1e8dfc3-495b-44ba-a436-35027d0db0a3</TrackFileId> </Resource> </ResourceList></cc:AncillaryDataSequence>
• <SourceEncoding> references an <EssenceDescriptor> defined in the <EssenceDescriptorList>
• The actual “external metadata” is referenced by the <TrackFileId> element
• Intrinsic timeline metadata is stored in plain XML in the CPL• Extrinsic timeline metadata is stored using “any” encoding method as long as
there is a MXF wrapping defined• Extrinsic timeline metadata can be transmitted as supplemental packages
• Extrinsic timeline metadata can have any size
• Extrinsic timeline metadata can be reused multiple times (multiple references)• Dynamics are defined by the encoding. No “generic animation mechanism”
IMF mechanisms for carrying metadata
Adding timeline metadata
• Product placement• Per-frame technical metadata (e.g. camera metadata, geo-localisation)• Creative changes (pan & scan), colorimetry (ST2094), etc.• Support for legacy essence formats (EBU STL, CEA 608/708, etc.)• New accessibility tracks• Live & post-live capture metadata• Etc.
IMF mechanisms for carrying metadata
Examples of timeline metadata
• Instructions on how to “process” CPLs• List of standard (SMPTE) & custom macros• Each OPL references a CPL• Can have <ExtensionProperties> too!• Metadata access in the CPL via CPL UUID• Metadata access in the CPL’s virtual tracks via UUID
IMF mechanisms for carrying metadata
Metadata processing via OPLs
IMF mechanisms for carrying metadata
Metadata processing via OPLs...<MacroList> <Macro type="mtio:FileOutMacroType"> <Name>macro_9313321f-7f2b-4ced-939d-efbf54ca8dff</Name> <mtio:InputImageSequence xmlns:mtio="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro"> <Handle>cpl/virtual-tracks/a8bb2208-3915-43b1-ab51-1b8ea48ae8ff</Handle> </mtio:InputImageSequence> ... <mtio:RDD6Sequence xmlns:mtio="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro"> <Handle>cpl/virtual-tracks/cff02228-faac-7543-dd23-cba6695aa3e2</Handle> </mtio:RDD6Sequence> ... <mtio:OutputFormat xmlns:mtfileout="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro"> <mtio:Container>mxfas11ukdpp</mtfileout:Container> </mtio:OutputFormat> </Macro></MacroList>...
IMF mechanisms for carrying metadata
How to choose a strategy?PLACE ROLE GLOBAL TIMELINE
CPL-related Related to specific moments in the timeline
INTRINSIC Essential to CPL interpretation, XML based, smaller than the CPL
Use a child element in <ExtensionProperties>
Store as a resource in a virtual track (could be custom)
EXTRINSIC Opposite of intrinsic Stored as external asset (e.g. QC report, EBUCore XML documents, etc.
Stored as Track file, referenced by Virtual Track in CPL (e.g. ST2094)
• IMF does not make use of TIMECODE
• CPLs have an optional timecode metadata• MXF supports timecode• IMF MXFs can carry the timecode you want/like (e.g. original/source timecode)• However timecode is asset-bound, not composition-bound• Stop thinking tape
• Start thinking files, structured edit/playlists & assets
IMF mechanisms for carrying metadata
Last but not least: Timecode!
• Current practice: UNREADABLE & INCOMPETENT NAMING CONVENTIONS
• SMPTE 35PM50 working on a specification to “ease the attachement” of sidecar
files• EBU IMF-TV working on this subject too!• Send comments/feedback, participate!
IMF mechanisms for carrying metadata
Conclusion