multimedia resource adaptation using xml within the mpeg ...timse/stud/privatissimum02(2x).pdf ·...

Privatissimum

Multimedia Resource Adaptation using XML within the MPEG-21 Multimedia FrameworkChristian Timmerer

Univ. of Klagenfurt: Hermann Hellwagner, Harald KoschSiemens Munich: Andreas Hutter, Jörg Heuer, Gabriel Panis

Klagenfurt, 2002/12/02

2002/12/02 Christian Timmerer, [email protected] 2

Privatissimum

Outline

• Introduction – Retrospection• ISO/IEC WD 21000-7: Digital Item

Adaptation• (generic) Bitstream Syntax

Description ((g)BSD)• Adaptation Experiments• Outcomes• Conclusion/Future Work


Privatissimum

Introduction - Retrospection

mpeg2ds

Adaptation

Synchronisation


Privatissimum

Introduction – Retrospection (cont’d)

• MPEG-7 Variation DS– Extraction of Descriptors

• DiffServ– discarded ;-)

• MPEG-21/DIA/BSDL– Analysis, Extensions, Experiments

• Synchronisation– never started ;-)


Privatissimum

Outline





Privatissimum

Digital Item Adaptation

• … provide interoperable and transparent access to multimedia content

• … without engaging in preparation of network and terminal devices

• … to consume and/or produce this content

To enable transparent and augmented use of multimedia resources across a wide range of networks and devices


Privatissimum

Digital Item Adaptation (cont’d)

A A’ AdaptationEngine (DIA)

Content Digital Item

(input)

Content Digital Item

(output)

Context Digital Items

B1 … Bn


Privatissimum

Digital Item Adaptation (cont’d)

• User Characteristics• Terminal Capabilities• Network Characteristics• Natural Environment Characteristics• Resource Adaptability

– Bitstream Syntax Description– Terminal and Network QoS (AdaptationQoS)– Metadata Adaptability

• Session Mobility


Privatissimum

Outline





Privatissimum

Bitstream Syntax Description

• Generic approach to provide interoperability in DIA by using XML

• … to describe the high-level structure of a bitstream in a scalable way

• Codec aware processor for generating description and/or bitstreamBSDL*

• Codec independent schema gBSSchema developed at Univ. of Klagenfurt in cooperation with Siemens Munich

* … Philips Research France


Privatissimum

Bitstream Syntax Description (cont’d)

Adaptation Architecture


Privatissimum

generic Bitstream Syntax Description

• Universal, coding format independent schema for describing bitstreams

• Means to adapt media resources w.r.t. semantic aspects (“semantic handles”) and/or syntactical labels

• Introduces elements for describing the bitstream in a hierarchical fashion

Privatissimum

gBS Schema


Privatissimum

gBSDSample

<?xml version="1.0" encoding="UTF-8"?> <gBSD xsi:schemaLocation="gBSD gBSSchema.xsd"

xmlns="urn:mpeg:mpeg21:dia:schema:gBSD:2003" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"> <Header>

<ClassificationAlias alias="MV4" href="urn:mpeg:mpeg4:video:cs:syntacticalLabels"/> <DefaultValues addressUnit="byte" addressMode="Absolute"

globalAddressInfo="Content/starwars.cmp"/> </Header> <gBSDUnit syntacticalLabel=":MV4:VO" start="0" length="26"/>

<gBSDUnit start="26" length="99983" marker="violent-5"> <gBSDUnit syntacticalLabel=":MV4:I_VOP" start="26" length="2877"/> <gBSDUnit syntacticalLabel=":MV4:P_VOP" start="2903" length="64"/>

<gBSDUnit syntacticalLabel=":MV4:P_VOP" start="2967" length="56"/>  </gBSDUnit>

<gBSDUnit start="456749" length="52009" marker="violent-4"> <gBSDUnit syntacticalLabel=":MV4:P_VOP" start="456749" length="1986"/> <gBSDUnit syntacticalLabel=":MV4:I_VOP" start="458735" length="1695"/>

 <gBSDUnit syntacticalLabel=":MV4:P_VOP" start="506268" length="1310"/> <gBSDUnit syntacticalLabel=":MV4:P_VOP" start="507578" length="1180"/>

</gBSDUnit> <gBSDUnit start="508758" length="44235" marker="violent-3"> <gBSDUnit syntacticalLabel=":MV4:I_VOP" start="508758" length="4648"/>

<gBSDUnit syntacticalLabel=":MV4:P_VOP" start="513406" length="3909"/>  </gBSDUnit>

 </gBSD>


Privatissimum

Outline





Privatissimum

Adaptation Experiments (CEs)

• Main issuesI. Multi-step adaptationII. Description generationIII. Compactness of the DescriptionIV. Functionality and feasibility of the marker

approach to control the adaptation process

• Resources (Content)

– MPEG-4 Visual Elementary Streams– JPEG2000 Images (performed by Siemens Munich)


Privatissimum

Adaptation Experiments (cont’d)

• Input– Test bitstreams– Schemas (BSDL-1, BSDL-2, BS Schemas, gBS Schema)– BintoBSD, BSDtoBin, BiM software– MPEG-7/21 Descriptions

• Output– gBSDs for test bitstreams– XSLT style sheets for generating gBSDs and performing the

adaptation– Adapted bitstreams and transformed gBSDs– gBSDtoBin software– Reports on results, conclusions and recommendations


Privatissimum

Adaptation Experiments (cont’d)Multi-step Adaptation Architecture

DI Provider Pre-Processing

First Adaptation

Second Adaptation

. . . . .. . . . .

Scalable Bitstream BintoXML

Codec (bitstream) specific BSDL Schema

BS Description

BSDLtogBSD(XSLT)

gBSDescript. 1

Med

ia R

eso

urc

e A

dap

tati

on

Seq

uen

ce

Adapted Scal. Bitstream

gBSDescript. 2

XSLT + gBSDtoBin

Adapted Bitstream

gBSDescript. 3

XSLT + gBSDtoBin

AdaptationHint

AdaptationHint

markerInfo

Privatissimum


• Feasibility and Functionality of gBSD– Correctness:

• gBSD descriptions and their (corrected) adaptations are valid descriptions of bitstreams and their adaptations

– Adaptations: • Described & automatically performed by XSLT style

sheets– Sequence of adaptations:

• Successfully performed– gBSDs could be binarized using BiM codec

• Outstanding– Control of adaptations


Privatissimum


• Control of the Adaptation Process– MPEG-7 ParentalRating Descriptor

• ICRAParentalRatingViolenceCS

<Classification>

<ParentalGuidance>

<ParentalRatinghref="urn:mpeg:mpeg7:cs:ICRAParentalRatingViolenceCS:2001:5"/>

<Region>de</Region>

</ParentalGuidance>

</Classification>

– Input to AdaptationHint

<xsl:template name="tplMarker"match="gbsd:gBSDUnit[@marker=document

('parameter.xml')/AdaptationHint/Component[@name='marker']/Value]">



</xsl:template>

<xsl:template name="tplMarker"match="gbsdl:gBSDUnit[@syntacticalLabel=document

('parameter.xml')/AdaptationHint/Component[@name='syntacticalLabel']/Value]">



</xsl:template>

<?xml version="1.0" encoding="UTF-8"?><AdaptationHint

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:noNamespaceSchemaLocation="../Schemas/AdaptationHint.xsd"><Component name="syntacticalLabel">

<Value>:MV4:B_VOP</Value></Component><Component name="marker">

<Value>violent-1</Value></Component><Component name="marker">

<Value>violent-2</Value></Component>

</AdaptationHint>

– XSLT transformation


Privatissimum

Adaptation Experiments (cont’d)Results

Filename/Method

Bitstream [bytes]

XML descr. [bytes]

BiM size[bytes]

XML/ Bitstream

BiM/ Bitstream

foreman

akiyo

starwars

BSDL*) 572100 38198 ? 6,68% ?

gBSD 572100 22287 6107 3,90% 1,07%

BSDL*) 165034 37821 ? 22,92% ?

gBSD 165034 21828 6085 13,23% 3,69%

BSDL*) 2515959 230474 ? 9,16% ?

gBSD 2515959 131955 36314 5,24% 1,44%

*) without MPEG-4 Visual ES Schema

6107


Privatissimum

Outline





Privatissimum

Outcomes

• 2 Proposals for MPEG-21 DIA*– Additional requirements for DIA– generic Bitstream Syntax Description (Language)

• 2 CE Reports*• 1 Publication submitted to Image Communication

Eurasip Journal*• Promotion to Working Draft during Shanghai

Meeting (Oct. 2002)*• 1 Ongoing CE*

– 7 participating Companies/Institutes/Departments

A MPEG-21 Tool for generic resource adaptation* … Co-Author


Privatissimum

Outline





Privatissimum

Conclusion

• gBSD provides a flexible and standardized way for describing resources

• gBSD enables coding format independent adaptation

• gBSD allows for an efficient, optimized implementation– E.g., no schema delivery to adaptation node

• gBSD’s marker concept:– Provides a handle to include semantic adaptation info– Supports powerful adaptations on “thin devices”

(through provider-side preparation)


Privatissimum

Future Work

• … finishing master thesis• … gBSD CD FCD FDIS• … integration of MPEG-21 DIA Engine

into Proxy-Server• … Digital Item Processing, Event

Reporting …• … parallel/distributed adaptation• … implementation/realisation …


Privatissimum

The Big Picture


Privatissimum

Backup Slides


Privatissimum

XSLT - BSDtogBSD <xsl:template name="calcLen"> <xsl:param name="nodes"/> <xsl:param name="index" select="1"/>

<xsl:param name="runningTotal" select="0"/> <xsl:variable name="start" select="substring-before($nodes[$index]/mp4:Payload, '-') - 5"/>

<xsl:variable name="currentLength" select="substring-after($nodes[$index]/mp4:Payload, '-') - $start + 1"/> <xsl:variable name="remainingLength">

<xsl:choose> <xsl:when test="$index=count($nodes)"> <xsl:value-of select="0"/>

</xsl:when> <xsl:otherwise> <xsl:call-template name="calcLen">

<xsl:with-param name="nodes" select="$nodes"/> <xsl:with-param name="index" select="$index+1"/> <xsl:with-param name="runningTotal"

select="$runningTotal+$currentLength"/> </xsl:call-template> </xsl:otherwise>

</xsl:choose> </xsl:variable> <xsl:value-of select="$currentLength+$remainingLength"/>

</xsl:template>


Privatissimum

XSLT – BSDtogBSD (cont’d)

<xsl:variable name="sumLen"> <xsl:call-template name="calcLen"> <xsl:with-param name="nodes"

select="following-sibling::mp4:VOP[position() < $lengthFrame]"/> <xsl:with-param name="index" select="1"/> <xsl:with-param name="runningTotal" select="0"/>

</xsl:call-template> </xsl:variable>

Privatissimum

MPEG-7 - ParentalRating<VideoSegment id="preface"> <CreationInformation> <Creation>

<Title xml:lang="en" type="popular">Segment 1</Title> <Abstract> <FreeTextAnnotation>

Preface of Starwars2 Trailer </FreeTextAnnotation> </Abstract>

</Creation> <Classification> <ParentalGuidance>

<ParentalRating href="urn:mpeg:mpeg7:cs:ICRAParentalRatingViolenceCS:2001:5"/> <Region>de</Region>

</ParentalGuidance> </Classification> </CreationInformation>

<MediaTime> <MediaRelIncrTimePoint>0</MediaRelIncrTimePoint> <MediaIncrDuration mediaTimeUnit="PT1N30F">

275 </MediaIncrDuration> </MediaTime>

</VideoSegment>


Privatissimum

Multi-step Adaptation<?xml version="1.0" encoding="UTF-8"?><Bitstream xmlns="mpeg4" xmlns:xsi="http://www.w3.org/2000/10/XMLSchema-instance" xsi:schemaLocation="mpeg4 MPEG4Bitstream.xsd"> <Header start=”0” end=”17”>akiyo.mpg4</Header> <I_VOP start=”18” end=”4658”>akiyo.mpg4</I_VOP> <P_VOP start=”4659” end=”4756”>akiyo.mpg4</P_VOP> <B_VOP start=”4757” end=”4772”>akiyo.mpg4</B_VOP> <B_VOP start=”4773” end=”4795”>akiyo.mpg4</B_VOP> <P_VOP start=”4796” end=”4973”>akiyo.mpg4</P_VOP> <B_VOP start=”4974” end=”5026”>akiyo.mpg4</B_VOP> <B_VOP start=”5027” end=”5065”>akiyo.mpg4</B_VOP> <P_VOP start=”5066” end=”5300”>akiyo.mpg4</P_VOP> </Bitstream>

<?xml version="1.0" encoding="UTF-8"?><Bitstream xmlns="mpeg4" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="mpeg4 Schema_MPEG4.xsd">

<Header>akiyo.mpg4#0-17</Header><I_VOP start="18" end="4658">akiyo.mpg4</I_VOP><P_VOP start="4659" end="4756">akiyo.mpg4</P_VOP><P_VOP start="4757" end="4934">akiyo.mpg4</P_VOP><P_VOP start="4935" end="5169">akiyo.mpg4</P_VOP>

</Bitstream>

NO gap in the addresses


Privatissimum

Multi-step Adaptation (cont’d)

• Address corrections– Address information may become outdated after

performing an adaptation

– Following attributes are affected• globalAddressInfo

• start (in case of absolute or offset address mode)

• length

• Two possibilities– … after adaptation process (in a separate process)

– … incorporated into bitstream generation process

multimedia resource adaptation using xml within the mpeg ...timse/stud/privatissimum02(2x).pdf ·...

Documents