application-specific constraints for multimedia presentation generation
DESCRIPTION
Application-specific constraints for multimedia presentation generation. Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman CWI Amsterdam email: Joost.Geurts @cwi.nl. Talk overview. Generating multimedia automatically Cuypers multimedia generation engine Multimedia and constraints - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/1.jpg)
1
Application-specific constraints for multimedia presentation generation
Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman
CWI Amsterdam
email: [email protected]
![Page 2: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/2.jpg)
2
Talk overview
•Generating multimedia automatically
•Cuypers multimedia generation engine
•Multimedia and constraints–Quantitative constraints
–Qualitative constraints
•Cuypers demo
•Conclusion, future directions
![Page 3: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/3.jpg)
3
Generating adaptive multimedia
•Content–Large multimedia database
•System profile–PC, PDA, WAP
•Network profile–Modem, Gigabit
•User profile–Language, Interests, Abilities, Preferences
Too costly to author manually
![Page 4: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/4.jpg)
4
Cuypers multimedia generation engine
![Page 5: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/5.jpg)
5
Automatic multimedia generation
•Designer does not specify complete presentation……but only specifies requirements
•System automatically finds a solution which meets requirements
•How should the requirements be specified?–Declarative constraints
![Page 6: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/6.jpg)
6
Traditional use of constraints
•Constraint solving used for problems with:–Many variables–Large domains
•Based on domain reduction paradigm•Quantitative constraints
– Integer domain–Reduction by arithmetic relations
• Greater than (>)• Less than (<)• Equals (=)
![Page 7: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/7.jpg)
7
Drawbacks of quantitative constraints
•Too many (trivial) solutions that differ by:–1 pixel position, or–1 milliseconds in timing
•Not sufficiently expressivee.g. cannot specify “no overlap” constraint
•Too low levele.g. A.X2 B.X1
![Page 8: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/8.jpg)
8
Solution: qualitative constraints
•Example “A not overlap B”, “B after C”•Advantages:
–More intuitive–More expressive–Smaller domains
•Qualitative solutions translate automatically to lower level quantitative constraints
•New problem: What if constraints are insoluble?
![Page 9: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/9.jpg)
9
Solution: Constraint Logic Programming
•Combine Prolog unification and backtracking with constraint solving
•Use Prolog rules to generate constraints•Backtrack when constraints are insoluble
![Page 10: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/10.jpg)
10
Cuypers generation engine
•Multiple layers:
–Prolog rules to generate
constraints
–Qualitative constraints
translate to quantitative
constraints
–Solution of both constraints
provides sufficient
information for final
presentation
![Page 11: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/11.jpg)
11
Cuypers demo: scenario
•Client: User is interested in Rembrandt and wants to know about about the “chiaroscuro” technique
•Server: Query database•Server: Generate constraints according to:
–System profile–User profile–Network profile
•Server: Solve constraints / revise constraints
•Server: Generate SMIL presentation•Client: Play presentation
![Page 12: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/12.jpg)
12
Conclusions
•Quantitative constraintsare insufficient for automatic multimediapresentation generation. Also need
•Qualitative constraintsto allow intuitive and effectivehigh level specification, and
•Backtrackingfor revising specific constraintsthat cause the entire set to fail
![Page 13: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/13.jpg)
13
Future directions
•Best-first instead of depth-first–Choose “best” among possible solutions–Needs evaluation criteria
•Improve knowledge management–Make design knowledge declarative and explicit
–Preserve metadata in final presentation–Use standardized and reusable profiles
![Page 14: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/14.jpg)
14
![Page 15: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/15.jpg)
15
Need to make trade-offs
•Semantics–Convey message
•Aesthetics–Clear / nice layout
•Resources–Screen size, bandwidth
•Dimension may result in conflicting goals
![Page 16: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/16.jpg)
16
Quantitative Constraints% csp(+Ids, -Boxes)csp([IdA,IdB],[box(IdA,[x1:AX1, …]), box(IdB,[x1:BX1,…])]) :-
% get valuesmaxX(MaxX), maxY(MaxY),height(IdA,HeightA),widtht(IdA,WidthA),…% define domains
[AX1,AX2,BX1,BX2]::[0..MaxX], [AY1,AY2,BY1,BY2]::[0..MaxY],
% set width & heightAX2 – AX1 #= WidthA,AY2 – AY1 #= HeightA,…% constraintsAX2 #< BX1, % A left-of BAY1 #= BY1, % A top-align B,…true.
![Page 17: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/17.jpg)
17
Multimedia and Constraints
•Constraint Logic Programming–Domain reduction–Backtracking–Unification (matching rules)
•Qualitative Constraints–Non-integer domain–Allen’s 13 temporal interval relations in three dimensions
![Page 18: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/18.jpg)
18
Qualitative Constraints
•Example:Two images, A,BA left or right of BA not above or
below B
![Page 19: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/19.jpg)
19
Qualitative Constraints
% csp(+Ids, -Graph)csp([IdA, Idb], [edge(IdA,IdB,x,NoOverlap),…]) :-
% define domainsNoOverlap :: [b,b-,m,m-],Overlap :: [d,d-,s,s-,f,f-,e],
% constraintsedge(IdA,IdB,x,NoOverlap), % B not-overlap Aedge(IdA,IdB,y,Overlap), % B overlap Atrue.
![Page 20: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/20.jpg)
20
Qualitative Constraints
•Reasoning– Inverse:
edge(A,B,D,Value) <=> inverse(Value,RValue),edge(B,A,D,RValue).
– Equalityedge(A,B,D,V1), edge(A,B,D,V2) => V1 #= V2
– Transitiveedge(A,B,D,VAB), edge(B,C,D,VBC) =>
tr(VAB,VBC,VAC), % rule generation algorithm
edge(A,C,D,VAC).
•Translation rules to quantitative domainedge(A,B,D,b) => node(A,D/2,V2), node(B,D/1,V1)
V1 #< V2.
![Page 21: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/21.jpg)
21
Problems in generating multimedia
•Text documents are flexible–Add page, scrollbar,
–Template models
–Wrap text around images
•Multimedia documents are less flexible–No pages or scrollbars, no line-breaking or hyphenation
–Not based on text-flow–Feedback needed
•Linear process model does not work for multimedia
![Page 22: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/22.jpg)
22
Quantitative Constraints
•Example:Two images,
A,BA left-of BA top-align B
![Page 23: Application-specific constraints for multimedia presentation generation](https://reader035.vdocument.in/reader035/viewer/2022081520/56814fe5550346895dbdafe6/html5/thumbnails/23.jpg)
23
Cuypers generation engine
•Rhetoric/Semantic–Sequence, Example
•Communicative devices–Bookshelf, Slideshow
•Qualitative Constraints–A before B
•Quantitative Constraints–A.X2 < B.X1
•Presentation–SMIL