continuity editing for 3d animation - inria · continuity editing for 3d animation quentin galvane...

Click here to load reader

Post on 14-Jul-2020

5 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Continuity Editing for 3D Animation

    Quentin Galvane Rémi Ronfard Christophe Lino Marc Christie

    Twenty-Ninth AAAI Conference

    2015

  • Motivation 2

    Grand Challenge: Automated film-making ( Mark Riedl AAAI – WICED 2014 )

    Specialized agents with cinematographic knowledge

    Director, actor, cameraman, film-editor

    With applications to games

  • 3 Objectives

    Read actions and dialogues from script

    Generate 3D animation

    Place cameras and lights, generate rushes

    Edit the rushes into a movie

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    4 Related work

    Idiom based solutions

    Scenario

    Virtual cinematographer [Christianson et al. 1996]

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    4 Related work

    Idiom based solutions

    Scenario

    Virtual cinematographer [Christianson et al. 1996]

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    4 Related work

    Idiom based solutions

    Scenario

    Virtual cinematographer [Christianson et al. 1996]

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    4 Related work

    Idiom based solutions

    Scenario

    Virtual cinematographer [Christianson et al. 1996]

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    4 Related work

    Idiom based solutions

    Scenario

    Virtual cinematographer [Christianson et al. 1996]

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    4 Related work

    Idiom based solutions

    Scenario

    Virtual cinematographer [Christianson et al. 1996]

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    5 Related work

    Scenario

    All cameras evaluated over the entire beat

    All transitions evaluated at beat changes

    [Riedl, M. et al., 2008]

    Optimization based approach

    Dynamic programming

  • … Goldie

    speaks to George

    George speaks to Goldie Goldie

    speaks to George

    Marty speaks to

    Goldie …

    6 Our approach

    Scenario

    Evaluate all possible transitions Evaluate the

    pacing

  • 7

    Film editing as an optimization problem

    Semi-Markov chains

    Create an editing graph that evaluates 3 aspects:

    Shot quality

    Cut quality

    Pacing

    Outline

  • Search over semi-Markov chains s = (rj, dj) given actions a(t)

    Minimize cost function:

    8

    Action cost (Shot quality)

    Transition cost (Cut quality)

    Rhythm cost (Pacing Quality)

    The final editing is given by the shortest path in the editing graph

    Film editing as optimization

  • Shot quality evaluated using:

    Action proximity

    Ensure that the cameras best cover the actions

    9 Shot Selection

  • Shot quality:

    Visibility computation

    10 Shot Selection

    Compute:

    Occluded area

    +

    Offscreen area

  • Shot quality:

    Hitchcock principle

    11 Shot Selection

    The size of a character on the screen should be proportional to its narrative importance in the story.

    • Narrative importance from script

    • Visible area V = S – O for each rush

  • Cut quality:

    Jump cuts

    Screen, motion and gaze continuity

    Left-to-right ordering

    12 Continuity editing

  • Pacing

    Shot durations follow a log-normal distribution

    13 Cutting rythm

  • 14 Results

  • 15 Results

    3 different versions edited:

    Original pacing

    Slower pacing

    Faster pacing

    User study: confirms that all 3 terms are important

    Shot quality, cut quality and pacing

  • Limitations & Future work 16

    Limitations

    Cameras must be pre-computed

    Cannot handle book-ending

    Context free grammar

    Cannot handle ellipsis or flashbacks

  • Limitations & Future work 17

    Future work

    Optimize over camera positions and movements

    Extend to real live video

    Learn cost function from real movies [Gandhi et al., 2014]

    [Galvane et al., 2014]

  • Conclusion 18

    Computational model for film editing

    Incorporate knowledge about shot composition and film grammar

    Efficient offline algorithm

    Suitable for automatic film making

    Dataset and results: https://team.inria.fr/imagine/continuity-editing/

  • 19 Continuity editing

    Diapositive numéro 1Diapositive numéro 2Diapositive numéro 3Diapositive numéro 4Diapositive numéro 5Diapositive numéro 6Diapositive numéro 7Diapositive numéro 8Diapositive numéro 9Diapositive numéro 10Diapositive numéro 11Diapositive numéro 12Diapositive numéro 13Diapositive numéro 14Diapositive numéro 15Diapositive numéro 16Diapositive numéro 17Diapositive numéro 18Diapositive numéro 19Diapositive numéro 20Diapositive numéro 21Diapositive numéro 22Diapositive numéro 23Diapositive numéro 24