-
Continuity Editing for 3D Animation
Quentin Galvane Rémi Ronfard Christophe Lino Marc Christie
Twenty-Ninth AAAI Conference
2015
-
Motivation 2
Grand Challenge: Automated film-making ( Mark Riedl AAAI – WICED 2014 )
Specialized agents with cinematographic knowledge
Director, actor, cameraman, film-editor
With applications to games
-
3 Objectives
Read actions and dialogues from script
Generate 3D animation
Place cameras and lights, generate rushes
Edit the rushes into a movie
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
4 Related work
Idiom based solutions
Scenario
Virtual cinematographer [Christianson et al. 1996]
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
4 Related work
Idiom based solutions
Scenario
Virtual cinematographer [Christianson et al. 1996]
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
4 Related work
Idiom based solutions
Scenario
Virtual cinematographer [Christianson et al. 1996]
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
4 Related work
Idiom based solutions
Scenario
Virtual cinematographer [Christianson et al. 1996]
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
4 Related work
Idiom based solutions
Scenario
Virtual cinematographer [Christianson et al. 1996]
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
4 Related work
Idiom based solutions
Scenario
Virtual cinematographer [Christianson et al. 1996]
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
5 Related work
Scenario
All cameras evaluated over the entire beat
All transitions evaluated at beat changes
[Riedl, M. et al., 2008]
Optimization based approach
Dynamic programming
-
… Goldie
speaks to George
George speaks to Goldie Goldie
speaks to George
Marty speaks to
Goldie …
6 Our approach
Scenario
Evaluate all possible transitions Evaluate the
pacing
-
7
Film editing as an optimization problem
Semi-Markov chains
Create an editing graph that evaluates 3 aspects:
Shot quality
Cut quality
Pacing
Outline
-
Search over semi-Markov chains s = (rj, dj) given actions a(t)
Minimize cost function:
8
Action cost (Shot quality)
Transition cost (Cut quality)
Rhythm cost (Pacing Quality)
The final editing is given by the shortest path in the editing graph
Film editing as optimization
-
Shot quality evaluated using:
Action proximity
Ensure that the cameras best cover the actions
9 Shot Selection
-
Shot quality:
Visibility computation
10 Shot Selection
Compute:
Occluded area
+
Offscreen area
-
Shot quality:
Hitchcock principle
11 Shot Selection
The size of a character on the screen should be proportional to its narrative importance in the story.
• Narrative importance from script
• Visible area V = S – O for each rush
-
Cut quality:
Jump cuts
Screen, motion and gaze continuity
Left-to-right ordering
12 Continuity editing
-
Pacing
Shot durations follow a log-normal distribution
13 Cutting rythm
-
14 Results
-
15 Results
3 different versions edited:
Original pacing
Slower pacing
Faster pacing
User study: confirms that all 3 terms are important
Shot quality, cut quality and pacing
-
Limitations & Future work 16
Limitations
Cameras must be pre-computed
Cannot handle book-ending
Context free grammar
Cannot handle ellipsis or flashbacks
-
Limitations & Future work 17
Future work
Optimize over camera positions and movements
Extend to real live video
Learn cost function from real movies [Gandhi et al., 2014]
[Galvane et al., 2014]
-
Conclusion 18
Computational model for film editing
Incorporate knowledge about shot composition and film grammar
Efficient offline algorithm
Suitable for automatic film making
Dataset and results: https://team.inria.fr/imagine/continuity-editing/
-
19 Continuity editing
Diapositive numéro 1Diapositive numéro 2Diapositive numéro 3Diapositive numéro 4Diapositive numéro 5Diapositive numéro 6Diapositive numéro 7Diapositive numéro 8Diapositive numéro 9Diapositive numéro 10Diapositive numéro 11Diapositive numéro 12Diapositive numéro 13Diapositive numéro 14Diapositive numéro 15Diapositive numéro 16Diapositive numéro 17Diapositive numéro 18Diapositive numéro 19Diapositive numéro 20Diapositive numéro 21Diapositive numéro 22Diapositive numéro 23Diapositive numéro 24