david bordwell - watching you watch there will be blood

7/31/2019 David Bordwell - Watching You Watch THERE WILL BE BLOOD

1/12

3/9/12 Observations on film art : Watching you watch THERE WILL BE BLOOD

8/19www.davidbordwell.net/blog/2011/02/14/watching-you-watch-there-will-be-blood/

Yvonne TehZigzigger: On the Audiovisual and Beyond

Log in

The eyes mindTime piece

Watching you watch THERE WILL BE BLOOD

Monday | February 14, 2011 open printable version

DB here:

Todays entry is our first guest blog. It follows naturally from the last entry on how our eyes scan and

sample images. Tim Smith is a psychological researcher particularly interested in how movie viewerswatch. You can follow his work on his blog Continuity Boy andhis research site.

I asked Tim to develop some of his ideas for our readers, and he obliged by providing an experiment thattakes off from my analysis of staging in one scene ofThere Will Be Blood, postedhere back in 2008.

The result is almost unprecedented in film studies, I think: an effort to test a critics analysis againstmeasurable effects of a movie. What follows may well change the way you think about visual storytelling.

Tims colorful findings also suggest how research into art can benefit from merging humanistic andsocial-scientific inquiry. Kristin and I thank Tim for his willingness to share his work.

Tim Smith writes:

Davids previous post provided a nice introduction to eye tracking and its possible significance forunderstanding film viewing. Now it is my job to show you what we can do with it.

Continuity errors: How they escape us

Knowing where a viewer is looking is critical to beginning to understand how a viewer experiences afilm. Only the visual information at the centre of attention can be perceived in detail and encoded inmemory. Peripheral information is processed in much less detail and mostly contributes to our perceptionof space, movement and general categorisation and layout of a scene.

The incredibly reductive nature of visual attention explains why large changes can occur in a visual scenewithout our noticing. Clear examples of this are the glaring continuity errors found in some films.Lighting that changes throughout a scene, cigarettes that never burn down, and drinks that instantly refillplague films and television but we rarely notice them except on repeated or more deliberate viewing. Inmy PhD thesis I created a taxonomy of continuity errors in feature films and related them to various

failings during pre-production, filming, and post-production.

Our inability to detect continuity errors was elegantly demonstrated in a study by Dan Levin and DanSimons. In their study continuity errors were purposefully introduced into a film sequence of two women


2/12

3/9/12Observconversing across a dinner table. If you havent seen it before, watch the video here before continuing,and see how many continuity errors you can spot.

Two frames from the clip used by Levin and Simons (1997). Continuity errors were deliberately insertedacross cuts (e.g., the disappearing scarf), and viewers were asked after watching the video whether they

noticed any.

The short clip contained nine continuity errors, such as a scarf that changed colour, then disappeared,plates that changed colour and hands that changed position. During the first viewing, viewers were toldto pay close attention but were not informed about the continuity errors. When asked afterwards if they

noticed anything change, only one participant reported seeing anything and that was a vague sense thatthe posture of the actors changed. Even during a second viewing in which they were instructed to detectchanges, viewers only detected an average of 2 out of the 9 changes and tended to notice changes closestto the actors faces such as the scarf.

Although Levin and Simons did not record viewer eye movements, my own experiments investigatinggaze behaviour during film viewing indicate that our eyes will mostly be focussed on faces and spendvirtually no time on peripheral details. If you as a viewer dont fixate a peripheral object such as the plate,you are unable to represent the colour of the plate in memory and can, therefore not detect the change incolour when you later refixate it.

Tracking gaze

To see how reductive and tightly focused our gaze is whilst watching a film, consider Paul ThomasAndersons There Will Be Blood(TWBB; 2007). In an earlier post, David used a scene from this film asan example of how staging can be used to direct viewer attention without the need for editing.

The scene depicts Paul Sunday describing the location of his family farm on a map to Daniel Plainview,his partner Fletcher Hamilton, and his son H.W. The entire scene is treated in a long, static shot (with aslight movement in at the beginning). Most modern film and television productions would use rapidediting and close-up shots to shift attention between the map and the characters within this scene. Thisfrenetic style of filmmakingwhich David termed intensified continuity in his book The Way Hollywood

Tells It(2006)breaks a scene down into a succession of many viewpoints, rapidly and forcefullypresented to the viewer.

Intensified continuity is in stark contrast to the long-take style used in this scene from TWBB. The long-take style, which was common in the 1910s and recurred at intervals after that period, relies more onstaging and compositional techniques to guide viewer attention within a prolonged shot. For example,lighting, colour, and focal depth can guide viewer attention within the frame, prioritising certain parts ofthe scene over others. However, even without such compositional techniques, the director can stillinfluence viewer attention by co-opting natural biases in our attention: our sensitivity to faces, hands, andmovement.


3/12

3/9/12Observ

In order to see these biases in action during TWBB we need to record viewer eye movements. In a smallpilot study, I recorded the eye movements of 11 adults using an Eyelink 1000 (SR Research) eyetracker.This eyetracker uses an infrared camera to accurately track the viewers pupil every millisecond. Themovements of the pupil are then analysed to identifyfixations, when the eyes are relatively still and visualprocessing happens; saccadic eye movements (saccades), when the eyes quickly move between locationsand visual processing shuts down; smooth pursuit movements, when we process a moving object; andblinks.

Eye movements on their own can be interesting for drawing inferences about cognitive processing, butwhen thinking about film viewing, where a viewer looks is of most interest. As David demonstrated inhis last post, analysing where a viewer looks whilst viewing a static scene, such as Repins paintingAnUnexpected Visitor,is relatively simple. The gaze of a viewer can be plotted on to the static image andthe time spent looking at each region, such as a characters face or an object in the scene can be measured.

However, when the scene is moving, it is much more difficult to relate the gaze of a viewer on the screento objects in the scene. To overcome this difficulty, my colleagues and I developed new visualisationtechniques and analysis tools. These efforts were part of a large project investigating eye movementbehaviour during film and TV viewing (Dynamic Images and Eye Movements, what we call the DIEMproject). These techniques allow us to capture the dynamics of gaze during film viewing and display it inall its fascinating, frenetic glory.

To begin, the gaze location of each viewer is placed as a point on the corresponding frame of the movie.The point is represented as a circle with the size of the circle denoting how long the eyes have remainedin the same location, i.e.fixatedthat location. We then add the gaze location of all viewers on to the sameframe. Although the viewers watched the clip at different times, plotting all viewers together allows us tolook for similarities and differences between where people look and when they look there. This figureshows the gaze location of 8 viewers at one moment in the scene. (The remaining 3 viewers are blinkingat this moment.)

A snapshot of gaze locations of 8 viewers whilst watching the map sequence from There Will BeBlood

(2007). Each green circle represents the gaze location of one participant, with the size of thecircle indicating how long the eyes have been in fixation (bigger equals longer).


4/12

3/9/12ObservYou have a roving eye

Plotting static gaze points onto a single frame of the movie allows us to see what viewers were looking atin a particular frame, but we dont get a true sense of how we watch movies until we animate the gaze ontop of the movie as it plays back. Here is a video of the entire sequence from TWBB with superimposedgaze of 11 viewers.

You can also see it here. The main table-top map sequence we are interested begins at 3 minutes, 37

seconds.

The most striking feature of the gaze behaviour when it is animated in this way is the very fast pace atwhich we shift our eyes around the screen. On average, each fixation is about 300 milliseconds induration. (A millisecond is a thousandth of a second.) Amazingly, that means that each fixation of thefovea lasts only about 1/3 of a second. These fixations are separated by even briefer saccadic eyemovements, taking between 15 and 30 milliseconds!

Looking at these patterns, our gaze may appear unusually busy and erratic, but were moving our eyeslike this every moment of our waking lives. We are not aware of the frenetic pace of our attentionbecause we are effectively blind every time we saccade between locations. This process is known assaccadic suppression. Our visual system automatically stitches together the information encoded during

each fixation to effortlessly create the perception of a constant, stable scene.

In other experiments with static scenes, my colleagues and I have shown that even if the overall scene ishidden 150milliseconds into every fixation, we are still able to move our eyes around and find a desiredobject. Our visual system is built to deal with such disruptions and perceive a coherent world fromfragments of information encoded during each fixation.

The second most striking observation you may have about the video is how coordinated the gaze ofmultiple viewers is. Most of the time, all viewers are looking in a similar place. This is a phenomenon Ihave termedAttentional Synchrony. If several viewers examine a static scene like the Repin paintingdiscussed in Davids last post, they will look in similar places, but not at the same time. Yet as soon asthe image moves, we get a high degree of attentional synchrony. Something about the dynamics of amoving scene leads to all viewers looking at the same place, at the same time.

The main factors influencing gaze can be divided into bottom-up involuntary control by the visual sceneand top-down voluntary control by the viewers intentions, desires, and prior experience. As part of theDIEM project we were able to identify the influence of bottom-up factors on gaze during film viewingusing computer vision techniques. These techniques allowed us to dissect a sequence of film into itsvisual constituents such as colour, brightness, edges, and motion. We found that moments of attentionalsynchrony can be predicted by points of motion within an otherwise static scene (i.e. motion contrast).

You can see this for yourself when you watch the gaze video. Viewers gazes are attracted by the suddenappearance of objects, moving hands, heads, and bodies. The greater the motion contrast between thepoint of motion and the static background, the more likely viewers will look at it. If there is only onepoint of motion at a particular moment, then all viewers will look at the motion, creating attentional

synchrony.

This is a powerful technique for guiding attention through a film. But its of course not unique to film.Noticing points of motion is a natural bias which we have evolved by living in the real world. If we were


5/12

3/9/12Observnot sensitive to peripheral motion, then the tiger in the bushes might have killed our ancestors before theyhad chance to pass their genes down to us.

But points of motion do not exist in film without an object executing the movement. This brings us toDavids earlier analysis of the staging of this sequence from TWBB. This might be a good time to goback and read Davids analysis before we begin testing his hypotheses with eyetracking. Is David right inpredicting that, even in the absence of other compositional techniques such as lighting, cameramovement, and editing, viewer attention during this sequence is tightly controlled by staging?

All together now

To help us test Davids hypotheses I am going to perform a little visualisation trick. Making sense ofwhere people are looking by observing a swarm of gaze points can often be very tricky. Tosimplify things we can create a peekthrough heatmap. A virtual spotlight is cast around each gazepoint. This spotlight casts a cold, blue light on the area around the gaze point. If the gazes of multipleviewers are in the same location their spotlights combine and create a hotter/redder heatmap. Areas of theframe that are unattended remain black. By then removing the gaze points but leaving the heatmap weget a peekthrough to the movie which allows us to clearly see which parts of the frame are at the centreof attention, which are ignored and how coordinated viewer gaze is.

Here is the resulting peekthrough video; also available here. The map sequence begins at 3:38.

Here is the image of gaze location I showed above, now matched to the same frame of the peekthroughvideo.


6/12

3/9/12Observ

The gaze data from multiple viewers is used to create a peekthrough heatmap in which each gaze

location shines a virtual spotlight on the film frame. Any part of the frame not attended is black, and themore viewers look in the same location, the hotter the color.

Davids first hypothesis about the map sequence is that the faces and hands of the actors command ourattention. This is immediately apparent from the peekthrough video. Most gaze is focused on faces,shifting between them as the conversation switches from one character to another.

The map receives a few brief fixations at the beginning of the scene but the viewers quickly realise that itis devoid of information and spend the remainder of the scene looking at faces. The only time the map isfixated is when one of the characters gestures towards it (as above).

We can see the effect of turn-taking in the conversation on viewer attention by analyzing a fewexchanges. The sequence begins with Paul pointing at the map and describing the location of his familyfarm to Daniel. Most viewers gazes are focused on Pauls face as he talks, with some glances to otherfaces and the rest of the scene. When Paul points to the map, our gaze is channeled between his face andwhat he is gazing/pointing at.

Such gaze prompting and gesturing are powerful social cues for attention, directing attention along apersons sightline to the target of their gaze or gesture. Gaze cues form the basis of a lot of editingconventions such as the match an action, shot/reverse-shot dialogue pairings, and point-of-viewshots. However, in this scene gaze cuing is used in its more natural form to cue viewer attention within asingle shot rather than across cuts.


7/12

3/9/12Observ

As Paul finishes giving directions, Daniel asks him a question which immediately results in all viewersshifting the gaze to Daniels face. Gaze then alternates between Daniel and Paul as the conversationpasses between them. The viewers are both watching the speaker to see what he is saying and alsomonitoring the listeners responses in the form of facial expressions and body movement.

Daniel turns his back to the camera, creating a conflict between where the viewer wants to look (Danielsface) and what they can see (the back of his head). As David rightly predicted, by removing the currenttarget of our attention the probability that we attend to other parts of the scene is increased, such as H.

W., who up until this point has not played a role in the interaction. Viewers begin glancing towards HWand then quickly shift their gaze to him when he asks Paul how many sisters he has.

Gaze returns to Paul as he responds.

Gaze shifts from Paul to Daniel as he asks a short question, and then moves to Fletcher as he joins theconversation.


8/12

3/9/12Observ

The quick exchanges of dialogue ensure that viewers only have enough time to shift their gaze to thespeaker and then shift to the respondent. When gaze dwells longer on a speaker, such as during theexchange between Fletcher and Paul, there is an increase in glances away from the speaker to other partsof the scene such as the other silent faces or objects.

An object that receives more fixations as the scene develops is Pauls hat, which he nervously fiddleswith. At one point, when responding to Fletchers question about what they grow on the farm, Paulglances down at his hat. This triggers a large shift of viewer gaze, which slides down to the hat.Likewise, a subtle turn of the head creates a highly significant cue for viewers, steering them towardswhat Paul is looking at while also conveying his uneasiness.


9/12

3/9/12Observ

The most subtle gesture of the scene comes soon after as Fletcher asks about water at the farm. Paulstates that the water is generally salty and as he speaks Fletcher shifts his eyes slightly in the direction ofDaniel. This subtle movement is enough to cue three of the viewers to shift their gaze to Daniel,registering their silent exchange.

This small piece of information seems critical to Daniel and Fletchers decision to follow up Pauls lead,

but its significance can be registered by viewers only if they happened to be fixating Fletcher at the timehe glanced at Daniel. The majority of viewers are looking at Paul as he speaks and they miss the gesture.For these viewers, the significance of the statement may be lost, or they may have to deduce thesignificance either from their own understanding of oil prospecting or other information exchangedduring the scene.

The final and most significant gesture of the scene is Daniels threatening raised hand. As Paul goes toleave, Daniel stalls him by raising his hand centre frame in a confusing gesture hovering midwaybetween a menacing attack and a friendly handshake. In Davids earlier post he predicted that the handwould command our attention. Viewer gaze data confirm this prediction. Daniel draws all gazes to himas he abruptly states Listen.Paul, and lifts his hand.

Gaze then shifts quickly; the raised hand becomes a stopping off point on the way to Pauls face. . .


10/12

3/9/12Observ

. . . finally following Daniels hand down as he grasps Pauls in a handshake.

We like to watch

The rapid sequence of actions clearly guide our attention around the scene:Daniel Hand -Paul Hand. Davids analysis of how the staging in this scene tightly controls viewer attention was spot-on andcan be confirmed by eyetracking. At any one moment in the scene there is a principal action signifiedeither by dialogue or motion. By minimising background distractions and staging the scene in a clearsequential manner using basic principles of visual attention, P. T. Anderson has created a scene whichcommands viewer attention as precisely as a rapidly edited sequence of close-up shots.

The benefit of using a single long shot is the illusion of volition. Viewers think they are free to lookwhere they want but, due to the subtle influence of the director and actors, where they want to look isalso where the director wants them to look. A single static long shot also creates a sense of space, clearrelationship between the characters, and a calm, slow pace which is critical for the rest of the film. Thesame scene edited into close-ups would have left the viewer with a completely different interpretation ofthe scene.

I hope Ive shown how some questions about film form, style, practice, and spectatorship can beinformed by borrowing theory and methods from cognitive psychology. The techniques I have utilised inrecording viewer gaze and relating it to the visual content of a film are the same methods I would use if Iwas conducting an experiment on a seemingly unrelated topic such as visual search. (See this paper foran example.)

The key difference is that the present analysis is exploratory and simply describes the viewing behaviourduring an existing clip. What we cannot conclude from such a study is which aspects of the scene arecritical for the gaze behaviour we observe. For instance, how important is the dialogue for guiding

attention? To investigate the contribution of individual factors such as dialogue we need to manipulatethe film and test how gaze behaviour changes when we add or remove a factor. This type of empiricalmanipulation is critical to furthering our understanding of film cognition and employing all of the toolscognitive psychology has to offer.


11/12

3/9/12ObservBut I expect an objection. Isnt this sort of empirical inquiry too reductive to capture the complexities offilm viewing? In some respects, yes. This is what we do. Reducing complex processes down to simple,manageable, and controllable chunks is the main principle of empirical psychology. Understanding apsychological process begins with formalizing what it and its constituent parts are, and thensystematically manipulating and testing their effect. If we are to understand something as complex as howwe experience film we must apply the same techniques.

As in all empirical psychology the danger is always that we lose sight of the forest whilst measuring thetrees. This is why the partnership between film theorists and empiricists like myself is critical. Thedecades of film theory, analysis, practice and intuition provide the framework and Big Picture to whichwe empiricists contribute. By sharing forces and combining perspectives, we can aid each othersunderstanding of the film experience without losing sight of the majesty that drew us to cinema in the firstplace.

On the importance of foveal detail for memory encoding, see J. M. Findlay,Eye scanning and visualsearch, in The Interface of Language, Vision, and Action: Eye movements and the visual world, ed. J.M.Henderson and F. Ferreira (New York: Psychology Press, 2004), pp. 134-159. Levin and Simonscontinuity-error experiment is explained in D. T. Levin and D. J. Simons, Failure to detect changes toattended objects in motion pictures, Psychonomic Bulletin and Review4 (1997), pp. 501-506.

A note about our equipment and experimental procedure. We presented the film on a 21 inch CRT

monitor at a distance of 90cm and a resolution of 720328, 25fps. Eye movements were recorded usingan Eyelink 1000 eyetracker and a chinrest to keep the viewers head still. This eye tracker consists of abank of infrared LEDs used to illuminate the participants face and a high-speed infrared camera filmingthe face. The infrared light reflects of the face but not the pupil, creating a dark spot that the eyetrackerfollows. The eyetracker also detects the infrared reflecting off the outside of the eye (the cornea) whichappears as a glint. By analysing how the glint and the centre of the pupil move as the viewer looksaround the screen the eyetracker is able to calculate where the viewer is looking every millisecond.

As for the heatmaps, the greater the number of viewers, the more consistent the heatmaps. The presentpilot study used gaze from only 11 viewers, which introduces a lot of noise into the visualisations.Compare the scattered nature of the gaze in the TWBB video to a similar scene visualised with the gaze of48 viewers. We would probably see the same degree of coordination in the TWBB clip if we had used

more viewers.

For a comprehensive discussion of attentional synchrony and its cause, see Mital, P.K., Smith, T. J., Hill,R. and Henderson, J. M., Clustering of gaze during dynamic scene viewing is predicted bymotion, Cognitive Computation (in press). Social cues for attention, like shared looks, are discussedin Langton, S. R. H., Watt, R. J., & Bruce, V., Do the eyes have it? Cues to the direction of socialattention, Trends in Cognitive Sciences 4, 2, pp. 50-59. For more on our inability to detect smalldiscontinuities, see Smith, T. J. and Henderson, J. M., Edit Blindness: The relationship betweenattention and global change blindness in dynamic scenes,Journal of Eye Movement Research (2008) 2(2), 6, pp. 1-17.

For further information on the Dynamic Images and Eye Movement project (DIEM) please visit

http://thediemproject.wordpress.com/. This research was funded by the Leverhulme Trust (Grant RefF/00-158/BZ) and the ESRC (RES 062-23-1092). To view more visualisations from the project visit this

site. The DIEM project partners are myself, Prof. John M Henderson, Parag Mital, and Dr. Robin Hill.

Gaze data and visualisation tools (CARPE: Computational and Algorithmic Representation and Processing

of Eye-Movements) can also be downloaded from the website. When using or referring to any of the

work from DIEM, please reference the Cognitive Computation paper cited above.

Wonderful work in this area has already been conducted by Dan Levin (Vanderbilt), Gery dYdewalle(Leuven), Stephan Schwan (KMRC, Tbingen), and the grandfather of the recent revival in empiricalcognitive film theory, Julian Hochberg. I am indebted to their pioneering work and excited about takingthis research area forward.

Finally, I would like to thank David and Kristin for inviting me to describe some of my work on theirwonderful blog. I have been an avid follower of their work for years and David has been a greatsupporter of my research.

DB PS 26 February: The response to Tims blog has been astonishing and gratifying. Tens of


12/12

3/9/12Observthousands of visitors have read his essay here, and his videos have been viewed over 700,000 times onsites across the Web. Im very happy that so many non-psychologistsscholars, critics, and filmmakershave found something of value here. The extended discussion on Jim Emersons scanners site, in which Iparticipated a little, is especially worth reading. For more comments and replies from Tim and his team,go to Tims Continuity Boy blogpage and the DIEM teams Vimeo page. At Continuity Boy, Tim willpost more videos based on his groups experimental efforts.

DB PS 18 October: Tim has posted a new, equally interesting experiment on tracking non-visible (!)movement on his blogsite.

Last Modified: Tuesday | October 18, 2011 @ 22:23 open printable version

This entry was posted on Monday | February 14, 2011 at 8:07 am and is filed under Directors: Anderson, Paul Thomas,

Film comments, Film technique: Cinematography, Film technique: Performance, Film theory: Cognitivism, Readers'

Favorite Entries. Responses are currently closed, but you can trackback from your own site.

Comments are closed.

comments about the state of this website go to Meg Hamel.

hosted by www.topped-with-meat.com

david bordwell - watching you watch there will be blood

Documents