Introduction to Digital Media & New MediaWilliam J. MonerSeptember 5, 2012RTF 319 – Intro to Digital Media

Reading Due Today

• Manovich, L. (2002). The Language of New Media. pp. 43 – 74• Sefton-Green, J. (2005). “Timelines,

Timeframes and Special Effects: software and creative media production.”

HOUSEKEEPING

• Moner’s office hours are now MONDAY 9 – 12, CMA A6.117• Labs begin next week

(Tuesday/Wednesday)• Introduction to Classes shared drive

TODAY

• Introduction to NEW MEDIA• Discussion of SOFTWARE STUDIES as

part of MEDIA STUDIES• Characteristics of DIGITAL MEDIA

Lev Manovich

http://www.manovich.net

Director of the Software Studies Initiative at California Institute for Telecommunications and Information Technology. (CALIT2).

Professor at the Visual Arts Department, University of California - San Diego (UCSD)

Specializations:digital art and historytheory of digital culturedigital humanities

Julian Sefton-Green

Principal Research Fellow, London School of Economics, University of London, Department of Media and Communications

Specializations in youth media and technologyCommunity and education policyCreativity, learning, and arts researchhttp://www.julianseftongreen.net/

THE LANGUAGE OF NEW MEDIALev Manovich (2002)

NEW MEDIA

• Convergence of two distinct historical trajectories• COMPUTATIONAL MEDIA• Babbage’s early computer• Information systems, computers

• AUDIOVISUAL MEDIA• Daguerreotypes• Photographs, movies• Recording technologies• Transmission: television, radio

The Language of New Media• The arts and media have distinct visual

languages• strategies to organize information and viewers’

experiences

• Material properties of the computer• its use in modern society, the structure of its

interface and key software applications

• Contemporary visual culture• The internal organization, iconography, iconology

and viewer experience of various visual sites in our culture

• Contemporary information culture• maps, wayfinding, navigation, visual cues

Representation in New Media

• Any new media object represents, as well as helps to construct, some outside referent• a physically existing object; historical information

presented in other documents; a system of categories currently employed by culture as a whole or by some social groups or interests.

• New media representations are also always biased. • They represent / construct some features of

physical reality at the expenses of others, one world view among many, one possible system of categories among numerous others possible.

Principles of New Media: Numeric

All new media objects are composed of digital code (bits: 0s and 1s)• Can be described formally• Subject to algorithmic manipulation

Principles of New Media: Modularity

• Founded on various “discrete samples” • Image = “pixels” (picture elements, 1x1

square)• 3D image = “polygons” (triangles + for

meshes)• 3D object = “voxels” (3D elements, 1x1x1

cube)

• Objects can be combined into new media compositions without losing their individual characteristics

Principles of New Media: Automation

• Role of programming, scripting, and sequencing• Algorithmic in nature• “Automatic” rendering of scenes,

backgrounds, etc.• In computer games, enemies can only

perform certain actions in reaction to the avatar’s movements (avatar = on-screen representation of user)

Automation…

• Web and distributed systems rely on automation• Consider how Google *exists*• What does Google *do*?

From Manovich (2002)“By the end of the twentieth century, the problem became no longer how to create a new media object such as an image; the new problem was how to find the object which already exists somewhere.”

“This led to the next stage in media evolution: the need for new technologies to store, organize and efficiently access these media materials. These new technologies are all computer-based.”

“The emergence of new media coincides with this second stage of a media society, now concerned as much with accessing and re-using existing media as with creating new one.” (p. 55)

Principles of New Media: Variability

• Closely connected to automation and modularity• Automation: “instead of identical

copies a new media object typically gives rise to many different versions … in part automatically assembled by a computer.”• Modularity: “media elements

maintain their separate identity and can be assembled into numerous sequences under program control.”

Variability

• Media elements are stored in a database• Can separate “content” from interface

(think of the Yelp website versus the Yelp app)• User information can lead to automatic

customization (think Yelp’s geographic awareness)

Variability

• Branching interactivity • pathways emerge from user interactive

choices

• Hypermedia experiences assembled through different pathways• think of how your experience differs from

that of someone else’s when playing Sims Online or browsing YouTube

Variability

• Periodic updates ensure the user experience is never quite the same from version to version or moment to moment• Scalability in the visual and

informational sense• Think of all of the information contained in

Google Maps versus the way we interface with a traditional paper map• Except in virtual worlds, we could zoom

infinitely!

From Manovich (2002)In a post-industrial society, every citizen can construct her own custom lifestyle and "select" her ideology from a large (but not infinite) number of choices. Rather than pushing the same objects/information to a mass audience, marketing now tries to target each individual separately.”

New social logic: “Every visitor to a Web site automatically gets her own custom version of the site created on the fly from a database. The language of the text, the contents, the ads displayed — all these can be customized” by interpreting data. (p. 60)

Variability

• In new media (particularly interactive media), the user rests control from the storyteller• “By passing these choices to the user,

the author also passes the responsibility to represent the world and the human condition in it.” (p. 62)

Principles of New Media: Transcoding

New media’s structures now follow the established conventions of computer's organization of data.

The examples of these conventions:• data structures such as lists, records and

arrays• substitution of all constants by variables• separation between algorithms and data

structures• modularity

Implications for storytelling

Two distinct layers: the “cultural layer” and the “computer layer.”

Cultural layer:• Encyclopedia and

a short story• Story and plot• Composition and

point of view• Mimesis and

catharsis• Comedy and

tragedy

Computer layer:• process and packet

transmission• sorting and matching• function and variable• computer language

and a data structure

Manovich’s Project

• Distinguish “digital” from “new media”• Establish a domain of study called software studies• Moves beyond traditional media studies

domains (radio, television, film, media industries, media technology) to introduce the logic of computerization, digital communication, and human-computer interface (HCI)• Finds a middle ground between HCI and

media/cultural studies

PRODUCTION SOFTWARE

Software Functions

• Cut and paste• Creating new / opening / saving

documents• Image manipulation includes:• Cropping• Rotating• Resizing• Framing

Creative Media Production

• Sefton-Green asserts that mastery of Photoshop is a core or key digital production skill• Competency in digital imaging

becomes an important concern in media literacy• The image (creation, manipulation)

underpins nearly all digital media production endeavors

Filtering

• Addition of special effects to digital images• Blurs, warps, distortions• Preset options and customizations

available

Filters in other software

• Audio effects (envelopes, distortion, reverb, autotune)• Video filters (lighting and gels,

diffusion)• Motion graphics (motion blurs,

tweening)

QUESTION:

• Do you believe that once you’re familiar with one production suite, you can easily map the software skills to other production suites?• (e.g. once you know Final Cut, do you

think you could master Maya more easily?)

Photoshop and Layers

• Key concept• Layers can be used to “stack”

elements of an image• Also, you can:• Blend layers using various algorithms• Mix light sources and colors together• Combine and merge layers into a single

layer

From Layers to Timelines

• Timelines are suited for image sequences played back in rapid fashion frame by frame• VIDEO, ANIMATION

• Timelines are also suited for audio editing and sequencing

Channels and Tracks

• In audio, layers may be referred to as channels or tracks in an homage to traditional audio editing practices• In video, layers may be referred to as

channels

The Takeaway

• Software for digital production bear the burden of using traditional metaphors for their given spaces

PHOTOSHOP = DARKROOM EDITINGAUDACITY = TAPE-BASED AUDIO STUDIO ENGINEERINGAVID = REEL-TO-REEL VIDEOTAPE EDITING

Moving beyond metaphors

PROGRAMMING AND SCRIPTING• Adobe Flash permits (and, quite

frankly, requires) a level of aptitude in programming to unlock the value of the IDE (integrated development environment) using Actionscript

• After Effects (a timeline-based visual effects editing app) permits use of Javascript for common scripting behaviors (looping, randomization)

Programming

• Flash makes use of objects which emerge from a class of items (think Object = baseball; Class = ball)• Objects can possess properties and

have methods (or actions) associated with them• Object-oriented programming becomes

a fundamental competency when creating modern games, interactive environments, and sequences

Interactivity

With the advent of software and gaming, the communicative capabilities of the internet (communication protocols), the visual nature of the web (HTML/CSS) and now with the abundance of multi-touch screens (HCI, UX/UI), interactivity becomes a new mode of thinking about visual experiences and storytelling

Databases

How do we organize digital materials?• Libraries of common sound effects• Archival footage• Attaching tags, labels, information to

media

Digital technologies work via the logic of data

Synaesthesis / Translation / Comparative Effects

• Be aware / “beware” of metaphors• iTunes does not just sell “tunes”• Channels in Photoshop refer to channels

of light (Red/Green/Blue)• Metaphors may be a “head fake”• Translation of experiences between one

discipline to another• Be cognizant of the differences

For Monday

• Read Manovich (2002) (pp. 43-74; 115 – 160)• [the entire text is recommended reading]

• Read Kleinrock (2010)

[RECOMMENDED]• Read Okin (2005) & Jordan (1999)