alice and media computation: goals for this session show interesting things you can do combining...

Alice and Media Computation:Goals for this sessionShow interesting things you can do combining

Alice and Media Computation.Provides a story-telling motivation to explore image and

sound manipulation

To see how one integrates Media Computation images and sounds into Alice.

In slide set:A brief Java example of ChromakeyA set of Python utilities to make simple image and sound

manipulation easier. To focus more on Alice and less on textual

programming, to start.

“Can’t I do everything in Alice?”Why do programming outside of Alice?Sometimes you want special images or

sounds in your Alice story.You can make them by programming those

images and sounds, then bringing them into Alice.

Sometimes you want to add in special movie effects to your Alice movie.

Overall, programming outside of Alice is the special effects studio for Alice.

Alice and Media Computation Intro3

Sound Frequency ProjectCreate an Alice world

with 3 frogsResize one to be bigger

and one to be smaller

Using croak.wavCreate a higher sound

Take every other sampleCreate a lower sound

Take every sample twice

Import the sound files and have the frogs play their sounds and tap their foot

Alice and Media Computation Intro4

Using ChromakeyCreate objects in Alice

and position them as desiredDelete the ground Set the color of the

background to green in the world properties

Use a conditional to replace the green colorYou may also need to

scale one of the pictures

Alice and Media Computation Intro5

Drawing ProjectsDraw something on an Alice

picture Like a hat, glasses, or a scarf Fireworks

Create images in Media Computation that you use as textures in Alice like a gradient for the ground

Clip an Alice picture To look like a porthole

Create a poster advertisement for your Alice movie. With your name on it

Create a comic panel with Alice pictures

Alice and Media Computation Intro6

Comic Strip ProjectCreate several scenes in Alice and take

pictures of the scenesUse Media Computation to create a Comic

Panel and/or Comic StripsDraw different types of text balloons

Speech balloon, thought balloon, scream balloon

Alice and Media Computation Intro 7

Add Yourself to an Alice Movie

Alice and Media Computation Intro8

Add Alice Characters to a Real Movie

Split out frames from a movie of peopleCapture frames from an Alice movie using MovieCapturePanelUse chromakey to copy the Alice items to the real movie

How to use image manipulation with AliceA. Create an image to appear as a Billboard

in an Alice world.B. Create images in Alice that you can

manipulate as snapshots.Like with chromakey

C. Make movies in Alice that you can then manipulate as video.

A. Loading images as billboards in AliceStep 1: Make a cool image that you want to

appear as an object in your Alice world.Step 2: Import the image as a Billboard.

Step 3: Move it around and use it like an object

B. Manipulating an Alice world snapshotStep 1: Set up an Alice world the way you

want it.If you want to chromakey,

drag the “Ground” into the trash make the “Atmosphere Color” of the world blue

(default) or green.

Creating the snapshotStep 2: Program your objects to move

where you want them to go.Step 3: Play your world, and at the right

time, hit “Pause” and “Take Picture”

Snapshot from Tutorial,Moved to the Moon

>>> capturedbirds = makePicture( getMediaPath("capture00.jpeg"))>>> capturedbirdsPicture, filename W:\mediasources\capture00.jpeg height 360 width 666>>> birds = scale(capturedbirds, 0.75)>>> birdsPicture, filename None height 271 width 500>>> chromakeyBlue(birds,moon)

C. Manipulating an Alice videofrom Python

Step 1. Tell Alice not to remove frames after making a video! (Change Preferences.)Only have to dothis once.

Record your videoStep 2: From File menu, choose “Export

Video.”Click “Record” then “Play”When done, “Stop Recording” and

“Export Video.”

Your frames are now saved

Doing Chromakey to a Set of Framesimport osdef chromakeySet(frames,background): newdir = "C:/temp/" for file in os.listdir(frames): if file.endswith(".jpg"): printNow(file) frame = makePicture(frames+file) chromakeyBlue(frame,background) writePictureTo(frame,newdir+file)

Using Chromakey to put Penguin in the jungle

>>> jungle = makePicture( getMediaPath("jungle.jpg"))>>> chromakeySet( "W:/mediasources/frames/", jungle)

Using newly created sounds in Alice

Making your characters play your sound

Choosing to import a sound

All set!

A Java-based exampleDoing Chromakey with an Alice-generated

image, from Java, to demonstrate the analagous methods.

Get a picture from Alice for Chromakey from JavaClick on Edit and then

on Preferences and then on the Screen Grab tabChange image format

to png (has loseless compression)

Set up the scene in AliceAdd the objects and

position them as desired

ChromakeyAlice 25

Get a picture from Alice for ChromakeyRight click on ground in

the object treeAnd then select delete

If you need to change the sky (atmosphere) colorClick on world in the

object treeThen click on properties

in the detail windowChange the

atmosphereColor to green

ChromakeyAlice 26

Save the picture to a fileClick on the Play

button in AliceThen click on the

Pause button to stop the action when you want to take a picture

Click on the Take Picture button to save a picture Usually on the

desktop as captureXX.png

ChromakeyAlice 27

Chromakey Method /** * Method to do chromakey using the passed background * color and the distance to the color * @param newBg the new background image to use to replace * @param color the background color to compare to * @param dist the distance that limits the chromakey * it will happen if the distance is less than or equal * to this value */ public void chromakey(Picture newBg, Color color, double dist) { Pixel currPixel = null; Pixel newPixel = null;

ChromakeyAlice 28

Chromakey Method - cont// loop through the columns for (int x=0; x<getWidth(); x++) {

// loop through the rows for (int y=0; y<getHeight(); y++) {

// get the current pixel currPixel = this.getPixel(x,y);

/* if the color at the current pixel is within the passed distance of the * chromakey color then change the background */ double currDist = currPixel.colorDistance(color);

ChromakeyAlice 29

Chromkey Method - cont if (currDist <= dist) { newPixel = newBg.getPixel(x,y); currPixel.setColor(newPixel.getColor()); } } } }

ChromakeyAlice 30

Chromakey onto current picture /** * Method to do chromakey by copying the pixels that have a larger

distance * to the passed source color than the passed distance. * @param sourcePict the picture with the background color * @param sourceColor the background color to compare to * @param dist the distance from the color to use * @param xSourceStart the starting X for the source picture * @param ySourceStart the starting Y for the source picture * @param xSourceEnd the ending X for the source picture * @param ySourceEnd the ending Y for the source picture * @param xTarget where to copy to in target (this) * @param yTarget where to copy to in target (this) */

ChromakeyAlice 31

Chromakey Method public void chromakey(Picture sourcePict,

Color sourceColor,

double dist,

int xSourceStart, int ySourceStart,

int xSourceEnd, int ySourceEnd,

int xTarget, int yTarget) {

Pixel currPixel = null;

Pixel newPixel = null;

// loop through the columns and rows

for (int x=xSourceStart, tX = xTarget; x<xSourceEnd; x++, tX++) {

for (int y=ySourceStart, tY = yTarget; y<ySourceEnd; y++, tY++) {

ChromakeyAlice 32

Chromakey - cont // get the current pixel currPixel = sourcePict.getPixel(x,y);

/* if the color at the current pixel is greater than the * passed distance from the source color then copy it */ if (currPixel.colorDistance(sourceColor) > dist && tX < this.getWidth() && tY < this.getHeight()) { newPixel = this.getPixel(tX,tY); newPixel.setColor(currPixel.getColor()); } } } }

ChromakeyAlice 33

Prepare for chromakeyGet the color that

the background isBlue = (73,124,204)

Figure out the boundary of what you want to copy0,0, width-5,height

Figure out where to copy to155,170

ChromakeyAlice 34

Chromakey ResultPicture dino = new Picture(FileChooser.pickAFile())Picture beach = new Picture("beach.jpg");import java.awt.Color;beach.chromakey(dino,new Color(73, 124, 204),10,0,0, dino.getWidth()-

5,dino.getHeight(),155,170)beach.explore();

ChromakeyAlice 35

ChallengeTry to use the

chromakey method on an Alice pictureYou can use one you

have createdOr use one from the

AlicePicts folder

ChromakeyAlice 36

Introducing Image Manipulation for AliceWe’ll start with Python, introducing some

very simple code for making it easy to do the image and sound manipulations for Alice.Available in the python-book folder.

We’ll do one example in Java.

What color shirt? Where is it?

Y=210

X=220X=400

>>> shirt = makePicture(pickAFile())>>> explore(shirt)>>> print shirtPicture, filename C:\mediasources\IMG_0808.JPG height 480 width 640

distance() function tells how close colors are>>> print whitecolor r=255 g=255 b=255>>> print blackcolor r=0 g=0 b=0>>> print distance(white,black)441.6729559300637>>> print redcolor r=255 g=0 b=0>>> print pinkcolor r=255 g=175 b=175>>> print distance(red,pink)247.48737341529164

Changing the whole shirtdef changeAllShirt(picture): myGreen = makeColor(132,240,204) for pixel in getPixels(picture): color = getColor(pixel) if distance(color,myGreen) < 200: setColor(pixel,pink)

Results

Just doing part of the picture, and changing fewer pixelsdef changeShirt(picture): myGreen = makeColor(132,240,204) for x in range(220,400): for y in range(210,480): pixel = getPixel(picture,x,y) color = getColor(pixel) if distance(color,myGreen) < 50: setColor(pixel,pink)

Results

Coloring the shirt from a different picture

>>> beach = makePicture(pickAFile())>>> shirt = makePicture(pickAFile())

Swap a new background for a colordef newBackgroundShirt(picture,background): myGreen = makeColor(132,240,204) for x in range(220,400): for y in range(210,480): pixel = getPixel(picture,x,y) color = getColor(pixel) if distance(color,myGreen) < 100: backgroundPixel = getPixel(background,x,y) backgroundColor = getColor(backgroundPixel) setColor(pixel,backgroundColor)

Results

How do we change the size of a picture?To copy a picture, get the color values from

pixels in one picture, and set those color values in the pixels in the other picture.

def copyPicture(picture): returnPicture =

makeEmptyPicture(getWidth(picture),getHeight(picture)) for pixel in getPixels(picture): color = getColor(pixel)

setColor(getPixel(returnPicture,getX(pixel),getY(pixel)),returnPicture)

return returnPicture

To scale the picturesmaller = scale(picture,0.5)How does it work?

To get it to be only ½ the size (in both horizontal and vertical) directions, we need to lose some pixels.

Easiest way: Skip a few. Every other pixel gets copied.

Can generalize this for any size scaling.

Code to scaledef scale(picture,factor): newHeight = int(factor*getHeight(picture))+1 newWidth = int(factor*getWidth(picture))+1 returnPic = makeEmptyPicture(int(newWidth),int(newHeight)) sx = 0 for tx in range(0,newWidth): sy = 0 for ty in range(0,newHeight): if (int(sx) < getWidth(picture)) and (int(sy) < getHeight(picture)): sp = getPixel(picture,int(sx),int(sy)) tp = getPixel(returnPic,tx,ty) setColor(tp,getColor(sp)) sy = sy + (1/factor) sx = sx + (1/factor) return returnPic

How do we grab part of a picture? guys =

makePicture(getMediaPath("IMG_0805.JPG"))james = copyPartPicture(guys,352,217,618,475)

copyPartPicture(picture,startX,startY,endX,endY)Gives you a new picture inside the box defined by upper-

left corner and lower-right corner.

Code to grab part of a picturedef copyPartPicture(picture,x1,y1,x2,y2): returnPicture = makeEmptyPicture(x2-x1,y2-y1) targetx = 0 for srcx in range(x1,x2): targety = 0 for srcy in range(y1,y2): srcpixel = getPixel(picture,srcx,srcy) targetPixel = getPixel(returnPicture,targetx,targety) setColor(targetPixel,getColor(srcpixel)) targety = targety + 1 targetx = targetx + 1 return returnPicture

How do we combine parts of a picture?grayscale(betsy)clearRed(james)pastePicture(betsy,scale(james,0.35),0,0)

pastePicture(canvas,partial,startX,startY)Pasting “partial” picture onto the “canvas”

starting from (startX,startY)

Code to paste a picturedef pastePicture(canvas,source,startx,starty): targetx = startx for x in range(0,getWidth(source)): targety = starty for y in range(0,getHeight(source)): srcpixel = getPixel(source,x,y) if (targetx < getWidth(canvas)) and (targety <

getHeight(canvas)): targetPixel = getPixel(canvas,targetx,targety) setColor(targetPixel,getColor(srcpixel)) targety = targety + 1 targetx = targetx + 1

How do use programming to replace re-typing code?Building a collage of pictures:def makeACollage(): betsy = makePicture(getMediaPath("IMG_0802.JPG")) hunter = makePicture(getMediaPath("IMG_0808.JPG")) guys = makePicture(getMediaPath("IMG_0805.JPG")) james = copyPartPicture(guys,352,217,618,475) grayscale(betsy) clearRed(james) pastePicture(betsy,scale(james,0.35),0,0) maxBlue(guys) pastePicture(betsy,scale(guys,0.35),450,340) negative(hunter) pastePicture(betsy,scale(hunter,0.35),0,340) writePictureTo(betsy,"C:/collage.jpg")

Result:

How do we replace the background of a picture with something else?

Take a picture with a background that is easy to test and isn’t in the foreground.

Two examples. Which one do you think will work?

Code to chromakey for any background colordef chromakey(picture,background,color,threshold=100.0): for pixel in getPixels(picture): pixelColor = getColor(pixel) if distance(color,pixelColor) < threshold: newColor =

getColor(getPixel(background,getX(pixel),getY(pixel))) setColor(pixel,newColor)

Trying to make the yellow background work>>> chromakey(yellowBarb,

moon, makeColor(167,159,110))

>>> writePictureTo(yellowBarb, "C:/yellowBarb-th100.jpg")

>>> chromakey( yellowBarb, moon, makeColor(167,159,110),50)

>>> writePictureTo( yellowBarb, "C:/yellowBarb-th50.jpg")

Yellow is too close to Barb’s skin tone to work.

Using the general chromakey form for a blue background

>>> chromakey(blueJenny,moon,makeColor(36,62,95))

Doing a specific version of ChromakeyComing up with a test for being “truly blue”

def chromakeyBlue(picture,background): for pixel in getPixels(picture): pixelColor = getColor(pixel) if getRed(pixel)<getBlue(pixel) and

getGreen(pixel)<getBlue(pixel): newColor =

getColor(getPixel(background,getX(pixel),getY(pixel))) setColor(pixel,newColor)

Nicely chromakeyed