cs380: introduction to computer graphics sampling chapter 16...

18/05/08

MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012

CS380:IntroductiontoComputerGraphicsSampling

Chapter16

MinH.KimKAISTSchoolofComputing

SUMMARYTexturemapping

18/05/08

Texturemapping•  Wehavealreadyseenandusedtexture

mapping•  Inbasictexturing,wesimply‘glue’partofan

imageontoatrianglebyspecifyingtexturecoordinatesatthethreevertices.

Normalmapping

Originalmesh4Mtriangles

Simplifiedmesh500triangles

Simplifiedmeshandnormalmapping500triangles

18/05/08

Environmentcubemaps•  Duringtheshadingofapoint,wecantreatthe

materialatthatpointasaperfectmirrorandfetchtheenvironmentdatafromtheappropriateincomingdirection.

Projectortexturemapping

•  Therearetimeswhenwewishtoglueourtextureontoourtrianglesusingaprojectormodel,insteadoftheaffinegluingmodel.

18/05/08

Shadowmapping

•  Two-passrendering

2ndpassrendering1stpassrendering

SAMPLINGChapter16

Aliasing

Anti-aliasing(multi-sampling)

Anti-aliasing(super-sampling)

18/05/08

Sampling•  Pointsampling:

continuousvectoràdiscretepixel•  Ourscenesaredescribedwithtrianglesgivinga

continuous2Dcolorfield.•  Ourimagesaredigital/discretemadeupofagridof

dots.•  Needtomakeabridge

betweenthesetwoworlds(continuousvs.discrete).

•  Elsewewillgetsomeunnecessaryartifactscalled“aliasing”artifacts.–  Jaggies,moirepatterns,

flickering9

Sampling•  Theseoccurwhenthereistoomuchdetailtofit

inonepixel.•  Wecanmitigatetheseartifactsbyaveragingup

thecolorswithinapixel’ssquare.•  Thisiscalledanti-aliasing.

18/05/08

Twomodels•  Acontinuousimage,,isabivariate

function.–  rangeisalinearcolorspace.

•  Adiscreteimage,I[i][j],isatwo-dimensionalarrayofcolorvalues.

•  Weassociateeachpairofintegersi,j,withthecontinuousimagecoordinates

I(xw , yw )

xw = i and yw = j

Aliasing•  Thesimplestandmostobviousmethodtogo

fromacontinuoustoadiscreteimageisbypointsampling.

•  Toobtainthevalueofapixeli,j,wesamplethecontinuousimagefunctionatasingleintegervalueddomainlocation:

•  Thiscanresultsinunwantedartifacts.

I[i][j]← I(i , j)

18/05/08

Aliasing•  Scenemadeupofblackandwhitetriangles:jaggies

atboundaries–  Jaggieswillcrawlduringmotion

•  Iftrianglesaresmallenoughthenwegetrandomvaluesorweirdpatterns– Willflickerduringmotion

•  Theheartoftheproblem:toomuchinformationinonepixel

Anti-aliasing

•  Intuitively:thesinglesampleisabadvalue,wewouldbebetteroffsettingthepixelvalueusingsomekindofaveragevalueoversomeappropriateregion.

•  Intheaboveexamples,perhapssomegrayvalue.

18/05/08

Anti-aliasingsolutions•  Forexample,mathematicallythiscanbe

modeledusingFourieranalysis.– Breaksupthedataby“frequencies”andfiguresout

whattodowiththeun-representablehighfrequencies.

NyquistSamplingTheorem •  Itstatesthefollowing:

– “Thesamplingfrequencyshouldbeatleasttwicethehighestfrequencycontainedinthesignal.”

cs ff 2≥

18/05/08

Example •  Anoriginal,continuoussignalwithf=1Hz

Example •  Ifwesamplebymorethan2Hz,wecan

reconstructthesignalfunctioncorrectly

2Hz 3Hz

18/05/08

Example •  Ifwesampleby1.5Hz(<=2Hz),theremightbe

anambiguityaboutthesignalshape.

1.5Hz Groundtruth

Anti-aliasing•  Wecanalsomodelthisasanoptimizationproblem.•  Theseapproachesleadto:

•  whereissomefunctionthattellsushowstronglythecontinuousimagevalueatshouldinfluencethepixelvalue

I[i][j]← I(x , y)Ω∫∫ Fi , j(x , y)dxdy

Fi , j(x , y)

[x , y]t i,j

I[i][j]

I(xw , yw )

I(x , y)←

ΩFi , jxw = i , yw = j

Fi , j(x , y)

I(x , y)

Verticalsectionview

18/05/08

Anti-aliasing•  Inthissetting,thefunctioniscalledafilter.–  Inotherwords,thebestpixelvalueisdeterminedby

performingsomecontinuousweightedaveragingnearthepixel’slocation.

– Effectively,thisislikeblurringthecontinuousimagebeforepointsamplingit.

– Forexamples,boxfilter,Gaussianfilter,Lanczosfilter,bilaterialfilter

Fi, j (x, y)

Boxfilter•  Weoftenchoosethefilterstobe

somethingnon-optimal,butthatcanmoreeasilycomputedwith.

•  Thesimplestsuchchoiceisaboxfilter,whereiszeroeverywhereexceptoverthe1-by-1squarecenterat.

•  CallingthissquarethatcoverstheareaI[i][j],wearriveat:

Fi, j (x, y)

Fi, j (x, y)x = i, y = j

Ωi , j

Ωi, j

I[i][j]← I(x , y)

Ωi , j∫∫ dxdy

x ⊂ xw = i , y ⊂ yw = j

I[i][j] I(xw , yw )

I(x , y)←

Ωi , j Ωi , j

18/05/08

Boxfilter•  Inthiscase,thedesiredpixelvalueissimplythe

averageofthecontinuousimageoverthepixel’ssquaredomain.

•  Eventhatintegralisnotreallyreasonabletocompute•  Instead,itisapproximatedbysomesumoftheform:

wherekindexessomesetoflocationscalledthesamplelocations,so-calledover-sampling.

•  Therendererfirstproducesa“highresolution”colorandz-buffer“image”,– wherewewillusethetermsampletorefertoeachof

thesehighresolutionpixels.

Over-sampling

I[i][j]← 1

nI(xk , yk )

(xk , yk )

apixel

18/05/08

Over-sampling•  Then,oncerasterizationiscomplete,groupsof

thesesamplesareaveragedtogether,tocreatethefinallowerresolutionimage.

Super-sampling•  Ifthesamplelocationsforthehighresolution

imageformaregular,highresolutiongrid,thenthisiscalledsupersampling.

•  Wecanalsochooseothersamplingpatternsforthehighresolution“image”,– Suchlessregularpatterns(so-calledjittering)can

helpusavoidsystematicerrorsthatcanarisewhenusingthesumtoreplacetheintegral.

apixel

18/05/08

Multi-sampling•  Rendertoa“highresolution”colorandz-buffer•  Duringtherasterizationofeachtriangle,

“coverage”andz-valuesarecomputedatthissamplelevelofhighresolution.

•  Butforefficiency,thefragmentshaderisonlycalledonlyonceperfinalresolutionpixel.– Thiscolordataissharedbetweenallofthesamples

hitbythetriangleinasingle(finalresolution)pixel.

•  Oncerasterizationiscomplete,groupsofthesehighresolutionsamplesareaveragedtogether.

Multi-sampling•  Multisamplingcanbeaneffectiveanti-aliasing

method•  Withouttexturemapping,colorstendtovaryquite

slowlyovereachtriangle,andthustheydonotneedtobecomputedathighspatialresolution.

•  Todealwithaliasingthatoccursduringtexturemapping,wehavetheadvantageofpossessingthetextureimageinhandattheoutsetoftherenderingprocess.

•  Thisleadstospecializedtechniquessuchasmipmapping.

18/05/08

MIPmapping•  AnacronymoftheLatinphrase,multumin

parvo(muchinlittle).•  Mipmapsareprecalculatedscaledversionsofan

originaltexture.

MIPmapping•  Observethepatternofeachmipmapasit

decreasesinsize;itishalfthedimensionofthepreviousone.

•  Thispatternisrepeateduntilthelastmipmap’sdimensionis1×1.

•  Interpolatemulti-resolutiontexturesviatrilinearinterpolation

18/05/08

Aliasingvs.anti-aliasing

Aliasing

Anti-aliasing(multi-sampling)

Anti-aliasing(super-sampling)

Camera•  Indigitalcameras,anti-aliasingisaccomplished

byacombinationofthespatialintegrationthathappensovertheextentofeachpixelsensor,

•  aswellasbytheopticalblurring(similartofilter),so-calleddiffraction,thathappensatduetothelensàpoint-spreadfunction(PSF)

•  Somecamerasalsoincludeadditionalopticalelementsspecificallytoblurthecontinuousimagedatabeforeitissampledatthesensors.

18/05/08

PrincipleofSolid-StateSensor

•  Howtocollectrainoveralargeareawithmanybucketsoffixedsize?

Electrons

Cells Photons

Camera:Demosaicingproblem

•  Imagineablack-on-whitecorner•  Coloraliasinghappensduetothedemosaicing

processoftheBayer-patternfilteredinputBayer-patterndemosaicing

18/05/08

Humanvision•  Inhumanvision,aliasingartifactsarenot

typicallyencountered.– Mostoftheanti-aliasing,atleastinthefoveal

(central)regionofvision,isduetotheopticalblurringoflight(Airydisk),whichhappenswellbeforeithitsthereceptorcells.

– Theirregularspatiallayoutofthesensorcellsintheretinaalsohelpsbyeffectivelyprovidingspatialjitter(randomness)whichturnsnoticeablealiasingintolessconspicuousnoise.

Imagecompositing(layering)•  Giventwodiscreteimages,aforeground,,

andbackground,,thatwewanttocombineintooneimage.

•  Simple:incomposite,useforegroundpixelswheretheyaredefined.Elseusebackgroundpixels.

•  Thiswillgiveusajaggedboundary.•  Realimagewouldhave“boundary”pixelswith

blendedcolors.•  Butthisrequiresusing“sub-pixel”information.

18/05/08

Imagecompositing

Alphablending(akamatting)•  Associatewitheachpixelineachimagelayer,a

value,,thatdescribestheoverallopacityorcoverageoftheimagelayeratthatpixel.– Analphavalueof1representsafullyopaque/

occupiedpixel,whileavalueof0representsafullytransparent/emptyone.

– Afractionalvaluerepresentsapartiallytransparent(partiallyoccupied)pixel.

•  Alphawillbeusedduringcompositing.

α[i][j]

18/05/08

Alphadefinition•  Morespecifically,letbeacontinuous

image,andletbeabinaryvaluedcoveragefunctionoverthecontinuousdomain,withavalueof1atanypointwheretheimageis“occupied”and0whereitisnot.

•  Letusstoreinourdiscreteimagethevalues:

I(x, y)C(x, y)

(x, y)

I[i][j]← I(x , y)

Ωi , j∫∫ C(x , y)dxdy

α[i][j]← C(x , y)dxdy

Ωi , j∫∫

x ⊂ xw = i , y ⊂ yw = j

I[i][j] I(xw , yw )

I(x , y)←

Ωi , j Ωi , j

Overoperation•  Tocompose,wecompute

thecompositeimagecolors,,using

•  Thatis,theamountofobservedbackgroundcoloratapixelisproportionaltothetransparencyoftheforegroundlayeratthatpixel.

•  Likewise,alpha(coverage)forthecompositeimagecanbecomputedas:

If[i][j] over Ib[i][j]Ic[i][j]

Ic[i][j]← If [i][j]+Ib[i][j](1-α f [i][j])

αc[i][j]← α f [i][j]+α b[i][j](1-α f [i][j])

18/05/08

Overoperation•  Ifbackgroundisopaque,sothecompositepixel

isopaque.•  Butwecanmodelmoregeneralcaseaspartof

blendingmultiplelayers.

Overproperties•  Thisprovidesareasonableapproximationtothe

correctlyrenderedimage.•  Onecaneasilyverifythattheoveroperationis

associativebutnotcommutative.Thatis,but,

•  NBpullingmattesfromrealimagesagainstcontrolledoresp.uncontrolledbackgroundsisanareaofresearch.àmattingalgorithmsinCV

Ia over ( Ib over Ic ) = (Ia over Ib ) over Ic

Ia overIb ≠ Ib overIa

18/05/08

•  Idealcontinuouscomposition

Alphablendingerror

Ic[i][j]← I f (x , y)C f (x , y)+ Ib(x , y)Cb(x , y)− Ib(x , y)Cb(x , y)C f (x , y)dxdy

Ωi , j∫∫

= I f (x , y)C f (x , y)dxdyΩi , j∫∫ + Ib(x , y)Cb(x , y)dxdy

Ωi , j∫∫

− Ib(x , y)Cb(x , y)C f (x , y)dxdyΩi , j∫∫

I[i][j]← I(x , y)

Ωi , j∫∫ C(x , y)dxdy

α[i][j]← C(x , y)dxdy

Ωi , j∫∫

x ⊂ xw = i , y ⊂ yw = j

I[i][j] I(xw , yw )

I(x , y)←

Ωi , j Ωi , j

•  Whatwedowithalphablendingis:

Alphablendingerror

Ic[i][j]← If [i][j]+Ib[i][j](1-α f [i][j])

= I f (x , y)C f (x , y)dxdyΩi , j∫∫

+ Ib(x , y)Cb(x , y)dxdyΩi , j∫∫⎛

⎝⎜

⎞⎠⎟ 1− C f (x , y)dxdy

Ωi , j∫∫⎛

⎝⎜

⎞⎠⎟

= I f (x , y)C f (x , y)dxdyΩi , j∫∫ + Ib(x , y)Cb(x , y)dxdy

Ωi , j∫∫

− Ib(x , y)Cb(x , y)dxdyΩi , j∫∫ ⋅ C f (x , y)dxdy

Ωi , j∫∫

18/05/08

Alphablendingerror•  Thedifferencebetweentwo

•  Thereforetheerroristhedifferencebetweentheintegralofaproductandaproductofintegrals.

•  Itistheamountof“correlation(covariance)”betweenthedistributionofforegroundcoverageinsomepixelandthedistributionofthebackgrounddatawithinthatpixel.

error= Ib(x , y)Cb(x , y)C f (x , y)dxdyΩi , j∫∫ −

Ib(x , y)Cb(x , y)dxdy× C f (x , y)dxdyΩi , j∫∫Ωi , j

∫∫

cov(X ,Y )=E[XY ]−E[X ]E[Y ]

Realalphablending

Idealcontinuouscomposition

cs380: introduction to computer graphics sampling chapter 16...

Documents

slide17 -...

radiometric characterization of spectral imaging for...

case storage cs380 - silverstone

arrays strings and regular expressions basic php syntax...

cs 380: artificial intelligence problem solving: informed...

cs380 final - sha1v2

cs380: introduction to computer graphics track-/arc-ball...

cs 380 web programming instructor: xenia mountrouidou cs380...

cs380 lab iii opengl jonghyeob lee reference1. [opengl...

cs380: introduction to computer graphics linear...

cs 380: artificial intelligence...

cs380 lab i opengl donghyuk kim reference1. [opengl course...

words slide17

electrothermal mems parallel plate rotation for...

cs380 lab i opengl

prey behavior through influence maps alicia yeargin cs380,...

cs380 lab ii opengl

cs380: introduction to computer graphics varying variables...

cs380: introduction to computer graphics reconstruction...

cs380 lab iv opengl jonghyeob lee reference1. [opengl course...