martin damek - digital matting for image processing and composition

8/9/2019 Martin Damek - Digital Matting for Image Processing and Composition

1/31

MASARYK UNIVERSITYFACULTY OF INFORMATICS

Digital Matting for Image Processing and

Composition

BACHELOR THESIS

Martin Dámek

Brno, !""


2/31

# INSERT $Co%& o' t(e t(e)i) a))i*nment+ HERE


3/31

Statement

I -e./are t(at t(i) t(e)i) i) m& ori*ina/ a0t(oria/ 1ork t(at I e/a2orate- 2& m&)e/'3

A// re)o0r.e), )o0r.e) an- /iterat0re, t(at I 0)e- in t(e %re%aration or -re1 'rom, I 40ote

in t(e t(e)i) %ro%er/&, 1it( )tatin* t(e '0// re'eren.e to t(e )o0r.e3

55555555555555555555555

6


4/31

Acknowledgements

I 1o0/- /ike to e7%re)) m& t(ank) to a// 1(o (e/%e- me an- )0%%orte- me -0rin* t(e

1ritin* o' t(i) t(e)i)3 Name/& m& )0%er8i)or -o.3 In*3 9i:; So.(or, C).3 'or ).(o/ar/& a-8i.e an-

kin- a%%roa.(3 To Mr3 Ser*e& Bo.(kano8 'or -e8e/o%in* an o%en )o0r.e /i2rar& o' matri7

o%eration) a/*orit(m) AL


5/31

Abstract

T(ere are )e8era/ met(o-) 'or e7tra.tion o' o2>e.t) 'rom ima*e)3 T(e %ro.e)) i) .a//e-

mattin* an- in8o/8e) .reatin* a matte ?an o%a.it& ma)k@ an- )e%aratin* t(e ima*e into t1o

/a&er) 'ore*ro0n- an- 2a.k*ro0n-3 T(e main %ro2/em i) -eterminin* t(eir .o/or tone)

at t(e o2>e.t) 2o0n-arie), 1(ere t(e& are mi7e- to*et(er in 8ario0) ratio)3 A )0..e))'0//&

)e%arate- 'ore*ro0n- /a&er .an t(en 2e .om2ine- 1it( anot(er 2a.k*ro0n- 0)in* t(e matte3

T(i) re8er)e %ro.e)) i) .a//e- ima*e .om%o)itin*3

One o' t(e mattin* met(o-) i) t(e Ba&e)ian a%%roa.(3 T(i) met(o- i) 2a)e-

on %ro2a2i/it& -i)tri20tion .om%0tation) an- a%%ro7imatin* t(e mo)t /ike/& .o/or an- o%a.it&

8a/0e)3 T(e a/*orit(m a.(ie8e) *oo- re)0/t) e8en 'or o2>e.t) 1it( .om%/e7 2o0n-arie),

)0.( a) '0r or tran)/0.ent materia/)3


6/31

Keywords

mattin*, -i*ita/ ima*e, 'ore*ro0n- -ete.tion, .o/or a%%ro7imation, %ro2a2i/it&,

/ike/i(oo-, a/%(a .(anne/, o%a.it&, tran)%aren.&, .o/or 40antiation


7/31


8/31

1 Introduction

In t(e re.ent -e.a-e) 1e are 1itne)) to a)toni)(in* a-8an.ement) in t(e 'ie/-

o' in'ormation te.(no/o*&3 Mo-ern .om%0ter) .om2ine- 1it( .om%/e7 )o't1are .an o'ten

%er'orm 1(at a /ai. 1o0/- ea)i/& .a// ma*i.3 I am a 'an o' ma*i. an- I am 'on- o' %i.t0re)3

(at mattin* met(o-) .an -o 1it( a %i.t0re ma& in-ee- )eem /ike ma*i.3 In 'a.t it i) a

mi7t0re o' mat(, name/& )tati)ti.), an- (0man *eni0)3 A) m& )0%er8i)or, -o.3 So.(or, )ai-

$Do not tr0)t a -i*ita/ ima*e3+ T(i) 1ork )(o1) 1(&3

11 +#at is matting,

Mattin* i) -eri8e- 'rom $matte+, 1(i.( i) a 1or- 0)e- in %(oto*ra%(& an- 'i/mmakin*3

It 0)e- to 2e a )%e.ia/ 'i/m 'rame, %art) o' 1(i.( 1ere 2/a.k an- t(e re)t 1a) tran)%arent3 (en

%0t o8er a %i.t0re 'rame, t(i) matte 1o0/- 'i/ter o0t t(e .o8ere- %art) ?2a.k*ro0n-@ an- /ea8in*

on/& t(e 'ore*ro0n-3 Matte .an a/)o mean )omet(in* */are/e)) or im%ermea2/e3 In t(at )en)e,

ima*e mattin* .an a/)o 2e 0n-er)too- a) .o8erin* .ertain %art) o' it, )o t(at t(e& are no /on*er

8i)i2/e3

O2>e.t) )im%/& .0t o0t o' an ima*e in )0.( a 1a& 1o0/- /ook 0nnat0ra/, t(o0*(3

Com%o)ition) ma-e a'ter 0)in* t(i) te.(ni40e 1o0/- 2e im%er'e.t an- an&one 1o0/- kno1

at t(e 'ir)t */an.e, t(at t(e ima*e (a) 2een tam%ere- 1it(3 O28io0)/&, t(e 2o0n-arie)

o' t(e o2>e.t are t(e %ro2/em3 I' on/& 1e .o0/- make t(o)e o2>e.t) 'it t(e ne1 2a.k*ro0n-

)eam/e))/&3 An- t(at i) e7a.t/& 1(at mattin* met(o-) are tr&in* to a.(ie8e3

T(e ke& to 0n/o.kin* t(e )e.ret o' )moot( e-*e) /ie) in o%a.it&3 T(e mono.(romati.

matte) -o not re.o*nie o%a.it&3 Ea.( %oint o' t(em i) eit(er '0//& o%a40e or '0//& tran)%arent3

In nat0re, man& o2>e.t) (a8e 8er& .om%/e7 or 2/0rre- e-*e)3 Er*o, t(e more -e*ree) o' o%a.it&

.an 2e 0ti/ie-, t(e more nat0ra/ im%re))ion t(e ima*e make)3 A/)o, 1(en a .amera take)

a %i.t0re, .onto0r) o' an o2>e.t 2e.ome me/-e- into t(e 2a.k*ro0n- a /itt/e an- it) .o/or tone)

are a--e- to t(e 2a.k*ro0n- .o/or)3 To 1(at e7tent t(i) (a%%en), -e%en-) on t(e /i*(t.on-ition), t(e -e%t( o' 'ie/- o' t(e o2>e.ti8e, 'o.0), an- re)o/0tion o' t(e .amera3

(at matter) i) t(at t(e .o/or) are )ti// t(ere, on/& t(e& are o8er/a%%in*, o..0%&in*

In )%ite o' t(e 'a.t t(at t(i) /o)t in'ormation .an ne8er 2e '0//& re.o8ere-, t(ere are met(o-) 'or

e)timatin* it 1it( a .ertain %ro2a2i/it&3 Mo)t o' t(e)e met(o-) )ti// re40ire *0i-an.e 'rom t(e

0)er to o0t/ine t(e 'ore*ro0n- an- 2a.k*ro0n- area) 'or t(e a/*orit(m to .o//e.t t(e initia/ -ata3


9/31

more %re.i)e t(e re)0/t i)3 it( eno0*( .are an- a rea)ona2/& -i8er)e %i.t0re, t(e 'ore*ro0n-

o2>e.t .an 2e e7tra.te- )o 1e//, t(at t(e 'o//o1in* %(otomonta*e )eem) 8er& rea/3

1" Digital representation of images

Com%0ter) -o not .om%re(en- 1(at yellow or blue mean), t(e& on/& $t(ink+ in 2inar&3

A'ter a//, .o/o0r) are mere/& a (0man %er.e%tion o' 8ario0) 1a8e/en*t() o' /i*(t3 e (a8e

to -e'ine a 1a& o' ma%%in* t(e )et o' .o/or) on a )et o' n0m2er) to 2e a2/e to re%re)ent .o/or)

-i*ita//&3 T(e mo)t .ommon re%re)entation o' .o/or) in n0m2er), 'or t(e %0r%o)e o' -i*ita/

ima*in*, i) t(e R


10/31

t(e a/%(a .(anne/, t(e .o/or) o' t(e .om%o)ite ima*e .an 2e .om%0te- a..or-in* to t(i)

e40ation

C = α F + (1 – α B -1.

1(ere C i) t(e re)0/tin* .o/or o' t(e .om%o)ite, F i) t(e 'ore*ro0n- .o/or, B i)

t(e 2a.k*ro0n- .o/or an- α i) t(e o%a.it&3

T(e re8er)e %ro.e)), e7tra.tion, i) not t(at ea)&, a) )ome in'ormation (a) 2een /o)t3

At a 'ir)t */an.e t(e on/& in'ormation 1e (a8e a8ai/a2/e i) t(e .o/or C 3 T(e remainin* 8aria2/e)

F , B, α are 0nkno1n3 S0.( e40ation a/one 1o0/- 2e im%o))i2/e to )o/8e3

L0.ki/& eno0*(, 1(en -ea/in* 1it( rea/ 1or/- -ata, in t(i) .a)e %(oto*ra%(), 1e 'in- t(at

t(ere are .ertain r0/e) .on)trainin* t(e 8a/0e) o' F , B an- α an- 2in-in* t(em to*et(er3

For e7am%/e t(e )%a.ia/ .on'ormit& t(e .o/or) o' near2& area) are o'ten t(e )ame or a 8er&)imi/ar .o/or3 T(e)e %ro%ertie) o' t(e rea/ 1or/- -ata .an 2e 0)e- in o0r a-8anta*e3 Ho1e8er,

not(in* .an 2e taken 'or *rante- an- a) mo)t r0/e) it (a) e7.e%tion), t(0) t(e nee- 'or t(e 0)e

o' )tati)ti.) ari)e)3

T(e *oa/ o' t(i) t(e)i) i) to e7%/ore t(i) %ro2a2i/i)ti. a%%roa.( to -i*ita/ mattin*,

im%/ement an a/*orit(m to )o/8e t(e mattin* %ro2/em an- -e8e/o% an a%%/i.ation t(at a//o1) to

te)t it, a) 1e// a) 0)e it .a)0a//& a) a *ra%(i.) )o't1are too/3

"!


11/31

" T#e $ayesian Approac#

A) )tate- 2e'ore, t(e Ba&e)ian met(o- 1ork) 1it( %ro2a2i/itie) an- e)timation)3

T(e %ro2/em o' )o/8in* an e40ation o' t(ree 8aria2/e), 1(i.( i) im%o))i2/e, i) re%/a.e-

2& e)timatin* t(e mo)t /ike/& 8a/0e) o' )ai- 8aria2/e)3

"1 /istory

T(e met(o- i) name- a'ter an En*/i)( mat(emati.ian Re83 T(oma) Ba&e) ?"! ""@1(o .ontri20te- *reat/& to t(e area o' )tati)ti.) an- %ro2a2i/it&3

F!"ure 1 – #or$ra!$ o% Tho&a' Baye' Ba&e)"

T(oma) Ba&e) ?See Fi*0re "@ 1a) 2orn in Lon-on to t(e 'ami/& o' a Non.on'ormi)t

mini)ter, 1(ere (e (a) a/)o re.ei8e- a %ri8ate e-0.ation3 Later (e 1a) or-inate- a mini)ter /ike

(i) 'at(er3

Mat(emati.) .a0*(t (i) intere)t), name/& t(e 'ie/- o' %ro2a2i/it& an- )tati)ti.)3 He

-e8ote- (i) 1ork to it an- 1rote man& %a%er) on >oint %ro2a2i/it& -i)tri20tion an- in'eren.e3

T(e mo)t 'amo0) %a%er o' (i), E''ay Toward' Sol!n" a #roble& !n $he )oc$r!ne o% Chance' ,

1a) %02/i)(e- t(ree &ear) a'ter (i) -eat(3 Ba&e)

""


12/31

(at i) re/e8ant to t(i) t(e)i) i) t(at (e e)ta2/i)(e- a mat(emati.a/ 2a)i) 'or %ro2a2i/it&

in'eren.e t(e 1a& o' .a/.0/atin* t(e %ro2a2i/it& o' an e8ent 'rom o2)er8ation) o' t(e e8ent)

o..0rren.e)3

"" T#eory of likeli#ood

A nat0ra/ *ra)% o' %ro2a2i/it& *oe) t(e o%%o)ite 1a&3 e kno1 t(e %ro2a2i/it& o'

an e8ent, t(en 1e are a2/e to .a/.0/ate t(e 're40en.& 1it( 1(i.( t(e e8ent 1i// o..0r on a8era*e

anot(er )e.on-ar& .(ara.teri)ti.)3

Ba&e) 'orm0/ate- a re/ation 2et1een a .on-itiona/ %ro2a2i/it& an- it) in8er)e 1(i.( i)

.a//e- t(e Ba&e) t(eorem3

#(B* #(Ba&e) t(eorem #(*B = ,,,,,,,, -".

#(B

(ere #(*B i) t(e .on-itiona/ %ro2a2i/it& o' e8ent o..0rrin* *i8en t(e o..0rren.e

o' e8ent B an- 1(ere #(B* i) it) in8er)e, t(e .on-itiona/ %ro2a2i/it& o' B *i8en 3

T(e in8er)e %ro2a2i/it& i) a/)o .a//e- t(e l!-el!hood3 T(e meanin* o' t(e /ike/i(oo-

40antit& .an 2e 0n-er)too- a) t(e .(an.e o' an e8ent (a8in* a .ertain %ro2a2i/it& *i8en

o2)er8e- o..0rren.e) o' it3 Forma//&, a /ike/i(oo- . i) a '0n.tion o' t(e )e.on- ar*0ment o'

t(e ori*ina/ .on-itiona/ %ro2a2i/it& J 1it( it) 'ir)t ar*0ment (e/- 'i7e-3 iki"

I' b / #(*B = b0

t(en .(b* = #(*B = b

Like/i(oo- i) 2a)i.a//& )ti// a %ro2a2i/it&, it) 8a/0e) .an on/& 2e in t(e ran*e #!, "3

De)%ite t(at, t(e& are not t(e )ame mat(emati.a/ 40antit&3

To 2e a2/e to 2etter ima*ine t(e -i''eren.e 2et1een t(e t1o, I %re)ent an e7am%/e3 Let)

)a& 1e (a8e an or-inar& t1o)i-e- .oin t(at .an 2e to))e- an- 1it( a %ro2a2i/it& #(2 /an-) on

it) $(ea-)+ )i-e or 1it( t(e %ro2a2i/it& #(T = 1 – #(2 /an-) on it) $tai/)+ )i-e3 T(e %ro2a2i/it&

i) in i-ea/ .a)e #(2 = #(T = 343 Let) a))0me 'or t(e e7am%/e) )ake t(at 1e -o not kno1

t(i)3

"


13/31

No1, i' 1e to)) t(e .oin on.e an- t(e o2)er8e- o0t.ome i) tai/), 1e kno1 'or .ertain

on/& t(at #(T 5 33 Yo0 )ee, 1e a))0me t(at t(e .oin .o0/- 2e .rooke-3 T(e on/& a))0m%tion 1e

.an make i) t(at t(ere i) a .(an.e o' t(at .oin (a8in* #(T = 343 It i) not t(e )ame t(o0*(,

a) )a&in* t(at e8ent 6TT6 o..0r) 1it( a %ro2a2i/it&3 T(ere i) a /ike/i(oo- '0n.tion t(at

-e).ri2e) t(i) .a)e

F!"ure 7 – .!-el!hood %unc$!on plo$ iki"

A) &o0 .an )ee, a'ter o2)er8in* 6TT6 t(ere i) )ti// a .(an.e t(at #(2 5 33 On/& it i) not

8er& l!-ely3 T(e mo)t /ike/& 8a/0e o' #(T i) no1 " an- t(e /ike/i(oo- o' #(T 2ein* " i) . = 13

T(e /ea)t /ike/& 8a/0e o' #(T i) !, 2e.a0)e a'ter o2)er8in* 6TT6 it i) /ike/& t(at t(e .oin 'a8o0r)

it) $tai/)+3

No1 /et) )a& 1e a-- one to)) o' t(e .oin an- .ome 0% 1it( a $(ea-)+3 T(e a/to*et(er

o2)er8an.e i) no1 6TT26 an- t(e /ike/i(oo- '0n.tion o' #(T .(an*e) a) 'o//o1in*

F!"ure 7 – .!-el!hood %unc$!on plo$ B iki"

"6


14/31

T(e mo)t /ike/& %ro2a2i/it& o' t(e e8ent $tai/)+ i) no1 #(T = 789 an- it) /ike/i(oo- i)

. = 3143 T(e /ea)t /ike/& are t(e %o))i2i/itie) o' #(T 2ein* ! or ", an- t(e /ike/i(oo- o' t(o)e

%o))i2i/itie) i) . = 3, 2e.a0)e a'ter o2)er8in* 2ot( $(ea-)+ an- $tai/)+, it i) not 8er& /ike/&

?in 'a.t it i) im%o))i2/e@ t(at eit(er #(2 or #(T i) eit(er " or !3

T(e re/ation 2et1een /ike/i(oo- an- .on-itiona/ %ro2a2i/it& %ro8e) 0)e'0/ 1(en 1e nee-

to )o/8e a .om%/e7 %ro2/em )0.( a) t(e e)timation o' 'ore*ro0n-, 2a.k*ro0n- an- o%a.it&

8a/0e) o' t(e mer*e- area) in t(e ima*e a) -e).ri2e- in .(a%ter "3

"% Problem formulation I

T(e aim i) to 'in- 'or ea.( %i7e/ o' t(e ima*e )0.( 8a/0e) o' 'ore*ro0n- .o/or F ,

2a.k*ro0n- .o/or B an- o%a.it& α0 t(at t(e %ro2a2i/it& o' t(em mer*in* into one '0//& o%a40e

%i7e/ o' t(e *i8en .o/or, i) t(e (i*(e)t %o))i2/e3 I 'o//o1 t(e te.(ni40e -e).ri2e- in t(e %a%er

Baye'!an pproach $o )!"!$al :a$$!n" Matt3

To 'orma//& e7%re)) t(e %ro2/em at (an-, 1e )ear.( 'or a ma7im0m %ro2a2i/it& 'or

F, B an- α *i8en C

&a; #(F0 B0 α * C

F, B, Q

I' 1e 0)e t(e Ba&e) t(eorem to )%/it t(i) .on-itiona/ %ro2a2i/it&, 1e *et

&a; #(F0 B0 α * C = &a; #(C * F0 B0 α #(F #(B #(α 8 #(C F0 B0 α F0 B0 α

e .an omit t(e term #(C 2e.a0)e it i) a .on)tant an- t(en tran)'orm t(e e40ation

/o*arit(mi.a//& E7%re))in* t(e re)0/t a) )0m o' /o*/ike/i(oo-) 1e *et

&a; #(F0 B0 α * C = &a; .(C * F0 B0 α + .(F + .(B + .(α -%.

F0 B0 α F0 B0 α

1(ere . i) t(e nat0ra/ /o*arit(m o' t(e /ike/i(oo- '0n.tion3 It (a) t(e )ame

.(ara.teri)ti.), 20t it i) ea)ier to 1ork 1it(3 In t(i) in)tan.e, .(α i) a))0me- .on)tant a) 1e//

an- i) e7./0-e- 'rom t(e ma7imiation3 T(e e7%re))ion o' a .om%/e7 .on-itiona/ %ro2a2i/it& i)

no1 )im%/i'ie- an- 1e on/& nee- to -etermine t(e %artia/ /o* /ike/i(oo-) .(C * F0 B0 α, .(F

an- .(B3

"=


15/31

T(e term .(C * F0 B0 α i) t(e /ike/i(oo- o' C 2ein* 1(at i) o2)er8e-, *i8en t(e 8a/0e)

F0 B an- α 3 It .an 2e e7%re))e- a) t(e -i''eren.e 2et1een t(e ori*ina/ .o/or C an- αF+(1


16/31

F!"ure > – Fore"round and bac-"round F!"ure 4 – Clu'$er'0 $he!r &ean' andne!"hborhood' o% a p!;el Matt e!"enec$or' !n RGB 'pace Matt

F0rt(ermore, t(e nei*(2or(oo-) .ontain 8arietie) o' %i7e/), )ome o' t(em are o' more or

/e)) )imi/ar )(a-e) o' .o/o0r), -e%en-in* on t(e .om%/e7it& o' t(e a.t0a/ ima*e3 In or-er to

e)timate t(e mo)t /ike/& 8a/0e) o' F , B an- α e''i.ient/&, it i) .on8enient to -i8i-e t(e

nei*(2or(oo-) into %art) .a//e- ./0)ter), t(at 1o0/- ea.( .ontain t(e %i7e/) o' )imi/ar .o/or3

Ea.( ./0)ter (a) it) mean 8a/0e, 1(i.( i) t(e a8era*e .o/or 8a/0e o' a// it) %i7e/) an-

an ei*en8e.tor, 1(i.( i) a 0nit 8e.tor in t(e -ire.tion o' t(e ./0)ter) -ominant -e8iation3

?See Fi*0re @

"%" Color uanti2ation and clustering

0antiation i) t(e %ro.e)) o' re-0.in* a %a/ette o' .o/or) )o t(at t(ere are 'e1er -i''erent

)(a-e) o' .o/or), 20t 1(i.( are a) ./o)e a) %o))i2/e to a) man& a) %o))i2/e o' t(e ori*ina/ .o/or)3

It (a) 2een 0)e- to .reate .o/or %a/ette) 'or -i)%/a& -e8i.e) t(at (a- a /imite- n0m2er o' .o/or)

t(e& .o0/- -i)%/a& )im0/taneo0)/&3 It .an a/)o 2e 0)e- to ./0)ter t(e %i7e/) nei*(2or(oo-) an-

t(0) re-0.e t(e n0m2er o' .om2ination) 'or )o/8in* t(e /ike/i(oo- e40ation)3

T(ere are )e8era/ met(o-) o' .o/or 40antiation, one o' t(em 0)e) t(e 2inar& tree -ata

)tr0.t0re 1(i.( I am not *oin* to e7%/ain (ere3 T(i) an- a 'e1 ot(er met(o-) are -e).ri2e- in

-etai/ in 0ant3 In m& t(e)i) I 0)e a mo-i'ie- 8er)ion o' t(e 2inar& tree ./0)terin*3 T(e 2a)i.

%rin.i%/e o' 40antiation i) 'o//o1in*

"


17/31

"3 Let t(e .o/or) 'orm a )et

3 Ca/.0/ate it) )tati)ti.a/ %ro%ertie)

63 S%/it t(e )et in a 1a& t(at im%ro8e) t(o)e %ro%ertie)

=3 Re%eat 6 'or t(e ne1 )et) 0nti/ t(e -e)ire- )tati)ti.a/ %ro%ertie) are rea.(e-

A'ter t(e 'ina/ )%/ittin*, t(e)e )et) .on)tit0te t(e ./0)ter)3 T(e %i7e/) nei*(2or(oo- i)

-i8i-e- into *ro0%) o' %i7e/) o' .o/or) )imi/ar to t(e -e*ree 1e .an .(oo)e 2& )ettin* t(e tar*et

ma7im0m -e8iation 'rom t(e mean or a -e)ire- ma7im0m n0m2er o' ./0)ter)3 B& takin* t(e

mean .o/or 8a/0e 'rom ea.( ./0)ter a) a re%re)entant, 1e .an re-0.e t(e n0m2er o'

.om2ination) o' t(e 2a.k*ro0n- an- 'ore*ro0n- .o/or) 1it(o0t /oo)in* m0.( a..0ra.&3

Let) take a ./o)er /ook at t(e 8er& %ro.e)) o' ./0)terin* an- 1(at are t(e )tati)ti.a/

%ro%ertie) t(at e7%re)) t(e -e*ree o' 8ariation in t(e )et o' .o/o0r)3

For t(e %0r%o)e) o' 40antiation, .o/o0r) .an 2e 0n-er)too- a) 8e.tor) in t(e R


18/31

For t(e .riteria o' 1(en to )to% t(e )%/ittin* )er8e) t(e ei*en8a/0e o' a ./0)ter

n = @ ? ;! – ?n@ en7

i ∈N

(en t(e ei*en8a/0e o' ea.( ./0)ter i) e40a/ or /e)) t(an a -e)ire- tar*et ei*en8a/0e,

t(e ./0)terin* i) .om%/ete3 Re)0/t i) a )et o' ./0)ter) t(at ea.( .ontain )imi/ar .o/o0r) ?)imi/ar in

t(e )en)e t(at t(e en- %oint) o' t(eir .o/o0r 8e.tor) are ./o)er to ea.( ot(er in t(e R


19/31

B& re%eatin* t(e %ro.e)) 0nti/ t(e /a)t ./0)ter meet) t(e ei*en8a/0e .on-ition I .an ./0)ter

t(e nei*(2or(oo- more e''e.ti8e/& 'or t(e %0r%o)e) o' t(e mattin* a/*orit(m3 T(e -i)tri20tion

o' .o/or tone) in t(e ./0)ter) i) )moot(er an- on a8era*e t(ere are /e)) ./0)ter) a/to*et(er

?1(i.( i) *oo-@ 'or a *i8en ei*en8a/0e .ei/in*3

Anot(er t(in* 1ort( mentionin* i) t(at )in.e I -ont nee- t(e .o8arian.e matri.e)

'or ./0)terin*, 20t on/& /ater, 'or t(e a.t0a/ %i7e/ a%%ro7imation, t(e n0m2er o' .o8arian.e

matri7 .om%0tation) i) re-0.e- )i*ni'i.ant/&3

I 2e/ie8e t(e)e )02t/e -i''eren.e) e7i)t an- t(e rea)on m& a->0)tment) %ro-0.e- 2etter

re)0/t) i) t(at 0ant %re)ent) a 0ni8er)a/ a/*orit(m t(at 1a) ori*ina//& inten-e- to 40antie

.o/or %a/ette) 'or -i)%/a& -e8i.e) 1it( a /imite- n0m2er o' )im0/taneo0)/& -i)%/a&a2/e .o/or)3

A/)o, in t(e mattin* in)tan.e, it i) %o))i2/e to 0ti/ie )%a.ia/ 1ei*(tin* o' t(e %i7e/), t(0)

%ro8i-in* e7tra in'ormation t(at (e/%) to i-enti'& t(e o%tima/ $)%/ittin* %/ane)+3 ?Te.(ni.a//&,

1(en )%/it, t(e ./0)ter) are -i8i-e- 2& a %/ane in t(e R


20/31

it( t(i) 1ei*(tin* '0n.tion 1e -e'ine t(e tota/ 1ei*(t o' a ./0)ter

P @ w!

i ∈N

an- t(e ./0)ter 1ei*(te- mean

Ne7t 1e -e'ine t(e ./0)ter .o8arian.e matri7

Fina//&, t(e /a)t t1o term) o' t(e e40ation ?%@ are e7%re))e- a) 'o//o1in*

?Ana/o*i.a//& 'or t(e 2a.k*ro0n- /ike/i(oo-3@

e (a8e -e'ine- a// term) nee-e- 'or .a/.0/atin* t(e o8era// %ro2a2i/it& #(F0 B0 α * C

!


21/31

"& Alternating iterati3e appro0imation

T(e ne7t )te% i) to a.t0a//& .a/.0/ate t(e /ike/i(oo-) o' ea.( %air o' .o/or) 2ein*t(e 'ore*ro0n- an- 2a.k*ro0n- an- .(oo)in* t(e mo)t /ike/& one3

In Matt t(e 8a/0e) o' F , B an- α are -e'ine- a) t(e )o/0tion) o' t(e )et o' e40ation)

-'.

-(.

To )o/8e 'or t(e o%tima/ )o/0tion, 1e .ommit t(e)e t1o e40ation) to an iterati8e

a%%ro7imation met(o-3 It i) a n0meri.a/ o%timiation met(o- t(at i) *oin* to 2e %er'orme- 2&

t(e .om%0ter3

e 'ir)t e)timate α a) t(e mean α 8a/0e o' t(e nei*(2or(oo- an- .om%0te t(e 'ir)ta%%ro7imation o' F an- B 'rom it 0)in* e40ation ?'@3 it( t(e)e 8a/0e) 1e t(en )o/8e e40ation

?(@, .om%0te t(e )e.on- a%%ro7imation o' α an- )o on3 T(e 8a/0e) o' F, B an- α )(o0/-

.on8er*e to t(e o%tima/ )o/0tion3

Like 1it( ot(er n0meri.a/ met(o-), 1(en t(e -e)ire- %re.i)ion i) rea.(e-, t(e a/ternatin*

iteration )to%) an- t(e .0rrent 8a/0e) F, B, α are .on)i-ere- t(e re)0/t)3

"


22/31

"' Algorit#mi2ation

For a %ra.ti.a/ 0)e, a// o' t(e a2o8e nee-) to 2e tran)/ate- into a /an*0a*e .om%0ter)0n-er)tan-, )o t(at it .an -o t(e tremen-o0) amo0nt o' a.t0a/ .om%0tation 'or 0)3 Here 'o//o1)

t(e a/*orit(m 'or Ba&e)ian mattin* in a %)e0-o .o-e3 R02in

in%0t ima*e C an- trima% M

initia/ie a/%(a matte α , 'ore*ro0n- F an- 2a.k*ro0n- B a..or-in* to M

'or a// 0nkno1n %i7e/) -o

➢ ./0)ter t(e F an- B nei*(2o0r(oo-)

➢ 'or a// %air) o' F an- B ./0)ter) -o

– )o/8e e40ation) 'or F, B, α 0)in* a/ternatin* iteration

– .a/.0/ate /ike/i(oo- L?C@ L?F@ L?B@

➢ a))i*n t(e mo)t /ike/& 8a/0e) to F, B, α

o0t%0t F, B, α

T(e ./0)terin* a/*orit(m in -etai/

in%0t nei*(2or(oo- 1it( one ./0)ter

.a/.0/ate t(e ei*en8a/0e o' t(e 'ir)t ./0)ter

'or t(e /i)t o' a// -o

➢ 1(i/e t(e ./0)ter) ei*en8a/0e ma7 e*ien8a/0e

– a-- a ne1 ./0)ter at t(e en- o' t(e /i)t

– )%/it t(e .0rrent ./0)ter a..or-in* to t(e )et o' ine40ation) ?&@

– mo8e t(e $o0ter (a/'+ o' %i7e/) into t(e /a)t ./0)ter

– .a/.0/ate t(e ei*en8a/0e o' t(e .0rrent ./0)ter

o0t%0t nei*(2or(oo- 1it( n ./0)ter), /e)) t(an ma7 ei*en8a/0e ea.(

A) t(e a/*orit(m %ro*re))e), t(e %ro.e))e- %i7e/) are a--e- to t(e initia/ -ata, )o t(at

t(e remainin* %i7e/) .an 2e a%%ro7imate- more %re.i)e/&3 T(at mean) t(e re)0/t -e%en-) on

t(e or-er in 1(i.( t(e %i7e/) are %ro.e))e-3 Here 'o//o1) t(e %at(in* a/*orit(m t(at *oe)

aro0n- t(e 0nkno1n area


23/31

in%0t trima%

1(i/e t(ere are an& 0nkno1n %i7e/) in t(e trima%

➢ %ro.e)) a// 'ore*ro0n-a->a.ent 0nkno1n %i7e/)

➢ %ro.e)) a// 2a.k*ro0n-a->a.ent 0nkno1n %i7e/)

➢ mark a// t(e %ro.e))e- %i7e/) a) kno1n in trima%

T(e a/*orit(m a/)o -&nami.a//& a->0)t) t(e nei*(2or(oo- ra-i0) a) it *oe), a..or-in* to

t(e minim0m re40ire- n0m2er o' %i7e/) in a nei*(2or(oo- an- )ie o' t(e near2& 0nkno1n

area, in or-er to a.(ie8e 2etter %re.i)ion3

In rare .a)e), )ome %i7e/) %ro-0.e a nonin8erti2/e .o8arian.e matri7, or 'or )ome ot(er

rea)on t(e o%timiation a/*orit(m -oe) not .on8er*e3 (en t(at o..0r), t(e %i7e/ i) /e't

0nkno1n, )ki%%e- in t(i) /oo% an- (o%e'0//& *et) )0..e))'0//& %ro.e))e- in t(e ne7t /oo%3 More

're40ent/& t(i) )it0ation ari)e) 1(ene8er a nei*(2or(oo- .ontain) /e)) nonero 1ei*(te- %i7e/)

t(an i) a re40ire- minim0m3 T(i) .on-ition en)0re) t(e )a'e amo0nt o' -ata 'or .o/or

e)timation3

6


24/31

% Application

%1 Platform and portability

I (a8e -e.i-e- to im%/ement t(e a/*orit(m in t(e CW /an*0a*e 'or it) a-8anta*e) an-

%re8io0) e7%erien.e3 T(e a%%/i.ation 1a) -e8e/o%e- 0)in* t(e Mi.ro)o't Vi)0a/ St0-io !"!

-e8e/o%ment inter'a.e3 T(e %ro*ram r0n) on an& ma.(ine 1it( 3NET 'rame1ork 8er)ion 63

or /ater3 An O%en So0r.e /i2rar& ALe.t 1(i.( i) re-i)tri20ta2/e 0n-er a


25/31

%% 4ser interface

T(e a%%/i.ation 1a) -e)i*ne- to 2e minima/i)ti. an- ea)& to 0)e3 T(e %0r%o)e i) to te)t

t(e Ba&e)ian mattin* a/*orit(m on rea/ -ata3 It .an o' .o0r)e 2e 0)e- 'or re.reationa/ %0r%o)e)a) 1e//3

Here i) a 40i.k o8er8ie1 o' t(e 0)er inter'a.e

T(e .ommon 0)e .a)e ).enario i) 'o//o1in*

"3 O%en an ima*e 'i/e3

3 Loa- a %re%are- trima% 'orm a 'i/e3 A/ternati8e/& &o0 .an .reate oneX, 20t it i) re.ommen-e-

to 0)e a 2etter *ra%(i. e-itor to -o it more .om'orta2/&3

63 e.t &o0 1i)( to e7tra.t 0)in* mo0)e -ire.t/& into t(e %i.t0re 2o73

(at &o0 -ra1 i) -ire.t/& -i)%/a&e- a) a tran)%arent o8er/a&3 T(e .onto0r i) t(e 0nkno1n area

to 2e e)timate-3 It i) -ra1n in *ra& .o/or3 (en &o0re -one, )e/e.t $Trima% Mark

'ore*ro0n-+ o%tion an- ./i.k on t(e in)i-e o' t(e o2>e.t3 A) /on* a) t(e .onto0r i) %ro%er/&

./o)e- 0%, t(e 'ore*ro0n- an- 2a.k*ro0n- area) )(o0/- 'i// o0t 1it( 1(ite an- 2/a.k .o/or) a'ter

a 1(i/e, an- &o0r .0)tom trima% i) rea-& to 0)e3


26/31

T(e /a&er) are -i)%/a&e- 0%on ./i.kin* on t(e .orre)%on-in* 20tton3 T(e i.on 2e)i-e t(e

-ro%-o1n men0 20tton) in-i.ate) 1(i.( /a&er i) .0rrent/& -i)%/a&e- to (e/% &o0 na8i*ate3 I'

&o0 1i)( to )e/e.t anot(er o%tion 1it(o0t )1it.(in* t(e -i)%/a&e- /a&er, 0)e t(e ri*(t )i-e o' t(e

20tton) marke- 2& a -ro%-o1n trian*/e3

Yo0 .an )a8e an& /a&er in a 'i/e 1(ene8er &o0 1i)( to 2& )e/e.tin* t(e Sa8e o%tion 'rom

t(e .orre)%on-in* -ro%-o1n men03 T(e trima% an- t(e 2a.k*ro0n- m0)t 2e t(e )ame )ie a)

t(e ima*e an- t(e a%%/i.ation -oe) not )0%%ort oomin* or %o)itionin* o' t(e /a&er) ?&et@3 I

a%o/o*ie 'or t(e in.on8enien.e3

%& Testing and results

J/ea)e )ee t(e a%%en-i7 'or t(e re)0/t) on t(e te)tin* )am%/e) 'rom Matt an- R02in3

T(eir) are a8ai/a2/e to .om%are on t(e 1e2 )ite) .ite- in t(e re'eren.e)3 Yo0 .an 'in-

.om%ari)on) 1it( ot(er mattin* a/*orit(m) t(ere, too3


27/31

' Conclusion

T(e a/*orit(m %er'orme- 1e// in a// 20t t(e mo)t .om%/i.ate- .a)e)3 E8en non.or%orea/

o2>e.t) /ike )moke i) e7tra.te- 1it( )ati)'a.tor& %re.i)ion3 ?See e7am%/e)@

A/t(o0*( t(e .o/or a%%ro7imation it)e/' i) ./ear/& -e'ine- 2& ?@ an- ?@, t(e 'ina/ )(a%e

o' t(e matte -e%en-) on )e8era/ %arameter), )0.( a) t(e )ie an- )(a%e o' nei*(2or(oo-),

%re.i)ion t(re)(o/- 'or ./0)terin* an- /ike/i(oo- .on8er*en.e, t(e or-er o' %ro.e))in* %i7e/)

an- (an-/in* o' t(e o..a)iona//& o..0rrin* -e*enerate .a)e)3

In t(e -emon)tration a%%/i.ation I (a8e t0ne- t(e %arameter) into a .om%romi)e 2et1een

)%ee- an- %re.i)ion, 2a)e- on n0mero0) /i8e te)tin*3 It i) %o))i2/e to .(an*e an& o' t(em in t(e

)o0r.e .o-e, )(o0/- &o0 1ant to t1eak t(e %er'orman.e, 20t )ome 0n1i)e/& .(o)en 8a/0e) ma&

.a0)e in)ta2i/it&, 'reein* or e7treme/& )/o1 %ro.e))in*3

A) one o' t(e %o))i2/e '0t0re im%ro8ement), I 1o0/- /ike to a-- a 0)er inter'a.e 'or t(e)e

)ettin*), )a'et& .(e.k) an- more 1a&) to -&nami.a//& a->0)t t(e %arameter) $on t(e r0n+ 2a)e-

on t(e -ata3


28/31

)eferences

Ba&e)" T(e In)tit0te o' Mat(emati.a/ Stati)ti.)3 The Reerend Tho&a' Baye'0 FRS <

131


29/31

Appendi0

Ori*ina/ A/%(a matte Com%o)ite


30/31

Ori*ina/ A/%(a matte Com%o)ite

6!


31/31

Ori*ina/ Com%o)ite an- a/%(a -etai/

6"

martin damek - digital matting for image processing and composition

Documents