applying the haar wavelet transform to time series information

7/13/2015 ApplyingtheHaarWaveletTransformtoTimeSeriesInformation

http://www.bearcave.com/misl/misl_tech/wavelets/haar.html 1/27

ApplyingtheHaarWaveletTransformtoTimeSeriesInformation

Contents

Introduction(/misl/misl_tech/wavelets/haar.html#Introduction)

Whatalong,strangetripsitsbeen(/misl/misl_tech/wavelets/haar.html#StrangeTrip)

Thewavelettechniqueforanalyzingasignalortimeseries(/misl/misl_tech/wavelets/haar.html#WaveletTechnique)

TheLanguageofWavelets(/misl/misl_tech/wavelets/haar.html#language)

ApplyingWaveletsandJavaSourceCode(/misl/misl_tech/wavelets/haar.html#ApplyingWavelets)

FinancialTimeSeries(/misl/misl_tech/wavelets/haar.html#FinancialTimeSeries)

WhyHaarWavelets?(/misl/misl_tech/wavelets/haar.html#WhyHaar)

HaarWavelets(/misl/misl_tech/wavelets/haar.html#HaarWavelets)

FilteringSpectrum(/misl/misl_tech/wavelets/haar.html#FilteringSpectrum)

NoiseFilters(/misl/misl_tech/wavelets/haar.html#NoiseFilters)

Waveletsvs.SimpleFilters(/misl/misl_tech/wavelets/haar.html#SimpleFilters)

LimitationsoftheHaarWaveletTransform(/misl/misl_tech/wavelets/haar.html#limitations)

WaveletsandParallelism(/misl/misl_tech/wavelets/haar.html#parallelism)



JavaSourceCodeDownload(/misl/misl_tech/wavelets/haar.html#download)

Resources(/misl/misl_tech/wavelets/haar.html#Resources)

References,Books(/misl/misl_tech/wavelets/haar.html#ReferencesBooks)

References,Webpublished(/misl/misl_tech/wavelets/haar.html#ReferencesWeb)

Linkstosubpages

DaubechiesWavelets(/software/java/wavelets/daubechies/index.html)

FilteringUsingHaarWavelets(/misl/misl_tech/wavelets/spectrum_plots/index.html)

TheHaarWaveletTransformandNoiseFilters(/misl/misl_tech/wavelets/close_images/index.html)

WaveletNoiseThresholding(/misl/misl_tech/wavelets/noise.html)

Waveletsvs.SimpleFilters(/misl/misl_tech/wavelets/simple_filters/index.html)

WaveletsinJava(includeshistogrammingandsimplestatisticalalgorithms)(/software/java/wavelets/index.html)

IntroductionThiswasthefirstwebpageIwroteonWavelets.Fromthisseedgrewotherwebpageswhichdiscussavarietyofwaveletrelatedtopics.Fora"tableofcontents"seeWaveletsandSignalProcessing(/misl/misl_tech/wavelets/index.html).Thiswebpageappliesthewavelettransformtoatimeseriescomposedofstockmarketcloseprices.Laterwebpagesexpandonthisworkinavarietyofareas(e.g.,compression,spectralanalysisandforecasting).



WhenIstartedoutIthoughtthatIwouldimplementtheHaarwaveletandthatsomeofmycolleaguesmightfindituseful.Ididnotexpectsignalprocessingtobesuchaninterestingtopic.NordidIunderstandwhomanydifferentareasofcomputerscience,mathematics,andquantitativefinancewouldbetouchedbywavelets.Ikeptfindingthat"onethingleadtoanother",makingitdifficulttofindalogicalstoppingplace.Thiswanderingpathofdiscoveryonmypartalsoaccountsforthesomewhatorganicgrowthofthesewebpages.Ihavetriedtotamethisgrowthandorganizeit,butIfearthatitstillreflectsthefactthatIdidnotknowwhereIwasgoingwhenIstarted.

TheJavacodepublishedalongwiththiswebpagereflectthefirstworkIdidonwavelets.Moresophisticated,liftingschemebased,algorithms,implementedinJavacanbefoundonotherwebpages.Thewaveletliftingschemecode,publishedonotherwebpages,issimplerandeasiertounderstand.Thewaveletliftingschemealsoprovidesanelegantandpowerfulframeworkforimplementingarangeofwaveletalgorithms.

Inimplementingwaveletpacketalgorithms,IswitchedfromJavatoC++.ThewaveletpacketalgorithmIusedissimplerandmoreelegantusingC++'soperatoroverloadingfeatures.C++alsosupportsgenericdatastructures(templates),whichallowedmetoimplementagenericclasshierarchyforwavelets.Thiscodeincludesseveraldifferentwaveletalgoriths,includingHaar,linearinterpolationandDaubechiesD4.

Likethewaveletalgorithms,thefinancialmodelingdonehererepresentsveryearlywork.WhenIstartedworkingonthesewebpagesIhadnoexperiencewithmodelingfinancialtimeseries.Theworkdescribedonthiswebpageleadtomoreintensiveexperimentswithwaveletfiltersinfinancialmodels,whichIcontinuetoworkon.OnthiswebpageIusestockmarketcloseprices.Infinancialmodelingoneusuallyusesreturns,sincewhatyouaretryingtopredictisfuturereturn.

Ibecameinterestedinwaveletsbyaccident.Iwasworkingonsoftwareinvolvedwithfinancialtimeseries(e.g.,equityopenandcloseprice),soIsupposethatitwasanaccidentwaitingtohappen.IwasreadingtheFebruary2001issueofWIREDmagazinewhenIsawthegraphincludedbelow.EverymonthWIREDrunsvariousgraphicvisualizationsoffinancialdataandthiswasoneofthem.



GraphandquotefromWIREDMagazine,February2001,page176

Ifstockpricesdoindeedfactorinallknowableinformation,acompositepricegraphshouldproceedinanorderlyfashon,asnewinformationnudgesperceivedvalueagainstthepullofestablishedtendencies.Waveletanalysis,widelyusedincommunicationstoseparatesignal(patternedmotion)fromnoise(randomactivity),suggestsotherwise.

ThisimageshowstheresultsofrunningaHaartransformthefundamentalwaveletformulaonthedailycloseoftheDowandNASDQsince1993.Thebluemountainsconstitutesignal.Theembeddedredspikesrepresentnoise,ofwhichtheyellowlinefollowsa50daymovingaverage.

Noise,whichcanberegardedasinvestorignorance,hasrisenalongwiththevalueofbothindices.ButwhilenoiseintheDowhasgrown500percentonaverage,NASDAQnoisehasballooned3,000percent,faroutstrippingNASDAQ'sspectacular500percentgrowthduringthesameperiod.Mostofthisincreasehasoccurredsince1997,withanextraordinarysurgesinceJanuary2000.PerhapstherewasaY2KglichafterallonethatderailednotoperatingsystemsandCPUs,but>investorpsychology.ClemChambers([email protected]).



IamaPlatonist.Ibelievethat,intheabstract,thereistruth,butthatwecanneveractuallyreachit.Wecanonlyreachanapproximation,orashadowoftruth.ModernscienceexpressesthisasHeisenberguncertainty.

APlatonistviewofafinancialtimeseriesisthatthereisa"true"timeseriesthatisobscuredtosomeextentbynoise.Forexample,aclosepriceorbid/asktimeseriesforastockmovesonthebasisofthesupplyanddemandforshares.Inthecaseofabid/asktimeseries,thesupply/demandcurvewillbesurroundedbythenoisecreatedbyrandomorderarrival.If,somehow,thenoisecouldbefilteredout,wewouldseethe"true"supply/demandcurve.Softwarewhichusesthisinformationmightbeabletodoabetterjobbecauseitwouldnotbeconfusedbyfalsemovementscreatedbynoise.

TheWIREDgraphabovesuggeststhatwaveletanalysiscanbeusedtofilterafinancialtimeseriestoremovetheassociatednoise.OfcoursethereisavastareathatisnotaddressedbytheWIREDquote.What,forexample,constitutesnoise?WhatarewaveletsandHaarwavelets?Whyarewaveletsusefulinanalyzingfinancialtimeseries?WhenIsawthisgraphIknewanswerstononeofthesequestions.

TheanalysisprovidedinthebriefWIREDparagraphisshallowaswell.Noiseinthetimeseriesincreaseswithtradingvolume.Inordertoclaimthatnoisehasincreased,thenoiseshouldbenormalizedfortradingvolume.

Whatalong,strangetripitsbeenReadingisadangerousthing.Itcanlaunchyouoffintostrangedirections.ImovedfromCaliforniatoSantaFe,NewMexicobecauseIreadabook(http://www.bearcave.com/bookrev/predictors.html).ThatonegraphinWIREDmagazinelaunchedmedownapaththatIspentmanymonthsfollowing.Likeanyadventure,I'mnotsureifIwouldhaveembarkedonthisoneifIhadknownhowlongand,attimes,difficult,thejourneywouldbe.

Yearsago,whenitfirstcameout,IboughtacopyofthebookTheWorldAccordingtoWaveletsbyBarbaraHubbard,onthebasisofareviewIreadinthemagazineScience.ThebooksatonmyshelfunreaduntilIsawtheWIREDgraph.

Waveletshavebeensomewhatofafad,abuzzwordthatpeoplehavethrownaround.BarbaraHubbardstartedwritingTheWorldAccordingtoWaveletswhenthewaveletfadwasstartingtocatchfire.Sheprovidesaninterestinghistoryofhowwaveletsdevelopedinthemathematicaland



engineeringworlds.Shealsomakesavaliantattempttoprovideanexplanationofwhatthewavelettechniqueis.Ms.Hubbardisasciencewriter,notamathematician,butshemasteredafairamountofbasiccalculusandsignalprocessingtheory(whichIadmireherfor).WhenshewroteTheWorldAccordingtoWaveletstherewerefewbooksonwaveletsandnointroductorymaterial.AlthoughIadmireBarbaraHubbard'sheroiceffort,IhadonlyasurfaceunderstandingofwaveletsafterreadingTheWorldAccordingtoWavelets.

Thereisavastliteratureonwaveletsandtheirapplications.Fromthepointofviewofasoftwareengineer(withonlyayearofcollegecalculus),theproblemwiththewaveletliteratureisthatithaslargelybeenwrittenbymathematicians,eitherforothermathematiciansorforstudentsinmathematics.I'mnotamemberofeithergroup,soperhapsmyproblemisthatIdon'thaveafluentgraspofthelanguageofmathematics.IcertianlyfeelthiswheneverIreadjournalarticlesonwavelets.However,Ihavetriedtoconcentrateonbooksandarticlesthatareexplicitlyintroductoryandtutorial.Eventhesehaveproventobedifficult.

ThefirstchapterofthebookWaveletsMadeEasybyYvesNievergeltstartsoutwithanexplainationofHaarwavelets(thesearethewaveletsusedtogeneratethegraphpublishedinWIRED).ThischapterhasnumerousexamplesandIwasabletounderstandandimplementHaarwaveletsfromthismaterial(linkstomyJavacodeforHaarwaveletscanbefoundbelow).AlaterchapterdiscussestheDaubechieswavelettransform.Unfortunately,thischapterofWaveletsMadeEasydoesnotseemtobeasgoodasthematerialonHaarwavelets.ThereappeartobeanumberoferrorsinthischapterandimplementingthealgorithmdescribedbyNievergeltdoesnotresultinacorrectwavelettransform.Amongotherthings,thewaveletcoefficientsfortheDaubechieswaveletsseemtobewrong.MywebpageontheDaubechieswavelettransformcanbefoundhere(/software/java/wavelets/daubechies/index.html).ThebookRipplesinMathematics(seethereferencesattheendofthewebpage)isabetterreference.

ThewavelettechniqueforanalyzingasignalortimeseriesThereisavastliteratureonwavelets.Thisincludesthousandsofjournalarticlesandmanybooks.ThebooksonwaveletsrangefromrelativelyintroductoryworkslikeNievergelt'sWaveletsMadeEasy(whichisstillnotlightreading)tobooksthatareaccessableonlytograduatestudentsinmathematics.ThereisalsoagreatdealofwaveletmaterialontheWeb.Thisincludesanumberoftutorials(seeWebbasedreference(/misl/misl_tech/wavelets/haar.html#ReferencesWeb),below).

Giventhevastliteratureonwavelets,thereisnoneedforyetanothertutorial.Butitmightbe



worthwhiletosummarizemyviewofwaveletsastheyareappliedto1Dsignalsortimeseries(animageis2Ddata).Atimeseriesissimplyasampleofasignalorarecordofsomething,liketemperature,waterlevelormarketdata(likeequitycloseprice).

Waveletsallowatimeseriestobeviewedinmultipleresolutions.Eachresolutionreflectsadifferentfrequency.Thewavelettechniquetakesaveragesanddifferencesofasignal,breakingthesignaldownintospectrum.AllthewaveletalgorithmsthatI'mfamiliarwithworkontimeseriesapoweroftwovalues(e.g.,64,128,256...).Eachstepofthewavelettransformproducestwosetsofvalues:asetofaveragesandasetofdifferences(thedifferencesarereferredtoaswaveletcoefficients).Eachstepproducesasetofaveragesandcoefficientsthatishalfthesizeoftheinputdata.Forexample,ifthetimeseriescontains256elements,thefirststepwillproduce128averagesand128coefficients.Theaveragesthenbecometheinputforthenextstep(e.g.,128averagesresultinginanewsetof64averagesand64coefficients).Thiscontinuesuntiloneaverageandonecoefficient(e.g.,2 )iscalculated.

Theaverageanddifferenceofthetimeseriesismadeacrossawindowofvalues.Mostwaveletalgorithmscalculateeachnewaverageanddifferencebyshiftingthiswindowovertheinputdata.Forexample,iftheinputtimeseriescontains256values,thewindowwillbeshiftedbytwoelements,128times,incalculatingtheaveragesanddifferences.Thenextstepofthecalculationusestheprevioussetofaverages,alsoshiftingthewindowbytwoelements.Thishastheeffectofaveragingacrossafourelementwindow.Logically,thewindowincreasesbyafactoroftwoeachtime.

Inthewaveletliteraturethistreestructuredrecursivealgorithmisreferredtoasapyramidalalgorithm.

Thepoweroftwocoefficient(difference)spectrumgeneratedbyawaveletcalculationreflectchangeinthetimeseriesatvariousresolutions.Thefirstcoefficientbandgeneratedreflectsthehighestfrequencychanges.Eachlaterbandreflectschangesatlowerandlowerfrequencies.

Thereareaninfinitenumberofwaveletbasisfunctions.Themorecomplexfunctions(liketheDaubechieswavelets)produceoverlappingaveragesanddifferencesthatprovideabetteraveragethantheHaarwaveletatlowerresolutions.However,thesealgorithmsaremorecomplicated.

TheLanguageofWaveletsEveryfieldofspecialtydevelopsitsownsublanguage.Thisiscertainlytrueofwavelets.I've

0



listedafewdefinitionsherewhich,ifIhadunderstoodtheirmeaningwouldhavehelpedmeinmywanderingsthroughthewaveletliterature.

Wavelet

Afunctionthatresultsinasetofhighfrequencydifferences,orwaveletcoefficients.Inliftingscheme(/misl/misl_tech/wavelets/lifting/index.html)termsthewaveletcalculatesthedifferencebetweenapredictionandanactualvalue.

Ifwehaveadatasamples ,s ,s ...theHaarwaveletequationsis

Wherec isthewaveletcoefficient.

ThewaveletLiftingScheme(/misl/misl_tech/wavelets/lifting/index.html)usesaslightlydifferentexpressionfortheHaarwavelet:

ScalingFunction

Thescalingfunctionproducesasmootherversionofthedataset,whichishalfthesizeoftheinputdataset.Waveletalgorithmsarerecursiveandthesmootheddatabecomestheinputforthenextstepofthewavelettransform.TheHaarwaveletscalingfunctionis

wherea isasmoothedvalue.

TheHaartransformpreservestheaverageinthesmoothedvalues.Thisisnottrueofallwavelettransforms.

Highpassfilter

i i+1 i+2

i

i



Indigitalsignalprocessing(DSP)terms,thewaveletfunctionisahighpassfilter.Ahighpassfilterallowsthehighfrequencycomponentsofasignalthroughwhilesuppressingthelowfrequencycomponents.Forexample,thedifferencesthatarecapturedbytheHaarwaveletfunctionrepresenthighfrequencychangebetweenanoddandanevenvalue.

Lowpassfilter

Indigitalsignalprocessing(DSP)terms,thescalingfunctionisalowpassfilter.Alowpassfiltersuppressesthehighfrequencycomponentsofasignalandallowsthelowfrequencycomponentsthrough.TheHaarscalingfunctioncalculatestheaverageofanevenandanoddelement,whichresultsinasmoother,lowpasssignal.

Orthogonal(orOrthonormal)Transform

Thedefinitionoforthonormal(a.k.a.orthogonal)tranformsinWaveletMethodsforTimeSeriesAnalysisbyPercivalandWalden,CambridgeUniversityPress,2000,Chaper3,section3.1,isoneofthebestI'veseen.I'vequotedthisbelow:

Intermsofwavelettransformsthismeansthattheoriginaltimeseriescanbeexactlyreconstructedfromthetimeseriesaverageandcoefficientsgeneratedbyanorthogonal(orthonormal)wavelettransform.

Signalestimation

Thisisalsoreferredtoas"denoising".Signalestimationalgorithmsattempttocharacterizeportionsofthetimeseriesandremovethosethatfallintoaparticularmodelofnoise.

ApplyingWaveletsandJavaSourceCodeTheseWebpagespublishsomeheavilydocumentedJavasourcecodefortheHaarwavelet

Orthonormaltransformsareofinterstbecausetheycanbeusedtoreexpressatimeseriesinsuchawaythatwecaneasilyreconstructtheseriesfromitstransform.Inaloosesense,the"information"inthetransformisthusequivalenttothe"information"istheoriginalseriestoputitanotherway,theseriesanditstransformcanbeconsideredtobetworepresentationsofthesamemathematicalentity.



transform.BookslikeWaveletsMadeEasyexplainsomeofthemathematicsbehindthewavelettransform.Ihavefound,however,thattheimplemationofthiscodecanbeatleastasdifficultasunderstandingthewaveletequations.Forexample,theinplaceHaarwavelettransformproduceswaveletcoefficientsinabutterflypatternintheoriginaldataarray.TheJavasourcepublishedhereincludescodetoreorderthebutterflyintocoefficientspectrumswhicharemoreusefulwhenitcomestoanalyzingthedata.Althoughthiscodeisnotlarge,ittookmemostofaSaturdaytoimplementthecodetoreorderthebutterflydatapattern.

ThewaveletLiftingScheme,developedbyWimSweldensandothersprovidesasimplerwaytolookasmanywaveletalgorithms.IstartedtoworkonLiftingSchemewaveletimplementationsafterIhadwrittenthiswebpageanddevelopedthesoftware.TheHaarwaveletcodeismuchsimplerwhenexpressedintheliftingscheme.SeemywebpageTheWaveletLiftingScheme(/misl/misl_tech/wavelets/lifting/index.html).

ThelinktotheJavasourcedownloadWebpageisbelow.

FinancialTimeSeriesThereareavarietyofwaveletanalysisalgorithms.Differentwaveletalgorithmsareappplieddependingonthenatureofthedataanalyzed.TheHaarwavelet,whichisusedhereisveryfastandworkswellforthefinancialtimeseries(e.g.,theclosepriceforastock).Financialtimeseriesarenonstationary(touseasignalprocessingterm).Thismeansthatevenwithinawindow,financialtimeseriescannotbedescribedwellbyacombinationofsinandcosterms.Norarefinancialtimeseriescyclicalinapredictablefashion(unlessyoubelieveinElliotwaves(http://www.crbindex.com/techtip/tipv2n19.htm)).FinancialtimeserieslendthemselvestoHaarwaveletanalysissincegraphsoffinancialtimeseriestendtojagged,withoutalotofsmoothdetail.Forexample,thegraphbelowshowsthedailyclosepriceforAppliedMaterialsoveraperiodofabouttwoyears.



DailyclosepriceforAppliedMaterials(symbol:AMAT),12/18/97to12/30/99.

TheHaarwaveletalgorithmsIhaveimplementedworkondatathatconsistsofsamplesthatareapoweroftwo.Inthiscasethereare512samples.

WhyHaarWavelets?Thereareawidevarietyofpopularwaveletalgorithms,includingDaubechies(/software/java/wavelets/daubechies/index.html)wavelets,MexicanHatwaveletsandMorletwavelets.Thesewaveletalgorithmshavetheadvantageofbetterresolutionforsmoothlychangingtimeseries.ButtheyhavethedisadvantageofbeingmoreexpensivetocalculatethantheHaarwavelets.Thehigerresolutionprovidedbythesewavletsisnotworththecostforfinancialtimeseries,whicharecharacterizedbyjaggedtransitions.

HaarWaveletsTheHaarwaveletalgorithmspublishedhereareappliedtotimeserieswherethenumberof



samplesisapoweroftwo(e.g.,2,4,8,16,32,64...)TheHaarwaveletusesarectangularwindowtosamplethetimeseries.Thefirstpassoverthetimeseriesusesawindowwidthoftwo.Thewindowwidthisdoubledateachstepuntilthewindowencompassestheentiretimeseries.

Eachpassoverthetimeseriesgeneratesanewtimeseriesandasetofcoefficients.Thenewtimeseriesistheaverageoftheprevioustimeseriesoverthesamplingwindow.Thecoefficientsrepresenttheaveragechangeinthesamplewindow.Forexample,ifwehaveatimeseriesconsistingofthevaluesv ,v ,...v ,anewtimeseries,withhalfasmanypointsiscalculatedbyaveragingthepointsinthewindow.Ifitisthefirstpassoverthetimeseries,thewindowwidthwillbetwo,sotwopointswillbeaveraged:

for(i=0;i



Thewaveletcoefficientsarecalcalculatedalongwiththenewaveragetimeseriesvalues.Thecoefficientsrepresenttheaveragechangeoverthewindow.Ifthewindowswidthistwothiswouldbe:

for(i=0;i



PlotoftheHaarcoefficientspectrum.Thesurfaceplotsthehighestfrequencyspectruminthefrontandthelowestfrequencyspectrumintheback.Notethatthehighestfrequencyspectrumcontainsmostofthenoise.

FilteringSpectrumThewavelettransformallowssomeorallofagivenspectrumtoberemovedbysettingthecoefficientstozero.Thesignalcanthenberebuiltusingtheinversewavelettransform.PlotsoftheAMATclosepricetimeserieswithvariousspectrumfilteredoutareshownhere.(/misl/misl_tech/wavelets/spectrum_plots/index.html)

NoiseFiltersEachspectrumthatmakesupatimeseriescanbeexaminedindependently.Anoisefiltercanbeappliedtoeachspectrumremovingthecoefficientsthatareclassifiedasnoisebysettingthecoefficientstozero.



Thiswebpage(/misl/misl_tech/wavelets/close_images/index.html)showsahistogramanalysisofthethreehighestfrequencyspectrumoftheAMATcloseprice.Theresultofafilterthatremovesthepointsthatfallwithinagaussiancurveineachspectrumisalsoshown.Thegaussiancurvehasameanandstandarddeviationofthecoefficientsinthatspectrum.

Anotherwaytoremovenoiseistousethresholding.Mywebpageoutliningonethresholdingalgorithmcanbefoundhere(/misl/misl_tech/wavelets/noise.html).

Waveletsvs.SimpleFiltersHowdoHaarwaveletfilterscomparetosimplefilters,likewindowedmeanandmedianfilters?AplotoftheAMATtimeseries,filteredwithamedianfilter(whichinthiscaseisvirtuallyidenticaltoameanfilter)isshownherehere(/misl/misl_tech/wavelets/simple_filters/index.html).Thesefilterscanbecomparedtothespectrumfilters(whereagivenwaveletcoefficientspectrumisfileredout)here.(/misl/misl_tech/wavelets/spectrum_plots/index.html).

Whetherawaveletfilterisbetterthanawindowedmeanfilterdependsontheapplication.Thewaveletfilterallowsspecificpartsofthespectrumtobefiltered.Forexample,theentirehighfrequencyspectrumcanberemoved.Orselectedpartsofthespectrumcanberemoved,asisdonewiththegaussiannoisefilter.ThepowerofHaarwaveletfiltersisthattheycanbeefficientlycalculatedandtheyprovidealotofflexibility.Theycanpotentiallyleavemoredetailinthetimeseries,comparedtothemeanormedianfilter.Totheextentthatthisdetailisusefulforanapplication,thewaveletfilterisabetterchoice.

LimitationsoftheHaarWaveletTransformTheHaarwavelettransformhasanumberofadvantages:

Itisconceptuallysimple.Itisfast.Itismemoryefficient,sinceitcanbecalculatedinplacewithoutatemporaryarray.Itisexactlyreversiblewithouttheedgeeffectsthatareaproblemwithotherwavelettrasforms.

TheHaartransformalsohaslimitations,whichcanbeaproblemforsomeapplications.

Ingeneratingeachsetofaveragesforthenextlevelandeachsetofcoefficients,theHaartransformperformsanaverageanddifferenceonapairofvalues.Thenthealgorithmshiftsover



bytwovaluesandcalculatesanotheraverageanddifferenceonthenextpair.

Thehighfrequencycoefficientspectrumshouldreflectallhighfrequencychanges.TheHaarwindowisonlytwoelementswide.Ifabigchangetakesplacefromanevenvaluetoanoddvalue,thechangewillnotbereflectedinthehighfrequencycoefficients.

Forexample,inthe64elementtimeseriesgraphedbelow,thereisalargedropbetweenelements16and17,andelements44and45.

Sincethesearehighfrequencychanges,wemightexpecttoseethemreflectedinthehighfrequencycoefficients.However,inthecaseoftheHaarwavelettransformthehighfrequencycoefficientsmissthesechanges,sincetheyareoneventooddelements.

Thesurfacebelowshowsthreecoefficientspectrum:32,16and8(wherethe32elementcoefficientspectrumisthehighestfrequency).Thehighfrequencyspectrumisplottedontheleadingedgeofthesurface.thelowestfrequencyspectrum(8)isthefaredgeofthesurface.



Notethatbothlargemagnitudechangesaremissingfromthehighfrequencyspectrum(32).Thefirstchangeispickedupinthenextspectrum(16)andthesecondchangeispickedupinthelastspectruminthegraph(8).

Manyotherwaveletalgorithms,liketheDaubechieswaveletalgorithm,useoverlappingwindows,sothehighfrequencyspectrumreflectsallchangesinthetimeseries.LiketheHaaralgorithm,Daubechiesshiftsbytwoelementsateachstep.However,theaverageanddifferencearecalculatedoverfourelements,sothereareno"holes".

Thegraphbelowshowsthehighfrequencycoefficientspectrumcalculatedfromthesame64elementtimeseries,butwiththeDaubechiesD4waveletalgorithm.Becauseoftheoverlappingaveragesanddifferencesthechangeisreflectedinthisspectrum.



The32,16and8coefficientspectrums,calculatedwiththeDaubechiesD4waveletalgorithm,areshownbelowasasurface.Notethatthechangeinthetimeseriesisreflectedinallthreecoefficientspectrum.

WaveletsandParallelism



Waveletalgorithmsarenaturallyparallel.Forexample,ifenoughprocessingelementsexist,thewavelettransformforaparticularspectrumcanbecalculatedinonestepbyassigningaprocessorforeverytwopoints.Theparallelisminthewaveletalgorithmmakesitattractiveforhardwareimplementation.

JavaSourceCodeDownloadTheWebpagefordownloadingtheHaarwaveletsourcecodecanbefoundhere(/software/java/wavelets/index.html).ThisJavacodeisextensivelydocumentedandthiswebpageincludesalinktotheJavadocgenerateddocumentation.

AsimplerversionoftheHaarwaveletalgorithmcanbefoundviamywebpageTheWaveletLiftingScheme(/misl/misl_tech/wavelets/lifting/index.html).

ResourcesGnuPlot

Theplotsabovearegeneratedwithgnuplot(http://www.gnuplot.org)forWindowsNT.SeemywebpageofGnuplotlinkshere(/misl/misl_tech/plotting.html).Iamonlymarginallystatisifiedwithgnuplot.ThesoftwareiseasytouseandtheWindowsNTversioncomeswithaniceGUIandanicehelpsystem.However,whenitcomesto3Dplots,thesoftwareleavessomethingstobedesired.Thehiddenlineremovalconsumesvastamountsofvirtualmemory.WhenItriedtoplotoneofthecoefficientssurfaceswiththexandzaxesswitched,itranoutofmemoryonaWindowsNTsystemwith256Kofvirtualmemory.Also,thesurfacewouldbemucheasiertounderstandifitcouldbecoloredwithaspectrum.Ifyouknowofabetter3DplottingpackagethatrunsonWindowsNT,pleasedropmeanote.

Ihavealsohadahardtimegettinggnuplottogenerate2Dplotswithmultiplelinesthathavedifferentcolors.Ihavesucceededindoingthisonlywhenthedataforeachlinewasinaseparatefile,whichcanbeawkward.

ROOT:AnObjectOrientedDataAnalysisFramework(http://root.cern.ch/)

IwassentthereferencetoRootbyaphysicist,CostasA.RootisadataanalysisframeworkthatistargetedatthemassiveamountsofdatageneratedbyhighenergyphysicsexperimentsatCERNandelsewhere.



AlthoughRootleansheavilytowardphysics,itlookstomelikeRootwouldbeusefulinotherareas.Someofthestatisticaltechniquesthatareusedtoanalyzeresultsinexperimentalphysicsisalsousedinquantitivefinance,forexample.

RoothasdifferentgoalsthangnuPlot.Itistargetedatamuchmorechallengingdataanalysisenviroment(terabytesofdata).ButithasalargelearningcurveandI'mskepticalifitcanbeeasilyusedbythosewhodonothaveasophisticatedcommandofC++.IncontrastgnuPlotisasimpleplottingenvironment.Somysearchforabetterplottingenvironmentcontinues.IknowthatsuchenvironmentsaresupportedbyMatlabandMathematics,butthesepackagesaretooexpensiveformylimitedsoftwarebudget.

ReferencesBooks

RipplesinMathematics:theDiscreteWaveletTransformbyJensenandlaCourHarbo,2001

SofarthisisthebestbookI'vefoundonwavelets.IreadthisbookafterIhadspentmonthsreadingmanyofthereferencesthatfollow,soI'mnotsurehoweasythisbookwouldbeforsomeonewithnopreviousexposuretowavelets.ButIhaveyettofindany"easy"reference.RipplesinMathematicscoversLiftingSchemewaveletswhichareeasiertoimplementandunderstand.Thebookiswrittenatarelativelyintroductorylevelandisaimedatengineers.Theauthorsprovideimplementationsforanumberofwaveletalgorithms.RipplesalsocoverstheproblemofapplyingwaveletalgorithmslikeDaubechiesD4tofinitedatasets(e.g.,theycoversomesolutionsfortheedgeproblemsencounteredforDaubechieswavelets).

WaveletsandFilterBanksbyGilbertStrangandTruongNguyen,WellesleyCambridgePr,1996

Acolleaguerecommendthisbook,althoughhecouldnotloadittomesinceitispackedawayinabox.Sadlythisbookishardtofind.Iboughtmycopyviaabebooks.com,used,fromabookdealerinAustralia.WhileIwaswaitingforthebookIreadafewofGilbertStrang'sjournalarticles.GilbertStrangisoneofthebestwritersI'veencounteredinmathematics.Ihaveonlyjuststartedworkingthroughthisbook,butitlookslikeanexcellent,althoughmathematical,bookonwavelets.

WaveletsMadeEasybyYvesNievergelt,Birkhauser,1999

ThisbookshastwoexcellentchaptersonHaarwavelets(Chapter1covers1DHaar



waveletsandChapter2covers2Dwavelets).AtleastinhiscoverageofHaarwavelts,Prof.Nievergeltwritesclearlyandincludesplentyofexamples.ThecoverageofHaarwaveletsusesonlybasicmathematics(e.g.,algebra).

FollowingthechapteronHaarwaveletsthereisachapteronDaubechieswavelets.Daubechieswaveletsarederivedfromageneralclassofwavelettransforms,whichincludesHaarwavelets.Daubechieswaveletsarebetterforsmoothlychangingtimeseries,butareprobablyoverkillforfinancialtimeseries.AsWaveletsMadeEasyprogresses,itgetslesseasy.FollowingthechapteronDaubechieswaveletsisadiscussionofFouriertransforms.Thelaterchaptersdelveintothemathematicsbehindwavelets.Prof.NievergeltprettymuchleftmebehindatthechapteronFouriertransforms.ForanapproachablediscussionofFouriertransforms,seeUnderstandingDigitalSignalProcessingbyRichardG.Lyons(below).

AsWaveletsMadeEasyprogresses,itbecomeslessandlessusefulforwaveletalgorithmimplementation.Infact,whilethemathematicsNievergeltusestodescribeDaubechieswaveletsiscorrect,thealgorithmhedescribestoimplementtheDaubechiestransformandinversetransformseemstobewrong.

WaveletsMadeEasydoesnotliveuptothe"easy"partofitstitle.GiventhisandtheapparenterrorsintheDaubechiescoverage,IamsorrytosaythatIcan'trecommendthisbook.SaveyourmoneyandbuyacopyofRipplesinMathematics.

DiscoveringWaveletsbyEdwardAboufadelandStevenSchlicker

At125pages,thisisoneofthemostexpensivewaveletbooksI'vepurchased,onaperpagebasis.ItsellsonAmazon(http://www.amazon.com)for$64.95US.Iboughtitusedfor$42.50.

IfDiscoveringWaveletsprovidedashort,cleardescriptionofwavelets,thelengthwouldbeavirtue,notafault.Sadlythisisnotthecase.DiscoveringWaveletsseemstobeabookwrittenforcollegestudentswhohavecompletedcalculusandlinearalgebra.Thebookisheavyontheorms(whichareincompletelyexplained)andverysortonusefulexplaination.Ifoundthedescriptionofwaveletsunnecessarilyobscure.Forexample,Haarwaveletsaredescribedintermsoflinearalgebra.Theycanbemuchmoresimplydescribedintermsofsums,differencesandthesocalledpyramidalalgorithm.

WhileDiscoveringWaveletscoverssomeimportantmaterial,itscoverageissoobscureand



cursorythatIfoundthebookuseless.Thebookresemblesasetoflecturenotesandisoflittleusewithoutthelecture(fortheirstudent'ssakeIhopethatAboufadelandSchlickerarebetterteachersthanwriters).ThisisabookthatIwishIhadnotpurchased.

WaveletMethodsforTimeSeriesAnalysisbyDonaldB.PercivalandAndrewT.Walden,CambridgeUniversityPress,2000

I'mnotamathematicianandIdon'tplayoneontelevision.Sothisbookisheavygoingforme.Nevertheless,thisisagoodbook.Forsomeonewithabettermathematicalbackgroundthismightbeanexcellentbook.Theauthorsprovideacleardiscussionofwaveletsandavarietyoftimeseriesanalsysistechniques.Unlikesomemathematicians,PercivalandWaldenactuallycodedupthewaveletalgorithmsandunderstandthedifficultiesofimplementation.Theycomparevariouswaveletfamiliesforvariousapplicationsandchosethesimplestone(Haar)insomecases.

OneofthegreatbenifitsofWaveletMethodsforTimeSeriesAnalysisisthatitprovidesaclearsummaryofagreatdealoftherecentresearch.ButPercivalandWaldenputtheresearchinanappliedcontext.ForexampleDonohoandJohnstonepublishedanequationforwaveletnoisereduction.IhavebeenunabletofindalloftheirpapersontheWebandIhaveneverunderstoodhowtocalculatesomeofthetermsintheequationinpractice.IfoundthisdefinitioninWaveletMethods.

TheWorldAccordingtoWavelets:TheStoryofaMathematicalTechniqueintheMakingbyBarbaraBurkeHubbard,A.K.Peters,1996

Thisbookprovidesaninterestinghistoryofthedevelopmentofwavelets.Thisincludessketchesofmanyofthepeopleinvolvedinpioneeringtheapplicationandmathematicaltheorybehindwavelets.AlthoughMs.Hubbardmakesaheroiceffort,Ifoundtheexplainationofwaveletsdifficulttofollow.

TheCartoonGuideToStatisticsbyLarryGonicandWoollcottSmith,HarperCollins

Iworkwithanumberofmathematicians,soit'sabitembarrassingtohavethisbookonmydisk.Inevertookstatistics.IncollegeeveryoneIknewwhotookstatisticsdidn'tlikeit.Sinceitwasnotrequiredformymajor(ascalculuswas),Ididnottakestatistics.I'vecometounderstandhowusefulstatisticsis.IwantedtofilteroutGaussiannoise,soIneededtounderstandnormalcurves.Althoughthetitleisabitembarrassing,TheCartoonGuideto



Statisticsprovidedaveryrapidandreadableintroductiontostatistics.

UnderstandingDigitalSignalProcessingbyRichardG.Lyons.

Thisbookisfantastic.Perhapsthebestintroductorybookeverwrittenondigitalsignalprocessing.Itisthebookonsignalprocessingforsoftwareengineerslikemyselfwithtepidmathematicalbackgrounds.ItprovidesthebestcoverageI'veeverseenonDFTsandFFTs.Infact,thisbookhasinspiredmetotryFFTsonfinancialtimeseries(http://www.bearcave.com/misl/misl_tech/signal/nonstat/index.html)(aninterestingexperiment,butwaveletsproducebetterresultsandFouriertransformsonnonstationarytimeseries).

SeemywebpageANotebookCompiledWhileReadingUnderstandingDigitalSignalProcessingbyLyons(/misl/misl_tech/signal/index.html)

WebbasedreferencesMywebpage(/misl/misl_tech/wavelets/lifting/index.html)onthewaveletLiftingScheme.TheHaarwaveletalgorithmexpressedusingthewaveletLiftingSchemeisconsiderablysimplerthanthealgorithmreferencedabove.TheLiftingSchemealsoallowsHaarwavelettobeextendedintoawaveletalgorithmsthathaveperfectreconstructionandhavebettermultiscaleresolutionthanHaarwavelets.

HaarWaveletTransform(http://dmr.ath.cx/gfx/haar/)byEmilMikulic

EmilMikulichaspublishedasimpleexplainationoftheHaartransform,forboth1Dand2Ddata.Forthosewhofindmyexplainationobscure,thismightbeagoodresource.

TheWaveletTutorial(http://engineering.rowan.edu/~polikar/WAVELETS/WTtutorial.html):TheEngineer'sUltimateGuidetoWaveletAnalysis,byRobiPolikar.

The"ultimateguide"towaveletanalysishasyettobewritten,atleastformypurposes.ButProf.Polikar'sWaveletTutorialisexcellent.WhenitcomestoexplainingWaveletsandFouriertransforms,thisisoneofthebestoverviewsI'veseen.Prof.PolikarputagreatdealofworkintothistutorialandIamgreatefulforhiseffort.However,therewasnotsufficientdetailinthistutorialtoallowmetocreatemyownwaveletandinversewavelettranformsoftware.

AReallyFriendlyGuidetoWavelets



(http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html)

ThisWebpage(whichisalsoavailableinPDF)providesaniceoverviewofthetheorybehindwavelets.ButaswithRobiPolikar'swebpage,itsabigstepfromthismaterialtoasoftwareimplementation.WhetherthisWebpageis"reallyfriendly"dependsonwhoyourfriendsare.Ifyoufriendsarecalculusandtaylorseries,thenthispaperisforyou.AfterworkingmywaythroughagoodpartofWaveletsMadeEasythispaperfilledinsomeholeforme.ButIwouldnothaveunderstooditifIhadreaditbeforeWaveletsMadeEasy.

BellLabsWaveletsGroupHomePage(http://cm.belllabs.com/who/jelena/Wavelet/)

WimSweldens,whohaspublishedalotofmaterialontheWeb(heistheeditorofWaveletDigest(http://www.wavelet.org))andelsewhereonWaveletsisamemberofthisgroup.Aninterestingsitewithlotsofgreatlinkstootherwebresources.

SeealsoWimSwelden'sWaveletCascadeJavaApplet(http://netlib.belllabs.com/cm/ms/who/wim/cascade/index.html)

LiftingSchemeWavelets

WinSweldensandIngridDaubechiesinventedanewwavelettechniqueknownastheliftingscheme.GabrielFernandezhaspublishedanexcellentbibliographyontheliftingschemewaveletswhichcanbefoundhere(http://www.cse.sc.edu/~fernande/liftpack/liftbibl.html).ThisbibliographyhasapointertoWimSweldens'andPeterSchroder'sliftingschemetutorialBuildingYourOwnWaveletsatHome.

ClemensValenshaswrittenatutorial(http://perso.wanadoo.fr/polyvalens/clemens/lifting/lifting.html)onthefastliftingwavelettransform.Thisisarathermathematicallyorientedtutorial.Formany,WimSweldens'paperBuildingYourOwnhWavletsatHomemaybeeasiertounderstand(althoughIstillfoundthispaperheavygoing).

GabrielFernandezhasdevelopedLiftPack.TheLiftPackHomePage(http://www.cse.sc.edu/~fernande/liftpack/index.html)publishestheLiftPacksoftware.ThebibliographyisasubpageoftheLiftPackHomepage.

SeealsomywebpageonTheWaveletLiftingScheme



(/misl/misl_tech/wavelets/lifting/index.html).

WaveletsinComputerGraphis

OneofthepapersreferencedinGabrielFernandez'sliftingschemebibliographyisWimSweldensandPeterSchroder'spaperBuildingYourOwnWaveletsatHome.ThisispartofacourseonWaveletsinComputerGraphics(http://www.multires.caltech.edu/teaching/courses/waveletcourse/)givenatSigGraph1994,1995and1996.ThesigGraphcoursecoverdanamazingamountofmaterial.BuildingYourOwnWaveletsatHomewasapparentlycoveredinamorning.Therearealotofmathematicallygiftedpeopleincomputergraphics.Butevenforthesepeople,thislooksliketoughgoingforamorning.I'vespenthoursreadingandrereadingthistutorialbeforeIunderstooditenoughtoimplementthepolynomialinterpolationwaveletsthatitdiscusses.

WaveletShrinkage(denoising)()

D.Donoho(http://wwwstat.stanford.edu/~donoho/)WaveletShrinkageandW.V.D.ATenMinuteTour(http://wwwstat.stanford.edu/~donoho/Reports/1993/toulouse.ps.Z),(figures)(http://wwwstat.stanford.edu/~donoho/Reports/1993/toulouse_figs.ps.Z),1993.

D.Donoho(http://wwwstat.stanford.edu/~donoho/)DeNoisingBySoftThresholding(http://wwwstat.stanford.edu/~donoho/Reports/1992/denoiserelease3.ps.Z),IEEETrans.onInformationTheory,Vol41,No.3,pp.613627,1995.

D.Donoho(http://wwwstat.stanford.edu/~donoho/)AdaptingtoUnknownSmoothnessviaWaveletShrinkage(http://spib.rice.edu/spib/papers/dsp/1994.02/009.ps.Z),JASA,1995.

CalTechMultiResolutionModelingGroupPublications

TheWaveletsinComputerGraphicspage,referencedabove,isoneofthelinksfromtheCalTechMultiresolutionModelingGroupPublications(http://www.multires.caltech.edu/pubs/)webpage.Thewaveletpublicationsreferencedonthispageconcentrateonwaveletapplicationsforcomputergraphics.

TutorialonContinuousWaveletAnalysisofExperiementalData(http://www.mame.syr.edu/faculty/lewalle/tutor/tutor.html)byJacquesLewalle,Syracuse



University.

Thisisyetanother"introductory"tutorialbyamathematician.Itgivesafeelingforwhatyoucandowithwavelets,butthereisnotenoughdetailtounderstandthedetailsofimplementingwaveletcode.

Amara'sWaveletPage(http://www.amara.com/current/wavelet.html)

AmaraGraps'webpageprovidessomegoodbasicintroductorymaterialonwaveletsandsomeexcellentlinkstootherWebresources.Thereisalsoalinktotheauthor's(Amara)IEEEComputationalSciencesandEngineeringarticleonwavelets.

Wave++fromRyersonPolytechnicUniversityComputationalSignalsAnalysisGroup

Wave++(http://www.scs.ryerson.ca/~lkolasa/CppWavelets.html)isaC++classlibraryforwaveletandsignalanalysis.Thislibraryisprovidedinsourceform.Ihavenotexamineditindetailyet.

Waveletandsignalprocessingalgorithmsareusuallyfairlysimple(theyconsistofarelativelysmallamountofcode).Myexperiencehasbeenthattheimplementationofthealgorithmsisnotastimeconsumingasunderstandingthealgorithmsandhowtheycanbeapplied.Sinceoneofthebestwaystounderstandthealgorithmsistoimplementandapplythem,I'mnotsurehowmuchleverageWave++providesunlessyoualreadyunderstandwaveletalgorithms.

WaveletCompressionArrives(http://www.seyboldreports.com/SRIP/wavelet/)byPeterDyson,SeyboldReports,April1998.

Thisisanincreasinglydateddiscussiononwaveletcompressionproducts,especiallyforimages.Thedescriptionofthecompressionproductsstrengthsandweaknessesisgood,butthedescriptionofwaveletsispoor.

WebpageofZbigniewR.Struzik(http://www.cwi.nl/~zbyszek/)

Prof.ZbigniewR.StruzikofCentrumvoorWiskundeenInformaticaintheNetherlandshasdonesomeveryinterestingworkwithwaveletsinavarietyofareas,includingdatamininginfinance.ThiswebpagehasalinktoProf.Struzik'spublications(atthebottomoftheWeb



page).Prof.Struzik'sworkalsoshowssomeinterestingconnectionsbetweenfractalsandwavelets.

DisclaimerThiswebpagewaswrittenonnightsandweekends,usingmycomputerresources.ThisWebpagedoesnotnecessarilyreflecttheviewsofmyemployer(atthetimethiswebpagewaswritten).Nothingpublishedhereshouldbeinterpretedasareflectiononanytechniquesusedbymyemployer(atthattime).

IanKaplan,July2001Revised:February2004

(mailto:[email protected])

applying the haar wavelet transform to time series information

Documents