Modeling and Veri� ation of DSP Designs in HOLBehzad Akbarpour

A ThesisinThe DepartmentofEle tri al and Computer EngineeringPresented in Partial Ful�llment of the Requirementsfor the Degree of Do tor of Philosophy atCon ordia UniversityMontr�eal, Qu�ebe , Canada

April 2005 Behzad Akbarpour, 2005

CONCORDIA UNIVERSITYDivision of Graduate StudiesThis is to ertify that the thesis preparedBy: Behzad AkbarpourEntitled: Modeling and Veri� ation of DSP Designs in HOLand submitted in partial ful�lment of the requirements for the degree ofDo tor of Philosophy omplies with the regulations of this University and meets the a epted standardswith respe t to originality and quality.Signed by the �nal examining ommittee: Dr. Krzyzak, AdamDr. Harrison, John R.Dr. Hassan, Ibrahim G.Dr. Ait Mohamed, OtmaneDr. Lyn h, William E.Dr. Tahar, So��eneApproved by Chair of the ECE Department2005 Dean of Engineering

iiiABSTRACTModeling and Veri� ation of DSP Designs in HOL

Behzad Akbarpour, Ph.D.Con ordia University, 2005In this thesis we propose a framework for the in orporation of formal methodsin the design ow of DSP (Digital Signal Pro essing) systems in a rigorous way. Inthe proposed approa h we model and verify DSP des riptions at di�erent abstra tionlevels using higher-order logi based on the HOL (Higher Order Logi ) theoremprover. This framework enables the formal veri� ation of DSP designs whi h in thepast ould only be done partially using onventional simulation te hniques. To thisend, we provide a shallow embedding of DSP des riptions in HOL at the oating-point, �xed-point, behavioral, RTL (Register Transfer Level), and netlist gate levels.We make use of existing formalization of oating-point theory in HOL and introdu ea parallel one for �xed-point arithmeti . The high ability of abstra tion in HOLallows a seamless hierar hi al veri� ation en ompassing the whole DSP design path,starting from top level oating- and �xed-point algorithmi des riptions down toRTL, and gate level implementations. We illustrate the new veri� ation frameworkusing di�erent ase studies su h as digital �lters and FFT (Fast Fourier Transform)algorithms.

ivTo My Family

vACKNOWLEDGEMENTSI have been very fortunate to have Dr. So��ene Tahar as my supervisor. Iam deeply grateful for his strong support and en ouragement through out my Ph.Dstudies. His expertise and ompetent advi e have shaped the hara ter of my re-sear h.Throughout my study in Con ordia many people have en ouraged and helpedme through many obsta les. I have enjoyed studying and working with my ol-leagues in the HVG group in Con ordia University, wishing to thank all of them fortheir support and the ni e time we have spent together.I would like to express my gratitute and thanks to Dr. Harrison from Intel fora epting to be my external examiner. I ould not have a better expert than himworld wide.Many thanks also to the HOL ommunity experts who helped me out through-out the thesis, in parti ular Dr. Hurd, Dr. Slind, Dr. Norrish, ... just to name afew. I also wish to express my gratitude to the examination ommittee members,Dr. Hassan, Dr. Ait Mohamed, and Dr. Lyn h, for reviewing my thesis and givingme invaluable feedba k.I would like to reserve my deepest thanks to my family for their perpetual loveand en ouragement. I an never thank them enough.

TABLE OF CONTENTSLIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixLIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xLIST OF ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi1 Introdu tion 11.1 General Obje tive: System Veri� ation . . . . . . . . . . . . . . . . . 11.2 Spe i� Obje tives: DSP Veri� ation . . . . . . . . . . . . . . . . . . 21.3 State of the Art: Simulation . . . . . . . . . . . . . . . . . . . . . . . 41.4 Proposed Solution: Formal Veri� ation . . . . . . . . . . . . . . . . . 41.5 Proposed DSP Veri� ation Framework . . . . . . . . . . . . . . . . . 61.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.6.1 Error Analysis in Formal Veri� ation . . . . . . . . . . . . . . 91.6.2 Floating-Point Formal Veri� ation . . . . . . . . . . . . . . . . 101.6.3 Error Analysis of Digital Filters . . . . . . . . . . . . . . . . . 131.6.4 Error Analysis of FFT Algorithms . . . . . . . . . . . . . . . . 151.6.5 Formalization and Veri� ation of FFT Algorithms . . . . . . . 161.7 Contributions of the Thesis . . . . . . . . . . . . . . . . . . . . . . . 171.8 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . 182 Formalization of Fixed-Point Arithmeti in HOL 192.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2 HOL Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.3 Fixed-Point Arithmeti . . . . . . . . . . . . . . . . . . . . . . . . . . 232.3.1 Fixed-Point Numbers . . . . . . . . . . . . . . . . . . . . . . . 232.3.2 Fixed-Point Operations . . . . . . . . . . . . . . . . . . . . . . 252.4 Formalizing Fixed-Point Arithmeti in HOL . . . . . . . . . . . . . . 302.4.1 Fixed-Point Numbers Representation . . . . . . . . . . . . . . 31vi

2.4.2 Fixed-Point Type . . . . . . . . . . . . . . . . . . . . . . . . . 322.4.3 Fixed-Point Valuation . . . . . . . . . . . . . . . . . . . . . . 332.4.4 Ex eption Handling . . . . . . . . . . . . . . . . . . . . . . . . 352.4.5 Quantization . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.4.6 Fixed-Point Arithmeti Operations . . . . . . . . . . . . . . . 382.5 Veri� ation of Fixed-Point Operations . . . . . . . . . . . . . . . . . 402.6 Appli ation with SPW . . . . . . . . . . . . . . . . . . . . . . . . . . 462.7 Con lusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513 Error Analysis of Digital Filters in HOL 523.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.2 Error Analysis Models . . . . . . . . . . . . . . . . . . . . . . . . . . 543.2.1 Floating-Point Error Model . . . . . . . . . . . . . . . . . . . 543.2.2 Fixed-Point Error Model . . . . . . . . . . . . . . . . . . . . . 583.3 Error Analysis of Digital Filters using HOL . . . . . . . . . . . . . . 603.3.1 First-Order Filter . . . . . . . . . . . . . . . . . . . . . . . . . 633.3.2 Se ond-Order Filter . . . . . . . . . . . . . . . . . . . . . . . . 673.3.3 Lth-Order Filter (Dire t Form) . . . . . . . . . . . . . . . . . 723.3.4 Lth-Order Filter (Parallel Form) . . . . . . . . . . . . . . . . 803.3.5 Lth-Order Filter (Cas ade Form) . . . . . . . . . . . . . . . . 883.4 Con lusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 964 Veri� ation of FFT Algorithms in HOL 974.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974.2 Error Analysis of FFT Algorithms in HOL . . . . . . . . . . . . . . . 1004.3 FFT Design Implementation Veri� ation . . . . . . . . . . . . . . . . 1164.4 Con lusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123vii

5 Con lusions and Future Work 1245.1 Con lusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1245.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126Bibliography 127

viii

LIST OF TABLES2.1 HOL Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.2 Fixed-Point Quantization Modes . . . . . . . . . . . . . . . . . . . . 272.3 Fixed-Point Over ow Modes . . . . . . . . . . . . . . . . . . . . . . . 28

ix

LIST OF FIGURES1.1 DSP design ow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Propsed DSP spe i� ation and veri� ation approa h . . . . . . . . . . 72.1 The behavior of �xed-point quantization modes . . . . . . . . . . . . 282.2 The behavior of �xed-point over ow modes . . . . . . . . . . . . . . . 302.3 Corre tness riteria for �xed-point addition . . . . . . . . . . . . . . . 402.4 Fixed-point values on the real axis . . . . . . . . . . . . . . . . . . . 442.5 SPW design of an integrator . . . . . . . . . . . . . . . . . . . . . . . 473.1 Error Analysis Approa h . . . . . . . . . . . . . . . . . . . . . . . . . 533.2 Basi forms of digital �lter realizations . . . . . . . . . . . . . . . . . 613.3 Error owgraph for the �rst-order �lter . . . . . . . . . . . . . . . . . 653.4 Error owgraph for the se ond-order �lter . . . . . . . . . . . . . . . 683.5 Error owgraph for Lth-order �lter (Dire t form) . . . . . . . . . . . 743.6 Error owgraph for Lth-order �lter (Parallel form) . . . . . . . . . . . 813.7 Error owgraph for Lth-order �lter (Cas ade form) . . . . . . . . . . 894.1 Proposed FFT spe i� ation and veri� ation approa h . . . . . . . . . 994.2 Signal owgraph of de imation-in-frequen y FFT, N = 24 . . . . . . . 1024.3 Signal owgraph of de imation-in-time FFT, N = 24 . . . . . . . . . 1034.4 Error owgraph for de imation-in-frequen y FFT . . . . . . . . . . . 1064.5 Radix-4 16-point pipelined FFT implementation . . . . . . . . . . . . 1164.6 Signal owgraph of radix-4 16-point FFT . . . . . . . . . . . . . . . . 118

x

LIST OF ACRONYMSACL2 A Computational Logi Appli ative Common LispBDD Binary De ision DiagramCAD Computer Aided DesignDFT Dis rete Fourier TransformDIF De imation-in-Frequen yDIT De imation-in-TimeDSP Digital Signal Pro essingFFT Fast Fourier TransformFRIDGE Fixed-point pRogrammIng DesiGn EnvironmentHDL Hardware Des ription LanguageHOL Higher Order Logi IEEE Inistitute of Ele tri al and Ele troni s EngineersIFT Inverse Fourier TransformLCF Logi for Computable Fun tionsLSB Least Signi� ant BitML Meta LanguageMSB Most Signi� ant BitOFDM Orthogonal Frequen y Division MultiplexingPVS Prototype Veri� ation SystemRTL Register Transfer LevelSMV Symboli Model Veri�erSPW Signal Pro essing WorkSystemSTE Symboli Traje tory EvaluationVLSI Very Large S ale Integrationxi

Chapter 1Introdu tion1.1 General Obje tive: System Veri� ationToday, hardware and software systems are widely used in appli ations where failureis una eptable: ele troni ommer e, telephone swit hing networks, highway andair traÆ ontrol systems, medi al instruments, and other examples too numerousto list. We frequently read of in idents where some failure is aused by an errorin a hardware or software system. A re ent example of su h a failure is the Ari-ane 5 ro ket, whi h exploded on June 4, 1996, less than forty se onds after it waslaun hed. The ommittee that investigated the a ident found that it was ausedby a software error in the omputer that was responsible for al ulating the ro ket'smovement. During the laun h, an ex eption o urred when a large 64-bit oatingpoint number was onverted to a 16-bit signed integer. This onversion was notprote ted by ode for handling ex eptions and aused the omputer to fail. Thesame error also aused the ba kup omputer to fail. As a result in orre t altitudedata was transmitted to the on-board omputer, whi h aused the destru tion ofthe ro ket. The team investigating the failure suggested that several measures betaken in order to prevent similar in idents in the future, in luding the veri� ation ofthe Ariane 5 software. Clearly, the need for reliable hardware and software systems1

is riti al. As the involvement of su h systems in our lives in reases, so too doesthe burden for ensuring their orre tness. Unfortunately, it is no longer feasible toshut down a malfun tioning system in order to restore safety. We are very mu hdependent on su h systems for ontinuous operation; in fa t, in some ases, devi esare less safe when they are shut down. Even when failure is not life-threatening,the onsequen es of having to repla e riti al ode or ir uitry an be e onomi allydevastating. Be ause of the su ess of the Internet and embedded systems in auto-mobiles, airplanes, and other safety riti al systems, we are likely to be ome evenmore dependent on the proper fun tioning of omputing devi es in the future. Infa t, the pa e of hange will likely a elerate in oming years. Be ause of this rapidgrowth in te hnology, it will be ome even more important to develop methods thatin rease our on�den e in the orre tness of su h systems.1.2 Spe i� Obje tives: DSP Veri� ationDigital system design is hara terized by ever in reasing system omplexity thathas to be implemented within redu ed time, resulting in minimum osts and shorttime-to-market. These hara teristi s all for a seamless design ow that allows toperform the design steps on the highest suitable level of abstra tion. For most digitalsignal pro essing systems, the design has to result in a �xed-point implementation.This is due to the fa t that these systems are sensitive to power onsumption, hip size and pri e per devi e. Fixed point realizations outperform oating-pointrealizations by far with regard to these riteria. A typi al DSP design ow is depi tedin Figure 1.1 [45℄. An algorithm design starts from a oating-point des ription.This allows to ignore the e�e ts of �nite wordlengths and �xed exponents and toabstra t from all implementation details. Additionally, the use of oating-pointmodels o�ers a maximum degree of reusability. On the �xed-point level, all operandsare assigned a �xed word length and a �xed exponent, while the ontrol stru ture and2

the operations of the oating point program remain un hanged. This des riptionis used to analyze whether the �xed-point model ful�lls the algorithmi systemrequirements. The transformation to the �xed-point is quite tedious and error-prone. On the implementation level, the �xed-point model of the algorithm hasto be transferred into the best suited target des ription, either using a hardwaredes ription language (HDL) or a programming language. These requirements havebeen the motivation for the development of CAD tools for DSP design. Examplesof su h tools are SPW (Caden e) [12℄, CoCentri (Synopsys) [15℄, Matlab-Simulink(Mathworks) [49℄, and FRIDGE (Aa hen UT) [45℄.Idea

Floating-PointAlgorithm

OK?No

Yes

Quantization

Fixed-PointAlgorithm

NoOK?

Yes

Code Generation

ArchitecturalDescription

OK?No

Yes

Target System

Impl

emen

tatio

n Le

vel

Algo

rithm

ic L

evel

Fixe

d-Po

int

Floa

ting-

Poin

t

Figure 1.1: DSP design owUsually the onforman e of the �xed-point implementation with respe t tothe des riptions in oating-point algorithmi , and RT and gate levels is veri�edby simulation te hniques whi h annot over all design errors, espe ially for largesystems. On the other hand, adopting formal veri� ation [43℄ in system designgenerally means using methods of mathemati al proof rather than simulation toensure the quality of the design, to improve the robustness of a design and to speedup the development. The overall aim for the proposed resear h is to model the DSP3

des riptions at di�erent abstra tion levels based on the shallow embedding approa hto enable their formal veri� ation in the HOL theorem proving environment.1.3 State of the Art: SimulationToday, the usual validation method to dis over the errors in the design ow of DSPsystems is still simulation. In this method, a simulation run must be performed inea h level of abstra tion su h as oating-point, �xed-point, RT and gate level to he k if the required hara teristi s are preserved. With simulation, input signalsare inje ted at ertain points in the system and the resulting signals at other pointsare observed. These methods an be a ost-eÆ ient way to �nd errors. However,in order to get full on�den e in the design we would have to perform a ompletesimulation whi h overs all possible input ombinations. Exhaustive simulationof even moderately-sized ir uits is impossible, and partial simulation o�ers onlypartial assuran e of orre tness. This is an espe ially serious problem in safety- riti al appli ations, where failure due to design errors may ause loss of life orextensive damage. In these appli ations, fun tional errors in ir uit designs annotbe tolerated. But even where safety is not the primary onsideration, there maybe important e onomi reasons for doing everything possible to eliminate designerrors, and to eliminate them early in the design pro ess. A awed design may mean ostly and time- onsuming refabri ation, and mass-produ ed devi es may have tobe re alled and repla ed.1.4 Proposed Solution: Formal Veri� ationA solution to these problems is one of the goals of formal methods [52℄ for veri� ationof the orre tness of hardware designs, sometimes just alled hardware veri� ation.With this approa h, the behaviour of hardware devi es is des ribed mathemati ally,4

and formal proof is used to verify that they meet rigorous spe i� ations of intendedbehaviour.However, formal veri� ation is not the golden rule in ir uit testing be auseof some limitations. A orre tness proof annot guarantee that the real devi e willnever malfun tion; the design model of the devi e may be proved orre t, but thehardware a tually built an still behave in a way unintended by the designer (thisis the ase for simulation too). Wrong spe i� ations an play a major role in this,be ause it has been veri�ed that the system will fun tion as spe i�ed, but it has notbeen veri�ed that it will work orre tly. Defe ts in physi al fabri ation an ausethis problem too. In formal veri� ation a model of the design is veri�ed, not the realphysi al implementation. Therefore, a fault in the modeling pro ess an give falsenegatives (errors in the design whi h do not exist). Although sometimes, the fault overs some real errors. Be ause of these limitations we an onsider simulationand hardware veri� ation as omplementary te hniques, the methods have to playtogether.Formal veri� ation methods an be ategorized in two main groups [67℄, theo-rem proving and model he king. Theorem proving refers to the use of axioms andproof rules to prove the orre tness of the systems. In this method, one expressesthe system model and spe i� ations in a suitable logi , and onstru ts a proof in thelogi that the system model implies the spe i� ations. The powerful mathemati alte hniques su h as indu tion and abstra tion are the strengths of theorem provingand make it a very exible and powerful veri� ation te hnique. It makes it possi-ble to onstru t a model at almost every abstra tion level and proves properties onall lasses of systems. However, it is a time onsuming pro ess whi h an involvegenerating and proving literally hundreds of lemmas in painstaking detail. Model he king, on the other hand, is more limited in s ope, but is fast and fully auto-mated. The system model is in essen e a �nite state ma hine, and spe i� ations arewritten in temporal logi . These logi s are limited with respe t to the very powerful5

logi s handled by general theorem provers, but are quite simple and on ise, and an express a wide variety of useful properties.1.5 Proposed DSP Veri� ation FrameworkIn this thesis we propose a methodology for applying formal methods to the design ow of DSP systems in a rigorous way. The orresponding ommutating diagram isshown in Figure 1.2. Thereafter, we model the ideal real spe i� ation of the DSPalgorithms and the orresponding oating- and �xed-point representations as wellas the RT and gate level implementations as predi ates in higher-order logi . Theoverall methodology for the formal spe i� ation and veri� ation of DSP algorithmswill be based on the idea of shallow embedding of languages [4℄ using the HOLtheorem proving environment [23℄. In the proposed approa h, we �rst fo us on thetransition from real to oating- and �xed-point levels. For this, we make use ofexisting theories in HOL on the onstru tion of real [27℄ and omplex [32℄ numbers,the formalization of IEEE-754 standard based oating-point arithmeti [28, 29℄, andthe formalization of �xed-point arithmeti . We use valuation fun tions to �nd thereal values of the oating- and �xed-point DSP outputs and de�ne the error as thedi�eren e between these values and the orresponding output of the ideal real spe -i� ation. Then we establish fundamental lemmas on the error analysis of oating-and �xed-point roundings and arithmeti operations against their abstra t mathe-mati al ounterparts. Finally, based on these lemmas, we derive expressions for thea umulation of roundo� error in oating- and �xed-point DSP algorithms usingre ursive de�nitions and initial onditions. While theoreti al work on omputingthe errors due to �nite pre ision e�e ts in the realization of DSP algorithms with oating- and �xed-point arithmeti s has been extensively studied sin e the late six-ties [41℄, this thesis ontains the �rst formalization and proof of this analysis usinga me hani al theorem prover, here HOL. The formal results are found to be in good6

agreement with the theoreti al ones.After handling the transition from real to oating- and �xed-point levels, weturn to the HDL representation. At this point, we use well known te hniques tomodel the DSP design at the RTL level within the HOL environment. The laststep is to verify this level using a lassi al hierar hi al proof approa h in HOL [52℄.In this way, we hierar hi ally prove that the DSP RTL implementation implies thehigh level �xed-point algorithmi spe i� ation, whi h has already been related tothe oating-point des ription and the ideal real spe i� ation through the error anal-ysis. The veri� ation an be extended, following similar manner, down to gate levelnetlist either in HOL or using other ommer ial veri� ation tools as depi ted inFigure 1.2. This analysis is not overed in this thesis.FP Error

FP Real Value( HOL )

( HOL )

FP( HOL )

FXP

FP

(Convert)

(Convert) Analysis

REALREALShallow

Shallow

Shallow Valuation

Valuation

Embedding

Embedding

Embedding

AnalysisFXP Error

( HOL )FXP Real ValueFXP

( HOL )

AnalysisFP to FXP Error

Embedding

(Synthesize)

RTL

(Synthesize)ImplicationLogical

( HOL )

Netlist

Shallow

Shallow Netlist( HOL )Embedding

ImplicationLogical

RTL

Figure 1.2: Propsed DSP spe i� ation and veri� ation approa hThe pro ess of spe ifying a hardware des ription language in higher-order logi is ommonly known as semanti embedding. There are two main approa hes [4℄:deep embedding and shallow embedding. In deep embedding, the abstra t syntax ofa design des ription is represented by terms, whi h are then interpreted by semanti fun tions de�ned in the logi that assign meaning to the design. With this method,7

it is possible to reason about lasses of designs, sin e one an quantify over thesynta ti stru tures. However, setting up HOL types of abstra t syntax and semanti fun tions an be very tedious. In a shallow embedding on the other hand, thedesign is modeled dire tly by a formal spe i� ation of its fun tional behavior. Thiseliminates the e�ort of de�ning abstra t syntax and semanti fun tions, but it alsolimits the proofs to fun tional properties. In this thesis, sin e our main on ern isto he k the orre tness of the designs based on their fun tionality, we propose theshallow embedding for DSP des riptions: translate the intended meaning of DSPblo k desings as des ribed in its do umentation into HOL and then omplete theformal proof in the HOL theorem prover.In this thesis, we demonstrate how the methodology presented in this se tion an be used for the veri� ation of a parametri L order digital �lter and the fastFourier transform (FFT) algorithms implemented in di�erent anoni al forms of re-alization. Similar dis ussion an be applied to other types of �ltering and signalanalysis algorithms.When a linear re ursive di�eren e equation digital �lter is realized with oating-and �xed-point arithmeti , on a omputer or with spe ial-purpose hardware, errorsand onstraints due to �nite word length are unavoidable. The main ategories of�nite pre ision e�e ts are errors due to roundo� in the arithmeti operations, errorsdue to quantization of input, and e�e ts of oeÆ ient ina ura ies. These errorproblems have already been studied extensively [47℄. In this thesis, as the �rst asestudy we show how this error analysis an be me hani ally performed using HOLtheorem prover. We have used our veri� ation methodology to derive expressionsfor the a umulation of roundo� error in a parametri L order digital �lter, for ea hof the three anoni al forms of realization: dire t, parallel, and as ade.The fast Fourier transform (FFT) is an algorithm to ompute the dis reteFourier transform with a substantial time saving over onventional methods. FFTalgorithms are based on the fundamental prin iple of de omposing the omputation8

of the dis rete Fourier transform of a sequen e of length N into su essively smallerdis rete Fourier transforms. The manner in whi h this prin iple is implementedleads to a variety of di�erent algorithms, all with omparable improvements in om-putational speed. Two basi lasses of FFT algorithms are the de imation-in-timeand de imation-in-frequen y. As the se ond ase studty in this proje t, we onsiderthe formal veri� ation of the de imation-in-time and de imation-in-frequen y FFTalgorithms. We used our methodology to derive expressions for the a umulation ofroundo� error in oating- and �xed-point FFT algorithms by re ursive de�nitionsand initial onditions, onsidering the e�e ts of input quantization and ina ura yin the oeÆ ients. Based on the extensively studied theoreti al work on omputingthe errors due to �nite pre ision e�e ts in the realization of FFT algorithms with oating- and �xed-point arithmeti s [41℄, we perform a similar analysis using theHOL theorem proving environment. The formal results are found to be in goodagreement with the theoreti al ones.1.6 Related Work1.6.1 Error Analysis in Formal Veri� ationPrevious work on the error analysis in formal veri� ation was done by Harrison [29℄who veri�ed the oating-point algorithms su h as the exponential fun tion againsttheir abstra t mathemati al ounterparts using the HOL Light theorem prover. Asthe main theorem, he proved that the oating-point exponential fun tion has a orre t over ow behavior, and in the absen e of over ow the error in the result isbounded to a ertain amount. He also reported on an error in the hand proof mostlyrelated to forgetting some spe ial ases in the analysis. This error analysis is verysimilar to the type of analysis performed for DSP algorithms. The major di�eren e,however, is the use of statisti al methods and mean square error analysis for DSPalgorithms whi h is not overed in the error analysis of the mathemati al fun tions9

used by Harrison. In this method, the error quantities are treated as independentrandom variables uniformly distributed over a spe i� interval depending on the typeof arithmeti and the rounding mode. Then the error analysis is performed to deriveexpressions for the varian e and mean square error. To perform su h an analysis inHOL, we need to develop a me hanized theory on the properties of random variablesand random pro esses. This type of analysis is not addressed in this thesis and isa part of our future work. Huhn et al. [34℄ proposed a hybrid formal veri� ationmethod ombining di�erent state-of-the-art te hniques to guide the omplete design ow of impre isely working arithmeti ir uits starting at the algorithmi down tothe register transfer level. The usefulness of the method is illustrated with theexample of the dis rete osine transform algorithms. In parti ular, the authorshave shown the use of omputer algebra systems like Mathemati a or Maple atthe algorithmi level to reason about real numbers and to determine ertain errorbounds for the results of numeri al operations. In ontrast to [34℄, we proposean error analysis for digital �lters using the HOL theorem prover. Although the omputer algebrai systems su h as Maple or Mathemati a are mu h more popularand have many powerful de ision pro edures and heuristi s, theorem provers aremore expressive, more pre ise, and more reliable [33℄. One option is to ombinethe rigour of the theorem provers with the power of omputer algebrai systems asproposed in [33℄.1.6.2 Floating-Point Formal Veri� ationThere exist several related work in the open literature on the formalization andveri� ation of IEEE standard based oating-point arithmeti . For instan e, Barrett[2℄ spe i�ed parts of the IEEE-754 standard in Z, and Miner [54℄ formalized theIEEE-854 oating-point standard in PVS. The latter de�ned the relation between oating-point numbers and real numbers, rounding, and some arithmeti operationson both �nite and in�nite operands. He used this formalization to verify abstra t10

mathemati al des riptions of the main operations and their relation to the orre-sponding oating-point implementations. His work was one of the earliest on theformalization of oating-point standards using theorem proving. His formal spe i�- ation was then used by Miner and Leathrum [53℄ to verify in PVS a general lassof IEEE ompliant subtra tive division algorithms.Carreno [11℄ formalized the same IEEE-854 standard in HOL. He interpretedthe lexi al des riptions of the standard into mathemati al onditional des riptionsand organized them in tables, whi h were then formalized in HOL. He dis usseddi�erent standard aspe ts su h as pre isions, ex eptions and traps, and many otherarithmeti operations su h as addition, multipli ation, and square-root of oating-point numbers.Harrison [27℄ onstru ted the real numbers in HOL. He then developed inHOL a generi oating-point library [28℄ to de�ne the most fundamental terms ofthe IEEE-754 standard and to prove the orresponding orre tness analysis lemmas.He used this library to formalize and verify oating-point algorithms of omplexarithmeti operations su h as the square root, the exponential fun tion [29℄, andthe trans endental fun tions [30℄ against their abstra t mathemati al ounterparts.He also used the oating-point library for the veri� ation of the lass of divisionalgorithms used in the Intel IA-64 ar hite ture [31℄.Moore et al. [56℄ have veri�ed the AMD-K5 oating-point division algorithmusing the ACL2 theorem prover. Also, Russino� [64℄ has developed a oating-point library for the ACL2 prover and applied it su essfully to verify the oating-point multipli ation, division, and square root algorithms of the AMD-K5 and AMDAthlon pro essors.Aagaard and Seger [1℄ ombined BDD-based model- he king and theoremproving te hniques in the Voss hardware veri� ation system to verify the IEEE omplian e of the gate-level implementation of a oating-point multiplier. O'Learyet al. [62℄ reported on the spe i� ation and veri� ation of the Intel Pentium r Pro11

pro essor's oating-point exe ution unit at the gate level using a ombination ofmodel- he king and theorem proving. Leeser et al. [46℄ veri�ed a subtra tive radix-2 square root algorithm and its hardware implementation using the higher-orderlogi theorem proving system Nuprl. Chen and Bryant [14℄ used word-level SMVto verify a oating-point adder. Cornea-Hasegan [17℄ used iterative approa hes andmathemati al proofs to verify the orre tness of the IEEE oating-point square root,divide, and remainder algorithms.More re ently, Daumas et al. [19℄ have presented a generi library for reasoningabout oating-point numbers within the Coq system. This library was then usedin the veri� ation of IEEE- ompliant oating-point arithmeti algorithms [8℄ andhardware units [7℄. Berg et al. [3℄ have formally veri�ed a theory of IEEE roundingpresented in [57℄ using the theorem prover PVS. They have used a formal de�nitionof rounding based on Miner's formalization of the standard [54℄. This theory wasthen used to prove the orre tness of a fully IEEE ompliant oating-point unitused in the VAMP pro essor [6℄. Sawada and Gamboa [65℄ formally veri�ed the orre tness of a oating-point square root algorithm used in the IBM Power4TMpro essor. The veri� ation was arried out with the ACL2(r) theorem prover whi his an extension of the ACL2 theorem prover that performs reasoning on real numbersusing non-standard analysis. The proof required the analysis of the approximationerror on Chebyshev series by proving Taylor's theorem. Kaivola et al. [38, 40, 42℄presented the formal veri� ation of the oating-point multipli ation, division, andsquare root units of the Intel IA-32 Pentium r 4 mi ropro essor. The veri� ationwas arried out using the Forte veri� ation framework, a ombined model- he kingand theorem-proving system built on top of the Voss system. Model he king wasdone via symboli traje tory evaluation (STE), and theorem proving was done inthe ThmTa proof tool.While all of the above work are on erned with oating-point representationand arithmeti , there is no report in the open literature on any ma hine- he ked12

formalization of properties of �xed-point arithmeti . Therefore, the formalizationpresented in this thesis is to our best knowledge, the �rst of its kind. Our formal-ization of the �xed-point arithmeti has been inspired mostly by the work done byHarrison [29℄ and Carreno [11℄ on oating-point. Harrison's work was more orientedtowards veri� ation purposes. Indeed, we used an analogous set of lemmas to hiswork, to he k the validity of operation results and to arry out the error analysisof the quantized �xed-point result. For ex eption handling whi h is not overed byHarrison [29℄, we followed Carreno [11℄ who formalized oating-point ex eptions andtheir handling in more details.1.6.3 Error Analysis of Digital FiltersWork on the analysis of the errors due to the �nite pre ision e�e ts in the realizationof the digital �lters has always existed sin e their early days, however, using theo-reti al paper-and-pen il proofs and simulation te hniques. For digital �lters realizedwith the �xed-point arithmeti , error problems have been studied extensively. Forinstan e, Knowles and Edwards [44℄ proposed a method for analysis of the �niteword length e�e ts in �xed-point digital �lters. Gold and Radar [25℄ arried out adetailed analysis of the roundo� error for the �rst-order and se ond-order �xed-point�lters. Ja kson [37℄ analyzed the roundo� noise for the as ade and parallel realiza-tions of the �xed-point digital �lters. While the roundo� noise for the �xed-pointarithmeti enters into the system additively, it is a multipli ative omponent in the ase of the oating-point arithmeti . This problem is analyzed �rst by Sandberg[66℄, who dis ussed the roundo� error a umulation and input quantization e�e tsin the dire t realization of the �lter ex ited by a deterministi input. He also de-rived a bound on the time average of the squared error at the output. Liu andKaneko [47℄ presented a general approa h to the error analysis problem of digital�lters using the oating-point arithmeti and al ulated the error at the output dueto the roundo� a umulation and input quantization. Expressions are derived for13

the mean square error for ea h of the three anoni al forms of realization: dire t, as ade, and parallel. Upper bounds that are useful for a spe ial lass of the �ltersare given. Oppenheim and Weinstein [60℄ dis ussed in some details the e�e ts of the�nite register length on implementations of the linear re ursive di�eren e equationdigital �lters, and the fast Fourier transform (FFT) algorithm. Comparisons of theroundo� noise in the digital �lters using the di�erent types of arithmeti s have alsobeen reported in [71℄.In order to validate the error analysis, most of the above work ompare thetheoreti al results with orresponding experimental simulations. In this thesis, weshow how the above error analysis an be me hani ally performed using the HOLtheorem prover, providing a superior approa h to validation by simulation. Ourfo us will be on the pro ess of translating the hand proofs into equivalent proofsin HOL. The analysis we propose is mostly inspired by the work done by Liu andKaneko [47℄, who de�ned a general approa h to the error analysis problem of digital�lters using the oating-point arithmeti . Following a similar approa h, we have ex-tended this theoreti al analysis for �xed-point digital �lters. In both ases, a goodagreement between the HOL formalized and the theoreti al results are obtained.Through our work, we on�rmed and strengthened the main results of the pre-viously published theoreti al error analysis, though we un overed some minor errorsin the hand proofs and lo ated a few subtle orners that are overlooked informally.For example, in the theoreti al �xed-point error analysis it is always assumed thatthe �xed-point addition auses no error and only the roundo� error in the �xed-point multipli ation is analyzed [60℄. This is under the assumption that there isno over ow in the result and also the input operands have the same attributes asthe output. Using a me hani al theorem prover, we provide a more general erroranalysis in whi h we over the roundo� errors in both the �xed-point addition andmultipli ation operations. On top of that, for the oating-point error analysis, wehave used the formalization in HOL of the IEEE-754 [28℄, a standard whi h has not14

yet been established at the time of the above mentioned theoreti al error analysis.This enabled us to over a more omplete set of rounding and over ow modes anddegenerate ases whi h are not dis ussed in earlier theoreti al work.1.6.4 Error Analysis of FFT AlgorithmsAnalysis of errors in FFT realizations due to �nite pre ision e�e ts has traditionallyrelied on paper-and-pen il proofs and simulation te hniques. The roundo� error inusing the FFT algorithms depends on the algorithm, the type of arithmeti , theword length, and the radix. For FFT algorithms realized with �xed-point arith-meti , the error problems have been studied extensively. For instan e, Wel h [73℄presented an analysis of the �xed-point a ura y of the radix-2 de imation-in-timeFFT algorithm. Tran-Thong and Liu [68℄ presented a general approa h to the erroranalysis of the various versions of the FFT algorithm when �xed-point arithmeti isused. While the roundo� noise for �xed-point arithmeti enters into the system addi-tively, it is a multipli ative omponent in the ase of oating-point arithmeti . Thisproblem is analyzed �rst by Gentleman and Sande [22℄, who presented an upperbound on the mean-squared error for oating-point de imation-in-frequen y FFTalgorithm. Weinstein [72℄ presented a statisti al model for roundo� errors of the oating-point FFT. Kaneko and Liu [41℄ presented a detailed analysis of roundo�error in the FFT de imation-in-frequen y algorithm using oating-point arithmeti .This analysis is later extended by the same authors to the FFT de imation-in-timealgorithm [48℄. Oppenheim and Weinstein [60℄ dis ussed in some detail the e�e tsof �nite register length on implementations of digital �lters, and FFT algorithms.In order to validate the error analysis, most of the above work ompare thetheoreti al results with experimental simulation. In this thesis, we show how theabove error analyses for the FFT algorithms an be me hani ally performed usingthe HOL theorem prover, providing a superior approa h to validation by simulation.Our fo us will be on the pro ess of translating the hand proofs into equivalent proofs15

in HOL. The analysis we develop is mainly inspired by the work done by Kanekoand Liu [41℄, who proposed a general approa h to the error analysis problem of thede imation-in-frequen y FFT algorithm using oating-point arithmeti . Followinga similar idea, we have extended this theoreti al analysis for the de imation-in-timeand �xed-point FFT algorithms. In all ases, good agreements between formal andtheoreti al results were obtained.1.6.5 Formalization and Veri� ation of FFT AlgorithmsRelated work on the formalization and me hani al veri� ation of the FFT algorithmwas done by Gamboa [21℄ using the ACL2 theorem prover. The author formalizedthe FFT as a re ursive data-parallel algorithm, using the powerlist data stru ture.He also presented an ACL2 proof of the orre tness of the FFT algorithm, by trans-lating the hand proof taken from Misra's seminal paper on powerlists [55℄ into ame hani al proof in ACL2. In the same line, Capretta [10℄ presented the formaliza-tion of the FFT using the type theory proof tool Coq. To fa ilitate the de�nitionof the transform by stru tural re ursion, Capretta used the stru ture of polynomialtrees whi h is similar to the data stru ture of powerlists introdu ed by Misra. Fi-nally, he proved its orre tness and the orre tness of the inverse Fourier transform(IFT).Bjesse [5℄ des ribed the veri� ation of FFT hardware at the netlist level withan automati ombination of symboli simulation and theorem proving using theLava hardware development platform. He proved that the sequential pipelined im-plementation of the radix-4 de imation-in-time FFT is equivalent to the orrespond-ing ombinational ir uit. He also proved that the abstra t implementation of theradix-2 and the radix-4 FFT are equivalent for sizes that are an exponent of four.While [21℄ and [10℄ prove the orre tness of the high level FFT algorithm against theDFT, the veri� ation of [5℄ is performed at the netlist level. In ontrast, our work16

tries to lose this gap by formally spe ifying and verifying the FFT algorithm real-izations at di�erent levels of abstra tion based on di�erent data types. Besides, thede�nition used for the FFT in [21, 10℄ is based on the radix-2 de imation-in-timealgorithm. We over both de imation-in-time and de imation-in-frequen y algo-rithms, and radi es other than 2. The methodology we propose in this paper is, tothe best of our knowledge, the �rst proje t of its kind that overs the formal spe i-� ation and veri� ation of integrated FFT algorithms at di�erent abstra tion levelsstarting from real spe i� ation to oating- and �xed-point algorithmi des riptions,down to RT and netlist gate levels.1.7 Contributions of the ThesisIn light of the above related work review and dis ussions, we believe the ontributionsof the thesis an be spe i�ed as follows:1. Formalization in higher-order logi of �xed-point arithmeti . We en odedthe �xed-point number system and spe i�ed the di�erent quantization andover ow modes and ex eptions. An error analysis is then performed to he kthe orre tness of the quantized result of basi arithmeti operations.2. Me hani al analysis of �nite word length e�e ts in digital �lters using HOLtheorem prover. We derived expressions for the a umulation of roundo� errorin parametri Lth-order digital �lters, for ea h of the three anoni al forms ofrealization: dire t, parallel, and as ade. The HOL formalization and proofsare found to be in a good agreement with existing theoreti al paper-and-pen il ounterparts.3. Formal spe i� ation and veri� ation of fast Fourier transform (FFT) algo-rithms at di�erent abstra tion levels based on the HOL theorem prover. We17

derive expressions for the a umulation of roundo� error in FFT designs. Fi-nally, we use a lassi al hierar hi al proof approa h in HOL to prove that theFFT implementations at the register transfer and gate levels imply the or-responding high level �xed-point and oating-point algorithmi spe i� ationstaking into a ount the �nite pre ision e�e ts.1.8 Organization of the ThesisThe rest of the thesis is organized as follows: Chapter 2 des ribes the �xed-pointarithmeti and the details of its formalization in higher-order-logi . Chapter 3 de-s ribes the error analysis of digital �lters using HOL theorem proving. Chapter4 presents the veri� ation of FFT algorithms in HOL from real spe i� ation togate level implementation. Chapter 5 on ludes the thesis and outlines the futureresear h dire tions.

18

Chapter 2Formalization of Fixed-PointArithmeti in HOL2.1 Introdu tionUsually the onforman e of the �xed-point implementation with respe t to the oating-point spe i� ation is veri�ed by simulation te hniques whi h annot overthe entire input spa e yielded by the oating-point representation. The obje tive ofthis work is to formalize the �xed-point arithmeti in higher-order logi as a basisfor he king the orre tness of the implementation of DSP designs against higherlevel algorithmi des riptions in oating-point and �xed-point representations.Unlike oating-point arithmeti whi h is standardized in IEEE-754 [35℄ andIEEE-854 [36℄, urrent �xed-point arithmeti does not follow any parti ular standardand depends on the tool and the language used to design the DSP hip. For instan e,in SPW (Signal Pro essing Worksystem), a �xed-point number is de�ned as a binarystring and a set of attributes. Attributes spe ify how the binary string is interpretedusing three arguments for the total number of bits, the number of integer bits, andthe sign format. For arithmeti operations, it supports three kinds of ex eptionssu h as loss-of-sign or over ow, two over ow modes, and �ve quantization modes.19

In Matlab Simulink Fixed-Point Blo kset [50℄, �xed-point numbers are stored in datatypes that are hara terized by their word size (up to 128 bits), a radix point, andwhether they are signed or unsigned. The radix point is used to support integers,fra tionals, and generalized �xed-point data types. The Matlab Blo kset providesfour quantization modes orresponding to those supported by SPW. It also supportssaturation and wrapping to deal with over ow for all �xed-point data types. Anotherexample is the Synopsys CoCentri tool, whi h uses �xed-point as des ribed in theSystemC language [61℄. It supports signed and unsigned �xed-point data types,as well as limited pre ision (53 bits mantissa) �xed-point, alled fast �xed-point tospeed up simulation. SystemC supports seven quantization modes, of whi h four orrespond exa tly to the quantization modes of SPW. The other three modes arespe i� to SystemC and are not supported by the other tools. SystemC supports �veover ow modes overing those of SPW. With the obje tive of providing a generalmethodology for the formalization and veri� ation of �xed-point arithmeti usinghigher-order logi , we de�ne in this hapter a omplete ommon set of �xed-pointarithmeti as supported by most of the DSP tools, in parti ular SPW and SystemC.Based on higher-order logi , we propose to en ode a �xed-point number bya pair omposed of a Boolean word, and a triplet indi ating the word length, thelength of the integer portion, and the sign format. Then, we formalize the on eptsof valuation and quantization as fun tions that onvert respe tively a �xed-pointnumber to a real number and vi e versa, taking into a ount di�erent quantizationand over ow modes. Fixed-point arithmeti operations are formalized as fun tionsperforming operations on the real numbers orresponding to the �xed-point operandsand then applying the quantization on the real number result. Finally, we provevarious lemmas regarding the error analysis of the �xed-point quantization and orre tness of the basi operations like addition, multipli ation, and division. Thehigher-order logi formalization and proof were done using the HOL theorem prover[26℄. They were developed into a full �xed-point arithmeti library, whi h was20

re ently in luded in the last release of HOL (HOL4, Kananaskis-2).The rest of this hapter is organized as follows: Se tion 2.3 des ribes the �xed-point arithmeti de�nitions adopted in this thesis in luding the format of the �xed-point numbers, arithmeti operations, ex eptions dete tion and their handling, andthe di�erent over ow and quantization modes. Se tion 2.4 des ribes in detail theirformalization in HOL. In Se tion 2.5, we dis uss the veri� ation of basi �xed-pointarithmeti operations, su h as addition and multipli ation. Se tion 2.6 presents anillustrative example on how this formalization an be used through the modelingand veri� ation of an Integrator ir uit. Finally, Se tion 2.7 on ludes the hapter.2.2 HOL PreliminariesThe HOL theorem prover is a me hanizaed proof-assistant developed by Mike Gor-don at the University of Cambridge for ondu ting proofs in higher-order logi [26℄.It was expli itly designed for the formal veri� ation of hardware, though it has alsobeen applied to other areas in luding software veri� ation and formalization of puremathemati s.HOL is based on LCF approa h to intera tive theorem proving and has manyfeatures in ommon with LCF systems developed at Cambridge and Edinbergh [24℄.Like LCF, the HOL system supports se ure theorem proving by representating itslogi in the strongly-typed fun tional programming language ML [63℄. Propositionsand theorems of the logi are represented by ML abstra t data types, and intera tionwith the theorem prover takes pla e by exe uting ML pro edures that operate onvalues of these data types. In addition to the usual programming language expres-sions, ML has expressions that evaluate to terms, types, formulas, and theorems ofHOL's dedu tive apparatus. The HOL system supports a natural dedu tion styleof proof, with driven rules formed from eight primitive inferen e rules. All inferen erules are implemented using ML fun tions, and their appli ation is the only way to21

obtain theorems in the system. On e proved, theorems an be saved and used infuture proofs.There are four types of HOL terms: onstants, variables, fun tion appli ations,and lambda-terms (denoted fun tion abstra tions). Polymorphism, types ontainingtype variables, is a spe ial feature supported by this logi . Semanti ally, typesdenote sets and terms denote members of these sets. Formulas, sequen es, axioms,and theorems are represented by using terms of Boolean types. The main task of thehigher-order logi theorem prover is the derivation of proofs. A epting de�ned typesand fun tions of new types, give us the ability to prove properties of those types andfun tions. The sets of types, type operators, onstants and axioms available in HOLare organized in the form of theories. There are two main primitive theories, booland ind, for booleans and individuals (a primitive type to denote distin t elements),respe tively. Theorems an be derived based on these two main theories and addedto the system.HOL supports two styles of intera tive proof: forward proof and ba kwardproof. In the forward proof style, inferen e rules are simply applied in sequen e topreviously proved theorems until the desired theorem is obtained. The user spe i�eswhi h rule to be applied at ea h step of the proof, either intera tively or by writingan ML program that alls the appropriate sequen e of pro edures. Forward proofis not the easiest way of doing a proof, sin e the exa t details of a proof are rarelyknown in advan e. An important advan e in proving using HOL was made by RobinMilner in the early 1970s when he invented the notion of ta ti , introdu ing a newproof methodology alled the ba kward, or goal-dire ted, proof style. A ta ti isan ML fun tion that breaks goals down into in reasingly simple subgoals, until thesubgoals obtained an be proved dire tly from theorems already derived. Again,the user spe i�es whi h ta ti to use at ea h step. In addition to breaking a goaldown into subgoals, a ta ti also onstru ts a sequen e of forward inferen e stepswhi h an be used to prove the goal, on e the subgoals have themselves been proved.22

This is ne essary be ause all theorems in the system must ultimately be obtainedby forward proof. Table 2.2 summarizes some of the HOL symbols used in thisthesis and their meanings [26℄. The HOL type system does not support subtypes, sothe real numbers (R) have formally a di�erent type from the natural numbers (N).Therefore, the unary operator ampersand (&) is used to map between them. Thusthe real number numerals an be written as &0;&1, et [29℄.HOL Symbol Standard Symbol Meaning�x: t "x: t An x su h that t (x) holds�x: t �x: t Fun tion that maps x to t (x)& (none) Natural map operator (N ! R): t : t Not t: x � x Unary negation of xinv (x) x�1 Multipli ative inverse of xabs (x) j x j Absolute value of xx pow n xn Real x raised to natural number power nm EXP n mn Natural number m raised to exponent nTable 2.1: HOL Symbols2.3 Fixed-Point Arithmeti In this se tion we des ribe the �xed-point arithmeti de�nitions on whi h we baseour formalization. While we tried to keep these de�nitions as general as possible,the �xed-point numbers format, arithmeti operations, over ow and quantizationmodes, and ex eption handling adopted are to some extent in uen ed by the �xed-point arithmeti de�ned by Caden e SPW [12℄ and Synopsys SystemC [61℄.2.3.1 Fixed-Point NumbersA �xed-point number has a �xed number of binary digits and a �xed position forthe de imal point with respe t to that sequen e of digits. Fixed-point numbers an23

be either unsigned (always positive) or signed (in two's omplement representation).For example, onsider the ase of four bits being used to represent the �xed-pointnumbers. If the numbers are unsigned and if the de imal point or, more properly, thebinary point is �xed at the position after the se ond digit (XX.XX), the representablereal values range from 0:0 to 3:75. In two's omplement format, the most signi� antbit is the sign bit. The remaining bits spe ify the magnitude. If four bits representthe �xed-point numbers, and the binary point is �xed at the position after the se onddigit following the sign bit (SXX.X), the real values range from �4:0 to +3:5.Fixed-point numbers are expressed as a pair onsisting of a binary string and aset of attributes, (Binary String ;Attributes). The attributes spe ify how the binarystring is interpreted. Generally, the attributes are spe i�ed in the following format:(wl; iwl; sign) (2.1)whi h onsists of the following parameters:� wl: Total word length, spe ifying the total number of bits used to representthe �xed-point binary string, in luding integer bits, fra tional bits, and signbit, if any. Word length must be in the range of 1 to 256.� iwl: Integer word length, spe ifying the number of integer bits (the numberof bits to the left of the binary point, ex luding the sign bit, if any). If thisnumber is negative, repeated leading sign bits or zeros are added to generatethe equivalent binary value. If this number is greater than the total wordlength, trailing zeroes are added to generate the equivalent binary value.� sign: A letter spe ifying the sign format: \u" for unsigned, and \t" for two's omplement.Example: A ording to the above de�nitions, the real value �0:75 is representedby (111101; (6; 3; t)). If we onsider the same bit string with unsigned attributes24

(111101; (6; 3; u)), then the equivalent number is 111:101 or +7:625. On the otherhand, (111101; (6;�3; u)) represents the value :000111101 whi h is +0:119140625.2.3.2 Fixed-Point OperationsA DSP design tool usually provides a library in luding basi �xed-point signalpro essing blo ks su h as adders, multipliers, delay blo ks, and ve tor blo ks. Italso supports �xed-point hardware blo ks su h as multiplexers, bu�ers, inverters, ip- ops, bit manipulation and general-purpose ombinational logi blo ks. Theseblo ks a urately model the behavior of �xed-point digital signal pro essing sys-tems. In this thesis, we will fo us on the arithmeti and logi operations, but theidea an be generalized to the remaining operations. Operations performed on �xed-point data types are done using arbitrary and full pre ision. After the operationis omplete, the resulting operand is ast to �t the �xed-point data type obje t.The asting operation applies the quantization behavior of the target obje t to thenew value and assigns the new value to the target obje t. Then, the appropriateover ow behavior is applied to the result of the pro ess whi h gives the �nal value.In addition to the parameters orresponding to the input operands and output re-sult, the arithmeti operations take spe i� parameters de�ning the over ow andquantization (loss of pre ision) modes. These parameters are as follows:� q mode: Quantization mode. This parameter determines the behavior of the�xed-point operations when the result generates more pre ision in the leastsigni� ant bits (LSB) than is available.� o mode: Over ow mode. This parameter determines the behavior of the�xed-point operations when the result generates more pre ision in the mostsigni� ant bits (MSB) than is available.� n bits: Number of saturated bits. This parameter is only used for over owmode and spe i�es how many bits will be saturated if a saturation behavior25

is spe i�ed and an over ow o urs.Example: Consider a blo k that serves as a primitive �xed-point multiplier, whi htrun ates the results when loss of pre ision o urs and wraps the result when over- ow o urs. We an make a all to the multiplier routine through the fun tionfxpMul (Wrap j Trun ate; In1 ; In2 ;Out), in whi h In1 and In2 are the input �xed-point operands, Out is a parameter orresponding to the output attributes, andWrap and Trun ate indi ate the over ow and quantization modes, respe tively.Fixed-Point Ex eption HandlingFixed-point arithmeti operations that do not ompute and return an exa t resultresort to an ex eption-handling pro edure. This pro edure is ontrolled by theex eption ags. There are three kinds of ex eptions that an be tested [12℄:� Loss of Sign: The result was negative but the result storage area was un-signed. Zero is stored.� Over ow: The result was too big to be represented in the result storage area.The over ow mode determines the returned value.� Invalid: No result an be meaningfully represented (e.g., divide by zero). Thiserror an also o ur if the �xed-point number itself is invalid.Fixed-Point Quantization ModesQuantization e�e ts are used to determine what happens to the LSBs of a �xed-pointtype when more bits of pre ision are required than are available. The quantizationmodes are listed in Table 2.2.Figure 2.1 shows the behavior of ea h quantization mode. The X axis isthe result of the previous arithmeti operation and the Y axis is the value after26

Quantization Mode NameQuantization to Plus In�nity RNDQuantization to Zero RND ZEROQuantization to Minus In�nity RND MIN INFQuantization to In�nity RND INFConvergent Quantization RND CONVTrun ation TRNTrun ation to Zero TRN ZEROTable 2.2: Fixed-Point Quantization Modesquantization. The diagonal line represents the ideal number representation givenin�nite bits. The small horizontal lines show the e�e t of the quantization. Anyvalue of the X axis within the range of the line will be onverted to the value ofthe Y axis. The symbol q in the �gure refers to the quantization step, that is,the resolution of the data type. Ea h non integer value on the X axis is lo atedin a quantization interval surrounded by two su essive integer multiples of q asits losest representable quantized numbers, one greater and one smaller than theoriginal value. If the value is exa tly in the middle of the quantization interval, thenthe two losest representable numbers are equally distan ed apart from the originalvalue. As shown in this �gure modes RND, RND ZERO, RND MIN INF, RND INF,and RND CONV will quantize a value to the losest representable number if thetwo nearest representable numbers are not equally distan ed apart from the originalvalue. Otherwise, quantization towards plus in�nity, to zero, towards minus in�nity,towards plus in�nity if positive or minus in�nity if negative, and towards nearesteven will be performed, respe tively (Figure 2.1 (a-e)). The TRN mode is the defaultfor �xed-point types and will be used if no other value is spe i�ed. The result isalways quantized towards minus in�nity (Figure 2.1 (f)). In other words, the resultvalue is the �rst representable number lower than the original value. Finally, forTRN ZERO the result is the nearest representable value to zero (Figure 2.1 (g))27

[61℄.

q

q2q3qY

d) RND_INFc) RND_MIN_INFb) RND_ZEROa) RND

X3q2qq

q2q3qY

X3q2q

2q

e) RND_CONV f) TRN g) TRN_ZERO

X3q2qq

q2q3qY

X3q2qq

q2q3qY

X3q

q

q2q3qY

X3q2qq

q2q3qY

X3q2qq

Y3q

q2q

Figure 2.1: The behavior of �xed-point quantization modesFixed-Point Over ow ModesIn addition to quantization modes, we an use over ow modes to approximate ahigher range for �xed-point operations. Usually, over ow o urs when the result ofan operation is too large or too small for the available bit range. Spe i� over owmodes an then be implemented to redu e the loss of data. Over ow modes arespe i�ed by the o mode and n bits parameters, and are listed in Table 2.3.Over ow Mode NameSaturation SATSaturation to Zero SAT ZEROSymmetri al Saturation SAT SYMWrap-Around WRAPSign Magnitude Wrap-Around WRAP SMTable 2.3: Fixed-Point Over ow Modes28

Figure 2.2 shows the behavior of ea h over ow mode for a 3 bit �xed-pointdata type. The diagonal line represents the ideal value if in�nite bits are available forrepresentation. The dots represent the values of the result. The X axis is the originalvalue and the Y axis is the result. From this �gure, it an be seen thatMAX = 3 andMIN = �4 for a 3 bit �xed-point data type. The SAT mode will onvert the spe i�edvalue to MAX for an over ow or MIN for an under ow ondition (Figure 2.2 (a)).The SAT ZERO mode will set the result to 0 for any input value that is outside therepresentable range of the �xed-point type. If the result value is greater than MAXor smaller than MIN, the result will be 0 (Figure 2.2 (b)). In the SAT SYM mode,positive over ow will generate MAX and negative over ow will generate �MAX forsigned numbers or MIN for unsigned numbers (Figure 2.2 ( )). With the WRAPmode, the value of an arithmeti operand will wrap around from MAX to MIN asMAX is rea hed. There are two di�erent ases within this mode. The �rst is withthe n bits parameter set to 0 or having a default value of 0. All bits ex ept for thedeleted bits are opied to the result number (Figure 2.2 (d)). The se ond is whenthe n bits parameter is a nonzero value. In this ase the spe i�ed number of mostsigni� ant bits of the result number are saturated with preservation of the originalsign, the other bits are simply opied. Positive numbers remain positive and negativenumbers remain negative. A graph showing this behavior with n bits = 1 is given inFigure 2.2 (e). Note that positive numbers wrap around to 0 while negative valueswrap around to �1. The WRAP SM over ow mode uses sign magnitude wrapping.This over ow mode behaves in two di�erent styles depending on the value of then bits parameter. When n bits is 0, no bits are saturated. This mode will �rst deleteany MSB bits that are outside the result word length. The sign bit of the result isset to the value of the least signi� ant deleted bit. If the most signi� ant remainingbit is di�erent from the original MSB, then all the remaining bits are inverted. If theMSBs are the same, the other bits are opied from the original value to the resultvalue. A graph showing the result of this over ow mode is provided in Figure 2.229

(f). As the value of X in reases, the value of Y in reases to MAX and then slowlystarts to de rease until MIN is rea hed. The result is a sawtooth like waveform.With n bits greater than 0, n bits MSB bits are saturated to 1. A graph showingthis behavior with n bits = 1 is given in Figure 2.2 (g). Note that while the graphlooks somewhat like a sawtooth waveform, positive numbers do not dip below 0 andnegative numbers do not ross �1 [61℄.

1 2 3 4 5 6 7 8 9 X

Y54321

1 2

1

4321

1 2 3 4 5 6 8 9 X

Y5432

3

4321

1 2 3 4 5 6 X

a) SAT b) SAT_ZERO c) SAT_SYM

d) WRAP, n_bits = 0 e) WRAP, n_bits = 1

f) WRAP_SM, n_bits = 0 g) WRAP_SM, n_bits = 1

5

4 5 6 X

Y54321

1 2 3 4 5 6 X

Y

5

X

Y54321

1 2 3 4 5 6 7 X98

Y

9

Y54321

1 2 3 4 5 6 7

Figure 2.2: The behavior of �xed-point over ow modes2.4 Formalizing Fixed-Point Arithmeti in HOLIn this se tion, we present formalization of the �xed-point arithmeti in higher-orderlogi , based on the general purpose HOL theorem prover. The HOL system supportsboth forward and ba kward proofs. The forward proof style applies inferen e rules30

to existing theorems to obtain new theorems and eventually the desired theorem.Ba kward or goal oriented proofs start with the goal to be proven. Ta ti s areapplied to the goal and subgoals until the goal is de omposed into simpler existingtheorems or axioms. The system basi language in ludes the natural numbers andBoolean type. It also in ludes other spe i� extensions like reals library [27℄, whi hwas proved to be essential for our �xed-point arithmeti formalization.2.4.1 Fixed-Point Numbers RepresentationThe a tual �xed-point numbers are represented in HOL by a pair of elements repre-senting the binary string and the set of attributes. The extra tors for the two �eldsof a �xed-point number are de�ned as follows:`def string (s,a) = s`def attrib (s,a) = aThe binary string is treated as a Boolean word (type: bool word). For ex-ample, the bit string 1010 is represented by WORD [T;F;T;F℄. In this way, we usethe de�nitions and theorems already available in the HOL word library [70℄ to fa ilitatethe manipulation of binary words. The attributes are represented by a triplet of naturalnumbers for the total number of bits, the integer bits and the sign format.In HOL, we de�ne fun tions to extra t the primitive parameters for arbitrary at-tributes.`def wordlength (w,iw,s) = w`def intbits (w,iw,s) = iw`def sign (w,iw,s) = sWe also de�ne predi ates partitioning the �xed-point numbers into signed and un-signed numbers.`def is_signed X = (sign X = 1)`def is_unsigned X = (sign X = 0) 31

The number of digits on the right hand side of the binary point of a �xed-pointnumber is de�ned as fra bits. It an be derived as the di�eren e between the total numberof bits and the number of integer bits, onsidering the sign bit in the ase of signednumbers.`def fra bits X =if (is_unsigned X) then (wordlength X � intbits X)else (wordlength X � intbits X � 1)Two useful derived predi ates test the validity of a set of attributes and a �xed-point number based on the de�nition in Se tion 2.3.1. In a valid set of attributes, thewordlength should be in the range of 1 and 256, the sign an be either 0 or 1, and thenumber of integer bits is less than or equal to the wordlength. A valid �xed-point numbermust have a valid set of attributes and the length of its binary string must be equal to thewordlength.`def validAttr X =wordlength X > 0 ^ wordlength X < 257 ^intbits X < wordlength X + 1 ^ sign X < 2`def is_valid a =validAttr (attrib a) ^ (WORDLEN (string a) = wordlength (attrib a))where WORDLEN is a prede�ned fun tion of the HOL word library, whi h returns thesize of a word.2.4.2 Fixed-Point TypeNow we de�ne the a tual HOL type for the �xed-point numbers. The type is de�nedto be in bije tion with the appropriate subset of (bool word � N3), with the bije tionswritten in HOL as fxp : (bool word� N3)! fxp, and defxp : fxp! (bool word�N3). Thebije tion maps the set of all elements of type (bool word � N3) to the set of valid �xed-point numbers spe i�ed by the fun tion is valid as de�ned in the previous se tion. For32

this purpose, we make use of built-in fa ilities in HOL for de�ning new bije tion types[69℄. A similar te hnique was used in [29℄ for de�ning type bije tions for the oating-pointnumbers ( oat,de oat) in HOL.fxp_tybij =` (8a. fxp (defxp a) = a) ^ (8r. is_valid r = (defxp (fxp r) = r))We spe ialize the previous fun tions and predi ates to the fxp type, as follows:`def String a = string (defxp a)`def Attrib a = attrib (defxp a)`def Wordlength a = wordlength (Attrib a)`def Intbits a = intbits (Attrib a)`def Fra bits a = fra bits (Attrib a)`def Sign a = sign (Attrib a)`def Issigned a = is_signed (Attrib a)`def Isunsigned a = is_unsigned (Attrib a)`def Isvalid a = is_valid (defxp a)Note that we start the name of the fun tions manipulating �xed-point numbers by apital letters to distinguish them from those taking pairs and triplets as argument.2.4.3 Fixed-Point ValuationNow we spe ify the real number valuation of �xed-point numbers. We use two separateformulas for signed and unsigned numbers:� Unsigned: (1=2M ) � (N�1Xn=0 2n � vn) (2.2)� Signed: (1=2M ) � [N�1Xn=0 2n � vn � 2N � vN�1℄ (2.3)33

where vn represents the nth bit of the binary string in the �xed-point number1, and Mand N are respe tively fra bits and wordlength. In HOL, we de�ne the valuation fun tionvalue that returns the orresponding real value of a �xed-point number.`def value a =if (Isunsigned a) then &(BNVAL (String a)) / 2 pow Fra bits aelse (&(BNVAL (String a)) � &((2 EXP Wordlength a) *BV (MSB (String a)))) / 2 pow Fra bits awhere BNVAL is a fun tion whi h returns the numeri value of a Boolean word, BV is afun tion for mapping between a single bit and a number, and MSB is a onstant for themost signi� ant bit of a word, available in the HOL word library.We also de�ne the real value of the smallest (MIN ) and largest (MAX ) representablenumbers for a given set of attributes. The maximum is de�ned for both signed andunsigned numbers using the following formula:MAX = 2a � 2�b (2.4)where a is the intbits and b the fra bits. The minimum value for unsigned numbers is zeroand for signed numbers is omputed using the following formula:MIN = � 2a (2.5)Thereafter, we obtain the orresponding fun tions in HOL.`def MAX X = 2 pow intbits X � inv (2 pow fra bits X)`def MIN X = if (is_unsigned X) then 0 else :(2 pow intbits X)The onstants for the smallest (bottomfxp) and largest (topfxp) representable �xed-point numbers for a given set of attributes an be de�ned as follows:1We adopt the onvention that bits are indexed from the right hand side.34

`def topfxp X =if (is_unsigned X) then fxp (WORD (REPLICATE (wordlength X) T),X)else fxp (WCAT (WORD [F℄,WORD (REPLICATE (wordlength X � 1) T)),X)`def bottomfxp X =if (is_unsigned X) then fxp (WORD (REPLICATE (wordlength X) F),X)else fxp (WCAT (WORD [T℄,WORD (REPLICATE (wordlength X � 1) F)),X)where WCAT denotes the on atenation of two words, and REPLICATE makes a list onsisting of a value repli ated a spe i�ed number of times, whi h are prede�ned fun tionsin HOL.2.4.4 Ex eption HandlingOperations on �xed-point numbers an signal ex eptions as des ribed in Se tion 2.3.2.These are de lared as a new HOL data type.`def Ex eption = no_ex ept j overflow j invalid j loss_signwhere no ex ept is reserved for the ase without ex eption.Five over ow modes are also represented via an enumerated type de�nition.`def overflow_mode = SAT j SAT_ZERO j SAT_SYM j WRAP j WRAP_SMA ording to the de�nition of over ow modes in Se tion 2.3.2 for Saturation, ifthe number is greater than MAX or less than MIN, we return topfxp and bottomfxp, asthe losest representable values to the right result, respe tively. For Saturation to Zeroover ow, we will return zero in any ase. For Symmetri al Saturation, if the number isgreater than MAX, we return topfxp. If the number is less than MIN, we return the two's omplement of the maximum value, de�ned by the fun tion minustopfxp for signed, andbottomfxp for unsigned numbers, respe tively. For Wrap-around and Sign magnitude, wemust �rst onvert the real number to a binary format. Then we dis ard the extra bitsa ording to the output attributes, and saturate the required bits based on the parameter35

n bits. The details are de�ned as fun tionsWRAP AROUND andWRAP AROUND SM.Therefore, we de�ne the �xed-point over ow fun tion in HOL as follows:`def fxp_overflow X o_mode n_bits x =if (x > MAX X) thenif (o_mode = SAT) then topfxp Xelse if (o_mode = SAT_ZERO) thenfxp (WORD (REPLICATE (wordlength X) F),X)else if (o_mode = SAT_SYM) then topfxp Xelse if (o_mode = WRAP) thenWRAP_AROUND X n_bits xelse WRAP_AROUND_SM X n_bits xelse if (x < MIN X) thenif (o_mode = SAT) then bottomfxp Xelse if (o_mode = SAT_ZERO) thenfxp (WORD (REPLICATE (wordlength X) F),X)else if (o_mode = SAT_SYM) thenif (is_unsigned X) then bottomfxp Xelse minustopfxp Xelse if (o_mode = WRAP) thenWRAP_AROUND X n_bits xelse WRAP_AROUND_SM X n_bits xelse Nullwhere Null is a onstant that represents the result of an invalid operation, de�ned as:`def Null = �a. : (Isvalid a)Note that if the number is in the representable range of the given attributes, i.e. itsvalue is neither greater than MAX nor less than MIN, then the over ow is meaninglessand Null will be returned as the result.2.4.5 QuantizationFixed-point quantization takes an in�nitely pre ise real number and onverts it into a�xed-point number. Seven quantization modes are spe i�ed in Se tion 2.3.2, whi h we36

formalize using the following data type.`def quantization_mode =RND j RND_ZERO j RND_MIN_INF j RND_INF j RND_CONV j TRN j TRN_ZEROThen we de�ne the �xed-point quantization operation by a fun tion, whi h is de�ned ase by ase on the quantization modes as follows:`def fxp_quantize X q_mode x =if (q_mode = RND) then losest value (� a. value a � x)fa j (Isvalid a) ^ (Attrib a = X)g xelse if (q_mode = RND_ZERO) then losest value (� a. abs (value a) � abs x)fa j (Isvalid a) ^ (Attrib a = X)g xelse if (q_mode = RND_MIN_INF) then losest value (� a. value a � x)fa j (Isvalid a) ^ (Attrib a = X)g xelse if (q_mode = RND_INF) then losest value(� a. (if 0 � x then value a � x else value a � x))fa j (Isvalid a) ^ (Attrib a = X)g xelse if (q_mode = RND_CONV) then losest value (� a. LSB (String a) = F)fa j (Isvalid a) ^ (Attrib a = X)g xelse if (q_mode = TRN) then losest value (� a. T)fa j (Isvalid a) ^ (Attrib a = X) ^ (value a � x)g xelse losest value (� a. T)fa j (Isvalid a) ^ (Attrib a = X) ^(abs (value a) � abs x)g xThe �xed-point quantization fun tion takes as arguments a real number, a quantiza-tion mode, and an output attributes, and returns the orresponding �xed-point number.Similar to the oating-point ase [29℄, its de�nition is based on the following predi ate37

meaning that a is an element of the set s that provides a best approximation to x, assum-ing a valuation fun tion v :`def is_ losest v s x a =((a IN s) ^ 8b. (b IN s) =) (abs (v a � x) � abs (v b � x)))However, we still need to de�ne a fun tion that pi ks out a best approximation in ase there are more than one losest number, based on a given property like even. This an be done in HOL as follows:`def losest v p s x =�a. ((is_ losest v s x a) ^((9b. (is_ losest v s x b) ^ (p b)) =) (p a)))Finally, we de�ne the a tual �xed-point rounding fun tion for an arbitrary outputattributes.`def fxp_round X o_mode q_mode n_bits x =if (x > MAX X _ x < MIN X) then((fxp_overflow X o_mode n_bits x),overflow)else ((fxp_quantize X q_mode x),no_ex ept)where fxp over ow is the �xed-point over ow fun tion as de�ned in the previous se tionand supports all over ow modes, and fxp quantize is the �xed-point quantization fun tionthat supports all quantization modes. The �xed-point rounding fun tion takes as argumenta real number, an output attributes, the quantization and over ow modes, and the numberof saturated bits. It returns a �xed-point number and an ex eption ag. The fun tion�rst he ks for over ow, and in ase of over ow returns the result based on the over owmode, and sets the ex eption ag to over ow. Otherwise, it performs the quantizationbased on the quantization mode, and sets the ex eption ag to no ex ept.2.4.6 Fixed-Point Arithmeti OperationsFixed-point arithmeti operations su h as addition or multipli ation take two �xed-pointinput operands and store the result into a third. The attributes of the inputs and output38

need not mat h one another. Both unsigned and two's omplement inputs and outputare allowed. The result is formatted into the output as spe i�ed by the output attributesand by the over ow and loss of pre ision mode parameters. In our formalization, we �rstdeal with ex eptional ases su h as invalid operation and loss of sign. If any of the inputnumbers is invalid, then the result is Null and the ex eption ag invalid is raised. Ifthe result is negative but the output is unsigned then zero is returned and the ex eption ag loss sign is raised. Also in the ase of division by zero, the output value is for edto zero and the invalid ag is raised. Otherwise, we take the real value of the inputarguments, perform the operation as in�nite pre ision, then quantize the result a ordingto the desired quantization and over ow modes. Formally, the operations for addition,subtra tion, multipli ation, and division are de�ned as follows:`def fxpAdd X o_mode q_mode n_bits a b =if :(Isvalid a ^ Isvalid b) then (Null,invalid)else if (value a + value b < 0 ^ is_unsigned X) then(fxp (WORD (REPLICATE (wordlength X) F),X),loss_sign)else fxp_round X o_mode q_mode n_bits (value a + value b)`def fxpSub X o_mode q_mode n_bits a b =if :(Isvalid a ^ Isvalid b) then (Null,invalid)else if (value a � value b < 0 ^ is_unsigned X) then(fxp (WORD (REPLICATE (wordlength X) F),X),loss_sign)else fxp_round X o_mode q_mode n_bits (value a � value b)`def fxpMul X o_mode q_mode n_bits a b =if :(Isvalid a ^ Isvalid b) then (Null,invalid)else if (value a * value b < 0 ^ is_unsigned X) then(fxp (WORD (REPLICATE (wordlength X) F),X),loss_sign)else fxp_round X o_mode q_mode n_bits (value a * value b)39

`def fxpDiv X o_mode q_mode n_bits a b =if :(Isvalid a ^ Isvalid b) then (Null,invalid)else if (value b = 0) then(fxp (WORD (REPLICATE (wordlength X) F),X),invalid)else if (value a / value b < 0 ^ is_unsigned X) then(fxp (WORD (REPLICATE (wordlength X) F),X),loss_sign)else fxp_round X o_mode q_mode n_bits (value a / value b)2.5 Veri� ation of Fixed-Point OperationsA ording to the dis ussion in Se tion 2.4.3, ea h �xed-point number has a orrespondingreal number value. The orre tness of a �xed-point operation an be spe i�ed by ompar-ing its output with the true mathemati al result, using the valuation fun tion value that onverts a �xed-point to an in�nitely pre ise number. For example, the orre tness of a�xed-point adder fxpAdd is spe i�ed by omparing it with its ideal ounterpart +. Thatis, for ea h pair of �xed-point numbers (a,b), we ompare value (a)+ value (b) and value(fxpAdd (a,b)). In other words, we he k if the diagram in Figure 2.3 ommutes.value

fxpAdd (a,b)

value

value (a) + value (b)~~ ?

value (fxpAdd (a,b))

+

a , bfxpAdd

value (a) , value (b)

Figure 2.3: Corre tness riteria for �xed-point additionFor this purpose we de�ne the error resulting from quantizing a real number to a�xed-point value as follows:`def fxperror X o_mode q_mode n_bits x =value (FST (fxp_round X o_mode q_mode n_bits x)) � xand then establish the orre tness theorems for all four �xed-point arithmeti operations.40

Theorem 1: FXP_ADD_THM` (Isvalid a) ^ (Isvalid b) ^ validAttr (X) =)(Isvalid (FST (fxpAdd (X) o_mode q_mode n_bits a b))) ^(value (FST (fxpAdd (X) o_mode q_mode n_bits a b)) =value (a) + value (b) +(fxperror (X) o_mode q_mode n_bits (value (a) + value (b))))Theorem 2: FXP_SUB_THM` (Isvalid a) ^ (Isvalid b) ^ validAttr (X) =)(Isvalid (FST (fxpSub X o_mode q_mode n_bits a b))) ^(value (FST (fxpSub X o_mode q_mode n_bits a b)) =value (a) � value (b) +(fxperror X o_mode q_mode n_bits (value a � value b)))Theorem 3: FXP_MUL_THM` (Isvalid a) ^ (Isvalid b) ^ validAttr (X) =)(Isvalid (FST (fxpMul X o_mode q_mode n_bits a b))) ^(value (FST (fxpMul X o_mode q_mode n_bits a b)) =(value a * value b) +(fxperror X o_mode q_mode n_bits (value a * value b)))Theorem 4: FXP_DIV_THM` (Isvalid a) ^ (Isvalid b) ^ validAttr (X) =)(Isvalid (FST (fxpDiv X o_mode q_mode n_bits a b))) ^(value (FST (fxpDiv X o_mode q_mode n_bits a b)) =(value a / value b) +(fxperror X o_mode q_mode n_bits (value a / value b)))The theorems are omposed of two parts. The �rst part is about the validity of the�xed-point arithmeti operation output and states that if the input �xed-point numbersand the output attributes are valid then the result of the �xed-point operation is valid.The se ond part of the theorem relates the result of the �xed-point arithmeti operationsto the real result based on the orresponding error fun tion. To prove these main the-orems, a number of lemmas have been established. We �rst proved lemmas on erning41

the approximation of a real number with a �xed-point number. We proved that in a �-nite non-empty set of �xed-point numbers, we an �nd the best approximation to a realnumber based on a given valuation fun tion (Lemma 1 ).Lemma 1: FXP_IS_CLOSEST_EXISTS` FINITE (s) =) :(s = EMPTY) =) 9 (a: fxp). is_ losest v s x aThen, we proved that the hosen best approximation to a real number satisfying aproperty p from a �nite and non-empty set of �xed-point numbers is unique (Lemma 2 ),and is itself a member of the set (Lemma 3 ), and is itself the best approximation of thereal number (Lemma 4 ).Lemma 2: FXP_CLOSEST_IS_EVERYTHING` FINITE (s) =) :(s = EMPTY) =)is_ losest v s x ( losest v p s x) ^((9b. is_ losest v s x b ^ p b) =) p ( losest v p s x))Lemma 3: FXP_CLOSEST_IN_SET` FINITE (s) =) :(s = EMPTY) =) ( losest v p s x) IN sLemma 4: FXP_CLOSEST_IS_CLOSEST` FINITE (s) =) :(s = EMPTY) =) is_ losest v s x ( losest v p s x)Finally, we proved that the hosen best approximation to a real number satisfyinga property p from the set of all valid �xed-point numbers with a given attributes is itselfa valid �xed-point number (Lemma 5 ).Lemma 5: IS_VALID_CLOSEST` (validAttr X) =)Isvalid ( losest v p fa j Isvalid a ^ ((Attrib a) = X)g x)Besides, we proved that the set of all valid �xed-point numbers with a given at-tributes is �nite (Lemma 6 ).Lemma 6: FINITE_VALID_ATTRIB` FINITE fa j Isvalid a ^ (Attrib a = X)g42

The proof of this lemma is a bit ompli ated. For this purpose we made use of somebuilt-in theorems about �nite sets in the HOL pred sets library [51℄. Among these arethe two fundamental theorems FINITE EMPTY and FINITE INSERT, whi h state thatthe empty set is indeed �nite and the insertion of an element to a �nite set onstru ts a�nite set. Other theorems state that the union of two �nite sets (FINITE UNION ), theimage of a fun tion on a �nite set (IMAGE FINITE ), a singleton set2 (FINITE SING),the ross ombination of two �nite sets (FINITE CROSS ), and any subset of a �nite set(SUBSET FINITE ) is itself a �nite set. Using these theorems together with the de�nitionof a valid �xed-point number helped us to break down the proof of the �niteness of allvalid �xed-point numbers to the proof of �niteness of the set of all Boolean words with agiven word length (WORD FINITE ) and the set of all natural numbers less than a givenvalue (FINITE COUNT ). The last lemmas are proved by indu tion on the word lengthof the Boolean word and the maximum limit of the natural numbers, respe tively.We also proved that the set of all valid �xed-point numbers is nonempty (Lemma7 ).Lemma 7: IS_VALID_NONEMPTY` (validAttr X) =) :(fa j Isvalid a ^ (Attrib a = X)g = EMPTY)Finally, we proved that the result of quantizing a real number, whi h is in the rangerepresentable by a given valid attributes, is a valid �xed-point number (Lemma 8 ).Lemma 8: IS_VALID_QUANTIZATION` (validAttr X) =) Isvalid (FST (fxp_round X o_mode q_mode n_bits x))The validity of the quantization dire tly implies validity of the �xed-point operationoutput, and this ompletes the proof of the �rst parts of the theorems. The se ond parts ofthe theorems are proved using the properties of the real arithmeti in HOL and rewritingwith the de�nitions of the fxpAdd, fxpSub, fxpMul, fxpDiv, and fxperror fun tions.The se ond main theorem on �xed-point error analysis on erns bounding the quan-tization error. The error an be absolutely quanti�ed as follows:2a set that ontains pre isely one element. 43

Theorem 5: FXP_ERROR_BOUND_THM` (validAttr X) ^ :(x > MAX (X)) ^ : (x < MIN (X)) =)abs (fxperror X o_mode q_mode n_bits x) � inv (&2 pow fra bits X)A ording to this theorem, the error in quantizing a real number whi h is in the rangerepresentable by a given set of attributes X is less than the quantity 1 = 2fra bits (X). Thistheorem is valid for all �xed-point quantization modes. However, for RND, RND ZERO,RND MIN INF, RND INF, and RND CONV modes, whi h quantize to the nearest repre-sentable value, the error an be bounded to 1 = 2(fra bits (X)+1) by extending the theorem.To explain the theorem, we onsider the following fa t that relates the de�nition ofthe �xed-point numbers to the rationals.An N -bit binary word, when interpreted as an unsigned �xed-point number, antake on values from a subset P of the non-negative rationals given byP = fp=2b j 0 � p � 2N � 1; p 2 Zg (2.6)Similarly, for signed two's omplement representation, we haveP = fp=2b j �2N�1 � p � 2N�1 � 1; p 2 Zg (2.7)Note that P ontains 2N elements and b represents the fra tional bits in ea h ase.Based on this fa t, we an depi t the range of values overed for ea h ase as shownin Figure 2.4. MIN1=2b p=2b (2N � 1)=2b(2N � 2)=2b0 x a MAX

b) Signed�2N�1=2b (2N�1 � 1)=2bp=2b2=2b1=2b (2N�1 � 2)=2b0(�2N�1 + 1)=2b ax MAXMIN a) Unsigned2=2b

Figure 2.4: Fixed-point values on the real axis44

Thereafter, the representable range of �xed-point numbers is divided into 2N eq-uispa ed quantization steps with the distan e between two representable steps equal to1 = 2b. Suppose that x 2 R is approximated by a �xed-point number a. The position ofthese values are labeled in Figure 2.4. The error j x � a j is hen e less than the length ofone interval, or 1 = 2b, as mentioned in the se ond theorem.In HOL, we �rst proved that the quantization result is the nearest value to a realnumber and the orresponding error is minimum ompared to the other �xed-point num-bers (Lemma 9 ).Lemma 9: FXP_ERROR_AT_WORST_LEMMA` (validAttr X) ^ :(x > MAX (X)) ^ :(x < MIN (X)) ^(Isvalid a) ^ (Attrib a = X) =)abs (fxperror X o_mode q_mode n_bits x) � abs (value a � x)Then we proved that ea h representable real value x an be surrounded by tworepresentable rational numbers (Lemma 10 ).Lemma 10: FXP_ERROR_BOUND_LEMMA1` (validAttr X) ^ :(x > MAX (X)) ^ :(x < MIN (X)) =)9k. (k < 2 EXP wordlength X) ^ (&k / (&2 pow fra bits X) � x) ^(x < (&(SUC k) / (&2 pow fra bits (X))))Also we proved that the di�eren e between the real number and the surroundingrationals is less than 1 = 2fra bits (X) (Lemma 11 ).Lemma 11: FXP_ERROR_BOUND_LEMMA2` (validAttr X) ^ :(x > MAX (X)) ^ :(x < MIN (X)) =)9k. (k � 2 EXP wordlength X) ^abs (x � &k / (&2 pow (fra bits (X)))) � inv (&2 pow (fra bits (X)))Finally, we proved that for ea h real value we an �nd a �xed-point number withthe required error hara teristi s (Lemma 12 ).Lemma 12: FXP_ERROR_BOUND_LEMMA3` (validAttr X) ^ :(x > MAX (X)) ^ :(x < MIN (X)) =) 9(w: bool word).abs (value (fxp (w,X)) � x) � inv (&2 pow (fra bits X)) ^(WORDLEN w = wordlength X) 45

Sin e the quantization produ es the minimum error as stated in Lemma 9, the proofof the se ond main theorem (Theorem 5 ) is a dire t onsequen e of Lemma 12. In theseproofs, we have treated the ase of signed and unsigned numbers separately sin e they havedi�erent de�nitions for MAX, MIN, and value fun tions. For signed numbers a spe ialattention needs also to be paid to deal with negative numbers.2.6 Appli ation with SPWIn this se tion we demonstrate how to apply the formalization of �xed-point arithmeti presented in the previous se tions for the veri� ation of the transition from oating-pointto �xed-point algorithmi levels. We have hosen SPW as appli ation tool and the aseof an Integrator as an example ir uit. A digital integrator is a dis rete time systemthat transforms a sequen e of input numbers into another sequen e of output, by meansof a spe i� omputational algorithm. To des ribe the general fun tionality of a digitalintegrator, let fxtg, fwtg, and a denote the input sequen e, output sequen e, and onstant oeÆ ient of the integrator, respe tively. Then the integrator an be spe i�ed by thedi�eren e equation: wt = xt�1 + a wt�1 (2.8)Thereafter, the output sequen e at time t is equal to the input sequen e at time t - 1,added to the output at time t - 1 multiplied by the integrator oeÆ ient.Figure 2.5 shows the SPW design of an integrator. The integrator is �rst designedand simulated using the SPW prede�ned oating-point blo ks and parameters (Figure2.5 (a)). The design is omposed of an adder (M1), a multiplier by onstant (M2), anda delay (M3 ) blo k, together with signal sour e (M4 ) and sink (M5 ) elements. Theinput signal, the output signal, and the output of the adder and multiplier blo ks arelabeled by IN', OUT', S1', and S2', respe tively. Figure 2.5 (b) shows the onverted�xed-point design in whi h ea h blo k is repla ed with the orresponding �xed-point blo k(M1';M2';M3';M4';M5'). Fixed-point blo ks are shown by double ir les and squares todistinguish them from the oating-point blo ks. The attributes of all �xed-point blo koutputs are set to (64; 31; t) to ensure that over ow and quantization do not a�e t the46

SIGNAL

a’ = 0.997137

M3M4

SOURCE

SIGNAL

S2’

OUT’S1’IN’ M5M1

M2

-1SIGNALSINK

(64,31,t)

b) Fixed-Point Design

a) Floating-Point Design

(64,31,t)

(64,31,t)

a’’ = 0.997137

SOURCE SINK

SIGNAL

S2’’

OUT’’S1’’

M1’IN’’

M5’M3’M4’

M2’

-1 (64,31,t)

Figure 2.5: SPW design of an integratorsystem operation. The orresponding �xed-point signals are labeled by IN", OUT", S1",and S2".In HOL, we �rst model the design at ea h level as predi ates in higher-order logi .The predi ates orresponding to the oating-point design are as follows:`def Float_Gain_Blo k a0 b0 0 = (8t. 0 t = a0 t float_mul b0)`def Float_Delay_Blo k a0 b0 = (8t. b0 t = a0 (t � 1))`def Float_Add_Blo k a0 b0 0 = (8t. 0 t = a0 t float_add b0 t)`def Float_Integrator_Imp X a0 IN0 OUT0 =9 S10 S20.Float_Add_Blo k IN0 S20 S10 ^Float_Delay_Blo k S10 OUT0 ^Float_Gain_Blo k OUT0 a0 S20where X is the oating-point format. In these de�nitions, we have used available formal-ization of oating-point arithmeti in HOL [29℄. Floating-point data types are stored inSPW in the standard IEEE 64 bit double pre ision format.The HOL des ription of the �xed-point implementation is as follows:47

`def Fxp_Gain_Blo k a00 b00 00 = (8t. 00 t = a00 t fxp_mul b00)`def Fxp_Delay_Blo k a00 b00 = (8t. b00 t = a00 (t � 1))`def Fxp_Add_Blo k a00 b00 00 = (8t. 00 t = a00 t fxp_add b00 t)`def Fxp_Integrator_Imp X0 o_mode q_mode n_bits a00 IN00 OUT00 =9 S100 S200.Fxp_Add_Blo k IN00 S200 S100 ^Fxp_Delay_Blo k S100 OUT00 ^Fxp_Gain_Blo k OUT00 a00 S200where X' is the �xed-point format, and the fun tions fxp add and fxp mul are de�ned asfollows:`def a00 fxp_add b00 = FST (fxpAdd X0 o_mode q_mode n_bits a00 b00)`def a00 fxp_mul b00 = FST (fxpMul X0 o_mode q_mode n_bits a00 b00)In the next step, we des ribe ea h design as a di�eren e equation relating the inputand output samples a ording to the equation (4.4).`def FLOAT_Integrator_Spe X a0 IN0 OUT0 =8t. OUT0 t = (IN0 (t � 1) float_add (a0 float_mul OUT0 (t � 1)))`def FXP_Integrator_Spe X0 o_mode q_mode n_bits a00 IN00 OUT00 =8t. OUT00 t = (IN00 (t � 1) fxp_add (a00 fxp_mul OUT00 (t � 1)))The following lemmas ensure that the implementation at ea h level satis�es the orresponding spe i� ation.Lemma 13: FLOAT_INTEGRATOR_IMP_SPEC` Float_Integrator_Imp X a0 IN0 OUT0 =)Float_Integrator_Spe X a0 IN0 OUT0Lemma 14: FXP_INTEGRATOR_IMP_SPEC` Fxp_Integrator_Imp X0 o_mode q_mode n_bits a00 IN00 OUT00 =)Fxp_Integrator_Spe X0 o_mode q_mode n_bits a00 IN00 OUT0048

Now we assume that the oating-point and �xed-point input sequen es are therounded versions of an in�nite pre ision ideal input IN, and we have`def IN0 t = round X To_nearest (IN t)`def IN00 t = FST (fxp_round X0 o_mode q_mode n_bits (IN t))where round is the oating-point rounding fun tion, and To nearest is the orrespondingmode for rounding to nearest oating-point number [29℄. We also make some other as-sumptions on �niteness and validity of oating-point and �xed-point inputs, oeÆ ients,and intermediate results, in order to have �nite and valid �nal outputs. Using these as-sumptions and based on the theorems FXP ADD THM and FXP MUL THM (Se tion2.5) and the orresponding ones in oating-point theory [29℄, we prove the following the-orem on erning the error between the real values of the oating-point and �xed-pointpre ision integrator output samples.Theorem 6: INTEGRATOR_THM` Float_Integrator_Imp X a0 IN0 OUT0 ^Fxp_Integrator_Imp X0 o_mode q_mode n_bits a00 IN00 OUT00=)Val (OUT0 t) � value (OUT00 t) =Val a0 * Val (OUT0 (t � 1)) �value a00 * value (OUT00 (t � 1)) +error (IN (t � 1)) +error (Val a0 * Val (OUT0 (t � 1))) +error (Val (IN0 (t � 1)) + Val (a0 float_mul OUT0 (t � 1))) �fxperror X0 o_mode q_mode n_bits (IN (t � 1)) �fxperror X0 o_mode q_mode n_bits(value a00 * value (OUT00 (t � 1))) �fxperror X0 o_mode q_mode n_bits(value (IN00 (t � 1)) + value (a00 fxp_mul OUT00 (t � 1)))where Val is the oating-point valuation fun tion, and error is the oating-point roundingerror fun tion [29℄. A ording to Theorem 6, for a valid and �nite set of input and outputsequen es at time (t - 1) to the integrator design at the oating-point and �xed-point49

levels, we an have �nite and valid outputs at time t, and the di�eren e in the realvalues orresponding to these output samples an be expressed as the di�eren e in inputand output values multiplied by the orresponding oeÆ ients, taking into a ount thee�e ts of �nite pre ision in oeÆ ients and arithmeti operations. To �nd a onstantupper bound for the di�eren e between the outputs, we use Theorem 5 on the �xed-pointerror quanti� ation. Similarly, for the oating-point error bound analysis we proved thefollowing lemma:Lemma 15: ERROR_BOUND_NORM_STRONG_NORMALIZE` normalizes X x =)9 j. abs (error x) � (2 pow j / 2 pow (bias X + fra width X))where normalizes de�nes the riteria for an arbitrary real number to be in the range ofnormalized oating-point numbers, bias de�nes the exponent bias in the oating-pointformat whi h is a onstant used to make the exponent's range non-negative, and fra widthextra ts the fra tion width parameter from the oating-point format. A ording to Lemma15, if the absolute value of a real number is in the representable range of the normalized oating-point numbers with the format X and lo ated in the j 'th binade (the oating-point numbers between two adja ent powers of 2), then the absolute value of the error isless than or equal to 2j=2(bias X + fra width X). The lemma is proved based on the general oating-point absolute error bound theorem developed in [29℄.Finally, we proved the following theorem (Theorem 7 ) that bounds the output errorof the integrator design in the transition from the oating-point to �xed-point levels.Theorem 7: INTEGRATOR_FP_TO_FXP_ERROR_BOUND_THM` Float_Integrator_Imp X a0 IN0 OUT0 ^Fxp_Integrator_Imp X0 o_mode q_mode n_bits a00 IN00 OUT00=)9 j1 j2 j3.abs (Val (OUT0 t) � value (OUT00 t)) �2 * abs (a) * M +(2 pow j1 + 2 pow j2 + 2 pow j3) / 2 pow (bias X + fra width X) +3 / (2 pow (fra bits X0)) 50

In the proof of this theorem, we have assumed that the real values of the oating-point and �xed-point integrator oeÆ ients are equal (Val a' = value a"= a), hen e ignoring the e�e ts of ina ura ies in the integrator oeÆ ient. We have alsoassumed that the oating-point and �xed-point output values are bounded to a onstantvalue (M ). The parameters j1, j2, and j3 are related to the binades in whi h the realvalued arguments of the three oating-point error expressions in Theorem 6 are lo ated.2.7 Con lusionIn this hapter, we established the formalization of �xed-point arithmeti in the HOLtheorem prover. The formalization presented in this hapter an be onsidered as a om-plement to the oating-point formalizations whi h are widely available in the literature.Based on the proposed �xed-point formalization, in the next hapters we will fo us onthe veri� ation of the error analysis between the real numbers and the oating-point and�xed-point algorithmi levels for digital �lters and FFT algorithms. We also dis uss thetransitions from the oating-point and �xed-point algorithmi levels to hardware imple-mentations for FFT algorithms.

51

Chapter 3Error Analysis of Digital Filters inHOL3.1 Introdu tionDigital �lters are a parti ularly important lass of DSP (Digital Signal Pro essing) systems.A digital �lter is a dis rete time system that transforms a sequen e of input numbers intoanother sequen e of output, by means of a omputational algorithm [39℄. Digital �ltersare used in a wide variety of signal pro essing appli ations, su h as spe trum analysis,digital image and spee h pro essing, and pattern re ognition. Due to their well-knownadvantages, digital �lters are often repla ing lassi al analog �lters. The three distin tand most outstanding advantages of the digital �lters are their exibility, reliability, andmodularity. Ex ellent methods have been developed to design these �lters with desired hara teristi s. The design of a �lter is the pro ess of determination of a transfer fun tionfrom a set of spe i� ations given either in the frequen y domain, or in the time domain,or for some appli ations, in both. The design of a digital �lter starts from an ideal realspe i� ation. In a theoreti al analysis of the digital �lters, we generally assume that signalvalues and system oeÆ ients are represented in the real number system and are expressedto an in�nite pre ision. When implemented as a spe ial-purpose digital hardware or as a omputer algorithm, we must represent the signals and oeÆ ients in some digital number52

system that must always be of a �nite pre ision. Therefore, arithmeti operations must be arried out with an a ura y limited by this �nite word length. There is a variety of typesof arithmeti used in the implementation of digital systems. Among the most ommonare the oating-point and �xed-point. Here, all operands are represented by a spe ialformat or assigned a �xed word length and a �xed exponent, while the ontrol stru tureand the operations of the ideal program remain un hanged. The transformation from thereal to the oating-point and �xed-point forms is quite tedious and error-prone. On theimplementation side, the �xed-point model of the algorithm has to be transformed intothe best suited target des ription, either using a hardware des ription or a programminglanguage.AnalysisFXP Error

AnalysisFP Error

Valuation

Valuation

( HOL )FXP Real Value

AnalysisFP to FXP Error

FP Real Value( HOL )( HOL )

FP

REAL( HOL )

( HOL )FXP Figure 3.1: Error Analysis Approa hIn this hapter we des ribe the error analysis of digital �lters using the HOL theoremproving environment [23℄ based on the ommutating diagram shown in Figure 3.1. There-after, we �rst model the ideal real �lter spe i� ation and the orresponding oating-pointand �xed-point implementations as predi ates in higher-order logi . For this, we makeuse of existing theories in HOL on the onstru tion of real numbers [27℄, the formalizationof IEEE-754 standard based oating-point arithmeti [28, 29℄, and the formalization of�xed-point arithmeti des ribed in Chapter 2. We use valuation fun tions to �nd thereal values of the oating-point and �xed-point �lter outputs and de�ne the errors as thedi�eren es between these values and the orresponding output of the ideal real spe i� a-tion. Then we establish fundamental lemmas on the error analysis of the oating-pointand �xed-point roundings and arithmeti operations against their abstra t mathemati al53

ounterparts. Finally, we use these lemmas as a model to derive expressions for the a - umulation of the roundo� error in parametri Lth-order digital �lters, for ea h of thethree basi forms of realization: dire t, parallel, and as ade [59℄. Using these forms, ourveri� ation methodology an be s aled up to any larger-order �lter, either dire tly or byde omposing the design into a ombination of internal sub-blo ks. While the theoreti alwork on omputing the errors due to �nite pre ision e�e ts has been extensively studiedsin e the late sixties [47℄, it is for the �rst time in this thesis, that a formalization andproof of this analysis for digital �lters is done using a me hani al theorem prover, herethe HOL. Our results are found to be in a good agreement with the theoreti al ones.The rest of this hapter is organized as follows: Se tion 3.2 introdu es the funda-mental lemmas in HOL for the error analysis of the oating-point and �xed-point roundingand arithmeti operations. Se tion 3.3 des ribes the details of the error analysis in HOLof the lass of linear di�eren e equation digital �lters implemented in the three basi formsof realization. Finally, Se tion 3.4 on ludes the hapter.3.2 Error Analysis ModelsIn this se tion we introdu e the fundamental error analysis theorems [74, 20℄, and the orresponding lemmas in HOL for the oating-point [28, 29℄ and �xed-point arithmeti s.These theorems are then used in the next se tions as a model for the analysis of theroundo� error in digital �lters.3.2.1 Floating-Point Error ModelIn analyzing the e�e ts of oating-point roundo�, the e�e ts of rounding will be repre-sented multipli atively. The following theorem is the most fundamental in the oating-point rounding-error theory [74, 20℄.Theorem 1: If the real number x lo ated within the oating-point range, is rounded tothe losest oating-point number xR, then 54

xR = x(1 + Æ); where jÆj � 2�p (3.1)and p is the pre ision of the oating-point format.In HOL, we proved this theorem in the IEEE single pre ision oating-point formatfor the ase of rounding to nearest as follows:Lemma 1: FLOAT_ROUND_RELATIVE_ERROR` normalizes x =) 9 e. abs (e) < (1 / 2 pow ((fra width X) + 1)) ^(Val (float (round X To_nearest x)) = x * (1 + e))where the fun tion normalizes de�nes the riteria for an arbitrary real number to be inthe normalized range of oating-point numbers [28℄, fra width extra ts the fra tion widthparameter from the oating-point format X, Val is the oating-point valuation fun tion, oat is the bije tion fun tion that onverts a triple of natural numbers into the oating-point type, and round is the oating-point rounding fun tion [29℄.To prove this theorem [20℄, we �rst proved the following lemma whi h lo ates a realnumber in a binade (the oating-point numbers between two adja ent powers of 2):Lemma 2: REAL_IN_BINADE` normalizes x =) 9 j. j � ((emax X) � 2) ^(2 pow (j + 1) / 2 pow (bias X)) � abs x ^abs x < (2 pow (j + 2) / 2 pow (bias X))where the fun tion emax de�nes the maximum exponent in a given oating-point format,and bias de�nes the exponent bias in the oating-point format whi h is a onstant usedto make the exponent's range nonnegative. Using this lemma we an rewrite the general oating-point absolute error bound theorem (ERROR BOUND NORM STRONG) developed in [29℄as follows:Lemma 3: ERROR_BOUND_NORM_STRONG_NORMALIZE` normalizes x =)9 j. abs (error x) � (2 pow j / 2 pow (bias X + fra width X))whi h states that if the absolute value of a real number is in the representable range ofthe normalized oating-point numbers, then the absolute value of the error is less than55

or equal to 2j=2(bias X + fra width X). The fun tion error, de�nes the error resulting fromrounding a real number to a oating-point value whi h is de�ned as follows [29℄:`def error x = (Val (float (round X To_nearest x)) � x)Sin e (2(j+1) = 2(bias X)) � jxj for the real numbers in the normalized region as provedin Lemma 2, we have (jerror xj = jxj) � (2j = 2(bias X + fra width X)) =(2(j+1) = 2(bias X)) or (jerror xj = jxj) � (1 = 2((fra width X) + 1)). Finally, de�ninge = (error x = x) will omplete the proof of the oating-point relative error bound theo-rem as des ribed in Lemma 1.Next, we apply the oating-point relative rounding error analysis theorem (Theo-rem 1) to the veri� ation of the arithmeti operations. The goal is to prove the followingtheorem in whi h oating-point arithmeti operations su h as addition, subtra tion, mul-tipli ation, and division are related to their abstra t mathemati al ounterparts a ordingto the orresponding errors.Theorem 2: Let � denote any of the oating-point operations +, -, � , /. Thenfl (x � y) = (x � y)(1 + Æ); where jÆj � 2�p (3.2)and p is the pre ision of the oating-point format. The notation (.) is used to denotethat the operation is performed using the oating-point arithmeti .To prove this theorem in HOL, we start from the already proved lemmas on absoluteanalysis of rounding error in oating-point arithmeti operations (FLOAT ADD,FLOAT SUB,FLOAT MUL,FLOAT DIV) developed in [29℄. We have onverted these lemmasto the following relative error analysis version, using the relative error bound analysis of oating-point rounding (Lemma 1):56

Lemma 4: FLOAT_ADD_RELATIVE` Finite a ^ Finite b ^ normalizes (Val a + Val b)=) Finite (a + b) ^ 9 e. abs e � (1 / 2 pow ((fra width X) + 1)) ^(Val (a + b) = (Val a + Val b) * (1 + e))Lemma 5: FLOAT_SUB_RELATIVE` Finite a ^ Finite b ^ normalizes (Val a � Val b)=) Finite (a � b) ^ 9 e. abs e � (1 / 2 pow ((fra width X) + 1)) ^(Val (a � b) = (Val a � Val b) * (1 + e))Lemma 6: FLOAT_MUL_RELATIVE` Finite a ^ Finite b ^ normalizes (Val a * Val b)=) Finite (a * b) ^ 9 e. abs e � (1 / 2 pow ((fra width X) + 1)) ^(Val (a * b) = (Val a * Val b) * (1 + e))Lemma 7: FLOAT_DIV_RELATIVE` Finite a ^ Finite b ^ : Iszero b ^ normalizes (Val a / Val b)=) Finite (a / b) ^ 9 e. abs e � (1 / 2 pow ((fra width X) + 1)) ^(Val (a / b) = (Val a / Val b) * (1 + e))where the fun tion Finite de�nes the �niteness riteria for the oating-point numbers, andthe fun tion Iszero he ks if a given oating-point number is equal to zero. Note that weuse the onventional symbols for arithmeti operations on oating-point numbers usingthe operator overloading feature of HOL. The lemmas are omposed of two parts. The�rst part is about the �niteness of the oating-point operation output. It states that forea h pair of �nite oating-point numbers, if the real result is in the representable range ofnormalized oating-point numbers, then the output result is also �nite. For oating-pointdivision, the se ond operand should be nonzero to avoid the division by zero. The se ondpart of the lemmas states that the result of a oating-point operation is the exa t result,perturbed by a relative error of bounded magnitude.57

3.2.2 Fixed-Point Error ModelWhile the rounding error for the oating-point arithmeti enters into the system multi-pli atively, it is an additive omponent for the �xed-point arithmeti . In this ase thefundamental error analysis theorem an be stated as follows [74℄.Theorem 3: If the real number x lo ated in the range of the �xed-point numbers withformat X', is rounded to the losest �xed-point number x0R, thenx0R = x + �; where j�j � 2�fra bits (X0) (3.3)and fra bits is a fun tion that extra ts the number of bits that are to the right of thebinary point in the given �xed-point format.This theorem is proved in HOL as follows:Lemma 5: FXP_ROUND_ABSOLUTE_ERROR_BOUND` (validAttr X0) ^ (representable X0 x) =)abs (Fxp_error X0 x) � (1 / 2 pow (fra bits X0))where the fun tion validAttr de�nes the validity of the �xed-point format, representablede�nes the riteria for a real number to be in the representable range of the �xed-pointformat, and Fxp error de�nes the �xed-point rounding error.The veri� ation of the �xed-point arithmeti operations using the absolute erroranalysis of the �xed-point rounding (Theorem 3) an be stated as in the following theoremin whi h the �xed-point arithmeti operations are related to their abstra t mathemati al ounterparts a ording to the orresponding errors.Theorem 4: Let � denote any of the �xed-point operations +, -, � , /, with a givenformat X'. Thenfxp (x � y) = (x � y) + �; where j�j � 2�fra bits (X0) (3.4)and the notation fxp (.) is used to denote that the operation is performed using the�xed-point arithmeti . This theorem is proved in HOL using the following lemmas:58

Lemma 9: FXP_ADD_ABSOLUTE` (Isvalid a) ^ (Isvalid b) ^ validAttr (X0) ^representable X0 (value a + value b) =) (Isvalid (FxpAdd X0 a b)) ^9 e. abs e � inv (2 pow (fra bits X0)) ^value (FxpAdd X0 a b) = (value a + value b) + eLemma 10: FXP_SUB_ABSOLUTE` (Isvalid a) ^ (Isvalid b) ^ validAttr (X0) ^representable X0 (value a � value b) =) (Isvalid (FxpSub X0 a b)) ^9 e. abs e � inv (2 pow (fra bits X0)) ^value (FxpSub X0 a b) = (value a � value b) + eLemma 11: FXP_MUL_ABSOLUTE` (Isvalid a) ^ (Isvalid b) ^ validAttr (X0) ^representable X0 (value a * value b) =) (Isvalid (FxpMul X0 a b)) ^9 e. abs e � inv (2 pow (fra bits X0)) ^value (FxpMul X0 a b) = (value a * value b) + eLemma 12: FXP_DIV_ABSOLUTE` (Isvalid a) ^ (Isvalid b) ^ : (value b = 0) ^ validAttr (X0) ^representable X0 (value a / value b) =) (Isvalid (FxpDiv X0 a b)) ^9 e. abs e � inv (2 pow (fra bits X0)) ^value (FxpDiv X0 a b) = (value a / value b) + ewhere the fun tion Isvalid de�nes the validity of a �xed-point number, value is the �xed-point valuation fun tion, and FxpAdd, FxpSub, FxpMul, and FxpDiv are the orrespondingfun tions for �xed-point addition, subtra tion, multipli ation, and division operations,respe tively. A ording to these lemmas, if the input �xed-point numbers and the outputattributes are valid, then the result of �xed-point operations is valid. For �xed-pointdivision, the se ond operand should be nonzero to avoid the division by zero. The resultof the �xed-point operations is the exa t result, perturbed by an absolute error of boundedmagnitude.59

3.3 Error Analysis of Digital Filters using HOLIn this se tion, the prin ipal results for roundo� a umulation in digital �lters using the-orem proving are derived and summarized. We shall employ the models for oating- and�xed-point roundo� errors in HOL presented in the previous se tion. To illustrate ourapproa h, we �rst onsidered the ase of �rst- and se ond-order digital �lters. Then, weextended this analysis to the general ase of the dire t form realization of a parametri Lth-order �lter of whi h the �rst- and se ond-order �lters are spe ial ases. Finally, weapplied our approa h to the parallel and as ade forms. Using these forms, larger-order�lters an be treated as a ombination of �rst- and se ond-order �lters. Then, the totalerror is omputed by a umulating the error in all internal sub-�lters. In the following,we will �rst des ribe in details the theory behind the analysis and then explain how ea hstep of this analysis is performed in HOL.The lass of digital �lters onsidered in this paper is that of linear onstant oeÆ ient�lters spe i�ed by the di�eren e equation:wn = MXi=0 bi xn�i � LXi=1 ai wn�i (3.5)where fxng is the input sequen e and fwng is the output sequen e. L is the order ofthe �lter, and M an be any positive number less than L. There are three basi forms ofrealizing a digital �lter, namely the dire t, parallel, and as ade forms (Figure 3.2) [59℄.If the output sequen e is al ulated by using the equation (3.5), the digital �lter issaid to be realized in the dire t form. Figure 3.2 (a) illustrates the dire t form realizationof the �lter using the orresponding blo ks for the addition, multipli ation by a onstantoperations, and the delay element.The implementation of a digital �lter in the parallel form is shown in Figure 3.2 (b)in whi h the entire �lter is visualized as the parallel onne tion of the simpler �ltersHi of alower order. In this ase, K intermediate outputs fwing; i = 1,2,. . . ,K are �rst al ulatedand then summed to form the total output fwng. Therefore, for the input sequen e fxngwe have: 60

..............

....

xn Z�1Z�1Z�1 Z�1

Z�1Z�1

xnxn

wn

w1nw2nwKn wnwKn = wnw2nw1n

�a1�a2bMb2b1b0

H1H2HKHKH2H1

�aLa) Dire t formb) Parallel form ) Cas ade formFigure 3.2: Basi forms of digital �lter realizationswin = fixn + gixn�1 � iwin�1 � diwin�2 (3.6)where the parameters fi; gi; i; and di are obtained from the parameters ai and bi inequation (3.5) using the parallel expansion. The output of the entire �lter wn, is thenrelated to win by: wn = w1n + w2n + � � �+ wKn (3.7)The implementation of a digital �lter in the as ade form is shown in Figure 3.2( ) in whi h the �lter is visualized as a as ade of lower �lters. From the input fxng, the61

intermediate output fw1ng is �rst al ulated, and then this is the input to the se ond �lter.Continuing in this manner, the �nal output wKn = wn is al ulated. Sin e the output ofthe ith se tion (win) is the input of the (i+1)th se tion, the following equation holds:wi+1n = win + kiwin�1 + liwin�2 � iwi+1n�1 � diwi+1n�2 (3.8)where the parameters ki; li; i; and di are obtained from the parameters ai and bi inequation (3.5) using the serial expansion.There are three ommon sour es of errors asso iated with the �lter of the equation(3.5), namely [47℄:1. input quantization: aused by the quantization of the input signal fxng into aset of dis rete levels.2. oeÆ ient ina ura y: aused by the representation of the �lter oeÆ ients fakgand fbkg by a �nite word length.3. round-o� a umulation: aused by the a umulation of roundo� errors at arith-meti operations.Our on ern in this thesis is round-o� a umulation e�e t only. However, the results an be extended by minor modi� ation to onsider other sour es of error. Therefore, forthe digital �lter of the equation (3.5) the a tual omputed output referen e is in generaldi�erent from fwng. We denote the a tual oating-point and �xed-point outputs by fyngand fvng, respe tively. Then, we de�ne the orresponding errors at the nth output sampleas: en = yn � wn (3.9)e0n = vn � wn (3.10)e00n = vn � yn (3.11)where en and e0n are de�ned as the errors between the a tual oating-point and �xed-point implementations and the ideal real spe i� ation, respe tively. e00n is the error in thetransition from the oating-point to �xed-point levels.62

3.3.1 First-Order FilterTo illustrate our approa h for the analysis of roundo� errors with oating- and �xed-pointarithmeti , let us onsider a �rst-order �lter. Let xn, wn, and a denote the ideal real inputsignal, output response, and the oeÆ ient of the �lter, that is, the �lter parameters withno roundo� noise and x0n, yn, a0 and x00n, vn, a00 denote the orresponding a tual oating-point and �xed-point �lter parameters in the presen e of roundo� noise, respe tively. Thenwe an write: wn = awn�1 + xn (3.12)The orresponding omputed oating- and �xed-point outputs are:yn = fl [a0yn�1 + x0n℄ (3.13)vn = fxp [a00vn�1 + x00n℄ (3.14)The notations fl (:) and fxp (:) are used to denote that the operations are performedusing the oating- and �xed-point arithmeti s, respe tively. In HOL, we spe i�ed the�rst-order digital �lter in real, oating-, and �xed-point abstra tion levels, as predi atesin higher-order logi . The orresponding odes are as follows.`def First_Order_Filter_Ideal_Spe a x w =8n. w n = a * w (n � 1) + x n`def First_Order_Filter_Float_Imp a0 x0 y =8n. y n = a0 * y (n � 1) + x0 n`def First_Order_Filter_Fxp_Imp X a00 x00 v =8n. v n = (FxpAdd X (FxpMul X a00 (v (n � 1))) (x00 n))The al ulation of Equation (3.13) is to be performed in the following manner. Firstthe produ t a0yn�1 is al ulated separately. Then it is added to x0n to obtain yn. Similardis ussion an be applied for the al ulation of the �xed-point output vn a ording tothe Equation (3.14). Following Sandberg [66℄, a owgraph as given in Figure 3.3 may be63

drawn by using the fundamental error analysis theorems on oating-point and �xed-pointarithmeti operations introdu ed in Se tion 3.2 as given in Equations (3.2) and (3.4).Formally, a owgraph is a network of dire ted bran hes that onne t at nodes.Asso iated with ea h node is a variable or node value. Ea h bran h has an input signal andan output signal with a dire tion indi ated by an arrowhead on it. In a linear owgraph,the output of a bran h is a linear transformation of the input to the bran h. The simplestexamples are onstant multipliers and adders, i.e., when the output of the bran h is simplya multipli ation or an addition of the input to the bran h with a onstant value, whi hare the only lasses we onsider in this paper. The linear operation represented by thebran h is typi ally indi ated next to the arrowhead showing the dire tion of the bran h.For the ase of a onstant multiplier and adder, the onstant is simply shown next to thearrowhead. When an expli it indi ation of the bran h operation is omitted, this indi atesa bran h transmittan e of unity, or identity transformation. By de�nition, the valueat ea h node in a owgraph is the sum of the outputs of all the bran hes entering thenode. To omplete the de�nition of the owgraph notation, we de�ne two spe ial typesof nodes. (1) Sour e nodes that have no entering bran hes. They are used to representthe inje tion of the external inputs or signal sour es into a owgraph. (2) Sink nodes thathave only entering bran hes. They are used to extra t the outputs from a owgraph [59℄.Note that we have used one owgraph to represent both the oating-point and �xed-point ases, simultaneously. For oating-point errors, the bran h operations are interpreted as onstant multipli ations, while for �xed-point errors the bran h operations are interpretedas onstant additions.The quantities �n and �n are errors aused by roundo� at ea h oating-point arithmeti step. The orresponding error quantities for �xed-point roundo� are �0n and �0n.Therefore the a tual yn is seen to be given expli itly byyn = [ayn�1(1 + �n) + xn℄(1 + �n) (3.15)In HOL, we established the following lemma to ompute the real value of the oating-point output for the �rst-order �lter a ording to the Equation (3.15).64

1 1 + �n1 + �n�0n�0nxn

ynvna yn�1a vn�1

Figure 3.3: Error owgraph for the �rst-order �lterLemma 13: FIRST_ORDER_FILTER_FLOAT_OUTPUT_VALUE` First_Order_Filter_Float_Imp a0 x0 y =)9 e1 e2. abs e1 � (1 / 2 pow 24) ^ abs e2 � (1 / 2 pow 24) ^(Val (y n) = (Val (a0) * Val (y (n � 1)) * (1 + e1) +Val (x0 n)) * (1 + e2))Similarly, the a tual �xed-point output vn is given expli itly byvn = [avn�1 + �0n + xn℄ + �0n (3.16)and the orresponding lemma is established in HOL as follows:Lemma 14: FIRST_ORDER_FILTER_FXP_OUTPUT_VALUE` First_Order_Filter_Fxp_Imp X a00 x00 v =)9 e1 e2. abs e1 � inv (2 pow (fra bits X)) ^abs e2 � inv (2 pow (fra bits X)) ^value (FxpAdd X (FxpMul X a00 (v (n � 1))) (x00 n)) =((value (a00) * value (v (n � 1)) + e1) + value (x00 n)) + e2For error analysis, we need to al ulate the yn and vn sequen es from Equations(3.15) and (3.16), and ompare them with the ideal output sequen e wn spe i�ed by theEquation (3.12) to obtain the orresponding errors en, e0n, and e00n, a ording to the Equa-tions (3.9), (3.10), and (3.11). Therefore, the di�eren e equations for the errors betweendi�erent levels showing the a umulation of roundo� error are derived as follows:65

1) Floating-Point Error Analysis:en � a en�1 = a yn�1(�n + �n + �n�n) + xn�n (3.17)To prove this theorem in HOL, we �rst de�ned the error as the di�eren e betweenthe output of the real �lter spe i� ation, and the orresponding real value of the oating-point �lter implementation (Float Error). Then, we established the following lemma forthe a umulation of round-o� error in oating-point realization of the �rst-order �lter,a ording to the Equation (3.17).Lemma 15: FIRST_ORDER_FILTER_FLOAT_TO_REAL_THM` First_Order_Filter_Ideal_Spe a x w ^First_Order_Filter_Float_Imp a0 x0 y =)9 e1 e2. abs e1 � (1 / 2 pow 24) ^ abs e2 � (1 / 2 pow 24) ^(Float_Error n � a * Float_Error (n � 1) =a * Val (y (n � 1)) * (e1 + e2 + e1 * e2) + (x n) * e2)2) Fixed-Point Error Analysis:e0n � a e0n�1 = �0n + �0n (3.18)To prove this theorem in HOL, we �rst de�ned the error as the di�eren e betweenthe output of the real �lter spe i� ation, and the orresponding real value of the �xed-point �lter implementation (Fxp Error). Then, we established the following lemma for thea umulation of round-o� error in �xed-point realization of the �rst-order �lter, a ordingto the Equation (3.18).Lemma 16: FIRST_ORDER_FILTER_FXP_TO_REAL_THM` First_Order_Filter_Ideal_Spe a x w ^First_Order_Filter_Fxp_Imp X a00 x00 v =)9 e1 e2. abs e1 � inv (2 pow (fra bits X)) ^abs e2 � inv (2 pow (fra bits X)) ^(Fxp_Error n � a * Fxp_Error (n � 1) = e1 + e2)3) Floating- to Fixed-Point Error Analysis:e00n � a e00n�1 = �0n + �0n � a yn�1(�n + �n + �n�n)� xn�n (3.19)66

To prove this theorem in HOL, we �rst de�ned the error as the di�eren e between thereal value of the output of the �xed-point �lter implementation, and the orresponding realvalue of the oating-point �lter implementation (Float Fxp Error). Then, we establishedthe following lemma for the a umulation of round-o� error in transition from oating-point to �xed-point levels of the �rst-order �lter, a ording to the Equation (3.19).Lemma 17: FIRST_ORDER_FILTER_FXP_TO_FLOAT_THM` First_Order_Filter_Ideal_Spe a x w ^First_Order_Filter_Float_Imp a0 x0 y ^First_Order_Filter_Fxp_Imp X a00 x00 v =)9 e1 e2 e3 e4. abs e1 � inv (2 pow (fra bits X)) ^abs e2 � inv (2 pow (fra bits X)) ^abs e3 � (1 / 2 pow 24) ^ abs e4 � (1 / 2 pow 24) ^(Float_Fxp_Error n � a * Float_Fxp_Error (n � 1) =e1 + e2 � a * Val (y (n � 1)) * (e3 + e4 + e3 * e4) � (x n) * e4)We proved these lemmas using the fundamental error analysis lemmas (Lemmas4,5, and 6 for oating-point, and Lemmas 9,10, and 11 for �xed-point), based on the errormodels presented in Se tion 3.2.3.3.2 Se ond-Order FilterA se ond-order �lter is spe i�ed bywn = b0xn � (a1wn�1 + a2wn�2) (3.20)The orresponding omputed oating- and �xed-point outputs areyn = fl [b0xn � (a1yn�1 + a2yn�2)℄ (3.21)vn = fxp [b0xn � (a1vn�1 + a2vn�2)℄ (3.22)In HOL, we spe i�ed the se ond-order digital �lter in real, oating-, and �xed-pointabstra tion levels, as predi ates in higher-order logi . The orresponding odes are asfollows. 67

`def Se ond_Order_Filter_Ideal_Spe a b x w =8n. w n = (b 0 * x n � (a 1 * w (n � 1) + a 2 * w (n � 2)))`def Se ond_Order_Filter_Float_Imp a0 b0 x0 y =8n. y n = (b0 0 * x0 n � (a0 1 * y (n � 1) + a0 2 * y (n � 2)))`def Se ond_Order_Filter_Fxp_Imp X a00 b00 x00 v =8n. v n = (FxpSub X (FxpMul X (b00 0) (x00 n))(FxpAdd X (FxpMul X (a00 1) (v (n � 1)))(FxpMul X (a00 2) (v (n � 2)))))The al ulation of Equation (3.21) is performed in the following manner. First, theprodu ts a1yn�1, a2yn�2, and b0xn are al ulated separately. Then a1yn�1 and a2yn�2are added. Finally, this sum is subtra ted from b0xn to obtain yn. Similar dis ussion an be applied for the al ulation of the �xed-point output vn a ording to the Equation(3.22). A owgraph for the error of this ase is drawn in Figure 3.4. The quantities Æn;0,�n;1, �n;2, �n, �n are errors aused by oating-point roundo� at ea h arithmeti step. The orresponding error quantities for �xed-point roundo� are Æ0n;0, �0n;1, �0n;2, �0n, �0n.1 + Æn;0Æ0n;0 1 + �n;11 + �n;21 + �n�0n �0n;2b0xn �0n a1yn�1

a2yn�2�0n;11 + �n

ynvna1vn�1a2vn�2Figure 3.4: Error owgraph for the se ond-order �lterTherefore the a tual yn is seen to be given expli itly byyn = b0 �n;0 xn � 2Xk=1 ak �n;k yn�k (3.23)where 68

�n;0 = (1 + Æn;0)(1 + �n)�n;1 = (1 + �n;1)(1 + �n)(1 + �n)�n;2 = (1 + �n;2)(1 + �n)(1 + �n)In HOL, we established the following lemma to ompute the real value of the oating-point output for the se ond-order �lter a ording to the Equation (3.23).Lemma 18: SECOND_ORDER_FILTER_FLOAT_OUTPUT_VALUE` Se ond_Order_Filter_Float_Imp a0 b0 x0 y =)9 t f. Val (y n) = Val (b0 0) * (t 0) * Val (x0 n) �sum (1,2) (� i. Val (a0 i) * (f i) * Val (y (n � i))) ^9 e1 e2 e3 e4 e5. abs e1 � (1 / 2 pow 24) ^abs e2 � (1 / 2 pow 24) ^ abs e3 � (1 / 2 pow 24) ^abs e4 � (1 / 2 pow 24) ^ abs e5 � (1 / 2 pow 24) ^t 0 = (1 + e1) * (1 + e5) ^f 1 = (1 + e2) * (1 + e4) * (1 + e5) ^f 2 = (1 + e3) * (1 + e4) * (1 + e5)Similarly, the a tual �xed-point output vn is given expli itly byvn = [b0xn � (a1vn�1 + a2vn�2)℄ + Æ0n;0 + �0n;1 + �0n;2 + �0n + �0n (3.24)and the orresponding lemma is established in HOL as follows:Lemma 19: SECOND_ORDER_FILTER_FXP_OUTPUT_VALUE` Se ond_Order_Filter_Fxp_Imp X a00 b00 x00 v =)9 e1 e2 e3 e4 e5. abs e1 � inv (2 pow (fra bits X)) ^abs e2 � inv (2 pow (fra bits X)) ^abs e3 � inv (2 pow (fra bits X)) ^abs e4 � inv (2 pow (fra bits X)) ^abs e5 � inv (2 pow (fra bits X)) ^value (v n) = value (b00 0) * value (x00 n) �(value (a00 1) * value (v (n � 1)) +value (a00 2) * value (v (n � 2))) +e1 + e2 + e3 + e4 + e5 69

For error analysis, we need to al ulate the yn and vn sequen es from Equations(3.23) and (3.24), and ompare them with the ideal output sequen e wn spe i�ed by theEquation (3.20) to obtain the orresponding errors en, e0n, and e00n, a ording to the Equa-tions (3.9), (3.10), and (3.11). Therefore, the di�eren e equations for the errors betweendi�erent levels showing the a umulation of roundo� error are derived as follows:1) Floating-Point Error Analysis:en + a1 en�1 + a2 en�2 = b0xn(�n;0 � 1) � [a1yn�1 (�n;1 � 1) + (3.25)a2yn�2 (�n;2 � 1)℄To prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in oating-point realization of the se ond-order �lter, a ordingto the Equation (3.25).Lemma 20: SECOND_ORDER_FILTER_FLOAT_TO_REAL_THM` Se ond_Order_Filter_Ideal_Spe a b x w ^Se ond_Order_Filter_Float_Imp a0 b0 x0 y =)9 t f. Float_Error n + a 1 * Float_Error (n � 1) +a 2 * Float_Error (n � 2) = b 0 * x n * (t 0 � 1) �(a 1 * Val (y (n � 1)) * (f 1 � 1) +a 2 * Val (y (n � 2)) * (f 2 � 1)) ^9 e1 e2 e3 e4 e5. abs e1 � (1 / 2 pow 24) ^abs e2 � (1 / 2 pow 24) ^ abs e3 � (1 / 2 pow 24) ^abs e4 � (1 / 2 pow 24) ^ abs e5 � (1 / 2 pow 24) ^t 0 = (1 + e1) * (1 + e5) ^f 1 = (1 + e2) * (1 + e4) * (1 + e5) ^f 2 = (1 + e3) * (1 + e4) * (1 + e5)2) Fixed-Point Error Analysis:e0n + a1 e0n�1 + a2 e0n�2 = Æ0n;0 + �0n;1 + �0n;2 + �0n + �0n (3.26)70

To prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in �xed-point realization of the se ond-order �lter, a ording tothe Equation (3.26).Lemma 21: SECOND_ORDER_FILTER_FXP_TO_REAL_THM` Se ond_Order_Filter_Ideal_Spe a b x w ^Se ond_Order_Filter_Fxp_Imp X a00 b00 x00 v =)9 e1 e2 e3 e4 e5. abs e1 � (inv (&2 pow (fra bits X))) ^abs e2 � inv (2 pow (fra bits X)) ^abs e3 � inv (2 pow (fra bits X)) ^abs e4 � inv (2 pow (fra bits X)) ^abs e5 � inv (2 pow (fra bits X)) ^Fxp_Error n + a 1 * Fxp_Error (n � 1) + a 2 * Fxp_Error (n � 2) =e1 + e2 + e3 + e4 + e53) Floating- to Fixed-Point Error Analysis:e00n + a1 e00n�1 + a2 e00n�2 = Æ0n;0 + �0n;1 + �0n;2 + �0n + �0n � (3.27)b0xn(�n;0 � 1) + a1yn�1 (�n;1 � 1) + a2yn�2 (�n;2 � 1)To prove this theorem in HOL, we established the following lemma for the a u-mulation of round-o� error in transition from oating-point to �xed-point levels of these ond-order �lter, a ording to the Equation (3.27).

71

Lemma 22: SECOND_ORDER_FILTER_FXP_TO_FLOAT_THM` Se ond_Order_Filter_Ideal_Spe a b x w ^Se ond_Order_Filter_Float_Imp a0 b0 x0 y ^Se ond_Order_Filter_Fxp_Imp X a00 b00 x00 v =)9 t f e10 e20 e30 e40 e50.abs e1 � inv (2 pow (fra bits X)) ^abs e2 � inv (2 pow (fra bits X)) ^abs e3 � inv (2 pow (fra bits X)) ^abs e4 � inv (2 pow (fra bits X)) ^abs e5 � inv (2 pow (fra bits X)) ^(Float_Fxp_Error n + a 1 * Float_Fxp_Error (n � 1) +a 2 * Float_Fxp_Error (n � 2) =e10 + e20 + e30 + e40 + e50 � b 0 * x n * (t 0 � 1) +(a 1 * Val (y (n � 1)) * (f 1 � 1) +a 2 * Val (y (n � 2)) * (f 2 � 1))) ^9 e1 e2 e3 e4 e5. abs e1 � (1 / 2 pow 24) ^abs e2 � (1 / 2 pow 24) ^ abs e3 � (1 / 2 pow 24) ^abs e4 � (1 / 2 pow 24) ^ abs e5 � (1 / 2 pow 24) ^t 0 = (1 + e1) * (1 + e5) ^f 1 = (1 + e2) * (1 + e4) * (1 + e5) ^f 2 = (1 + e3) * (1 + e4) * (1 + e5)We proved these lemmas using the fundamental error analysis lemmas (Lemmas4,5, and 6 for oating-point, and Lemmas 9,10, and 11 for �xed-point), based on the errormodels presented in Se tion 3.2.3.3.3 Lth-Order Filter (Dire t Form)The dire t form realization of a parametri Lth-order �lter is spe i�ed by Equation (3.5).The orresponding omputed oating- and �xed-point outputs areyn = fl ( MXk=0 bk xn�k � LXk=1 ak yn�k) (3.28)and 72

vn = fxp ( MXk=0 bk xn�k � LXk=1 ak vn�k) (3.29)In HOL, we spe i�ed the dire t form realization of a parametri Lth-order digital�lter in real, oating-, and �xed-point abstra tion levels, as predi ates in higher-orderlogi . The real spe i� ation is de�ned in HOL using Equation (3.5). For this we usedthe expression sum (m;n) f denoting Pm+n�1i = m f(i), whi h is a fun tion available in theHOL real library [27℄ and de�nes the �nite summation on the real numbers. The oating-and �xed-point spe i� ations are de�ned in HOL a ording to the Equations (3.28) and(3.29). For these ases, we de�ned similar fun tions for �nite summation on the oating-point ( oat sum) and �xed-point (fxp sum) numbers, using re ursive de�nition in HOL.The orresponding odes in HOL are as follows.`def L_Order_Filter_Dire t_Form_Ideal_Spe a b x w M L =8n. w n = sum (0,SUC M) (� i. b i * x (n � i)) �sum (1,L) (� i. a i * w (n � i))`def 8 f n m. (float_sum (n,0) f = float (0,0,0)) ^(float_sum (n,SUC m) f = float_sum (n,m) f + f (n + m))`def L_Order_Filter_Dire t_Form_Float_Imp a0 b0 x0 y M L =8 n. y n = float_sum (0,SUC M) (� i. b0 i * x0 (n � i)) �float_sum (1,L) (� i. a0 i * y (n � i))`def 8 X f n m. (fxp_sum (n,0) X f =(fxp (WORD (REPLICATE (streamlength X) F),X))) ^(fxp_sum (n,SUC m) X f =FxpAdd X (fxp_sum (n,m) X f) f (n+m))`def L_Order_Filter_Dire t_Form_Fxp_Imp X a00 b00 v M L =8 n. v n = FxpSub X(fxp_sum (0,SUC M) X (� i. FxpMul X b00 i x00 (n � i)))(fxp_sum (1,L) X (� i. FxpMul X a i y (n � i)))73

The al ulation of Equation (3.28) is to be performed in the following manner. First,the output produ ts ak yn�k, k = 1; 2; :::; L are al ulated separately and then summed.Next, the same is done for the input produ ts bk xn�k, k = 0; 1; :::;M . Finally, theoutput summation is subtra ted from the input one to obtain the main oating-pointoutput yn. Similar dis ussion an be applied for the al ulation of the �xed-point outputvn a ording to the Equation (3.29). The orresponding owgraph showing the e�e t ofroundo� error using the fundamental error analysis theorems (Theorems 2 and 4) a ordingto the Equations (3.2) and (3.4), is given by Figure 3.5 whi h also indi ates the order of the al ulation. The quantities Æn;k, k = 0; 1; :::;M , �n;k, k = 1; 2; :::; L, �n;k, k = 1; 2; :::;M ,�n;k, k = 2; 3; :::; L, and �n are errors aused by oating-point roundo� at ea h arithmeti step. The orresponding error quantities for �xed-point roundo� are Æ0n;k, k = 0; 1; :::;M ,�0n;k, k = 1; 2; :::; L, � 0n;k, k = 1; 2; :::;M , �0n;k, k = 2; 3; :::; L, and �0n.1 + Æn;0Æ0n;0Æ0n;2

1 + Æn;M1 + Æn;21 + Æn;1b0xn�2b0xn�1

bMxn�M

b0xn

Æ0n;M �0n;L 1 + �n;L� 0n;M1 + �n;2Æ0n;1 1 + �n;1� 0n;2� 0n;1

1 + �n;M 1 + �n�0n1 + �n;3 �0n;3�0n;21 + �n;2 �0n;31 + �n;3�0n;2

1 + �n;L�0n;La2yn�2a1yn�1

aLyn�La3yn�31 + �n;2�0n;11 + �n;1

ynaLvn�La3vn�3a2vn�2a1vn�1

vnFigure 3.5: Error owgraph for Lth-order �lter (Dire t form)Therefore, the a tual oating-point output yn is seen to be given expli itly by:yn = MXk=0 bk �n;k xn�k � LXk=1 ak �n;k yn�k (3.30)74

where �n;0 = (1 + �n)(1 + Æn;0) MYi=1(1 + �n;i)�n;j = (1 + �n)(1 + Æn;j) MYi=j(1 + �n;i) j = 1; 2; :::;M�n;1 = (1 + �n)(1 + �n;1) LYi=2(1 + �n;i)�n;j = (1 + �n)(1 + �n;j) LYi=j(1 + �n;i) j = 2; 3; :::; LIn HOL, we �rst de�ned �nite multipli ation on the real numbers re ursively as theexpression mul (m;n) f denoting Qm+n�1i = m f(i) as follows:`def 8 f n m. (mul (n,0) f = 1) ^(mul (n,SUC m) f = mul (n,m) f * f (n + m))Then, we established the following lemma to ompute the real value of the oating-point �lter output for the dire t form of realization a ording to the Equations (3.30).Lemma 23: L_ORDER_FILTER_DIRECT_FORM_FLOAT_OUTPUT_VALUE` L_Order_Filter_Dire t_Form_Float_Imp a0 b0 x0 y M L =)9 t f. (Val (y n) = (if L = 0 thensum (0,SUC M) (� i. (Val (b0 i) * t i * Val (x0 (n � i))))else sum (0,SUC M) (� i. (Val (b0 i) * t i * Val (x0 (n � i)))) �sum (1,L) (� i. (Val (a0 i) * f i * Val (y (n � i)))))) ^9 k d p e z. abs k � (1 / 2 pow 24) ^(8 i. (i � M) =) (abs (d i) � (1 / 2 pow 24))) ^(8 i. (i � M) =) (abs (p i) � (1 / 2 pow 24))) ^(8 i. (i � L) =) (abs (e i) � (1 / 2 pow 24))) ^(8 i. (i � L) =) (abs (z i) � (1 / 2 pow 24))) ^(t 0 = (1 + k) * (1 + d 0) * (mul (1,M) (� i. (1 + p i)))) ^(8 j. (1 � j ^ j � M) =)(t j = (1 + k) * (1 + d j) *(mul (j,(M � (j � 1))) (� j. (1 + p j))))) ^(f 1 = (1 + k) * (1 + e 1) * (mul (2,(L � 1)) (� i. (1 + z i)))) ^(8 j. (2 � j ^ j � L) =)(f j = (1 + k) * (1 + e j) * (mul (j,(L � j + 1)) (� j. (1 + z j)))))75

Similarly, the a tual �xed-point output vn is given expli itly byvn = MXk=0 bk xn�k � LXk=1 ak vn�k + MXk=0 Æ0n;k + MXk=1 � 0n;k + LXk=1 �0n;k + LXk=2 �0n;k + �0n (3.31)and the orresponding lemma is established in HOL as follows:Lemma 24: L_ORDER_FILTER_DIRECT_FORM_FXP_OUTPUT_VALUE_EXPAND` L_Order_Filter_Dire t_Form_Fxp_Imp X a00 b00 x00 v M L =)9 k d p e z. abs k � (inv (2 pow (fra bits X))) ^(8 i. (i � M) =) abs (d i) � inv (2 pow (fra bits X))) ^(8 i. (i � M) =) abs (p i) � inv (2 pow (fra bits X))) ^(8 i. (i � L) =) abs (e i) � inv (2 pow (fra bits X))) ^(8 i. (i � L) =) abs (z i) � inv (2 pow (fra bits X))) ^(value (v n) = if (L = 0) thensum (0,SUC M) (� i. value (b00 i) * value (x00 (n � i))) +sum (0,SUC M) (� i. d i) + sum (1,M) (� j. p j) + k elsesum (0,SUC M) (� i. value (b00 i) * value (x00 (n � i))) �sum (1,L) (� i. value (a00 i) * value (v (n � i))) +sum (0,SUC M) (� i. d i) + sum (1,M) (� j. p j) +sum (1,L) (� i. e i) + sum (2,(L � 1)) (� j. z j) + k)For error analysis, we need to al ulate the yn and vn sequen es from Equations(3.30) and (3.31), and ompare them with the ideal output sequen e wn spe i�ed by theEquation (3.5) to obtain the orresponding errors en, e0n, and e00n, a ording to the Equa-tions (3.9), (3.10), and (3.11). Therefore, the di�eren e equations for the errors betweendi�erent levels showing the a umulation of roundo� error are derived as follows:1) Floating-Point Error Analysis:en + LXk=1 ak en�k = MXk=0 bk (�n;k � 1) xn�k � LXk=1 ak (�n;k � 1) yn�k (3.32)To prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in oating-point realization of the dire t form �lter, a ording tothe Equation (3.32). 76

Lemma 25: L_ORDER_FILTER_DIRECT_FORM_FLOAT_TO_REAL_THM` L_Order_Filter_Dire t_Form_Ideal_Spe a b x w M L ^L_Order_Filter_Dire t_Form_Float_Imp a0 b0 x0 y M L =)9 t f. (if L = 0 then (Float_Error n =sum (0,SUC M) (� i. Val (b0 i) * (t i � 1) * Val (x0 (n � i)))) else(((Float_Error n) + sum (1,L) (� i. a i * (Float_Error (n � i))) =sum (0,SUC M) (� i. Val (b0 i) * (t i � 1) * Val (x0 (n � i))) �sum (1,L) (� i. Val (a0 i) * (f i � 1) * Val (y (n � i)))))) ^9 k d p e z. (abs k � (1 / 2 pow 24)) ^(8 i. (i � M) =) (abs (d i) � (1 / 2 pow 24))) ^(8 i. (i � M) =) (abs (p i) � (1 / 2 pow 24))) ^(8 i. (i � L) =) (abs (e i) � (1 / 2 pow 24))) ^(8 i. (i � L) =) (abs (z i) � (1 / 2 pow 24))) ^(t 0 = (1 + k) * (1 + d 0) * (mul (1,M) (� i. (1 + p i)))) ^(8 j. (1 � j ^ j � M) =)(t j = (1 + k) * (1 + d j) *(mul (j,(M � (j � 1))) (� j. (1 + p j))))) ^(f 1 = (1 + k) * (1 + e 1) * (mul (2,(L � 1)) (� i. (1 + z i)))) ^(8 j. (2 � j ^ j � L) =)(f j = (1 + k) * (1 + e j) * (mul (j,(L � j + 1)) (� j. (1 + z j)))))2) Fixed-Point Error Analysis:e0n + LXk=1 ak e0n�k = MXk=0 Æ0n;k + MXk=1 � 0n;k + LXk=1 �0n;k + LXk=2 �0n;k + �0n (3.33)To prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in �xed-point realization of the dire t form �lter, a ording to theEquation (3.33).77

Lemma 26: L_ORDER_FILTER_DIRECT_FORM_FXP_TO_REAL_THM` L_Order_Filter_Dire t_Form_Ideal_Spe a b x w M L ^L_Order_Filter_Dire t_Form_Fxp_Imp X a00 b00 x00 v M L =)9 k d p e z. abs k � inv (2 pow (fra bits X)) ^(8 i. (i � M) =) abs (d i) � inv (2 pow (fra bits X))) ^(8 i. (i � M) =) abs (p i) � inv (2 pow (fra bits X))) ^(8 i. (i � L) =) abs (e i) � inv (2 pow (fra bits X))) ^(8 i. (i � L) =) abs (z i) � inv (2 pow (fra bits X))) ^(if (L = 0) then (Fxp_Error n =sum (0,SUC M) (� i. d i) + sum (1,M) (� j. p j) + k) else(Fxp_Error n + sum (1,L) (� i. a i * Fxp_Error (n � i)) =sum (0,SUC M) (� i. d i) + sum (1,M) (� j. p j) +sum (1,L) (� i. e i) + sum (2,(L � 1)) (� j. z j) + k))3) Floating- to Fixed-Point Error Analysis:e00n + LXk=1 ak e00n�k = MXk=0 Æ0n;k + MXk=1 � 0n;k + LXk=1 �0n;k + LXk=2 �0n;k + �0n � (3.34)MXk=0 bk (�n;k � 1) xn�k + LXk=1 ak (�n;k � 1) yn�kTo prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in transition from oating-point to �xed-point levels of the dire tform �lter, a ording to the Equation (3.34).

78

Lemma 27: L_ORDER_FILTER_DIRECT_FORM_FXP_TO_FLOAT_THM` L_Order_Filter_Dire t_Form_Ideal_Spe a b x w M L ^L_Order_Filter_Dire t_Form_Float_Imp a0 b0 x0 y M L =)9 t f k1 d1 p1 e1 z1.(abs k1 � inv (2 pow (fra bits X))) ^(8 i. (i � M) =) abs (d1 i) � inv (2 pow (fra bits X))) ^(8 i. (i � M) =) abs (p1 i) � inv (2 pow (fra bits X))) ^(8 i. (i � L) =) abs (e1 i) � inv (2 pow (fra bits X))) ^(8 i. (i � L) =) abs (z1 i) � inv (2 pow (fra bits X))) ^(if (L = 0) then(Float_Fxp_Error n = sum (0,SUC M) (� i. d1 i) +sum (1,M) (� j. p1 j) + k1 � (sum (0,SUC M)(� i. Val (b0 i) * (t i � 1) * Val (x0 (n � i))))) else(Float_Fxp_Error n + sum (1,L) (� i. a i * Float_Fxp_Error (n � i)) =sum (0,SUC M) (� i. d1 i) + sum (1,M) (� j. p1 j) +sum (1,L) (� i. e1 i) + sum (2,(L � 1)) (� j. z1 j) + k1 �sum (0, (SUC M)) (� i. Val (b0 i) * (t i � 1) * Val (x0 (n � i))) +sum (1,L) (� i. Val (a0 i) * (f i � 1) * Val (y (n � i))))) ^9 k2 d2 p2 e2 z2. abs k2 � (1 / 2 pow 24) ^(8 i. (i � M) =) abs (d2 i) � (1 / 2 pow 24)) ^(8 i. (i � M) =) abs (p2 i) � (1 / 2 pow 24)) ^(8 i. (i � L) =) abs (e2 i) � (1 / 2 pow 24)) ^(8 i. (i � L) =) abs (z2 i) � (1 / 2 pow 24)) ^(t 0 = (1 + k2) * (1 + d2 0) * (mul (1,M) (� i. (1 + p2 i)))) ^(8 j. (1 � j ^ j � M) =) (t j = (1 + k2) * (1 + d2 j) *(mul (j,(M � (j � 1))) (� j. (1 + p2 j))))) ^(f 1 = (1 + k2) * (1 + e2 1) * (mul (2,(L � 1)) (� i. (1 + z2 i)))) ^(8 j. (2 � j ^ j � L) =) (f j = (1 + k2) * (1 + e2 j) *(mul (j,(L � j + 1)) (� j. (1 + z2 j)))))We proved these lemmas using the fundamental error analysis lemmas (Lemmas4,5, and 6 for oating-point, and Lemmas 9,10, and 11 for �xed-point), based on the errormodels presented in Se tion 3.2. The lemmas are proved by indu tion on parameters Land M for the dire t form of realization. 79

3.3.4 Lth-Order Filter (Parallel Form)For parallel form of realization, the ith parallel path is des ribed by Equation (3.6). The orresponding omputed oating- and �xed-point outputs areyin = fl [fixn + gixn�1 � iyin�1 � diyin�2℄ (3.35)vin = fxp [fixn + gixn�1 � ivin�1 � divin�2℄ (3.36)The output of the entire parallel form �lter is des ribed by Equation (3.7). The orresponding omputed oating- and �xed-point outputs areyn = fl [y1n + y2n + :::+ yKn ℄ (3.37)vn = fxp [v1n + v2n + :::+ vKn ℄ (3.38)In HOL, we �rst spe i�ed the ith parallel path in real, oating-, and �xed-pointabstra tion levels, using Equations (3.6), (3.35), and (3.36). Then, we spe i�ed the en-tire output as de�ned in Equations (3.7), (3.37), and (3.38) using the �nite summationfun tions. The orresponding odes are as follows.`def Parallel_Form_Ideal_Spe d f g x ww w K =8 n. w n = sum (1, K) (� i. ww i n) ^8 i. ww i n = f i * x n + g i * x (n � 1) � i * ww i (n � 1) � d i * ww i (n � 2)`def Parallel_Form_Float_Imp 0 d0 f0 g0 x0 yy y K =8 n. y n = float_sum (1,K) (� i. yy i n) ^8 i. (i � 1 ^ i � K) =) yy i n =f0 i * x0 n + g0 i * x0 (n � 1) �( 0 i * yy i (n � 1) + d0 i * yy i (n � 2))80

`def Parallel_Form_Fxp_Imp X 00 d00 f00 g00 x00 vv v K =8 n. v n = fxp_sum (1,K) X (� i. vv i n) ^8 i. (i � 1 ^ i � K) =) vv i n = FxpSub X(FxpAdd X (FxpMul X (f00 i) (x00 n))(FxpMul X (g00 i) (x00 (n � 1))))(FxpAdd X (FxpMul X ( 00 i) (vv i (n � 1)))(FxpMul X (d00 i) (vv i (n � 2))))Figure 3.6 shows the error owgraph for the parallel form realization of a parametri Lth-order �lter.

11

1

1

1 + �in;1 1 + �in;1Æ0in;1 � 0in;1 �0in;2�0in;1fixngixn�1

iyin�1diyin�21 + Æin;2 1 + Æin;1 �0in;11 + �in;1

1 + �in�0inÆ0in;2 1 + �in;2 ivin�1divin�2yinyina) ith parallel path

yny1n � 0n;K� 0n;2b) Final parallel output1 + �n;2 1 + �n;K vnv1ny2n y3nv2n v3n yKnvKn

Figure 3.6: Error owgraph for Lth-order �lter (Parallel form)The orresponding error owgraph for the ith parallel path is shown in Figure 3.6(a). The a tual oating-point output sequen e for the ith parallel path is therefore givenby yin = fixn�in;1 + gixn�1�in;2 � iyin�1�in;1 � diyin�2�in;2 (3.39)where �in;1 = (1 + Æin;1)(1 + �in;1)(1 + �in)81

�in;2 = (1 + Æin;2)(1 + �in;1)(1 + �in)�in;1 = (1 + �in;1)(1 + �in;1)(1 + �in)�in;2 = (1 + �in;2)(1 + �in;1)(1 + �in)If the summation of (3.37) is arried out from the left to the right, a orresponding owgraph an be drawn as given in Figure 3.6 (b). Thusyn = KXi=1 n;iyin (3.40)where n;i = 8>>>>><>>>>>: KYj=2(1 + �n;j); i = 1KYj=i(1 + �n;j); i � 2In HOL, we established the following lemma to ompute the real value of the oating-point output of the parallel form of realization a ording to the Equations (3.39) and(3.40).Lemma 28: PARALLEL_FORM_FLOAT_OUTPUT_VALUE` Parallel_Form_Float_Imp 0 d0 f0 g0 x0 yy y K =)(9 t f. Val (yy i n) = Val (f0 i) * Val (x0 n) * (t 1) +Val (g0 i) * Val (x0 (n � 1)) * (t 2) �Val ( 0 i) * Val (yy i (n � 1)) * (f 1) �Val (d0 i) * Val (yy i (n � 2)) * (f 2)) ^9 k d1 d2 p e1 e2 z. abs (k i) � (1 / 2 pow 24) ^82

abs (d1 i) � (1 / 2 pow 24) ^abs (d2 i) � (1 / 2 pow 24) ^abs (p i) � (1 / 2 pow 24) ^abs (e1 i) � (1 / 2 pow 24) ^abs (e2 i) � (1 / 2 pow 24) ^abs (z i) � (1 / 2 pow 24) ^t 1 = (1 + d1 i) * (1 + p i) * (1 + k) ^t 2 = (1 + d2 i) * (1 + p i) * (1 + k) ^f 1 = (1 + e1 i) * (1 + z i) * (1 + k) ^f 2 = (1 + e2 i) * (1 + z i) * (1 + k) ^9 s. (Val (y n) = sum (1,K) (� i. s i * Val (yy i n)) ^9 k. s i = (if (i = 1) then (mul (2,(K � 1)) (� i. (1 + k i))) else(mul (i,(K � i + 1)) (� i. (1 + k i)))))Similarly, the a tual �xed-point outputs of the parallel form of realization vin, andvn are given expli itly byvin = fixn + gixn�1 � ivin�1 � divin�2 + Æ0in;1 + Æ0in;2 + � 0in;1 + �0in;1+ (3.41)�0in;2 + �0in;1 + �0inand vn = KXi=1 vin + KXi=2 � 0n;i (3.42)and the orresponding lemma is established in HOL as follows:Lemma 29: PARALLEL_FORM_FXP_OUTPUT_VALUE` Parallel_Form_Fxp_Imp X 00 d00 f00 g00 x00 vv v K =)9 k0 d10 d20 p0 e10 e20 z0.abs (k0 i) � inv (2 pow (fra bits X)) ^abs (d10 i) � inv (2 pow (fra bits X)) ^abs (d20 i) � inv (2 pow (fra bits X)) ^83

abs (p0 i) � inv (2 pow (fra bits X)) ^abs (e10 i) � inv (2 pow (fra bits X)) ^abs (e20 i) � inv (2 pow (fra bits X)) ^abs (z0 i) � inv (2 pow (fra bits X)) ^value (vv i n) = value (f00 i) * value (x00 n) +value (g00 i) * value (x00 (n � 1)) �value ( 00 i) * value (vv i (n � 1)) �value (d00 i) * value (vv (n � 2)) +d10 i + d20 i + p0 i + e10 i + e20 i + z0 i + k0 i ^9 s0. value (v n) = sum (1,K) (� i. value (vv i n)) +sum (2,(K � 1)) (� j. s0 j)For error analysis of the parallel form, we �rst de�ne the orresponding errors atthe ith parallel path output sample as:ein = yin � win (3.43)e0in = vin � win (3.44)e00in = vin � win (3.45)Then, we al ulate the yin, yn, vin, and vn sequen es from Equations (3.39), (3.40),(3.41), and (3.42), respe tively and ompare them with the ideal output sequen es win,and wn spe i�ed by the Equations (3.6), and (3.7) to obtain the orresponding errors ein,e0in, e00in , en, e0n, and e00n a ording to the Equations (3.43), (3.44), (3.45), (3.9), (3.10), and(3.11), respe tively. Therefore, the di�eren e equations for the errors between di�erentlevels showing the a umulation of roundo� error are derived as follows:1) Floating-Point Error Analysis:ein + iein�1 + diein�2 = fixn(�in;1 � 1) + gixn�1(�in;2 � 1)� iyin�1(�in;1 � 1) � (3.46)84

diyin�2(�in;2 � 1)and en � KXi=1 ein = KXi=1( n;i � 1) yin (3.47)To prove these theorems in HOL, we established the following lemma for the a u-mulation of round-o� error in oating-point realization of the parallel form �lter, a ordingto the Equations (3.46), and (3.47).Lemma 30: PARALLEL_FORM_FLOAT_TO_REAL_THM` Pararllel_Form_Ideal_Spe d f g x ww w K ^Parallel_Form_Float_Imp 0 d0 f0 g0 x0 yy y K =)9 t f. Float_Error i n + i * Float_Error i (n � 1) +d i * Float_Error i (n � 2) =f i * x n * (t 1 � 1) + g i * x (n � 1) * (t 2 � 1) � i * Val (y (n � 1)) * (f 1 � 1) �d i * Val (y (n � 2)) * (f 2 � 1) ^9 k d1 d2 p e1 e2 z. abs (k i) � (1 / 2 pow 24) ^abs (d1 i) � (1 / 2 pow 24) ^abs (d2 i) � (1 / 2 pow 24) ^abs (p i) � (1 / 2 pow 24) ^abs (e1 i) � (1 / 2 pow 24) ^abs (e2 i) � (1 / 2 pow 24) ^abs (z i) � (1 / 2 pow 24) ^t 1 = (1 + d1 i) * (1 + p i) * (1 + k) ^t 2 = (1 + d2 i) * (1 + p i) * (1 + k) ^f 1 = (1 + e1 i) * (1 + z i) * (1 + k) ^f 2 = (1 + e2 i) * (1 + z i) * (1 + k) ^9 s. (Float_Error n = sum (1,K) (� i. Float_Fxp_Error i n) +sum (1, K) (� i. (((s i) � 1) * Val (yy i n))) ^9 k. s i = if (i = 1) then mul (2, (K � 1)) (� i. (1 + (k i))) elsemul (i,(K � i + 1)) (� i. (1 + (k i))))2) Fixed-Point Error Analysis: 85

e0in + ie0in�1 + die0in�2 = Æ0in;1 + Æ0in;2 + � 0in;1 + �0in;1 + �0in;2 + �0in;1 + �0in (3.48)and e0n � KXi=1 e0in = KXi=1 � 0n;i (3.49)To prove these theorems in HOL, we established the following lemma for the a u-mulation of round-o� error in �xed-point realization of the parallel form �lter, a ordingto the Equations (3.48), and (3.49).Lemma 31: PARALLEL_FORM_FXP_TO_REAL_THM` Parallel_Form_Ideal_Spe d f g x ww w K ^Parallel_Form_Fxp_Imp X 00 d00 f00 g00 x00 vv v K =)9 k0 d10 d20 p0 e10 e20 z0.abs (k0 i) � inv (2 pow (fra bits X)) ^abs (d10 i) � inv (2 pow (fra bits X)) ^abs (d20 i) � inv (2 pow (fra bits X)) ^abs (p0 i) � inv (2 pow (fra bits X)) ^abs (e10 i) � inv (2 pow (fra bits X)) ^abs (e20 i) � inv (2 pow (fra bits X)) ^abs (z0 i) � inv (2 pow (fra bits X)) ^Fxp_Error i n + i * Fxp_Error i (n � 1) +d i * Fxp_Error i (n � 2) =d10 i + d20 i + p0 i + e10 i + e20 i + z0 i + k0 i ^9 s0. Fxp_Error n = sum (1,K) (� i. Fxp_Error i n) +sum (2,(K � 1)) (� j. s0 j)3) Floating- to Fixed-Point Error Analysis:e00in + ie00in�1 + die00in�2 = Æ0in;1 + Æ0in;2 + � 0in;1 + �0in;1 + �0in;2 + �0in;1 + �0in� (3.50)fixn(�in;1 � 1)� gixn�1(�in;2 � 1) + iyin�1(�in;1 � 1) + diyin�2(�in;2 � 1)86

and e00n � KXi=1 e00in = KXi=1 � 0n;i � KXi=1( n;i � 1) yin (3.51)To prove these theorems in HOL, we established the following lemma for the a u-mulation of round-o� error in transition from oating-point to �xed-point realizations ofthe parallel form �lter, a ording to the Equations (3.50), and (3.51).Lemma 32: PARALLEL_FORM_FXP_TO_FLOAT_THM` Parallel_Form_Ideal_Spe d f g x ww w K ^Parallel_Form_Float_Imp 0 d0 f0 g0 x0 yy y K ^Parallel_Form_Fxp_Imp X 00 d00 f00 g00 x00 vv v K =)9 t f k0 d10 d20 p0 e10 e20 z0.abs (k0 i) � inv (2 pow (fra bits X)) ^abs (d10 i) � inv (2 pow (fra bits X)) ^abs (d20 i) � inv (2 pow (fra bits X)) ^abs (p0 i) � inv (2 pow (fra bits X)) ^abs (e10 i) � inv (2 pow (fra bits X)) ^abs (e20 i) � inv (2 pow (fra bits X)) ^abs (z0 i) � inv (2 pow (fra bits X)) ^Float_Fxp_Error i n + i * Float_Fxp_Error i (n � 1) +d i * Float_Fxp_Error i (n � 2) =d10 i + d20 i + p0 i + e10 i + e20 i + z0 i + k0 i �(f i * x n * (t 1 � 1) +g i * (x (n � 1)) * (t 2 � 1) �( i * Val (y (n � 1)) * (f 1 � 1) +d i * Val (y (n � 2)) * (f 2 � 1))) ^9 k d1 d2 p e1 e2 z. abs (k i) � (1 / 2 pow 24) ^abs (d1 i) � (1 / 2 pow 24) ^abs (d2 i) � (1 / 2 pow 24) ^abs (p i) � (1 / 2 pow 24) ^abs (e1 i) � (1 / 2 pow 24) ^abs (e2 i) � (1 / 2 pow 24) ^abs (z i) � (1 / 2 pow 24) ^ 87

t 1 = (1 + d1 i) * (1 + p i) * (1 + k) ^t 2 = (1 + d2 i) * (1 + p i) * (1 + k) ^f 1 = (1 + e1 i) * (1 + z i) * (1 + k) ^f 2 = (1 + e2 i) * (1 + z i) * (1 + k) ^9 s s0. Float_Fxp_Error n = sum (1,K) (� i. Float_Fxp_Error i n) +sum (2,(K � 1)) (� j. s0 j) �sum (1, K) (� i. ((s i � 1) * Val (yy i n))) ^9 k. s i = if (i = 1) then mul (2, (K � 1)) (� i. (1 + k i)) elsemul (i,(K � i + 1)) (� i. (1 + k i))We proved these lemmas using the fundamental error analysis lemmas (Lemmas4,5, and 6 for oating-point, and Lemmas 9,10, and 11 for �xed-point), based on the errormodels presented in Se tion 3.2. The lemmas are proved by indu tion on the parameter Kwhi h is de�ned as the number of internal sub-�lters onne ted in parallel form to generatethe �nal output, a ording to the Equation (3.7).3.3.5 Lth-Order Filter (Cas ade Form)The as ade form realization of a parametri Lth-order �lter is spe i�ed by Equation (3.8).The orresponding omputed oating- and �xed-point outputs areyi+1n = fl[yin + kiyin�1 + liyin�2 � iyi+1n�1 � diyi+1n�2℄ (3.52)vi+1n = fxp[vin + kivin�1 + livin�2 � ivi+1n�1 � divi+1n�2℄ (3.53)In HOL, we spe i�ed the se ond-order digital �lter in real, oating-, and �xed-pointabstra tion levels, using re ursive de�nitions as predi ates in higher-order logi . The orresponding odes are as follows.88

`def Cas ade_Form_Ideal_Spe d k l x w =8 n. (w 0 n = x n ^ 8 i.w (SUC i) n = ((((w i n + k i * w i (n � 1)) +l i * w i (n � 2)) � i * w (SUC i) (n � 1)) �d i * w (SUC i) (n � 2)))`def Cas ade_Form_Float_Imp 0 d0 k0 l0 x0 y =8 n. (y 0 n = x0 n ^ 8 i.y (SUC i) n = ((((y i n + (k0 i * y i (n � 1))) +(l0 i * y i (n � 2))) � ( 0 i * y (SUC i) (n � 1))) �(d0 i * y (SUC i) (n � 2))))`def Cas ade_Form_Fxp_Imp X 00 d00 k00 l00 x00 v =8 n. (v 0 n = x00 n ^ 8 i. v (SUC i) n =FxpSub X (FxpSub X (FxpAdd X (FxpAdd X v i n(FxpMul X (k00 i) v i (n � 1)))(FxpMul X (l00 i) v i (n � 2)))(FxpMul X ( 00 i) v (i + 1) (n � 1)))(FxpMul X (d00 i) v (i + 1) (n � 2)))Figure 3.6 shows the error owgraph for the as ade form realization of a parametri Lth-order �lter.1 � 0i+1n;1 � 0in;2 �0i+1n;21 + �in;2 iyi+1n�1

diyi+1n�2�0i+1n 1 + �i+1n �0i+1n;21 + �i+1n;1 1 + �i+1n;21 + Æi+1n;1 yi+1n 1 + �i+1n;2Æ0i+1n;1 �0i+1n;11 + �i+1n;1Æ0i+1n;2 1 + Æi+1n;2yinkiyin�1

vinkivin�1

ivi+1n�1divi+1n�2vi+1n

liyin�2livin�2

Figure 3.7: Error owgraph for Lth-order �lter (Cas ade form)The a tual oating-point output sequen e for the as ade form is therefore given byyi+1n = yin�i+1n;0 + fiyin�1�i+1n;1 + giyin�2�i+1n;2 � iyin�1�i+1n;1 � diyin�2�i+1n;2 (3.54)89

where �i+1n;0 = (1 + �i+1n;1 )(1 + �i+1n;2 )(1 + �i+1n )�i+1n;1 = (1 + Æi+1n;1 )(1 + �i+1n;1 )(1 + �i+1n;2 )(1 + �i+1n )�i+1n;2 = (1 + Æi+1n;2 )(1 + �i+1n;1 )(1 + �i+1n )�i+1n;1 = (1 + �i+1n;1 )(1 + �i+1n;1 )(1 + �i+1n )�i+1n;2 = (1 + �i+1n;2 )(1 + �i+1n;1 )(1 + �i+1n )In HOL, we established the following lemma to ompute the real value of the oating-point output of the parallel form of realization a ording to the Equation (3.54).Lemma 33: CASCADE_FORM_FLOAT_OUTPUT_VALUE` Cas ade_Form_Float_Imp 0 d0 k0 l0 x0 y =)9 t f. Val (y (SUC i) n) = Val (y i n) * (t 0) +Val (k0 i) * Val (y i (n � 1)) * (t 1) +Val (l0 i) * Val (y i (n � 2)) * (t 2) �Val ( 0 i) * Val (y i (n � 1)) * (f 1) �Val (d0 i) * Val (y i (n � 2)) * (f 2) ^9 k d1 d2 p1 p2 e1 e2 z. abs (k i) � (1 / 2 pow 24) ^abs (d1 i) � (1 / 2 pow 24) ^abs (d2 i) � (1 / 2 pow 24) ^abs (p1 i) � (1 / 2 pow 24) ^abs (p2 i) � (1 / 2 pow 24) ^abs (e1 i) � (1 / 2 pow 24) ^abs (e2 i) � (1 / 2 pow 24) ^abs (z i) � (1 / 2 pow 24) ^90

t 0 = (1 + p1 i) * (1 + p2 i) * (1 + k) ^t 1 = (1 + d1 i) * (1 + p1 i) * (1 + p2 i) * (1 + k) ^t 2 = (1 + d2 i) * (1 + p2 i) * (1 + k) ^f 1 = (1 + e1 i) * (1 + z i) * (1 + k) ^f 2 = (1 + e2 i) * (1 + z i) * (1 + k)Similarly, the a tual �xed-point outputs of the as ade form of realization is givenexpli itly byvi+1n = vin + fivin�1 + givin�2 � ivi+1n�1 � divi+1n�2 + Æ0i+1n;1 + � 0i+1n;1 + Æ0i+1n;2 + � 0in;2+ (3.55)�0i+1n;1 + �0i+1n;2 + �0i+1n;2 + �0i+1nand the orresponding lemma is established in HOL as follows:Lemma 34: CASCADE_FORM_FXP_OUTPUT_VALUE` Cas ade_Form_Fxp_Imp X 00 d00 k00 l00 x00 v =)9 k0 d10 d20 p10 p20 e10 e20 z0.abs (k0 i) � inv (2 pow (fra bits X)) ^abs (d10 i) � inv (2 pow (fra bits X)) ^abs (d20 i) � inv (2 pow (fra bits X)) ^abs (p10 i) � inv (2 pow (fra bits X)) ^abs (p20 i) � inv (2 pow (fra bits X)) ^abs (e10 i) � inv (2 pow (fra bits X)) ^abs (e20 i) � inv (2 pow (fra bits X)) ^abs (z0 i) � inv (2 pow (fra bits X)) ^value (v (SUC i) n) = value (v i n) +value (k00 i) * value (v i (n � 1)) +value (l00 i) * value (v i (n � 2)) +value (l00 i) * value (v i (n � 2)) +value ( 00 i) * value (v (SUC i) (n � 1)) +value (d00 i) * value (v (SUC i) (n � 2)) +d10 i + p10 i + d20 i + p20 i + e10 i + e20 i + z0 i + k0 i91

For error analysis of the as ade form, we �rst de�ne the orresponding errors as:ei+1n = yi+1n � wi+1n (3.56)e0i+1n = vi+1n � wi+1n (3.57)e00i+1n = vi+1n � wi+1n (3.58)Then, we al ulate the yi+1n , and vi+1n sequen es from Equations (3.54), (3.55),respe tively and ompare them with the ideal output sequen es wi+1n spe i�ed by theEquations (3.8) to obtain the orresponding errors ei+1n , e0i+1n , and e00i+1n a ording to theEquations (3.56), (3.57), (3.58), respe tively. Therefore, the di�eren e equations for theerrors between di�erent levels showing the a umulation of roundo� error are derived asfollows:1) Floating-Point Error Analysis:ei+1n = ein + fiein�1 + giein�2 � iei+1n�1 � diei+1n�2 + yin(�i+1n;0 � 1) + fiyin�1(�i+1n;1 � 1)+ (3.59)giyin�2(�i+1n;2 � 1)� iyin�1(�i+1n;1 � 1)� diyin�2(�i+1n;2 � 1)To prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in oating-point realization of the as ade form �lter, a ordingto the Equation (3.59).

92

Lemma 35: CASCADE_FORM_FLOAT_TO_REAL_THM` Cas ade_Form_Ideal_Spe d k l x w ^Cas ade_Form_Float_Imp 0 d0 k0 l0 x0 y =)9 t f. Float_Error (SUC i) n = Float_Error i n +k i * Float_Error i (n � 1) + l i * Float_Error i (n � 2) � i * Float_Error (SUC i) (n � 1) �d i * Float_Error (SUC i) (n � 2) +Val (y i n) * (t 0 � 1) +(k i) * Val (y i (n � 1) (t 1 � 1) +(l i) * Val (y i (n � 2)) (t 2 � 1) �( i) * Val (y i (n � 1)) (f 1 � 1) �(d i) * Val (y i (n � 2)) (f 2 � 1) ^9 k d1 d2 p1 p2 e1 e2 z. abs (k i) � (1 / 2 pow 24) ^abs (d1 i) � (1 / 2 pow 24) ^abs (d2 i) � (1 / 2 pow 24) ^abs (p1 i) � (1 / 2 pow 24) ^abs (p2 i) � (1 / 2 pow 24) ^abs (e1 i) � (1 / 2 pow 24) ^abs (e2 i) � (1 / 2 pow 24) ^abs (z i) � (1 / 2 pow 24) ^t 0 = (1 + p1 i) * (1 + p2 i) * (1 + k) ^t 1 = (1 + d1 i) * (1 + p1 i) * (1 + p2 i) * (1 + k) ^t 2 = (1 + d2 i) * (1 + p2 i) * (1 + k) ^f 1 = (1 + e1 i) * (1 + z i) * (1 + k) ^f 2 = (1 + e2 i) * (1 + z i) * (1 + k)2) Fixed-Point Error Analysis:e0i+1n = e0in + fie0in�1 + gie0in�2 � ie0i+1n�1 � die0i+1n�2+ (3.60)Æ0i+1n;1 + � 0i+1n;1 + Æ0i+1n;2 + � 0i+1n;2 + �0i+1n;1 + �0i+1n;2 + �0i+1n;2 + �0i+1nTo prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in oating-point realization of the as ade form �lter, a ording93

to the Equation (3.60).Lemma 36: CASCADE_FORM_FXP_TO_REAL_THM` Cas ade_Form_Ideal_Spe d k l x w ^Cas ade_Form_Fxp_Imp X 00 d00 k00 l00 x00 v =)9 k0 d10 d20 p10 p20 e10 e20 z0.abs (k0 i) � inv (2 pow (fra bits X)) ^abs (d10 i) � inv (2 pow (fra bits X)) ^abs (d20 i) � inv (2 pow (fra bits X)) ^abs (p10 i) � inv (2 pow (fra bits X)) ^abs (p20 i) � inv (2 pow (fra bits X)) ^abs (e10 i) � inv (2 pow (fra bits X)) ^abs (e20 i) � inv (2 pow (fra bits X)) ^abs (z0 i) � inv (2 pow (fra bits X)) ^Fxp_Error (SUC i) n = Fxp_Error i n + k i * Fxp_Error i (n � 1) +l i * Fxp_Error i (n � 2) � i * Fxp_Error (SUC i) (n � 1) �d i * Fxp_Error (SUC i) (n � 2) +d10 i + p10 i + d20 i + p20 i + e10 i + e20 i + z0 i + k0 i3) Floating- to Fixed-Point Error Analysis:e00i+1n = e00in + fie00in�1 + gie00in�2 � ie00i+1n�1 � die00i+1n�2 + Æ0i+1n;1 + � 0i+1n;1 + Æ0i+1n;2 + � 0i+1n;2 + (3.61)�0i+1n;1 + �0i+1n;2 + �0i+1n;2 + �0i+1n � yin(�i+1n;0 � 1)� fiyin�1(�i+1n;1 � 1)� giyin�2(�i+1n;2 � 1)+ iyin�1(�i+1n;1 � 1) + diyin�2(�i+1n;2 � 1)To prove this theorem in HOL, we established the following lemma for the a umu-lation of round-o� error in oating-point realization of the as ade form �lter, a ordingto the Equation (3.61).94

Lemma 37: CASCADE_FORM_FXP_TO_FLOAT_THM` Cas ade_Form_Ideal_Spe d k l x w ^Cas ade_Form_Float_Imp 0 d0 k0 l0 x0 y ^Cas ade_Form_Fxp_Imp X 00 d00 k00 l00 x00 v =)9 t f k0 d10 d20 p10 p20 e10 e20 z0.abs (k0 i) � inv (2 pow (fra bits X)) ^abs (d10 i) � inv (2 pow (fra bits X)) ^abs (d20 i) � inv (2 pow (fra bits X)) ^abs (p10 i) � inv (2 pow (fra bits X)) ^abs (p20 i) � inv (2 pow (fra bits X)) ^abs (e10 i) � inv (2 pow (fra bits X)) ^abs (e20 i) � inv (2 pow (fra bits X)) ^abs (z0 i) � inv (2 pow (fra bits X)) ^Float_Fxp_Error (SUC i) n = Float_Fxp_Error i n +k i * Float_Fxp_Error i (n � 1) +l i * Float_Fxp_Error i (n � 2) � i * Float_Fxp_Error (SUC i) (n � 1) �d i * Fxp_Error (SUC i) (n � 2) +d10 i + p10 i + d20 i + p20 i + e10 i + e20 i + z0 i + k0 i �Val (y i n) * (t 0 � 1) �(k i) * Val (y i (n � 1) (t 1 � 1) �(l i) * Val (y i (n � 2)) (t 2 � 1) +( i) * Val (y i (n � 1)) (f 1 � 1) +(d i) * Val (y i (n � 2)) (f 2 � 1) ^9 k d1 d2 p1 p2 e1 e2 z. abs (k i) � (1 / 2 pow 24) ^abs (d1 i) � (1 / 2 pow 24) ^abs (d2 i) � (1 / 2 pow 24) ^abs (p1 i) � (1 / 2 pow 24) ^abs (p2 i) � (1 / 2 pow 24) ^abs (e1 i) � (1 / 2 pow 24) ^abs (e2 i) � (1 / 2 pow 24) ^abs (z i) � (1 / 2 pow 24) ^t 0 = (1 + p1 i) * (1 + p2 i) * (1 + k) ^t 1 = (1 + d1 i) * (1 + p1 i) * (1 + p2 i) * (1 + k) ^t 2 = (1 + d2 i) * (1 + p2 i) * (1 + k) ^95

f 1 = (1 + e1 i) * (1 + z i) * (1 + k) ^f 2 = (1 + e2 i) * (1 + z i) * (1 + k)We proved these lemmas using the fundamental error analysis lemmas (Lemmas4,5, and 6 for oating-point, and Lemmas 9,10, and 11 for �xed-point), based on the errormodels presented in Se tion 3.2.3.4 Con lusionIn this hapter, we des ribed our omprehensive methodology for the error analysis ofgeneri digital �lters using the HOL theorem prover. We believe this is the �rst time a omplete formal framework is onsidered using me hani al proofs in HOL for the erroranalysis of digital �lters. In the next hapter, we des ribe the formal veri� ation of FFTalgorithms. We perform a similar error analysis between the real numbers and the oating-point and �xed-point algorithmi levels. We also perform the veri� ation for the transitionfrom the oating-point and �xed-point algorithmi levels to hardware implementations forFFT algorithms.

96

Chapter 4Veri� ation of FFT Algorithms inHOL4.1 Introdu tionThe fast Fourier transform (FFT) [9, 16℄ is a highly eÆ ient method for omputing thedis rete Fourier transform (DFT) oeÆ ients of a �nite sequen e of omplex data. Be auseof the substantial time saving over onventional methods, the fast Fourier transform hasfound important appli ations in a number of diverse �elds su h as spe trum analysis,spee h and opti al signal pro essing, and digital �lter design. FFT algorithms are basedon the fundamental prin iple of de omposing the omputation of the dis rete Fouriertransform of a �nite-length sequen e of length N into su essively smaller dis rete Fouriertransforms. The manner in whi h this prin iple is implemented leads to a variety ofdi�erent algorithms, all with omparable improvements in omputational speed. Thereare two basi lasses of FFT algorithms for whi h the number of arithmeti multipli ationsand additions as a measure of omputational omplexity is proportional to N log N ratherthan N2 as in the onventional methods. The �rst proposed by Cooley and Tukey [18℄, alled de imation-in-time (DIT), derives its name from the fa t that in the pro ess ofarranging the omputation into smaller transformations, the input sequen e (generallythought of as a time sequen e) is de omposed into su essively smaller subsequen es. In97

the se ond general lass of algorithms proposed by Gentleman and Sande [22℄, the sequen eof dis rete Fourier transform oeÆ ients is de omposed into smaller subsequen es, hen eits name, de imation-in-frequen y (DIF).In Chapter 1, Figure 1.1 illustrates a generi DSP (digital signal pro essing) design ow as used in leading industrial proje ts for the design of FFT algorithms. Thereafter, thedesign pro ess starts from an ideal real spe i� ation used for the theoreti al analysis of thefast Fourier transform. Here signal values and system oeÆ ients are represented with realnumbers expressed to in�nite pre ision. When implemented as a spe ial-purpose digitalhardware or as a omputer algorithm, these must be represented in some digital numbersystem of �nite pre ision. There is hen e an inherent a ura y problem in al ulating theFourier oeÆ ients, sin e the arithmeti operations must be arried out with an a ura ylimited by the �nite word length of signals. Among the most ommon types of arithmeti used in the implementation of FFT systems are oating- and �xed-point [59℄. Here, alloperands are represented by a spe ial format or assigned a �xed word length and a �xedexponent, while the ontrol stru ture and the operations of the ideal program remainun hanged. The transformation from real to oating- and �xed-point is quite tedious anderror-prone. On the implementation side, the �xed-point model of the algorithm has tobe transformed into the best suited target des ription, either using a hardware des riptionlanguages (HDL) or a software programming language.The onforman e of the �xed-point implementation with respe t to the des riptionsin oating-point or real algorithm on one hand, and the RT (Register Transfer) and gatelevels on the other hand is veri�ed by simulation te hniques. Simulation is, however,known to provide partial veri� ation as it annot over all design errors, espe ially forlarge systems. In this hapter, we are proposing the use of formal methods for the mod-eling and veri� ation of FFT algorithms. Adopting formal veri� ation generally meansusing methods of mathemati al proof to ensure the quality of the design, to improve therobustness of a design and to speed up the overall system design and development y les.The proposed veri� ation approa h is depi ted in the ommutating diagram shownin Figure 4.1, where we model the ideal real spe i� ation of the FFT algorithms and the orresponding oating- and �xed-point representations as well as the RT and gate level98

FP Real Value

FP to FXP ErrorAnalysis

FXP Real Value( HOL )

( HOL )

( HOL )

( HOL )

( HOL )

( HOL )

Valuation

Valuation

FXP ErrorAnalysis

FP ErrorAnalysis

FFT REAL

FFT FP

FFT FXP

FFT RTL

FFT Netlist

( HOL )

LogicalImplication

LogicalImplicationFigure 4.1: Proposed FFT spe i� ation and veri� ation approa himplementations as predi ates in higher-order logi . The overall methodology for the for-mal spe i� ation and veri� ation of FFT algorithms will be based on the idea of shallowembedding of languages [4℄ using the HOL theorem proving environment [23℄. In theproposed approa h, we �rst fo us on the transition from real to oating- and �xed-pointlevels. For this, we make use of existing theories in HOL on the onstru tion of real [27℄and omplex [32℄ numbers, the formalization of IEEE-754 standard based oating-pointarithmeti [28, 29℄, and the formalization of �xed-point arithmeti . We use valuationfun tions to �nd the real values of the oating- and �xed-point FFT outputs and de�nethe error as the di�eren e between these values and the orresponding output of the idealreal spe i� ation. Then we establish fundamental lemmas on the error analysis of oating-and �xed-point rounding and arithmeti operations against their abstra t mathemati al ounterparts. Finally, based on these lemmas, we derive, for ea h of the two anoni alforms of realization, expressions for the a umulation of roundo� error in oating- and�xed-point FFT algorithms using re ursive de�nitions and initial onditions. While theo-reti al work on omputing the errors due to �nite pre ision e�e ts in the realization of FFTalgorithms with oating- and �xed-point arithmeti s has been extensively studied sin ethe late sixties [41℄, this thesis ontains the �rst formalization and proof of this analysis99

using a me hani al theorem prover, here HOL. The formal results are found to be in goodagreement with the theoreti al ones.After handling the transition from real to oating- and �xed-point levels, we turn tothe HDL representation. At this point, we use well known te hniques to model the FFTdesign at the RTL level within the HOL environment. The last step is to verify this levelusing a lassi al hierar hi al proof approa h in HOL [52℄. In this way, we hierar hi allyprove that the FFT RTL implementation implies the high level �xed-point algorithmi spe i� ation, whi h has already been related to the oating-point des ription and theideal real spe i� ation through the error analysis. The veri� ation an be extended, fol-lowing similar manner, down to gate level netlist either in HOL or using other ommer ialveri� ation tools as depi ted in Figure 4.1, whi h is not overed in this paper.The rest of the hapter is organized as follows: Se tion 4.2 des ribes the details ofthe error analysis in HOL of the FFT algorithms at the real, oating-point and �xed-pointlevels. Se tion 4.3 des ribes the veri� ation of the FFT algorithms in the transition to theRTL and gate level netlist for a radix-4 16-point FFT implementation. Finally, Se tion4.4 on ludes the hapter.4.2 Error Analysis of FFT Algorithms in HOLIn this se tion, the prin ipal results for roundo� a umulation in FFT algorithms usingHOL theorem proving are derived and summarized. For the most part, the followingdis ussion is phrased in terms of the de imation-in-frequen y form of radix-2 algorithm.The results, however, are appli able with only minor modi� ation to the de imation-in-time form. Furthermore, most of the ideas employed in the error analysis of the radix-2algorithms an be utilized in the analysis of other algorithms. In the following, we will�rst des ribe in detail the theory behind the analysis and then explain how this analysisis performed in HOL.The dis rete Fourier transform of a sequen e fx(n)gN�1n=0 is de�ned as [59℄A(p) = N�1Xn=0 x(n) (WN )np; p = 0; 1; 2; : : : ; N � 1 (4.1)100

where WN = e�j2�=N and j = p�1. The multipli ative fa tors (WN )np are alledtwiddle fa tors. For simpli ity, our dis ussion is restri ted to the radix-2 FFT algorithm, inwhi h the number of points N to be Fourier transformed satisfy the relationshipN = 2m,wherem is an integer value. The results an be extended to radi es other than 2. By usingthe FFT method, the Fourier oeÆ ients fA(p)gN�1p=0 an be al ulated in m = log2Niterative steps. At ea h step, an array of N omplex numbers is generated by usingonly the numbers in the previous array. To explain the FFT algorithm, let ea h integerp; p = 0; 1; 2; : : : ; N � 1, be expanded into a binary form asp = 2m�1p0 + 2m�2p1 + � � � + 2pm�2 + pm�1; pk = 0 or 1 (4.2)and let p� denote the number orresponding to the reverse bit sequen es of p, i.e.,p� = 2m�1pm�1 + 2m�2pm�2 + � � �+ 2p1 + p0 (4.3)� De imation-in-Frequen y (DIF) FFT Algorithm:Let fAk(p)gN�1p=0 denote the N omplex numbers al ulated at the kth step. Thenthe de imation in frequen y (DIF) FFT algorithm an be expressed as [41℄Ak+1(p) = 8<: Ak(p) +Ak(p+ 2m�1�k) if pk = 0[Ak(p� 2m�1�k)�Ak(p)℄ wk(p) if pk = 1 (4.4)where wk(p) is a power of WN given by wk(p) = (WN )zk(p), wherezk(p) = 2k (2m�1�kpk + 2m�2�kpk+1 + � � �+ 2pm�2 + pm�1)� 2m�1pk (4.5)Equation (4.4) is arried out for k = 0; 1; 2; : : : ;m� 1; with A0(p) = x(p). It an beshown [22℄ that at the last step fAm(p)gN�1p=0 are the dis rete Fourier oeÆ ients inrearranged order. Spe i� ally, Am(p) = A(p�) with p and p� expanded and de�nedas in Equations (4.2) and (4.3), respe tively. Figure 4.2 shows the signal owgraphof the a tual omputation for the ase N = 24.101

Formally, a owgraph onsists of nodes and dire ted bran hes. Ea h bran h has aninput signal and an output signal with a dire tion indi ated by an arrowhead onit. Ea h node represents a variable whi h is the weighted sum of the variables atthe originating nodes of the bran hes that terminate on that node. The weights,if other than unity, are shown for ea h bran h. Sour e nodes have no enteringbran hes. They are used to represent the inje tion of the external inputs or signalsour es into the owgraph. Sink nodes have only entering bran hes. They are usedto extra t the outputs from the owgraph [59℄.

W 0NW 1NW 2NW 3NW 4NW 5NW 6NW 7N

W 0NW 2NW 4NW 6N

W 0NW 2NW 4NW 6N

W 0NW 4NW 0NW 4NW 0NW 4NW 0NW 4N W 0N

W 0NW 0NW 0NW 0NW 0N

A (15)A (11)A (3)

A (8)A (0)A (4)A (12)A (2)A (10)A (6)A (14)A (1)A (9)A (5)A (13)

-1-1-1-1-1-1-1-1-1-1

-1-1-1-1-1-1-1-1

-1-1

-1-1-1

-1-1-1-1-1-1-1-1

-1

A (7)

W 0NW 0Nx (0)x (1)x (2)x (3)x (4)x (5)x (6)x (7)x (8)x (9)x (10)x (11)x (12)x (13)x (14)x (15)

fA1(p)g fA2(p)g fA3(p)gfx(p)g = fA0(p)g fA4(p)g = fA(p�)g

Figure 4.2: Signal owgraph of de imation-in-frequen y FFT, N = 24� De imation-in-Time (DIT) FFT Algorithm:Let fAk(p)gN�1p=0 denote the N omplex numbers al ulated at the kth step. Thenthe de imation in time (DIT) FFT algorithm an be expressed as [48℄102

Ak+1(p) = 8<: Ak(p) + wk(p) Ak(p+ 2k) if pm�1�k = 0Ak(p� 2k)� wk(p) Ak(p) if pm�1�k = 1 (4.6)where wk(p) is a power of WN given by wk(p) = (WN )zk(p), wherezk(p) = 2m�1�k (2kpm�1�k+2k�1pm�k+ � � �+2pm�2+pm�1)�2m�1pm�1�k (4.7)Equation (4.6) is arried out for k = 0; 1; 2; : : : ;m � 1; with A0(p) = x(p�) with pand p� expanded and de�ned as in Equations (4.2) and (4.3), respe tively. It an beshown [18℄ that at the last step fAm(p)gN�1p=0 are the dis rete Fourier oeÆ ients inthe normal order. Spe i� ally, Am(p) = A(p). Figure 4.3 shows the signal owgraphof the a tual omputation for the ase N = 24.

-1-1-1-1-1-1-1-1

W 0NW 1NW 2NW 3NW 4NW 5NW 6NW 7N-1-1-1-1

-1-1-1-1W 0NW 2NW 4NW 6N

W 0NW 2NW 4NW 6N

-1-1-1-1-1-1-1-1

W 0NW 4NW 0NW 4NW 0NW 4NW 0NW 4N-1

-1-1-1-1-1-1-1

W 0NW 0NW 0NW 0NW 0NW 0NW 0NW 0N A (0)A (1)A (2)A (3)A (4)A (5)A (6)A (7)A (8)A (9)A (10)A (11)A (12)A (13)A (14)A (15)

x (0)x (8)x (4)x (12)x (2)x (10)x (6)x (14)x (1)x (9)x (5)x (13)x (3)x (11)x (7)x (15)

fA4(p)g = fA(p)gfx(p�)g = fA0(p)g fA1(p)g fA2(p)g fA3(p)g

Figure 4.3: Signal owgraph of de imation-in-time FFT, N = 24103

There are three ommon sour es of errors asso iated with the FFT algorithms,namely [41℄:1. Input Quantization: aused by the quantization of the input signal fxng into aset of dis rete levels.2. CoeÆ ient A ura y: aused by the representation of the oeÆ ients fwk(p)g bya �nite word length.3. Round-O� A umulation: aused by the a umulation of roundo� errors atarithmeti operations.Therefore, the a tual array omputed by using equations (4.4) and (4.6) are in gen-eral di�erent from fAk(p)gN�1p=0 . We denote the a tual oating- and �xed-point omputedarrays by fA0k(p)gN�1p=0 and fA00k(p)gN�1p=0 , respe tively. Then, we de�ne the orrespondingerrors of the pth element at step k asek(p) = A0k(p)�Ak(p) (4.8)e0k(p) = A00k(p)�Ak(p) (4.9)e00k(p) = A00k(p)�A0k(p) (4.10)where ek(p) and e0k(p) are de�ned as the error between the a tual oating- and �xed-point implementations and the ideal real spe i� ation, respe tively. e00k(p) is the error intransition from oating- to �xed-point levels.In analyzing the e�e t of oating-point roundo�, the e�e t of rounding will berepresented multipli atively. Letting � denote any of the arithmeti operations +, -, � ,/, as proved in Se tion 3.2, if p represents the pre ision of the oating-point format, thenfl (x � y) = (x � y)(1 + Æ); where jÆj � 2�p (4.11)The notation fl (:) is used to denote that the operation is performed using oating-point arithmeti . The theorem relates the oating-point arithmeti operations su h asaddition, subtra tion, multipli ation, and division to their abstra t mathemati al oun-terparts a ording to the orresponding errors.104

While the rounding error for oating-point arithmeti enters into the system multi-pli atively, it is an additive omponent for �xed-point arithmeti . In this ase the funda-mental error analysis theorem for �xed-point arithmeti operations against their abstra tmathemati al ounterparts as shown in Se tion 3.2 an be stated asfxp (x � y) = (x � y) + �; where j�j � 2�fra bits (X) (4.12)and fra bits is the number of bits that are to the right of the binary point in the given �xed-point format X. The notation fxp (:) is used to denote that the operation is performedusing �xed-point arithmeti . We have proved equations (4.11) and (4.12) as theorems inhigher-order logi within HOL. The theorems are proved under the assumption that thereis no over ow or under ow in the operation result. This means that the input values ares aled so that the real value of the result is lo ated in the ranges de�ned by the maximumand minimum representable values of the given oating-point and �xed-point formats.In equation (4.4) the fAk(p)g are omplex numbers, so their real and imaginaryparts are al ulated separately. LetBk(p) = Re [Ak(p)℄ Ck(p) = Im [Ak(p)℄Uk(p) = Re [wk(p)℄ Vk(p) = Im [wk(p)℄ (4.13)where the notations Re [:℄ and Im [:℄ denote, respe tively, the real and imaginary parts ofthe quantity inside the bra ket [:℄. Equation (4.4) an be rewritten asBk+1(p) = Bk(p) +Bk(q)Ck+1(p) = Ck(p) + Ck(q) 9=; if pk = 0 (4.14)Bk+1(p) = [Bk(r)�Bk(p)℄ Uk(p)� [Ck(r)� Ck(p)℄ Vk(p)Ck+1(p) = [Ck(r)� Ck(p)℄ Uk(p) + [Bk(r)�Bk(p)℄ Vk(p) 9=; if pk = 1where q = p+ 2m�1�k and r = p� 2m�1�k. Similarly, we an express the real and imag-inary parts of A0k+1(p), B0k+1(p) and C 0k+1(p), and A00k+1(p), B00k+1(p) and C 00k+1(p), usingthe oating- and �xed-point operations, respe tively. The orresponding error owgraphshowing the e�e t of roundo� error using the fundamental oating- and �xed-point erroranalysis theorems a ording to the equations (4.11) and (4.12), respe tively, is given in105

Figure 4.4, whi h also indi ates the order of the al ulation.The quantities 0k;p, 00k;p, Æ0k;p, Æ00k;p, �0k;p, �00k;p, � 0k;p, � 00k;p, �0k;p, �00k;p, �0k;p, and �00k;pin Figure 4.4 are errors aused by oating-point roundo� at ea h arithmeti step. The orresponding error quantities for �xed-point roundo� are k;p, 000k;p, Æk;p, Æ000k;p, �k;p, �000k;p,�k;p, � 000k;p, �k;p, �000k;p, �k;p, and �000k;p. Thereafter, the a tual real and imaginary parts of the oating- and �xed-point outputs A0k+1(p) and A00k+1(p), respe tively are seen to be givenexpli itly byB00k(q) C 0k(p)C 00k (p) C 0k(q)C 00k (q)

1 + �00k;p�k;pUk �Vk Uk Vk

pk = 11 + � 00k;p 1 + �00k;p�k;p 1 + �0k;p �000k;p

B0k(q) 000k;p 1 + 00k;pB0k+1(p) C 0k+1(p) k;p 1 + 0k;p pk = 0

C 00k+1(p)B00k+1(p)

B0k(p)B00k(p)

1 + �00k;p

�1B0k(r)B00k(r) B0k(p)B00k(p) C 0k(r)C 00k (r) C 0k(p)C 00k (p) C 0k(r)C 00k (r) C 0k(p)C 00k (p) B0k(r)B00k(r) B0k(p)B00k(p)�1

B0k+1(p) C 0k+1(p)

1 + �0k;p �000k;pÆ000k;p1 + Æ0k;p 1 + Æ00k;pÆk;p1 + �0k;p�k;p �000k;p�k;p � 000k;p1 + � 0k;p

C 00k+1(p)B00k+1(p)

�1�1

Figure 4.4: Error owgraph for de imation-in-frequen y FFTB0k+1(p) = [B0k(p) +B0k(q)℄(1 + 0k;p)C 0k+1(p) = [C 0k(p) + C 0k(q)℄(1 + 00k;p) 9=; if pk = 0 (4.15)B0k+1(p) = [B0k(r)�B0k(p)℄ Uk(p)(1 + Æ0k;p)(1 + � 0k;p)(1 + �0k;p)� [C 0k(r)� C 0k(p)℄ Vk(p)(1 + Æ00k;p)(1 + � 00k;p)(1 + �0k;p)C 0k+1(p) = [C 0k(r)� C 0k(p)℄ Uk(p)(1 + �0k;p)(1 + �0k;p)(1 + �00k;p)+ [B0k(r)�B0k(p)℄ Vk(p)(1 + �00k;p)(1 + �00k;p)(1 + �00k;p)9>>>>>>=>>>>>>; if pk = 1and 106

B00k+1(p) = [B00k(p) +B00k(q)℄ + k;pC 00k+1(p) = [C 00k (p) + C 00k (q)℄ + 000k;p 9=; if pk = 0 (4.16)B00k+1(p) = [B00k(r)�B00k(p) + Æk;p℄ Uk(p) + �k;p�([C 00k (r)� C 00k (p) + Æ000k;p℄ Vk(p) + � 000k;p) + �k;pC 00k+1(p) = [C 00k (r)� C 00k (p) + �k;p℄ Uk(p) + �k;p+([B00k(r)�B00k(p) + �000k;p℄ Vk(p) + �000k;p) + �000k;p9>>>>>>=>>>>>>; if pk = 1The errors ek(p), e0k(p), and e00k(p) de�ned in equations (4.8), (4.9), and (4.10) are omplex and an be rewritten asek(p) = B0k(p)�Bk(p) + j[C 0k(p)� Ck(p)℄ (4.17)e0k(p) = B00k(p)�Bk(p) + j[C 00k (p)� Ck(p)℄ (4.18)e00k(p) = B00k(p)�B0k(p) + j[C 00k (p)� C 0k(p)℄ (4.19)k = 1; 2; : : : ;m; p = 0; 1; : : : ; N � 1with e0(p) = e00(p) = e000(p) = 0; p = 0; 1; : : : ; N � 1 (4.20)From equations (4.14), (4.15), (4.16), (4.17), (4.18), and (4.19), we derive the followingerror analysis ases:1. FFT Real to Floating-Point:ek+1(p) = 8<: ek(p) + ek(q) + fk(p) if pk = 0[ek(r)� ek(p)℄ wk(p) + fk(p) if pk = 1 (4.21)where fk(p) is given by

fk(p) =8>>>>>>>>>>>><>>>>>>>>>>>>:

0k;p[B0k(p) +B0k(q)℄+j 00k;p[C 0k(p) + C 0k(q)℄ if pk = 0[(1 + Æ0k;p)(1 + � 0k;p)(1 + �0k;p)� 1℄[B0k(r)�B0k(p)℄Uk(p)�[(1 + Æ00k;p)(1 + � 00k;p)(1 + �0k;p)� 1℄[C 0k(r)� C 0k(p)℄Vk(p)+j[(1 + �0k;p)(1 + �0k;p)(1 + �00k;p)� 1℄[C 0k(r)� C 0k(p)℄Uk(p)+j[(1 + �00k;p)(1 + �00k;p)(1 + �00k;p)� 1℄[B0k(r)�B0k(p)℄Vk(p)if pk = 1 (4.22)107

2. FFT Real to Fixed-Point:e0k+1(p) = 8<: e0k(p) + e0k(q) + f 0k(p) if pk = 0[e0k(r)� e0k(p)℄ wk(p) + f 0k(p) if pk = 1 (4.23)where f 0k(p) is given byf 0k(p) = 8>>><>>>: k;p + j 000k;p if pk = 0Æk;pUk(p) + �k;p � Æ000k;pVk(p)� � 000k;p + �k;p+j(�k;pUk(p) + �k;p + �000k;pVk(p) + �000k;p + �000k;p) if pk = 1 (4.24)3. FFT Floating- to Fixed-Point:e00k+1(p) = 8<: e00k(p) + e00k(q) + f 0k(p)� fk(p) if pk = 0[e00k(r)� e00k(p)℄ wk(p) + f 0k(p)� fk(p) if pk = 1 (4.25)where fk(p) and f 0k(p) are given by equations (4.22) and (4.24).The a umulation of roundo� error is determined by the re ursive equations (4.21),(4.22), (4.23), (4.24), and (4.25), with initial onditions given by equation (4.20).In HOL, we �rst onstru ted omplex numbers on reals similar to [32℄. We de�nedin HOL a new type for omplex numbers, to be in bije tion with R � R. The bije tionsare written in HOL as omplex : R2 ! C and oords : C ! R2 .`def (8 a. omplex ( oords a) = a) ^ (8 r. oords ( omplex r) = r)We used onvenient abbreviations for the real (Re) and imaginary (Im) parts of a omplex number.`def Re z = FST ( oords z)`def Im z = SND ( oords z)and also de�ned arithmeti operations su h as addition, subtra tion, and multipli ationon omplex numbers. We overloaded the usual symbols (+;�;�) for C and R.108

`def ompadd a b = (FST a + FST b,SND a + SND b)`def ompsub a b = (FST a � FST b,SND a � SND b)`def ompmul a b = (FST a * FST b � SND a * SND b,FST a * SND b � SND a * FST b)`def w omplex_add z = omplex ( ompadd ( oords w) ( oords z))`def w omplex_sub z = omplex ( ompsub ( oords w) ( oords z))`def w omplex_mul z = omplex ( ompmul ( oords w) ( oords z))Furthermore, we de�ned using re ursive de�nition in HOL expressions for the �nitesummation on omplex numbers.`def ( omplex_sum (n,0) f = omplex (0,0)) ^( omplex_sum (n,SUC m) f = omplex_sum (n,m) f + f (n + m))Similarly, we onstru ted omplex numbers on oating-point numbers ( oat omplex, oat oords, oat Re, oat Im, oat omplex add, oat omplex sub, oat omplex mul, oat omplex sum) and �xed-point numbers (fxp omplex, fxp oords, fxp Re, fxp Im, fxp omplex add, fxp omplex sub, fxp omplex mul, fxp omplex sum). We also de�ned round-ing and valuation fun tions for oating-point ( oat omplex round, oat omplex Val) and�xed-point (fxp omplex round, fxp omplex value) omplex numbers.Then we de�ned the prin ipal N -roots on unity (e�j2�n=N = os (2�n=N) �j sin (2�n=N)), and its powers (OMEGA) as a omplex number using the sine and osinefun tions available in the trans endental theory of the HOL reals library [27℄.`def prin ipal_root_1 n N = omplex ( os :2 * pi * & n / & N , sin :2 * pi * & n / & N)109

We spe i�ed expressions in HOL for expansion of a natural number into a binaryform in normal and rearranged order a ording to the equations (4.2), (4.3), and (4.5).`def DIG n m = (m DIV 2 ** n) MOD 2`def Binary_Form p m = (� k. DIG (m � 1 � k) p)`def Log_2 p = �k. p = 2 ** k`def (num_sum (n,0) f = 0) ^(num_sum (n,SUC m) f = num_sum (n,m) f + f (n + m))`def Z k p N = 2 ** k * num_sum (k,Log_2 N � k)(� i. 2 **(Log_2 N � 1 � i) * DIG i p) �2 ** (Log_2 N � 1) * DIG k p`def p_star p m = num_sum (0,m) (� i. 2 ** m * DIG i p)The above enables us to spe ify the FFT algorithms in real (FFT ), oating- (FLOAT FFT ),and �xed-point (FXP FFT ) abstra tion levels using re ursive de�nitions in HOL as de-s ribed in equation (4.4).`def (FFT x N 0 = (� p. x p)) ^FFT x N (SUC k) =(� p. (if DIG k p = 0 thenFFT x N k p + FFT x N k (p + 2 ** (Log_2 N � 1 � k))else(FFT x N k (p � 2 ** (Log_2 N � 1 � k)) � FFT x N k p) *OMEGA k p N))Then we de�ne the real and imaginary parts of the FFT algorithm (FFT REAL,FFT IMAGE ) and powers of the prin ipalN -roots on unity (OMEGA REAL,OMEGA IMAGE )a ording to the equation (4.13). 110

`def FFT_REAL x N k p = Re (FFT x N k p)`def FFT_IMAGE x N k p = Im (FFT x N k p)`def OMEGA_REAL k p N = Re (OMEGA k p N)`def OMEGA_IMAGE k p N = Im (OMEGA k p N)Later, we prove in separate lemmas that the real and imaginary parts of the FFTalgorithm in real, oating-point, and �xed-point levels an be expanded as in equation(4.14). In following, we show the HOL expansion theorem (Lemma 1 ) for real numbers.Similar lemmas have been derived for the oating- and �xed-point levels.Lemma 1:8 x N k p.(if DIG k p = 0 then(FFT_REAL x N (SUC k) p =FFT_REAL x N k p +FFT_REAL x N k (p + 2 ** (Log_2 N � 1 � k))) ^(FFT_IMAGE x N (SUC k) p =FFT_IMAGE x N k p +FFT_IMAGE x N k (p + 2 ** (Log_2 N � 1 � k)))else(FFT_REAL x N (SUC k) p =(FFT_REAL x N k (p � 2 ** (Log_2 N � 1 � k)) �FFT_REAL x N k p) * OMEGA_REAL k p N �(FFT_IMAGE x N k (p � 2 ** (Log_2 N � 1 � k)) �FFT_IMAGE x N k p) * OMEGA_IMAGE k p N) ^(FFT_IMAGE x N (SUC k) p =(FFT_IMAGE x N k (p � 2 ** (Log_2 N � 1 � k)) �FFT_IMAGE x N k p) * OMEGA_REAL k p N +(FFT_REAL x N k (p � 2 ** (Log_2 N � 1 � k)) �FFT_REAL x N k p) * OMEGA_IMAGE k p N))111

Then we prove lemmas to introdu e an error in ea h of the arithmeti steps in realand imaginary parts of the oating-point and �xed-point FFT algorithms a ording tothe equations (4.15), and (4.16). In following, we show the HOL theorem (Lemma 2 ) orresponding to the error analysis of the transition from real to oating-point. Similartheorems have been proven for the transitions from, respe tively, real and oating-pointto the �xed-point level.Lemma 2:8 x N k p.9 e.8 i.1 � i ^ i � 12 =)e i � 1 / 2 pow 24 ^(if DIG k p = 0 then(Val (FLOAT_FFT_REAL x N (SUC k) p) =(Val (FLOAT_FFT_REAL x N k p) +Val(FLOAT_FFT_REAL x N k (p + 2 ** (Log_2 N � 1 � k)))) *(1 + e 1)) ^(Val (FLOAT_FFT_IMAGE x N (SUC k) p) =(Val (FLOAT_FFT_IMAGE x N k p) +Val(FLOAT_FFT_IMAGE x N k(p + 2 ** (Log_2 N � 1 � k)))) * (1 + e 2))

112

else(Val (FLOAT_FFT_REAL x N (SUC k) p) =(Val(FLOAT_FFT_REAL x N k (p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_REAL x N k p)) *Val (FLOAT_OMEGA_REAL k p N) * (1 + e 3) * (1 + e 4) *(1 + e 5) �(Val(FLOAT_FFT_IMAGE x N k (p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_IMAGE x N k p)) *Val (FLOAT_OMEGA_IMAGE k p N) * (1 + e 6) * (1 + e 7) *(1 + e 5)) ^(Val (FLOAT_FFT_IMAGE x N (SUC k) p) =(Val(FLOAT_FFT_IMAGE x N k (p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_IMAGE x N k p)) *Val (FLOAT_OMEGA_REAL k p N) * (1 + e 8) * (1 + e 9) *(1 + e 10) +(Val(FLOAT_FFT_REAL x N k (p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_REAL x N k p)) *Val (FLOAT_OMEGA_IMAGE k p N) * (1 + e 11) * (1 + e 12) *(1 + e 10)))We prove these lemmas using the fundamental error analysis lemmas for basi arith-meti operations a ording to the equations (4.11) and (4.12). Then we de�ned in HOLthe error of the pth element of the oating- (FLOAT TO REAL FFT ERROR) and �xed-point (FXP TO REAL FFT ERROR) FFT algorithms at step k, and the orrespondingerror in transition from oating- to �xed-point (FLOAT TO FXP FFT ERROR), a ord-ing to the equations (4.8), (4.9), and (4.10).`def FLOAT_TO_REAL_FFT_ERROR x N k p =float_ omplex_Val (FLOAT_FFT x N k p) � FFT x N k p113

Thereafter, we prove lemmas to rewrite the errors as omplex numbers using thereal and imaginary parts a ording to the equations (4.17), (4.18), and (4.19), respe tively.The HOL theorem (Lemma 3 ) for the real numbers to oating-point transition is givenbelow.Lemma 3:8 x N k p.FLOAT_TO_REAL_FFT_ERROR x N k p = omplex(Val (FLOAT_FFT_REAL x N k p) � FFT_REAL x N k p,Val (FLOAT_FFT_IMAGE x N k p) � FFT_IMAGE x N k p)Finally, we prove a set of lemmas to determine the a umulation of roundo� errorin oating- and �xed-point FFT algorithms by re ursive equations and initial onditionsa ording to the equations (4.20), (4.21), (4.22), (4.23), (4.24), and (4.25). Lemma 4represents the HOL theorem for the transition from real numbers to oating-point.Lemma 4:8x N k p.(FLOAT_TO_REAL_FFT_ERROR x N 0 p = omplex (0,0)) ^9f.(FLOAT_TO_REAL_FFT_ERROR x N (SUC k) p =(if DIG k p = 0 thenFLOAT_TO_REAL_FFT_ERROR x N k p +FLOAT_TO_REAL_FFT_ERROR x N k (p + 2 ** (Log_2 N � 1 � k)) +f x N k pelse(FLOAT_TO_REAL_FFT_ERROR x N k (p � 2 ** (Log_2 N � 1 � k)) �FLOAT_TO_REAL_FFT_ERROR x N k p) * OMEGA k p N + f x N k p)) ^9e.8i.1 � i ^ i � 12 =)e i � 1 / 2 pow 24 ^(f x N k p =(if DIG k p = 0 then 114

omplex(e 1 *(Val (FLOAT_FFT_REAL x N k p) +Val(FLOAT_FFT_REAL x N k(p + 2 ** (Log_2 N � 1 � k)))),e 2 *(Val (FLOAT_FFT_IMAGE x N k p) +Val(FLOAT_FFT_IMAGE x N k(p + 2 ** (Log_2 N � 1 � k)))))else omplex(((1 + e 3) * (1 + e 4) * (1 + e 5) � 1) *(Val(FLOAT_FFT_REAL x N k(p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_REAL x N k p)) * OMEGA_REAL k p N �((1 + e 6) * (1 + e 7) * (1 + e 5) � 1) *(Val(FLOAT_FFT_IMAGE x N k(p � 2 ** (Log_2 N � 1 � k))) �FFT_IMAGE x N k p) * OMEGA_IMAGE k p N,((1 + e 8) * (1 + e 9) * (1 + e 10) � 1) *(Val(FLOAT_FFT_IMAGE x N k(p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_IMAGE x N k p)) * OMEGA_REAL k p N �((1 + e 11) * (1 + e 12) * (1 + e 10) � 1) *(Val(FLOAT_FFT_REAL x N k(p � 2 ** (Log_2 N � 1 � k))) �Val (FLOAT_FFT_REAL x N k p)) *OMEGA_IMAGE k p N)))115

4.3 FFT Design Implementation Veri� ationIn this se tion, we des ribe the appli ation of the proposed approa h for the veri� ationin HOL of the transition from real, oating- and �xed-point spe i� ations to RTL andgate level netlist implementations of an FFT algorithm. We have hosen the ase studyof a radix-4 pipelined 16-point omplex FFT ore available as a VHDL RTL model inthe Xilinx Coregen library [76℄. We have also used Synopsys tools to generate the gatelevel netlist of the design. All proofs have been ondu ted in HOL, hen e establishing a orre tness of the FFT design implementation with respe t to its high level algorithmi spe i� ations.Figure 4.5 shows the overall blo k diagram of the Radix-4 16-point pipelined FFTdesign. The basi elements are memories, delays, multiplexers, and dragon ies. In gen-eral, the 16-point pipelined FFT requires the al ulation of two radix-4 dragon y ranks.Ea h radix-4 dragon y is a su essive ombination of a radix-4 butter y with four twid-dle fa tor multipliers. The FFT ore a epts naturally ordered data on the input busesin a ontinuous stream, performs a omplex FFT, and streams out the DFT samples onthe output buses in a natural order. These buses are respe tively the real and imaginary omponents of the input and output sequen es. An internal input data memory ontrollerorders the data into blo ks to be presented to the FFT pro essor. The twiddle fa tors arestored in oeÆ ient memories. The real and imaginary omponents of omplex input andoutput samples and the phase fa tors are represented as 16-bit 2's omplement numbers.The uns rambling operation is performed using the output bit-reversing bu�er.STAGE 1

CONTROLCONTROL

MemoryBufferInput

Bitreverse

OutputBuffer

D

OUTPUTINPUT

CONTROLMemory

CoefficientMemory

CoefficientCONTROL

D

STAGE 2

XUM

YALE

XUM

YALE

DragonflyRadix_4

DragonflyRadix_4

Figure 4.5: Radix-4 16-point pipelined FFT implementationTo de�ne the radix-4 FFT algorithm [9, 59℄, we represent the indi es p and n in116

equation (4.1) in a base 4 (quaternary number system) asp = 4p1 + p0; p1; p0 = 0; 1; 2; 3 (4.26)n = 4n1 + n0; n1; n0 = 0; 1; 2; 3 (4.27)It is easy to verify that as n0 and n1 take on all possible values in the range indi ated,n goes through all possible values from 0 to 15 with no values repeated. This is also truefor the frequen y index p. Using these index mappings, we an express the radix-4 16-pointFFT algorithm re ursively asA1(p0; n0) = 3Xn1=0x(n1; n0) (W16)4p0n1 (4.28)A2(p0; p1) = 3Xn0=0A1(p0; n0) (W16)(4p1+p0)n0 (4.29)The �nal result an be written asA(p1; p0) = A2(p0; p1) (4.30)Thus, as in the radix-2 algorithm, the results are in reversed order. Based onequations (4.28), (4.29), and (4.30) we an develop a signal owgraph for the radix-416-point FFT algorithm as shown in Figure 4.6, whi h is an expanded version of thepipelined implementation of Figure 4.5. The graph is omposed of two su essive radix-4dragon y stages. To alleviate onfusion in this graph we have shown only one of theradix-4 butter ies in the �rst stage. Also, we have not shown the multipliers for theradix-4 butter ies in the se ond stage sin e they are similar to the representative butter yof the �rst stage. Figure 4.6 also illustrates the uns rambling pro edure for the radix-4algorithm.In HOL, we �rst modeled the RTL des ription of a radix-4 butter y as a predi atein higher-order logi (radix 4 butter y RTL).117

W 016W 316W 216W 116

Radix-4 Dragon y Radix-4 Dragon y

A (0)A (4)A (8)A (12)A (1)A (5)A (9)A (13)A (2)A (6)A (10)A (14)A (3)A (7)A (11)A (15) ReversingBit

jW 216

x (4)x (3)x (2)x (1)W 016

x (0)fx(p)g = fA0(p)gInput fA1(p)g fA2(p)g = fA(p�)g fA(p)gOutputUns rambled

�1�1�1�j

�1�jj

W 916W 616W 316W 016W 616W 416

A (14)A (13)A (12)A (11)A (9)A (8)A (7)A (6)A (4)A (3)A (2)A (1)

A (10)

A (5)

A (0)

A (15)x (15)x (14)x (13)x (12)x (11)x (10)x (9)x (8)x (7)x (6)x (5)

Fa torsTwiddleButter yRadix-4 Fa torsTwiddleButter yRadix-4Figure 4.6: Signal owgraph of radix-4 16-point FFT`def radix_4_butterfly_RTL N ar ai br bi r i dr di q1r q1i q2r q2i q3rq3i q4r q4i =(9 y1r y1i y2r y2i.N_ omplex_add_RTL N ar ai br bi y1r y1i ^N_ omplex_add_RTL N r i dr di y2r y2i ^N_ omplex_add_RTL N y1r y1i y2r y2i q1r q1i) ^(9 y3r y3i y4r y4i y5r y5i y6r y6i.N_ omplex_mul_two_ omp_RTL N br bi (NBWORD N 0) (NBWORD N 1) y3r y3i ^N_ omplex_sub_RTL N ar ai y3r y3i y4r y4i ^N_ omplex_mul_two_ omp_RTL N dr di (NBWORD N 0) (NBWORD N 1) y5r y5i ^N_ omplex_sub_RTL N y5r y5i r i y6r y6i ^N_ omplex_add_RTL N y4r y4i y6r y6i q2r q2i) ^(9 y7r y7i y8r y8i.N_ omplex_sub_RTL N ar ai br bi y7r y7i ^N_ omplex_sub_RTL N r i dr di y8r y8i ^N_ omplex_add_RTL N y7r y7i y8r y8i q3r q3i) ^118

9 y9r y9i y10r y10i y11r y11i y12r y12i.N_ omplex_mul_two_ omp_RTL N br bi (NBWORD N 0) (NBWORD N 1) y9r y9i ^N_ omplex_add_RTL N ar ai y9r y9i y10r y10i ^N_ omplex_mul_two_ omp_RTL N dr di (NBWORD N 0) (NBWORD N 1) y11r y11i ^N_ omplex_add_RTL N y11r y11i r i y12r y12i ^N_ omplex_sub_RTL N y10r y10i y12r y12i q4r q4iThe blo k takes a ve tor of four omplex input data and performs the operationsas depi ted in the owgraph of Figure 4.6, to generate a ve tor of four omplex outputsignals. The real and imaginary parts of the input and output signals are represented as16-bit Boolean words. We de�ned separate fun tions in HOL for arithmeti operationssu h as addition (N omplex add RTL), subtra tion (N omplex sub RTL), and multipli a-tion (N omplex mul two omp RTL) on omplex two's omplement 16-bit Boolean words.Then, we built the omplete butter y stru ture using a proper ombination of these prim-itive operations.Thereafter, we des ribed a radix-4 dragon y blo k (radix 4 dragon y RTL) as a onjun tion of a radix-4 butter y and four 16-bit twiddle fa tor omplex multipliers asshown in Figure 4.6.`def radix_4_dragonfly_RTL N ar ai br bi r i dr di wr wiq1r q1i q2r q2i q3r q3i q4r q4i =9 s1 s2 s3 s4 s5 s6 s7 s8.radix_4_butterfly_RTL N ar ai br bi r i dr dis1 s2 s3 s4 s5 s6 s7 s8 ^N_ omplex_mul_two_ omp_RTL N s1 s2 (wr 1) (wi 1) q1r q1i ^N_ omplex_mul_two_ omp_RTL N s3 s4 (wr 2) (wi 2) q2r q2i ^N_ omplex_mul_two_ omp_RTL N s5 s6 (wr 3) (wi 3) q3r q3i ^N_ omplex_mul_two_ omp_RTL N s7 s8 (wr 4) (wi 4) q4r q4iFinally, we modeled the omplete RTL des ription of the radix-4 16-point FFTstru ture (radix 4 16 point DIF FFT RTL) in HOL. The FFT blo k is de�ned as a on-jun tion of 8 instantiations of radix-4 dragon y blo ks a ording to Figure 4.6, by applyingthe proper time instan es of the input and output signals to ea h blo k.119

`def radix_4_16_point_DIF_FFT_RTL xr xi ar ai wr wi =9 a1r a1i.radix_4_dragonfly_RTL N (xr 0) (xi 0) (xr 4) (xi 4) (xr 8) (xi 8)(xr 12) (xi 12) (wr 0) (wi 0) (a1r 0) (a1i 0) (a1r 4) (a1i 4)(a1r 8) (a1i 8) (a1r 12) (a1i 12) ^radix_4_dragonfly_RTL N (xr 1) (xi 1) (xr 5) (xi 5) (xr 9) (xi 9)(xr 13) (xi 13) (wr 1) (wi 1) (a1r 1) (a1i 1) (a1r 5) (a1i 5)(a1r 9) (a1i 9) (a1r 13) (a1i 13) ^radix_4_dragonfly_RTL N (xr 2) (xi 2) (xr 6) (xi 6) (xr 10)(xi 10) (xr 14) (xi 14) (wr 2) (wi 2) (a1r 2) (a1i 2) (a1r 6)(a1i 6) (a1r 10) (a1i 10) (a1r 14) (a1i 14) ^radix_4_dragonfly_RTL N (xr 3) (xi 3) (xr 7) (xi 7) (xr 11)(xi 11) (xr 15) (xi 15) (wr 3) (wi 3) (a1r 3) (a1i 3) (a1r 7)(a1i 7) (a1r 11) (a1i 11) (a1r 15) (a1i 15) ^radix_4_dragonfly_RTL N (a1r 0) (a1i 0) (a1r 1) (a1i 1) (a1r 2)(a1i 2) (a1r 3) (a1i 3) (wr 4) (wi 4) (ar 0) (ai 0) (ar 4)(ai 4) (ar 8) (ai 8) (ar 12) (ai 12) ^radix_4_dragonfly_RTL N (a1r 4) (a1i 4) (a1r 5) (a1i 5) (a1r 6)(a1i 6) (a1r 7) (a1i 7) (wr 5) (wi 5) (ar 1) (ai 1) (ar 5)(ai 5) (ar 9) (ai 9) (ar 13) (ai 13) ^radix_4_dragonfly_RTL N (a1r 8) (a1i 8) (a1r 9) (a1i 9) (a1r 10)(a1i 10) (a1r 11) (a1i 11) (wr 6) (wi 6) (ar 2) (ai 2) (ar 6)(ai 6) (ar 10) (ai 10) (ar 14) (ai 14) ^radix_4_dragonfly_RTL N (a1r 12) (a1i 12) (a1r 13) (a1i 13)(a1r 14) (a1i 14) (a1r 15) (a1i 15) (wr 7) (wi 7) (ar 3) (ai 3)(ar 7) (ai 7) (ar 11) (ai 11) (ar 15) (ai 15)Following similar steps, we des ribed a radix-4 16-point FFT stru ture as �xed-point (radix 4 16 point DIF FFT fxp), oating-point (radix 4 16 point DIF FFT oat),and real (radix 4 16 point DIF FFT real) domains in HOL using the orresponding om-plex data types and arithmeti operations.For the formal veri� ation of the ase study of the radix-4 de imation in frequen yFFT algorithm based on the ommutating diagram of Figure 1.2, we proved that the FFT120

RTL des ription implies the orresponding �xed-point model (Lemma 5 ).Lemma 5:8 N xr xi ar ai wr wi.radix_4_16_point_DIF_FFT_RTL N xr xi ar ai wr wi =)radix_4_16_point_DIF_FFT_FXP N (FXP_VECT_COMPLEX N xr xi)(FXP_VECT_COMPLEX N ar ai) (FXP_VECT_COMPLEX N wr wi)The proof of the FFT blo k is then broken down into the orresponding proof ofthe dragon y blo k, whi h itself is broken down to the proof of butter y and primitivearithmeti operations.Lemma 6:8 N ar ai br bi r i dr di q1r q1i q2r q2i q3r q3i q4r q4i wr wi.radix_4_dragonfly_RTL ar ai br bi r i dr di wr wi q1r q1i q2rq2i q3r q3i q4r q4i =)radix_4_dragonfly_FXP (N,N � 1,1) (fxp_ omplex (FXP N ar,FXP N ai))(fxp_ omplex (FXP N br,FXP N bi)) (fxp_ omplex (FXP N r,FXP N i))(fxp_ omplex (FXP N dr,FXP N di)) (FXP_VECT_COMPLEX N wr wi)(fxp_ omplex (FXP N q1r,FXP N q1i)) (fxp_ omplex (FXP N q2r,FXP N q2i))(fxp_ omplex (FXP N q3r,FXP N q3i)) (fxp_ omplex (FXP N q4r,FXP N q4i))We used the data abstra tion fun tions FXP and FXP VECT COMPLEX to on-vert a omplex ve tor of 16-bit two's omplement Boolean words into the orresponding�xed-point ve tor.For the error analysis of the radix-4 de imation in frequen y FFT algorithm andfollowing the dis ussions in Se tion 4.2, we proved the theorems below, whi h state theerror between the real values of, respe tively, the oating-point (Lemma 7 ) and �xed-point(Lemma 8 ) pre ision output samples and the orresponding ideal real spe i� ation. Wealso proved a theorem (Lemma 9 ) on the error from the transition from oating-point to�xed-point spe i� ations.121

Lemma 7:8 N xr xi ar ai wr wi.radix_4_16_point_DIF_FFT_FLOAT N (FLOAT_VECT_COMPLEX N xr xi)(FLOAT_VECT_COMPLEX N ar ai) (FLOAT_VECT_COMPLEX N wr wi) =)radix_4_16_point_DIF_FFT_REAL N (REAL_VECT_COMPLEX N xr xi)(REAL_VECT_COMPLEX N ar ai) (REAL_VECT_COMPLEX N wr wi) ^FLOAT_TO_REAL_FFT_ERROR N xr xi ar ai wr wiLemma 8:8 N xr xi ar ai wr wi.radix_4_16_point_DIF_FFT_FXP N (FXP_VECT_COMPLEX N xr xi)(FXP_VECT_COMPLEX N ar ai) (FXP_VECT_COMPLEX N wr wi) =)radix_4_16_point_DIF_FFT_REAL N (REAL_VECT_COMPLEX N xr xi)(REAL_VECT_COMPLEX N ar ai) (REAL_VECT_COMPLEX N wr wi) ^FXP_TO_REAL_FFT_ERROR N xr xi ar ai wr wiLemma 9:8 N xr xi ar ai wr wi.radix_4_16_point_DIF_FFT_FXP N (FXP_VECT_COMPLEX N xr xi)(FXP_VECT_COMPLEX N ar ai) (FXP_VECT_COMPLEX N wr wi) =)radix_4_16_point_DIF_FFT_FLOAT N (FLOAT_VECT_COMPLEX N xr xi)(FLOAT_VECT_COMPLEX N ar ai) (FLOAT_VECT_COMPLEX N wr wi) ^FLOAT_TO_FXP_FFT_ERROR N xr xi ar ai wr wiA ording to these theorems, the oating-point and �xed-point implementationsand the real spe i� ation of a radix-4 de imation in frequen y FFT algorithm are relatedto ea h other based on the orresponding data abstra tion (FLOAT VECT COMPLEX,FXP VECT COMPLEX, REAL VECT COMPLEX ), and error analysis (FLOAT TO REAL FFT ERROR, FXP TO REAL FFT ERROR, FLOAT TO FXP FFT ERROR)fun tions. These errors are already quanti�ed using the theorems mentioned in Se tion4.2. Finally, using the obtained theorems (Lemma 5, Lemma 8 ), we an easily dedu eour ultimate theorem (Lemma 10 ) proving the orre tness of the real spe i� ation fromthe RTL implementation, taking into a ount the error analysis omputed beforehand.122

Lemma 10:8 N xr xi ar ai wr wi.radix_4_16_point_DIF_FFT_RTL N xr xi ar ai wr wi =)radix_4_16_point_DIF_FFT_REAL N (REAL_VECT_COMPLEX N xr xi)(REAL_VECT_COMPLEX N ar ai) (REAL_VECT_COMPLEX N wr wi) ^FXP_TO_REAL_FFT_ERROR N xr xi ar ai wr wi4.4 Con lusionIn this hapter, we des ribed a omprehensive methodology for the veri� ation of generi fast Fourier transform algorithms using the HOL theorem prover. We believe this isthe �rst time a omplete formal framework has been proposed for the spe i� ation andveri� ation of the fast Fourier transform algorithms at di�erent levels of abstra tion. Themethodology presented in this hapter opens new avenues in using formal methods forthe veri� ation of digital signal pro essing (DSP) systems as omplement to traditionaltheoreti al (analyti al) and simulation te hniques.

123

Chapter 5Con lusions and Future Work5.1 Con lusionsIn this thesis, we �rst established the formalization of �xed-point arithmeti in the HOLtheorem prover. Unlike oating-point arithmeti , there is no standard for the �xed-point ounterpart. We hen e de�ned in this thesis a omplete ommon set of the �xed-pointarithmeti supported by most DSP tools, in parti ular SPW and SystemC. We started�rst by en oding the �xed-point arithmeti in HOL onsidering di�erent quantizationand over ow modes, as well as ex eption handling. We then proved two main theoremsstating that the operations on �xed-point numbers are losely related to the orrespondingoperations on in�nitely pre ise values, onsidering some error. The error is bounded to a ertain absolute value whi h is a fun tion of the output pre ision. We have also shownby an example how these theorems an be used as a basis for analysis of the quantizationerrors in the design of �xed-point DSP subsystems. The formalization presented in thisthesis an be onsidered as a omplement to the oating-point formalizations whi h arewidely available in the literature. The developed theories have been a epted by the HOLdevelopers to be in luded in the new publi release of HOL.Based on the developed �xed-point theories, we proposed a omprehensive method-ology for the error analysis of generi digital �lters using the HOL theorem prover. Theproposed approa h overs the three basi forms (dire t, parallel and as ade) of realization124

entirely spe i�ed in HOL. We made use of existing theories in HOL on real, IEEE standardbased oating-point, and �xed-point arithmeti to model the ideal �lter spe i� ation andthe orresponding implementations in higher-order logi . We used valuation fun tionsto de�ne the errors as the di�eren es between the real values of the oating-point and�xed-point �lter implementation outputs and the orresponding output of the ideal real�lter spe i� ation. Finally, we established fundamental analysis lemmas as our model toderive expressions for the a umulation of the roundo� error in digital �lters. Relatedwork did exist sin e the late sixties using theoreti al paper-and-pen il proofs and simula-tion te hniques. The authors believe this is the �rst time a omplete formal framework is onsidered using me hani al proofs in HOL for the error analysis of digital �lters.Furthermore, we established a more elaborated methodology for the veri� ation ofgeneri fast Fourier transform algorithms using the HOL theorem prover. The approa h overs the two anoni al forms (de imation-in-time, and de imation-in-frequen y) of re-alization of the FFT algorithm using real, oating-, and �xed-point arithmeti as well astheir RT implementations, ea h entirely spe i�ed in HOL. We proved lemmas to deriveexpressions for the a umulation of roundo� error in oating- and �xed-point designs om-pared to the ideal real spe i� ation. Then we proved that the FFT RTL implementationimplies the orresponding spe i� ation at the �xed-point level using lassi al hierar hi alveri� ation in HOL, hen e bridging the gap between hardware implementation and highlevels of mathemati al spe i� ation. In this work we also have ontributed to the upgradeand appli ation of established real, omplex real, oating- and �xed-point theories in HOLto the analysis of errors due to �nite pre ision e�e ts, and applied them on the realizationof the FFT algorithms. Error analyses using theoreti al paper-and-pen il proofs do existsin e the late sixties while design veri� ation is ex lusively done by simulation te hniques.We believe this is the �rst time a omplete formal framework has been proposed for thespe i� ation and veri� ation of the fast Fourier transform algorithms at di�erent levels ofabstra tion.125

5.2 Future WorkThe methodology presented in this thesis opens new avenues in using formal methods forthe veri� ation of DSP systems as a omplement to the traditional theoreti al (analyti al)and simulation te hniques. There are many opportunities for further work to improve ourapproa h on verifying DSP systems.� Extend the error analysis lemmas to analyse the worst- ase, average, and varian eerrors.� Develop a me hanized theory on the properties of random variables and pro essesfor statisti al error analysis in HOL.� Link HOL with omputer algebra systems (Maple [13℄, Mathemati a [75℄) to reatea sound, reliable, and powerful system for the veri� ation of DSP systems.� Prove the orre tness of automati transitions from oating-point to �xed-pointlevels.� Investigate the veri� ation of omplex wired and wireless ommuni ation systems,whose building blo ks, heavily make use of several instan es of the FFT algorithmssu h as OFDM (Orthogonal Frequen y Division Multiplexing) modems [58℄.

126

Bibliography[1℄ M. D. Aagaard and C. -J. H. Seger, \The Formal Veri� ation of a Pipelined Double-Pre ision IEEE Floating-Point Multiplier," In Pro eedings International Conferen eon Computer Aided Design, pp. 7-10, San Jose, California, USA, November 1995.[2℄ G. Barrett, \Formal Methods Applied to a Floating Point Number System," IEEETransa tions on Software Engineering, SE-15 (5): 611-621, May 1989.[3℄ C. Berg and C. Ja obi, \Formal Veri� ation of the VAMP Floating Point Unit," InCorre t Hardware Design and Veri� ation Methods, LNCS 2144, pp. 325-339, Springer-Verlag, 2001.[4℄ R. Boulton, A. Gordon, M. Gordon, J. Harrison, J. Herbert, and J. Van-Tassel, \Expe-rien e with Embedding Hardware Des ription Languages in HOL," In Theorem Proversin Cir uit Design, pp. 129-156, North-Holland, 1992.[5℄ P. Bjesse, \Automati Veri� ation of Combinational and Pipelined FFT Cir uits," InComputer Aided Veri� ation, LNCS 1633, pp. 380-393, Springer-Verlag, 1999.[6℄ S. Beyer, C. Ja obi, D. Kr�oning, D. Leinenba h, and W. J. Paul, \InstantiatingUninterpreted Fun tional Units and Memory System: Fun tional Veri� ation of theVAMP," In Corre t Hardware Design and Veri� ation Methods, LNCS 2860, pp. 51-65,Springer-Verlag, 2003.[7℄ S. Boldo and M. Daumas, \Properties of Two's Complement Floating Point Nota-tions," Software Tools for Te hnology Transfer, 5 (2-3): 237-246, Mar h 2004.127

[8℄ S. Boldo, M. Daumas, and L. Th�ery, \Formal Proofs and Computations in FinitePre ision Arithmeti ," In Pro eedings of the 11th Symposium on the Integration ofSymboli Computation and Me hanized Reasoning, pp. 101-111, Rome, Italy, Septem-ber 2003.[9℄ E. O. Brigham, \The Fast Fourier Transform," Prenti e Hall, 1974.[10℄ V. Capretta, \Certifying the Fast Fourier Transform with Coq," In Theorem Provingin Higher Order Logi s, LNCS 2152, pp. 154-168, Springer-Verlag, 2001.[11℄ V. A. Carreno, \Interpretation of IEEE-854 Floating-Point Standard and De�nitionin the HOL System," NASA TM-110189, September 1995.[12℄ Caden e Design Systems, In ., \Signal Pro essing WorkSystem (SPW) User's Guide,"USA, July 1999.[13℄ B. W. Char, K. O. Geddes, G. H. Gonnet, B. L. Leong, M. B. Monagan, S. M. Watt,\A Tutorial Introdu tion to Maple V," Springer-Verlag, 1992.[14℄ Y. -A. Chen and R. E. Bryant, \Veri� ation of Floating Point Adders," In ComputerAided Veri� ation, LNCS 1427, pp. 488-499, Springer-Verlag, 1998.[15℄ Synopsys, In ., \CoCentri TM System Studio User's Guide," USA, Aug. 2001.[16℄ W. T. Co hran et. al., \What is the Fast Fourier Transform," IEEE Transa tions onAudio and Ele troa ousti s, AU-15: 45-55, Jun. 1967.[17℄ M. Cornea-Hasegan, \Proving the IEEE Corre tness of Iterative Floating-PointSquare Root, Divide, and Remainder Algorithms," Intel Te hnology Journal, Q2: 1-11,1998.[18℄ J. W. Cooley and J. W. Tukey, \An Algorithm for Ma hine Cal ulation of ComplexFourier Series," Mathemati s of Computation, 19: 297-301, Apr. 1965.[19℄ M. Daumas, L. Rideau, and L. Th�ery, \A Generi Library for Floating-Point Numbersand Its Appli ation to Exa t Computing," In Theorem Proving in Higher Order Logi s,LNCS 2152, pp. 169-184, Springer-Verlag, 2001.128

[20℄ G. Forsythe and C. B. Moler, \Computer Solution of Linear Algebrai Systems,"Prenti e-Hall, 1967.[21℄ R. A. Gamboa, \The Corre tness of the Fast Fourier Transform: A Stru tural Proofin ACL2," Formal Methods in System Design, Spe ial Issue on UNITY, Jan. 2002.[22℄ W. M. Gentleman and G. Sande, \Fast Fourier Transforms - For Fun and Pro�t," InAFIPS Fall Joint Computer Conferen e, Vol. 29, pp. 563-578, Spartan Books, Wash-ington, DC, 1966.[23℄ M. J. C. Gordon and T. F. Melham, \Introdu tion to HOL: A Theorem ProvingEnvironment for Higher-Order Logi ," Cambridge University Press, 1993.[24℄ M.J.C. Gordon, R. Milner, and C. P. Wadsworth, \Edinburgh LCF: A Me hanisedLogi of Computation," Le ture Notes in Computer S ien e, vol. 78, Springer-Verlag,1979.[25℄ B. Gold, and C. M. Radar, \E�e ts of Quantization Noise in Digital Filters," InPro eedings AFIPS Spring Joint Computer Conferen e, vol. 28, pp. 213-219, 1966.[26℄ M. J. C. Gordon and T. F. Melham, \Introdu tion to HOL: A Theorem ProvingEnvironment for Higher-Order Logi ," Cambridge University Press, 1993.[27℄ J. R. Harrison, \Constru ting the Real Numbers in HOL," Formal Methods in SystemDesign, 5 (1/2): 35-59, 1994.[28℄ J. R. Harrison, \A Ma hine-Che ked Theory of Floating-Point Arithmeti ," In The-orem Proving in Higher Order Logi s, LNCS 1690, pp. 113-130, Springer-Verlag, 1999.[29℄ J. R. Harrison, \Floating-Point Veri� ation in HOL Light: The Exponential Fun -tion," Formal Methods in System Design, 16 (3): 271-305, 2000.[30℄ J. R. Harrison, \Formal Veri� ation of Floating Point Trigonometri Fun tions," InFormal Methods in Computer-Aided Design, LNCS 1954, pp. 217-233, Springer-Verlag,2000. 129

[31℄ J. R. Harrison, \Formal Veri� ation of IA-64 Division Algorithms," In Theorem Prov-ing in Higher Order Logi s, LNCS 1869, pp. 234-251, Springer-Verlag, 2000.[32℄ J. R. Harrison, \Complex Quanti�er Elimination in HOL," In Supplemental Pro eed-ings of the International Conferen e on Theorem Proving in Higher Order Logi s, pp.159-174, Edinburgh, S otland, UK, Sep. 2001.[33℄ J. R. Harrison and L. Th�ery, \A Skepti 's Approa h to Combining Hol and Maple,"Journal of Automated Reasoning, 21: 279-294, 1998.[34℄ M. Huhn, K. S hneider, T. Kropf, and G. Logothetis, \Verifying Impre isely WorkingArithmeti Cir uits," In Pro eedings Design Automation and Test in Europe Confer-en e, pp. 65-69, Muni h, Germany, Mar h 1999.[35℄ The Institute of Ele tri al and Ele troni Engineers, In ., \IEEE, Standard for BinaryFloating-Point Arithmeti ," ANSI/IEEE Standard 754, USA, 1985.[36℄ The Institute of Ele tri al and Ele troni Engineers, In ., \IEEE, Standard for Radix-Independent Floating-Point Arithmeti ," ANSI/IEEE Std 854, USA, 1987.[37℄ L. B. Ja kson, \Roundo�-Noise Analysis for Fixed-Point Digital Filters Realized inCas ade or Parallel Form," IEEE Transa tions on Audio and Ele troa ousti s, AU-18:107-122, June 1970.[38℄ R. Kaivola and M. D. Aagaard, \Divider Cir uit Veri� ation with Model Che kingand Theorem Proving," In Theorem Proving in Higher Order Logi s, LNCS 1869, pp.338-355, Springer-Verlag, 2000.[39℄ J. F. Kaiser, \Digital Filters," In System Analysis by Digital Computer, F. F. Kuoand J. F. Kaiser, Eds., pp. 218-285, Wiley, 1966.[40℄ R. Kaivola and K. R. Kohatsu, \Proof Engineering in the Large: Formal Veri� ationof Pentium r 4 Floating-Point Divider," Software Tools for Te hnology Transfer, 4(3): 323-334, 2003. 130

[41℄ T. Kaneko and B. Liu, \A umulation of Round-O� Error in Fast Fourier Trans-forms," Journal of Asso iation for Computing Ma hinery, 17 (4): 637-654, O t. 1970.[42℄ R. Kaivola and N. Narasimhan, \Formal Veri� ation of the Pentium r 4 Floating-Point Multiplier," In Pro eedings Design Automation and Test in Europe Conferen e,pp. 20-27, Paris, Fran e, Mar h 2002.[43℄ C. Kern and M. Greenstreet, \Formal Veri� ation in Hardware Design: A Survey,"ACM Transa tions on Design Automation of Ele toni Systems, 4: 123-193, April1999.[44℄ J. B. Knowles and R. Edwards, \E�e ts of a Finite-Word-Length Computer in aSampled-Data Feedba k System," IEE Pro eedings, 112: 1197-1207, June 1965.[45℄ H. Keding, M. Willems, M. Coors, and H. Meyr, \FRIDGE: A Fixed-Point Designand Simulation Environment," In Pro eedings Design Automation and Test in EuropeConferen e, pp. 429-435, Paris, Fran e, February 1998.[46℄ M. Leeser and J. O'Leary, \Veri� ation of a Subtra tive Radix-2 Square Root Al-gorithm and Implementation," In Pro eedings International Conferen e on ComputerDesign, pp. 526-531, Austin, Texas, USA, O tober 1995.[47℄ B. Liu and T. Kaneko, \Error Analysis of Digital Filters Realized with Floating-PointArithmeti ," Pro eedings of the IEEE, 57: 1735-1747, O tober 1969.[48℄ B. Liu and T. Kaneko, \Roundo� Error in Fast Fourier Transforms (De imation inTime)," Pro eedings of the IEEE (Pro eedings Letters), 991-992, Jun. 1975.[49℄ Mathworks, In ., \Simulink Referen e Manual," USA, 1996.[50℄ Mathworks, In ., \Fixed-Point Blo kset, For Use with Simulink, User's Guide," USA,2004.[51℄ T. F. Melham, \The HOL pred sets Library," University of Cambridge, ComputerLaboratory, February 1992. 131

[52℄ T. Melham, \Higher Order Logi and Hardware Veri� ation," Cambridge Tra ts inTheoreti al Computer S ien e 31, Cambridge University Press, 1993.[53℄ P. S. Miner and J. F. Leathrum, \Veri� ation of IEEE Compliant Subtra tive DivisionAlgorithms," In Formal Methods in Computer-Aided Design, LNCS 1166, pp. 64-78,Springer-Verlag, 1996.[54℄ P. S. Miner, \De�ning the IEEE-854 Floating-Point Standard in PVS," NASA TM-110167, June 1995.[55℄ J. Misra, \Powerlists: A Stru ture for Parallel Re ursion," In ACM Transa tions onProgramming Languages and Systems, 16 (6): 1737-1767, Nov. 1994.[56℄ J. S. Moore, T. Lyn h, and M. Kaufmann, \A Me hani ally Che ked Proof of theCorre tness of the Kernel of the AMD5K86 Floating-Point Division Algorithm," IEEETransa tions on Computers, 47 (9): 913-926, 1998.[57℄ S. M. Mueller andW. J. Paul, \Computer Ar hite ture. Complexity and Corre tness,"Springer-Verlag, 2000.[58℄ R. V. Nee and R. Prasad, \OFDM for Wireless Multimedia Communi ations," Arte hHouse, Boston, 2000.[59℄ A. V. Oppenheim and R. W. S hafer, \Dis rete-Time Signal Pro essing," Prenti e-Hall, 1989.[60℄ A. V. Oppenheim and C. J. Weinstein, \E�e ts of Finite Register Length in DigitalFiltering and the Fast Fourier Transform," Pro eedings of the IEEE, 60 (8): 957-976,August 1972.[61℄ Open SystemC Initiative, \SystemC Language Referen e Manual," USA, 2004.[62℄ J. O' Leary, X. Zhao, R. Gerth, and C.-J.H. Seger, \Formally Verifying IEEE Com-plian e of Floating-Point Hardware," Intel Te hnology Journal, Q1: 1-14, 1999.[63℄ L.C. Paulson, \ML for the Working Programmer," Cambridge University Press, U.K.,2nd edition, 1996. 132

[64℄ D. M. Russino�, \A Case Study in Formal Veri� ation of Register-Transfer Logi with ACL2: The Floating-Point Adder of the AMD Athlon Pro essor," In FormalMethods in Computer-Aided Design, LNCS 1954, pp. 3-36, Springer-Verlag, 2000.[65℄ J. Sawada and R. Gamboa, \Me hani al Veri� ation of a Square Root Algorithmusing Taylor's Theorem," In Formal Methods in Computer-Aided Design, LNCS 2517,pp. 274-291, Springer-Verlag, 2002.[66℄ I. W. Sandberg, \Floating-Point-Roundo� A umulation in Digital Filter Realiza-tion," The Bell System Te hni al Journal, 46: 1775-1791, O tober 1967.[67℄ C.J.Seger, \An Introdu tion to Formal Hardware Veri� ation," Te hni al Report92-13, Dept. of Computer S ien e, University of British Columbia, Van ouver, B.C.,Canada, June 1992.[68℄ T. Thong and B. Liu, \Fixed-Point Fast Fourier Transform Error Analysis," IEEETransa tions on A ousti s, Spee h, and Signal Pro essing, ASSP 24 (6): 563-573, De .1976.[69℄ University of Cambridge, \The HOL System Referen e," Computer Laboratory, Cam-bridge, UK, Mar h 2004.[70℄ W. Wong, \Modeling Bit Ve tors in HOL: The Word Library," In Higher Order Logi and Its Appli ations, LNCS 780, pp. 371-384, Springer-Verlag, 1994.[71℄ C. Weinstein and A. V. Oppenheim, \A Comparison of Roundo� Noise in FloatingPoint and Fixed Point Digital Filter Realizations," Pro eedings of the IEEE (Pro eed-ings Letters), 57: 1181-1183, June 1969.[72℄ C. J. Weinstein, \Roundo� Noise in Floating Point Fast Fourier Transform Compu-tation," IEEE Transa tions on Audio and Ele troa ousti s, AU-17 (3): 209-215, Sep.1969.[73℄ P. D. Wel h, \A Fixed-Point Fast Fourier Transform Error Analysis," IEEE Trans-a tions on Audio and Ele troa ousti s, AU-17 (2): 151-157, Jun. 1969.133

[74℄ J. H. Wilkinson, \Rounding Errors in Algebrai Pro esses," Prenti e-Hall, 1963.[75℄ S. Wolfram, \Mathemati a, A System for Doing Mathemati s by Computer,"Addison-Wesley, 1988.[76℄ Xilinx, In ., \High-Performan e 16-Point Complex FFT/IFFT V1.0.5, Produ t Spe -i� ation," USA, Jul. 2000, http://xilinx. om/ip enter.

134

Biography� Edu ation{ Con ordia University: Montreal, Quebe , CanadaPh.D andidate, in Ele tri al Engineering, 5/00-present{ Sharif University of Te hnology: Tehran, IranM.S ., in Ele tri al Engineering, 9/94 - 3/97{ Shiraz University: Shiraz, IranB.S ., in Ele tri al Engineering, 3/88 - 9/93� Work Experien e{ Resear h Assistant: 5/00-presentHardware Veri� ation Group (HVG), Con ordia University{ Design Engineer: 9/98-4/00Emad Semi on. Co. Ltd., Tehran, Iran� Publi ations{ Journal Papers1. B. Akbarpour, S. Tahar, and A. Dekdouk, \Formalization of Fixed-PointArithmeti in HOL," To appear in Formal Methods in Systems Design,Springer-Verlag. [33 pages℄2. B. Akbarpour and S. Tahar, \Error Analysis of Digital Filters using HOLTheorem Proving," Submitted to IEEE Transa tions on Cir uits and Sys-tems I. [35 pages℄3. B. Akbarpour and S. Tahar, \An Approa h for the Formal Veri� ation ofFFT Algorithms using Theorem Proving," Submitted to IEEE Transa -tions on CAD of Integrated Cir uits and Systems. [24 pages℄{ Conferen e Papers 135

1. B. Akbarpour and S. Tahar, \A Methodology for the Formal Veri� a-tion of FFT Algorithms in HOL," In Formal Methods in Computer-AidedDesign, LNCS 3312, pp. 37-51, Springer-Verlag, 2004.2. B. Akbarpour and S. Tahar, \Error Analysis of Digital Filters using The-orem Proving," In Theorem Proving in Higher Order Logi s, LNCS 3223,pp. 1-16, Springer-Verlag, 2004.3. B. Akbarpour and S. Tahar, \Modeling SystemC Fixed-Point Arithmeti in HOL," In Formal Methods and Software Engineering, LNCS 2885, pp.206-225, Springer-Verlag, 2003.4. B. Akbarpour and S. Tahar, \The Appli ation of Formal Veri� ation toSPW Designs," In Pro eedings Euromi ro Symposium on Digital Sys-tem Design, IEEE Computer So iety Press, pp. 325 -332, Belek, Turkey,September 2003.5. B. Akbarpour, S. Tahar, and A. Dekdouk, \Formalization of Caden eSPW Fixed-Point Arithmeti in HOL," In Integrated Formal Methods,LNCS 2335, pp. 185-204, Springer-Verlag, 2002.{ Te hni al Reports1. B. Akbarpour and S. Tahar, \Veri� ation of the Fast Fourier Transformusing HOL Theorem Proving;," Te hni al Report, Con ordia University,Department of Ele tri al and Computer Engineering, Mar h 2004. [40pages℄2. B. Akbarpour and S. Tahar, \Error Analysis of Digital Filters using HOLTheorem Proving," Te hni al Report, Con ordia University, Departmentof Ele tri al and Computer Engineering, February 2004. [36 pages℄3. B. Akbarpour, S. Tahar, \Formalization of Fixed-Point Arithmeti inHOL," Te hni al Report, Con ordia University, Department of Ele tri- al and Computer Engineering, September 2002. [21 pages℄136