stepanov rose from.mathematics.to.generic.programming
Post on 13-Dec-2015
17 Views
Preview:
DESCRIPTION
TRANSCRIPT
AboutThiseBook
ePUBisanopen,industry-standardformatforeBooks.However,supportofePUBanditsmanyfeaturesvariesacrossreadingdevicesandapplications.Useyourdeviceorappsettingstocustomizethepresentationtoyourliking.Settingsthatyoucancustomizeoftenincludefont,fontsize,singleordoublecolumn,landscapeorportraitmode,andfiguresthatyoucanclickortaptoenlarge.Foradditionalinformationaboutthesettingsandfeaturesonyourreadingdeviceorapp,visitthedevicemanufacturer’sWebsite.Manytitlesincludeprogrammingcodeorconfigurationexamples.Tooptimizethepresentationofthese
elements,viewtheeBookinsingle-column,landscapemodeandadjustthefontsizetothesmallestsetting.Inadditiontopresentingcodeandconfigurationsinthereflowabletextformat,wehaveincludedimagesofthecodethatmimicthepresentationfoundintheprintbook;therefore,wherethereflowableformatmaycompromisethepresentationofthecodelisting,youwillseea“Clickheretoviewcodeimage”link.Clickthelinktoviewtheprint-fidelitycodeimage.Toreturntothepreviouspageviewed,clicktheBackbuttononyourdeviceorapp.
FromMathematicstoGenericProgramming
AlexanderA.StepanovDanielE.Rose
UpperSaddleRiver,NJ•Boston•Indianapolis•SanFranciscoNewYork•Toronto•Montreal•London•Munich•Paris•Madrid
Capetown•Sydney•Tokyo•Singapore•MexicoCity
Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirproductsareclaimedastrademarks.Wherethosedesignationsappearinthisbook,andthepublisherwasawareofatrademarkclaim,thedesignationshavebeenprintedwithinitialcapitallettersorinallcapitals.
Theauthorsandpublisherhavetakencareinthepreparationofthisbook,butmakenoexpressedorimpliedwarrantyofanykindandassumenoresponsibilityforerrorsoromissions.Noliabilityisassumedforincidentalorconsequentialdamagesinconnectionwithorarisingoutoftheuseoftheinformationorprogramscontainedherein.
Forinformationaboutbuyingthistitleinbulkquantities,orforspecialsalesopportunities(whichmayincludeelectronicversions;customcoverdesigns;andcontentparticulartoyourbusiness,traininggoals,marketingfocus,orbrandinginterests),pleasecontactourcorporatesalesdepartmentatcorpsales@pearsoned.comor(800)382-3419.
Forgovernmentsalesinquiries,pleasecontactgovernmentsales@pearsoned.com.
ForquestionsaboutsalesoutsidetheUnitedStates,pleasecontactinternational@pearsoned.com.
VisitusontheWeb:informit.com/aw
LibraryofCongressCataloging-in-PublicationData
Stepanov,AlexanderA.Frommathematicstogenericprogramming/AlexanderA.Stepanov,DanielE.Rose.pagescmIncludesbibliographicalreferencesandindex.ISBN978-0-321-94204-3(pbk.:alk.paper)1.Genericprogramming(Computerscience)—Mathematics.2.Computeralgorithms.I.Rose,DanielE.II.Title.QA76.6245.S742015005.1'1—dc232014034539
Copyright©2015PearsonEducation,Inc.
Allrightsreserved.PrintedintheUnitedStatesofAmerica.Thispublicationisprotectedbycopyright,andpermissionmustbeobtainedfromthepublisherpriortoanyprohibitedreproduction,storageinaretrievalsystem,ortransmissioninanyformorbyanymeans,electronic,mechanical,photocopying,recording,orlikewise.Toobtainpermissiontousematerialfromthiswork,pleasesubmitawrittenrequesttoPearsonEducation,Inc.,PermissionsDepartment,OneLakeStreet,UpperSaddleRiver,NewJersey07458,oryoumayfaxyourrequestto(201)236-3290.
Photocreditsarelistedonpage293.
ISBN-13:978-0-321-94204-3ISBN-10:0-321-94204-3TextprintedintheUnitedStatesonrecycledpaperatRRDonnelleyinCrawfordsville,Indiana.Firstprinting,November2014
Contents
AcknowledgmentsAbouttheAuthorsAuthors’Note
1WhatThisBookIsAbout1.1ProgrammingandMathematics1.2AHistoricalPerspective1.3Prerequisites1.4Roadmap
2TheFirstAlgorithm2.1EgyptianMultiplication2.2ImprovingtheAlgorithm2.3ThoughtsontheChapter
3AncientGreekNumberTheory3.1GeometricPropertiesofIntegers3.2SiftingPrimes3.3ImplementingandOptimizingtheCode3.4PerfectNumbers3.5ThePythagoreanProgram3.6AFatalFlawintheProgram3.7ThoughtsontheChapter
4Euclid’sAlgorithm4.1AthensandAlexandria4.2Euclid’sGreatestCommonMeasureAlgorithm4.3AMillenniumwithoutMathematics4.4TheStrangeHistoryofZero4.5RemainderandQuotientAlgorithms4.6SharingtheCode4.7ValidatingtheAlgorithm4.8ThoughtsontheChapter
5TheEmergenceofModernNumberTheory5.1MersennePrimesandFermatPrimes5.2Fermat’sLittleTheorem
5.3Cancellation5.4ProvingFermat’sLittleTheorem5.5Euler’sTheorem5.6ApplyingModularArithmetic5.7ThoughtsontheChapter
6AbstractioninMathematics6.1Groups6.2MonoidsandSemigroups6.3SomeTheoremsaboutGroups6.4SubgroupsandCyclicGroups6.5Lagrange’sTheorem6.6TheoriesandModels6.7ExamplesofCategoricalandNon-categoricalTheories6.8ThoughtsontheChapter
7DerivingaGenericAlgorithm7.1UntanglingAlgorithmRequirements7.2RequirementsonA7.3RequirementsonN7.4NewRequirements7.5TurningMultiplyintoPower7.6GeneralizingtheOperation7.7ComputingFibonacciNumbers7.8ThoughtsontheChapter
8MoreAlgebraicStructures8.1Stevin,Polynomials,andGCD8.2GöttingenandGermanMathematics8.3NoetherandtheBirthofAbstractAlgebra8.4Rings8.5MatrixMultiplicationandSemirings8.6Application:SocialNetworksandShortestPaths8.7EuclideanDomains8.8FieldsandOtherAlgebraicStructures8.9ThoughtsontheChapter
9OrganizingMathematicalKnowledge9.1Proofs
9.2TheFirstTheorem9.3EuclidandtheAxiomaticMethod9.4AlternativestoEuclideanGeometry9.5Hilbert’sFormalistApproach9.6PeanoandHisAxioms9.7BuildingArithmetic9.8ThoughtsontheChapter
10FundamentalProgrammingConcepts10.1AristotleandAbstraction10.2ValuesandTypes10.3Concepts10.4Iterators10.5IteratorCategories,Operations,andTraits10.6Ranges10.7LinearSearch10.8BinarySearch10.9ThoughtsontheChapter
11PermutationAlgorithms11.1PermutationsandTranspositions11.2SwappingRanges11.3Rotation11.4UsingCycles11.5Reverse11.6SpaceComplexity11.7Memory-AdaptiveAlgorithms11.8ThoughtsontheChapter
12ExtensionsofGCD12.1HardwareConstraintsandaMoreEfficientAlgorithm12.2GeneralizingStein’sAlgorithm12.3Bézout’sIdentity12.4ExtendedGCD12.5ApplicationsofGCD12.6ThoughtsontheChapter
13AReal-WorldApplication13.1Cryptology
13.2PrimalityTesting13.3TheMiller-RabinTest13.4TheRSAAlgorithm:HowandWhyItWorks13.5ThoughtsontheChapter
14ConclusionsFurtherReading
ANotation
BCommonProofTechniquesB.1ProofbyContradictionB.2ProofbyInductionB.3ThePigeonholePrinciple
CC++forNon-C++ProgrammersC.1TemplateFunctionsC.2ConceptsC.3DeclarationSyntaxandTypedConstantsC.4FunctionObjectsC.5Preconditions,Postconditions,andAssertionsC.6STLAlgorithmsandDataStructuresC.7IteratorsandRangesC.8TypeAliasesandTypeFunctionswithusinginC++11C.9InitializerListsinC++11C.10LambdaFunctionsinC++11C.11ANoteaboutinline
BibliographyIndex
Acknowledgments
Wewouldliketothankallthepeoplewhocontributedtomakingthisbookareality.OurmanagementatA9.comactivelysupportedthisprojectfromthebeginning.BillStasiorinitiatedthecreationofthecoursethisbookisbasedon,andselectedthetopicfromamongseveraloptionsweoffered.BrianPinkertonnotonlyattendedthecourse,butalsostronglyencouragedourideaofturningthematerialintoabook.WealsowouldliketothankMatMarcus,whocollaboratedwithAlexonasimilarcourseatAdobein2004–2005.TheothermembersoftheFundamentalDataStructuresandAlgorithmsforSearchteamplayed
importantrolesthroughouttheprocess.AnilGangollihelpedshapethecontentofthecourse,RyanErnstprovidedmuchoftheprogramminginfrastructure,andParamjitOberoigaveinvaluablefeedbackduringthewritingstage.Wehaveenjoyedworkingwithallofthemandaregratefulfortheirinput.Wearegratefultooureditors,PeterGordonandGregDoench,andtotheteamofexpertsassembledby
Addison-Wesley,includingmanagingeditorJohnFuller,productioneditorMaryKeselWilson,copyeditorJillHobbs,andcompositor/LaTeXexpertLoriHughesforalltheirworkinturningourroughmanuscriptintoapolishedbook.Finally,we’dliketothankthemanyfriends,familymembers,andcolleagueswhoreadearlierdraftsof
thebookand/orgaveuscomments,corrections,suggestions,advice,orotherhelp:GašperAžman,JohnBanning,CynthiaDwork,HernanEpelman,RyanErnst,AnilGangolli,SusanGruber,JonKalb,RobertLehr,DmitryLeshchiner,TomLondon,MarkManasse,PaulMcJones,NicolasNicolov,GorNishanov,ParamjitOberoi,SeanParent,FernandoPelliccioni,JohnReiser,RobertRose,StefanVargyas,andAdamYoung.Thebookismuchbetterasaresultoftheircontributions.
AbouttheAuthors
AlexanderA.StepanovstudiedmathematicsatMoscowStateUniversityfrom1967to1972.Hehasbeenprogrammingsince1972:firstintheSovietUnionand,afteremigratingin1977,intheUnitedStates.Hehasprogrammedoperatingsystems,programmingtools,compilers,andlibraries.HisworkonfoundationsofprogramminghasbeensupportedbyGE,PolytechnicUniversity,BellLabs,HP,SGI,Adobe,and,since2009,A9.com,Amazon’ssearchtechnologysubsidiary.In1995hereceivedtheDr.Dobb’sJournalExcellenceinProgrammingAwardforthedesignoftheC++StandardTemplateLibrary.DanielE.RoseisaresearchscientistwhohasheldmanagementpositionsatApple,AltaVista,Xigo,Yahoo,andA9.com.Hisresearchfocusesonallaspectsofsearchtechnology,rangingfromlow-levelalgorithmsforindexcompressiontohuman-computerinteractionissuesinwebsearch.RoseledtheteamatApplethatcreateddesktopsearchfortheMacintosh.HeholdsaPh.D.incognitivescienceandcomputersciencefromUniversityofCalifornia,SanDiego,andaB.A.inphilosophyfromHarvardUniversity.
Authors’Note
Theseparationofcomputersciencefrommathematicsgreatlyimpoverishesboth.Thelecturesthatthisbookisbasedonweremyattempttoshowhowthesetwoactivities—anancientonegoingbacktotheverybeginningsofourcivilizationandthemostmodernone—canbebroughttogether.IwasveryfortunatethatmyfriendDanRose,underwhosemanagementourteamwasapplying
principlesofgenericprogrammingtosearchenginedesign,agreedtoconvertmyrathermeanderinglecturesintoacoherentbook.Bothofushopethatourreaderswillenjoytheresultofourcollaboration.
—A.A.S.
Thebookyouareabouttoreadisbasedonnotesfroman“AlgorithmicJourneys”coursetaughtbyAlexStepanovatA9.comduring2012.ButasAlexandIworkedtogethertotransformthematerialintobookform,werealizedthattherewasastrongerstorywecouldtell,onethatcenteredongenericprogramminganditsmathematicalfoundations.Thisledtoamajorreorganizationofthetopics,andremovaloftheentiresectiononsettheoryandlogic,whichdidnotseemtobepartofthesamestory.Atthesametime,weaddedandremoveddetailstocreateamorecoherentreadingexperienceandtomakethematerialmoreaccessibletolessmathematicallyadvancedreaders.WhileAlexcomesfromamathematicalbackground,Idonot.I’vetriedtolearnfrommyownstruggles
tounderstandsomeofthematerialandtousethisexperiencetoidentifyideasthatrequireadditionalexplanation.Ifinsomecaseswedescribesomethinginaslightlydifferentwaythanamathematicianwould,orusingslightlydifferentterminology,orusingmoresimplesteps,thefaultismine.
—D.E.R.
1.WhatThisBookIsAbout
Itisimpossibletoknowthingsofthisworldunlessyouknowmathematics.
RogerBacon,OpusMajus
Thisbookisaboutprogramming,butitisdifferentfrommostprogrammingbooks.Alongwithalgorithmsandcode,you’llfindmathematicalproofsandhistoricalnotesaboutmathematicaldiscoveriesfromancienttimestothe20thcentury.Morespecifically,thebookisaboutgenericprogramming,anapproachtoprogrammingthatwas
introducedinthe1980sandstartedtobecomepopularfollowingthecreationoftheC++StandardTemplateLibrary(STL)inthe1990s.Wemightdefineitlikethis:
Definition1.1.Genericprogrammingisanapproachtoprogrammingthatfocusesondesigningalgorithmsanddatastructuressothattheyworkinthemostgeneralsettingwithoutlossofefficiency.
Ifyou’veusedSTL,atthispointyoumaybethinking,“Waitaminute,that’sallthereistogenericprogramming?Whataboutallthatstuffabouttemplatesanditeratortraits?”Thosearetoolsthatenablethelanguagetosupportgenericprogramming,andit’simportanttoknowhowtousethemeffectively.Butgenericprogrammingitselfismoreofanattitudetowardprogrammingthanaparticularsetoftools.Webelievethatthisattitude—tryingtowritecodeinthisgeneralway—isonethatallprogrammers
shouldembrace.Thecomponentsofawell-writtengenericprogramareeasiertouseandmodifythanthoseofaprogramwhosedatastructures,algorithms,andinterfaceshardcodeunnecessaryassumptionsaboutaspecificapplication.Makingaprogrammoregenericrendersitsimultaneouslybothsimplerandmorepowerful.
1.1ProgrammingandMathematicsSowheredoesthisgenericprogrammingattitudecomefrom,andhowdoyoulearnit?Itcomesfrommathematics,andespeciallyfromabranchofmathematicscalledabstractalgebra.Tohelpyouunderstandtheapproach,thisbookwillintroduceyoutoalittlebitofabstractalgebra,whichfocusesonhowtoreasonaboutobjectsintermsofabstractpropertiesofoperationsonthem.It’satopicnormallystudiedonlybyuniversitystudentsmajoringinmath,butwebelieveit’scriticalinunderstandinggenericprogramming.Infact,itturnsoutthatmanyofthefundamentalideasinprogrammingcamefrommathematics.Learning
howtheseideascameintobeingandevolvedovertimecanhelpyouthinkaboutsoftwaredesign.Forexample,Euclid’sElements,abookwrittenmorethan2000yearsago,isstilloneofthebestexamplesofhowtobuildupacomplexsystemfromsmall,easilyunderstoodpieces.Althoughtheessenceofgenericprogrammingisabstraction,abstractionsdonotspringintoexistence
fullyformed.Toseehowtomakesomethingmoregeneral,youneedtostartwithsomethingconcrete.Inparticular,youneedtounderstandthespecificsofaparticulardomaintodiscovertherightabstractions.Theabstractionsthatappearinabstractalgebralargelycomefromconcreteresultsinoneoftheoldest
branchesofmathematics,callednumbertheory.Forthisreason,wewillalsointroducesomekeyideasfromnumbertheory,whichdealswithpropertiesofintegers,especiallydivisibility.Thethoughtprocessyou’llgothroughinlearningthismathcanimproveyourprogrammingskills.But
we’llalsoshowhowsomeofthemathematicalresultsthemselvesturnouttobecrucialtosomemodernsoftwareapplications.Inparticular,bytheendofthebookwe’llshowhowsomeoftheseresultsareusedincryptographicprotocolsunderlyingonlineprivacyandonlinecommerce.Thebookwillmovebackandforthbetweentalkingaboutmathandtalkingaboutprogramming.In
particular,we’llinterweaveimportantideasinmathematicswithadiscussionofbothspecificalgorithmsandgeneralprogrammingtechniques.We’llmentionsomealgorithmsonlybriefly,whileotherswillberefinedandgeneralizedthroughoutthebook.Acoupleofchapterswillcontainonlymathematicalmaterial,andacouplewillcontainonlyprogrammingmaterial,butmosthaveamixtureofboth.
1.2AHistoricalPerspectiveWe’vealwaysfoundthatit’seasierandmoreinterestingtolearnsomethingifit’spartofastory.Whatwasgoingonatthetime?Whowerethepeopleinvolved,andhowdidtheycometohavetheseideas?Wasoneperson’sworkanattempttobuildonanother’s—oranattempttorejectwhatcamebefore?Soasweintroducethemathematicalideasinthisbook,we’lltrytotellyouthestoryofthoseideasandofthepeoplewhocameupwiththem.Inmanycases,we’veprovidedshortbiographicalsketchesofthemathematicianswhoarethemaincharactersinourstory.Thesearen’tcomprehensiveencyclopediaentries,butratheranattempttogiveyousomecontextforwhothesepeoplewere.Althoughwetakeahistoricalperspective,thatdoesn’tmeanthatthebookisintendedasahistoryof
mathematicsoreventhatalltheideasarepresentedintheorderinwhichtheywerediscovered.We’lljumparoundinspaceandtimewhennecessary,butwe’lltrytogiveahistoricalcontextforeachoftheideas.
1.3PrerequisitesSincealotofthebookisaboutmathematics,youmaybeconcernedthatyouneedtohavetakenalotofmathclassestounderstandit.Whileyou’llneedtobeabletothinklogically(somethingyoushouldalreadybegoodatasaprogrammer),wedon’tassumeanyspecificmathematicalknowledgebeyondhighschoolalgebraandgeometry.Inacoupleofsections,weshowsomeapplicationsthatusealittlelinearalgebra(vectorsandmatrices),butyoucansafelyskiptheseifyouhaven’tbeenexposedtothebackgroundmaterialbefore.Ifyou’reunfamiliarwithanyofthenotationweuse,it’sexplainedinAppendixA.Animportantpartofmathematicsisbeingabletoprovesomethingformally.Thisbookcontainsquitea
fewproofs.You’llfindthebookeasiertounderstandifyou’vedonesomeproofsbefore,whetherinhighschoolgeometry,inacomputerscienceclassonautomatatheory,orinlogic.We’vedescribedsomeofthecommonprooftechniquesweuse,alongwithexamples,inAppendixB.Weassumethatifyou’rereadingthisbook,you’realreadyaprogrammer.Inparticular,youshouldbe
reasonablyproficientinatypicalimperativeprogramminglanguagelikeC,C++,orJava.OurexampleswilluseC++,butweexpectyou’llbeabletounderstandthemevenifyou’veneverprogrammedinthatlanguagebefore.WhenwemakeuseofaconstructuniquetoC++,weexplainitinAppendixC.IrrespectiveofouruseofC++,webelievethattheprinciplesdiscussedinthisbookapplytoprogrammingingeneral.Manyoftheprogrammingtopicsinthisbookarealsocoveredfromadifferentperspective,andmore
formally,inElementsofProgrammingbyStepanovandMcJones.Readersinterestedinadditionaldepthmayfindthatbooktobeausefulcompaniontothisone.Throughoutthisbook,weoccasionallyreferinterestedreaderstoarelevantsectionofElementsofProgramming.
1.4RoadmapBeforedivingintothedetails,it’susefultoseeabriefoverviewofwherewe’reheaded:
•Chapter2tellsthestoryofanancientalgorithmformultiplication,andhowtoimproveit.•Chapter3looksatsomeearlyobservationsaboutpropertiesofnumbers,andanefficientimplementationofanalgorithmforfindingprimes.•Chapter4introducesanalgorithmforfindingthegreatestcommondivisor(GCD),whichwillbethebasisforsomeofourabstractionsandapplicationslateron.•Chapter5focusesonmathematicalresults,introducingacoupleofimportanttheoremsthatwillplayacriticalrolebytheendofthebook.•Chapter6introducesthemathematicalfieldofabstractalgebra,whichprovidesthecoreideaforgenericprogramming.•Chapter7showshowthesemathematicalideasallowustogeneralizeourmultiplicationalgorithmbeyondsimplearithmetictoavarietyofpracticalprogrammingapplications.•Chapter8introducesnewabstractmathematicalstructures,andshowssomenewapplicationstheyenable.•Chapter9talksaboutaxiomsystems,theories,andmodels,whichareallbuildingblocksofgenericprogramming.•Chapter10introducesconceptsingenericprogramming,andexaminesthesubtletiesofsomeapparentlysimpleprogrammingtasks.•Chapter11continuestheexplorationofsomefundamentalprogrammingtasks,examininghowdifferentpracticalimplementationscanexploittheoreticalknowledgeoftheproblem.•Chapter12looksathowhardwareconstraintscanleadtoanewapproachforanoldalgorithm,andshowsnewapplicationsofGCD.•Chapter13putsthemathematicalandalgorithmicresultstogethertobuildanimportantcryptographyapplication.•Chapter14isasummaryofsomeoftheprincipalideasinthebook.
Thestrandsofprogrammingandmathematicsareinterwoventhroughout,thoughoneortheothermayliehiddenforachapterortwo.Buteverychapterplaysapartintheoverallchainofreasoningthatsummarizestheentirebook:
Tobeagoodprogrammer,youneedtounderstandtheprinciplesofgenericprogramming.Tounderstandtheprinciplesofgenericprogramming,youneedtounderstandabstraction.Tounderstandabstraction,youneedtounderstandthemathematicsonwhichit’sbased.
That’sthestorywe’rehopingtotell.
2.TheFirstAlgorithm
Mosesspeedilylearnedarithmetic,andgeometry....ThisknowledgehederivedfromtheEgyptians,
whostudymathematicsaboveallthings.PhiloofAlexandria,LifeofMoses
Analgorithmisaterminatingsequenceofstepsforaccomplishingacomputationaltask.Algorithmsaresocloselyassociatedwiththenotionofcomputerprogrammingthatmostpeoplewhoknowthetermprobablyassumethattheideaofalgorithmscomesfromcomputerscience.Butalgorithmshavebeenaroundforliterallythousandsofyears.Mathematicsisfullofalgorithms,someofwhichweuseeveryday.Eventhemethodschoolchildrenlearnforlongadditionisanalgorithm.Despiteitslonghistory,thenotionofanalgorithmdidn’talwaysexist;ithadtobeinvented.Whilewe
don’tknowwhenalgorithmswerefirstinvented,wedoknowthatsomealgorithmsexistedinEgyptatleastasfarbackas4000yearsago.
***
AncientEgyptiancivilizationwascenteredontheNileRiver,anditsagriculturedependedontheriver’sfloodstoenrichthesoil.TheproblemwasthateverytimetheNileflooded,allthemarkersshowingtheboundariesofpropertywerewashedaway.TheEgyptiansusedropestomeasuredistances,anddevelopedproceduressotheycouldgobacktotheirwrittenrecordsandreconstructthepropertyboundaries.Aselectgroupofpriestswhohadstudiedthesemathematicaltechniqueswereresponsibleforthistask;theybecameknownas“rope-stretchers.”TheGreekswouldlatercallthemgeometers,meaning“Earth-measurers.”Unfortunately,wehavelittlewrittenrecordoftheEgyptians’mathematicalknowledge.Onlytwo
mathematicaldocumentssurvivedfromthisperiod.TheoneweareconcernedwithiscalledtheRhindMathematicalPapyrus,namedafterthe19th-centuryScottishcollectorwhoboughtitinEgypt.Itisadocumentfromabout1650BCwrittenbyascribenamedAhmes,whichcontainsaseriesofarithmeticandgeometryproblems,togetherwithsometablesforcomputation.Thisscrollcontainsthefirstrecordedalgorithm,atechniqueforfastmultiplication,alongwithasecondoneforfastdivision.Let’sbeginbylookingatthefastmultiplicationalgorithm,which(asweshallseelaterinthebook)isstillanimportantcomputationaltechniquetoday.
2.1EgyptianMultiplicationTheEgyptians’numbersystem,likethatofallancientcivilizations,didnotusepositionalnotationandhadnowaytorepresentzero.Asaresult,multiplicationwasextremelydifficult,andonlyafewtrainedexpertsknewhowtodoit.(ImaginedoingmultiplicationonlargenumbersifyoucouldonlymanipulatesomethinglikeRomannumerals.)Howdowedefinemultiplication?Informally,it’s“addingsomethingtoitselfanumberoftimes.”
Formally,wecandefinemultiplicationbybreakingitintotwocases:multiplyingby1,andmultiplyingbyanumberlargerthan1.Wedefinemultiplicationby1likethis:
Nextwehavethecasewherewewanttocomputeaproductofonemorethingthanwealready
computed.Somereadersmayrecognizethisastheprocessofinduction;we’llusethattechniquemoreformallylateron.
Onewaytomultiplynbyaistoaddinstancesofatogetherntimes.However,thiscouldbeextremelytediousforlargenumbers,sincen–1additionsarerequired.InC++,thealgorithmlookslikethis:Clickheretoviewcodeimage
intmultiply0(intn,inta){
if(n==1)returna;
returnmultiply0(n-1,a)+a;
}
Thetwolinesofcodecorrespondtoequations2.1and2.2.Bothaandnmustbepositive,astheywerefortheancientEgyptians.ThealgorithmdescribedbyAhmes—whichtheancientGreeksknewas“Egyptianmultiplication”and
whichmanymodernauthorsrefertoasthe“RussianPeasantAlgorithm”1—reliesonthefollowinginsight:
1ManycomputerscientistslearnedthisnamefromKnuth’sTheArtofComputerProgramming,whichsaysthattravelersin19th-centuryRussiaobservedpeasantsusingthealgorithm.However,thefirstreferencetothisstorycomesfroma1911bookbySirThomasHeath,whichactuallysays,“Ihavebeentoldthatthereisamethodinusetoday(somesayinRussia,butIhavenotbeenabletoverifythis),....”
Thisoptimizationdependsonthelawofassociativityofaddition:
a+(b+c)=(a+b)+c
Itallowsustocomputea+aonlyonceandreducethenumberofadditions.Theideaistokeephalvingnanddoublinga,constructingasumofpower-of-2multiples.Atthetime,
algorithmswerenotdescribedintermsofvariablessuchasaandn;instead,theauthorwouldgiveanexampleandthensay,“Nowdothesamethingforothernumbers.”Ahmeswasnoexception;hedemonstratedthealgorithmbyshowingthefollowingtableformultiplyingn=41bya=59:
Eachentryontheleftisapowerof2;eachentryontherightistheresultofdoublingthepreviousentry(sinceaddingsomethingtoitselfisrelativelyeasy).Thecheckedvaluescorrespondtothe1-bitsinthebinaryrepresentationof41.Thetablebasicallysaysthat
41×59=(1×59)+(8×59)+(32×59)
whereeachoftheproductsontherightcanbecomputedbydoubling59thecorrectnumberoftimes.Thealgorithmneedstocheckwhethernisevenandodd,sowecaninferthattheEgyptiansknewof
thisdistinction,althoughwedonothavedirectproof.ButancientGreeks,whoclaimedthattheylearned
theirmathematicsfromtheEgyptians,certainlydid.Here’showtheydefined2evenandodd,expressedinmodernnotation:3
2Thedefinitionappearsinthe1st-centuryworkIntroductiontoArithmetic,BookI,ChapterVII,byNicomachusofGerasa.Hewrites,“Theevenisthatwhichcanbedividedintotwoequalpartswithoutaunitinterveninginthemiddle;andtheoddisthatwhichcannotbedividedintotwoequalpartsbecauseoftheaforesaidinterventionofaunit.”3Thearrowsymbol“ ”isread“implies.”SeeAppendixAforasummaryofthemathematicalnotationusedinthisbook.
Wewillalsorelyonthisrequirement:
odd(n) half(n)=half(n–1)
ThisishowweexpresstheEgyptianmultiplicationalgorithminC++:Clickheretoviewcodeimage
intmultiply1(intn,inta){
if(n==1)returna;
intresult=multiply1(half(n),a+a);
if(odd(n))result=result+a;
returnresult;
}
Wecaneasilyimplementodd(x)bytestingtheleastsignificantbitofx,andhalf(x)byasinglerightshiftofx:Clickheretoviewcodeimage
boolodd(intn){returnn&0x1;}
inthalf(intn){returnn>>1;}
Howmanyadditionsismultiply1goingtodo?Everytimewecallthefunction,we’llneedtodotheadditionindicatedbythe+ina+a.Sincewearehalvingthevalueaswerecurse,we’llinvokethefunctionlogntimes.4Andsomeofthetime,we’llneedtodoanotheradditionindicatedbythe+inresult+a.Sothetotalnumberofadditionswillbe
#+(n)= logn +(ν(n)–1)
4Throughoutthisbook,whenwewrite“log,”wemeanthebase2logarithm,unlessspecifiedotherwise.
wherev(n)isthenumberof1sinthebinaryrepresentationofn(thepopulationcountorpopcount).SowehavereducedanO(n)algorithmtoonethatisO(logn).Isthisalgorithmoptimal?Notalways.Forexample,ifwewanttomultiplyby15,thepreceding
formulawouldgivethisresult:
#+(15)=3+4–1=6
Butwecanactuallymultiplyby15withonly5additions,usingthefollowingprocedure:Clickheretoviewcodeimage
intmultiply_by_15(inta){
intb=(a+a)+a;//b==3*a
intc=b+b;//c==6*a
return(c+c)+b;//12*a+3*a
}
Suchasequenceofadditionsiscalledanadditionchain.Herewehavediscoveredanoptimaladditionchainfor15.Nevertheless,Ahmes’salgorithmisgoodenoughformostpurposes.
Exercise2.1.Findoptimaladditionchainsforn<100.
Atsomepointthereadermayhaveobservedthatsomeofthesecomputationswouldbeevenfasterifwefirstreversedtheorderoftheargumentswhenthefirstisgreaterthanthesecond(forexample,wecouldcompute3×15moreeasilythan15×3).That’strue,andtheEgyptiansknewthis.Butwe’renotgoingtoaddthatoptimizationhere,becauseaswe’llseeinChapter7,we’reeventuallygoingtowanttogeneralizeouralgorithmtocaseswheretheargumentshavedifferenttypesandtheorderoftheargumentsmatters.
2.2ImprovingtheAlgorithmOurmultiply1functionworkswellasfarasthenumberofadditionsisconcerned,butitalsodoeslogn recursivecalls.Sincefunctioncallsareexpensive,wewanttotransformtheprogramtoavoidthisexpense.Oneprinciplewe’regoingtotakeadvantageofisthis:Itisofteneasiertodomoreworkratherthan
less.Specifically,we’regoingtocompute
r+na
whererisarunningresultthataccumulatesthepartialproductsna.Inotherwords,we’regoingtoperformmultiply-accumulateratherthanjustmultiply.Thisprincipleturnsouttobetruenotonlyinprogrammingbutalsoinhardwaredesignandinmathematics,whereit’softeneasiertoproveageneralresultthanaspecificone.Here’sourmultiply-accumulatefunction:
Clickheretoviewcodeimage
intmult_acc0(intr,intn,inta){
if(n==1)returnr+a;
if(odd(n)){
returnmult_acc0(r+a,half(n),a+a);
}else{
returnmult_acc0(r,half(n),a+a);
}
}
Itobeystheinvariant:r+na=r0+n0a0,wherer0,n0anda0aretheinitialvaluesofthosevariables.
Wecanimprovethisfurtherbysimplifyingtherecursion.Noticethatthetworecursivecallsdifferonlyintheirfirstargument.Insteadofhavingtworecursivecallsfortheoddandevencases,we’lljustmodifythevalueofrbeforewerecurse,likethis:Clickheretoviewcodeimage
intmult_acc1(intr,intn,inta){
if(n==1)returnr+a;
if(odd(n))r=r+a;
returnmult_acc1(r,half(n),a+a);
}
Nowourfunctionistail-recursive—thatis,therecursionoccursonlyinthereturnvalue.We’lltakeadvantageofthisfactshortly.Wemaketwoobservations:•nisrarely1.•Ifniseven,there’snopointcheckingtoseeifit’s1.
Sowecanreducethenumberoftimeswehavetocomparewith1byafactorof2,simplybycheckingforoddnessfirst:Clickheretoviewcodeimage
intmult_acc2(intr,intn,inta){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
returnmult_acc2(r,half(n),a+a);
}
Someprogrammersthinkthatcompileroptimizationswilldothesekindsoftransformationsforus,butthat’srarelytrue;theydonottransformonealgorithmintoanother.Whatwehavesofarisprettygood,butwe’reeventuallygoingtowanttoeliminatetherecursionto
avoidthefunctioncalloverhead.Thisiseasierifthefunctionisstrictlytail-recursive.
Definition2.1.Astrictlytail-recursiveprocedureisoneinwhichallthetail-recursivecallsaredonewiththeformalparametersoftheprocedurebeingthecorrespondingarguments.
Again,wecanachievethissimplybyassigningthedesiredvaluestothevariableswe’llbepassingbeforewedotherecursion:Clickheretoviewcodeimage
intmult_acc3(intr,intn,inta){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
n=half(n);
a=a+a;
returnmult_acc3(r,n,a);
}
Nowitiseasytoconvertthistoaniterativeprogrambyreplacingthetailrecursionwithawhile(true)construct:Clickheretoviewcodeimage
intmult_acc4(intr,intn,inta){
while(true){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
n=half(n);
a=a+a;
}
}
Withournewlyoptimizedmultiply-accumulatefunction,wecanwriteanewversionofmultiply.Ournewversionwillinvokeourmultiply-accumulatehelperfunction:Clickheretoviewcodeimage
intmultiply2(intn,inta){
if(n==1)returna;
returnmult_acc4(a,n-1,a);
}
Noticethatweskiponeiterationofmult_acc4bycallingitwithresultalreadysettoa.Thisisprettygood,exceptwhennisapowerof2.Thefirstthingwedoissubtract1,whichmeansthat
mult_acc4willbecalledwithanumberwhosebinaryrepresentationisall1s,theworstcaseforouralgorithm.Sowe’llavoidthisbydoingsomeoftheworkinadvancewhenniseven,halvingit(anddoublinga)untilnbecomesodd:Clickheretoviewcodeimage
intmultiply3(intn,inta){
while(!odd(n)){
a=a+a;
n=half(n);
}
if(n==1)returna;
returnmult_acc4(a,n-1,a);
}
Butnowwenoticethatwe’remakingmult_acc4dooneunnecessarytestforn=1,becausewe’recallingitwithanevennumber.Sowe’lldoonehalvinganddoublingontheargumentsbeforewecallit,givingusourfinalversion:Clickheretoviewcodeimage
intmultiply4(intn,inta){
while(!odd(n)){
a=a+a;
n=half(n);
}
if(n==1)returna;
//even(n–1) n–1≠1
returnmult_acc4(a,half(n-1),a+a);
}
RewritingCodeAswehaveseenwithourtransformationsofthemultiplyalgorithm,rewritingcodeisimportant.Noonewritesgoodcodethefirsttime;ittakesmanyiterationstofindthemostefficientorgeneralwaytodosomething.Noprogrammershouldhaveasingle-passmindset.Atsomepointduringtheprocessyoumayhavebeenthinking,“Onemoreoperationisn’tgoing
tomakeabigdifference.”Butitmayturnoutthatyourcodewillbereusedmanytimesformanyyears.(Infact,atemporaryhackoftenbecomesthecodethatlivesthelongest.)Furthermore,thatinexpensiveoperationyou’resavingnowmaybereplacedbyaverycostlyoneinsomefutureversionofthecode.Anotherbenefitofstrivingforefficiencyisthattheprocessforcesyoutounderstandtheproblem
inmoredepth.Atthesametime,thisincreaseddepthofunderstandingleadstomoreefficientimplementations;it’savirtuouscircle.
2.3ThoughtsontheChapterStudentsofelementaryalgebralearnhowtokeeptransformingexpressionsuntiltheycanbesimplified.InoursuccessiveimplementationsoftheEgyptianmultiplicationalgorithm,we’vegonethroughananalogousprocess,rearrangingthecodetomakeitclearerandmoreefficient.Everyprogrammerneedstogetinthehabitoftryingcodetransformationsuntilthefinalformisobtained.We’veseenhowmathematicsemergedinancientEgypt,andhowitgaveusthefirstknownalgorithm.
We’regoingtoreturntothatalgorithmandexpandonitquiteabitlaterinthebook.Butfornowwe’regoingtomoveaheadmorethanathousandyearsandtakealookatsomemathematicaldiscoveriesfromancientGreece.
3.AncientGreekNumberTheory
Pythagoreansappliedthemselvestothestudyofmathematics....Theythoughtthatitsprinciplesmustbetheprinciplesofallexistingthings.
Aristotle,Metaphysics
Inthischapter,we’regoingtolookatsomeoftheproblemsstudiedbyancientGreekmathematicians.Theirworkonpatternsand“shapes”ofnumbersledtothediscoveryofprimenumbersandthebeginningsofafieldofmathematicscallednumbertheory.Theyalsodiscoveredparadoxesthatultimatelyproducedsomemathematicalbreakthroughs.Alongtheway,we’llexamineanancientalgorithmforfindingprimes,andseehowtooptimizeit.
3.1GeometricPropertiesofIntegersPythagoras,theGreekmathematicianandphilosopherwhomostofusknowonlyforhistheorem,wasactuallythepersonwhocameupwiththeideathatunderstandingmathematicsisnecessarytounderstandtheworld.Healsodiscoveredmanyinterestingpropertiesofnumbers;heconsideredthisunderstandingtobeofgreatvalueinitsownright,independentofanypracticalapplication.AccordingtoAristotle’spupilAristoxenus,“Heattachedsupremeimportancetothestudyofarithmetic,whichheadvancedandtookoutoftheregionofcommercialutility.”
Pythagoras(ca.570BC–ca.490BC)
PythagoraswasbornontheGreekislandofSamos,whichwasamajornavalpoweratthetime.Hecamefromaprominentfamily,butchosetopursuewisdomratherthanwealth.AtsomepointinhisyouthhetraveledtoMiletustostudywithThales,thefounderofphilosophy(seeSection9.2),whoadvisedhimtogotoEgyptandlearntheEgyptians’mathematicalsecrets.DuringthetimePythagoraswasstudyingabroad,thePersianempireconqueredEgypt.
PythagorasfollowedthePersianarmyeastwardtoBabylon(inwhatisnowIraq),wherehelearnedBabylonianmathematicsandastronomy.Whilethere,hemayhavemettravelersfromIndia;whatweknowisthathewasexposedtoandbeganespousingideaswetypicallyassociatewithIndianreligions,includingthetransmigrationofsouls,vegetarianism,andasceticism.PriortoPythagoras,theseideaswerecompletelyunknowntotheGreeks.AfterreturningtoGreece,PythagorasstartedasettlementinCroton,aGreekcolonyinsouthern
Italy,wherehegatheredfollowers—bothmenandwomen—whosharedhisideasandfollowedhisasceticlifestyle.Theirliveswerecenteredonthestudyoffourthings:astronomy,geometry,numbertheory,andmusic.Thesefoursubjects,laterknownasthequadrivium,remainedafocusofEuropeaneducationfor2000years.Eachofthesedisciplineswasrelated:themotionofthestarscouldbemappedgeometrically,geometrycouldbegroundedinnumbers,andnumbersgeneratedmusic.Infact,Pythagoraswasthefirsttodiscoverthenumericalstructureoffrequenciesinmusicaloctaves.Hisfollowerssaidthathecould“hearthemusicofthecelestialspheres.”AfterthedeathofPythagoras,thePythagoreansspreadtoseveralotherGreekcoloniesinthe
areaanddevelopedalargebodyofmathematics.However,theykepttheirteachingssecret,somanyoftheirresultsmayhavebeenlost.TheyalsoeliminatedcompetitionwithintheirranksbycreditingalldiscoveriestoPythagorashimself,sowedon’tactuallyknowwhichindividualsdidwhat.AlthoughthePythagoreancommunitiesweregoneafteracoupleofhundredyears,theirwork
remainsinfluential.Aslateasthe17thcentury,Leibniz(oneoftheinventorsofcalculus)describedhimselfasaPythagorean.
Unfortunately,Pythagorasandhisfollowerskepttheirworksecret,sononeoftheirwritingssurvive.However,weknowfromcontemporarieswhatsomeofhisdiscoverieswere.Someofthesecomefromafirst-centurybookcalledIntroductiontoArithmeticbyNicomachusofGerasa.Theseincludedobservationsaboutgeometricpropertiesofnumbers;theyassociatednumberswithparticularshapes.
Triangularnumbers,forexample,whichareformedbystackingrowsrepresentingthefirstnintegers,arethosethatformedthefollowinggeometricpattern:
Oblongnumbersarethosethatlooklikethis:
Itiseasytoseethatthenthoblongnumberisrepresentedbyann×(n+1)rectangle:
n=n(n+1)
It’salsocleargeometricallythateachoblongnumberistwiceitscorrespondingtriangularnumber.Sincewealreadyknowthattriangularnumbersarethesumofthefirstnintegers,wehave
Sothegeometricrepresentationgivesustheformulaforthesumofthefirstnintegers:
AnothergeometricobservationisthatthesequenceofoddnumbersformstheshapeofwhattheGreekscalledgnomons(theGreekwordforacarpenter’ssquare;agnomonisalsothepartofasundialthatcaststheshadow):
Combiningthefirstngnomonscreatesafamiliarshape—asquare:
Thispicturealsogivesusaformulaforthesumofthefirstnoddnumbers:
Exercise3.1.Findageometricproofforthefollowing:takeanytriangularnumber,multiplyitby8,andadd1.Theresultisasquarenumber.(ThisproblemcomesfromPlutarch’sPlatonicQuestions.)
3.2SiftingPrimesPythagoreansalsoobservedthatsomenumberscouldnotbemadeintoanynontrivialrectangularshape(ashapewherebothsidesoftherectanglearegreaterthan1).Thesearewhatwenowcallprimenumbers—numbersthatarenotproductsofsmallernumbers:
2,3,5,7,11,13,...
(“Numbers”fortheGreekswerealwayswholenumbers.)SomeoftheearliestobservationsaboutprimescomefromEuclid.Whileheisusuallyassociatedwithgeometry,severalbooksofEuclid’sElementsactuallydiscusswhatwenowcallnumbertheory.Oneofhisresultsisthistheorem:
Theorem3.1(EuclidVII,32):Anynumberiseitherprimeordivisiblebysomeprime.
Theproof,whichusesatechniquecalled“impossibilityofinfinitedescent,”goeslikethis:11Euclid’sproofofVII,32actuallyreliesonhispropositionVII,31(anycompositenumberisdivisiblebysomeprime),whichcontainsthe
reasoningshownhere.
Proof.ConsideranumberA.Ifitisprime,thenwearedone.Ifitiscomposite(i.e.,nonprime),thenitmustbedivisiblebysomesmallernumberB.IfBisprime,wearedone(becauseifAisdivisiblebyBandBisprime,thenAisdivisiblebyaprime).IfBiscomposite,thenitmustbedivisiblebysomesmallernumberC,andsoon.Eventually,wewillfindaprimeor,asEuclidremarksinhisproofofthepreviousproposition,“aninfinitesequenceofnumberswilldividethenumber,eachofwhichislessthantheother;andthisisimpossible.”
ThisEuclideanprinciplethatanydescendingsequenceofnaturalnumbersterminatesisequivalenttotheinductionaxiomofnaturalnumbers,whichwewillencounterinChapter9.
***
Anotherresult,whichsomeconsiderthemostbeautifultheoreminmathematics,isthefactthatthereareinfinitelymanyprimes:
Theorem3.2(EuclidIX,20):Foranysequenceofprimes{p1,...,pn},thereisaprimepnotinthesequence.
Proof.Considerthenumber
wherepiistheithprimeinthesequence.Becauseofthewayweconstructedq,weknowitisnotdivisiblebyanypi.Theneitherqisprime,inwhichcaseitisitselfaprimenotinthesequence,orqisdivisiblebysomenewprime,whichbydefinitionisnotinthesequence.Therefore,thereareinfinitelymanyprimes.
Oneofthebest-knowntechniquesforfindingprimesistheSieveofEratosthenes.Eratostheneswasa3rd-centuryGreekmathematicianwhoisrememberedinpartforhisamazinglyaccuratemeasurementofthecircumferenceoftheEarth.Metaphorically,theideaofEratosthenes’sieveisto“sift”allthenumberssothatthenonprimes“fallthrough”thesieveandtheprimesremainattheend.Theactualprocedureistostartwithalistofallthecandidatenumbersandthencrossouttheonesknownnottobeprimes(sincetheyaremultiplesofprimesfoundsofar);whateverisleftaretheprimes.TodaytheSieveofEratosthenesisoftenshownstartingwithallpositiveintegersuptoagivennumber,butEratosthenesalreadyknewthatevennumberswerenotprime,sohedidn’tbothertoincludethem.FollowingEratosthenes’convention,we’llalsoincludeonlyoddnumbers,sooursievewillfind
primesgreaterthan2.Eachvalueinthesieveisacandidateprimeuptowhatevervaluewecareabout.Ifwewanttofindprimesuptoamaximumofm=53,oursieveinitiallylookslikethis:
Ineachiteration,wetakethefirstnumber(whichmustbeaprime)andcrossoutallthemultiplesexceptitselfthathavenotpreviouslybeencrossedout.We’llhighlightthenumbersbeingcrossedoutinthecurrentiterationbyboxingthem.Here’swhatthesievelookslikeafterwecrossoutthemultiplesof3:
Nextwecrossoutthemultiplesof5thathavenotyetbeencrossedout:
Andthentheremainingmultiplesof7:
Weneedtorepeatthisprocessuntilwe’vecrossedoutallthemultiplesoffactorslessthanorequalto,wheremisthehighestcandidatewe’reconsidering.
Inourexample,m=53,sowearedone.Allthenumbersthathavenotbeencrossedoutareprimes:
Beforewewriteourimplementationofthealgorithm,we’llmakeafewobservations.Let’sgobackto
whatthesievelookedlikeinthemiddleoftheprocess(say,whenwewerecrossingoutmultiplesof5)andaddsomeinformation—namely,theindex,orpositioninthelist,ofeachcandidatebeingconsidered:
Noticethatwhenwe’reconsideringmultiplesoffactor5,thestepsize—thenumberofentriesbetweentwonumbersbeingcrossedout,suchas25and35—is5,thesameasthefactor.Anotherwaytosaythisisthatthedifferencebetweentheindexesofanytwocandidatesbeingcrossedoutinagiveniterationisthesameasthefactorbeingused.Also,sincethelistofcandidatescontainsonlyoddnumbers,thedifferencebetweentwovaluesistwiceasmuchasthedifferencebetweentwoindexes.Sothedifferencebetweentwonumbersbeingcrossedoutinagiveniteration(e.g.,between25and35)istwicethestepsizeor,equivalently,twicethefactorbeingused.You’llseethatthispatternholdsforallthefactorsweconsideredinourexampleaswell.Finally,weobservethatthefirstnumbercrossedoutineachiterationisthesquareoftheprimefactor
beingused.Thatis,whenwe’recrossingoutmultiplesof5,thefirstonethatwasn’tpreviouslycrossedoutis25.Thisisbecausealltheothermultipleswerealreadyaccountedforbypreviousprimes.
3.3ImplementingandOptimizingtheCodeAtfirstglanceitseemslikeouralgorithmwillneedtomaintaintwoarrays:onecontainingthecandidatenumberswe’resifting—the“values”—andanothercontainingBooleanflagsindicatingwhetherthecorrespondingnumberisstillthereorhasbeencrossedout.However,afterabitofthoughtitbecomesclearthatwedon’tactuallyneedtostorethevaluesatall.Mostofthevalues(namely,allthenonprimes)areneverused.Whenwedoneedavalue,wecancomputeitfromitsposition;weknowthatthefirstvalueis3andthateachsuccessivevalueis2morethanthepreviousone,sotheithvalueis2i+3.SoourimplementationwillstorejusttheBooleanflagsinthesieve,usingtrueforprimeandfalse
forcomposite.Wecalltheprocessof“crossingout”nonprimesmarkingthesieve.Here’safunctionwe’llusetomarkallthenonprimesforagivenfactor:Clickheretoviewcodeimage
template<RandomAccessIteratorI,IntegerN>
voidmark_sieve(Ifirst,Ilast,Nfactor){
//assert(first!=last)
*first=false;
while(last-first>factor){
first=first+factor;
*first=false;
}
}
Weareusingtheconventionof“declaring”ourtemplateargumentswithadescriptionoftheirrequirements.Wewilldiscusstheserequirements,knownasconcepts,indetaillateroninChapter10;fornow,readerscanconsultAppendixCasareference.(IfyouarenotfamiliarwithC++templates,thesearealsoexplainedinthisappendix.)Aswe’llseeshortly,we’llcallthisfunctionwithfirstpointingtotheBooleanvaluecorresponding
tothefirst“uncrossed-out”multipleoffactor,whichaswesawisalwaysfactor’ssquare.Forlast,we’llfollowtheSTLconventionofpassinganiteratorthatpointsjustpastthelastelementinourtable,sothatlast-firstisthenumberofelements.
***
Beforeweseehowtosift,weobservethefollowingsiftinglemmas:•Thesquareofthesmallestprimefactorofacompositenumbercislessthanorequaltoc.•Anycompositenumberlessthanp2issiftedby(i.e.,crossedoutasamultipleof)aprimelessthanp.•Whensiftingbyp,startmarkingatp2.•Ifwewanttosiftnumbersuptom,stopsiftingwhenp2>m.
Wewillusethefollowingformulasinourcomputation:
stepbetweenmultiplekandmultiplek+1ofvalueati:
indexofsquareofvalueati:
Wecannowmakeourfirstattemptatimplementingthesieve:Clickheretoviewcodeimage
template<RandomAccessIteratorI,IntegerN>
voidsift0(Ifirst,Nn){
std::fill(first,first+n,true);
Ni(0);
Nindex_square(3);
while(index_square<n){
//invariant:index_square=2i^2+6i+3
if(first[i]){//ifcandidateisprime
mark_sieve(first+index_square,
first+n,//last
i+i+3);//factor
}
++i;
index_square=2*i*(i+3)+3;
}
}
ItmightseemthatweshouldpassinareferencetoadatastructurecontainingtheBooleansequence,sincethesieveworksonlyifwesiftthewholething.Butbyinsteadpassinganiteratortothebeginningoftherange,togetherwithitslength,wedon’tconstrainwhichkindofdatastructuretouse.ThedatacouldbeinanSTLcontainerorinablockofmemory;wedon’tneedtoknow.Notethatweusethesizeofthetablenratherthanthemaximumvaluetosiftm.
Thevariableindex_squareistheindexofthefirstvaluewewanttomark—thatis,thesquareofthecurrentfactor.Onethingwenoticeisthatwe’recomputingthefactorweusetomarkthesieve(i+i+3)andotherquantities(showninslantedtext)everytimethroughtheloop.Wecanhoistcommonsubexpressionsoutoftheloop;thechangesareshowninbold:Clickheretoviewcodeimage
template<RandomAccessIteratorI,IntegerN>
voidsift1(Ifirst,Nn){
Ilast=first+n;
std::fill(first,last,true);
Ni(0);
Nindex_square(3);
Nfactor(3);
while(index_square<n){
//invariant:index_square=2i^2+6i+3,
//factor=2i+3
if(first[i]){
mark_sieve(first+index_square,last,factor);
}
++i;
factor=i+i+3;
index_square=2*i*(i+3)+3;
}
}
Theastutereaderwillnoticethatthefactorcomputationisactuallyslightlyworsethanbefore,sinceithappenseverytimethroughtheloop,notjustoniterationswhentheiftestistrue.However,weshallseelaterwhymakingfactoraseparatevariablemakessense.Abiggerissueisthatwestillhavearelativelyexpensiveoperation—thecomputationofindex_square,whichinvolvestwomultiplications.Sowewilltakeacuefromcompileroptimizationanduseatechniqueknownasstrengthreduction,whichwasdesignedtoreplacemoreexpensiveoperationslikemultiplicationwithequivalentcodethatuseslessexpensiveoperationslikeaddition.2Ifacompilercandothisautomatically,wecancertainlydoitmanually.
2Whilemultiplicationisnotnecessarilyslowerthanadditiononmodernprocessors,thegeneraltechniquecanstillleadtousingfeweroperations.
Let’slookatthesecomputationsinmoredetail.SupposewereplacedClickheretoviewcodeimage
factor=i+i+3;
index_square=3+2*i*(i+3);
withfactor+=δfactor;index_square+=δindex_square;
whereδfactorandδindex_squarearethedifferencesbetweensuccessive(ithandi+1st)valuesoffactorandindex_square,respectively:
δfactoriseasy;thevariablescancelandwegettheconstant2.Buthowdidwesimplifytheexpressionforδindex_square?Weobservethatbyrearrangingtheterms,wecanexpressitusingsomethingwealreadyhave,factor(i),andsomethingweneedtocomputeanyway,factor(i+1).(Whenyouknowyouneedtocomputemultiplequantities,it’susefultoseeifonecanbecomputedintermsofanother.Thismightallowyoutodolesswork.)Withthesesubstitutions,wegetourfinalversionofsift;again,ourimprovementsareshowninbold:
Clickheretoviewcodeimage
template<RandomAccessIteratorI,IntegerN>
voidsift(Ifirst,Nn){
Ilast=first+n;
std::fill(first,last,true);
Ni(0);
Nindex_square(3);
Nfactor(3);
while(index_square<n){
//invariant:index_square=2i^2+6i+3,
//factor=2i+3
if(first[i]){
mark_sieve(first+index_square,last,factor);
}
++i;
index_square+=factor;
factor+=N(2);
index_square+=factor;
}
}
Exercise3.2.Timethesieveusingdifferentdatasizes:bit(usingstd::vector<bool>),uint8_t,uint16_t,uint32_t,uint64_t.
Exercise3.3.Usingthesieve,graphthefunction
π(n)=numberofprimes<n
fornupto107andfinditsanalyticapproximation.Wecallprimesthatreadthesamebackwardandforwardpalindromicprimes.Herewe’vehighlighted
theonesupto1000:
Interestingly,therearenopalindromicprimesbetween1000and2000:
Exercise3.4.Aretherepalindromicprimes>1000?Whatisthereasonforthelackofthemintheinterval[1000,2000]?Whathappensifwechangeourbaseto16?Toanarbitraryn?
3.4PerfectNumbersAswesawinSection3.1,theancientGreekswereinterestedinallsortsofpropertiesofnumbers.Oneideatheycameupwithwasthatofaperfectnumber—anumberthatisthesumofitsproperdivisors.3Theyknewoffourperfectnumbers:
3Aproperdivisorofanumbernisadivisorofnotherthannitself.
Perfectnumberswerebelievedtoberelatedtonatureandthestructureoftheuniverse.Forexample,thenumber28wasthenumberofdaysinthelunarcycle.WhattheGreeksreallywantedtoknowwaswhethertherewasawaytopredictotherperfectnumbers.
Theylookedattheprimefactorizationsoftheperfectnumberstheyknew:
andnoticedthefollowingpattern:
Theresultofthisexpressionisperfectwhenthethesecondtermisprime.ItwasEuclidwhopresentedtheproofofthisfactaround300BC.
Theorem3.3(EuclidIX,36):
UsefulFormulasBeforewelookattheproof,itisusefultorememberacoupleofalgebraicformulas.Thefirstisthedifferenceofpowers:
Thisresultcaneasilybederivedusingthesetwoequations:
Theleftandrightsidesof3.2and3.3areequalbythedistributivelaw.Ifwethensubtract3.3from3.2,weget3.1.Thesecondusefulformulaisforthesumofoddpowers:
whichwecanderivebyconvertingthesumtoadifferenceandrelyingonourpreviousresult:
Wecangetawaywiththisbecause–1toanoddpowerisstill–1.Wewillrelyheavilyonbothoftheseformulasintheproofsahead.
Nowweknowthatforn>0
bythedifferenceofpowersformula:
2n–1=(2–1)(2n–1+2n–2+...+2+1)
(orjustthinkofthebinarynumberyougetwhenyouaddpowersof2).
Exercise3.5.UsingEquation3.1,provethatif2n–1isprime,thennisprime.
WearegoingtoproveEuclid’stheoremthewaythegreatGermanmathematicianCarlGaussdid.(We’lllearnmoreaboutGaussinChapter8.)First,wewilluseEquation3.5,substituting2n–1forbothoccurrencesof inEuclid’stheorem,torestatethetheoremlikethis:
If2n–1isprime,then2n–1(2n–1)isperfect.
Next,wedefineσ(n)tobethesumofthedivisorsofn.Iftheprimefactorizationofnis
thenthesetofalldivisorsconsistsofeverypossiblecombinationoftheprimedivisorsraisedtoeverypossiblepoweruptoai.Forexample,24=23·31,sothedivisorsare{20·30,21·30,22·30,20·31,21
·31,22·31,23·31}.Theirsumis20·30+21·30+22·30+20·31+21·31+22·31+23·31=(20+21+22+23)(30+31)Thatis,wecanwritethesumofthedivisorsforanynumbernasaproductofsums:
wherethelastlinereliesonusingthedifferenceofpowersformulatosimplifythenumerator.(Inthisexample,andfortherestofthebook,whenweusepasanintegervariableinourproofs,weassumeit’saprime,unlesswesayotherwise.)
Exercise3.6.Provethatifnandmarecoprime(havenocommonprimefactors),then
σ(nm)=σ(n)σ(m)
(Anotherwaytosaythisisthatσisamultiplicativefunction.)Wenowdefineα(n),thealiquotsum,asfollows:
α(n)=σ(n)–n
Inotherwords,thealiquotsumisthesumofallproperdivisorsofn—allthedivisorsexceptnitself.Nowwe’rereadyfortheproofofTheorem3.3,alsoknownasEuclidIX,36:
If2n–1isprime,then2n–1(2n–1)isperfect.
Proof.Letq=2n–1(2n–1).Weknow2isprime,andthetheorem’sconditionisthat2n–1isprime,so2n–1(2n–1)isalreadyaprimefactorizationoftheform ,wherem=2,p1=2,a1=n–1,p2=2n–1,anda2=1.Usingthesumofdivisorsformula(Equation3.6):
Then
α(q)=σ(q)–q=2q–q=q
Thatis,qisperfect.
WecanthinkofEuclid’stheoremassayingthatifanumberhasacertainform,thenitisperfect.Aninterestingquestioniswhethertheconverseistrue:ifanumberisperfect,doesithavetheform2n–1(2n–1)?Inthe18thcentury,Eulerprovedthatifaperfectnumberiseven,thenithasthisform.Hewasnotabletoprovethemoregeneralresultthateveryperfectnumberisofthatform.Eventoday,thisisanunsolvedproblem;wedon’tknowifanyoddperfectnumbersexist.
Exercise3.7.Provethateveryevenperfectnumberisatriangularnumber.
Exercise3.8.Provethatthesumofthereciprocalsofthedivisorsofaperfectnumberisalways2.Example:
3.5ThePythagoreanProgramForPythagoreans,mathematicswasnotaboutabstractsymbolmanipulation,asitisoftenviewedtoday.Instead,itwasthescienceofnumbersandspace—thetwofundamentalperceptibleaspectsofourreality.Inadditiontotheirfocusonunderstandingfiguratenumbers(suchassquare,oblong,andtriangularnumbers),theybelievedthattherewasdiscretestructuretospace.Theirchallenge,then,wastoprovideawaytogroundgeometryinnumbers—essentially,tohaveaunifiedtheoryofmathematicsbasedonpositiveintegers.Todothis,theycameupwiththeideathatonelinesegmentcouldbe“measured”byanother:
Definition3.1.AsegmentVisameasureofasegmentAifandonlyifAcanberepresentedasafiniteconcatenationofcopiesofV.
Ameasuremustbesmallenoughthatanexactintegralnumberofcopiesproducesthedesiredsegment;thereareno“fractional”measures.Ofcourse,differentmeasuresmightbeusedfordifferentsegments.Ifonewantedtousethesamemeasurefortwosegments,ithadtobeacommonmeasure:
Definition3.2.AsegmentVisacommonmeasureofsegmentsAandBifandonlyifitisameasureof
both.
Foranygivensituation,thePythagoreansbelievedthereisacommonmeasureforalltheobjectsofinterest.Therefore,spacecouldberepresenteddiscretely.
***
Sincetherecouldbemanycommonmeasures,theyalsocameupwiththeideaofthegreatestcommonmeasure:
Definition3.3.AsegmentVisthegreatestcommonmeasureofAandBifitisgreaterthananyothercommonmeasureofAandB.
ThePythagoreansalsorecognizedseveralpropertiesofgreatestcommonmeasure(GCM),whichwerepresentinmodernnotationasfollows:
Usingtheseproperties,theycameupwiththemostimportantprocedureinGreekmathematics—perhapsinallmathematics:awaytocomputethegreatestcommonmeasureoftwosegments.ThecomputationalmachineryoftheGreeksconsistedofrulerandcompassoperationsonlinesegments.UsingC++notation,wemightwritetheprocedurelikethis,usingline_segmentasatype:Clickheretoviewcodeimage
line_segmentgcm(line_segmenta,line_segmentb){
if(a==b)returna;
if(b<a)returngcm(a-b,b);
/*if(a<b)*/returngcm(a,b-a);
}
Thiscodemakesuseofthetrichotomylaw:thefactthatifyouhavetwovaluesaandbofthesametotallyorderedtype,theneithera=b,a<b,ora>b.Let’slookatanexample.What’sgcm(196,42)?
Sowe’redone:gcm(196,42)=14.Ofcourse,whenwesaygcm(196,42),wereallymeanGCMofsegmentswithlength196and42,but
fortheexamplesinthischapter,we’lljustusetheintegersasshorthand.We’regoingtouseversionsofthisalgorithmforthenextfewchapters,soit’simportanttounderstandit
andhaveagoodfeelforhowitworks.Youmaywanttotrycomputingafewmoreexamplesbyhandto
convinceyourself.
3.6AFatalFlawintheProgramGreekmathematiciansfoundthatthewell-orderingprinciple—thefactthatanysetofnaturalnumbershasasmallestelement—providedapowerfulprooftechnique.Toprovethatsomethingdoesnotexist,provethatifitdidexist,asmalleronewouldalsoexist.Usingthislogic,thePythagoreansdiscoveredaproofthatunderminedtheirentireprogram.4We’re
goingtousea19th-centuryreconstructionofthisproofbyGeorgeChrystal.4Wedon’tknowifPythagorashimselfmadethisdiscovery,oroneofhisearlyfollowers.
Theorem3.4:Thereisnosegmentthatcanmeasureboththesideandthediagonalofasquare.
Proof.Assumethecontrary,thattherewereasegmentthatcouldmeasureboththesideandthediagonalofsomesquare.5Letustakethesmallestsuchsquareforthissegment:
5Thisisanexampleofproofbycontradiction.Formoreaboutthisprooftechnique,seeAppendixB.1.
Usingarulerandcompass,6wecanconstructasegment withthesamelengthas ,andthencreateasegmentstartingatFandperpendicularto .
6Althoughmodernreadersmaythinkofarulerasbeingusedtomeasuredistances,forEucliditwasonlyawaytodrawstraightlines.Forthisreason,somepeoplepreferthetermstraightedgetodescribeEuclid’sinstrument.Similarly,althoughamoderncompasscanbefixedtomeasureequaldistances,Euclid’scompasswasusedonlytodrawcircleswithagivenradius;itwascollapsible,soitdidnotpreservedistancesoncelifted.
Nowweconstructtwomoreperpendicularsegments, and :
Weknowthat∠CFE=90°(byconstruction)andthat∠ECF=45°(sinceit’sthesameas∠BCA,whichistheangleformedbythediagonalofasquare,andthereforeishalfof90°).Wealsoknowthatthethreeanglesofatrianglesumto180°.Therefore
∠CEF=180°–∠CFE–∠ECF=180°–90°–45°=45°
So∠CEF=∠ECF,whichmeansCEFisanisoscelestriangle,sothesidesoppositeequalanglesareequal—thatis, .Finally,weaddonemoresegment :
TriangleABFisalsoisosceles,with∠ABF=∠AFB,sinceweconstructed .And∠ABC=∠AFE,sincebothwereconstructedwithperpendiculars.So
Now,weknow ismeasurablesincethat’spartofourpremise,andweknow ismeasurable,sinceit’sthesameas ,whichisalsomeasurablebyourpremise.Sotheirdifference isalsomeasurable.SincewejustshowedthatΔCEFandΔBEFarebothisosceles,
weknow ismeasurable,againbyourpremise,andwe’vejustshownthat ,andtherefore ,ismeasurable.So ismeasurable.Wenowhaveasmallersquarewhoseside anddiagonal arebothmeasurablebyourcommon
unit.Butouroriginalsquarewaschosentobethesmallestforwhichtherelationshipheld—acontradiction.Soouroriginalassumptionwaswrong,andthereisnosegmentthatcanmeasureboththesideandthediagonalofasquare.Ifyoutrytofindone,you’llbeatitforever—ourline_segment_gcm(a,b)procedurewillnotterminate.
Toputitanotherway,theratioofthediagonalandthesideofasquarecannotbeexpressedasarationalnumber(theratiooftwointegers).Todaywewouldsaythatwiththisproof,thePythagoreanshaddiscoveredirrationalnumbers,andspecificallythat isirrational.Thediscoveryofirrationalnumberswasunbelievablyshocking.ItunderminedthePythagoreans’s
entireprogram;itmeantthatgeometrycouldnotbegroundedinnumbers.Sotheydidwhatmanyorganizationsdowhenfacedwithbadnews:theysworeeveryonetosecrecy.Whenoneoftheorderleakedthestory,legendhasitthatthegodspunishedhimbysinkingtheshipcarryinghim,drowningallonboard.
***
Eventually,Pythagoras’followerscameupwithanewstrategy.Iftheycouldn’tunifymathematicsonafoundationofnumbers,theywouldunifyitonafoundationofgeometry.Thiswastheoriginoftheruler-and-compassconstructionsstillusedtodaytoteachgeometry;nonumbersareusedorneeded.Latermathematicianscameupwithanalternate,number-theoreticproofoftheirrationalityof .One
versionwasincludedasproposition117insomeeditionsofBookXofEuclid’sElements.WhiletheproofpredatesEuclid,itwasaddedtoElementssometimeafterthebook’soriginalpublication.Inanycase,itisanimportantproof:
Theorem3.5: isirrational.
Proof.Assume isrational.Thenitcanbeexpressedastheratiooftwointegersmandn,wherem/nisirreducible:
m2iseven,whichmeansthatmisalsoeven,7sowecanwriteitas2timessomenumberu,substitutetheresultintotheprecedingequation,anddoabitmorealgebraicmanipulation:
7Thisiseasilyshown:Theproductoftwooddnumbersisanoddnumber,soifmwerenoteven,m2couldnotbeeven.EuclidprovedthisandmanyotherresultsaboutoddandevennumbersearlierinElements.
n2iseven,whichmeansthatnisalsoeven.Butifmandnarebotheven,thenm/nisnotirreducible—acontradiction.Soourassumptionisfalse;thereisnowaytorepresent astheratiooftwointegers.
3.7ThoughtsontheChapterTheancientGreeks’fascinationwith“shapes”ofnumbersandotherpropertiessuchasprimeandperfectwerethebasisofthemathematicalfieldofnumbertheory.Someofthealgorithmstheyused,suchastheSieveofEratosthenes,arestillveryelegant,thoughwesawhowtoimprovetheirefficiencyfurtherbyusingsomemodernoptimizationtechniques.
***
Towardtheendofthechapter,wesawtwodifferentproofsthat isirrational,onegeometricandonealgebraic.Thefactthatwehavetwocompletelydifferentproofsofthesameresultisgood.Itisactuallyessentialformathematicianstolookformultipleproofsofthesamemathematicalfact,sinceitincreasestheirconfidenceintheresult.Forexample,Gaussspentmuchofhiscareercomingupwithmultipleproofsforoneimportanttheorem,thequadraticreciprocitylaw.ThediscoveryofirrationalnumbersemergedfromthePythagoreans’attemptstorepresentcontinuous
realitywithdiscretenumbers.Whileatfirstglancewemightthinktheywerenaivetobelievethattheycouldaccomplishthis,computerscientistsdothesamethingtoday—weapproximatetherealworldwithbinarynumbers.Infact,thetensionbetweencontinuousanddiscretehasremainedacentralthemeinmathematicsthroughthepresentday,andwillprobablybewithusforever.Butratherthanbeingaproblem,thistensionhasactuallybeenthesourceofgreatprogressandrevolutionaryinsights.
4.Euclid’sAlgorithm
Thewholestructureofnumbertheoryrestsonasinglefoundation,namelythealgorithmforfindingthegreatestcommondivisor.
Dirichlet,LecturesonNumberTheoryInthepreviouschapter,wemetPythagorasandthesecretiveorderhefoundedtostudyastronomy,geometry,numbertheory,andmusic.WhilethePythagoreans’failuretofindacommonmeasureofthesideandthediagonalofasquareendedthedreamofreducingtheworldtonumbers,theideaofagreatestcommonmeasure(GCM)turnedouttobeanimportantoneformathematics—andeventuallyforprogramming.Inthischapter,we’llintroduceanancientalgorithmforGCMthatwe’llbeexploringthroughouttherestofthebook.
4.1AthensandAlexandriaTosetthestageforthediscoveryofthisalgorithm,wenowturntooneofthemostamazingtimesandplacesinhistory:Athensinthe5thcenturyBC.For150yearsfollowingthemiraculousdefeatoftheinvadingPersiansinthebattlesofMarathon,Salamis,andPlatea,Athensbecamethecenterofculture,learning,andscience,layingthefoundationsformuchofWesterncivilization.ItwasinthemiddleofthisperiodofAthenianculturaldominancethatPlatofoundedhisfamous
Academy.AlthoughwethinkofPlatotodayasaphilosopher,thecenteroftheAcademy’sprogramwasthestudyofmathematics.AmongPlato’sdiscoverieswerewhatwenowcallthefivePlatonicsolids—theonlythree-dimensionalshapesinwhicheveryfaceisanidenticalregularpolygon.
Plato(429BC–347BC)
PlatowasbornintooneoftheancientnoblefamiliesofAthens.Asayoungman,hebecameafollowerofSocrates,oneofthefoundersofphilosophy,whotaughtandlearnedbyquestioning,especiallyexaminingone’sownlifeandassumptions.Socrateswasugly,bug-eyed,shabbilydressed,old,andonlyalowlystonemasonbytrade,but
hisideaswererevolutionary.Atthetime,self-proclaimedwisemen(“Sophists”)promisedtoteachtheirstudentstotakeanysideofanargumentandmanipulatethevoters.SocrateschallengedtheSophists,questioningtheirsupposedwisdomandmakingthemlookfoolish.WhiletheSophistschargedsubstantialfeestosharetheirknowledge,Socrates’followersreceivedhistrainingforfree.Tothisday,thetechniqueSocratesintroducedofaskingquestionstogetatthetruthisknown
astheSocraticmethod.AlthoughSocrateswasadmiredbysome,andsomeofhisfollowerswentontobeprominentleaders,hewasgenerallyconsideredtobeanotorioustroublemaker,andwaspubliclyridiculedinAristophanes’famousplayClouds.Eventually,in399BC,Socrateswasputontrialforcorruptingthecity’syouth,andwascondemnedtodeathbypoisoning.PlatowasprofoundlyinfluencedbySocrates,andmostofhisownwritingstaketheformof
dialoguesbetweenSocratesandvariousopponents.PlatowasdevastatedbySocrates’execution,andbythefactthatasocietywoulddestroyitswisestandmostjustmember.HeleftAthensindespair,studyingforawhilewiththepriestsinEgypt,andlaterlearningmathematicsfromthePythagoreansinsouthernItaly.Adecadeorsolater,hereturnedtoAthensandfoundedwhatisessentiallytheworld’sfirstuniversityataplacecalledtheAcademy,namedafteranancientheroAcademus.UnlikethesecretteachingsofthePythagoreans,theAcademy’sprogramofstudywaspublicandavailabletoeveryone:menandwomen,Greeksandbarbarians,freeandslave.ManyofPlato’sdialogues,suchasApology,Phaedo,andSymposium,areasbeautifullywritten
asanypoetry.AlthoughPlato’sbest-knownworkstodayareconcernedwithavarietyofethicalandmetaphysicalissues,mathematicsplayedacentralroleinthecurriculumoftheAcademy.Infact,Platohadtheinscription“Letnooneignorantofgeometryenter”writtenovertheentrance.HegatheredmanytopmathematiciansofthetimetoteachattheAcademyanddevelopauniformcourseofstudy.WhilePlatodidnotleaveusanymathematicalworks,manymathematicalideasarespreadthroughouthisdialogues,andoneofthem,Meno,isdesignedtodemonstratethatmathematicalreasoningisinnate.Onseveraloccasions,PlatotraveledtoSyracusetoinfluencethelocalrulertointroduceajust
society.Hewasunsuccessful;infact,oneofthetripsannoyedtherulersomuchthathearrangedforPlatotobesoldintoslavery.Fortunately,thephilosopherwasquicklyransomedbyanadmirer.ItishardtoexaggeratetheinfluenceofPlatoonEuropeanthought.AstheprominentBritish
philosopherWhiteheadsaid,“ThesafestgeneralcharacterizationoftheEuropeanphilosophicaltraditionisthatitconsistsofaseriesoffootnotestoPlato.”
AthenianculturespreadthroughouttheMediterranean,especiallyduringthereignofAlexandertheGreat.Amonghisachievements,AlexanderfoundedtheEgyptiancityofAlexandria(namedforhimself),whichbecamethenewcenterofresearchandlearning.Morethanathousandscholarsworkedinwhatwewouldnowthinkofasaresearchinstitute,theMouseion—the“InstitutionoftheMuses”—fromwhichwegetourword“museum.”Thescholars’patronsweretheGreekkingsofEgypt,thePtolemys,whopaidtheirsalariesandprovidedfreeroomandboard.PartoftheMouseionwastheLibraryofAlexandria,whichwasgiventhetaskofcollectingalltheworld’sknowledge.Supposedlycontaining500,000scrolls,thelibrarykeptalargestaffofscribesbusycopying,translating,andeditingscrolls.
***
ItwasduringthisperiodthatEuclid,oneofthescholarsattheMouseion,wrotehisElements,oneofthemostimportantbooksinthehistoryofmathematics.Elementsincludesderivationsoffundamentalresultsingeometryandnumbertheory,aswellastheruler-and-compassconstructionsthatstudentsstilllearntoday.
Euclid(flourishedca.300BC)
WeknowverylittleaboutEuclid—notevenexactlywhenhelived.Whatwedoknowisthathetookgeometryveryseriously.AccordingtoastorytoldbythephilosopherProclusDiadochus,oneofPlato’slatersuccessorsasheadoftheAcademy:“Ptolemy[thekingofEgypt]onceaskedEuclidwhethertherewasanyshorterwaytoaknowledgeofgeometrythanbystudyoftheElements,whereuponEuclidansweredthattherewasnoroyalroadtogeometry.”ItisprobablethatEuclidstudiedattheAcademysometimeafterPlato’sdeathandbroughtthemathematicshelearnedtoAlexandria.AlthoughweknowalmostnothingelseaboutEuclid’slife,wedoknowabouthiswork.His
Elementsincorporatedmathematicalresultsandproofsfromseveralexistingtexts.Acarefulreadingrevealssomeoftheselayers;forexample,sinceancienttimestheworkonthetheoryofproportionsinBookVhasgenerallybeenbelievedtobebasedontheworkofPlato’sstudent,Eudoxus.ButitwasEuclidwhowovetheseideastogethertoformacarefullycraftedcoherentstory.InBookI,hestartswiththefundamentaltoolsforgeometricconstructionwithrulerandcompassandendswithwhatwenowcallthePythagoreanTheorem(PropositionI,47).Inthe13thandfinalbook,heshowshowtoconstructthefivePlatonicsolids,andprovesthattheyaretheonlyregularpolyhedra(bodieswhosefacesarecongruent,regularpolygons)thatexist.Euclid’sElementsshowsasenseofpurposeuniqueinthehistoryofmathematics.Each
propositionandproofisthereforareason;nounnecessaryresultsarepresented.Howeverbeautiful,notheoremispresentedunlessitisneededforthelargerstory.Euclidalsoprefersproofsthatconstructasmanyusefulresultsaspossiblewiththefewestruler-and-compassoperations.Hisapproachisreminiscentofamodernprogrammerstrivingforminimalelegantalgorithms.Fromitspublicationaround300BCuntilthebeginningofthe20thcentury,Euclid’sElements
wasusedasthebasisofmathematicaleducation.ItwasnotonlyscientistsandmathematicianswhostudiedEuclid;greatpoliticalleaderssuchasThomasJeffersonandAbrahamLincolnalsoadmiredandstudiedElementsthroughouttheirlives.Evennow,manypeoplebelievethatstudentswouldstillbenefitfromthisapproach.
4.2Euclid’sGreatestCommonMeasureAlgorithmBookXofEuclid’sElementscontainedaconcisetreatmentofincommensurablequantities:
Proposition2.If,whenthelessoftwounequalmagnitudesiscontinuallysubtractedinturnfromthegreater,thatwhichisleftnevermeasurestheonebeforeit,thenthetwomagnitudesareincommensurable.
Essentially,Euclidissayingwhatweobservedearlierinthechapter:ifourprocedureforcomputinggreatestcommonmeasureneverterminates,thenthereisnocommonmeasure.EuclidthengoesontoexplicitlydescribethealgorithmandprovethatitcomputestheGCM.This
diagrammaybeusefulinfollowingtheproof:
Sincethisisthefirstalgorithmterminationproofinhistory,we’reincludingtheentiretext,usingSirThomasHeath’stranslation:
Proposition3.Giventwocommensurablemagnitudes,tofindtheirgreatestcommonmeasure.
Proof.
LetthetwogivencommensurablemagnitudesbeAB,CDofwhichABistheless;thusitisrequiredtofindthegreatestcommonmeasureofAB,CD.NowthemagnitudeABeithermeasuresCDoritdoesnot.Ifthenitmeasuresit—anditmeasuresitselfalso—ABisacommonmeasureofAB,CD.Anditismanifestthatitisalsothegreatest;foragreatermagnitudethanthemagnitudeABwillnot
measureAB.Next,letABnotmeasureCD.Then,ifthelessbecontinuallysubtractedinturnfromthegreater,thatwhichisleftoverwill
sometimemeasuretheonebeforeit,becauseAB,CDarenotincommensurable;[cf.X.2]letAB,measuringED,leaveEClessthanitself,letEC,measuringFB,leaveAFlessthanitself,andletAFmeasureCE.Since,then,AFmeasuresCE,whileCEmeasuresFB,thereforeAFwillalsomeasureFB.Butitmeasuresitselfalso;thereforeAFwillalsomeasurethewholeAB.ButABmeasuresDE;thereforeAFwillalsomeasureED.ButitmeasuresCEalso;thereforeitalsomeasuresthewholeCD.ThereforeAFisacommonmeasureofAB,CD.Isaynextthatitisalsothegreatest.For,ifnot,therewillbesomemagnitudegreaterthanAFwhichwillmeasureAB,CD.LetitbeG.SincethenGmeasuresAB,whileABmeasuresED,thereforeGwillalsomeasureED.ButitmeasuresthewholeCDalso;thereforeGwillalsomeasuretheremainderCE.ButCEmeasuresFB;thereforeGwillalsomeasureFB.ButitmeasuresthewholeABalso,anditwillthereforemeasuretheremainderAF,thegreater
[measuring]theless:whichisimpossible.ThereforenomagnitudegreaterthanAFwillmeasureAB,CD;thereforeAFisthegreatestcommon
measureofAB,CD.ThereforethegreatestcommonmeasureofthetwogivencommensurablemagnitudesAB,CDhas
beenfound.
This“continualsubtraction”approachtoGCMisknownasEuclid’salgorithm(orsometimestheEuclideanalgorithm).It’saniterativeversionofthegcmfunctionwesawinChapter3.Aswedidbefore,wewilluseC++-likenotationtoshowitsimplementation:Clickheretoviewcodeimage
line_segmentgcm0(line_segmenta,line_segmentb){
while(a!=b){
if(b<a)a=a-b;
elseb=b-a;
}
returna;
}
InEuclid’sworld,segmentscannotbezero,sowedonotneedthisasaprecondition.
Exercise4.1.gcm0isinefficientwhenonesegmentismuchlongerthantheother.Comeupwithamoreefficientimplementation.Rememberyoucan’tintroduceoperationsthatcouldn’tbedonebyruler-and-compassconstruction.
Exercise4.2.Provethatifasegmentmeasurestwoothersegments,thenitmeasurestheirgreatestcommonmeasure.
Toworktowardamoreefficientversionofline_segment_gcm,we’llstartbyrearranging,checkingforb<aaslongaswecan:Clickheretoviewcodeimage
line_segmentgcm1(line_segmenta,line_segmentb){
while(a!=b){
while(b<a)a=a-b;
std::swap(a,b);
}
returna;
}
Wecouldavoidaswapinthecasewherea=b,butthatwouldrequireanextratest,andwe’renotquitereadytooptimizethecodeanyway.Instead,weobservethattheinnerwhileloopiscomputingtheremainderofaandb.Let’sfactoroutthatpieceoffunctionality:Clickheretoviewcodeimage
line_segmentsegment_remainder(line_segmenta,line_segmentb){
while(b<a)a=a-b;
returna;
}
Howdoweknowtheloopwillterminate?It’snotasobviousasitmightappear.Forexample,ifourdefinitionofline_segmentincludedthehalflinestartingatapointandcontinuinginfinitelyinonedirection,thecodewouldnotterminate.Therequiredassumptionsareencapsulatedinthefollowing
axiom:AxiomofArchimedes:Foranyquantitiesaandb,thereisanaturalnumbernsuchthata≤nb.Essentially,whatthissaysisthattherearenoinfinitequantities.
***
NowwecanrewriteourGCMfunctionwithacalltosegment_remainder:Clickheretoviewcodeimage
line_segmentgcm(line_segmenta,line_segmentb){
while(a!=b){
a=segment_remainder(a,b);
std::swap(a,b);
}
returna;
}
Sofarwehaverefactoredourcodebutnotimproveditsperformance.Mostoftheworkisdoneinsegment_remainder.Tospeedupthatfunction,wewillusethesameideaasinEgyptianmultiplication—doublingandhalvingourquantities.Thisrequiresknowingsomethingabouttherelationshipofdoubledsegmentstoremainder:
Lemma4.1(RecursiveRemainderLemma):Ifr=segment_remainder(a,2b),then
Suppose,forexample,thatwewantedtofindtheremainderofsomenumberndividedby10.We’lltrytotaketheremainderofndividedby20.Iftheresultislessthan10,we’redone.Iftheresultisbetween11and20,we’lltakeaway10fromtheresultandgettheremainderthatway.Usingthisstrategy,wecanwriteourfasterfunction:
Clickheretoviewcodeimage
line_segmentfast_segment_remainder(line_segmenta,
line_segmentb){
if(a<=b)returna;
if(a-b<=b)returna-b;
a=fast_segment_remainder(a,b+b);
if(a<=b)returna;
returna-b;
}
It’srecursive,butit’salessintuitiveformofupwardrecursion.Inmostrecursiveprograms,wegodownfromnton–1whenwerecurse;here,we’remakingourargumentbiggereverytime,goingfromnto2n.It’snotobviouswheretheworkisdone,butitworks.Let’slookatanexample.Supposewehaveasegmentaoflength45andasegmentboflength6,and
wewanttofindtheremainderofadividedbyb:
RememberthatsincetheGreekshadnonotionofazero-lengthsegment,theirremainderswereintherange[1,n].Westillhavetheoverheadofrecursion,sowe’lleventuallywanttocomeupwithaniterativesolution,
butwe’llputthatasidefornow.Finally,wecanplugthiscodeintoourGCMfunction,providingasolutiontoExercise4.1:
Clickheretoviewcodeimage
line_segmentfast_segment_gcm(line_segmenta,line_segmentb){
while(a!=b){
a=fast_segment_remainder(a,b);
std::swap(a,b);
}
returna;
}
Ofcourse,nomatterhowfastitis,thiscodewillstillneverterminateifaandbdonothaveacommonmeasure.
4.3AMillenniumwithoutMathematicsAswehaveseen,ancientGreecewasasourceofseveralcenturiesofastonishingmathematicaldevelopments.Bythe3rdcenturyBC,mathematicswasaflourishingfieldofstudy,withArchimedes(bestknowntodayforastoryaboutdiscoveringtheprincipleofbuoyancyinhisbathtub)itsmostdominantfigure.Unfortunately,theriseofRomanpowerledtoastagnationinWesternmathematicsthatwouldlastforalmost1500years.WhiletheRomansbuiltgreatworksofengineering,theyweregenerallyuninterestedinadvancingthemathematicsthatmadethesestructurespossible.AsthegreatRomanstatesmanCicerosaidinhisTusculanDisputations:
AmongtheGreeksgeometrywasheldinhighesthonor;nothingcouldoutshinemathematics.Butwehavelimitedtheusefulnessofthisarttomeasuringandcalculating.
WhiletherewereGreekmathematiciansworkinginRomantimes,itisaremarkablefactthatthereisnorecordofanyoriginalmathematicaltextwritteninLatin(thelanguageofancientRome)atthattime.TheperiodofhistorythatfollowedwasnotkindtotheformerlygreatsocietiesofEurope.In
Byzantium,theGreek-speakingEasternremnantoftheformerRomanEmpire,mathematicswasstillstudied,butinnovationdeclined.Bythe6thto7thcenturies,scholarsstillreadEuclid,butusuallyjustthefirstbookofElements;Latintranslationsdidn’tevenbothertoincludetheproofs.Bytheendofthefirstmillennium,ifyouwereaEuropeanwhowantedtostudymathematics,youhadtogotocitieslikeCairo,Baghdad,orCordobaintherealmoftheArabs.
OtherMathematicalTraditionsThroughoutancienttimes,mathematicsdevelopedinmanypartsoftheworld.Civilizationdependsonmathematics.Allmajorcivilizationsdevelopednumbersystems,whichwereafundamentalrequirementfortwocriticalcivicactivities:collectingtaxesandcomputingcalendarstodeterminecultivationdates.Furthermore,allmajorcivilizationsdevelopedcommonmathematicalconcepts,suchas
Pythagoreantriples(setsofthreeintegersa,b,cwherea2+b2=c2).WhilesomehavearguedthatthisimpliesacommonNeolithicsourceofmathematicalknowledgethatspreadthroughouttheworld,thereisnoevidenceforthisclaim.Todayitseemsmorelikelythatthisissimplythemathematicalequivalentofconvergentevolutioninbiology,wherethesamecharacteristicsevolveindependentlyinunrelatedspecies.Thefactthatthesesamemathematicalideaswererediscoveredindependentlysuggeststheirfundamentalnature.Manycivilizationsdevelopedimportantmathematicaltraditionsatsomepointintheirhistory.
Forexample,inChina,3rd-centurymathematicianandpoetLiuHuiwroteimportantcommentariesonanearlierbook,NineChaptersontheMathematicalArt,andextendedthework.Amongotherdiscoveries,hedemonstratedthatthevalueofπmustbegreaterthan3,andprovidedseveralgeometrictechniquesforsurveying.InIndia,5th-centurymathematicianandastronomerAryabhatawroteafoundationaltextcalledtheAryabhatiya,whichincludedalgorithmsforcomputingsquareandcuberoots,aswellasgeometrictechniques.IdeasofIndianmathematicswerefurtherdevelopedbyArab,Persian,andJewishscholars,allwritinginArabic,whointurnheavilyinfluencedtherebirthofEuropeanmathematicsintheearly13thcentury.ComputerscienceemergedfromthisreinvigoratedEuropeanmathematics,sothisiswhatwe
arefocusingon.Asprogrammers,weareallheirsofthistradition.
4.4TheStrangeHistoryofZeroThenextdevelopmentinthehistoryofEuclid’salgorithmrequiredsomethingtheGreeksdidn’thave:zero.Youmayhaveheardthatancientsocietieshadnonotionofzero,andthatitwasinventedbyIndiansorArabs,butthisisonlypartiallycorrect.Infact,Babylonianastronomerswereusingzeroasearlyas1500BC,togetherwithapositionalnumbersystem.However,theirnumbersystemusedbase60.Therestoftheirsocietyusedbase10—forexample,incommerce—withouteitherzeroorpositionalnotation.Amazingly,thisstateofaffairspersistedforcenturies.GreekastronomerseventuallylearnedtheBabyloniansystemandusedit(stillinbase60)fortheirtrigonometriccomputations,butagain,thisapproachwasusedonlyforthisoneapplicationandwasunknowntotherestofsociety.(ItwasalsotheseGreekastronomerswhostartedusingtheGreekletteromicron,whichlooksjustlikeourletter“O,”torepresentzero.)Whatisparticularlysurprisingaboutthelackofzerooutsideofastronomyisthatitpersisteddespite
thefactthattheabacuswaswellknownandcommonlyusedforcommerceinnearlyeveryancientcivilization.Abaciconsistofstonesorbeadsarrangedincolumns;thecolumnscorrespondto1s,10s,100s,andsoon,andeachbeadrepresentsoneunitofagivenpowerof10.Inotherwords,ancientsocietiesusedadevicethatrepresentednumbersinbase10positionalnotation,yettherewasnocommonlyusedwrittenrepresentationofzerountil1000yearslater.TheunificationofawrittenformofzerowithadecimalpositionalnotationisduetoearlyIndian
mathematicianssometimearoundthe6thcenturyAD.ItthenspreadtoPersiabetweenthe6thand9thcenturiesAD.Arabscholarslearnedthetechniqueandspreaditacrosstheirempire,fromBaghdadintheeasttoCordobainthewest.ThereisnoevidencethatzerowasknownanywhereinEuropeoutsidethisempire(evenintherestofSpain);300yearswouldpassbeforethisinnovationcrossedfromoneculturetotheother.Thebreakthroughcamein1203whenLeonardoPisano,alsoknownasFibonacci,publishedLiber
Abaci(“TheBookofCalculation”).Inadditiontointroducingzeroandpositionaldecimalnotation,thisastonishingbookdescribedtoEuropeans,forthefirsttime,thestandardalgorithmsfordoingarithmeticthatwearenowtaughtinelementaryschool:longaddition,longsubtraction,longmultiplication,andlongdivision.Withonestroke,LeonardobroughtmathematicsbacktoEurope.
LeonardoPisano(1170–ca.1240)
TheItaliancityofPisa,whichtodayislandlocked,wasamajorportandnavalpowerinthe12thand13thcentury.ItcompetedwithVeniceasthedominanttradingcenterintheMediterranean.ThousandsofPisantraderscrisscrossedthesearoutestotheMiddleEast,Byzantium,NorthAfrica,andSpain,andthePisangovernmentsenttraderepresentativestomajorcitiestoensuretheirsuccess.Oneoftheserepresentatives,GuglielmoBonacci,waspostedtoAlgeria.HedecidedtobringhissonLeonardoalong,adecisionthatchangedthecourseofmathematicalhistory.Leonardolearned“Hindudigits”fromtheArabs,andcontinuedhisstudiesduringbusinesstrips
toEgypt,Syria,Sicily,Greece,andProvence.InhisbookLiberAbaci,hewouldgoontointroducetheirinnovations(includingzero)toEurope.ButLiberAbaciwasnotjustatranslationofotherpeople’swork:itwasafirst-classmathematicaltreatisewithmanyfundamentalnewcontributions.Leonardowouldgoontowriteseveralmorebooksonvariousbranchesofmathematics,includingsomeofthemostimportantmathematicaldevelopmentsincenturies.HecalledhimselfLeonardoPisano(“LeonardothePisan”),althoughsincethe19thcenturyhe
hasusuallybeenknownasFibonacci,anabbreviationoffiliusBonacci(“sonofBonacci”).Leonardo’sfamereachedtheHolyRomanEmperor,FrederickII,agreatintellectualconversant
inmanylanguagesandapatronofscienceandmathematics,whosecourtwasinPalermo,Sicily.FrederickcametoPisaandorganizedachallengetoLeonardobyhiscourtmathematicians.Leonardoperformedwellandimpressedthevisitingdignitaries.Towardtheendofhislife,thecityofPisagavehimasalaryasarewardforhisgreatcontributions.
LeonardoPisano’slaterworkLiberQuadratorum(“TheBookofSquares”),publishedin1225,isprobablythegreatestworkonnumbertheoryinthetimespanbetweenDiophantus1000yearsearlierandthegreatFrenchmathematicianPierredeFermat400yearslater.Hereisoneoftheproblemsfromthebook:
Exercise4.3(easy).Provethat .
WhywasaproblemlikethisdifficultfortheGreeks?Theyhadnoterminatingprocedureforcomputingcuberoots(infact,itwaslaterproventhatnosuchprocessexists).Sofromtheirperspective,the
problemstartsout:“First,executeanonterminatingprocedure....”Leonardo’sinsightwillbefamiliartoanymiddle-schoolalgebrastudent,butitwasrevolutionaryin
the13thcentury.Basically,whathesaidwas,“EventhoughIdon’tknowhowtocompute ,I’lljustpretendIdoandassignitanarbitrarysymbol.”Here’sanotherexampleofthekindofproblemLeonardosolved:
Exercise4.4.ProvethefollowingpropositionfromLiberQuadratorum:Foranyoddsquarenumberx,thereisanevensquarenumbery,suchthatx+yisasquarenumber.
Exercise4.5(hard).ProvethefollowingpropositionfromLiberQuadratorum:Ifxandyarebothsumsoftwosquares,thensoistheirproductxy.(ThisisanimportantresultthatFermatbuildson.)
4.5RemainderandQuotientAlgorithmsOncezerowaswidelyusedinmathematics,itactuallytookcenturieslongerbeforeitoccurredtoanyonethatasegmentcouldhavezerolength—specifically,thesegment .Zero-lengthsegmentsforceustorethinkourGCMandremainderprocedures,becauseArchimedes’
axiomnolongerholds—wecanaddazero-lengthsegmentforever,andwe’llneverexceedanonzerosegment.Sowe’llallowthefirstargumentatobezero,butweneedapreconditiontoensurethatthesecondargumentbisnotzero.Havingzeroalsoletsusshiftourremainderstotherange[0,n–1],whichwillbecrucialformodulararithmeticandotherdevelopments.Here’sthecode:Clickheretoviewcodeimage
line_segmentfast_segment_remainder1(line_segmenta,
line_segmentb){
//precondition:b!=0
if(a<b)returna;
if(a-b<b)returna-b;
a=fast_segment_remainder1(a,b+b);
if(a<b)returna;
returna-b;
}
Theonlythingwe’vechangedaretheconditions;everywhereweusedtosaya<=b,wenowchecka<b.Let’sseeifwecangetridoftherecursion.Everytimewerecursedown,wedoubleb,sointhe
iterativeversion,we’dliketoprecomputethemaximumamountofdoublingwe’llneed.Wecandefineafunctionthatfindsthefirstrepeateddoublingofbthatexceedsthedifferencea–b:Clickheretoviewcodeimage
line_segmentlargest_doubling(line_segmenta,line_segmentb){
//precondition:b!=0
while(a-b>=b)b=b+b;
returnb;
}
Nowweneedouriterativefunctiontodothesamecomputationthathappensonthewayoutoftherecursion.Eachtimeitreturns,thevalueofbhasthevalueithadbeforethemostrecentrecursivecall(i.e.,themostrecentdoubling).Sotosimulatethis,theiterativeversionneedstorepeatedly“undouble”thevalue,whichitwilldobycallingafunctionhalf.Remember,we’restill“computing”withrulerandcompass.Fortunately,thereisaEuclideanprocedurefor“halving”asegment,1sowecanuseahalffunction.Nowwecanwriteaniterativeversionofremainder:
Clickheretoviewcodeimage
line_segmentremainder(line_segmenta,line_segmentb){
//precondition:b!=0
if(a<b)returna;
line_segmentc=largest_doubling(a,b);
a=a-c;
while(c!=b){
c=half(c);
if(c<=a)a=a-c;
}
returna;
}
1Drawacirclewiththecenteratoneendofthesegmentandradiusequaltothesegment;repeatfortheotherend.Userulertoconnectthetwopointswherethecirclesintersect.Theresultinglinewillbisecttheoriginalsegment.
Thefirstpartofthefunction,whichfindsthelargestdoublingvalue,doeswhatthe“downward”recursiondoes,whilethelastpartdoeswhathappensonthewaybackupoutoftherecursivecalls.Let’slookagainatourexampleoffindingtheremainderof45dividedby6,thistimewiththenewremainderfunction:
Noticethatthesuccessivevaluesofcintheiterativeimplementationarethesameasthevaluesofbfollowingeachrecursivecallintherecursiveimplementation.Also,comparethistothetraceofourearlierversionofthealgorithmattheendofSection4.2.Observehowtheresultsofthefirstpart(c=24anda=21)herearethesameastheinnermostrecursionintheoldexample.Thisisanextremelyefficientalgorithm,nearlyasfastasthehardwareimplementedremainder
operationinmodernprocessors.
***
Whatifwewantedtocomputequotientinsteadofremainder?Itturnsoutthatthecodeisalmostthesame.Allweneedareacoupleofminormodifications,showninbold:Clickheretoviewcodeimage
integerquotient(line_segmenta,line_segmentb){
//Precondition:b>0
if(a<b)returninteger(0);
line_segmentc=largest_doubling(a,b);
integern(1);
a=a-c;
while(c!=b){
c=half(c);n=n+n;
if(c<=a){a=a-c;n=n+1;}
}
returnn;
}
Quotientisthenumberoftimesonelinesegmentfitsintoanother,soweusethetypeintegertorepresentthiscount.Basically,wearegoingtocountmultiplesofb.Ifa<b,thenwedon’thaveanymultiplesofbandwereturn0.Butifa≥b,weinitializethecounterto1,thendoubleiteachtimewehalvec,addingonemoremultipleforeachiterationwhenitfits.Again,let’sworkthroughanexample.Thistime,insteadoffindingtheremainderof45dividedby6,we’llfindthequotientof45dividedby6.
Essentially,thisistheEgyptianmultiplicationalgorithminreverse.AndAhmesknewit:aprimitivevariantofthisalgorithm,knowntotheGreeksasEgyptiandivision,appearsintheRhindpapyrus.
4.6SharingtheCodeSincethemajorityofthecodeissharedbetweenquotientandremainder,itwouldmakemuchmoresensetocombinethemintoasinglefunctionthatreturnsbothvalues;thecomplexityofthecombinedfunctionisthesameaseitherindividualfunction.NotethatC++11allowsustouseinitializerlistsyntax{x,y}toconstructthepairthatthefunctionreturns:Clickheretoviewcodeimage
std::pair<integer,line_segment>
quotient_remainder(line_segmenta,line_segmentb){
//Precondition:b>0
if(a<b)return{integer(0),a};
line_segmentc=largest_doubling(a,b);
integern(1);
a=a-c;
while(c!=b){
c=half(c);n=n+n;
if(c<=a){a=a-c;n=n+1;}
}
return{n,a};
}
Infact,anyquotientorremainderfunctiondoesnearlyalltheworkoftheother.
ProgrammingPrinciple:TheLawofUsefulReturnOurquotient_remainderfunctionillustratesanimportantprogrammingprinciple,whichwecallthelawofusefulreturn:
Ifyou’vealreadydonetheworktogetsomeusefulresult,don’tthrowitaway.Returnittothecaller.
Thismayallowthecallertogetsomeextraworkdone“forfree”(asinthequotient_remaindercase)ortoreturndatathatcanbeusedinfutureinvocationsofthefunction.Unfortunately,thisprincipleisnotalwaysfollowed.Forexample,theCandC++programming
languageshaveseparatequotientandremainderoperators;thereisnowayforaprogrammertogetbothresultswithonecall—despitethefactthatmanyprocessorshaveaninstructionthatreturnsboth.
Mostcomputingarchitectures,whetherruler-and-compassormodernCPUs,provideaneasywaytocomputehalf;forus,it’sjusta1-bitrightshift.However,ifyoushouldhappentobeworkingwithanarchitecturethatdoesn’tsupportthisfunctionality,thereisaremarkableversionoftheremainderfunctiondevelopedbyRobertFloydandDonaldKnuththatdoesnotrequirehalving.It’sbasedontheideaoftheFibonaccisequence—anotherofLeonardoPisano’sinventions,whichwewilldiscussmoreinChapter7.Insteadofthenextnumberbeingdoublethepreviousone,we’llmakethenextnumberbethesumofthetwopreviousones:2
Clickheretoviewcodeimage
line_segmentremainder_fibonacci(line_segmenta,line_segmentb){
//Precondition:b>0
if(a<b)returna;
line_segmentc=b;
do{
line_segmenttmp=c;c=b+c;b=tmp;
}while(a>=c);
do{
if(a>=b)a=a-b;
line_segmenttmp=c-b;c=b;b=tmp;
}while(b<c);
returna;
}
2Notethatthissequencestartsatb,sothevalueswillnotbethesameasthetraditionalFibonaccisequence.
Thefirstloopisequivalenttocomputinglargest_doublinginourpreviousalgorithm.Thesecondloopcorrespondstothe“halving”partofthecode.Butinsteadofhalving,weusesubtractiontogetbacktheearliernumberintheFibonaccisequence.Thisworksbecausewealwayskeeponepreviousvalue
aroundinatemporaryvariable.
Exercise4.6.Tracetheremainder_fibonaccialgorithmasitcomputestheremainderof45and6,inthewaywetracedtheremainderalgorithminSection4.5.
Exercise4.7.Designquotient_fibonacciandquotient_remainder_fibonacci.
Nowthatwehaveanefficientimplementationoftheremainderfunction,wecanreturntoouroriginalproblem,thegreatestcommonmeasure.Usingournewremainderfunctionfromp.54,wecanrewriteEuclid’salgorithmlikethis:Clickheretoviewcodeimage
line_segmentgcm_remainder(line_segmenta,line_segmentb){
while(b!=line_segment(0)){
a=remainder(a,b);
std::swap(a,b);
}
returna;
}
Sincewenowallowremaindertoreturnzero,theterminationconditionforthemainloopiswhenb(thepreviousiteration’sremainder)iszero,insteadofcomparingaandbaswedidoriginally.Wewillusethisalgorithmforthenextfewchapters,leavingitsstructureintactbutexploringhowit
appliestodifferenttypes.Wewillleaveourruler-and-compassconstructionsbehindandimplementthealgorithmwithadigitalcomputerinmind.Forexample,forintegers,theequivalentfunctionisthegreatestcommondivisor(GCD):Clickheretoviewcodeimage
integergcd(integera,integerb){
while(b!=integer(0)){
a=a%b;
std::swap(a,b);
}
returna;
}
Thecodeisidentical,exceptthatwehavereplacedline_segmentwithintegerandusedthemodulusoperator%tocomputetheremainder.Sincecomputershaveinstructionstocomputetheintegerremainder(asinvokedbytheC++modulusoperator),it’sbettertousethemthantorelyondoublingandhalving.
4.7ValidatingtheAlgorithmHowdoweknowthattheintegerGCDalgorithmworks?Weneedtoshowtwothings:first,thatthealgorithmterminates,andsecond,thatitcomputestheGCD.Toprovethatthealgorithmterminates,werelyonthefactthat
0≤(amodb)<b
Therefore,ineachiteration,theremaindergetssmaller.Sinceanydecreasingsequenceofpositiveintegersisfinite,thealgorithmmustterminate.ToprovethatthealgorithmcomputestheGCD,westartbyobservingthatineachiteration,the
algorithmcomputesaremainderofaandb,whichbydefinitionis
r=a–bq
whereqistheintegerquotientofadividedbyb.Sincegcd(a,b)bydefinitiondividesaandalsodividesb(andthereforebq),itmustalsodivider.Wecanrewritetheremainderequationasfollows:
a=bq+r
Bythesamereasoning,sincegcd(b,r)bydefinitiondividesb(andthereforebq),andalsodividesr,itmustalsodividea.Sincepairs(a,b)and(b,r)havethesamecommondivisors,theyhavethesamegreatestcommondivisor.Thereforewehaveshownthat
Ateachiteration,thealgorithmreplacesgcd(a,b)withgcd(b,r)bytakingtheremainderandswappingthearguments.Hereisthelistofremainders,startingwitha0andb0,theinitialargumentstothefunction:
Usingthedefinitionofremainder,werewritethesequencecomputedbythealgorithmlikethis:
WhatEquation4.1guaranteesisthattheGCDstaysthesameeachtime.Inotherwords:
gcd(a0,b0)=gcd(b0,r1)=gcd(r1,r2)=···=gcd(rn–1,rn)
Butweknowthattheremainderofrn-1andrnis0,becausethat’swhattriggerstheterminationofthealgorithm.Andgcd(x,0)=x.So
gcd(a0,b0)=···=gcd(rn–1,rn)=gcd(rn,0)=rn
whichisthevaluereturnedbythealgorithm.Therefore,thealgorithmcomputestheGCDofitsoriginalarguments.
4.8ThoughtsontheChapterWe’veseenhowanancientalgorithmforcomputingthegreatestcommonmeasureoftwolinesegmentscouldbeturnedintoamodernfunctionforcomputingtheGCDofintegers.We’velookedatvariantsofthealgorithmandseenitsrelationshiptofunctionsforfindingthequotientandtheremainder.DoestheGCDalgorithmworkforotherthingsbesideslinesegmentsandintegers?Inotherwords,isthereawaytomakethealgorithmmoregeneral?We’llcomebacktothatquestionlaterinthebook.
5.TheEmergenceofModernNumberTheory
Mathematicianshavetriedinvaintothisdaytodiscoversomeorderinthesequenceofprimenumbers,
andwehavereasontobelievethatitisamysteryintowhichthehumanmindwillneverpenetrate.
LeonhardEulerInthepreviouschapter,wesawhowthefledgingfieldofnumbertheory,whichhadfascinatedtheancientGreeks,wasrevivedinmedievalEuropeafteralongperiodofdormancy.Butnumbertheoryinitsmodernsensereallyemergedafewhundredyearslater,in17th-centuryFrance.Forthischapter,wearegoingtoputprogrammingasideforabitandlearnsomeoftheresultsdiscoveredby17th-centuryFrenchmathematicians,whichwe’lluseforsomeimportantcomputerapplicationslateron.
5.1MersennePrimesandFermatPrimesMathematiciansoftheRenaissancerekindledtheancientGreeks’fascinationwithprimenumbers.Theywonderedwhethertherewerecertainpredictablepatternsofprimes.Theywereparticularlyinterestedinprimesoftheform2n-1,since(aswesawinSection3.4)thiswasthesourceofperfectnumbers.Mathematiciansfromthe15thtothe18thcenturies,liketheGreeksbeforethem,feltthatthesenumbershadspecialimportance.Lettersofthe17th-centurymathematiciansFermat,Mersenne,andDescartescontainmanyreferencestoperfectnumbers,aswellasacloselyrelatedconcept,amicablenumbers.Inthe18thcentury,thegreatmathematicianLeonhardEulerstillfoundthesubjecttobeofprimaryimportance.AswesawinChapter3,theGreeksknewthattheycouldgenerateperfectnumbersfromprimesofthe
form2n-1.Theyknewthatnumbersofthatformareprimeforn=2,3,5,and7,andpossibly13.In1536,HudalricusRegiusshowedthattheexpressionwasnonprimeforn=11,byfinding
211-1=2047=23x89
PietroCataldiaddedseveralmorevaluesofntothelistin1603—17,19,(23),(29),31,and(37)—buthalfofthese(showninparentheses)wereincorrect.PierredeFermatdiscoveredthat
Inhis1644bookCogitataPhysicoMathematica,theFrenchmathematicianMersennestatesthatforn≤257,2n-1isprimeifandonlyif
n=2,3,5,7,13,17,19,31,(67),117,(257)
Twoofthesewerewrong(showninparentheses),andhemissed89and107.BecauseofMersenne’sconjecture,primesofthisformbecameknownasMersenneprimes.Westilldonotknowwhetherthereisaninfinitenumberofthem,buteventodayMersennenumbersarestillusedtosearchforlargeprimes.
MarinMersenne(1588–1648)
Startingaround1624,whenCardinalRichelieubecamechiefminister,Francebegantoriseasamilitary,political,cultural,andscientificpower.WhilescholarsattraditionaluniversitiesstillinterpretedAristotle’sancientworks,philosopherslikeDescartesinFrance,workingoutsideoftheuniversitysystem,wererevolutionizingthewaypeoplethoughtabouttheworld.Richelieucreatedthefirstmodernstate,withacarefullyorganizedcentralbureaucracyandmilitary,andevenofficialcontroloftheFrenchlanguage.By1660,FrancehadbecometheundisputedleaderofEurope,andFrenchbecamethedominantlanguagefordiplomatsandaristocratsinmostWesterncountriesforthenext250years.ItwasduringthisperiodthatMarinMersenne,aFrenchpolymathandamemberofthestrictreligiousorderofMinims,hadanenormousimpactonscience.AlthougheducatedbytheintellectualJesuitsandanaccomplishedclassicalscholarandmathematician,MersennechosetheextremeasceticismoftheMinims,whoheldnoproperty(evenincommon),ateastrictvegandiet,anddranknoalcohol.Mersenne’shumilityextendedtohisprofessionallife;whileotherscientistsproclaimedtheirownimportance,hiscausewastohelpothersdisseminatetheirworkandlearnabouteachother’sresults.Mersennedidsomeimportantworkonthetheoryofsoundandotherareas,buthisgreatestcontributionwasthecreationofasharedscientificcommunity.Scientificjournalsdidnotyetexist,butMersenneservedasa“virtualscientificjournal,”byexchangingletterswithfriendsandinformingthemofeachother’sresults.Mersenne’sfriendsincludedpeoplelikeGalileo,Huygens,Torricelli,Descartes,Fermat,andPascal.Infact,MersennearrangedforpublicationofGalileo’sworkinProtestantHolland,despiteGalileo’scondemnationbytheCatholicChurch.LaterinMersenne’slife,scholarswouldmeettogetherinhiscellinakindofinformalweeklyconference.Whenhisletterswerepublishedafterhisdeath,theywereinessencetheworld’sfirstscientificproceedings.
InalettertoMersenneinJune1640,Fermatwrotethathisfactorizationof237-1dependsonthefollowingthreediscoveries:
1.Ifnisnotaprime,2n-1isnotaprime.2.Ifnisaprime,2n-2isamultipleof2n.
3.Ifnisaprime,andpisaprimedivisorof2n-1,thenp-1isamultipleofn.We’lllookattheproofofdiscovery1inabit,butfornowlet’sassumethatallthreestatementsaretrue.Fermatreasonedthatif237-1isnotprime,itmusthaveaprimefactorp,whichmustbeodd.By
observation3,p-1isamultipleof37,whichisequivalenttosayingthat
p=37u+1
Also,sincepisodd,p-1=37umustbeeven,soumustbeeven.Thatmeanswecanexpressuas2v,whichgivesus:
p=74v+1
Fermatthereforenarrowedthefactoringtaskfromtryingallpossiblenumberstojustthoseprimesproducedbythisformula.Testingtheseinsequence:Whataboutv=1?No,75isnotaprime.Whataboutv=2?No,149isprime,butisnotadivisorof237-1.Whataboutv=3?Yes!223isprime,andisadivisorof237-1.So237-1isnotprime.
***
Nowlet’slookatFermat’sproofofdiscovery1,whichwestateinitscontrapositive1form.1Anyimplicationp qislogicallyequivalenttoitscontrapositive,whichistheexpression¬q ¬p.See“Implicationandthe
Contrapositive”inAppendixAformoredetails.
Theorem5.1:If2n-1isprime,thennisprime.
Proof.Supposenisnotprime.Thentheremustbefactorsuandvsuchthat
n=uv,u>1,v>1
Then
wherethelaststepusesEquation3.1,thedifferenceofpowersformula.Sinceu>1,weknowthatbothofthefollowingaretrue:
So5.1showsthatwehavefactored2n-1intotwonumberseachgreaterthan1.Butthiscontradictstheconditionofthetheoremisthat2n-1isprime.Sotheinitialassumptioninourproofmustbefalse,andnmustbeprime.
Asfordiscoveries2and3,Fermatneversharedtheproofs.InalettertohisfriendFrenicle,Fermat
wrotethat“hewouldsend[theproof]ifhedidnotfearbeingtoolong.”Weshallreturntothemsoon.
PierredeFermat(1601–1665)
PierredeFermatwasalawyerandprovincialmagistratefromToulouseinthesouthofFrance.HewasaRenaissancemaninthetraditionofMontaigne,interestedinavarietyofsubjectsincludingclassicalliterature,andfluentinLatinandGreek.Thelastofthegreatamateurmathematicians,FermatbecameinterestedinnumbertheoryafterreadingBachet’stranslationofDiophantus’ancientGreektextArithmetic.Althoughhemadeenormouscontributionstomathematics,heneverpersonallyinteractedwithothermathematicians.Infact,MersennerepeatedlyinvitedhimtovisitParis,butasfarasweknow,Fermatneverwent.Fermatoftenboastedofhisresultswhilekeepinghismethodssecret.Hewouldoftensaythathehadaproofofsomething,yetcomeupwithanexcusenottoprovideit.Whenhedidpublicizearesult,hewouldtrytodivulgeaslittleaspossibleabouthowhedidit.Duringhislife,Fermatneverpublishedhiswork,althoughhecorrespondedwithMersenneandothersthroughletters.AfterFermat’sdeath,hissonpublishedtheeditionofDiophantuswithFermat’smarginalnotes.Thesenotescontainedmanytheorems,whichweregraduallyconfirmedbyothermathematiciansinlateryears.Thelasttobesolved—whichbecameknownasFermat’sLastTheorem—wasthestatementthatan+bn=cnhasnosolutionsinpositiveintegersforn>2.Itwasfinallyprovedin1994byAndrewWiles.Fermatnotoriouslywrote“theproofistoolargetofitinthemargin”nexttohisstatementoftheLastTheorem.Asmentionedearlier,thiswashiscommonpattern;heoftengavesimilarexcusestoavoidsharinghisproofs.Althoughallbutoneofhisconjectureshavebeenconfirmed,someoftheproofsaresocomplexandlengthythatlatermathematicianssuchasGausshavebeenskepticalthatFermatactuallydiscoveredthem.Inadditiontohisworkonnumbertheory,Fermatmademajorcontributionstootherareasofmathematics.Heinventedanalyticgeometry—thestudyofequationsofcurves—beforeDescartes,butdescribedtheworkinanunpublishedmanuscript.Healsoco-inventedprobabilitytheoryinthecourseofalengthycorrespondencewithBlaisePascal.
Fermatmadealotofconjecturesforwhichheleftnoproofs,buteveryonehassincebeenproventrue
exceptone:
2n+1isprime n=2i
(Thedouble-arrowsymbolisread“ifandonlyif”;seeAppendixAfordetails.)Sincethen,numbersofthisform(22
i+1)havebeenknownasFermatprimes.It’seasytoproveapartofhisconjecture:
Theorem5.2:2n+1isprime n=2i.
Proof.Supposen≠2i.Thenoneofn’sfactorsmustbeodd,sowecanexpressthatfactoras2q+1.Thisis>1,sowecanexpressnas
n=m(2q+1)
Substitutingm(2q+1)fornandthenusingtheformulaforsumofoddpowers(Equation3.4),wefactor2n+1:
Butfactoring2n+1contradictsthepremiseoftheconjecture;itcan’thavenontrivialfactorsifit’sprime.Soourinitialassumptionintheproofisfalse,andn=2i.
Whataboutotherprimesoftheform22i+1?Fermatstatesthat3,5,17,257,65537,4294967297,and
18446744073709551617areprime,andsoarealltherestofthisform.Unfortunately,hewaswrongabouttwoofhisexamples—onlythefirstfiveareprime—andabouthisconjecture.In1732,Eulershowedthat
232+1=4294967297=641×6700417
Infact,weknowthatfor5≤i≤32,thenumbersarecomposite.ArethereanymoreFermatprimesbesidesthesefive?Asofthiswriting,nooneknows.
5.2Fermat’sLittleTheoremWenowcometooneofthemostimportantresultsinnumbertheory.
Theorem5.3(Fermat’sLittleTheorem):
Ifpisprime,ap–1–1isdivisiblebypforany0<a<p.
Fermatclaimedtohaveaproofofthetheoremin1640,butdidnotpublishit.Leibnizdiscoveredaproofsometimebetween1676and1680,butdidnotpublishiteither.Finally,Eulerpublishedtwodifferentproofsin1742and1750.Wewillprovethetheoremhere,butfirstweneedtoderiveseveralotherresults.Whilethesemayatfirstseemtobeunrelated,wewillseeshortlyhowtheycometogether.
LeonhardEuler(1707–1783)
LeonhardEuler(pronounced“OILer”)wasborninawell-educatedmiddle-classfamilyinSwitzerland.Atalentedandwell-roundedstudentwithanamazingmemory,hestudiedwithJohannBernoulli,thegreatestmathematicianofthetimeandafriendofEuler’sfather.(BernoullihimselfwasastudentofLeibniz,theco-inventorofcalculus.)Formostofthe18thcentury,CzarPetertheGreatandhissuccessorsconductedaperiodofdramaticreformthat“Europeanized”Russiansocietyandculture.OneoftheresultsofthesereformswasthecreationoftheImperialAcademyofSciencesinSt.Petersburg,whichrecruitedEuropeanscholars.Itwastherein1727thatEuler,atage20,gotajobdoingmathematicalresearch.Within10years,hisresultsinmathematics,mechanics,andevenshipbuildingestablishedhisreputationasoneofthetopscientificmindsinEurope.BythetimeFredericktheGreatrecruitedhimtocometoBerlinin1741,Eulerwasaninternationalsuperstar.Atthetime,kingsandqueensconsideredassociatingwithscientistsandotherintellectualstobeanimportantwaytoincreasetheirownstatus.ThroughoutEuler’syearsinBerlin,theFrenchandRussianroyalcourtscompetedtowoohimaway.Eventually,in1766,hereturnedtoSt.Petersburg,wherehespenttherestofhiscareer.Euler’scontributionstomathematics(andphysics)wereenormous.Heworkedinmanyareas;hefoundedmoderngraphtheoryandmadefundamentaldiscoveriesinnumbertheory.However,hisgreatestachievementwasthedevelopmentofmodernanalysis—calculusanddifferentialequations—fromtheindividualtechniquesinventedbyNewtonandLeibniztoasystematicdiscipline.Histhreebooksoncalculus(IntroductiontoAnalysisoftheInfinite,DifferentialCalculus,andIntegralCalculus)werethedefinitivetextsfornearlyacenturyandstilldeservecarefulstudy.Eulerwrotethefirstbookonpopularscience,LetterstoaGermanPrincess,inwhichheexplainstheNewtonianviewoftheworldtoalayperson.Healsowroteanelementaryalgebratextbookintendedfornon-mathematicians,whichisstillinprint.Eulerwassoprolificthatafterhisdeath,theRussianAcademyofSciencestook60yearstopublishalltheadditionalworkhehadsubmitted.Hewasgenerallyregardedasthegreatestmathematicianintheworldinhistime,andafter200years,westillshareLaplace’sviewthat“heisthemasterofusall.”
OurfirststepisanotherpropositionfromEuclid:
Theorem5.4(EuclidVII,30):Theproductoftwointegerssmallerthanaprimepisnotdivisiblebyp.
(Anotherwaytosaythisisthatifpisprimeandaandbaresmallerthanp,thenabisnotdivisiblebyp.)Ifsomenumberxisdivisiblebysomeothernumbery,thenxisamultipleofy:x=my.Ifxisnotdivisiblebyy,thendividingxbyyleavesaremainderr:x=my+r.Sowecanrestatethepropositionlikethis:
pisprime∧0<a,b<p ab=mp+r∧0<r<p
Proof.Assumethecontrary,thatabisamultipleofp.Thenforagivena,letbbethesmallestintegersuchthatab=mp.Thensincepisprime,weknowdividingpbybleavesaremainderv<b:
p=bu+v∧0<v<b
Multiplyingbothsidesoftheequationbyaandthensubstitutingwithab=mpgives
Butthismeansthatvisanintegersmallerthanbsuchthatavisamultipleofp.That’sacontradiction,sincewechosebtobethesmallestsuchnumber.Soourassumptionisfalse,andabisnotdivisiblebyp.
ThisapproachwasactuallyacommonpatternforproofsinancientGreekmathematics:choosethesmallestofsomething,andthenshowthatcertainassumptionswouldleadtoasmallerone.
***
Next,weprovearesultaboutremainders:
Lemma5.1(PermutationofRemaindersLemma):Ifpisprime,thenforany0<a<p,
where
0<ri<p∧i≠j ri≠rj
Inotherwords,ifwetakeallthemultiplesofafrom1ato(p-1)a,andexpresseachmultipleintheformqp+r,everyremainderrwillbeuniqueandthesetofremainderswillbeapermutationof{1,...,p-1}.(Weknoweachremainderislessthanp,sowehavep-1uniquenumbersintherange[1,p-1].)Example:Ifp=7anda=4,thenthelemmasaysthat
{4,8,12,16,20,24}={0·7+4,1·7+1,1·7+5,2·7+2,2·7+6,3·7+3}sotheremaindersare
{4,1,5,2,6,3}
whichisapermutationof{1,...,7-1}.
Proof.Supposeri=rjandi<j;thatis,twooftheremaindersareequal.Thenwecouldtakethedifferenceofthetwocorrespondingelementsintheset,andtheremaindersriandrjwouldcancel:
Sincetheithandjthelementsofthesetaretheproductsaiandaj,wecouldequivalentlywritethedifferenceofthesetwoelementsasaj-ai.Thatis:
Butthisisoftheformab=mp,whichimpliesthattheproductoftwointegerssmallerthanpisdivisiblebyp.SincethiscontradictsEuclidVII,30,whichwejustproved,ourassumptionmustbefalse,andtheremaindersmustbeunique.
5.3CancellationNowwewilllookatsomeresultsthatdealwiththenotionofcancellation.Ifwearemultiplyingtwonumbersxandy,theycancel(i.e.,theirproductis1)whenoneisthemultiplicativeinverseoftheother.
CancellationandModularArithmeticOnewaytoviewcancellationisinthecontextofmodulararithmetic,whichwasintroducedbyCarlGauss,whomweshallmeetinChapter8.AlthoughEulerdidnotusethistechniqueinhisproofofFermat’sLittleTheorem,youmayfinditusefultounderstandthelogic.Agoodanalogyformodulararithmeticisastandard12-hourclock.Ifit’s10o’clock,andyouhavetodosomethingthat’sgoingtotake5hours,you’llbedoneat3o’clock.Inasense,you’resayingthat10+5=3.Moreprecisely,you’resayingthat(10+5)mod12=3.(Mathematicianswouldcallnoon“0,”though.)Ofcourse,wecandomodulararithmeticinanybase.Hereareacoupleofexamplesusing7:
(6+4)mod7=3
(3x3)mod7=(3+3+3)mod7=2
Noticeinthelattercasethatwecouldalsocalculateourproductinthetraditionalway,thenexpressitintermsofmultiplesofthemodularbaseandaremainder:
(3×3)=9=(1×7)+2
Inotherwords,avaluemodulonisequivalenttotheremainderafterusingnasadivisor.Inelementaryarithmetic(forexample,arithmeticofrationalnumbers),iftheproductoftwotermsxandyis1,thentheyaresaidtocancel,andxandyarecalledinversesofeachother.Thesameistrueinmodulararithmetic,onlytheinverseswillbothbeintegers.Forexample,
(2×4)mod7=1
so2and4cancel,andareeachother’sinverse.Anegativenumberxmodnisequaltothepositivenumbern-x;it’sthepositionyou’dgettoifyou“turnedtheclockback”byxhours.Inparticular,-1modn=n–1.Justaswithelementaryarithmetic,wecanwritemultiplicationtablesformodulararithmetic.Here’soneforintegersmodulo7:
Firstweexpresstheproductasamultipleof7andaremainder;themodularproductisthenjusttheremainder.Forexample,5×4=20=(2×7)+6=6mod7,sothereisa6inthetableattheintersectionofrow5andcolumn4.Observethateveryrowisapermutationofeveryotherrow,andthateveryrowcontainsa1.Recallthatiftheproductis1,thetwofactorsareinverses.Inthetableabove,wesee,forexample,that2and4areinversessince2×4=1mod7.Here’saversionofthetablewiththeinverseofeachfactorontheleftshowninthecolumnontheright:
Formally,forintegern>1andintegeru>0,wecallvamultiplicativeinversemodulonifthereisanintegerqsuchthatuv=1+qn.Inotherwords,uandvareinversesiftheirproductdividedbynyieldsaremainderof1.Wewillrelyheavilyonthisinthefollowingproofs.
***
Thenextresultreliesonthisgeneralizednotionofcancellation:
Lemma5.2(CancellationLaw):Ifpisprime,thenforany0<a<pthereisanumber0<b<psuchthatab=mp+1.
Inotherwords,aandbcancelmodulop.Example:Supposeagainthatp=7anda=4.Isthereavalueofbthatsatisfiestheequationab=mp
+1?Let’stryallthevaluesofbuntilwefindonethatworks:
Proof.BythePermutationofRemaindersLemma,weknowthatoneofthepossibleproductsintheset
a·{1,...,p–1}
willhavearemainderof1.Inthiscasewehavep-1uniqueremaindersgreaterthan0andlessthanp,sooneofthemmustbe1.Therefore,theremustbeanotherelementbthatcancelsa.
Notethat1andp-1areself-cancelingelements—thatis,ifyoumultiplyeachbyitself,theresultis1modp,or(equivalently)theresultcanbeexpressedintheformmp+1.It’sobviousthat1·1canbeexpressedinthisform,sinceit’s0p+1.Whataboutp-1?
(p-1)2=p2-2p+1=(p-2)p+1=mp+1
Infact,1andp-1aretheonlyself-cancelingelements,whichwe’llnowdemonstrate.
Lemma5.3(Self-CancelingLaw):
Forany0<a<p,a2=mp+1 a=1∨a=p–1
Proof.Assumethereisaself-cancelingathat’sneither1norp-1:
a≠1∧a≠p–1 1<a<p–1
Rearrangingtheconditionoftheproof,wehave
a2-1=mp
Factoringtheexpressionontheleft,wehave
(a-1)(a+1)=mp
Butsincebyourassumption0<a-1,a+1<p,whichmeanswehaveaproductoftwointegerssmallerthanpthatisdivisiblebyp,acontradictionwithEuclidVII,30(seep.70).Soourassumptionisfalse,andtheonlyself-cancelingelementsare1andp-1.
WearealmostreadytoproveFermat’sLittleTheorem,butwestillneedonemoreresult:Wilson’stheorem,announcedbyEdwardWaringin1770,whoattributedittohisstudent,JohnWilson.Atthetime,Waringstatedthathewasunabletoprovethetheoremsincehedidnothavetherightnotation—inresponsetowhichGausslaterremarked,“Oneneedsnotion,notnotation.”
Theorem5.5(Wilson’sTheorem):Ifpisprime,thereexistsanintegermsuchthat
(p-1)!=mp+(p-1)
orinotherwords
(p-1)!=(p-1)modp
Proof.Bydefinition,
(p-1)!=1·2·3...(p-1)
BytheCancellationLaw,everynumberabetween1andp-1hasanumberbinthatrangethat’sitsinverse;bytheSelf-CancelingLaw,only1andp-1aretheirowninverses.Soeveryothernumberintheproductexcept1andp-1iscancelledbyitsinverse;thatis,theirproductdividedbyphasremainder1.Inotherwords,wecouldexpressallthecancelledtermstogether—allthetermsbetween1andp-1—asnp+1forsomen.Westillhaveouruncanceledterms1andp-1,soourproductnowbecomes
Thenm=np-nsatisfiesthetheorem.
Exercise5.1.Provethatifn>4iscomposite,then(n–1)!isamultipleofn.
5.4ProvingFermat’sLittleTheoremFinally,usingtheresultswe’vejustderived,wecanproveFermat’sLittleTheorem:
Ifpisprime,ap–1–1isdivisiblebypforany0<a<p.
Proof.Considertheexpression .Wecanmovetheatermsoutsidetheproduct,sowehave
Wilson’sTheoremcanbewrittenas
ThereforewecanmaketheabovesubstitutionintoEquation5.2,giving
Nowlet’sreturntotheexpression .Itsexpansioncontainsalltheterms{a,2a,3a,...,(p-1)a},whichbythePermutationofRemaindersLemma(p.71)isthesameas{q1p+r1,...,qp–1p+rp–1}.Sowecanwrite
Whenweexpandtheproductontheright,wegetasumcontainingmanytermswithp,andonethatistheproductofallri.Wegroupalltheptermstogether;theygiveussomemultipleup.Whatremainsistheproductofallri:
NowwecanapplyWilson’sTheoremagaintotheproductontheright,thenagaingroupmultiplesofp:
wherew=u+v+1.Weknowexpressions5.3and5.4areequal,andneedonlysomesimplerearrangement:
Again,wecancombinemultiplesofpontheright,givingourdesiredresult:
ap-1-1=np
Soap-1-1isdivisiblebyp.
Wealsoobservethatap-2isaninverseofa,sinceap-2·a=ap-1,whichFermat’sLittleTheoremtellsusismp+1.(Rememberthatbeinganinversewithrespecttopmeanshavingaremainderof1afterdividingbyp.)
***
WhatabouttheconverseofFermat’sLittleTheorem?Toprovethat,weneedonemoreintermediateresult:
Lemma5.4(Non-invertibilityLemma):Ifn=uv∧u,v>1,thenuisnotinvertiblemodulon.
Proof.Letn=uvandwbeaninverseofu(i.e.,wu=mn+1).Then
Soifwedefinez=(w-mv),then
(w-mv)n=zn=v
Sincen>v,thenzn>v,whichisacontradictionwithzn=v.Soucannothaveaninverse.
Definition5.1.Twonumbersmandnarecoprimeifgcd(m,n)=1.Equivalently,mandnarecoprimeiftheyhavenocommonfactorsgreaterthan1.
TheNon-invertibilityLemmatellsusthatwhenwearedealingwithnumbersmodulon,wherenisnotprime,thereareinvertibleelementsandnon-invertibleelements;elementsthatarenotcoprimetonarenotinvertible.
Theorem5.6(ConverseofFermat’sLittleTheorem):Ifforalla,0<a<n,
an-1=1+qan
thennisprime.
Proof.Supposenisnotprime;thatis,n=uv.ThenbytheNon-invertibilityLemma,uisnotinvertible.Butbytheconditionofthetheorem,un-1=un-2u=1+qun.Inotherwords,uhasaninverseun-2,whichisacontradiction.Sonmustbeprime.
5.5Euler’sTheoremLikeanygreatmathematician,EulerwasnotsatisfiedwithjustprovingFermat’sLittleTheorem;hewantedtoseeifitcouldbegeneralized.SinceFermat’sLittleTheoremwasonlyforprimes,Eulerwonderedwhethertherewasasimilarresultthatwouldincludecompositenumbers.Butcompositenumbersdostrangethingsinmodulararithmetic.Toillustratethis,let’stakealookatthemultiplicationtablemodulo10,whichwe’veannotatedbyshowinginversesoftheleft-handfactorontheright-handsideofthetable:
Thetableshouldlookabitfamiliar,becauseit’sjustlikethetraditional10×10multiplicationtable,ifyoukeeponlythelastdigitofeachproduct.Forexample,7×9=63,whichis3mod10.Immediatelywecanseedifferencesfromthetablewedidfor7,whichwasprime(seep.74).Foronething,therowsarenolongerpermutationsofeachother.Moreimportantly,somerowsnowcontain0.That’saproblemformultiplication—howcantheproductoftwothingsbe0?Thatwouldmeanthatwegetintoasituationwherewecanneverescapezero—anyproductoftheresultwillbezero.Theotherpropertywenotedearlieraboutprimes—thatonly1and-1areself-canceling—happensto
betruefor10aswell,butisnotalwaystrueforcompositenumbers.(Theinteger8,forexample,hasfourself-cancelingelements:1,3,5,and7.)Let’slookatthemultiplicationtablefor10again,focusingoncertainentries:
Therowsthatcontainonly“good”products(i.e.,nozeros)aretheoneswhosefirstfactorisshownina
rectangularboxontheleft—whichalsohappentobetherowswherethatfactorhasaninverse,shownontherightsideofthetable.Whichrowshavethisproperty?Thosethatrepresentnumbersthatarecoprimewith10.(Remember,beingcoprimemeanshavingnocommonfactorsgreaterthan1.)Socouldwejustusethegoodrowsandleaveouttherest?Notquite,becausesomeoftheresultsin
goodrowswouldthemselvesleadtobadrowsifusedinasuccessiveproduct.(Forexample,3isagoodrow,but(3×5)×2=0.)Euler’sideawastouseonlytheentriesingoodcolumnsaswellasgoodrows—thenumbersinshadedcells.Noticethatthosenumbershaveallthenicepropertieswesawforprimes:theshadednumbersineachrowarepermutationsofeachother,eachsetofshadednumberscontainsa1,andsoon.
***
ToextendFermat’sLittleTheoremforcompositenumbers,Eulerusesonlytheseboldvalues.Hestartsbydefiningthesizeofthesetofcoprimes:
Definition5.2.Thetotientofapositiveintegernisthenumberofpositiveintegerslessthannthatarecoprimewithn.Itisgivenbytheformula:
φ(n)=|{0<i<n∧coprime(i,n)}|
ThisisknownastheEulertotientfunctionorEulerφfunction.φ(n)givesusthenumberofrowscontainingshadedentriesinthemultiplicationtablemodulon.For
example,φ(10)=4,andφ(7)=6,aswecanseefromthemultiplicationtablesgivenearlier.Sinceprimesbydefinitiondon’tshareanyprimefactorswithsmallernumbers,thetotientofaprime
numberis
φ(p)=p–1
Inotherwords,allnumberslessthanagivenprimearecoprimewithit.WhatEulerrealizedwasthatthep-1inFermat’stheoremisjustaspecialcase;it’swhatφhappensto
beforprimes.NowwecanstateEuler’sgeneralizationofFermat’sLittleTheorem.
Theorem5.7(Euler’sTheorem):coprime(a,n) aφ(n)-1isdivisiblebyn.
Exercise5.2.ProveEuler’sTheorembymodifyingtheproofofFermat’sLittleTheorem.Steps:
•ReplacePermutationofRemaindersLemmawithPermutationofCoprimeRemaindersLemma.(Essentially,usethesameproofbutlookonlyat“good”elements.)•Provethateverycoprimeremainderhasamultiplicativeinverse.(Wejustshowedthattheremaindersformapermutation,so1hastobesomewhereinthepermutation.)•UsetheproductofallcoprimeremainderswheretheproofofLittleFermathastheproductofallnonzeroremainders.
***
Wewouldliketobeabletocomputetheφfunctionforanyinteger.Sincewecanexpressanyintegerastheproductofpowersofprimes,we’llstartbyseeinghowtocomputethetotientofapowerofaprimep.
Wewanttoknowthenumberofcoprimesofpm.Weknowthereareatmostpm-1ofthem,becausethat’sallthepossiblenumberslessthanpm.Butwealsoknowthatthosedivisiblebyp(i.e.,multiplesofp)arenotcoprime,soweneedtosubtracthowevermanyofthesetherearefromourtotal:
Whathappensifwehaveφ(puqv),wherepandqarebothprimes?Again,westartwiththemaximumpossibleandthensubtractoffallthemultiples.Sowe’llsubtractthenumberofmultiplesofpandalsothenumberofmultiplesofq,butthenwehavetoaddbackmultiplesofbothpandq,becauseotherwisethey’dbesubtractedtwice.(Thisgeneraltechnique,knownastheinclusion-exclusionprinciple,isoftenusedincombinatorics.)Letusassumen=puqv:
Asaspecialcasewhenwehaveasimpleproductoftwoprimes,p1andp2,wenowknowthat
Forexample,since10=5×2,
φ(10)=φ(5)φ(2)=4
Althoughthecasewecaremostaboutistheonegivenhere,wecangeneralizetheformulatohandleaproductofanynumberofprimesraisedtopowers,notjusttwo.Forexample,ifwehadthreefactorsp,q,andr,we’dsubtractallthemultiplesofeach,thenaddbackthedouble-countedmultiplesofpq,pr,andqr,andthencompensateforourovercompensationbyagainsubtractingmultiplesofpqr.Extendingthistomprimesgivesthisformula,where :
Euler’sinterestinprovinghistheoremledtohisneedtocountcoprimes.Hisderivationoftheφfunctiongavehimatoolthatallowedhimtoefficientlycomputethiscountinthecaseswheretheprimedecompositionisknown.
5.6ApplyingModularArithmeticInSection5.3,wesawhowmodularmultiplicationwasrelatedtoremainders.Let’stakealookatacoupleofourimportantresultsfromearlierinthechapterandseewhatsomeexampleslooklikeifwedothemmodulo7.Wilson’sTheoremstatesthatforaprimep,thereexistssomemsuchthat
(p-1)!=(p-1)+mp
Anotherwaytosaythisis
(p-1)!=(p-1)modp
Let’sseeifwecanconfirmthatresultifpis7.p-1is6,sowestartbyexpanding6!intoitsfactors,rearrangingthem,andusingourmodularmultiplicationtabletocancelinverses:
whichiswhatWilson’sTheorempredicts.Similarly,let’susemodularmultiplicationtoseewhatFermat’sLittleTheoremsays.Theoriginalform
is
Ifpisprime,ap-1–1isdivisiblebypforany0<a<p.
Butwithmodulararithmetic,wecouldrestateitas
Ifpisprime,ap–1–1=0modpforany0<a<p.
or
Ifpisprime,ap–1=1modpforany0<a<p.
Again,let’susep=7,andtrya=2.Thistimewe’llexpandourexpression,multiplybothsidesby6!,andthenusemodularmultiplicationtocancelterms:
whichiswhatFermat’sLittleTheoremtellsus.
5.7ThoughtsontheChapterEarlier,wesawhowtheancientGreekswereinterestedinperfectnumbers.Therewasn’tanypracticalvaluetothiswork;theyweresimplyinterestedinexploringpropertiesofcertainkindsofnumbersfortheirownsake.Yetaswehaveseeninthischapter,overtimethesearchforthese“useless”perfectnumbersledtothediscoveryofFermat’sLittleTheorem,oneofthemostpracticallyusefultheoremsinallofmathematics.We’llseewhyit’ssousefulinChapter13.Thischapteralsogaveusafirstlookattheprocessofabstractioninmathematics.Eulerlookedat
Fermat’sLittleTheoremandrealizedthathecouldextenditfromonespecificsituation(primes)toamoregeneralone(integers).HesawthattheexponentinFermat’stheoremwasaspecialcaseofamoregeneralconcept,thenumberofcoprimes.Thatsameprocessofabstractionliesattheheartofgenericprogramming.Generalizingcodeislikegeneralizingtheoremsandtheirproofs.JustasEulersawhowtoextendFermat’sresultfromonetypeofmathematicalobjecttoanother,soprogrammerscantakeafunctionthatwasdesignedforonetypeofcomputationalobject(say,vectors)andextendittoworkequallywellonanother(perhapslinkedlists).
6.AbstractioninMathematics
Mathematiciansdonotstudyobjects,buttherelationsbetweenobjects;tothemitisamatterofindifferenceiftheseobjectsarereplacedbyothers,
providedthattherelationsdonotchange.Matterdoesnotengagetheirattention,theyareinterestedinformalone.
Poincaré,ScienceandHypothesisThehistoryofmathematicsisfilledwithdiscoveriesofnewabstractions:findingwaystosolveamoregeneralproblem.Forexample,wesawinChapter5howEulergeneralizedFermat’sLittleTheoremsoitwouldworkwithcompositenumbersaswellasprimes.Eventually,mathematiciansrealizedthattheycouldgeneralizebeyondnumbers,andderiveresultsaboutabstractentitiescalledalgebraicstructures—collectionsofobjectsthatfollowcertainrules.Thisledtothedevelopmentofanentirelynewbranchofmathematics,abstractalgebra.Inthischapter,we’llintroducethefirstexamplesoftheseabstractentities,andprovesomeoftheirproperties.Aswedidinthepreviouschapter,we’regoingtoputprogrammingasidewhilewebuildthefoundationsweneedtoderiveagenericalgorithminChapter7.
6.1GroupsThefirstandmostimportantofthesealgebraicstructures,discoveredbyFrenchmathematicianÉvaristeGaloisin1832,iscalledagroup.
Definition6.1.Agroupisasetonwhichthefollowingaredefined:
operations:x y,x–1
constant:eandonwhichthefollowingaxiomshold:
Theconstanteistheidentityelement(alsosometimeswrittenid),whichisoftenwrittenas1inmultiplicativecontexts.Theoperationx–1istheinverseoperation;applyingtheoperationtoanitemanditsinverseresultsintheidentityelement,asthelastaxiomshows.Thegroupoperationisbinary,whichsimplymeansthatittakestwoarguments(ithasnothingtodowiththebinaryrepresentationofnumbersusedincomputers).Thesymbol (sometimeswritten*)canrepresentanybinaryoperation,aslongasitfollowstheaxioms.Weoftentreatthegroupoperationasmultiplication,andevenreferto“multiplying”twoelementsofa
group,althoughwhatwereallymeanisapplyingthegroupoperation,whateveritmightbe.Justaswithmultiplication,thesymbolfortheoperationisoftendropped;thatis,x ymaybewrittenxy,andx x=xx=x2.Thegroupoperationisnotnecessarilycommutative(commutativitymeansthat∀x,y:x y=y x).
Whenwewanttorequirecommutativity,weneedtospecifyaparticularkindofgroup:
Definition6.2.Anabeliangroupisagroupwhoseoperationiscommutative.
Onekindofabeliangroupistheadditivegroup:
Definition6.3.Anadditivegroupisanabeliangroupwherethegroupoperationisaddition.
Additivegroupsaretheexceptiontothenamingconventionsgivenearlier.Foranadditivegroup,thesymbol+isusedtorepresentitsoperationand0itsidentityelement.Eventhoughthename“additivegroup”saysnothingaboutcommutativity,itisbyconventionassumedtobecommutative.Groupsareclosedundertheiroperation.Thismeansthatifyoutakeanytwoelementsofthegroupand
applythegroupoperation,theresultwillitselfbeamemberofthegroup.Similarly,theyareclosedundertheinversefunction:ifyoutaketheinverseofanyelementofthegroup,theresultisstillanelementofthegroup.Someexamplesofgroupsfollow:•Additivegroupofintegers:theelementsareintegersandtheoperationisaddition.•Multiplicativegroupofnonzeroremaindersmodulo7:theelementsarethenumbers1through6andtheoperationismultiplicationmodulo7.•Groupofrearrangementsofadeckofcards:theelementsarepermutationsofthedeckandtheoperationiscompositionofthesepermutations.•Multiplicativegroupofinvertiblematrices(thosewithnonzerodeterminants)withrealcoefficients:theelementsarematricesandtheoperationismatrixmultiplication.•Groupofrotationsoftheplane:theelementsaredifferentrotationsaroundtheoriginandtheoperationiscompositionoftheserotations.
Notethatintegersdonotformamultiplicativegroup,becausethemultiplicativeinverseofmostintegersisnotaninteger.Inotherwords,integersarenotclosedundermultiplicativeinverse.
Exercise6.1.Howmanyintegershavemultiplicativeinversesthatareintegers?Whatarethey?
Let’slookatoneoftheexamplesinabitmoredetail.InChapter5,welookedatthemultiplicationtableforintegersmodulo7:
Theuniquevaluesinthetable—theset{1,2,3,4,5,6}—arealsocalled“nonzeroremaindersmodulo7,”andaswenotedearlier,theseformamultiplicativegroup.Whatdoesthatmean?Sinceit’samultiplicativegroup,thegroupoperationismultiplication,anditsidentityelementis1.Wecanseefromthefirstrowandcolumnofthetablethat1istheidentity,becausetheproductofanyelementxand1isx.Sincegroupsareclosedundertheiroperation,ifwemultiplyanytwomembersofthegroup,weget
anothermemberofthegroup.Forexample:
Associativityandcommutativityofmodularmultiplicationfollowsfromassociativityandcommutativityofintegermultiplication.Thecommutativityorabeliannessofthegroupisevidentbyobservingthatthemultiplicationtableissymmetricwithrespecttothemaindiagonal.Sincegroupsareclosedunderinverse,ifwetaketheinverseofanymemberofthegroup,weget
anothermemberofthegroup.(Recallthattheinverseofanelementxistheelementthatproduces1whenmultipliedbyx.Fromthemultiplicationtable,youcanseethepairsofinversesbylookingattherowsandcolumnswhosecellscontain1s.)Forexample:
2–1=4mod7
4–1=2mod7
5–1=3mod7
ÉvaristeGalois(1811–1832)
TheconceptofgroupsstartedwiththeworkofÉvaristeGalois,ayoungFrenchcollegedropoutinvolvedinarevolutionarymovement,andthemostromanticfigureinthehistoryofmathematics.Intheearly19thcentury,aromanticspiritspreadthroughEurope;youngpeopleidolizedtheEnglishpoetByron,whodiedfightingforGreekindependence,andotherswhowerewillingtogivetheirlivesforacause.TheyrememberedNapoleonnotasatyrant,butasayoungherowhoabolishedfeudalismthroughoutEurope.Parisintheearly1830swasaflamewithrevolutionaryactivity.Galois,whowasabohemianhothead,joinedtherevolutionarymovement.Asaromanticrebel,Galoisdidnotfollowtheconventionalpaththroughauniversityeducation.Afterfailingtobeadmittedtooneschoolandbeingexpelledfromanother,hestudiedmathematicsonhisown,becominganexpertonLagrange’stheoryofpolynomials.Heservedbriefprisonsentencesforvariousprotestactivities,suchasmarchingthroughthestreetsintheuniformofabannednationalguardunitwhilecarryingseveralloadedweapons—butkeptdoingmathematicswhileinprison.Atage20,Galois,defendingthehonorofawomanwhomheapparentlybarelyknew,issuedachallenge(orwaschallenged)toaduel.Thenightbeforetheduel,certainofhisimpendingdeath,hewrotealonglettertoafrienddescribinghismathematicalideas.Thismanuscriptcontainedtheseedsofthetheoryofgroups,fields,andtheirautomorphisms(mappingsontothemselves).Theseideaslaidthefoundationsforamajornewfieldofmathematics,abstractalgebra.AccordingtomathematicianHermannWeyl,“Thisletter,ifjudgedbythenoveltyandprofundityofideasitcontains,isperhapsthemostsubstantialpieceofwritinginthewholeliteratureofmankind.”Thenextday,Galoisfoughttheduelanddiedasaresultofhiswounds.Itisironicthatwhileheonlyplayedatbeingarevolutionaryinpolitics,hewasatruerevolutionaryinmathematics.
6.2MonoidsandSemigroupsInsomesituations,weareinterestedinalgebraicstructuresthathavefewerrequirementsthangroups.(We’llseesomeoftheseapplicationsinthenextchapter.)Forexample,therearetimeswhenwedon’tneedtorequireaninverseoperation,butwewanttomaintaintheotherpropertiesofagroup.Thisiscalledamonoid.Moreformally:
Definition6.4.Amonoidisasetonwhichthefollowingaredefined:
operation:x y
constant:e
andonwhichthefollowingaxiomshold:
Thisdefinitionisliterallythesameastheoneforgroups,exceptwe’veleftouttheinverseoperationandtheaxiomofcancellationthatusesit.Aswithgroups,wecandefineparticularkindsofmonoidsbyspecifyingtheoperation,suchasanadditivemonoid(wheretheoperationisaddition)andamultiplicativemonoid(wheretheoperationismultiplication).Someexamplesofmonoidsfollow:•Monoidoffinitestrings(freemonoid):theelementsarestrings,theoperationisconcatenation,andtheidentityelementistheemptystring.•Multiplicativemonoidofintegers:theelementsareintegers,theoperationismultiplication,andtheidentityelementis1.
Wecanrelaxtherequirementsevenfurtherbydroppingtheidentityelement.Thisiscalledasemigroup:
Definition6.5.Asemigroupisasetonwhichthefollowingisdefined:
operation:x y
andonwhichthefollowingaxiomholds:
Again,allwe’vedoneistakenthepreviousdefinitionandremovedsomething—inthiscase,therequirementthattherebeanidentityelement,andtheaxiomthatusesit.Andaswedidwithgroupsandmonoids,wecandefineadditivesemigroupsandmultiplicativesemigroupsassemigroupsthatusethoseoperations.Someexamplesofsemigroupsfollow:•Additivesemigroupofpositiveintegers:theelementsarepositiveintegersandtheoperationisaddition.•Multiplicativesemigroupofevenintegers:theelementsareevenintegersandtheoperationismultiplication.
Aswementionedearlier,mathematicianswriterepeatedapplicationsofthesemigroup,monoid,orgroupoperationwiththesameconventionsasordinarymultiplication.Forexample:
x x x=xxx=x3
Moreformally,wedefineraisingasemigrouptoapowerasfollows:
Exercise6.2.Whycan’twedefinepowerforsemigroupsstartingwithn=0?
Equation6.1showsthesemigroupoperationhappeningontheleftsideofthepower(i.e.,itsaysxxn–1,notxn–1x).Whatifwewantedtowritetheexpansiontheotherway?Wecandothat,too,aswewillnowprove:
Lemma6.1:Forn≥2,xxn–1=xn–1x.
Proof.Weprovethisbyinduction.1
Basis:n=2.It’sobviouslytrue,because
xx1=xx=x1x
1Forarefresheronthisprooftechnique,seeAppendixB.2.
Astheinductionhypothesis,weassumethestatementistrueforn=k–1:
xx(k –1)–1=xxk–2=xk–2x=x(k–1)–1x
andthenderivetheresultforn=k:
Eventhoughasemigroupguaranteesonlyassociativityandnotcommutativityofitsoperation,itturnsoutthatpowersofagivensemigroupelementalwayscommute,whichwecanalsoprovebygeneralizingthisresult.Thisisperhapsthemostimportanttheoremonsemigroups.
Theorem6.1(CommutativityofPowers):xnxm=xmxn=xn+m
Proof.Proofbyinductiononm.
Basis:m=1:
Inductivestep:Assumetrueform=k.Showform=k+1:
Wehaveshownthatxnxm=xn+m.Therefore,itisalsotruethatxmxn=xm+n.Sinceintegeradditioniscommutative,xn+m=xm+n,soxnxm=xmxn.
Asemigroupistheweakestinterestingalgebraicstructure.Theonlyrequirementlefttorelaxistheassociativityaxiom.Analgebraicstructurecalledmagmadropseventhataxiom,butitturnsoutnottobeveryuseful.Sincetherearenoaxiomsleft,notheoremscanbeproved.
6.3SomeTheoremsaboutGroupsNowlet’sreturntogroupsandlookatsomeoftheirproperties.Animportantobservationisthatallgroupsaretransformationgroups.Inotherwords,everyelementa
ofthegroupGdefinesatransformationofGontoitself:
x→ax
Forexample,withtheadditivegroupofintegers,ifwechoosea=5,thenthisactsasa“+5”operation,transformingthesetofelementsxtothesetx+5.Thesetransformationsareone-to-onebecauseofourinvertibilityaxiom:
a–1(ax)→x
Inourexample,wecanundoour+5transformationbyapplyingtheinverse,–5.
Theorem6.2:Agrouptransformationisaone-to-onecorrespondence.2
2Aone-to-onecorrespondencebetweentwosetsisamappingbetweenthemthatisbothone-to-oneandonto.
ThisisequivalenttosayingthatforanyfinitesetSofelementsofgroupGandanyelementaofG,asetofelementsaShasthesamenumberofelementsasS.
Proof.IfS={s1,...,sn},thenaS={as1,...,asn}.WeknowthatthesetaScan’tcontainmoreuniqueelementsthanS,butcoulditcontainfewer?(ThatwouldbethecaseiftwooftheelementsinSweremappedtothesameelementinaS.)SupposetwoelementsofaSwerethesame:asi=asj.Then
Soiftworesultsofthetransformationaskareequal,thentheirinputsskmustbeequal.Equivalently(bythecontrapositiveofthepreviousstatement),iftheinputsarenotequal,thentheresultsofthetransformationmustnotbeequal.Sincewestartedwithndistinctarguments,wewillhavendistinctresults.Inotherwords,thesetaShasthesamenumberofelementsasS.
Hereareafewmoresimpleresultsaboutgroups:
Theorem6.3:Thereisauniqueinverseforeveryelement.
ab=e b=a–1
Proof.Supposeab=e.Thenwecanmultiplybothsidesbya–1ontheleft,likethis:
Theorem6.4:Theinverseofaproductisthereversedproductoftheinverses.
(ab)–1=b–1a–1
Proof.Thetwoexpressionsareequalifandonlyifmultiplyingonebytheinverseoftheotheryieldstheidentityelement.We’llusetheinverseof(ab)–1,whichbydefinitionis(ab),andmultiplyitbyb–1a–1:
Theorem6.5:Thepowerofaninverseistheinverseofthepower.
(x–1)n=(xn)–1
Proofbyinduction.Basis:n=1
(x−1)1=x−1=(x1)−1
Inductivestep:Assumetrueforn=k−1;thatis,(x−1)k−1=(xk−1)−1.Thenproveforn=k.Wewanttoshowthat(x–1)k=(xk)–1;thatis,theinverseofxkis(x–1)k.Ifthat’strue,thenwhenwe
multiplythemtogether,weshouldgettheidentityelement.Usingthedefinitionofpowerandthecommutativityofpowerstheorem,we’llrewritexkasxk –1xand(x–1)kasx–1(x–1)k–1,regroupthetermstogetsometocancel,andthensubstituteusingourinductiveassumption:
Therefore(xn)–1=(x–1)n.
Exercise6.3(veryeasy).Provethatanygrouphasatleastoneelement.
Definition6.6.Ifagrouphasn>0elements,niscalledthegroup’sorder.Ifagrouphasinfinitelymanyelements,itsorderisinfinite.
Thereisalsothenotionoftheorderofanelementinagroup:
Definition6.7.Anelementahasanordern>0ifan=eandforany0<k<n,ak≠e.(Inotherwords,theorderofaisthesmallestpowerofathatproducese.)Ifsuchndoesnotexist,ahasaninfiniteorder.
Exercise6.4(veryeasy).Whatistheorderofe?Provethateistheonlyelementofsuchorder.
***
Wenowcometoanimportanttheoremaboutgroups:
Theorem6.6:Everyelementofafinitegrouphasfiniteorder.
Proof.Ifnisanorderofthegroup,thenforanyelementa,{a,a2,a3,...,an+1}hasatleastonerepetitionaiandaj.Letusassumethat1≤i<j≤n+1,aiisthefirstrepeatedelementandajisitsfirstrepetition.Then
andj–i>0istheorderofa.Thisproofusesaversionofthepigeonholeprinciple.(Tolearnmoreaboutthepigeonholeprinciple
andhowtouseit,seeAppendixB.3.)Theresultguaranteesthatthissimplealgorithmforcomputingtheorderofanelementwillterminate:justkeepmultiplyingbyitselfuntilyougete.
Exercise6.5.Provethatifaisanelementofordern,thena–1=an–1.
6.4SubgroupsandCyclicGroupsDefinition6.8.AsubsetHofagroupGiscalledasubgroupofGif
Inotherwords,tobeasubgroup,Hmustbeasubsetandagroup.AssociativityoftheoperationonGimpliesitsassociativityonH,sowedonotneedtoexplicitlystatetheassociativityrequirementinthedefinition;bythesamereasoning,wedonotneedtoexplicitlyrestatetheidentityandcancellationaxioms.
Forexample,theadditivegroupofevennumbersisasubgroupoftheadditivegroupofintegers;soistheadditivegroupofnumbersdivisibleby5.Somegroupshavemanysubgroups,butalmostallgroupshaveatleasttwo:thegroupitselfandthe
groupconsistingofjusttheelemente.Thesetwosubgroupsarecalledtrivialsubgroups.(Theonlygroupthatdoesn’thaveatleasttwosubgroupsisthegroupthatconsistsofjusttheidentityelement.)Let’sreturnagaintothemultiplicativegroup{1,2,3,4,5,6}ofnonzeroremaindersmodulo7andits
multiplicationtable:
Ourgrouphasfourmultiplicativesubgroups:
{1},{1,6},{1,2,4},{1,2,3,4,5,6}
Howcanwetell?Tobeasubgroup,eachonefirstneedstobeasubsetoftheoriginalgroup.Eachoftheseisobviouslyasubsetof{1,2,3,4,5,6},sincealloftheirelementsarecontainedinthelargerset.Next,eachsubsetalsocontains1(theidentityelement)andisclosedunderitsoperation(multiplicationmodulo7)anditsinverseoperation.Forexample,considertheset{1,2,4}:ifwemultiplyanyelementofthesetbyitselforanyother
memberofthesetanynumberoftimes(mod7),wewillstillgetaresultintheset.
Exercise6.6.Findordersofeveryelementof:
•Themultiplicativegroupofremaindersmod7•Themultiplicativegroupofremaindersmod11
***
Thesimplestkindofgroupsarecyclicgroups.
Definition6.9.Afinitegroupiscalledcyclicifithasanelementasuchthatforanyelementb,thereisanintegernwhere
b=an
Inotherwords,everyelementinthegroupcanbegeneratedbyraisingoneparticularelementtodifferentpowers.Suchanelementiscalledageneratorofthegroup;agroupmayhavemultiplegenerators.Theadditivegroupofremaindersmodulonisanexampleofacyclicgroup.Inourpreviousexampleofremaindersmodulo7,wecantellthatthegeneratorsare3and5,because
theyarenotinanynontrivialsubgroupoftheoriginalgroup.
Exercise6.7.Provethatanysubgroupofacyclicgroupiscyclic.
Exercise6.8.Provethatacyclicgroupisabelian.
Lemma6.2:Powersofagivenelementinafinitegroupformasubgroup.
Inotherwords,everyelementofafinitegroupiscontainedinacyclicsubgroupgeneratedbythiselement.
Proof.Forasettobeasubgroup,itmustbeanonemptysubsetanditmustbeagroup.Tobeasubset,itmustbeclosedunderthegroupoperation.Weknowit’sclosedundertheoperationbecausetheproductoftwopowersisapower.Tobeagroup,itsoperationneedstobeassociative,itmustcontaintheidentityelement,anditmusthaveaninverseoperation.Weknowtheoperationisassociative,becauseit’sthesameoperationfromtheoriginalgroup.Weknowthesetofpowersofagivenelementhasanidentityelement,sinceweshowedearlier(Theorem6.6)thateveryelementofafinitegrouphasfiniteorder.3Andweknowthattheyhaveaninverse,becauseforeverypowerak,an–kisitsinverse,wherenistheorderofa.
3Recallthatpartofthedefinitionofanelementahavingafiniteorderisthatan=e.
6.5Lagrange’sTheoremOneoftheremarkablethingsaboutabstractalgebraisthatwecanproveresultsforstructuressuchasgroupswithoutknowinganythingabouteitherthespecificitemsinthegrouportheoperation.Toseeanimportantexampleofthis,wewillstartbyprovingafewsimpleresultsaboutcosets.
Definition6.10.IfGisagroupandH⊂GisasubgroupofG,thenforanya Gthe(left)cosetofabyHisaset
aH={g G|∃h H:g=ah}
Inotherwords,acosetaHisasetofallelementsinGobtainablebymultiplyingelementsofHbya.Asanexample,considertheadditivegroup4ofintegers anditssubgroup,integersdivisibleby4,4 .
(Theuseof asasymbolforintegerscomesfromtheGermanwordZahlen,whichmeans“numbers.”)Ithasfourdistinctcosets:4n,4n+1,4n+2,and4n+3.Addingotherintegerswillresultonlyinvaluesthatarealreadyinoneofthesecosets;forexample,thecoset4n+5willcontainthesameelementsasthecoset4n+1.(Theleftandrightcosetsarethesame,sinceintegeradditioniscommutative.)
4Remember,inanadditivegroup,theroleofgroup“multiplication”isplayedbyaddition.SothecosetaHconsistsofelementsofGobtainablebyaddingatoelementsofH.
Lemma6.3(SizeofCosets):InafinitegroupG,foranyofitssubgroupsH,thenumberofelementsinacosetaHisthesameasthenumberofelementsinthesubgroupH.
Proof.Wealreadyprovedtheone-to-onecorrespondenceoftransformationsaSwhenSisasubsetofG.Sinceasubgroupisbydefinitionalsoasubset,thenweknowthemappingfromHtoaHisalsoaone-to-onecorrespondence.Ifthereisaone-to-onecorrespondencebetweentwofinitesets,theyarethesamesize.
Lemma6.4(CompleteCoveragebyCosets):EveryelementaofagroupGbelongstosomecosetofsubgroupH.
Proof.a aH.Thatis,everyelementabelongstothecosetaHgeneratedbyitself,becauseH,beingasubgroup,containstheidentityelement.
Lemma6.5(Cosetsareeitherdisjointoridentical):IftwocosetsaHandbHinagroupGhaveacommonelementc,thenaH=bH.
Proof.Supposethecommonelementcisahainonecosetandbhbintheothercoset.
aha=bhb
Multiplyingbothsidesontherightby ,weget
ThetermontherightisbtimessomethingfromH(weknowit’sfromHbecausehbisfromHand isfromH,andsinceHisasubgroup,itisclosedundermultiplication).Nowlet’smultiplybothsidesontherightbyx,anarbitraryelementfromH:
WeknowbydefinitionthataxisinthecosetaH.WealsoknowthatthetermontherightisbtimessomethingfromH,soit’salsointhecosetbH.SincewecandothisforanyxinH,aH⊆bH.Wecanthenrepeattheprocessfromthebeginning,thistimeusing insteadof ,toshowthatbH⊆aH.SobH=aH.
Withtheseresults,wecanstateanimportanttheoremingrouptheory,whichillustratesthepowerofabstractreasoning.Ifyouwanttolearnonetheoremingrouptheory,thisistheone.Whileit’sverysimpletostate,itprovidesthefoundationforthetheoryoffinitegroups.
Theorem6.7(Lagrange’sTheorem):TheorderofasubgroupHofafinitegroupGdividestheorderofthegroup.
Proof.
1.ThegroupGiscoveredbycosetsofH.(Provedearlier,Lemma6.4)2.Differentcosetsaredisjoint.(Provedearlier,Lemma6.5)3.Theyareofthesamesizen,wherenistheorderofH.(Provedearlier,Lemma6.3.Specifically,theresultsaysthat|aH|=|H|foranya,soeverycosethassize|H|andthereforethesamesizeaseveryothercoset.)
ThereforetheorderofGisnm,wheremisthenumberofdistinctcosets,whichmeansthattheorderofGisamultipleoftheorderofH.Inotherwords,theorderofHdividestheorderofG.
Asanexample,supposeagroupGhastwodistinctcosetsofitssubgroupH.TheneveryelementofGmustbecoveredbyoneortheother(butnotboth)ofthosecosets,sotheorderofHmustbe|G|/2.Interestingly,theconverseofLagrange’sTheoremisnottrue:inagroupofordern,noteverydivisorof
nwillhaveasubgroupofcorrespondingorder.
Joseph-LouisLagrange(1736–1813)
TheleadingmathematicianinEuropeattheendofthe18thcenturywasJoseph-LouisLagrange,whowasasuccessorofLeonhardEulerbothintermsofintellectualleadershipandasholderofapositionasdirectorofmathematicsatthePrussianAcademyofSciencesinBerlin.LagrangewasbornGiuseppeLuigiLagranciainTurin,inthenorthernItalianregionofPiedmont.Ashisoriginalnamesuggests,hegrewupspeakingItalian,althoughhisfamilyclaimedFrenchancestry.LagrangediscoveredmathematicslargelyonhisownwhileastudentinTurin,andwithinafewyearsbecameaninstructorandbeganpublishinghiswork.Sometimearoundage20,hebegancorrespondingwithEuler,whowasinBerlinatthetime.ImpressedbyLagrange’swork,Euleractedasamentortotheyoungermathematician,encouraginghimandpromotinghisdiscoveries.EulerbeganlobbyingtobringLagrangetoBerlin,butbythetimethisplancametofruitionin1766,EulerhadmovedbacktoRussia.WithEulergone,Lagrangewasappointedtohismentor’sformerposition,andsoonestablishedhimselfasthesecondmostprominentmathematicianinEurope.Lagrangespentthenext20yearsinBerlin,anddidhismostimportantworkduringthisperiod,contributingtoseveralareasofmathematicsandphysics.HisbookAnalyticalMechanics,perhapsoneofthe10mostimportantbooksinthehistoryofmathematics,describedatop-downapproachtosolvingmechanicalproblemsthatwasmoregeneralthanusingNewtonianmechanics.MuchofmodernphysicsreliesonLagrange’swork.Healsodidextensiveworkonpolynomialequations,realizingthatcoefficientscouldbeexpressedasfunctionsofapolynomial’sroots,andlaidthegroundworkforalaterbreakthroughbyGalois.Andinnumbertheory,Lagrangefiguredoutwhencontinuedfractionswillbeperiodic,complementingEuler’searlierworkonthesubject.Afterthedeathofhispatronandfriend,KingFriedrichIIofPrussia,LagrangewasapproachedbytheFrenchambassador(actingonordersfromKingLouisXVI)toconvincethegreat
mathematiciantomovetoFrance.Lagrangeagreed,andwouldliveinParisfrom1786untilhisdeath.Despitehisprominence,Lagrangewasashy,unpretentiousmanwithfewfriendsandlittlesociallife.Hewaspronetoboutsofdepression,andmadelittleprogressonhisworkforyearsatatime,particularlyduringhisfirstyearsinFrance.However,theFrenchRevolutionrevivedhisinterest—althoughheoftenfearedhewouldneedtofleethecountryduetotherevolutionaries’expulsionofmostforeigners.Hebecameinvolvedineffortstocomeupwithanewsystemofweightsandmeasures,andwasoneofacommitteeoffiveprominentscientistswhovotedtoapprovewhatwenowcallthemetricsystem.Lagrangealsobeganteachingagain,althoughbymanyaccountshewasnotpopularwithstudents,whohadtroubleunderstandingbothhisideasandhisItalianaccent.Inhislateryears,LagrangewasfavoredbythenewemperorNapoleonBonaparte—himselfquiteanaccomplishedmathematician.NapoleonrecognizedLagrange’sgenius,andmadehimaCountoftheEmpire.
***
Nowwe’llproveacoupleofcorollariestoLagrange’sTheorem:Corollary6.7.1:Theorderofanyelementinafinitegroupdividestheorderofthegroup.
Proof.ThepowersofanelementofGformasubgroupofG.Sincetheorderofanelementistheorderofthesubgroup,andsincetheorderofthesubgroupmustdividetheorderofthegroup,thentheorderoftheelementmustdividetheorderofthegroup.
(Reminder:Theorderofanelementisequaltotheorderofthecyclicgroupofitspowers.)Corollary6.7.2:GivenagroupGofordern,ifaisanelementofG,thenan=e.
Proof.Ifahasanorderm,thenmdividesn(bythepreviouscorollary),son=qm.am=e(bydefinitionoforderofanelement).Therefore(am)q=eandan=e.
Notethatthisdoesn’tsaythattheorderofaisn;itcouldbesmaller.Lagrange’sTheoremletsuseasilyprovesomeimportantresultsfromChapter5inmuchsimpler
fashionthanwedidthefirsttime:Fermat’sLittleTheorem:Ifpisprime,ap–1–1isdivisiblebypforany0<a<p.
Proof.Letustakethemultiplicativegroupofremaindersmodulop.Itcontainsp–1nonzeroremainders.Sincep–1istheorderofthegroup,itfollowsimmediatelyfromCorollary6.7.2that
ap–1=e
Sincetheidentityelementforamultiplicativegroupis1(specifically,1modpinourgroupofremainders),wehave
ap–1=1modp
ap–1–1=0modp
whichiswhatitmeanstobedivisiblebyp.
Euler’sTheorem:For0<a<n,whereaandnarecoprime,aφ(n)–1isdivisiblebyn.
Proof.Letustakethemultiplicativegroupofinvertibleremaindersmodulon.Sinceφ(n)isbydefinitionthenumberofcoprimes,andeverycoprimeisinvertible,φ(n)givesustheorderofthegroup.ItfollowsimmediatelyfromCorollary6.7.2that
aφ(n)=e
aφ(n)=1modn
orequivalently
aφ(n)–1=0modn
Thelogicisexactlythesameasthepreviousproof.
Exercise6.9(veryeasy).Whataresubgroupsofagroupoforder101?
Exercise6.10.Provethateverygroupofprimeorderiscyclic.
6.6TheoriesandModelsGroups,monoids,andsemigroupsareexamplesofwhatmathematicianscalltheories.Peopleusetheword“theory”inmanyways,oftenmeaningsomethinglike“conjecture”—thatis,anunprovenexplanation.Buttomathematicians,“theory”hasaveryspecificmeaning,whichdoesnotincludethissenseofbeingunproven.
Definition6.11.Atheoryisasetoftruepropositions.
Fromnowon,whenweusetheword“theory,”we’llmeanthisspecificmathematicalsense.Herearesomeimportantfactsabouttheories:
•Atheorycanbegeneratedbyasetofaxiomsplusasetofinferencerules.•Atheoryisfinitelyaxiomatizableifitcanbegeneratedfromafinitesetofaxioms.•Asetofaxiomsisindependentifremovingonewilldecreasethesetoftruepropositions.•Atheoryiscompleteifforanyproposition,eitherthatpropositionoritsnegationisinthetheory.•Atheoryisconsistentiffornopropositionitcontainsboththatpropositionanditsnegation.
Let’slookataspecificexample:thenotionofgroupsthatwe’vebeendiscussingthroughoutthechapter.Agroupisatheoryinthesensewe’vejustdefined—specifically,atheorythat,givenoperationsx yandx–1andtheidentityelemente,hastheseaxioms:
x (y z)=(x y) z
x e=e x=x
x x–1=x–1 x=e
Startingwiththese,wecanderiveanynumberoftruepropositions(theorems),suchas
x y=x y=e
(x y)–1=y–1 x–1
Wearen’tenumeratingallthetruepropositionsthatconstitutethetheoryofgroups.Rather,we’regeneratingthepropositionsbyderivingthemfromtheaxiomsandfrompreviouslyprovenpropositions.Forexample,wecanprovethefirsttheorembymultiplyingbothsidesoftheequationbyx–1.Likebasisvectorsinlinearalgebra,axiomsformabasisforthetheory.Alsolikethelinearalgebracase,wecanhavemorethanonebasisforthesametheory.
***
Closelyassociatedwiththenotionofatheoryisthatofamodel.Again,themathematicalmeaningisquitedifferentfromtheeverydaymeaning:
Definition6.12.Asetofelementswherealltheoperationsinthetheoryaredefinedandallthepropositionsinthetheoryaretrueiscalleditsmodel.
Inasense,amodelisaparticularimplementationofatheory.Amodelgivesyouthespecificsetofelements;atheorydoesnot.Justastherecanbemultipleimplementationsof,say,analgorithm,sotherecanbemultiplemodelsofatheory.Forexample,theadditivegroupofintegersandthemultiplicativegroupofnonzeroremaindersmodulo7arebothmodelsofthetheoryofabeliangroups.Themore5propositionsthereareinatheory,thefewerdifferentmodelsthereare.Ifwegenerate
propositionsfromaxiomsandinferencerules,thenfeweraxiomsmeansfewerpropositions,andthereforemoremodels.Thismakessenseintuitively:axiomsandpropositionsareconstraintsonatheory;themoreofthemyouhave,theharderitistosatisfyallofthem,sothefewermodelstherewillbethatdoso.
5Thenotionof“more”thatwe’reinterestedinhereisthatofadditional.IfthesetofpropositionsfortheoryAcontainsallthepropositionsoftheoryB,plussomeadditionalones,thenwesayAhasmore,evenifbothhaveacountablyinfinitenumberofpropositions.
Conversely,themoremodelsthereareforatheory,thefewerpropositionsthereare.Iftherearemorewaystodosomething,theremustbefewerconstraintsonhowyoucandoit.
Definition6.13.Twomodelsareisomorphicifthereisaone-to-onecorrespondencebetweenthemthatpreservestheiroperations.Thismeanswecanapplythemapping(oritsinverse)beforeoraftertheoperationandwe’llgetthesameresult.
Forexample,wecanmapnaturalnumberstoevennaturalnumberswiththemapping“multiplyby2,”anduseadditionasouroperation.Ifweaddtwonaturalnumbersandthenapplythemapping(i.e.,multiplyby2),wegetthesameresultasifwefirstmultiplyby2andthenaddthenumbers:
Anisomorphismofamodelwithitselfiscalledanautomorphism.
Definition6.14.A(consistent)theoryiscalledcategoricalorunivalentifallofitsmodelsareisomorphic.6
6ThisistheoriginaldefinitionbyOswaldVeblen.Modernlogicianstalkaboutκ-categoricaltheories:allmodelsofthecardinalityκareisomorphic.Fulltreatmentofmodernmodeltheoryiswelloutsidethescopeofthisbook.
Aninconsistenttheoryisonethathasnomodels—there’snowaytosatisfyallthepropositionswithoutacontradiction.
CategoricalTheoriesversusSTLForalongtime,peoplebelievedthatonlycategoricaltheoriesweregoodforprogramming.WhentheC++StandardTemplateLibrary(STL)wasfirstproposed,somecomputerscientistsopposeditonthegroundsthatmanyofitsfundamentalconcepts,suchasIterator,wereunderspecified.Infact,itisthisunderspecificationthatgivesthelibraryitsgenerality.Whilelinkedlistsandarraysarenotcomputationallyisomorphic,manySTLalgorithmsaredefinedoninputiteratorsandworkonbothdatastructures.Ifyoucangetbywithfeweraxioms,youallowforawiderrangeofimplementations.
6.7ExamplesofCategoricalandNon-categoricalTheoriesLet’slookatanexampleofacategoricaltheorywithtwoisomorphicmodels:cyclicgroupsoforder4.Thefirstmodelis ,theadditivegroupofremaindersmodulo4(consistingof{0,1,2,3});thesecondmodelis( ),themultiplicativegroupofnonzeroremaindersmodulo5(consistingof{1,2,3,4}).Thesegroupshavethefollowing“multiplication”tables:
Eventhoughthenumbersaredifferent,thesetwomodelsareisomorphic;wecouldmapelementsofonetoelementsoftheother.Intheory,thereare4!=12possiblemappings—0inthefirstmodelcouldmapto1,2,3,or4inthesecond,then1couldmaptothethreeremainingchoices,andsoon.Butinthiscase,the
actualnumberofpossibilitiesismuchsmaller.Weobservethatinthefirstmodel,thevalues1and3aregeneratorsofthegroup—wecouldstartwith
eitherofthem,raiseittoapowerusingthegroupoperation,andgetalltheotherelements.Inthesecondmodel,2and3aregenerators.Thishelpsusnarrowthechoices:ageneratorfromonegrouphastomaptoageneratorfromtheothergroup,whichinthiscasegivesustwodifferentmappings.Forexample,wecansay,“theroleof1inthefirstmodelisplayedby2inthesecondmodel,and3inthefirstmodelcorrespondsto3inthesecond.”Whatabouttheothertwovalues?Weknowfromthemultiplicationtablesthat0inthefirstmodelisthe
identityelement,aroleplayedby1inthesecondmodel.Finally,weknowthat2inthefirstmodelmapsto4inthesecondmodel,becauseinbothcasesitistheonlynon-identityelementthatgivesidentitywhenappliedtoitself.Sowehavetwopossiblemappings:
Howdoweknowthesemappingsproduceoursecondmodel?Onewayistoseeifwecanusethemtotransformthe multiplicationtableintothe( )multiplicationtable.Let’stryitusingthesecondmapping.First,wereplacethevaluesfromthe tablewiththemappedvalues.Thatgivesus:
Thenwepermutetherowsandcolumnssotheheadersareintherightorder.We’llstartbyswappingthelasttwocolumnsandthelasttworows(shadedintheprecedingtable):
Finally,we’llswapthemiddletworowsandcolumns:
Thisisexactlythemultiplicationtablefor( )weshowedonp.105,whichiswhatwewanted.
***
Nowlet’slookatanexampleofanon-categoricaltheory:allgroupsoforder4.Whilethereisonlyonenon-isomorphicgroupoforder1,2,or3,therearetwonon-isomorphicgroupsoforder4:thecyclicgroup ,whichwejustdescribed,andagroupcalledtheKleingroup.TherearetwoimportantmodelsoftheKleingroup:themultiplicativegroupofunitsmodulo8(consistingof{1,3,5,7})andthegroupofisometriestransformingarectangleintoitself(theidentitytransform,verticalsymmetry,horizontalsymmetry,and180°rotation).Thesetwokindsofgroupshavethefollowingmultiplicationtables.Sincewedon’tknowthe
individualelementsofatheory,thistimewe’llwritethemusingetomeantheidentityelementanda,b,andctomeantheirotherelements:
Inthetableontheleft,additionisouroperationandwecanthinkof“e”asbeing0(theadditiveidentity)anda,b,andcasrepresentingtheintegers1,2,and3.So,forexample,a b=1+2=3=c;thereforethevalueatrowaandcolumnbisc.Arethesetwogroupsreallydifferent(i.e.,notisomorphic),oristhereawaytotransformoneintothe
other,aswedidbefore?Inotherwords,isthereadistinguishingpropositionforgroupsoforder4?Yes,theproposition
∀x G:x2=e
istruefortheKleingroupbutfalsefor .Wecanseethisbylookingatthediagonalofthemultiplicationtable.Anotherwaytoseethattheyaredifferentistonoticethatthecyclicgroupcontainstwogenerators,whiletheKleingroupcontainsnone.
6.8ThoughtsontheChapterInthischapterweintroducedtheideaofalgebraicstructures,abstractsetsofelementsthatobeycertainproperties.Welookedatgroups,themostimportantofthesestructures,andtheirweakercousinsmonoidsandsemigroups.Thesearesummarizedinthefollowingtable,whichwe’llbeaddingtolater:
Eachrowofthetableincludesallthepropertiesofthepreviousrow,withoneormoreadditions.Wecanviewtherelationshipsbetweenthestructureslikethis:
Forexample,thediagramshowsthatamonoidisasemigroupthatalsohasanidentityelement(andtheidentityaxiom).Afewadditionalstructureswetalkedaboutaremosteasilydefinedintermsofothers:
Wealsosawhowwecouldderivepropertiesofgroups(liketheonestatedbyLagrange’sTheorem)withoutknowinganythingabouttheparticularelementsbeingmanipulated.Inotherwords,wesawhowtoderiveresultsabouttheorieswithoutspecifyingaparticularmodel.Nowwe’rereadytoputthesealgebraicstructurestopracticaluse.
7.DerivingaGenericAlgorithm
Togeneralizesomethingmeanstothinkit.Hegel,PhilosophyofRight
Inthischapterwe’lltaketheEgyptianmultiplicationalgorithmfromChapter2and,byusingthemathematicalabstractionsintroducedinthepreviouschapter,generalizeittoapplytoawidevarietyofproblemsbeyondsimplearithmetic.
7.1UntanglingAlgorithmRequirementsTwostepsarerequiredtowriteagoodpieceofcode.Thefirststepistogetthealgorithmright.Thesecondstepistofigureoutwhichsortsofthings(types)itworksfor.Now,youmightbethinkingthatyoualreadyknowthetype—it’sintorfloatorwhateveryoustartedwith.Butthatmaynotalwaysbethecase;thingschange.Nextyearsomeonemaywantthecodetoworkwithunsignedintsordoublesorsomethingelseentirely.Wewanttodesignourcodesowecanreuseitinthesedifferentsituations.Let’stakeanotherlookatthemultiply-accumulateversionoftheEgyptianmultiplicationalgorithmwe
developedinChapter2.Recallthatwearemultiplyingnbyaandaccumulatingtheresultinr;alsorecallthatwehaveapreconditionthatneithernnoracanbezero:Clickheretoviewcodeimage
intmult_acc4(intr,intn,inta){
while(true){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
n=half(n);
a=a+a;
}
}
Thistimewe’vewrittensomeofthecodeinaslantedtypefaceandsomeofitinaboldtypeface.Noticethattheslantedandboldpartsaredisjointed;therearenoplaceswherea“slantedvariable”anda“boldvariable”arecombinedorinteractwitheachotherinanyway.Thismeansthattherequirementsforbeingaslantedvariabledon’thavetobethesameastherequirementsforbeingaboldvariable—ortoputitinprogramminglanguageterms,theycanbedifferenttypes.Sowhataretherequirementsforeachkindofvariable?Untilnow,we’vedeclaredthevariablesas
ints,butitseemsasifthealgorithmwouldworkformanyothersimilartypesaswell.Slantedvariablesrandamustbesometypethatsupportsadding—wemightsaythattheymustbea“plusable”type.Theboldvariablenmustbeabletobecheckedforoddness,comparedwith1,andmustsupportdivisionby2(itmustbe“halvable”).Notethatdivisionby2isamuchmorerestrictedoperationthandivisioningeneral.Forexample,anglescanbedividedby2withruler-and-compassconstruction,whiledividingthemby3isimpossibleinthatframework.We’veestablishedthatrandaarethesametype,whichwe’llwriteusingthetemplatetypenameA.
Similarly,wesaidthatnisadifferenttype,whichwe’llcallN.Soinsteadofinsistingthatalltheargumentsbeoftypeint,wecannowwritethefollowingmoregenericformoftheprogram:Clickheretoviewcodeimage
template<typenameA,typenameN>
Amultiply_accumulate(Ar,Nn,Aa){
while(true){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
n=half(n);
a=a+a;
}
}
Thismakestheproblemeasier—wecanfigureouttherequirementsforAseparatelyfromtherequirementsonN.Let’sdigabitdeeperintothesetypes,startingwiththesimplerone.
7.2RequirementsonAWhatarethesyntacticrequirementsonA?Inotherwords,whichoperationscanwedoonthingsbelongingtoA?Justbylookingathowvariablesofthistypeareusedinthecode,wecanseethattherearethreeoperations:
•Theycanbeadded(inC++,theymusthaveoperator+).•Theycanbepassedbyvalue(inC++,theymusthaveacopyconstructor).•Theycanbeassigned(inC++,theymusthaveoperator=).
Wealsoneedtospecifythesemanticrequirements.Thatis,weneedtosaywhattheseoperationsmean.Ourmainrequirementisthat+mustbeassociative,whichweexpressasfollows:
A(T) ∀a,b,c T:a+(b+c)=(a+b)+c
(InEnglish,wemightreadthepartbeforethecolonlikethis:“IftypeTisanA,thenforanyvaluesa,b,andcinT,thefollowingholds:....”)Eventhough+isassociativeintheory(andinmathgenerally),thingsarenotsosimpleoncomputers.
Therearereal-worldcaseswhereassociativityofadditiondoesnothold.Forexample,considerthesetwolinesofcode:
w=(x+y)+z;
w=x+(y+z)
Supposex,y,andzareoftypeint,andzisnegative.Thenitispossiblethatforsomeverylargevalues,x+ywilloverflow,whilethiswouldnothavehappenedifweaddedy+zfirst.Theproblemarisesbecauseadditionisnotwelldefinedforallpossiblevaluesofthetypeint;wesaythat+isapartialfunction.Toaddressthisproblem,weclarifyourrequirements.Werequirethattheaxiomsholdonlyinsidethe
domainofdefinition—thatis,thesetofvaluesforwhichthefunctionisdefined.11Foramorerigoroustreatmentofthistopic,seeSection2.1ofElementsofProgrammingbyStepanovandMcJones.
***
Infact,thereareacouplemoresyntacticrequirementsthatwemissed.Theyareimpliedbycopyconstructionandassignment.Forexample,copyconstructionmeanstomakeacopythatisequaltotheoriginal.Todothis,weneedtheabilitytotestthingsbelongingtoAforequality:
•Theycanbecomparedforequality(inC++,theymusthaveoperator==).
•Theycanbecomparedforinequality(inC++,theymusthaveoperator!=).Accompanyingthesesyntacticrequirementsaresemanticrequirementsforwhatwecallequational
reasoning;equalityonatypeTshouldsatisfyourexpectations:•Inequalityisthenegationofequality.
(a≠b) ¬(a=b)
•Equalityisreflexive,symmetric,andtransitive.
a=a
a=b b=a
a=b∧b=c a=c
•Equalityimpliessubstitutability.
foranyfunctionfonT,a=b f(a)=f(b)
Thethreeaxiomsinthemiddle(reflexivity,symmetry,andtransitivity)providewhatwecallequivalence,butequationalreasoningrequiressomethingmuchstronger,soweaddthesubstitutabilityrequirement.Wehaveaspecialnamefortypesthatbehaveinthe“usualway”—regulartypes:
Definition7.1.AregulartypeTisatypewheretherelationshipsbetweenconstruction,assignment,andequalityarethesameasforbuilt-intypessuchasint.
Forexample:•Ta(b);assert(a==b);unchanged(b);•a=b;assert(a==b);unchanged(b);•Ta(b);isequivalenttoTa;a=b;
Foranextensivetreatmentofregulartypes,seeChapter1ofElementsofProgramming.Alltypesthatweuseinthisbookareregular.
***
NowwecanformalizetherequirementsonA:•Regulartype•Providesassociative+
AswesawinChapter6,algebraicstructuresthathaveabinaryassociativeoperationarecalledsemigroups(seeDefinition6.5).Also,aregulartypeguaranteestheabilitytocomparetwovaluesforequality,whichweneedforourassociativityaxiom.ThereforewecansaythatAisasemigroup.Itsoperationisaddition,sowemightbetemptedtocallitanadditivesemigroup.Butrecallthatbyconvention,additivesemigroupsareassumedtobecommutative.Sincewedon’tneedcommutativityforouralgorithm,we’llsaythatAisanoncommutativeadditivesemigroup.Thismeansthatcommutativityisnotrequired;itdoesnotmeanthatcommutativityisnotallowed.Inotherwords,every(commutative)additivesemigroupisalsoanoncommutativeadditivesemigroup.
Definition7.2.Anoncommutativeadditivesemigroupisasemigroupwheretheassociativebinaryoperationis+.
Someexamplesofnoncommutativeadditivesemigroupsarepositiveevenintegers,negativeintegers,realnumbers,polynomials,planarvectors,Booleanfunctions,andlinesegments.Theseexampleshappentoalsobeadditivesemigroups,butthatisnotalwaysthecase.Asweshallsee,+mayhavedifferentinterpretationsforthesedifferenttypes,butitwillalwaysbeassociative.Formanycenturies,thesymbol“+”hasbeenused,byconvention,tomeanacommutativeoperationas
wellasanassociativeone.Manyprogramminglanguages(e.g.,C++,Java,Python)use+forstringconcatenation,anoncommutativeoperation.Thisviolatesstandardmathematicalpractice,whichisabadidea.Themathematicalconventionisasfollows:
•Ifasethasonebinaryoperationanditisbothassociativeandcommutative,callit+.•Ifasethasonebinaryoperationanditisassociativeandnotcommutative,callit*.
20th-centurylogicianStephenKleeneintroducedthenotationabtodenotestringconcatenation(sinceinmathematics*isusuallyelided).
TheNamingPrincipleIfwearecomingupwithanameforsomething,oroverloadinganexistingname,weshouldfollowthesethreeguidelines:1.Ifthereisanestablishedterm,useit.2.Donotuseanestablishedterminconsistentlywithitsacceptedmeaning.Inparticular,overloadanoperatororfunctionnameonlywhenyouwillbepreservingitsexistingsemantics.
3.Ifthereareconflictingusages,themuchmoreestablishedonewins.ThenamevectorinSTLwastakenfromtheearlierprogramminglanguagesSchemeand
CommonLisp.Unfortunately,thiswasinconsistentwiththemucholdermeaningoftheterminmathematicsandviolatesRule3;thisdatastructureshouldhavebeencalledarray.Sadly,ifyoumakeamistakeandviolatetheseprinciples,theresultmightstayaroundforalongtime.
7.3RequirementsonNNowthatweknowthatAmustbeanoncommutativeadditivesemigroup,wecanspecifythatinourtemplateinsteadofjustsayingtypename:Clickheretoviewcodeimage
template<NoncommutativeAdditiveSemigroupA,typenameN>
Amultiply_accumulate(Ar,Nn,Aa){
while(true){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
n=half(n);
a=a+a;
}
}
Herewe’reusingNonCommutativeAdditiveSemigroupasaC++concept,asetofrequirementsontypesthatwe’lldiscussinChapter10.Insteadofsayingtypename,wenametheconceptwewishto
use.Sinceconceptsarenotyetsupportedinthelanguageasofthiswriting,we’redoingabitofpreprocessorslight-of-hand:Clickheretoviewcodeimage
#defineNonCommutativeAdditiveSemigrouptypename
Asfarasthecompilerisconcerned,Aisjustatypename,butforus,it’saNonCommutativeAdditiveSemigroup.We’llusethistrickfromnowonwhenwewanttospecifythetyperequirementsintemplates.Althoughthisbehaviorisnotneededforabstractmathematics,inprogrammingweneedourvariables
tobeconstructibleandassignable,whicharealsoguaranteedbybeingregulartypes.Fromnowon,whenwespecifyanalgebraicstructureasaconcept,wewillassumethatweareinheritingalloftheregulartyperequirements.Whatabouttherequirementsonourotherargument’stype,N?Let’sstartwiththesyntactic
requirements.Nmustbearegulartypeimplementing•half•odd•==0•==1
HerearethesemanticrequirementsonN:•even(n) half(n)+half(n)=n•odd(n) even(n–1)•odd(n) half(n–1)=half(n)•Axiom:n≤1∨half(n)=1∨half(half(n))=1∨...
WhichC++typessatisfytheserequirements?Thereareseveral:uint8_t,int8_t,uint64_t,andsoon.TheconceptthattheysatisfyiscalledNInteger.
***
Nowwecanfinallywriteafullygenericversionofourmultiply-accumulatefunctionbyspecifiyingthecorrectrequirementsforbothtypes:Clickheretoviewcodeimage
template<NoncommutativeAdditiveSemigroupA,IntegerN>
Amultiply_accumulate_semigroup(Ar,Nn,Aa){
//precondition(n>=0);
if(n==0)returnr;
while(true){
if(odd(n)){
r=r+a;
if(n==1)returnr;
}
n=half(n);
a=a+a;
}
}
We’veaddedonemorelineofcode,whichreturnsrwhenniszero.Wedothisbecausewhenniszero,wedon’tneedtodoanything.However,thesameisnottrueformultiply,aswe’llseeinamoment.
Here’sthemultiplyfunctionthatcallstheprecedingcode;ithasthesamerequirements:Clickheretoviewcodeimage
template<NoncommutativeAdditiveSemigroupA,IntegerN>
Amultiply_semigroup(Nn,Aa){
//precondition(n>0);
while(!odd(n)){
a=a+a;
n=half(n);
}
if(n==1)returna;
returnmultiply_accumulate_semigroup(a,half(n-1),a+a);
}
WecanalsoupdateourhelperfunctionsoddandhalftoworkforanyInteger:Clickheretoviewcodeimage
template<IntegerN>
boolodd(Nn){returnbool(n&0x1);}
template<IntegerN>
Nhalf(Nn){returnn>>1;}
7.4NewRequirementsOurpreconditionformultiplysaysthatnmustbestrictlygreaterthanzero.(Wemadethisassumptionearlier,sincetheGreekshadonlypositiveintegers,butnowweneedtomakeitexplicit.)Whatshouldanadditivesemigroupmultiplicationfunctionreturnwhenniszero?Itshouldbethevaluethatdoesn’tchangetheresultwhenthesemigroupoperator—addition—isapplied.Inotherwords,itshouldbetheadditiveidentity.Butanadditivesemigroupisnotrequiredtohaveanidentityelement,sowecan’tdependonthisproperty.Inotherwords,wecan’trelyontherebeinganequivalentofzero.(Remember,anolongerhastobeaninteger;itcanbeanyNoncommutativeAdditiveSemigroup,suchaspositiveintegersornonemptystrings.)That’swhyncan’tbezero.Butthereisanalternative:insteadofhavingarestrictiononthedatarequiringn>0,wecanrequire
thatanytypeweuseknowhowtodealwith0.Wedothisbychangingtheconceptrequirementonnfromadditivesemigrouptomonoid.RecallfromChapter6thatinadditiontoanassociativebinaryoperation,amonoidcontainsanidentityelementeandanidentityaxiomthatsays
xοe=eοx=x
Inparticular,we’regoingtouseanoncommutativeadditivemonoid,wheretheidentityelementiscalled“0”:
x+0=0+x=x
Thisisthemultiplyfunctionformonoids:Clickheretoviewcodeimage
template<NoncommutativeAdditiveMonoidA,IntegerN>
Amultiply_monoid(Nn,Aa){
//precondition(n>=0);
if(n==0)returnA(0);
returnmultiply_semigroup(n,a);
}
Whatifwewanttoallownegativenumberswhenwemultiply?Weneedtoensurethat“multiplyingbyanegative”makessenseforanytypewemighthave.Thisturnsouttobeequivalenttosayingthatthetypemustsupportaninverseoperation.Again,wefindthatourcurrentrequirement—noncommutativeadditivemonoid—isnotguaranteedtohavethisproperty.Forthis,weneedagroup.Agroup,asyoumayrecallfromChapter6,includesallofthemonoidoperationsandaxioms,plusaninverseoperationx–1thatobeysthecancellationaxiom
xοx–1=x–1οx=e
Inourcase,wewantanoncommutativeadditivegroup,onewheretheinverseoperationisunaryminusandthecancellationaxiomis:
x+–x=–x+x=0
Havingstrengthenedourtyperequirements,wecanremoveourpreconditionsonntoallownegativevalues.Again,we’llwrapthelastversionofourfunctionwithournewone:Clickheretoviewcodeimage
template<NoncommutativeAdditiveGroupA,IntegerN>
Amultiply_group(Nn,Aa){
if(n<0){
n=-n;
a=-a;
}
returnmultiply_monoid(n,a);
}
7.5TurningMultiplyintoPowerNowthatourcodehasbeengeneralizedtoworkforanyadditivesemigroup(ormonoidorgroup),wecanmakearemarkableobservation:
Ifwereplace+with*(therebyreplacingdoublingwithsquaring),wecanuseourexistingalgorithmtocomputeaninsteadofn·a.
Here’stheC++functionwegetwhenweapplythistransformationtoourcodeformultiply_accumulate_semigroup:Clickheretoviewcodeimage
template<MultiplicativeSemigroupA,IntegerN>
Apower_accumulate_semigroup(Ar,Aa,Nn){
//precondition(n>=0);
if(n==0)returnr;
while(true){
if(odd(n)){
r=r*a;
if(n==1)returnr;
}
n=half(n);
a=a*a;
}
}
Thenewfunctioncomputesran.Theonlythingsthathavechangedarehighlightedinbold.Notethatwe’vechangedtheorderofargumentsaandn,soastomatchtheorderofargumentsinstandard
mathematicalnotation(i.e.,wesayna,butan).Here’sthefunctionthatcomputespower:
Clickheretoviewcodeimage
template<MultiplicativeSemigroupA,IntegerN>
Apower_semigroup(Aa,Nn){
//precondition(n>0);
while(!odd(n)){
a=a*a;
n=half(n);
}
if(n==1)returna;
returnpower_accumulate_semigroup(a,a*a,half(n-1));
}
Herearethewrappedversionsformultiplicativemonoidsandgroups:Clickheretoviewcodeimage
template<MultiplicativeMonoidA,IntegerN>
Apower_monoid(Aa,Nn){
//precondition(n>=0);
if(n==0)returnA(1);
returnpower_semigroup(a,n);
}
template<MultiplicativeGroupA,IntegerN>
Apower_group(Aa,Nn){
if(n<0){
n=-n;
a=multiplicative_inverse(a);
}
returnpower_monoid(a,n);
}
Justasweneededanadditiveidentity(0)forourmonoidmultiply,soweneedamultiplicativeidentity(1)inourmonoidpowerfunction.Also,justasweneededanadditiveinverse(unaryminus)forourgroupmultiply,soweneedamultiplicativeinverseforourgrouppowerfunction.There’snobuilt-inmultiplicativeinverse(reciprocal)operationinC++,butit’seasytowriteone:Clickheretoviewcodeimage
template<MultiplicativeGroupA>
Amultiplicative_inverse(Aa){
returnA(1)/a;
}
7.6GeneralizingtheOperationWe’veseenexamplesoftwosemigroups—additiveandmultiplicative—eachwithitsassociatedoperation(+and*,respectively).Thefactthatwecouldusethesamealgorithmforbothiswonderful,butitwasannoyingtohavetowritedifferentversionsofthesamecodeforeachcase.Inreality,therecouldbemanysuchsemigroups,eachwithitsassociativeoperations(forexample,multiplicationmod7)thatworkonthesametypeT.Ratherthanhavinganotherversionforeveryoperationwewanttouse,wecangeneralizetheoperationitself,justaswegeneralizedthetypesoftheargumentsbefore.Infact,therearemanysituationswhereweneedtopassanoperationtoanalgorithm;youmayhaveseenexamplesofthisinSTL.Here’stheaccumulateversionofourpowerfunctionforanarbitrarysemigroup.Westillrefertothe
functionitcomputesas“power,”eventhoughtheoperationit’srepeatedlyapplyingmaynotnecessarilybemultiplication.Clickheretoviewcodeimage
template<RegularA,IntegerN,SemigroupOperationOp>
//requires(Domain<Op,A>)
Apower_accumulate_semigroup(Ar,Aa,Nn,Opop){
//precondition(n>=0);
if(n==0)returnr;
while(true){
if(odd(n)){
r=op(r,a);
if(n==1)returnr;
}
n=half(n);
a=op(a,a);
}
}
Noticethatwe’veaddeda“requires”commenttoourtemplatethatsaysthedomainofoperationOpmustbeA.IffutureversionsofC++supportconcepts,thiscommentcouldbeturnedintoastatement(similartoanassertion)thatthecompilercouldusetoensurethecorrectrelationshipholdsbetweenthegiventypes.Asitis,we’llhavetomakesureasprogrammersthatwecallthisfunctiononlywithtemplateargumentsthatsatisfytherequirement.Also,sincewenolongerknowwhichkindofsemigrouptomakeA—itcouldbeadditive,
multiplicative,orsomethingelsealtogether,dependingonOp—wecanrequireonlythatAbearegulartype.The“semigroupness”willcomefromtherequirementthatOpbeaSemigroupOperation.Wecanusethisfunctiontowriteaversionofpowerforanarbitrarysemigroup:
Clickheretoviewcodeimage
template<RegularA,IntegerN,SemigroupOperationOp>
//requires(Domain<Op,A>)
Apower_semigroup(Aa,Nn,Opop){
//precondition(n>0);
while(!odd(n)){
a=op(a,a);
n=half(n);
}
if(n==1)returna;
returnpower_accumulate_semigroup(a,op(a,a),
half(n-1),op);
}
Aswedidbefore,wecanextendthefunctiontomonoidsbyaddinganidentityelement.Butsincewedon’tknowinadvancewhichoperationwillbepassed,wehavetoobtaintheidentityfromtheoperation:Clickheretoviewcodeimage
template<RegularA,IntegerN,MonoidOperationOp>
//requires(Domain<Op,A>)
Apower_monoid(Aa,Nn,Opop){
//precondition(n>=0);
if(n==0)returnidentity_element(op);
returnpower_semigroup(a,n,op);
}
Hereareexamplesofidentity_elementfunctionsfor+and*:Clickheretoviewcodeimage
template<NoncommutativeAdditiveMonoidT>
Tidentity_element(std::plus<T>){returnT(0);}
template<MultiplicativeMonoidT>
Tidentity_element(std::multiplies<T>){returnT(1);}
Eachofthesefunctionsspecifiesthetypeoftheobjectitexpectstobecalledwith,butdoesn’tnameit,sincetheobjectisneverused.Thefirstonesays,“Theadditiveidentityis0.”Ofcourse,therewillbedifferentidentityelementsfordifferentmonoids—forexample,themaximumvalueofthetypeTformin.
***
Toextendpowertogroups,weneedaninverseoperation,whichisitselfafunctionofthespecifiedGroupOperation:Clickheretoviewcodeimage
template<RegularA,IntegerN,GroupOperationOp>
//requires(Domain<Op,A>)
Apower_group(Aa,Nn,Opop){
if(n<0){
n=-n;
a=inverse_operation(op)(a);
}
returnpower_monoid(a,n,op);
}
Examplesofinverse_operationlooklikethis:Clickheretoviewcodeimage
template<AdditiveGroupT>
std::negate<T>inverse_operation(std::plus<T>){
returnstd::negate<T>();
}
template<MultiplicativeGroupT>
reciprocal<T>inverse_operation(std::multiplies<T>){
returnreciprocal<T>();
}
STLalreadyhasanegatefunction,but(duetoanoversight)hasnoreciprocal.Sowe’llwriteourown.We’lluseafunctionobject—aC++objectthatprovidesafunctiondeclaredbyoperator()andisinvokedlikeafunctioncall,usingthenameoftheobjectasthefunctionname.Tolearnmoreaboutfunctionobjects,seeAppendixC.Clickheretoviewcodeimage
template<MultiplicativeGroupT>
structreciprocal{
Toperator()(constT&x)const{
returnT(1)/x;
}
};
Thisisjustageneralizationofthemultiplicative_inversefunctionwewroteintheprevioussection.2
2Thistimewedonotneedapreconditionpreventingxfrombeingzero,becauseMultiplicativeGroupdoesnotcontainanon-invertiblezeroelement.IfinpracticewehaveatypesuchasdoublethatotherwisewouldsatisfytherequirementsofMultiplicativeGroupifitdidnotcontainazero,wecanaddapreconditiontoeliminatethatcase.
ReductionThepoweralgorithmisnottheonlyimportantalgorithmdefinedonsemigroups.Anotherkeyalgorithmisreduction,inwhichabinaryoperationisappliedsuccessivelytoeachelementofasequenceanditspreviousresult.Twocommonlyseenexamplesofthisinmathematicsarethesummation(∑)functionfor
additivesemigroupsandtheproduct(∏)functionformultiplicativesemigroups.Wecangeneralizethistoanarbitrarysemigroup.Thisgeneralizedversionofreductionwasinventedin1962bycomputerscientistKenIversonin
hislanguageAPL.InAPLterminology,the/representedthereductionoperator.Forexample,summationofasequenceisexpressedas+/123
Theideaofreductionhasappearedinmanycontextssincethen.JohnBackus,inventorofthefirsthigh-levelprogramminglanguage,includedasimilaroperatorcalledinsertinhislanguageFPin1977.(Hecalledoperators“functionalforms.”)Anearlypaperongenericprogramming,“OperatorsandAlgebraicStructures,”byKapur,Musser,andStepanov,extendedtheideatoparallelreductionin1981andclarifiedtherelationshiptoassociativeoperations.ThelanguageCommonLisp,popularinthe1980sforartificialintelligenceapplications,includedareducefunction.Google’sMapReducesystem,anditsopen-sourcevariantHadoop,isacurrentpracticalapplicationoftheseideas.
7.7ComputingFibonacciNumbersNote:Thissectionassumessomebasicknowledgeoflinearalgebra.Therestofthebookdoesnotdependonthematerialcoveredhere,andthissectionmaybeskippedwithoutaffectingthereader’sunderstanding.InChapter4wemettheearly13th-centurymathematicianLeonardoPisano,oftenknowntodayasFibonacci.Oneofthethingshe’sbestknownforisafamousproblemheposed:ifwestartwithonepairofrabbits,howmanypairswillwehaveafteracertainnumberofmonths?Tosimplifytheproblem,Leonardomadesomeassumptions:theoriginalpairofrabbitsandeverylitterafterwardconsistsofamaleandafemale;femalerabbitstakeonemonthtoreachsexualmaturityandhaveonelitterpermonthafterthat;rabbitsliveforever.Initiallywehave1pairofrabbits.Atthestartofmonth2,therabbitsmate,butwestillhaveonly1
pair.Atthestartofmonth3,thefemalegivesbirth,sowehave2pairs.Atthestartofmonth4,theinitialfemalegivesbirthanothertime,sowehave3pairs.Atthestartofmonth5,theinitialfemalegivesbirthagain,butsodoesthefemaleborninmonth3,sonowwehave5pairs,andsoon.Ifwesaythatwehad0rabbitsinmonth0(beforetheexperimentbegan),thenthenumberofpairsineachmonthlookslikethis:
0,1,1,2,3,5,8,13,21,34...
Eachmonth’spopulationcanbeobtainedsimplybyaddingthepopulationsofeachoftheprevioustwomonths.Today,elementsofthissequencearecalledFibonaccinumbers,andsuchasequenceisdefinedformallylikethis:
F0=0
F1=1
Fi=Fi–1+Fi–2
HowlongdoesittaketocomputethenthFibonaccinumber?The“obvious”answerisn–2,buttheobviousansweriswrong.ThenaivewaytoimplementthisinC++issomethinglikethis:
Clickheretoviewcodeimage
intfib0(intn){
if(n==0)return0;
if(n==1)return1;
returnfib0(n-1)+fib0(n-2);
}
However,thiscodedoesanawfullotofrepeatedwork.Considerthecalculationstocomputefib0(5):
Eveninthissmallexample,thecomputationdoes17additions,andjustthequantityF1+F0isrecomputed5times.
Exercise7.1.Howmanyadditionsareneededtocomputefib0(n)?
Recomputingthesamethingoverandoverisunacceptable,andthere’snoexcuseforcodelikethat.Wecaneasilyfixthecodebykeepingarunningstateoftheprevioustworesults:Clickheretoviewcodeimage
intfibonacci_iterative(intn){
if(n==0)return0;
std::pair<int,int>v={0,1};
for(inti=1;i<n;++i){
v={v.second,v.first+v.second};
}
returnv.second;
}
Thisisanacceptablesolution,whichtakesO(n)operations.Infact,giventhatwewanttofindthenthelementofasequence,itmightappeartobeoptimal.ButtheamazingthingisthatwecanactuallycomputethenthFibonaccinumberinO(logn)operations,whichformostpracticalpurposesislessthan64.SupposewerepresentthecomputationofthenextFibonaccinumberfromtheprevioustwousingthe
followingmatrixequation:3
3AbriefrefresheronmatrixmultiplicationmaybefoundatthebeginningofSection8.5.
ThenthenthFibonaccinumbermaybeobtainedby
Inotherwords,wecancomputethenthFibonaccinumberbyraisingacertainmatrixtoapower.Aswewillsee,matrixmultiplicationcanbeusedtosolvemanyproblems.Matricesareamultiplicativemonoid,sowealreadyhaveanO(logn)algorithm—ourpoweralgorithmfromSection7.6.
Exercise7.2.ImplementcomputingFibonaccinumbersusingpower.
Thisisaniceapplicationofourpoweralgorithm,butcomputingFibonaccinumbersisn’ttheonlythingwecando.Ifwereplacethe+withanarbitrarylinearrecurrencefunction,wecanusethesametechniquetocomputeanylinearrecurrence.
Definition7.3.Alinearrecurrencefunctionoforderkisafunctionfsuchthat
Definition7.4.Alinearrecurrencesequenceisasequencegeneratedbyalinearrecurrencefrominitialkvalues.
TheFibonaccisequenceisalinearrecurrencesequenceoforder2.Foranylinearrecurrencesequence,wecancomputethenthstepbydoingmatrixmultiplicationusing
ourpoweralgorithm:
Thelineof1sjustbelowthediagonalprovidesthe“shifting”behavior,sothateachvalueinthesequencedependsonthepreviousk.
7.8ThoughtsontheChapterWebeganthischapterbyanalyzingtherequirementsonourcodefromChapter2,abstractingthealgorithmstouseanassociativeoperationonarbitrarytypes.Wewereabletorewritethecodesothatitisdefinedoncertainalgebraicstructures:semigroups,monoids,andgroups.Next,wedemonstratedthatthealgorithmcouldbegeneralized,firstfrommultiplicationtopower,then
toarbitraryoperationsonouralgebraicstructures.We’llusethisgeneralizedpoweralgorithmagainlateroninthebook.Theprocesswewentthrough—takinganefficientalgorithm,generalizingit(withoutlosingefficiency)
sothatitworksonabstractmathematicalconcepts,andthenapplyingittoavarietyofsituations—istheessenceofgenericprogramming.
8.MoreAlgebraicStructures
ForEmmyNoether,relationshipsamongnumbers,functions,andoperationsbecametransparent,amenabletogeneralization,andproductiveonlyaftertheyhavebeen
dissociatedfromanyparticularobjectsandhavebeenreducedtogeneralconceptualrelationships.
B.L.vanderWaerdenWhenwefirstintroducedEuclid’salgorithminChapter4,itwasforcomputingthegreatestcommonmeasureoflinesegments.Thenweshowedhowtoextendittoworkforintegers.Doesitworkforotherkindsofmathematicalentities?Thisisthequestionwe’llbeinvestigatinginthischapter.Aswe’llsee,attemptstoansweritledtoimportantdevelopmentsinabstractalgebra.We’llalsoshowhowsomeofthesenewalgebraicstructuresenablenewprogrammingapplications.
8.1Stevin,Polynomials,andGCDSomeofthemostimportantcontributionstomathematicswereduetooneofitsleast-knownfigures,the16th-centuryFlemishmathematicianSimonStevin.Inadditiontohiscontributionstoengineering,physics,andmusic,Stevinrevolutionizedthewaywethinkaboutandoperateonnumbers.AsBartelvanderWaerdenwroteinhisHistoryofAlgebra:
[With]onestroke,theclassicalrestrictionsof“numbers”tointegersortorationalfractionswaseliminated.[Stevin’s]generalnotionofarealnumberwasacceptedbyalllaterscientists.
Inhis1585pamphletDeThiende(“TheTenth”),publishedinEnglishasDisme:TheArtofTenths,or,DecimallArithmetike,Stevinintroducesandexplainstheuseofdecimalfractions.ThiswasthefirsttimeanyoneinEuropeproposedusingpositionalnotationintheotherdirection—fortenths,hundredths,andsoon.Disme(pronounced“dime”)wasoneofthemostwidelyreadbooksinthehistoryofmathematics.ItwasoneofThomasJefferson’sfavorites,andisthereasonwhyU.S.currencyhasacoincalleda“dime”andusesdecimalcoinageratherthantheBritishpounds,shillings,andpenceinuseatthetime.
SimonStevin(1548–1620)
SimonStevinwasborninBruges,Flanders(nowpartofBelgium),butlatermovedtoLeideninHolland.Priortothisperiod,theNetherlandsprovinces(whichincludedbothFlandersandHolland)werepartoftheSpanishempire,ledbyitsHapsburgkingsandheldtogetherbyitsinvincibleprofessionalarmy.In1568,theDutchbeganawarofindependence,unitedbyacommoncultureandlanguage,ultimatelycreatingarepublicandthenanempireoftheirown.Stevin,aDutchpatriotandmilitaryengineer,joinedintherebellionandbecamefriendswithPrinceMauriceofOrange,itsleader.InparthelpedbyStevin’sdesignsforfortificationsandhiscleveruseofasystemofsluicestofloodinvadingSpanishtroops,therebellionsucceeded,creatinganindependentDutchnationcalledtheUnitedProvinces.ThisbegantheDutch“GoldenAge,”whenthecountrybecameacultural,scientific,andcommercialpower,rememberedtodayinthegreatworksofartistslikeRembrandtandVermeer.StevinwasatrueRenaissanceman,withmanyfar-reachinginterestsbeyondmilitaryengineering.WhileStevin’sofficialjobformostofhiscareerwasquartermaster-generalofthearmy,inpracticehealsobecameakindofscienceadvisortoPrinceMaurice.Inadditiontohisinventionofdecimalfractions,polynomials,andothermathematicalwork,Stevinmademanycontributionstophysics.Hestudiedstatics,realizingthatforcescouldbeaddedusingwhatwenowcallthe“parallelogramofforces,”andpavedthewayfortheworkofNewtonandothers.Hediscoveredtherelationshipoffrequenciesinadjacentnotesofa12-tonemusicalscale.Stevinevendemonstratedconstantaccelerationoffallingobjects,afewyearsbeforeGalileo.StevinwasalsoanardentproponentoftheDutchlanguage,whichuntilthenhadbeenconsideredasortofsecond-ratedialectofGerman.HehelpedPrinceMauricecreateanengineeringschoolwhereteachingwasinDutch,andwrotetextbooksinthelanguage.Heusedwordfrequencyanalysisandwordlengthto“prove”thatitwasthebest(mostefficient)languageandthebesttodoscience.StevininsistedonpublishinghisownresultsinDutchratherthanLatin,whichmayexplainwhyhedidnotbecomebetterknownoutsidehiscountry.
InDisme,Stevinexpandsthenotionofnumbersfromintegersandfractionsto“thatwhichexpresseththequantitieofeachthing”(asanEnglishtranslationatthetimeputit).Essentially,Stevininventedtheentireconceptofrealnumbersandthenumberline.Anyquantitycouldgoonthenumberline,including
negativenumbers,irrationalnumbers,andwhathecalled“inexplicable”numbers(bywhichhemayhavemeanttranscendentalnumbers).Ofcourse,Stevin’sdecimalrepresentationshadtheirowndrawbacks,particularlytheneedtowriteaninfinitenumberofdigitstoexpressasimplevalue,suchas
Stevin’srepresentationenabledthesolutionofpreviouslyunsolvableproblems.Forexample,heshowedhowtocomputecuberoots,whichhadgiventheGreekssomuchtrouble.HisreasoningwassimilartowhateventuallybecameknownastheIntermediateValueTheorem(seethe“OriginsofBinarySearch”sidebarinSection10.8),whichsaysthatifacontinuousfunctionisnegativeatonepointandpositiveatanother,thentheremustbeanintermediatepointwhereitsvalueiszero.Stevin’sideawastofindthenumber(initially,aninteger)wherethefunctiongoesfromnegativetopositive,thendividetheintervalbetweenthatnumberandthenextintotenths,andrepeattheprocesswiththetenths,hundredths,andsoon.Herealizedthatby“zoomingin,”anysuchproblemcouldbesolvedtowhateverdegreeofaccuracywasneeded,orasheputit,“onemayobtainasmanydecimalsof[thetruevalue]asonemaywishandcomeindefinitelyneartoit.”AlthoughStevinsawhowtorepresentanynumberasapointalongaline,hedidnotmaketheleapto
showingpairsofnumbersaspointsonaplane.Thatinvention—whatwenowcallCartesiancoordinates—camefromthegreatFrenchphilosopherandmathematicianRenéDescartes(RenatusCartesiusinLatin).
***
Stevin’snextgreatachievementwastheinventionof(univariate1)polynomials,alsointroducedin1585,inabookcalledArithmétique.Considerthisexpression:
4x4+7x3–x2+27x–3
1Univariatepolynomialsarepolynomialswithasinglevariable.Fortherestofthischapter,wewillassume“polynomial”meansunivariatepolynomial.
PriortoStevin’swork,theonlywaytoconstructsuchanumberwasbyperforminganalgorithm:Takeanumber,raiseittothe4thpower,multiplyitby4,andsoon.In,fact,onewouldneedadifferentalgorithmforeverypolynomial.Stevinrealizedthatapolynomialissimplyafinitesequenceofnumbers:{4,7,–1,27,–3}fortheprecedingexample.Inmoderncomputerscienceterms,wemightsaythatStevinwasthefirsttorealizethatcodecouldbetreatedasdata.WithStevin’sinsight,wecanpasspolynomialsasdatatoagenericevaluationfunction.We’llwrite
onethattakesadvantageofHorner’srule,whichusesassociativitytoensurethatweneverhavetomultiplypowersofxhigherthan1:
4x4+7x3–x2+27x–3=(((4x+7)x–1)x+27)x–3
Forapolynomialofdegreen,weneednmultiplicationsandn–madditions,wheremisthenumberofcoefficientsequaltozero.Usuallywewillsettlefordoingnadditions,sincecheckingwhethereachadditionisneededismoreexpensivethanjustdoingtheaddition.Usingthisrule,wecanimplementapolynomialevaluationfunctionlikethis,wheretheargumentsfirstandlastspecifytheboundsofasequenceofcoefficientsofthepolynomial:Clickheretoviewcodeimage
template<InputIteratorI,SemiringR>
Rpolynomial_value(Ifirst,Ilast,Rx){
if(first==last)returnR(0);
Rsum(*first);
while(++first!=last){
sum*=x;
sum+=*first;
}
returnsum;
}
Let’sthinkabouttherequirementsonthetypessatisfyingIandR.Iisaniterator,becausewewanttoiterateoverthesequenceofcoefficients.2Butthevaluetypeoftheiterator(thetypeofthecoefficientsofthepolynomial)doesnothavetobeequaltothesemiring3R(thetypeofthevariablexinthepolynomial).Forexample,ifwehaveapolynomiallikeax2+bwherethecoefficientsarerealnumbers,thatdoesn’tmeanxhastobearealnumber;infact,itcouldbesomethingcompletelydifferent,likeamatrix.
2We’llexplainiteratorsmoreformallyinChapter10,butfornowwecanthinkofthemasgeneralizedpointers.3Asemiringisanalgebraicstructurewhoseelementscanbeaddedandmultipliedandhasdistributivity.Wewillgiveitsformaldefinition
inSection8.5.
Exercise8.1.WhataretherequirementsonRandthevaluetypeoftheiterator?Inotherwords,whataretherequirementsoncoefficientsofpolynomialsandontheirvalues?
Stevin’sbreakthroughallowedpolynomialstobetreatedasnumbersandtoparticipateinnormalarithmeticoperations.Toaddorsubtractpolynomials,wesimplyaddorsubtracttheircorrespondingcoefficients.Tomultiply,wemultiplyeverycombinationofelements.Thatis,ifaiandbiaretheithcoefficientsofthepolynomialsbeingmultiplied(startingfromthelowest-orderterm)andciistheithcoefficientoftheresult,then
Todividepolynomials,weneedthenotionofdegree.
Definition8.1.Thedegreeofapolynomialdeg(p)istheindexofthehighestnonzerocoefficient(orequivalently,thehighestpowerofthevariable).
Forexample:deg(5)=0deg(x+3)=1deg(x3+x–7)=3
Nowwecandefinedivisionwithremainder:
Definition8.2.Polynomialaisdivisiblebypolynomialbwithremainderriftherearepolynomialsqandrsuchthat
a=bq+r∧deg(r)<deg(b)
(Inthisequation,qrepresentsthequotientofa÷b.)Doingpolynomialdivisionwithremainderisjustlikedoinglongdivisionofnumbers:
Exercise8.2.Provethatforanytwopolynomialsp(x)andq(x):
1.p(x)=q(x)·(x–x0)+r p(x0)=r
2.p(x0)=0 p(x)=q(x)·(x–x0)
***
StevinrealizedthathecouldusethesameEuclideanalgorithm(theonewelookedatintheendofSection4.6)tocomputetheGCDoftwopolynomials;allwereallyneedtodoischangethetypes:Clickheretoviewcodeimage
polynomial<real>gcd(polynomial<real>a,polynomial<real>b){
while(b!=polynomial<real>(0)){
a=remainder(a,b);
std::swap(a,b);
}
returna;
}
Theremainderfunctionthatweuseimplementsthealgorithmforpolynomialdivision,althoughwedonotcareaboutthequotient.ThepolynomialGCDisusedextensivelyincomputeralgebrafortaskssuchassymbolicintegration.Stevin’srealizationistheessenceofgenericprogramming:analgorithminonedomaincanbe
appliedinanothersimilardomain.JustasinSection4.7,weneedtoshowthatthealgorithmworks—specifically,thatitterminatesand
computestheGCD.Toshowthatthealgorithmterminates,weneedtoshowthatitcomputestheGCDinafinitenumberof
steps.Sinceitrepeatedlyperformspolynomialremainder,weknowbyDefinition8.2that
deg(r)<deg(b)
Soateverystep,thedegreeofrisreduced.Sincedegreeisanon-negativeinteger,thedecreasingsequencemustbefinite.ToshowthealgorithmcomputestheGCD,wecanusethesameargumentfromSection4.7;itappliesto
polynomialsaswellasintegers.
Exercise8.3(fromChrystal,Algebra).FindtheGCDofthefollowingpolynomials:
8.2GöttingenandGermanMathematicsInthe18thand19thcenturies,startinglongbeforeGermanyexistedasaunifiedcountry,Germancultureflourished.ComposerslikeBach,Mozart,andBeethoven,poetslikeGoetheandSchiller,andphilosopherslikeKant,Hegel,andMarxwerecreatingtimelessworksofdepthandbeauty.GermanuniversitiescreatedauniqueroleforGermanprofessorsascivilservantsboundbytheircommitmenttothetruth.Eventuallythissystemwouldproducethegreatestmathematiciansandphysicistsoftheirage,manyofthemteachingorstudyingattheUniversityofGötttingen.
TheUniversityofGöttingenThecenterofGermanmathematicswasaseeminglyunlikelyplace:theUniversityofGöttingen.UnlikemanygreatEuropeanuniversitiesthathadstartedhundredsofyearsearlierinmedievaltimes,Göttingenwasrelativelyyoung,foundedin1734.AndthecityofGöttingenwasnotamajorpopulationcenter.Despitethis,theUniversityofGöttingenwashometoanastonishingseriesoftopmathematicians,includingGauss,Riemann,Dirichlet,Dedekind,Klein,Minkowski,andHilbert,someofwhomwewilldiscusslaterinthebook.Bytheearly20thcenturyitscommunityofphysicistswasequallyimpressive,includingquantumtheoristsMaxBornandWernerHeisenberg.Göttingen’sgreatnesswasdestroyedin1933bytheNazis,whoexpelledallJewsfromthefacultyandstudentbody—includingmanyofthetopphysicistsandmathematicians.Someyearslater,theNaziMinisterofEducationaskedthegreatGermanmathematicianDavidHilbert,“HowismathematicsinGöttingennowthatithasbeenfreedofJewishinfluence?”Hilbertreplied,“MathematicsinGöttingen?Thereisnoneanymore.”
PerhapsthemostimportantmathematiciantocomeoutofGöttingenwasCarlFriedrichGauss,whowasthefounderofGermanmathematicsinthemodernsense.Amonghismanyaccomplishmentswashisseminalworkonnumbertheory,describedinhis1801bookDisquisitionesArithmeticae(“InvestigationsofArithmetic”).Gauss’sbookistonumbertheorywhatEuclid’sElementsistogeometry—thefoundation
onwhichalllaterworkinthefieldisbased.Amongotherresults,itincludestheFundamentalTheoremofArithmetic,whichstatesthateveryintegerhasauniqueprimefactorization.
CarlFriedrichGauss(1777–1855)
CarlFriedrichGaussgrewupinBrunswick,Germany,andwasrecognizedasachildprodigyearlyinhislife.Accordingtoafamousstory,hiselementaryschoolteachertriedtokeeptheclassoccupiedbyaskingthemtoaddalltheintegersfrom1to100.Nine-year-oldGausscameupwiththeanswerinseconds;hehadobservedthatthefirstandlastnumbersaddedto101,asdidthesecondandsecondtolast,andsoon,sohesimplymultiplied101by50.Gauss’stalentscametotheattentionoftheDukeofBrunswick,whopaidfortheyoungstudent’seducationstartingatage14,firstataschoolinhishometownandlaterattheUniversityofGöttingen.Initially,Gaussconsideredacareerinclassics,which,unlikemathematics,wasoneofthestrengthsoftheuniversityatthetime.However,hecontinuedtodomathematicsonhisown,andin1796hemadeadiscoverythathadeludedmathematicianssinceEuclid:howtoconstructa17-sidedregularpolygonusingarulerandacompass.Infact,Gausswentfurther,provingthatconstructionofaregularp-gonwaspossibleforprimeponlyifpisaFermatprime—thatis,anumberoftheform22k+1.Thisbreakthroughconvincedhimtopursueacareerinmathematics.Infact,hewassoproudofthediscoverythatheplannedtohavethe17-gonengravedonhistomb.InhisPh.D.dissertation,GaussprovedwhatbecameknownastheFundamentalTheoremofAlgebra,whichstatesthateverynonconstantpolynomialwithcomplexcoefficientshasacomplexroot.Gausswrotehisgreatnumbertheorytreatise,DisquisitionesArithmeticae,whilehewasstillastudent,andhaditpublishedin1801whenhewasjust24.Whilegreatmathematiciansthroughouthistory,suchasEuclid,Fermat,andEuler,hadworkedonnumbertheory,Gausswasthefirsttocodifythefieldandplaceitonaformalfoundationbyintroducingmodulararithmetic.Disquisitionesisstillstudiedtoday,and,infact,someimportantdevelopmentsin20th-centurymathematicsweretheresultofcarefulstudyofGauss’swork.Gaussbecameworldfamousin1801whenhepredictedthelocationoftheasteroidCeresusinghismethodofleastsquares.Becauseofthisresult,hewaslaterappointeddirectoroftheastronomicalobservatoryatGöttingen.Thispatternofpracticalproblemsinspiringhismathematicalresults
continuedthroughouthiscareer.Hisworkongeodesy(thescienceofmeasuringtheEarth)ledtohisinventionofthenewfieldofdifferentialgeometry.HisobservationsoferrorsindataledtotheideaoftheGaussiandistributioninstatistics.ThroughoutGauss’scareer,hechosehisworkverycarefully,andpublishedonlywhatheconsideredtobehisbestresults—asmallfractionofhispotentialoutput.Oftenhewoulddelaypublication,sometimesbyyears,untilhefoundtheperfectwaytoproveaparticularresult.Hismottowas“Few,butripe.”Becauseofthebreadthanddepthofhiscontributions,GausswasknownasPrincepsMathematicorum,thePrinceofMathematicians.
AnotherofGauss’sinnovationswasthenotionofcomplexnumbers.Mathematicianshadusedimaginarynumbers(xiwherei2=–1)forover200years,butthesenumberswerenotwellunderstoodandwereusuallyavoided.Thesamewastrueforthefirst30yearsofGauss’scareer;wehaveevidencefromhisnotebooksthatheusedimaginarynumberstoderivesomeofhisresults,butthenhereconstructedtheproofssothepublishedversionswouldnotmentioni.(“Themetaphysicsofiisverycomplicated,”hewroteinaletter.)Butin1831,Gausshadaprofoundinsight:herealizedthatnumbersoftheformz=x+yicouldbe
viewedaspoints(x,y)onaCartesianplane.Thesecomplexnumbers,hesaw,werejustaslegitimateandself-consistentasanyothernumbers.Hereareafewdefinitionsandpropertieswe’lluseforcomplexnumbers:
Theabsolutevalueofacomplexnumberzisthelengthofthevectorzonthecomplexplane,whiletheargumentistheanglebetweentherealaxisandthevectorz.Forexample,|i|=1andarg(i)=90°.JustasStevindidforpolynomials,Gaussdemonstratedthatcomplexnumberswereinfactfull-fledged
numberscapableofsupportingordinaryarithmeticoperations:
Multiplyingtwocomplexnumberscanalsobedonebyaddingtheargumentsandmultiplyingtheabsolutevalues.Forexample,ifwewanttofind ,weknowitwillalsohaveanabsolutevalueof1andanargumentof45°(since1·1=1and45+45=90).
***
GaussalsodiscoveredwhatarenowcalledGaussianintegers,whicharecomplexnumberswithintegercoefficients.Gaussianintegershavesomeinterestingproperties.Forexample,theGaussianinteger2is
notprime,sinceitcanbeexpressedastheproductoftwootherGaussianintegers,1+iand1–i.Wecan’tdofulldivisionwithGaussianintegers,butwecandodivisionwithremainder.Tocompute
theremainderofz1andz2,Gaussproposedthefollowingprocedure:
1.Constructagridonthecomplexplanegeneratedbyz2,iz2,−iz2,and−z2.2.Findasquareinthegridcontainingz1.3.Findavertexwofthegridsquareclosesttoz1.
4.z1–wistheremainder.
Gaussrealizedthatwiththisremainderfunction,hecouldapplyEuclid’sGCDalgorithmtocomplexintegers,aswe’vedonehere:Clickheretoviewcodeimage
complex<integer>gcd(complex<integer>a,complex<integer>b){
while(b!=complex<integer>(0)){
a=remainder(a,b);
std::swap(a,b);
}
returna;
}
Theonlythingwe’vechangedarethetypes.
***
Gauss’sworkwasextendedbyanotherGöttingenprofessor,PeterGustavLejeune-Dirichlet.WhileGauss’scomplexnumberswereoftheform(inDirichlet’sterminology) ,Dirichletrealizedthatthiswasaspecialcaseof whereadidnothavetobe1,andthatdifferentpropertiesfollowedfromtheuseofdifferentvalues.Forexample,thestandardGCDalgorithmworksonnumbersofthisformwhena=1,butitfailswhena=5sincethereendupbeingnumbersthatdon’thaveauniquefactorization.Forexample:
ItturnsoutthatifEuclid’salgorithmworks,thenthereisauniquefactorization.Sincewehavenouniquefactorizationhere,thenEuclid’salgorithmdoesn’tworkinthiscase.Dirichlet’sgreatestresultwashisproofthatifaandbarecoprime(thatis,ifgcd(a,b)=1),thenthere
areinfinitelymanyprimesoftheformak+b.MostofDirichlet’sresultsweredescribedinthesecondgreatbookonnumbertheory,appropriately
calledVorlesungenüberZahlentheorie(“LecturesonNumberTheory”).Thebookcontainsthefollowingimportantinsight,whichweusedinourepigraphforChapter4:
[T]hewholestructureofnumbertheoryrestsonasinglefoundation,namelythealgorithmforfindingthegreatestcommondivisoroftwonumbers.Allthesubsequenttheorems...arestillonlysimpleconsequencesoftheresultofthisinitialinvestigation....
ThebookwasactuallywrittenandpublishedafterDirichlet’sdeathbyhisyoungerGöttingencolleague,RichardDedekind,basedonDedekind’snotesfromDirichlet’slectures.DedekindwassomodestthathepublishedthebookunderDirichlet’sname,evenafteraddingmanyadditionalresultsofhisowninlatereditions.Unfortunately,Dedekind’smodestyhurthiscareer;hefailedtogettenureatGöttingenandendeduponthefacultyofaminortechnicaluniversity.DedekindobservedthatGaussianintegersandDirichlet’sextensionsofthemwerespecialcasesofa
moregeneralconceptofalgebraicintegers,whicharelinearintegralcombinationsofrootsofmonicpolynomials(polynomialswherethecoefficientofthehighest-ordertermis1)withintegercoefficients.Wesaythatthesepolynomialsgeneratethesesetsofalgebraicintegers.Forexample:
Dedekind’sworkonalgebraicintegerscontainedalmostallthefundamentalbuildingblocksofmodernabstractalgebra.ButitwouldtakeanothergreatGöttingenmathematician,EmmyNoether,tomakethebreakthroughtofullabstraction.
8.3NoetherandtheBirthofAbstractAlgebraEmmyNoether’srevolutionaryinsightwasthatitispossibletoderiveresultsaboutcertainkindsofmathematicalentitieswithoutknowinganythingabouttheentitiesthemselves.Inprogrammingterms,wewouldsaythatNoetherrealizedthatwecoulduseconceptsinouralgorithmsanddatastructures,withoutknowinganythingaboutwhichspecifictypeswouldbeused.Inaveryrealsense,Noetherprovidedthetheoryforwhatwenowcallgenericprogramming.Noethertaughtmathematicianstoalwayslookforthemostgeneralsettingofanytheorem.Inasimilarway,genericprogrammingdefinesalgorithmsintermsofthemostgeneralconcepts.
EmmyNoether(1882–1935)
EmmyNoether(pronouncedalmostlike“Nerter,”butwithoutfinishingthefirst“r”sound)wasbornintoanacademicGerman-Jewishfamily.HerfatherwasadistinguishedprofessorofmathematicsattheUniversityofErlangen.Althoughitwasveryunusualforwomenatthetime,Noetherwasabletostudyattheuniversityandgotadoctorateinmathematicsin1907.ShethenstayedonforseveralyearsatErlangen,assistingherfatherandteachingwithoutapositionorsalary.Womenhadbeenexcludedfromacademiccareersforcenturies.WiththesingleexceptionofSofiaKovalevskaya,aRussianmathematicianwhobecameaprofessorinStockholmin1884,therewerenowomeninfacultypositionsinmathematicsatuniversitiesatthetime.Twoofthegreatestmathematiciansoftheday,FelixKleinandDavidHilbert,recognizedNoether’stalent,andfeltthatshedeservedanacademicposition.Theyalsobelievedasamatterofprinciplethatwomenshouldnotbeexcludedfrommathematics.TheyarrangedforNoethertocometoGöttingenin1915.Unfortunately,shestillwasnotofficiallyallowedtoteach;thefacultyresistedherappointment.Forthenextfouryears,allofNoether’scourseswerelistedunderHilbert’sname;shewastreatedasakindofunofficialsubstituteteacher.Evenin1919,whenshewasfinallygiventherighttoteachunderherownname,itwasanunpaidpositionasaPrivatdozent,akindofadjunctprofessor.DuringhertimeatGöttingen,Noethermadeenormouscontributionsintwofields,physicsandmathematics.Inphysics,sheisresponsibleforNoether’stheorem,whichfundamentallyconnectedcertainsymmetriesandphysicalconservationlaws(e.g.,conservationofangularmomentum).AlbertEinsteinwasimpressedbyNoether’stheorem,whichisoneofthemostprofoundresultsintheoreticalphysics.Herresultunderliesmuchofmodernphysics,fromquantummechanicstothetheoryofblackholes.Inmathematics,Noethercreatedthefieldofabstractalgebra.AlthoughearliermathematicianssuchasCauchyandGaloishadworkedwithgroups,rings,andotheralgebraicobjects,theyalwaysusedspecificinstances.Noether’sbreakthroughwastorealizethatthesealgebraicstucturescouldbestudiedabstractly,withoutlookingatparticularimplementations.Noetherwasknownasanoutstandingteacher,andattractedstudentsfromallovertheworld.Underherleadership,theseyoungresearchers(oftencalled“Noether’sBoys”)werecreatinga
newkindofmathematics.In1933,whentheNazisexpelledJewsfromuniversities,NoetherfledtotheUnitedStates.Despitebeingoneofthegreatestmathematiciansintheworld,nomajorresearchuniversitywouldhireher,primarilybecauseshewasawoman.SheendedupwithavisitingappointmentatBrynMawr,asmallundergraduatewomen’scollege.Tragically,EmmyNoetherdiedin1935atage53,afewdaysaftersurgerytoremoveanovariancyst.Sincethen,hercontributionstomathematicshaveincreasinglybeenrecognizedasfundamentalandrevolutionary.
Noetherwaswellknownforherwillingnesstohelpstudentsandgivethemherideastopublish,butshepublishedrelativelylittleherself.Fortunately,ayoungDutchmathematician,BartelvanderWaerden,auditedhercourseandwroteabookbasedonherlectures(whichhecreditsonthetitlepage).CalledModernAlgebra,itwasthefirstbooktodescribetheabstractapproachshehaddeveloped.Thisbook,ModernAlgebra,ledtoafundamentalrethinkingofthewaymodernmathematicsis
presented.Itsrevolutionaryapproach—theideathatyouexpressyourtheoremsinthemostabstractterms—isNoether’screation.Mostofmathematics—notjustalgebra—changedasaresultofherwork;shetaughtpeopletothinkdifferently.
8.4RingsOneofNoether’smostimportantcontributionswasthedevelopmentofthetheoryofanalgebraicstructurecalledaring.4
4Theterm“ring,”coinedbyHilbert,wasintendedtousethemetaphorofabunchofpeopleinvolvedinacommonenterprise,likeacriminalring.Ithasnothingtodowithjewelryrings.
Definition8.3.Aringisasetonwhichthefollowingaredefined:
operations:x+y,–x,xy
constants:0R,1R
andonwhichthefollowingaxiomshold:
Rings5havethepropertiesweassociatewithintegerarithmetic—operatorsthatactlikeadditionandmultiplication,whereadditioniscommutativeandmultiplicationdistributesoveraddition.Indeed,rings
maybethoughtofasanabstractionofintegers,andthecanonicalexampleofaringisthesetofintegers,.Alsoobservethateveryringisanadditivegroupandthereforeanabeliangroup.The“addition”
operatorisrequiredtohaveaninverse,butthe“multiplication”operatorisnot.5Somemathematiciansdefineringswithoutthemultiplicativeidentity1anditsaxioms,andcallringsthatincludethemunitaryrings;we
donotmakethatdistinctionhere.
Inpractice,mathematicianswritethezeroeswithouttheirsubscripts,justaswe’vedoneintheaxioms.Forexample,indiscussingaringofmatrices,“0”refersnottothesingleintegerzerobuttotheadditiveidentitymatrix.Besidesintegers,otherexamplesofringsincludethefollowingsets:•n×nmatriceswithrealcoefficients•Gaussianintegers•Polynomialswithintegercoefficients
Wesaythataringiscommutativeifxy=yx.Noncommutativeringsusuallycomefromtherealmoflinearalgebrawherematrixmultiplicationdoesnotcommute.Incontrast,polynomialringsandringsofalgebraicintegersdocommute.Thesetwotypesofringsleadtotwobranchesofabstractalgebra,knownascommutativealgebraandnoncommutativealgebra.Ringsareoftennotexplicitlylabeledas“commutative”or“noncommutative”;instead,onetypeofringortheotherisassumedfromthebranchofalgebra.WiththeexceptionofSections8.5and8.6,therestofthisbookwilldealwithcommutativealgebra—thekindthatDedekind,Hilbert,andNoetherworkedon—sofromthenonwewillassumeourringsarecommutative.
Definition8.4.Anelementxofaringiscalledinvertibleifthereisanelementx–1suchthat
xx–1=x–1x=1
Everyringcontainsatleastoneinvertibleelement:1.Theremaybemorethanone;forexample,intheringofintegers ,both1and–1areinvertible.
Definition8.5.Aninvertibleelementofaringiscalledaunitofthatring.
Exercise8.4(veryeasy).Whichringcontainsexactlyoneinvertibleelement?WhatareunitsintheringofGaussianintegers?
Theorem8.1:Unitsareclosedundermultiplication(i.e.,aproductofunitsisaunit).
Proof.Supposeaisaunitandbisaunit.Then(bydefinitionofunits)aa–1=1andbb–1=1.So
1=aa–1=a·1·a–1=a(bb–1)a–1=(ab)(b–1a–1)
Similarly,a–1a=1andb–1b=1,so
1=b–1b=b–1·1·b=b–1(a–1a)b=(b–1a–1)(ab)
Wenowhaveatermthat,whenmultipliedbyabfromeitherside,gives1;thattermistheinverseofab:
(ab)–1=b–1a–1
Soabisaunit.
Exercise8.5.Provethat:
•1isaunit.•Theinverseofaunitisaunit.
Definition8.6.Anelementxofaringiscalledazerodivisorif:
1.x≠02.Thereexistsay≠0,xy=0
Forexample,inthering ofremaindersmodulo6,2and3arezerodivisors.
Definition8.7.Acommutativeringiscalledanintegraldomainifithasnozerodivisors.
It’scalled“integral”becauseitselementsactlikeintegers—youdon’tgetzerowhenyoumultiplytwononzerothings.Herearesomeexamplesofintegraldomains:
•Integers•Gaussianintegers•Polynomialsoverintegers•Rationalfunctionsoverintegers,suchas (Arationalfunctionistheratiooftwopolynomials.)
Theringofremaindersmodulo6isnotanintegraldomain.(Whetheraringofremaindersisintegraldependsonwhetherthemodulusisprime.)
Exercise8.6(veryeasy).Provethatazerodivisorisnotaunit.
8.5MatrixMultiplicationandSemiringsNote:Thissectionandthenextassumesomebasicknowledgeoflinearalgebra.Therestofthebookdoesnotdependonthematerialcoveredhere,andthesesectionsmaybeskippedwithoutimpactingthereader’sunderstanding.Inthepreviouschapter,wecombinedpowerwithmatrixmultiplicationtocomputelinearrecurrences.Itturnsoutthatwecanusethistechniqueformanyotheralgorithmsifweuseamoregeneralnotionofmatrixmultiplication.
LinearAlgebraReviewLet’squicklyreviewhowsomebasicvectorandmatrixoperationsaredefined.Innerproductoftwovectors:
Inotherwords,theinnerproductisthesumoftheproductsofallthecorrespondingelements.Theresultofinnerproductisalwaysascalar(asinglenumber).Matrix-vectorproduct:
Multiplyingann×mmatrixwithanm-lengthvectorresultsinann-lengthvector.Onewaytothinkoftheprocessisthattheithelementoftheresultistheinnerproductoftheithrowofthematrixwiththeoriginalvector.Matrix-matrixproduct:
InthematrixproductAB=C,ifAisak×mmatrixandBisanm×nmatrix,thenCwillbeak×nmatrix.TheelementinrowiandcolumnjofCistheinnerproductoftheithrowofAandthejthcolumnofB.Notethatmatrixmultiplicationisnotcommutative:thereisnoguaranteethatAB=BA.Indeed,it’softenthecasethatonlyoneofABandBAwillbewelldefined,sincethenumberofcolumnsofthefirsttermhastomatchthenumberofrowsofthesecond.Evenwhenbothproductsaredefined,theyarealmostalwaysdifferent.
Justaswegeneralizedourpowerfunctiontoworkwithanyoperation,wecannowgeneralizethenotionofmatrixmultiplication.Normallywethinkofmatrixmultiplicationasconsistingofaseriesofsumsofproducts,asshownintheearlierformula.Butwhat’smathematicallyessentialisactuallythattherebetwooperations,a“plus-like”onethatisassociativeandcommutative(denotedby⊕)anda“times-like”onethatisassociative(denotedby⊗),wherethelatteroperationdistributesoverthefirst:
a⊗(b⊕c)=a⊗b⊕a⊗c
(b⊕c)⊗a=b⊗a⊕c⊗a
We’vejustseenanalgebraicstructurethathasoperationslikethis,aring.However,ringshaveafewrequirementswedon’tneed,specificallythoseinvolvingtheadditiveinverseoperation.Instead,whatwewantisasemiring,aringwithoutminus(–).
Definition8.8.Asemiringisasetonwhichthefollowingaredefined:
operations:x+y,xy
constants:0R,1R
andonwhichthefollowingaxiomshold:
Ourdefinitionfollowsthemathematicalconventionofreferringtotheoperationsas+and×ratherthan⊕and⊗.Butaswithallthealgebraicstructureswe’vebeendiscussing,thesymbolsrefertoanytwooperationsthatbehaveinthemannerspecifiedbytheaxioms.Thecanonicalexampleofasemiringisthesetofnaturalnumbers .Whilenaturalnumbersdonot
haveadditiveinverses,youcaneasilyperformmatrixmultiplicationonmatriceswithnon-negativeintegercoefficients.(Infact,wecouldrelaxtherequirementsfurtherbyremovingtheadditiveidentity0andthemultiplicativeidentity1,aswellastheircorrespondingaxioms;matrixmultiplication6wouldstillwork.Wemightrefertoasemiringwithout0and1asaweaksemiring.)
6Hereweareassumingthestraightforwardalgorithmformatrixmultiplication;fasteralgorithmsrequirestrongertheories.
8.6Application:SocialNetworksandShortestPathsWecanusesemiringstosolveavarietyofproblems.Forexample,supposewehaveagraphoffriendships,asinasocialnetwork,andwewanttofindallthepeopleyouareconnectedtothroughanypath.Inotherwords,wewanttoknowwhoyourfriendsare,thefriendsofyourfriends,thefriendsofthefriendsofyourfriends,andsoon.Findingallsuchpathsisknownasfindingthetransitiveclosureofthegraph.Tocomputethetransitive
closure,wetakeann×nBooleanmatrixwhereentryxijis1iftherelationholdsbetweeniandj(inthiscase,ifpersoniisfriendswithpersonj),and0otherwise;we’llalsoassumepeoplearefriendswiththemselves.Here’sasmallexample:
Thematrixtellsuswhoeachperson’sfriendsare.Wecanapplygeneralizedmatrixmultiplicationwherewereplace⊕byBooleanOR(∨)and⊗byBooleanAND(∧).WesaythisisthematrixmultiplicationgeneratedbyaBooleanor{∨,∧}-semiring.Multiplyingthematrixbyitselfusingtheseoperationstellsuswhothefriendsofourfriendsare.Doingthismultiplicationn–1timeswilleventuallyfindallthepeopleineachnetworkoffriends.Sincemultiplyingthematrixbyitselfseveraltimesisjustraisingittoapower,wecanuseourexistingpoweralgorithmtodothecomputationefficiently.Ofcourse,wecanusethisideatocomputethetransitiveclosureofanyrelation.
Exercise8.7.UsingthepoweralgorithmfromChapter7withmatrixmultiplicationonBooleansemirings,writeaprogramforfindingtransitiveclosureofagraph.Applythisfunctiontofindthesocialnetworksofeachpersonintheprecedingtable.
Anotherexampleofaclassicproblemwecansolvethiswayisfindingtheshortestpathbetweenanytwonodesinadirectedgraphlikethisone:
Asbefore,wecanrepresentthegraphasann×nmatrix—thistimeonewhosevaluesaijrepresentthedistancefromnodeitonodej.Ifthereisnoedgefromonenodetoanother,we’llinitiallylistthedistanceasinfinity.
Thistime,weusematrixmultiplicationgeneratedbyatropicalor{min,+}-semiring:
Thatis,the⊕operationismin,andthe⊗operationis+.Again,weraisetheresultingmatrixtothen–1power.Theresulttellsustheshortestpathofanylengthupton–1steps.
Exercise8.8.UsingthepoweralgorithmfromChapter7withmatrixmultiplicationontropicalsemirings,writeaprogramforfindingthelengthoftheshortestpathinagraph.
Exercise8.9.ModifytheprogramfromExercise8.8toreturnnotjusttheshortestdistancebuttheshortestpath(asequenceofedges).
8.7EuclideanDomainsWebeganthischapterbyseeinghowEuclid’sGCDalgorithmcouldbegeneralizedbeyondintegers,firsttopolynomials,thentocomplexnumbers,andsoon.Howfarcouldthisgeneralizationgo?Inotherwords,whatarethemostgeneralmathematicalentitiesthattheGCDalgorithmworkson(thedomainorsettingforthealgorithm)?WiththeabstractionsNoetherhaddeveloped,shewasfinallyableanswerthisquestion:thedomainoftheGCDalgorithmiswhatNoethercalledtheEuclideandomain;itisalsosometimesknownasaEuclideanring.
Definition8.9.EisaEuclideandomainif:
•Eisanintegraldomain•Ehasoperationsquotientandremaindersuchthat
b≠0 a=quotient(a,b)·b+remainder(a,b)
•Ehasanon-negativenorm||x||:E→ satisfying
Theterm“norm”hereisameasureofmagnitude,butitshouldnotbeconfusedwiththeEuclideannormyoumaybefamiliarwithfromlinearalgebra.Forintegers,thenormistheirabsolutevalue;for
polynomials,itisthedegreeofthepolynomial;forGaussianintegers,itisthecomplexnorm.Theimportantideaisthatwhenyoucomputetheremainder,thenormdecreasesandeventuallygoestozero,sinceitmapsintonaturalnumbers.WeneedthispropertytoguaranteethatEuclid’salgorithmterminates.
***
NowwecanwritethefullygenericversionoftheGCDalgorithm:Clickheretoviewcodeimage
template<EuclideanDomainE>
Egcd(Ea,Eb){
while(b!=E(0)){
a=remainder(a,b);
std::swap(a,b);
}
returna;
}
Theprocesswe’vegonethroughintransformingtheGCDalgorithmfromsomethingthatworksonlyonlinesegmentstosomethingthatworksonverydifferenttypesillustratesthefollowingimportantprinciple:
Tomakesomethinggeneric,youdon’taddextramechanisms.Rather,youremoveconstraintsandstripdownthealgorithmtoitsessentials.
8.8FieldsandOtherAlgebraicStructuresAnotherimportantabstractionisthefield.7
7Theterm“field”reliesonthemetaphorofafieldofstudy,notafieldofwheat.
Definition8.10.Anintegraldomainwhereeverynonzeroelementisinvertibleiscalledafield.
Justasintegersarethecanonicalexampleofrings,sorationalnumbers( )arethecanonicalexampleoffields.Otherimportantexamplesoffieldsareasfollows:
•Realnumbers•Primeremainderfields•Complexnumbers
Aprimefieldisafieldthatdoesnothaveapropersubfield(asubfielddifferentfromitself).Itturnsoutthateveryfieldhasoneoftwokindsofprimesubfields: or .Thecharacteristicofafieldispifitsprimesubfieldis (thefieldofintegerremaindersmodulop),and0ifitsprimesubfieldis .
***
Allfieldscanbeobtainedbystartingwithaprimefieldandaddingelementsthatstillsatisfythefieldproperties.Thisiscalledextendingthefield.Inparticular,wecanextendafieldalgebraicallybyaddinganextraelementthatisarootofa
polynomial.Forexample,wecanextend with ,whichisnotarationalnumber,sinceitistherootofthepolynomialx2–2.Wecanalsoextendafieldtopologicallyby“fillingintheholes.”Rationalnumbersleavegapsinthe
numberline,butrealnumbershavenogaps,sothefieldofrealnumbersisatopologicalextensionofthefieldofrationalnumbers.Wecanalsoextendthefieldtotwodimensionswithcomplexnumbers.
Surprisingly,therearenootherfinitedimensionalfieldscontainingreals.88Therearefour-andeight-dimensionalfield-likestructurescalledquaternionsandoctonions.Thesearenotquitefields,becausethey
aremissingcertainaxioms;bothquaternionsandoctonionslackcommutativityofmultiplication,andoctonionsalsolackassociativityofmultiplication.Therearenootherfinite-dimensionalextensionsofrealnumbers.
Uptonow,everyalgebraicstructurewe’veintroducedinthisbookhasoperatedonasinglesetofvalues.Buttherearealsostructuresthataredefinedintermsofmorethanoneset.Forexample,animportantstructurecalledamodulecontainsaprimaryset(anadditivegroupG)andasecondaryset(aringofcoefficientsR),withanadditionalmultiplicationoperationR×G→Gthatobeysthefollowingaxioms:
IfringRisalsoafield,thenthestructureiscalledavectorspace.Agoodexampleofavectorspaceistwo-dimensionalEuclideanspace,wherethevectorsarethe
additivegroupandtherealcoefficientsarethefield.
8.9ThoughtsontheChapterInthischapter,wefollowedthehistoricaldevelopmentofgeneralizingtheideaof“numbers”andthecorrespondinggeneralizationoftheGCDalgorithm.Thisledtothedevelopmentofseveralnewalgebraicstructures,someofwhichweusedtogeneralizematrixmultiplicationandapplyittosomeimportantgraphproblemsincomputerscience.Let’sextendourtablefromSection6.8toincludethenewstructuresweintroducedinthischapter.
Notethateveryrowofthetableincludesalltheaxiomsfromearlierrows.(Inthecaseofsemiringsandrings,the“times”operationinheritsalltheaxiomsfrommonoids,whilethe“plus”operationinheritstheaxiomsfromabeliangroups.)Toillustratethis,we’vegrayedoutoperations,elements,andaxiomsthatappearedpreviouslyinthetable.
Aswedidbefore,wecanalsodefinesomeotherstructuresmoreconciselyintermsofothers:
Thisdiagramshowstherelationshipsbetweensomeofthemostimportantstructuresdiscussedinthischapter:
Thefirsttimeyouencounteralgebraicstructures,itmightseemasiftherearesomanyvarietiesthatit’shardtokeeptrackoftheirproperties.However,theyfitintoamanageabletaxonomythatmakestheirrelationshipsclear—ataxonomythathasenabledgreatprogressinmathematicsoverthelasthundredyears.
9.OrganizingMathematicalKnowledge
Allthetruthsofmathematicsarelinkedtoeachother,andallmeansofdiscoveringthemareequallyadmissible.
Legendre
Nowwe’regoingtolookatsomeofthebuildingblocksfororganizingknowledge,particularlymathematicalknowledge.We’llstartbyexploringthenotionofproofsandtheintroductionoftheideaoftheorems.Thenwe’llexaminesomeimportantexamplesofattemptstobuildupbodiesofknowledgefromaxioms.Mathematicianshavebeenthinkingabouthowtoorganizeknowledgeforthousandsofyears.As
programmers,wewillusetheirorganizationalprinciplesinourdomainofalgorithmsanddatastructures.
9.1ProofsPeoplehadbeendiscoveringandusingmathematicalresultslongbeforetheystartedprovingthem.Yetmathematicalproofsarealsoasurprisinglyoldinvention.Forcenturiesmathematiciansreliedonvisualproofs.TheancientGreeksrealizedthattheycoulduseourinnatespatialreasoningtoprovealgebraicfacts.Herearesomeexamplesofvisualproofs.Commutativityofaddition:a+b=b+a
Ifwehavetwostripsofpaperandtapethemtogethertomakeonestrip,wegetthesamelengthregardlessofwhichoneisontheleftandwhichoneisontheright.Wecanseethisbecausethefigureontherightisamirrorimageofthefigureontheleft.Associativityofaddition:(a+b)+c=a+(b+c)
Ifwehavethreestripsofpaperandwetapethepiecestogethertomakeonelongstrip,itdoesn’tmatterifwetapethefirsttwopiecestogetherandthentapethethirdonetotheresult,orifinsteadwetapethelasttwoandthenthefirst.Eitherwaywe’llendupwithastripofthesamelengthintheend.Commutativityofmultiplication:ab=ba
Arectanglehasacertainlengthandacertainwidth.Ifyouturnitsideways,you’vereversedlengthandwidth,butyouobviouslystillhavethesamerectangle.Infact,thisessentialargumentappearsina19th-centurybookbyDirichlet,whosaysthatwhetheryouarrangesoldiersinrowsorcolumns,youstillhavethesamenumber.Associativityofmultiplication:(ab)c=a(bc)
Whetheryouslicethisrectangularprismalongoneaxisoralonganother,whenyouputtheslicesbacktogether,youstillhavethesamevolume.(a+b)2=a2+2ab+b2:
It’sclearjustbylookingthattherectangleonthelowerleftisthesameareaastherectangleontheupperright:notonlydobothhaveareaab,butyoucouldliterallycutoneout,turnitsideways,andlayitontheother.π>3:
Herewe’veinscribedaregularunithexagon(onewhosesidesarealloflength1)inthecircle.It’sevidentthattheperimeterofthehexagonisshorterthanthecircumferenceofthecircle,becausewheneverwehavetwointersectionpointsbetweenthetwofigures,theshortestpathfromonepointtothenextisalongthehexagon,notthecircle.Sincethetrianglesthatmakeupthehexagonareequilateral,alltheirsidesarelength1,sothediameterofthecircleislength2.Sotheratioofthecircle’scircumferencetoitsdiameter(i.e.,π)mustbegreaterthantheratioofthehexagon’sperimeter(6)toitsdiameter(2).
Exercise9.1.Designvisualproofsforthefollowing:
Exercise9.2.Usingavisualproof,findanupperboundforπ.
***
Asusefulasvisualproofsare,thistechniqueisn’tsufficienttoproveeverytypeofpropositioninmathematics,andsomeoftheproofsarenolongerconsideredrigorousenough.Modernmathematicianshaveavarietyofprooftechniquesavailabletothem,someofwhichwe’veusedthroughoutthisbook,andwhicharesummarizedinAppendixB.Proofsshowconnectionsbetweendifferenttruths.Butwhatexactlyconstitutesaproof?Today,weusethefollowingdefinition:
Definition9.1.Aproofofapropositionis
•Anargument•Acceptedbythemathematicalcommunity•Toestablishthepropositionasvalid
Thesecondpointisoftenoverlooked:proofisfundamentallyasocialprocess,andonethatchangesovertime.Ourconfidenceinaproofincreasesasmorepeopleunderstandandagreewithit.Atthesametime,whatisconsideredavalidprooftodaymightnotbeconsideredavalidproof300yearsfromnow,justassomeproofsthatwereviewedasvalidbyEuler—thegreatest18th-centurymathematician—arefrownedupontoday.Nowwe’llturntoanotherbuildingblockofmathematicalknowledge,theorems.
9.2TheFirstTheoremAswediscussedinChapter2,ancientMediterraneancivilizationsbelievedthattheEgyptianswerethesourceofmathematicalknowledge.WhenGreekcivilizationwasjuststarting,Egyptiancivilizationhadalreadyexistedforthousandsofyears,soitisnotsurprisingthattheleadingthinkersofancientGreecewouldtraveltoEgypttostudywiththeirpriestsandlearntheirwisdom.ThefirstsuchpersonknowntousisThalesofMiletus.ThaleslearnedgeometryfromtheEgyptians,buthewentbeyondtheirwork.WhiletheEgyptianshadalgorithms,Thaleshadatheorem—infact,heinventedtheverynotionofatheorem,whichisapropositionderivablefromotherpropositions.TodayThalesisregardedasthefounderofWesternphilosophy,andmightalsobeconsideredthefirstmathematician.
ThalesofMiletus(flourishedearly6thcenturyBC)
Sometimearoundtheyear750BCanewsocietystartedtoappearindifferentcoastalregionsoftheMediterraneanandevenasfarnorthastheBlackSea.TheycalledthemselvesHellenes;wecallthemGreeks.Theycamefromasmallmountainouscountrywherethegeographypreventedtheemergenceofalargeunifiedkingdomaswascommonelsewhere.Greekslivedinsmall,independentcity-statesunifiednotbyacentralgovernment,butbyacommonlanguageandculture.Wheneveracity’spopulationexceededitsresources,itwouldsendsomeofitscitizenstosettleacolony,anewpracticallyindependentcityonsomeconvenientlylocatedbaywithariver.Within200yearsGreekssettledaroundtheMediterranean,asPlatoputit,“likefrogsaroundapond.”Bysomewherearound600BC,theGreekcoloniesinAsiaMinor(whatisnowTurkey)were
gettingwealthy.Insteadofspendingalltheextramoneyonluxuries,someofthemstartedsupportingintellectualpursuits.Forthefirsttimeinhistorytheylookedbeyondmythologyfortheanswerstoeternalquestionssuchaswhatthingsweremadeof.ThefirstpersontodothisinafundamentalwaywasThalesofMiletus.ThaleswastheoriginatorofwhatancientGreekswouldeventuallycallphilosophyandwhatwenowcallscience.Hewantedtofindthenatural,non-mythologicalexplanationofreality.Heproposedthatallvisiblerealityismadeoutofonesinglesubstance:water.Therefore,visiblerealityexistsinoneofthreestates—gas,liquid,orsolid—andtherearetransitionsbetweenthestates.
WhileinEgypt,Thalescollectedmanygeometricalgorithmsand,probably,someBabylonianastronomicalknowledge.HerodotusreportsthatThaleswasabletopredictatotalsolareclipseayearinadvance.Aristotle—usuallyareliablesource—tellsusthatThaleswasabletopredictanexceptionallylargeharvestofolivesbystudyingweatherpatternsand,bybuyingoptionsontheuseofalltheolivepressesintheregion,madeafortune.Therearemanyotherstoriesabouthisaccomplishments,suchashisdiscoveryofstaticelectricity.Whilewedonotknowexactlywhichstoriesaretrue,Thalesclearlyamassedalargebodyofscientificknowledgeandwasabletoapplyittopracticalproblems.Hisknowledgedidnotperishwithhim;hisstudentscarriedtheprogramforward.Butmoreimportantthananyofhisspecificdiscoverieswashisapproachtounderstandingtheworld,whichisstillthebasisofallscience.
Theorem9.1(Thales’Theorem):ForanytriangleABCformedbyconnectingthetwoendsofacircle’sdiameter(AC)withanyotherpointBonthecircle,∠ABC=90°.
Proof.ConsiderthetrianglesformedbyjoiningpointBwiththecenterofthecircle,D:
SinceDAandDBarebothradiiofthecircle,theyareequalandtriangleADBisisosceles.ThesameistrueforDB,DC,andtriangleBDC.Therefore
wherewegetthethirdequationbyaddingtheprevioustwo.Itwasalsoknownthattheanglesofatriangleaddupto180°,andwecanseethat∠CBAisthesumof∠DBAand∠DBC,so
∠DAB+∠DCB+∠DBA+∠DBC=180°
Bysubstitutingusingtheequalityweestablished,wecanwritethisasfollows:
WhywasThales’discoverysoimportant?Whatherealizedisthattruthsareconnected.Hesawthatifyouhaveonepieceofknowledge,youcanuseittofindanother.Furthermore,theoremsareessentialtotheideaofabstraction,forthevalueofatheoremisthatitappliestoallentitiesthathavecertainproperties.
9.3EuclidandtheAxiomaticMethodIfwewanttobuildupasystemofknowledge,proofsandtheoremsareessentialtools.Butwealsoneedtohaveasetofstartingassumptions,oraxioms,asafoundationforoursystem.Thefirstappearanceoftheaxiomaticmethod,inwhichanentiremathematicalsystemwasbuiltonthe
basisofafewformalprinciples,isinEuclid’sElements.Infact,forcenturiesEuclid’sweretheonlyknownexamplesofaxioms,andtheyappliedonlytogeometry.Eucliddividedhisprinciplesintothreegroups:definitions,postulates,andcommonnotions.Hestarts
withhis23definitions,whichrelatetogeometricfigures.Hereareafewofthem:11Asbefore,weuseSirThomasHeath’stranslationofEuclid’sElements.
1.Apointisthatwhichhasnoparts.2.Alineisabreadthlesslength.
23.Parallelstraightlinesarestraightlineswhich,beinginthesameplaneandbeingproducedindefinitelyinbothdirections,donotmeetoneanotherineitherdirection.
Next,hegavethefollowingfive“commonnotions”:1.Thingswhichareequaltothesamethingarealsoequaltooneanother.2.Ifequalsbeaddedtoequals,thewholeareequal.3.Ifequalsbesubtractedfromequals,theremaindersareequal.4.Thingswhichcoincidewithoneanotherareequaltooneanother.5.Thewholeisgreaterthanthepart.
Todaywewouldexpressthesenotionsasfollows:1.a=c∧b=c a=b2.a=b∧c=d a+c=b+d3.a=b∧c=d a−c=b−d4.a b a=b5.a<a+b
What’sinterestingaboutthesecommonnotionsisthat,unlikethe23definitions,thenotionsarenotlimitedtogeometry;theyalsoapplytopositiveintegers.Infact,thesecommonnotions,suchastransitivityofequality,areessentialtoprogramming.2
2ThedefinitionofregulartypesinChapter7isderivedfromtheseEuclideannotions.
Finally,Euclidintroducedhisfamousfivepostulates.Thesearestatedintermsofallowableoperationsinthe“computationalmachinery”ofhisgeometricsystem.Youcanreadthefirstthreeasbeingprefixedwithastatementlike“Thereisaprocedure...”:
1.Todrawastraightlinefromanypointtoanypoint.2.Toproduceafinitestraightlinecontinuouslyinastraightline.3.Todescribeacirclewithanycenteranddistance.4.Thatallrightanglesareequaltooneanother.5.That,ifastraightlinefallingontwostraightlinesmakestheinterioranglesonthesamesidelessthantworightangles,thetwostraightlines,ifproducedindefinitely,meetonthatsideonwhicharetheangleslessthanthetworightangles.
IfwewerewritingEuclid’ssystemtoday,wewouldconsiderboth“commonnotions”and“postulates”tobeaxioms—unprovableassumptionsonwhichtherestofthesystemisbuilt.Euclid’sfifthpostulate,whichprovidesthebasisforreasoningaboutparallellines,isthemost
importantaxiominthehistoryofmathematics.Alsoknownastheparallelpostulate,itexpressestherelationshowninthefollowingdiagram:
However,therearemanyequivalentwaystoexpressthesamenotion:•Givenalineandapointnotonit,atmostoneparalleltothegivenlinecanbedrawnthroughthepoint.3
3Thisformulation,whichisoftentaughtas“theparallelpostulate”insecondaryschoolgeometry,wasactuallypublishedbyScottishmathematicianJohnPlayfairin1795,andisproperlyknownasPlayfair’sAxiom.
•Thereexistsatrianglewhoseanglesaddupto180°.•Thereexisttwosimilartrianglesthatarenotcongruent.
9.4AlternativestoEuclideanGeometryAlmostfromthetimeEuclidstatedhisfivepostulates,mathematiciansfeltthattherewassomethingdifferentaboutthefifthone.Intuitively,theyfeltthatthefirstfourpostulatesweresomehowmorefundamental;perhapsthefifthpostulatecouldbederivedfromtheothers,andthereforewasnotatrueaxiom.Thusbegana2000-yearsearchforaproofofthefifthpostulate,onepursuedbysuchluminariesastheastronomer(andmathematician)Ptolemy(90–168),thepoet(andmathematician)OmarKhayyam(1050–1153),andtheItalianpriest(andmathematician)GiovanniGirolamoSaccheri,S.J.(1667–1733).SaccheriwroteabookcalledEuclidusVindicatus(“EuclidVindicated”)inwhichheconstructedawholegeometricalsystembasedonthetheassumptionthatthefifthpostulateisfalse,thenclaimedthattheconsequenceswouldbesobizarrethatthepostulatemustbetrue.
Whilemost18th-centurymathematiciansdidn’tcareaboutaxioms,themoodshiftedinthe19thcentury.Mathematiciansstartedtofocusonthefoundationsoftheirwork.Theyrevisitedgeometry,nolongertakingEuclidforgranted,butexamininghisassumptions.Around1824,RussianmathematicianNikolaiLobachevskywasworkingontheproblem.Atsome
point,herealizedthattheparallelpostulatewasjustonepossibleassumption,andthatthecontraryassumptionisequallyvalid.Insteadofsaying“thereisatmostonelinethroughapointparalleltoagivenline,”Lobachevskyessentiallyexploredtheideathat“therearemanylines....”UnlikeSaccheri,Lobachevskyrealizedthattheresultingsystemofgeometrywasentirelyconsistent.Inotherwords,heinventedanentirelynewnon-Euclideangeometry,sometimescalledhyperbolicgeometry.InLobachevsky’sgeometry,therearenosimilartrianglesexceptforcongruentones.Bywayof
analogy,thinkoftrianglesonthesurfaceofasphere.Forsmalltriangles,thesurfaceisalmostplanar,sothesumoftheanglesiscloseto180°.Butasthetrianglesgetbigger,theanglesneedtogetbiggerbecauseofthecurvatureofthesurface.Lobachevsky’smodelwassimilar,butwithspacecurvedintheoppositeway,sothatbiggertrianglescorrespondedtosmallerangles.Lobachevsky’sresults,firstpublishedin1826,weremetwithdismissalandscornfromtheRussian
mathematicalcommunity,andLobachevskyhimselfwasmarginalized.OnepersonwhodidrecognizethevalidityofLobachevsky’sworkwasGauss,wholearnedRussiantoreadLobachevsky’sbook.Butingeneral,itwouldtakemanyyearsbeforehisworkbecameanacceptedpartofmathematics.Today,Lobachevsky’sdiscoveryisconsideredtobeamonumentalturningpointinthehistoryofmathematics.
NikolaiIvanovichLobachevsky(1792–1856)
Intheearly19thcentury,Russiawasnotamajorcenterformathematics(despiteEulerspendingmuchofhiscareerinSt.Petersburg).TherewerenogreatRussianmathematicians.Yetbythemiddleofthe20thcentury,Russiawasamathematicalsuperpower.ThistransformationbeganwiththefirstgreatRussianmathematician,NikolaiIvanovichLobachevsky.Lobachevskydidnotcomefromamajorcity,nordidheattendoneofthetwogreatuniversities
(MoscowandSt.Petersburg);hewasnotsentabroadtolearnfromtheleadingthinkersofEurope.Hedidnotcomefromthearistocracyoreventheuppermiddleclass;heandhisbrotherwere
charitystudentsattheirlocalschool.HegrewupinKazan,aprovincialcityontheVolgariverthatdidnotevenhaveauniversityuntil1805.Lobachevskyenteredtherecentlyfoundeduniversityin1807.(Interestingly,TolstoyandLeninattendedthesameschooldecadeslater.)WhenLobachevskystartedattheUniversityofKazan,therewasnoonetoteachmathematics—
studentsstudiedontheirown.Fortunately,MartinBartels,oneofGauss’sformerprofessors,soonjoinedthefaculty.Afterreceivingamaster’sdegreeandcontinuingtostudyprivatelywithBartels,Lobachevskywasappointedasanadjunctprofessorin1814.Hewouldgoontospendmostofhiscareerattheuniversity,eventuallybeingelecteditsrector(similartopresident)in1827.Despitehishumbleorigins,Lobachevskywasneverafraidtochallengeconventionalopinions.
Hisgroundbreakingworkonnon-Euclideangeometrywassubmittedin1826,butwasnotwidelyknownuntilitwaspublishedasabookin1832.ThebookwaspubliclyridiculedinareviewbyOstrogradsky,animportantRussianmathematicianwhostudiedwithCauchy.Lobachevskycontinuedhisworkonnon-Euclideangeometryfortherestofhislife,refiningit
andpublishingbooksaboutitinvariouslanguages.Bythe1840s,Gaussrecognizedtheimportanceofthework,evenreadingsomeofLobachevsky’sbooksintheoriginalRussian.GaussnominatedhimformembershipintheGöttingenAcademyofSciences,agreathonoratthetime.YetLobachevskywasstillostracizedbytheRussianmathematicalestablishmentuntiltheendofhiscareer.Inhislateryears,Lobachevsky’slifetookatragicturn.Helosthisjobattheuniversity,his
house,andmostofhisproperty,sufferedthedeathsoftwoofhischildren,andthenbecameblind.Evenunderthesecircumstances,hepersistedinhiswork,dictatingamajornewbook,Pangeometry,justbeforehisdeathin1856.
Oftenwhenanewideaemergesinmathorscience,itisdiscoveredindependentlybymultiplepeopleatroughlythesametime.Thiswasthecasewithnon-Euclideangeometry.AtaboutthesametimeLobachevskywasworkinginKazan,ayoungHungarianmathematiciannamedJánosBolyaimadeasimilardiscovery.Afewyearslater,Bolyai’sfatherFarkasBolyai,awell-knownmathprofessorandfriendofGauss,includedtheson’sresultsasanappendixtooneofhisownbooks.FarkassentGaussthebook.AlthoughGaussprivatelyremarkedthatyoungBolyaiwasagenius,theletterhesentFarkashadadiscouragingmessage:
IfIcommencedbysayingthatIamunabletopraisethiswork,youwouldcertainlybesurprisedforamoment.ButIcannotsayotherwise.Topraiseitwouldbetopraisemyself.Indeedthewholecontentsofthework,thepathtakenbyyourson,theresultstowhichheisled,coincidealmostentirelywithmymeditations,whichhaveoccupiedmymindpartlyforthelastthirtyorthirty-fiveyears.
TheletteristypicalofGauss,bothinhisrefusaltogivecredittoothersandinhisinsistencethathisownunpublishedthoughtsgavehimpriority.(WenowknowthatGausshadindeeddiscoveredmanyofthesameideas,buthaddecidednottopublishthembecausehewasafraidofthereaction.)WhyheacknowledgedLobachevsky’sworkbutdismissedBolyai’swewillneverknow.Butwhateverthereason,theresultsweretragic.BolyaiwasdevastatedbyGauss’sresponseandneverattemptedtopublishinmathematicsagain.Evensadder,hebecamementallyunstable.WhenhecameacrossLobachevsky’sbooksometimelater,hewasconvincedthat“Lobachevsky”wasactuallyapseudonymforGauss,whomhebelievedhadstolenhisideas.
***
Oncenon-Euclideangeometrywasdiscovered,manymathematicianswrestledwithwhattheyconsideredtobeanimportantquestion:whichgeometryisactuallycorrect,Euclid’sorLobachevsky’s?Gausstookthequestionquiteseriously,andproposedaningeniousexperimenttotestthetheory.First,findthreemountainsformingatrianglethataresomedistanceapart,butcloseenoughsothata
personstandingontopofeachwithatelescopecanseetheothers.Thensetupsurveyingequipmentoneachpeaktoaccuratelymeasuretheanglesofthetriangle.Iftheanglesaddupto180°,thenEuclidisright;iftheirsumislessthan180°,thenLobachevskyis.Theactualexperimentwasneverconducted.Butovertime,thequestionbecamemoot.Other
mathematicianswouldultimatelyprovetheindependenceofthefifthpostulate,showingthatifEuclideangeometryisconsistent,thensoisLobachevskiangeometry.Meanwhile,mathematiciansbegantotreatquestionsofrealityasirrelevant.Whilemathwasoriginallyinventedtounderstandaspectsoftheworldwelivein,bytheendofthe19thcentury,itbegantobeseenasapurelyformalexercise.
9.5Hilbert’sFormalistApproachOnemustbeabletosay“tables,chairs,beer-mugs”eachtimeinplaceof“points,lines,planes.”
—DavidHilbert
DavidHilbert,perhapsthegreatestmathematicianoftheearly20thcentury,wastheleaderofthisformalistapproach.Inaviewthateventuallybecamestandardthroughoutmathematics,hesaidthatifatheorywasconsistent,itwastrue.WhileallofEuclid’stheoremsandproofsarecorrect,bymodernstandardstheaxiomsaresomewhat
shaky.Ittook2400yearsbeforeanyonetriedtocomeupwithabetterfoundationforgeometry.Hilbertspent10yearsrethinkingEuclidandconstructinghisownaxiomaticsystemforgeometry.Asthequotationsuggests,Hilbertbelievedthatthevalidityofhisaxiomaticsystemshouldnotrelyonanyintuitionsaboutgeometry.Hilbert’ssystemcontainedmanymoreaxiomsthanEuclid’s,makingexplicitmanythingsthatEuclidtookforgranted.Hilberthad:
•7axiomsofconnection(e.g.,iftwopointslieonaplane,thenallpointsonthelinegoingthroughthesepointsareonthisplane)•4axiomsoforder(e.g.,thereisapointbetweenanytwopointsonaline)•1axiomofparallels•6axiomsofcongruence(e.g.,twotrianglesarecongruentifside-angle-side...)•1Archimedes’axiom•1completenessaxiom
Hilbert’sgeometricsystemisquitecomplex,andwasthesubjectofseveralofhiscourses.Unfortunately,bythetimehewasdoneconstructingtheaxioms,hehadnoenergylefttoprovemanygeometrictheorems.Hilbert’sworkontheaxiomsofEuclideangeometrywasthelastmajorworkdoneonthatsubject.
DavidHilbert1862–1943
DavidHilbertwasbornintheGermancityofKönigsberg(nowKaliningrad,Russia).HestudiedmathematicsatthetheUniversityofKönigsberg,continuedforaPh.D.,andeventuallyjoinedthefaculty.Atage33,heacceptedanoffertobecomeaprofessorattheUniversityofGöttingen.Hewould
staytherefortherestofhiscareer.Aswesawearlier(Section8.2),Göttingenwasthecenterofthemathematicaluniverse,andHilberteventuallybecametheleaderofthemathematicalcommunitythereduringthepinnacleofthedepartment’sfame.ItisdifficulttoconveytheastoundingvarietyoffundamentalworkdonebyHilbertandthe
profoundeffecthehadonallofmathematics.Inhisinitialworkoninvarianttheory,Hilbertchampionedtheuseofnonconstructiveproofs,
whichwasaradicalideaatthetime.Infact,heinitiallybecamefamousasmuchfortheapproachasfortheactualresult.Today,nonconstructiveproofsarecommon.Whenaskedtosummarizeworkonalgebraicnumbertheory(theareaDedekindhadbeen
workingon),Hilbertwrotea600-pagevolumecalledZahlbericht(“ReportonNumbers”).Thisbookcapturedandexplainedallthemajordevelopmentsinthefield.Whilehemostlysummarized(andcredited)theworkofothers,Hilbert’sunificationdrovethefieldforward,eventuallyleadingtoNoether’sworkonabstractalgebra.Forthenext10years,whileworkingongeometry,HilbertwentbeyondLobachevskyand
examinedthevalidityofallofEuclid’saxioms.HisbookFoundationsofGeometrynotonlyintroducedhisnewaxioms,butalsotaughtpeopleforthefirsttimehowtothinkaboutandrigorouslyanalyzeanyaxiomaticsystem.Hilbertalsoworkedonphysics,co-inventinggeneralrelativitytheory,largelyindependentlyand
atroughlythesametimeasEinstein.HisinventionofHilbertspaces—anextensionofvectorspacestoinfinitedimensions—becameanimportantbuildingblockinthemathematicalfoundationofquantummechanics.In1900,HilbertgavealectureattheSorbonneinPariswherehelisted10importantunsolved
problemsinmathematicsandchallengedthecommunitytoworkonthem.Thelistwaslater
expandedto23problemsinapublishedpaper.Workontheseproblems,whichbecameknownasHilbert’sproblems,definedmuchofmathematicsinthe20thcentury.Healsospentmuchofthelast25yearsofhiscareerworkingonmechanizingthefoundationsofmathematics,aneffortknownas“Hilbert’sprogram.”AlthoughHilbert’sprogramwasshowntobeflawedbytheworkofKurtGödelandAlanTuring,thatsameworkledtothedevelopmentofthemoderntheoryofcomputation.Hilbertwasnotonlyagreatmathematician,butalsoagreatmentorandsupporterofyounger
colleagues.WhenhisbestfriendHermannMinkowskidied,HilbertspentseveralyearseditingandpublishingMinkowski’swork.HechampionedthecareerofEmmyNoether(Section8.3).Healsocollaboratedwithmanyresearchersinseveralfields;hislecturesonphysicsbecamethebasisofaclassictextco-authoredbyRichardCourant.Hilbert’soneblindspotwashisprideinGermanculture.Withgoodreason,hesawmathematics
inGermanyastheculminationof200yearsofadvances.HewelcomedpeoplefromallovertheworldtojointheGermanmathematicalcommunity.ButhealsobelievedthatonlytheresearchinGermanywasworthyofattention.PerhapsthemostegregiousexamplewashisunwillingnesstocitetheworkofGiuseppePeanoinItalyorrecognizeitsseminalimportancetothefoundationsofmathematics.Atthesametime,HilbertwascompletelyopposedtotheviewsoftheNazis(whocametopowerafewyearsafterhisretirementin1930),havingspentmuchofhiscareerpromotingtheworkofmanycolleagueswhohappenedtobeJewish,includinghisbestfriendMinkowskiandhisprotegéNoether.Sadly,Hilbertlivedtoseeeverythinghecaredaboutdestroyed.Hisfriendsweredriveninto
exile,hisonce-greatdepartmentwasreducedtomediocrity,andhisbelovedcountryembracedbeliefshedespised.Buthismathematicallegacywascarriedaroundtheworldbyhismanystudentsandcollaborators,andlivesontoday.
9.6PeanoandHisAxiomsCertainlyitispermittedtoanyonetoputforwardwhateverhypotheseshewishes,andtodevelopthelogicalconsequencescontainedinthosehypotheses.ButinorderthatthisworkmeritthenameofGeometry,itisnecessarythatthesehypothesesorpostulatesexpresstheresultofthemoresimpleandelementaryobservationsofphysicalfigures.
—GiuseppePeano
EvenbeforeHilbertannouncedhisprogramonformalizingmathematics,othershadbeenworkingonsimilarideasaboutformalizingmathematicalsystems.OneofthesewasItalianmathematicianGiuseppePeano.Asthequotationshows,Peanowasstillinterestedintheconnectionsbetweenmathematicsandreality.In1891,hebeganwritingFormularioMathematico(“MathematicalFormulas”),whichwouldbecomeacomprehensiveworkcontainingallessentialtheoremsinmathematicsexpressedinasymbolicnotationPeanoinvented.Muchofhisnotation,suchasthesymbolsforquantifiersandsetoperations,isstillusedtoday.In1889,Peanopublishedasetofaxiomsthatprovidedaformalbasisforarithmetic.Therewerefive,
justlikeEuclid’s:Thereisaset calledthenaturalnumbers:1.∃0 2.∀n :∃n′ -calleditssuccessor
3.∀ ⊂ :(0 ∧∀n:n n′ ) =4.∀n,m :n′=m′ n=m5.∀n :n′≠0InEnglish,wemightwritethemlikethis:1.Zeroisanaturalnumber.2.Everynaturalnumberhasasuccessor.3.Ifasubsetofnaturalnumberscontainszero,andeveryelementinthesubsethasasuccessorinthesubset,thenthesubsetcontainsallnaturalnumbers.
4.Iftwonaturalnumbershavethesamesuccessor,thentheyareequal.5.Zeroisnotthesuccessorofanynaturalnumber.
Thethirdaxiom,knownastheaxiomofinduction,isthemostimportant.ItsaysthatifwetakeanysubsetSof thatcontainszeroandobeystherulethatthesuccessorofeveryelementisalsoinS,thenSisthesameas .Anotherwaytoputthisis“therearenounreachablenaturalnumbers”;ifyoustartwithzeroandkeeptakingthesuccessor,you’lleventuallygettoeverynaturalnumber.Manymoderntextsputthisaxiomlast,butweusePeano’sorder.4
4Moderntextsoftenalsostartnaturalnumberswith1ratherthan0.
Peano’saxiomstransformedarithmetic.Infact,hewasbuildingonearlierworkbyRichardDedekindandHermannGrassman,bothofwhomshowedhowtoderivesomebasicprinciplesofarithmetic.ButPeanowentfurther,andhiscontributionsweresoimportantthatmathematicianssincethentalkaboutPeanoarithmetic,notjustarithmetic.
GiuseppePeano(1858–1932)
GiuseppePeanowasbornintoapeasantfamilynearTurininthenorthofItaly,rightaroundthetimeItalybecameaunifiedcountry.HeattendedtheUniversityofTurinandeventuallyjoinedthefacultythere.Later,healsobeganteachingattheRoyalMilitaryAcademy.Amonghisbest-knownachievementswasthediscoveryofthespace-fillingcurve,knownasaPeanocurve,which
providedacontinuousmappingfromaone-dimensionalsegmenttoeverypointonatwo-dimensionalsquare.Formostofthe1890s,Peanoworkedonthefoundationsofmathematicsandhisgreatbook
FormularioMathematico.Formulariowasmeanttobeacompendiumofallmathematicalresults,writtenformally.Itwasamasterpiece,notonlyprovidingafoundationformathematicsbutalsocoveringavarietyoftopics,togetherwithreferencestothesourcesintheiroriginallanguages.PeanogaveacopyofthebooktotheBritishphilosopherBertrandRussell,anditstronglyinfluencedRussell’sworkwithWhitehead,PrincipiaMathematica,whichwouldcometoplayanimportantroleinearlytheoriesofcomputation.Initially,PeanopublishedFormularioMathematicoinFrench,buthewasfrustratedbythe
ambiguityinherentinanynaturallanguage.Eventually,around1900,hedecidedthattheonlysolutionwastoinventanunambiguousuniversallanguageforscienceandmathematics,andtothenusethisforhiswriting.ThelanguagePeanodesignedwascalledLatinesineFlexione(“LatinwithoutInflection”),later
renamed“Interlingua.”HisideawastostartwithLatin,buttoreplaceallitsconfusingdeclensions,conjugations,andirregularwordswithasimple,logicalsetofrules.PeanorewroteFormularioinhisnewlanguage,andthiseditionwaspublishedin1908.Here’s
whathisfamousaxiomslookedlikeinInterlingua:0.N0esclasse,vel“numero”esnomencommune.
1.Zeroesnumero.2.Siaesnumero,tuncsuosuccessivoesnumero.3.N0esclasseminimo,quesatisfacadconditione0,1,2;[...]
4.Duonumero,quehabesuccessivoaequale,esaequaleinterse.5.0nonsequeullonumero.Peanoalsostartedusinghisformalnotationinhisteaching,whichprobablydidnotendearhim
tohisstudents.Inaddition,heturnedeverycoursehewassupposedtobeteachingintoadiscussionofthefoundationsofmathematics,whicheventuallycausedhimtolosehispositionatthemilitaryacademy.Peano’sdreamwasthatotherscientistswouldstartpublishingtheirworkinInterlingua,butthis
didnothappen.Infact,fewpeopleevenattemptedtoreadPeano’sbook,andhisworkwaslargelyignored.Towardtheendofhislife,PeanospentmuchofhistimetryingtopromoteInterlingua,andhewasmostlyforgottenbythemathematicalcommunity;theyweremoreinterestedintheworkofHilbertandothersatGöttingen.Eventoday,despiteembracingPeano’sfoundationalaxiomsofarithmetic,mostmathematicians
haveneverreadmorethanthefirstpageofhismonumentalwork.Ithasbeenoutofprintforyears,andhasneverbeentranslatedintoEnglish.
Toprovethateveryaxiomisneeded,weneedtoremoveeachonefromthesetanddemonstratethattheremainingsethasconsequencesthatdonotmeetourintent—inthiscase,thattheydonotcorrespondtowhatwemeanbynaturalnumbers.Removingexistenceof0axiom.Ifweremovethisaxiom,weareforcedtodropallaxiomsthatreferto
zero.Sincewehavenoelementstostartwith,theotheraxiomsneverapplyandcanbesatisfiedbytheemptyset,whichisclearlynotamodelofnaturalnumbers.
Removingtotalityofsuccessoraxiom.Ifweremovetherequirementthateveryvaluehaveasuccessor,thenweendupallowingfinitesetslike{0}or{0,1,2}.Clearly,nofinitesetsatisfiesournotionofnaturalnumbers.(However,oncomputers,wegiveupthisaxiom,sinceallofourdatatypesarefinite;forexample,auint64canexpressonlythefirst264integers.)Removinginductionaxiom.Ifweremovetheinductionaxiom,thenweendupwiththesituationwhere
wehavemoreinteger-likethingsthanthereareintegers.These“unreachable”numbersarecalledtransfiniteordinalsandaredesignatedbyω.Sowecouldendupwithsetslike{0,1,2,3,...,ω,ω+1,ω+2,...},{0,1,2,3,...,ω1,ω1+1,ω1+2,...,ω2,ω2+1,ω2+2,...},andsoon.
Removinginvertibilityofsuccessoraxiom.Ifweremovetherequirementthatequalsuccessorshaveequalpredecessors,thenwe’reallowing“ρ-shaped”structureswhereanitemcanhavemultiplepredecessors,someearlierinthesequenceandsomelater,suchas{0,1,1,1,...},{0,1,2,1,2,...},or{0,1,2,3,4,5,3,4,5,...}.Sinceallofthesestructuresarefinite,theyclearlydonotincludeallnaturalnumbers.Removing“nothinghas0asitssuccessor”axiom.Ifweremovethisaxiom,thenwe’dallow
structuresthatloopbacktozero,like{0,0,...}and{0,1,0,1,...}.Again,thesestructuresarefinite,sotheydonotcaptureournotionofnaturalnumbers.
9.7BuildingArithmeticNowthatwehaveestablishedthatallofPeano’saxiomsareindependent,andthereforenecessaryforournotionofnaturalnumbers,wecanbuilduparithmeticfromfirstprinciples.We’lldothisnowbydefiningexactlywhatitmeanstoaddandmultiplytwonaturalnumbers.DefinitionofAddition:
Wearenotprovingthesestatements;wearedefiningadditiontobethesestatements.Allpropertiesofaddingnaturalnumbersfollowfromthisdefinition.Forexample,here’showweprovethat0istheleftadditiveidentity:
Inthebasisstep,weassertthatit’struewhenaiszero.WeknowthisbecauseofEquation9.1inthedefinitionofaddition.Intheinductivestep,weassumeit’strueforanya.ByEquation9.2,weknowthat0+a′=(0+a)′.Butbytheassumptionoftheinductivestep,wecansubstituteafor0+a,soourresultisa′,andtherefore0+a′=a′.DefinitionofMultiplication:
Wecannowprovethat0·a=0,muchaswedidforaddition:
Definitionof1.Wealsodefine1asthesuccessorof0:
Nowweknowhowtoadd1:
Wealsoknowhowtomultiplyby1:
a·1=a·0′=a·0+a=0+a=a
Wecanderivefundamentalpropertiesofadditionaswell;theyfollowfromtheaxioms.AssociativityofAddition:(a+b)+c=a+(b+c)
inductivestep:
Togetcommutativity,we’llstartbyprovingitforthespecialcase:
inductivestep:
CommutativityofAddition:a+b=b+a
inductivestep:
Exercise9.3.Usinginduction,prove:
•Associativityandcommutativityofmultiplication•Distributivityofmultiplicationoveraddition
Exercise9.4.Usinginduction,definetotalorderingbetweennaturalnumbers.
Exercise9.5.Usinginduction,definethepartialfunctionpredecessoronnaturalnumbers.
***
DoPeanoaxiomsdefinenaturalnumbers?No;asPeanoputit,“number(positiveinteger)cannotbedefined(seeingthattheideasoforder,succession,aggregate,etc.,areascomplexasthatofnumber).”Inotherwords,ifyoudon’talreadyknowwhattheyare,Peano’sdefinitionswon’ttellyou.Instead,theydescribeourexistingideaofnumbers,formalizingournotionsofarithmetic,whichhelpsprovideawaytostructureproofs.Ingeneral,wecansaythataxiomsexplain,notdefine.Theexplanationmaynotbeconstructive;thatis,
itmightnotsayhowtheresultisachieved.Evenifitdoessuggestanalgorithm,thealgorithmcouldbecomputationallyveryinefficient.Nosanepersonwoulddoadditionbyrepeatedapplicationofthesuccessorfunction.Buttheseaxiomsstillserveausefulpurpose;theygetustothinkaboutwhichpropertiesofnaturalnumbersareessentialandwhicharenot.Thisapproachisagoodattitudetotakewhenstudyingthedocumentationforaprogramminginterface.
Whyisthatrequirementimposed?Whatwouldtheconsequencesbeifitwerenotthere?
9.8ThoughtsontheChapterWebeganthechapterbylookingatthenotionofproof,aformal—yetsocial—processfordemonstratingthetruthofaproposition.Wesawhowproofsshowconnectionsbetweentruths;proofsystemsareawaytoorganizeknowledge.Wealsolookedatthediscoveryoftheorems,andtheimportantabstractiontheyprovide.Nextwelookedataricherformalismfororganizingknowledge,theaxiomaticsystem,andsawhow
geometryandarithmeticcouldbebuiltupfromfirstprinciples.Thecriticalroleofaxiomaticsystemsistheirabilitytoreducethecomplexityofknowledge.Youdon’tneedtomemorizeallthetruepropositions,becauseyoucanderivethemfromafewaxiomsandinferencerules.However,it’simportanttorememberthathistorically,mathematiciansdidnotreallystartwithaxioms
andderivetheoremsfromthem.Theaxiomswereproposedonlyaftertheinterrelationshipsbetweenthe
theoremswerewellunderstoodandtheassumptionsunderlyingthemidentified.Thesameprocessholdsforprogramming:designinggoodabstractionsrequiresexaminingalargenumberofrealalgorithmsandunderstandingtheirinterrelationships.Whileaxiomaticsystemsallowustoorganizeknowledge,theypresupposethatwealreadyhavesome
knowledgetoorganize.Discoveryofatheoremisamoreimportantthingthanprovingit—youcannotattempttoprovesomethingunlessyouhavereasontobelieveitisatruth.Sometimesmodernmathematiciansforgettheempiricaloriginsofknowledge.InhisbookTheMethod,
thegreatGreekmathematicianArchimedesdiscussedhowanymeansforacquiringmathematicalknowledgewasvalid,includingmeasurementandexperimentation.Onlyafterdiscoveringmathematicaltruthsshouldoneattempttoderivearigorousproof.Thesameprincipleappliestoprogramming:beforetryingtoproveaprogramcorrect,weshouldtrytowritecorrectprograms—evenifourattemptsinvolvetrialanderror.
10.FundamentalProgrammingConcepts
Allhumansnaturallydesiretoknow.Aristotle,MetaphysicsI,1
Inthischapterwewillintroducesomeoftheimportantideasassociatedwithgenericprogramming,includingconceptsanditerators.We’llalsoconsidersomecommonprogrammingtasksthatrelyonthem.Butwe’llstartbylookingattheoriginsofthenotionofabstraction.
10.1AristotleandAbstractionTheSchoolofAthens,afamouspaintingbyItalianRenaissancepainterRaphael,depictsmanyancientGreekphilosophers(seedetailofpaintingonthenextpage).AtthecenterarePlatoandAristotle,thetwomostimportantphilosophersoftheancientworld.Platoispointingupward,whilehisstudentAristotleholdshishandoutovertheground.Accordingtopopularinterpretation,Platoispointingtotheheavens,indicatingthatweshouldcontemplatetheeternal,whileAristotleisindicatingthatweneedtostudytheworld.Infact,itcouldbesaidthatPlatoinventedmathematicsandAristotleinventedthestudyofeverythingelse,especiallyscience.Aristotle’sworkscovereverythingfromaestheticstozoology.
Aristotle(384BC–322BC)
AristotlecamefromStageira,acityinthefarnorthofGreece.Weknowverylittleabouthisearlylife,butweknowthatatsomepointhedecidedtomovetoAthensinsearchofwisdom.Hestudied(andatsomepoint,probablytaught)atPlato’sAcademyforabout20years.AroundthetimeofPlato’sdeath—perhapsdisappointedbecausehewasnotappointedPlato’ssuccessor—heleftAthens.In343BC,KingPhilipofMacedonappointedAristotletobeatutorforhissonAlexanderand
Alexander’scompanions.Wedon’tknowmuchaboutAristotle’srelationshipwiththeprince,butinlateryearswhenAlexanderbecamekingandbegantheAsianconqueststhatearnedhimthenickname“theGreat,”hesentAristotleexoticplantandanimalspecimensforthephilosopher’scollection.Around335BC,AristotlereturnedtoAthensandcreatedhisowngreatschool,theLyceum.Duringthe
next12years,heproducedthemostastonishingcollectionofknowledgeeverassembled.WhilehisteacherPlatohadfocusedonstudyingeternaltruths,Aristotlewantedtounderstandtheworldasitreallywas.Forexample,whenPlatowantedtowriteaboutpolitics,hedescribedwhattheidealsocietywouldbe.WhenAristotlewroteaboutpolitics,heaskedhisstudentstovisiteachoftheimportantGreekcity-statesandthenreportontheirconstitutions.Aristotle’sapproachwastoobserveeverything,describeit,andcomeupwithexplanationsforwhathesaw.Aristotletaught,andwroteabout,nearlyeverysubjectimaginable.Accordingtoseveralimportant
writersoftheperiod,Aristotlewrotebeautifully;unfortunatelythebooksheintendedforpublication,suchashisdialogues,haveallbeenlost.Whatwehaveinsteadaretersetreatisesthatwereprobablylecturenotes.Nevertheless,manyofhisworks,suchasNicomacheanEthics,Politics,andMetaphysicsarestillessentialreadingtoday.Andregardlessoftheirstyle,hiscollectedworksconstitutethefirstencyclopedictreatmentofknowledge.AlthoughAristotle’sscientificdescriptionsincludefactualerrors,hewasthefirstpersonto
systematicallydescribethescientificworld,withobservationsontopicsasdetailedashowtheoctopusreproduces.AristotleleftAthensaround322BCanddiedsoonafterward.Whileotherphilosophicaltraditions
(e.g.,Stoicism,Platonism)persistedinancienttimeslongaftertheirfounders’deaths,Aristotle’sdidnot.GreekphilosophybecameincreasinglyintrospectiveandlostinterestinAristotle’sideaofstudyingobservablereality.TheLyceumrapidlydeclinedinimportance,andfewscholarsinlaterGreekand
RomantimesconsideredthemselvesAristotelians.EvenwhenhisworkwasrediscoveredintheMiddleAges,medievalscholarsslavishlystudiedhiswritingsratherthanfollowinghismethodologyofgoingoutandobservingtheworld.Aristotle’sgreatlegacyishisempiricalapproach,whichisthefoundationalprincipleofallmodernscience.Furthermore,theorganizationofknowledgereflectedinthedepartmentsofmodernuniversitiesisadirectdescendentofthetaxonomythatAristotleproposed.
Aristotle’swritingswerepreservedtowardtheendofthefirstmillenniumADbyArabphilosophers.Inthe12thcentury,whenChristiankingdomsrecapturedmuchofSpainfromtheIslamicstateknownasAl-Andalus,theyfoundalibraryintheSpanishcityofToledocontainingagreatnumberofbooksinArabic,includingtranslationsofAristotle’sworks.EventuallythebooksweretranslatedfromtheoriginalGreekintoLatin,andanAristotelianrenaissancespreadthroughEurope.Aristotlebecameknownassimply“thePhilosopher,”andhisworksbecameapartofgenerallyacceptedknowledge.ThegreatAndalusianphilosopherIbnRushd(knownasAverroestoEuropeans)wrotewidelyreadcommentariesonAristotle,reconcilinghisphilosophywiththeteachingsofIslam;hebecameknownsimplyas“theCommentator.”Inthe13thcentury,ChristianscholarssuchasThomasAquinasandDunsScotussimilarlyshowedthatAristotelianismwascompatiblewithChristianity;thiswasoftendescribedas“baptizing”Aristotle.Asaresult,Aristotle’sworkswerepartoftherequiredstudyatEuropeanuniversitiesforliterallyhundredsofyears.AmongAristotle’smostimportantworksistheOrganon,acollectionofsixtreatisesonvarious
aspectsoflogicthatwoulddefinethefieldforthenext2600years.1InCategories,thefirstpartoftheOrganon,Aristotleintroducedthenotionofabstraction.Hewroteaboutthedistinctionbetweenanindividual,aspecies,andagenus.Whiletodaymostpeoplethinkoftheseasbiologicaldistinctions,forAristotletheyappliedtoeverything.Aspeciesincorporatesallthe“essential”propertiesofatypeofthing.Agenusmaycontainmanyspecies,eachidentifiedbyitsdifferentia—thethingsthatdifferentiateitfromotherspeciesinthegenus.
1UnlikeAristotle’sotherworks,aLatinversionoftheOrganonwasavailabletoEuropeansmuchearlier;Boethiusprovidedthetranslationintheearly6thcentury.
ItisAristotle’sideaofgenusthatinspiredthetermgenericprogramming—awaytothinkaboutprogrammingthatfocusesonthelevelofgenera(thepluralofgenus)ratherthanspecies.
10.2ValuesandTypesNowwewillseehowsomeoftheideaswe’vebeendiscussingfitincomputerprogramming.First,weneedafewdefinitions:
Definition10.1.Adatumisasequenceofbits.
01000001isanexampleofadatum.
Definition10.2.Avalueisadatumtogetherwithitsinterpretation.
Adatumwithoutaninterpretationhasnomeaning.Thedatum01000001mighthavetheinterpretationoftheinteger65,orthecharacter“A,”orsomethingelsealtogether.Everyvaluemustbeassociatedwithadatuminmemory;thereisnowaytorefertodisembodiedvaluesinalanguagelikeC++orJava.
Definition10.3.Avaluetypeisasetofvaluessharingacommoninterpretation.
Definition10.4.Anobjectisacollectionofbitsinmemorythatcontainavalueofagivenvaluetype.
Thereisnothinginthedefinitionthatsaysthatallthebitsofanobjectmustbecontiguous.Infact,it’squitecommonforpartsofanobjecttobelocatedatdifferentplacesinmemory;thesearecalledremoteparts.Anobjectisimmutableifthevalueneverchanges,andmutableotherwise.Anobjectisunrestrictedif
itcancontainanyvalueofitsvaluetype.
Definition10.5.Anobjecttypeisauniformmethodofstoringandretrievingvaluesofagivenvaluetypefromaparticularobjectwhengivenitsaddress.
Whatwecall“types”inprogramminglanguagesareobjecttypes.C++,Java,andotherprogramminglanguagesdonotprovidemechanismsfordefiningvaluetypes.2Everytyperesidesinmemoryandisanobjecttype.
2Theiteratortraitvalue_typeinC++,despiteitsname,doesnotactuallyreturnavaluetypeinthesensedescribedhere.Rather,itreturnstheobjecttypeofthevaluepointedtobytheiterator.
10.3ConceptsTheessenceofgenericprogrammingliesintheideaofconcepts.Aconceptisawayofdescribingafamilyofrelatedobjecttypes.Therelationshipbetweenconceptandtypeisexactlytherelationshipbetweentheoryandmodelinmathematics,andbetweengenusandspeciesinthescientifictaxonomyintroducedbyAristotle.
HerearesomeexamplesofconceptsandsomeoftheirtypesinC++:•Integral:3int8_t,uint8_t,int16_t,...
3Thisrefersspecificallytothelistofbuilt-inC++integraltypes.ThereisabroaderconceptIntegerthatincludesalloftheseplusotherrepresentationsofintegerssuchasinfinite-precisionintegers.
•UnsignedIntegral:uint8_t,uint16_t,...•SignedIntegral:int8_t,int16_t,...
Whileconceptsexistinmanylanguagesimplicitly,veryfewlanguagesprovideanexplicitwaytotalkaboutthem.4
4TherehavebeenproposalstoincludeconceptsinC++;theworkisstillinprogress.Therearealsoconcept-likefeaturesinsomefunctionalprogramminglanguagessuchasHaskell.
Manyprogramminglanguagesprovideamechanismtospecifytheinterfaceofatypetobeimplementedlater:abstractclassesinC++,interfacesinJava,andsoon.However,thesemechanismscompletelyspecifytheinterface,includingstrictrequirementsonthetypesofargumentsandreturnvalues.Incontrast,conceptsallowinterfacestobespecifiedintermsoffamiliesofrelatedtypes.Forexample,inbothJavaandC++,youcanspecifyaninterfacecontainingafunctionsize()returningavalueoftypeint32.Intheworldofconcepts,youcanhaveaninterfacewithafunctionsize()thatreturnsavalueofanyintegraltype—uint8,int16,int64,etc.
Aconceptcanbeviewedasasetofrequirementsontypes,orasapredicate5thattestswhethertypesmeetthoserequirements.Therequirementsconcern
5Apredicateisafunctionthatreturnstrueorfalse.
•Theoperationsthetypesmustprovide•Theirsemantics•Theirtime/spacecomplexity
Atypeissaidtosatisfyaconceptifitmeetstheserequirements.Whenfirstencounteringconcepts,programmersoftenwonderwhythethirdrequirement,forspace/time
complexity,isincluded.Isn’tcomplexityjustanimplementationdetail?Toanswerthisquestion,considerareal-worldexample.Supposeyoudefinedtheabstractdatatypestack,butyouimplementeditasanarray,insuchawaythateverytimeyoupushedsomethingontothearray,youhadtomoveeveryexistingelementtothenextpositiontomakeroom.Insteadofpushingonthestackbeingafastoperation(constanttime),it’snowaslowoperation(lineartime).Thisviolatesaprogrammer’sassumptionofhowstacksshouldbehave.Inasense,astackthatdoesn’thavefastpushandpopisnotreallyastack.Thereforetheseverybasiccomplexityconstraintsarepartofwhatitmeanstosatisfyaconcept.
***
Twousefulideasrelatedtoconceptsaretypefunctionsandtypeattributes.Atypefunctionisafunctionthat,givenatype,returnsanaffiliatedtype.Forexample,itwouldbenicetohavetypefunctionslikethis:
•value_type(Sequence)•coefficient_type(Polynomial)•ith_element_type(Tuple,size_t)
Unfortunately,mainstreamprogramminglanguagesdonotcontaintypefunctions,eventhoughtheywouldbeeasytoimplement.(Afterall,thecompileralreadyknowsthingslikethetypeofelementsinasequence.)Atypeattributeisafunctionthat,givenatype,returnsavaluerepresentingoneofitsattributes.For
example:•sizeof•alignment_of•Numberofmembersinastruct•Nameofthetype
Somelanguagesprovidesometypeattributes,likesizeof()inCandC++.
***
Let’slookatsomeverygeneralconcepts.ThefirstoneisRegular,6whichweintroducedbackinChapter7.Roughlyspeaking,atypeisregularifitsupportstheseoperations:
6Byconvention,wewriteconceptnameswithinitialcapitals,anddisplaythemwithaSansSeriftypeface.
•Copyconstruction•Assignment•Equality
•DestructionHavingacopyconstructorimplieshavingadefaultconstructor,sinceTa(b)shouldbeequivalenttoTa;a=b;.TodescribethesemanticsofRegular,we’llexpresstherequirementsasaxioms:
Thefirstaxiomsaysthatifyoucopyconstructafromb,thenanythingthatwasequaltobwillnowalsobeequaltoa.Thesecondaxiomsaysthatifyouassignbtoa,thenanythingthatwasequaltobwillnowalsobeequaltoa.Thethirdaxiomusesthenotionofaregularfunction(nottobeconfusedwitharegulartype),whichisonethatproducesequalresultsgivenequalinputs.It’stheresponsibilityoftheprogrammertospecifywhichfunctionsaresupposedtoberegular;onlythencanotherprogrammers(or,inthefuture,thecompiler)relyonthefactthatthesefunctionswillpreserveequality.ThecomplexityrequirementsonRegulararethateachoperationisnoworsethanlinearintheareaof
theobject,whereareaisdefinedasallspaceoccupiedbytheobject,bothitsheaderanditsremoteparts,bothitsdataanditsconnectors.7
7ForamoreformaltreatmentoftheconceptRegular,seeElementsofProgrammingSection1.5.
TheconceptRegularisuniversal—it’snotspecifictoanyprogramminglanguage.Atypeinanylanguageisregularifitsatisfiestherequirements.ArelatedconceptisSemiregular,whichisjustlikeRegularexceptthatequalityisnotexplicitly
defined.Thisisneededinafewsituationswhereitisverydifficulttoimplementanequalitypredicate.Eveninthesesituations,equalityisassumedtobeimplicitlydefined,sothataxiomsthatcontrolcopyingandassignmentarestillvalid.Afterall,aswesawearlier,themeaningofassigningatobisthatb’svaluewillbeequaltoa’svalueafterward.
10.4IteratorsAniteratorisaconceptusedtoexpresswhereweareinasequence.Infact,iteratorswereoriginallygoingtobecalled“coordinates”or“positions”;theymaybeviewedasageneralizationofpointers.Insomeprogramminglanguages,whattheycalliteratorsareheavyweightbundlesoffunctionality,buttheconceptofaniteratorjustexpressesthisverysimplenotionofposition.Tobeaniterator,atypemustsupportthreeoperations:•Regulartypeoperations•Successor•Dereference
Onewaytothinkofaniteratoris“somethingthatletsyoudolinearsearchinlineartime.”Theessenceofaniteratoristhenotionofsuccessor.Indeed,iteratorscometousdirectlyfromPeano’saxioms;essentially,theconceptIteratoris“atheorywithsuccessor.”However,ouriteratorconceptswillbelessstrict,becausewedon’trequireallofPeano’saxioms.Forexample,inPeanoarithmetic,everynumberhasasuccessor,whilewithiterators,thisisnotalwaysthecase—sometimeswegettotheendofourdata.Peanoalsotellsusthatifsuccessorsareequal,thepredecessorsmustbeequal,andthatwecan’thaveloops.Theserequirementsarealsonotthecaseforprogrammers;we’reallowedtohavedatastructuresthatlinkbacktoearlierelementsandformloops.Infact,thisissometimesexactlywhatweneedtodoacomputationaltaskefficiently.
Theseconditeratoroperation,dereferencing,isawaytogetfromaniteratortoitsvalue.Dereferencinghasatimecomplexityrequirement;it’sassumedtobe“fast,”whichmeansthatthereisnotafasterwayofgettingtodatathanthroughtheiterator.Iteratorsaresometimesbiggerthantraditionalpointers,insituationswheretheyneedtostoresomeadditionalstatetoenablefastnavigation.Iteratorsmayalsosupportspecialvaluesindicatingthatwearepasttheendoftheobject,aswellassingularvalueslikethenullpointerthatcannotbedereferenced.It’sokaythatdereferencingisapartialfunction(i.e.,thatitisn’tdefinedforallvalues);afterall,mathematicianshavenotroublesayingwhatdivisionis,eventhoughdivisionbyzeroisnotdefined.Dereferencingandsuccessorarecloselyconnected,8andthisrelationshipimposesthefollowing
restrictions:8SeeChapter6ofElementsofProgrammingformoreabouttherelationshipbetweendereferencingandsuccessor.
•Dereferencingisdefinedonaniteratorifandonlyifsuccessorisdefined.•Ifyouarenotattheendoftherange,youcandereference.
Whydoweneedequalityasarequirementforiterators?Inotherwords,whydoweneediteratorstoberegularratherthansemiregular?Becauseweneedtobeabletoseewhenoneiteratorreachesanother.
10.5IteratorCategories,Operations,andTraitsThereareseveralkindsofiterators,whichwecalliteratorcategories.Herearethemostimportant:
•Inputiteratorssupportone-directionaltraversal,butonlyonce,asisfoundinsingle-passalgorithms.Thecanonicalmodelofaninputiteratoristhepositioninaninputstream.Bytesarecomingoverthewireandwecanprocessthemoneatatime,butoncetheyareprocessed,theyaregone.Inparticular,withinputiteratorsi==jdoesnotimply++i==++j;forexample,ifyou’vealreadyconsumedacharacterfromaninputstream,youcan’tconsumethesamecharacteragainwithadifferentiterator.Keepinmindthatjustbecauseanalgorithmonlyrequiresinputiteratorsdoesnotmeanitislimitedtooperatingoninputstreams.•Forwarditeratorsalsosupportonlyone-directionaltraversal,butthistraversalcanberepeatedasneeded,asinmulti-passalgorithms.Thecanonicalmodelofaforwarditeratoristhepositioninasinglylinkedlist.9
9Weassumethatlinkstructureofthelistisnotmodifiedasitistraversed.
•Bidirectionaliteratorssupportbidirectionaltraversal,repeatedasneeded(i.e.,theyalsocanbeusedinmulti-passalgorithms).Thecanonicalmodelofabidirectionaliteratoristhepositioninadoublylinkedlist.Bidirectionaliteratorshaveaninvertiblesuccessorfunction:ifanelementxhasasuccessory,thenyhasapredecessor.•Random-accessiteratorssupportrandom-accessalgorithms;thatis,theyallowaccesstoanyelementinconstanttime(bothfarandfast).Thecanonicalmodelisthepositioninanarray.
Inaddition,thereisanothercommoniteratorcategorythatbehavesdifferentlyfromtheothers:•Outputiteratorssupportalternatingsuccessor(++)anddereference(*)operations,buttheresultsofdereferencinganoutputiteratorcanappearonlyontheleft-handsideofanassignmentoperator,andtheyprovidenoequalityfunction.Thecanonicalmodelofanoutputiteratoristhepositioninanoutputstream.Wecan’tdefineequalitybecausewecan’tevengettotheelementsoncethey’vebeenoutput.
WhiletheiteratorsdescribedsofararetheonlyonesincludedinC++,otherusefuliteratorconceptsalsoexist:
•Linkediteratorsworkinsituationswherethesuccessorfunctionismutable(forexample,alinkedlistwherethelinkstructureismodified).•Segmentediteratorsareforcaseswherethedataisstoredinnoncontiguoussegments,eachcontainingcontiguoussequences.std::deque,adatastructurethatisimplementedasasegmentedarray,wouldimmediatelybenefit;insteadofneedingeachsuccessoroperationtocheckwhethertheendofthesegmenthasbeenreached,a“toplevel”iteratorcouldfindthenextsegmentandknowitsbounds,whilethe“bottomlevel”iteratorcoulditeratethroughthatsegment.
Iteratorslikethesecaneasilybeimplemented.Justbecauseaconceptisnotbuiltintothelanguagedoesnotmeanit’snotuseful.Ingeneral,STLshouldbeviewedasasetofwell-chosenexamples,notanexhaustivecollectionofallusefulconcepts,datastructuresandalgorithms.
***
Asimplebutimportantthingwemaywanttodoisfindthedistancebetweentwoiterators.Foraninputiterator,wemightwriteourdistance()functionlikethis:Clickheretoviewcodeimage
template<InputIteratorI>
DifferenceType<I>distance(If,Il,std::input_iterator_tag){
//precondition:valid_range(f,l)
DifferenceType<I>n(0);
while(f!=l){
++f;
++n;
}
returnn;
}
Therearethreenotablethingsaboutthiscode:theuseofthetypefunctionDifferenceType,theuseoftheiteratortagargument,andtheprecondition.We’lldiscussallofthesesoon,butbeforewedo,let’scomparethistoadifferentimplementation—onethat’soptimizedforrandomaccessiterators:Clickheretoviewcodeimage
template<RandomAccessIteratorI>
DifferenceType<I>distance(If,Il,
std::random_access_iterator_tag){
//precondition:valid_range(f,l)
returnl-f;
}
Sincewehaverandomaccess,wedon’thavetorepeatedlyincrement(andcount)fromoneiteratortotheother;wecanjustuseaconstanttimeoperation—subtraction—tofindthedistance.Thedifferencetypeofaniteratorisanintegraltypethatislargeenoughtoencodethelargestpossible
range.Forexample,ifouriteratorswerepointers,thedifferencetypeinC++couldbeptrdiff_t.Butingeneralwedon’tknowinadvancewhichtypetheiteratorwillbe,soweneedatypefunctiontogetthedifferencetype.AlthoughC++doesnothaveageneralmechanismfortypefunctions,STLiteratorshaveaspecialsetofattributesknownasiteratortraits,oneofwhichgivesusthedifferencetype.Thecompletesetofiteratortraitsis
•value_type•reference•pointer
•difference_type•iterator_category
We’vementionedvalue_typebefore;itreturnsthetypeofthevaluespointedtobytheiterator.Thereferenceandpointertraitsarerarelyusedincurrentarchitectures,10buttheothersareveryimportant.
10EarlierversionsoftheIntelprocessorarchitectureincludeddifferenttypesforshorterandlongerpointers,soitwasimportanttoknowwhichtouseforagiveniterator.Today,ifthevaluetypeofaniteratorisT,thepointeriteratortraitwouldnormallybeT*.
Sincethesyntaxforaccessingiteratortraitsisratherverbose,we’llimplementourowntypefunctionforaccessingdifference_type,withtheusingconstructofC++11.(SeeAppendixCformoreinformationaboutusing.)Clickheretoviewcodeimage
template<InputIteratorI>
usingDifferenceType=
typenamestd::iterator_traits<I>::difference_type;
ThisgivesustheDifferenceTypetypefunctionusedintheearliercode.Theiteratortraititerator_categoryreturnsatagtyperepresentingthekindofiteratorwe’re
dealingwith.Objectsofthesetagtypescontainnodata.AswedidforDifferenceType,wedefinethefollowingtypefunction:Clickheretoviewcodeimage
template<InputIteratorI>
usingIteratorCategory=
typenamestd::iterator_traits<I>::iterator_category;
Nowwecanreturntotheuseoftheiteratortagargumentinthedistancefunctions.Theiteratortagsshownintheexamples(input_iterator_tagandrandom_access_iterator_tag)arepossiblevaluesoftheiteratorcategorytrait,sobyincludingthemasarguments,wearedistinguishingthetypesignatureofthetwofunctionimplementations.(WewillseemoreexamplesofthisinChapter11.)Thisallowsustoperformcategorydispatchonthedistancefunction;thatis,wecanwriteageneralformofthefunctionforanyiteratorcategory,andthefastestonewillbeinvoked:Clickheretoviewcodeimage
template<InputIteratorI>
DifferenceType<I>distance(If,Il){
returndistance(f,l,IteratorCategory<I>());
}
Notethatthethirdargumentisactuallyaconstructorcallcreatinganinstanceoftheappropriatetype,becausewecannotpasstypestofunctions.Whentheclientcallsdistance(),itusesthetwo-argumentversionshownhere.Thatfunctiontheninvokestheimplementationthatmatchestheiteratorcategory.Thisdispatchhappensatcompiletimeandthegeneralfunctionisinline,sothereisliterallynoperformancepenaltyforchoosingtherightversionofthefunction.Theuseoftagtypesasargumentstodistinguishversionsofthefunctionmayseemredundant,sincewe
alreadyspecifieddifferentconceptsinthetemplates.However,recallthatouruseofconceptsservesonlyasdocumentationfortheprogrammer;currentC++compilersdon’tknowanythingaboutconcepts.Onceconceptsareaddedtothelanguage,thearcaneiteratorcategorytagmechanismwillnolongerbeneeded.
10.6RangesArangeisawayofspecifyingacontiguoussequenceofelements.Rangescanbeeithersemi-openorclosed;11aclosedrange[i,j]includesitemsiandj,whileasemi-openrange[i,j)includesibutendsjustbeforej.Itturnsoutthatsemi-openrangesarethemostconvenientfordefininginterfaces.Thisisbecausealgorithmsthatoperateonsequencesofnelementsneedtobeabletoreferton+1positions.Forexample,therearen+1placestoinsertanewitem:beforethefirstelement,betweenanytwoelements,orafterthelastelement.Also,semi-openranges,unlikeclosedranges,candescribeanemptyrange.Furthermore,asemi-openemptyrangecanbespecifiedatanyposition;itprovidesmoreinformationthanasimple“nil”oremptylist.
11Inmathematics,therearealsoopenranges,buttheyarelessusefulinprogramming,sowedonotincludethemhere.
Arangecanbespecifiedinoneoftwoways:aboundedrangehastwoiterators(onepointingtothebeginningandonepointingjustpasttheend),whileacountedrangehasaniteratorpointingtothebeginningandanintegernindicatinghowmanyitemsareincluded.Thisgivesusfourkindsofrangesaltogether:
(Aclosedcountedrangemusthaven>0.)Asweshallsee,therearedifferentsituationswhereboundedorcountedrangesarepreferable.Whilemathematicaltextsindexsequencesfrom1,computerscientistsstartfrom0,andwewillusethe
latterconventionforourranges.Interestingly,although0-basedindexingincomputersciencewasinitiallyusedasawaytoindicatetheoffsetinmemory,thisconventionturnsouttobemorenaturalregardlessofimplementation,sinceitmeansthatforasequencewithnelements,theindicesareintherange[0,n)andanyiterationisboundedbythelength.
***
Nowwecanreturntothethirdnotablefeatureofourdistancefunctions:thevalid_rangeprecondition.Itwouldbeniceifwecouldhaveavalid_rangefunctionthatreturnedtrueiftherangespecifiedbythetwoiteratorswasvalidandfalseotherwise,butunfortunately,it’snotpossibletoimplementsuchafunction.Forexample,iftwoiteratorseachrepresentcellsinalinkedlist,wehavenowayofknowingifthere’sapathfromonetotheother.Butevenifwe’redealingwithsimplepointers,westillcannotcomputevalid_range:thereisnowayinCorC++todetermineiftwopointerspointtoasinglecontiguousblockofmemory;theremightbegapsinthemiddle.Sowecan’twriteavalid_rangefunction,butwecanstilluseitasaprecondition.Insteadof
guaranteeingthecorrectbehaviorincode,we’lluseaxiomsthat,ifsatisfied,ensurethatourdistancefunctionwillbehaveasintended.Specifically,wepostulatethefollowingtwoaxioms:
Thefirstaxiomsaysthatifit’sacontainer,therangefrombegin()toend()isvalid.Thesecondaxiomsaysthatif[x,y)isanonemptyvalidrange,thentherange[successor(x),y)isalsovalid.AllSTL-stylecontainers,aswellasC++arrays,mustobeytheseaxioms.Thisallowsustoprovethealgorithmscorrect.Forexample,ifyougobacktoouroriginaldistancefunctionforinputiteratorsinSection10.5,you’llseethatthesecondaxiomensuresthatifwestartwithavalidrange,we’llstillhaveoneeachtime
throughtheloop.
***
Inadditiontothesuccessor(++)anddistanceoperations,it’susefultohaveawaytomoveaniteratorbyseveralpositionsatonce.Wecallthisfunctionadvance.Asbefore,we’llimplementtwoversions,oneforinputiterators:Clickheretoviewcodeimage
template<InputIteratorI>
voidadvance(I&x,DifferenceType<I>n,std::input_iterator_tag){
while(n){
--n;
++x;
}
}
andanotherforrandomaccessiterators:Clickheretoviewcodeimage
template<RandomAccessIteratorI>
voidadvance(I&x,DifferenceType<I>n,
std::random_access_iterator_tag){
x+=n;
}
We’llalsoprovidewithatop-levelfunctionfordoingthedispatch:Clickheretoviewcodeimage
template<InputIteratorI>
voidadvance(I&x,DifferenceType<I>n){
advance(x,n,IteratorCategory<I>());
}
10.7LinearSearchLinearsearchisafundamentalprogrammingtaskthatallprogrammersshouldunderstand.Thesimplestideaoflinearsearchistoscanalinearlistuntilwefindaspecificelement.Butwewillgeneralizethattoafunctionthatscansthelistuntilitfindsanelementthatsatisfiesagivenpredicate.Soinadditiontobeingabletosearchforaspecificvalue,wecouldfind,forexample,thefirstoddelement,orthefirstelementthathasnovowels,orwhateverelsewelike.Ofcourse,theremightnotbesuchanelement,soweneedsomewaytoindicatethatnoitemisfound.We’llcallourfunctionfind_if—“finditifit’sthere”:12
12ThisnameoriginatedintheprogramminglanguageCommonLisp.
Clickheretoviewcodeimage
template<InputIteratorI,PredicateP>
Ifind_if(If,Il,Pp){
while(f!=l&&!p(*f))++f;
returnf;
}
Thisfunctionreliesonequality,dereference,andsuccessor.Ifnoitemthatsatisfiesthepredicateexists,thereturnedvalueoffwillbethesameasl,theiteratorthatpointspasttheendoftherange.Thecallingfunctionusesthiscomparisontodeterminewhetheramatchingitemhasbeenfound.CandC++guarantee
thatapointerisvalidonepositionpasttheendofanarray.However,suchapointershouldneverbedereferenced.AllSTLcontainersprovidesimilarguaranteesfortheiriterators.Thisfunctionhasanimplicitsemanticprecondition:thevaluetypeoftheiteratorandtheargumenttype
ofthepredicatemustbethesame;otherwise,thereisnowaytoapplythepredicatetotheitemsintherange.Here’savariationofourlinearsearchfunctionfortheinputiteratorcase.Althoughwecouldhave
overloadedthename,we’vedeliberatelyadded_ntoemphasizethatthisversionusesacountedrange:Clickheretoviewcodeimage
template<InputIteratorI,PredicateP>
std::pair<I,DifferenceType<I>>
find_if_n(If,DifferenceType<I>n,Pp){
while(n&&!p(*f)){++f;--n;}
return{f,n};
}
Whydowereturnapair?Wouldn’titbesufficienttoreturntheiteratorthatpointstothefoundelement,aswedidinthepreviousversion?No.Inthepreviousversion,thecallerhadthe“last”iteratortocompareto;here,itdoesnot.Sothesecondreturnedvaluetellsthecallerwhetherwe’reattheend,inwhichcasetheitemwasnotfoundandthereturnediteratorcannotbedereferenced.Butjustasimportantly,ifwedofindamatchingitem,thisallowsustorestartthesearchwhereweleftoff.Withoutthis,therewouldbenowaytosearcharangeforanythingbutthefirstoccurrenceofadesireditem.Thisillustratesanimportantpoint:justastherecanbebugsincode,sotherecanalsobebugsinthe
interface.We’llseeanexampleofoneinthenextsection.
10.8BinarySearchIfwehaveasortedsequence,wecansearchitmuchmoreefficientlybyusinganotheressentialalgorithm,binarysearch.Thisfunctioniseasytodescribe,buthardtoimplementcorrectlyandevenhardertospecify(designaninterfacefor)correctly.
OriginsofBinarySearchTheideaofbinarysearchoriginatedintheIntermediateValueTheorem(IVT),alsoknownastheBolzano-CauchyTheorem:Iffisacontinuousfunctioninaninterval[a,b]suchthatf(a)<f(b),then∀u [f(a),f(b)]thereisc [a,b]suchthatu=f(c).
Theproofofthetheoremconsistsofdoingbinarysearch.Suppose,asanexample,wehaveacontinuousfunctionsuchthatf(a)is–3andf(b)is5.TheIVTtellsusthatforaparticularpointintheimageofthefunction—let’ssay,0—thereisapointcinthedomainsuchthatf(c)=0.Howcanwefindthatpoint?Wecanstartbyfindingthepointx1thatishalfthedistancebetweenaandb,andcomputingf(x1).Ifitequals0,we’redone;we’vefoundc.Ifit’sgreaterthan0,werepeatwithapointx2that’shalfthedistancefromatox1.Ifit’sless,wetakehalfthedistancefromx1tob.Aswekeeprepeatingtheprocess,weasymptoticallyconvergetoc.SimonStevinhadasimilarideain1594,whenhedevisedaversionoftheIVTforpolynomials.
ButsinceStevinwasinterestedindoingeverythingwithdecimals,heactuallydividedtheintervalintotenthsratherthanhalves,andexaminedallofthemuntilhefoundthetenththatcontainedthedesiredvalue.ItwasLagrangewhofirstdescribedthebinaryapproachforpolynomialsin1795.BolzanoandCauchygeneralizedtheIVTintheearly19thcentury,anditistheirversionusedbymathematicianstoday.
Binarysearchwasfirstdiscussedasaprogrammingtechniquein1946byphysicistandcomputingpioneerJohnMauchly,co-creatoroftheENIAC(thefirstelectronicgeneral-purposecomputer).However,manydetailswereleftunspecified.Thefirst“correct”algorithmforperformingbinarysearchwaspublishedin1960byD.H.Lehmer,amathematicianwhohadworkedontheENIACyearsearlier.However,Lehmer’sversiondidnotusethecorrectinterface,andthiserrorwasrepeatedforseveraldecadesafterward.AnexampleoftheerroneousinterfaceremainstodayintheUNIXbsearch()function.Accordingto
thePOSIX13standard:13POSIXisthesetofstandardsforUNIX-likeoperatingsystems.Linux,forexample,isaPOSIX-compliantOS.
Thebsearch()functionshallreturnapointertoamatchingmemberofthearray,oranullpointerifnomatchisfound.Iftwoormorememberscompareequal,whichmemberisreturnedisunspecified.14
14http://www.unix.com/man-page/POSIX/3posix/bsearch/
Therearetwofundamentalflawswiththisinterface.Thefirstconcernsthereturnofanullpointertoindicatethattheitemisnotfound.Oftenyouaredoingthesearchbecauseyouwanttoinsertanitemifitisn’tthere,attheplaceitwouldhavebeenifitwerethere.Withthisinterface,youhavetostartfromscratchtofindyourinsertposition,thistimeusinglinearsearch!Furthermore,therearemanyapplicationswhereyouactuallywanttofindtheclosestornextvaluetowherethemissingitemwouldbe;theitemyou’researchingformaysimplybeaprefixofthekindsofitemsyouhopetofind.Thesecondflawconcernsthesituationwheretherearemultiplematches.Thematchesmaybekeysof
itemsthatyouneedtoretrieve.Sohowdoyouobtaintheentirerangeofequalitems,ifyoudon’tknowwhichoneyou’reon?Youhavetodolinearsearchbothbackwardandforwardtofindtheendsofthematchingsequence.
***
Therightwaytoimplementbinarysearchbeginswiththeideaofapartitionpoint.Assumethatwehaveasequenceofitems[f,l)thatisarrangedsuchthatacertainpredicateistrueoftheitemsintherange[f,m)andfalsefor[m,l).15Thenthepartitionpointisthepositionm.Formally,ifwehaveafunctionthatcomputesthepartitionpoint,thenitspreconditionis
15Inretrospect,itwouldhavebeenbettertohavethefalseitemsfirst,sincetheBooleanvaluefalsesortsbeforetrue;unfortunately,the“wrong”orderisnowpartoftheC++languagestandard.
(i.e.,theelementsarealreadypartitionedasdescribedearlier).Itspostconditionisthatitreturnsthevaluemfromtheprecondition.Notethatfinthisexpressionreferstothefirstelementintherange,nottoafunction.Foracountedrange,thepartitionpointalgorithmlookslikethis:
Clickheretoviewcodeimage
template<ForwardIteratorI,PredicateP>
Ipartition_point_n(If,DifferenceType<I>n,Pp){
while(n){
Imiddle(f);
DifferenceType<I>half(n>>1);
advance(middle,half);
if(!p(*middle)){
n=half;
}else{
f=++middle;
n=n-(half+1);
}
}
returnf;
}
Thisisanextremelyimportantalgorithm,andit’sworthspendingsometimetomakesureyouunderstandit.Itusesabinary-search-stylestrategyjustliketheIntermediateValueTheorem.Recallthatthegoalistoreturnthefirst“bad”element—thatis,thefirstelementforwhichthepredicateisfalse.Theouterloopcontinuesuntilniszero.Insidetheloop,wepositiontheiteratormiddletoapointhalfwaybetweenfandf+n.Ifourpredicateisfalsefortheelementatthatposition,wesetntohalfofitspreviousvalueandrepeat.Otherwise,weknowthepredicateistrue,sowesetourstartingpointftothenextvalueafterthemiddle,adjustntoreflectthenumberofitemsleft,andrepeat.Whenwearedone,thereturnvalueisthepartitionpoint:thepointafterthelasttruevalue.Noticethatourfunctionusesadvancetomovetheiterator.Sincewedon’tknowtheiteratortype,we
can’tassumethatadditionisallowed.However,ifwehavearandomaccessiterator,theadvancefunctionwewroteearlierwillcallthefastimplementation.(Ifwedon’thavearandomaccessiterator,wemighthavetomakeasmanyasnmoves,butineithercasewe’llneverhavetodomorethanlogncomparisons.)Ifwearegivenaboundedrangeinstead,wesimplycomputethedistanceandinvokethecountedrange
version:Clickheretoviewcodeimage
template<ForwardIteratorI,PredicateP>
Ipartition_point(If,Il,Pp){
returnpartition_point_n(f,distance(f,l),p);
}
Nowlet’sreturntothegeneralbinarysearchproblem.Tosolveit,we’regoingtomakeuseofthefollowinglemma:
Lemma10.1(BinarySearchLemma):Foranysortedrange[i,j)andavaluea(theitemyou’researchingfor),therearetwoiterators,lowerboundblandupperboundbusuchthat
wherevkisthevalueatpositionk.Theseboundsalwaysexist,thoughinthespecialcasewherethereisnomatchingitem,bl=bu.Itmayhelptopicturethedataintherange:
Exercise10.1.ProvetheBinarySearchLemma.
Nowwecanuseourpartitionpointfunctiontoperformbinarysearch.Ifwehavetotalordering,anysortedrangeispartitionedforanyvalueaaccordingtothepredicatex<a.STLactuallyprovidesafewfunctionsthatperformbinarysearch,dependingonourtask.Ifwewanttofindthefirstpositionwheretheitemisfound,weusethelower_boundfunction.UsingthefeaturesinC++11,wecanwritelower_boundlikethis:Clickheretoviewcodeimage
template<ForwardIteratorI>
Ilower_bound(If,Il,ValueType<I>a){
returnpartition_point(f,l,
[=](ValueType<I>x){returnx<a;});
}
Thelastlinedefinesanewanonymousfunction(alsoknownasalambdaexpression16)thatreturnstrueifitsargumentislessthana,thenpassesthatfunctionasthepredicateusedbypartition_point.Thelower_boundfunctionreturnsthepositionoftheitema,orthepositionitwouldbeinifitwerethere.ValueTypeisjustashorthandtypefunctionforaccessingtheappropriateiteratortrait,liketheonewewroteearlierforDifferenceType:Clickheretoviewcodeimage
template<InputIteratorI>
usingValueType=typenamestd::iterator_traits<I>::value_type;
16SeeAppendixCfordetailsonhowtouselambdaexpressions.
Inthecasewherethereturnedpositionisnotl,westillneedtoknowwhetherwefoundtheitem.Todothis,thecallerneedstocheckwhetherthedereferencedreturnvalueisequaltoa.Ifinsteadwewanttofindthelastpositionwheretheitemisfound,weuseupper_bound.Thecodeis
almostthesame,exceptthatwedefinethepredicatetocheckifthevalueislessthanorequaltoa,rather
thanstrictlylessthana:Clickheretoviewcodeimage
template<ForwardIteratorI>
Iupper_bound(If,Il,ValueType<I>a){
returnpartition_point(f,l,
[=](ValueType<I>x){returnx<=a;});
}
Somereadersmaybewonderingwhichfunctionisthe“real”binarysearch.Theansweristhatitdependsonthetask.Ifyouwanttofindthepositionofthefirstmatchingelement,thenlower_boundisthe“real”binarysearch.Ifyouwanttofindthepositionofthelastmatchingelement,thenit’supper_bound.Ifyouwanttoknowtheentirerangeofmatchingelements,STLprovidesathirdfunction,equal_range.Andifallyoucareaboutiswhethertherewasamatch,thereisafunctionbinary_search—butkeepinmindthatallit’sdoingiscallinglower_boundandtestingwhetherthedereferencedreturnvalueisequaltotheitem.Theadditionalfunctionalityofequal_rangeclearlybenefitsfromtheSTLconventionofusing
semi-openranges.Evenwhentheelementisnotpresent,thefunctionreturnsanemptyrangeatthepositionwhereitcouldbeinserted.
10.9ThoughtsontheChapterWebeganthischapterbyseeinghowAristotle’slevelsofabstraction(individual,species,genus)correspondtotheprogrammingnotionsofinstance,type,andconcept.Itisthenotionofconceptthatallowsagenericprogramtoworkinavarietyofsettings.Oneofyourcentralgoalsasaprogrammershouldbetoidentifyexistingconceptsinyourapplication.
Youwilloftendevelopnewalgorithms,occasionallydevelopanewdatastructure,andonlyrarelydefineanewconcept.Inthatraresituation,alotofworkisneededtoensurethatitisatrueconceptandnotjustacollectionofunrelatedrequirements.TorestateOccam’sRazor,oneshouldnotintroducenewconceptswithoutnecessity.Wethenintroducedtheconceptofiterators,andsawtheroletheyplayinsomefundamentalalgorithms.
Byusingcompile-timetypedispatchondifferentkindsofiterators,wecanensurethatthemostefficientimplementationgetsexecutedinagivensituation.Finally,wesawtheimportanceofwritingnotonlycorrectcode,butalsocorrectinterfaces.An
incorrectinterfacecanseverelylimittheutilityofafunction;acorrectinterfaceallowsittobeusedinavarietyofsituationswithoutlossofefficiency.
11.PermutationAlgorithms
Analgorithmmustbeseentobebelieved.DonaldKnuth
Complexcomputerprogramsarebuiltupfromsmallerpiecesthatperformcommonlyusedfundamentaltasks.Inthepreviouschapter,welookedatsometasksinvolvingsearchingfordata.Inthischapter,we’lllookattasksthatinvolveshiftingdataintonewpositions,andshowhowtoimplementtheminagenericway.We’llseehowthesetasksalsoendupusingtwoideasdiscussedearlierinthebook:groupsfromabstractalgebraandthegreatestcommondivisor(GCD)fromnumbertheory.Thetaskswewillfocuson—rotateandreverse—allowustointroducealgorithmsthatdothesametask
differentlydependingontheconceptoftheiteratortowhichtheyapply.Inadditiontoillustratingsomegenericprogrammingtechniques,thesealgorithmsareofgreatpracticalimportance.TherotatealgorithminparticularisprobablythemostusedalgorithminsidetheimplementationofSTLcomponentsfromvectortostable_sort.
11.1PermutationsandTranspositionsOurexplorationoftheGCDalgorithmledustolearnaboutgroupsandotheralgebraicstructures.Usingthisknowledge,we’regoingtostartinvestigatingthemathematicaloperationspermutationandtransposition,whichplayanimportantroleinsomefundamentalalgorithms.
Definition11.1.Apermutationisafunctionfromasequenceofnobjectsontoitself.
Theformalnotationforpermutations1lookslikethis:1Thisisthesamenotationmathematiciansuseformatrices;hopefullythecontextwillmakeitclearwhichinterpretationisintended.
Thefirstrowrepresentstheindexes(positions)ofasequenceofobjects;mathematiciansstartnumberingfrom1.Thesecondrowrepresentswheretheitemsinthosepositionsendupafterapplyingthepermutation.Inthisexample,theitemthatusedtobeatposition1willendupinposition2,theitemthatwasinposition2willendupinposition4,andsoon.Inpractice,permutationsareusuallywrittenusingashorthandformatthatomitsthefirstrow:
(2413)
Inotherwords,atpositioni,youwritewheretheithoriginalelementwillendup.Anexampleofapplyingapermutationis
(2413):{a,b,c,d}={c,a,d,b}
Wecanusethenotionofpermutationtodefineasymmetricgroup:
Definition11.2.ThesetofallpermutationsonnelementsconstitutesagroupcalledthesymmetricgroupSn.
Asymmetricgrouphasthefollowinggroupproperties:
binaryoperation:composition(associative)inverseoperation:inversepermutationidentityelement:identitypermutation
Thisisthefirstexamplewe’veseenwheretheelementsofagrouparethemselvesfunctions,andthegroupoperationisanoperationonfunctions.Ifwehaveapermutationxthatshiftsitemstwopositionstotheright,andanotherpermutationythatshiftsitemsthreepositionstotheright,thenthecompositionx yshiftsitemsfivepositionstotheright.Thisisperhapsthemostimportantgrouptoknowabout,sinceeveryfinitegroupisasubgroupofa
symmetricgroup.ItisknownasCayley’sTheorem.
Exercise11.1.ProveCayley’sTheorem.
Exercise11.2.WhatistheorderofSn?
Nowlet’slookataspecialcaseofpermutation,transposition.
Definition11.3.Atransposition(i,j)isapermutationthatexchangestheithandjthelements(i≠j),leavingtherestinplace.
Thenotationfortranspositionsindicateswhichtwopositionsshouldgetexchanged:
(23):{a,b,c,d}={a,c,b,d}
Inprogramming,wehaveasimplernamefortransposition:wecallitswap.InC++,wemightimplementitlikethis:
template<SemiregularT>
voidswap(T&x,T&y){
Ttmp(x);
x=y;
y=tmp;
}
TheswapoperationrequiresonlythatthetypesofitsargumentssatisfytheconceptSemiregular.2Wecanseethatswaprequirestheabilitytocopy-construct,assign,anddestructitsdata,sincethoseoperationsareusedinthecode.Itdoesnotneedtoexplicitlytestforequality,sowedonotneedthetypestobeRegular.Whenwedesignanalgorithm,we’llwanttoknowwhichconceptsthetypesneedtosatisfy,butwe’llalsowanttomakesurenottoimposeextrarequirementswedon’tneed.
2AdiscussionofC++movesemanticsisbeyondthescopeofthisbook.
***
Thetranspositionlemmademonstrateshowfundamentaltheswapoperationis:
Lemma11.1(TranspositionLemma):Anypermutationisaproductoftranspositions.
Proof.Onetranspositioncanputatleastoneelementintoitsfinaldestination.Therefore,atmostn–1
transpositionswillputallnelementsintotheirfinaldestinations.
Whydoweneedonlyn–1transpositions?Becauseoncen–1itemsareintherightplace,thenthitemmustalsobeintherightplace;there’snoplaceelseforittogo.
Exercise11.3.Provethatifn>2,Snisnotabelian.
Everypermutationdefinesadirectedgraphofnelements.Afterapplyingthepermutationenoughtimes,agivenelementwilleventuallybeputbackinitsoriginalposition,representingacycleinthegraph.Everypermutationcanbedecomposedintocycles.Forexample,considerthepermutation(235614).Theelementinposition4movestoposition6,andtheelementinposition6movestoposition4,soafterapplyingthepermutationtwice,bothofthoseelementswillendupwheretheystarted.Weseeasimilarpatternfortheelementsatpositions1,2,3,and5,althoughthistimeittakesfouroperationsbeforetheygetbacktothebeginning.Wesaythatthepermutation(235614)canbedecomposedintotwocycles,andweshowthemgraphicallylikethis:
Wewritethisdecompositionas(235614)=(1235)(46).Thecyclenotation,usedontheright,maybethoughtofasanextensionofthetranspositionnotation.
Althoughthenotationisambiguous(isthisacycleorapermutation?),itisusuallyclearfromthecontext.Also,permutationsalwayscontainalltheintegersfrom1ton,whilecyclesmightnot.Cyclesaredisjoint.Ifyouareatapositioninacycle,youcangettoallotherpositionsinthatcycle.
Therefore,iftwocyclesshareoneposition,theyshareallpositions;thatis,theyarethesamecycle.Sotheonlywaytohaveseparatecyclesisiftheydon’tshareanypositions.
Definition11.4.Acyclecontainingoneelementiscalledatrivialcycle.
Exercise11.4.Howmanynontrivialcyclescouldapermutationofnelementscontain?
Theorem11.1(NumberofAssignments):Thenumberofassignmentsneededtoperformanarbitrarypermutationinplaceisn–u+v,wherenisthenumberofelements,uisthenumberoftrivialcycles,andvisthenumberofnontrivialcycles.
Proof.Everynontrivialcycleoflengthkrequiresk+1assignments,sinceeachelementneedstobemoved,plusweneedtosavethefirstvaluebeingoverwritten;sinceeverynontrivialcyclerequiresoneextramove,movingallvcyclesrequiresvextramoves.Elementsintrivialcyclesdon’tneedtobemovedatall,andthereareuofthose.Soweneedtomoven–uelements,plusvadditionalmovesforthecycles.
Acommonpermutationthathasexactlyn/2cyclesisreverse.Insomesensethisisthe“hardest”permutationbecauseitrequiresthelargestnumberofassignments.We’lllookatreverseingreaterdetailinthenextchapter.
Exercise11.5.Designanin-place3reversealgorithmforforwarditerators;thatis,thealgorithmshouldworkforsinglylinkedlistswithoutmodifyingthelinks.
3We’lldiscussthenotionofin-placealgorithmsmoreinSection11.6;seeDefinition11.6.
11.2SwappingRangesSometimeswewanttoswapmorethanoneitematatime.Infact,acommonoperationinprogrammingistoswapallthevaluesinonerangeofdatawiththecorrespondingvaluesinanother(possiblyoverlapping)range.Wedoitinaloop,oneswapatatime:Clickheretoviewcodeimage
while(condition)std::swap(*iter0++,*iter1++);
whereiter0anditer1areiteratorspointingtotherespectivevaluesineachrange.RecallfromChapter10thatweprefertousesemi-openranges—thosewheretherangeincludesallelementsfromthefirstbounduptobutnotincludingthesecondbound.Whenweswaptworanges,oftenonlyoneofthemneedstobespecifiedexplicitly.Here,first0and
last0specifytheboundsofthefirstrange,whilefirst1referstothestartofthesecondrange:Clickheretoviewcodeimage
template<ForwardIteratorI0,ForwardIteratorI1>
//ValueType<I0>==ValueType<I1>
I1swap_ranges(I0first0,I0last0,I1first1){
while(first0!=last0)swap(*first0++,*first1++);
returnfirst1;
}
There’snopointinspecifyingtheendofthesecondrange,sincefortheswaptowork,itmustcontain(atleast)thesamenumberofelementsasthefirstrange.Whydowereturnfirst1?Becauseitmightbeusefultothecaller.Forexample,ifthesecondrangeislonger,wemightwanttoknowwheretheunmodifiedpartofthesecondrangebegins.It’sinformationthatthecallerdoesn’thave,anditcostsusalmostnothingtoreturn.Thisisagoodtimetoreviewthelawofusefulreturn,whichweintroducedinSection4.6.
TheLawofUsefulReturn,RevisitedWhenwritingcode,it’softenthecasethatyouendupcomputingavaluethatthecallingfunctiondoesn’tcurrentlyneed.Later,however,thisvaluemaybeimportantwhenthecodeiscalledinadifferentsituation.Inthissituation,youshouldobeythelawofusefulreturn:Aprocedureshouldreturnallthepotentiallyusefulinformationitcomputed.
Thequotient-remainderfunctionthatwesawinChapter4isagoodexample:whenwefirstwroteit,weneededonlytheremainder,butwehadalreadydonealltheworktofindthequotient.Later,wesawotherapplicationsthatusebothvalues.Thelawdoesnotimplydoingunneededextracomputations,nordoesitimplythatuselessinformationshouldbereturned.Forexample,inthecodegivenearlier,it’snotusefultoreturnfirst0,becausethealgorithmguaranteesthatattheendit’sequaltolast0,whichthecalleralreadyhas.It’suselesstogivemeinformationIalreadyhave.
TheLawofSeparatingTypesTheswap_rangescodeillustratesanotherimportantprogrammingprinciple,thelawofseparatingtypes:Donotassumethattwotypesarethesamewhentheymaybedifferent.
Ourfunctionwasdeclaredwithtwoiteratortypes,likethis:Clickheretoviewcodeimage
template<ForwardIteratorI0,ForwardIteratorI1>
//ValueType<I0>==ValueType<I1>
I1swap_ranges(I0first0,I0last0,I1first1);
ratherthanassumingthatthey’rethesametype,likethis:Clickheretoviewcodeimage
template<ForwardIteratorI>
Iswap_ranges(Ifirst0,Ilast0,Ifirst1);
Thefirstwaygivesusmoregeneralityandallowsustousethefunctioninsituationswewouldn’totherwisebeableto,withoutincurringanyadditionalcomputationcost.Forexample,wecouldswaparangeofelementsinalinkedlistwitharangeofelementsinanarrayanditwouldjustwork.However,justbecausetwotypesaredistinctdoesnotmeanthereisnorelationshipbetween
them.Inthecaseofswap_ranges,fortheimplementationtobeabletocallswaponthedata,weneedtoensurethatthevaluetypeofI0isthesameasthevaluetypeofI1.Whilethecompilercannotcheckthisconditiontoday,wecanindicateitwithacommentinthecode.
Insituationswherewe’renotsureifthesecondrangeislongenoughtoperformalltheneededswaps,wecanmakebothrangesexplicitandhaveourwhiletestchecktomakesureneitheriteratorhasrunofftheendoftherange:Clickheretoviewcodeimage
template<ForwardIteratorI0,ForwardIteratorI1>
std::pair<I0,I1>swap_ranges(I0first0,I0last0,
I1first1,I1last1){
while(first0!=last0&&first1!=last1){
swap(*first0++,*first1++);
}
return{first0,first1};
}
Thistimewedoreturnbothfirst0andfirst1,becauseonerangemaybeexhaustedbeforetheother,sowewon’tnecessarilyhavereachedlast0andlast1.Swappingcountedrangesisalmostthesame,exceptthatinsteadofcheckingtoseeifwereachedthe
endoftherange,wecountdownfromnto0:Clickheretoviewcodeimage
template<ForwardIteratorI0,ForwardIteratorI1,IntegerN>
std::pair<I0,I1>swap_ranges_n(I0first0,I1first1,Nn){
while(n!=N(0)){
swap(*first0++,*first1++);
--n;
}
return{first0,first1};
}
TheLawofCompletenessObservethatwecreatedbothswap_rangesandswap_ranges_n.Eventhoughinaparticularsituationtheprogrammermightneedonlyoneoftheseversions,lateronaclientmightneedtheotherversion.Thisfollowsthelawofcompleteness:Whendesigninganinterface,considerprovidingalltherelatedprocedures.
Iftherearedifferentwaysofinvokinganalgorithm,provideinterfacesforthoserelatedfunctions.Inourswapexample,we’vealreadyprovidedtworelatedinterfacesforboundedranges,andwe’llalsoprovideinterfacesforcountedranges.Thisruleisnotsayingthatyouneedtohaveasingleinterfacetohandlemultiplecases.It’s
perfectlyfinetohaveonefunctionforcountedrangesandanotherforboundedranges.Youespeciallyshouldnothaveasingleinterfacefordisparateoperations.Forexample,justbecausecontainersneedtoprovideboth“insertelement”and“eraseelement”functionalitydoesn’tmeanyourinterfaceshouldhaveasingle“insert_or_erase”function.
Countedrangesareeasierforthecompiler,becauseitknowsthenumberofiterations,calledthetripcount,inadvance.Thisallowsthecompilertomakecertainoptimizations,suchasloopunrollingorsoftwarepipelining.
Exercise11.6.Whydon’tweprovidethefollowinginterface?
Clickheretoviewcodeimage
pair<I0,I1>swap_ranges_n(I0first0,I1first1,N0n0,N1n1)
11.3RotationOneofthemostimportantalgorithmsyou’veprobablyneverheardofisrotate.It’safundamentaltoolusedbehindthescenesinmanycommoncomputingapplications,suchasbuffermanipulationintexteditors.Itimplementsthemathematicaloperationrotation.
Definition11.5.Apermutationofnelementsbykwherek≥0:
(kmodn,k+1modn,...,k+n–2modn,k+n–1modn)
iscalledannbykrotation.Ifyouimagineallnelementslaidoutinacircle,we’reshiftingeachone“clockwise”byk.Atfirstglance,itmightseemthatrotationcouldbeimplementedwithamodularshift,takingthe
beginningandendoftherange,togetherwiththeamounttoshift,asarguments.However,doingmodulararithmeticoneveryoperationisquiteexpensive.Also,itturnsoutthatrotationisequivalenttoexchangingdifferentlengthblocks,ataskthatisextremelyusefulformanyapplications.Viewedthisway,itisconvenienttopresentrotationwiththreeiterators:f,m,andl,where[f,m)and[m,l)arevalidranges.4Rotationtheninterchangesranges[f,m)and[m,l).Iftheclientwantstorotatekpositionsinarange[f,l),thenitshouldpassavalueofmequaltol–k.Asanexample,ifwewanttodoarotationwithk=5onasequencespecifiedbytherange[0,7),wechoosem=l–k=7–5=2:
4Thenamesf,m,andlaremeanttobemnemonicsforfirst,middle,andlast.
0123456
fml
whichproducesthisresult:2345601
Essentially,we’removingeachitem5spacestotheright(andwrappingaroundwhenwerunofftheend).
Exercise11.7.Provethatifwedorotate(f,m,l),itperformsdistance(f,l)bydistance(m,l)rotation.
AnimportantrotatealgorithmwasdevelopedbyDavidGries,aprofessoratCornellUniversity,togetherwithIBMscientistHarlanMills:Clickheretoviewcodeimage
template<ForwardIteratorI>
voidgries_mills_rotate(If,Im,Il){
//u=distance(f,m)&&v=distance(m,l)
if(f==m||m==l)return;//u==0||v==0
pair<I,I>p=swap_ranges(f,m,m,l);
while(p.first!=m||p.second!=l){
if(p.first==m){//u<v
f=m;m=p.second;//v=v-u
}else{//v<u
f=p.first;//u=u-v
}
p=swap_ranges(f,m,m,l);
}
return;//u==v
}
Thealgorithmfirstusestheregularswap_rangesfunctiontoswapasmanyelementsaswecan—asmanyelementsasthereareintheshorterrange.Theiftestcheckswhetherwe’veexhaustedthefirstrangeorthesecondrange.Dependingontheresult,weresetthestartpositionsoffandm.Thenwedoanotherswap,andrepeattheprocessuntilneitherrangehasremainingelements.Thisiseasiertofollowwithanexample.Let’slookathowarangeistransformed,andhowthe
iteratorsmove,asthealgorithmruns:Start:
0123456
fml
Swap[0,1]and[2,3].Haveweexhaustedbothranges?No,onlythefirstone,sowesetf=mandm=p.second,whichpointstothefirstelementinthesequencethathasn’tyetbeenmoved:
2301456
fml
Swap[0,1]and[4,5].Haveweexhaustedbothranges?No,onlythefirstone,soagainwesetf=mandm=p.second:
2345016
fml
Swap[0]with[6].Haveweexhaustedbothranges?No,onlythesecondonethistime,sowesetf=p.first:
2345610
fml
Swap[1]with[0].Haveweexhaustedbothranges?Yes,sowe’redone.2345601
fm
l
Nowlookatthecommentsinthegries_mills_rotatecode(showninboldface).Wecalluthelengthofthefirstrange[f,m),andvthelengthofthesecondrange[m,l).Wecanobservesomethingremarkable:theannotationsareperformingourfamiliarsubtractiveGCD!Attheendofthealgorithm,u=v=GCDofthelengthsoftheinitialtworanges.
Exercise11.8.Ifyouexamineswap_ranges,youwillseethatthealgorithmdoesunnecessaryiteratorcomparisons.Rewritethealgorithmsothatnounnecessaryiteratorcomparisonsaredone.
Itturnsoutthatmanyapplicationsbenefitiftherotatealgorithmreturnsanewmiddle:apositionwherethefirstelementmoved.Ifrotatereturnsthisnewmiddle,thenrotate(f,rotate(f,m,l),l)isanidentitypermutation.First,weneedthefollowing“auxiliaryrotate”algorithm:Clickheretoviewcodeimage
template<ForwardIteratorI>
voidrotate_unguarded(If,Im,Il){
//assert(f!=m&&m!=l)
pair<I,I>p=swap_ranges(f,m,m,l);
while(p.first!=m||p.second!=l){
f=p.first;
if(m==f)m=p.second;
p=swap_ranges(f,m,m,l);
}
}
ThecentralloopisthesameasintheGries-Millsalgorithm,justwrittendifferently.(Wecouldhavewrittenitthiswaybeforebutwantedtomaketheuandvcomputationsclearer.)Weneedtofindm′—theelementwhosedistancetothelastisthesameasm’sdistancefromthefirst.
It’sthevaluereturnedbythefirstcalltoswap_ranges.Togetitback,wecanembedacalltorotate_unguardedinourfinalversionofrotate,whichworksaslongaswehaveforwarditerators.Aswe’llexplainshortly,theforward_iterator_tagtypeintheargumentlistwillhelpusinvokethisfunctiononlyinthiscase:Clickheretoviewcodeimage
template<ForwardIteratorI>
Irotate(If,Im,Il,std::forward_iterator_tag){
if(f==m)returnl;
if(m==l)returnf;
pair<I,I>p=swap_ranges(f,m,m,l);
while(p.first!=m||p.second!=l){
if(p.second==l){
rotate_unguarded(p.first,m,l);
returnp.first;
}
f=m;
m=p.second;
p=swap_ranges(f,m,m,l);
}
returnm;
}
Howmuchworkdoesthisalgorithmdo?Untilthelastiterationofthemainloop,everyswapputsoneelementintherightplaceandmovesanotherelementoutoftheway.Butinthefinalcalltoswap_ranges,thetworangesarethesamelength,soeveryswapputsbothelementsitisswappingintotheirfinalpositions.Inessence,wegetanextramoveforfreeoneveryswap.Thetotalnumberofswapsisthetotalnumberofelementsn,minusthenumberoffreeswapswesavedinthelaststep.Howmanyswapsdidwesaveinthelaststep?Thelengthoftherangesattheend,aswesawearlier,isgcd(n–k,k)=gcd(n,k),wheren=distance(f,l)andk=distance(m,l).Sothetotalnumberofswapsisn–gcd(n,k).Also,sinceeachswaptakesthreeassignments(tmp=a;a=b;b=tmp),thetotalnumberofassignmentsis3(n–gcd(n,k)).
11.4UsingCyclesCanwefindafasterrotatealgorithm?Wecanifweexploitthefactthatrotations,likeanypermutations,havecycles.Considerarotationofk=2forn=6elements:
Theiteminposition0endsupinposition2,2endsupinposition4,and4endsupbackinposition0.Thesethreeelementsformacycle.Similarly,item1endsupin3,3in5,and5backin1,forminganothercycle.Sothisrotationhastwocycles.RecallfromSection11.1thatwecanperformanypermutationinn–u+vassignments,wherenisthenumberofelements,uisthenumberoftrivialcycles,andvisthenumberofnontrivialcycles.Sincewenormallydon’thaveanytrivialcycles,weneedn+vassignments.
Exercise11.9.Provethatifarotationofnelementshasatrivialcycle,thenithasntrivialcycles.(Inotherwords,arotationeithermovesallelementsornoelements.)
Itturnsoutthatthenumberofcyclesisgcd(k,n),soweshouldbeabletodotherotationinn+gcd(k,n)assignments,5insteadofthe3(n–gcd(n,k))weneededfortheGries-Millsalgorithm.Furthermore,inpracticeGCDisverysmall;infact,itis1(thatis,thereisonlyonecycle)about60%ofthetime.Soarotatealgorithmthatexploitscyclesalwaysdoesfewerassignments.
5SeeElementsofProgramming,Section10.4,fortheproof.
Thereisonecatch:Gries-Millsonlyrequiredmovingonestepforward;itworksevenforsinglylinkedlists.Butifwewanttotakeadvantageofcycles,weneedtobeabletodolongjumps.Suchanalgorithmrequiresstrongerrequirementsontheiterators—namely,theabilitytodorandomaccess.Tocreateournewrotatefunction,we’llfirstwriteahelperfunctionthatmoveseveryelementina
cycletoitsnextposition.Butinsteadofsaying“whichpositiondoestheiteminpositionxmoveto,”we’llsay“inwhichpositiondowefindtheitemthat’sgoingtomovetopositionx.”Eventhoughthesetwooperationsaresymmetricmathematically,itturnsoutthatthelatterismoreefficient,sinceitneedsonlyonesavedtemporaryvariablepercycle,insteadofoneforeveryitemthatneedstobemoved(exceptthelast).Here’sourhelperfunction:
Clickheretoviewcodeimage
template<ForwardIteratorI,TransformationF>
voidrotate_cycle_from(Ii,Ffrom){
ValueType<I>tmp=*i;
Istart=i;
for(Ij=from(i);j!=start;j=from(j)){
*i=*j;
i=j;
}
*i=tmp;
}
Notethatwe’reusingtheValueTypetypefunctionwedefinedneartheendofSection10.8.Howdoesrotate_cycle_fromknowwhichpositionanitemcomesfrom?Thatinformationwill
beencapsulatedinafunctionobjectfromthatwepassinasanargument.Youcanthinkoffrom(i)as“computewheretheelementmovingintopositionicomesfrom.”Thefunctionobjectwe’regoingtopasstorotate_cycle_fromwillbeaninstanceof
rotate_transform:Clickheretoviewcodeimage
template<RandomAccessIteratorI>
structrotate_transform{
DifferenceType<I>plus;
DifferenceType<I>minus;
Im1;
rotate_transform(If,Im,Il):
plus(m-f),minus(m-l),m1(f+(l-m)){}
//m1separatesitemsmovingforwardandbackward
Ioperator()(Ii)const{
returni+((i<m1)?plus:minus);
}
};
Theideaisthateventhoughweareconceptually“rotating”elements,inpracticesomeitemsmoveforwardandsomemovebackward(becausetherotationcausedthemtowraparoundtheendofourrange).Whenrotate_transformisinstantiatedforagivensetofranges,itprecomputes(1)howmuchtomoveforwardforitemsthatshouldmoveforward,(2)howmuchtomovebackwardforthingsthatmovebackward,and(3)whatthecrossoverpointisfordecidingwhentomoveforwardandwhentomovebackward.Nowwecanwritethecycle-exploitingversionofalgorithmforrotation,whichisavariationofthe
algorithmdiscoveredbyFletcherandSilverin1965:Clickheretoviewcodeimage
template<RandomAccessIteratorI>
Irotate(If,Im,Il,std::random_access_iterator_tag){
if(f==m)returnl;
if(m==l)returnf;
DifferenceType<I>cycles=gcd(m-f,l-m);
rotate_transform<I>rotator(f,m,l);
while(cycles-->0)rotate_cycle_from(f+cycles,rotator);
returnrotator.m1;
}
Afterhandlingsometrivialboundarycases,thealgorithmfirstcomputesthenumberofcycles(theGCD)andconstructsarotate_transformobject.Thenitcallsrotate_cycle_fromtoshiftalltheelementsalongeachcycle,andrepeatsthisforeverycycle.Let’slookatanexample.Considertherotationk=2forn=6elementsthatweusedatthebeginningof
thissection.Forsimplicity,we’llassumethatourvaluesareintegersstoredinanarray:012345
Wealsoassumeouriteratorsareintegeroffsetsinanarray,startingat0.(Becarefultodistinguishbetweenthevaluesatapositionandthepositionitself.)Toperformak=2rotation,we’llneedtopassthethreeiteratorsf=0,m=4,andl=6:
012345
fml
Theboundarycasesofournewrotatealgorithmdon’tapply,sothefirstthingitdoesiscomputethenumberofcycles,whichisequaltogcd(m–f,l–m)=gcd(4,2)=2.Thenitconstructstherotatorobject,initializingitsstatevariablesasfollows:
plus←m–f=4–0=4minus←m–l=4–6=–2m1←f+(l–m)=0+(6–4)=2
Themainloopofthefunctionrotatesallelementsofacycle,thenmovesontothenextcycle.Letusseewhathappenswhenrotate_cycle_fromiscalled.Initially,wepassf+d=0+2=2asthefirstargument.Soinsidethefunction,i=2.Wesavethevalue
atposition2,whichisalso2,toourtmpvariableandsetstarttoourstartingpositionof2.Nowwegothroughaloopaslongasanewvariable,j,isnotequaltostart.Eachtimethroughthe
loop,wearegoingtosetjbyusingtherotatorfunctionobjectthatwepassedinthroughthevariablefrom.Basically,allthatobjectdoesisaddthestoredvaluesplusorminustoitsargument,dependingonwhethertheargumentislessthanthestoredvaluem1.Forexample,ifwecallfrom(0),itwillreturn0+4,or4,since0islessthan2.Ifwecallfrom(4),itwillreturn4+(–2),or2,since4isnotlessthan2.
Here’showthevaluesinourarraychangeaswegothroughtheloopinrotate_cycle_from:
i←2,j←from(2)=0
012345
ji
*i←*j
010345
ji
i←j=0,j←from(0)=4
010345
ij
*i←*j
410345
ij
i←j=4,j←from(4)=2whichisstart,soloopends
410345
ji
*i←tmp
410325
ji
Thiscompletesthefirstcalltorotate_cycle_frominthewhileloopofourrotatefunction.
Exercise11.10.Continuetotracetheprecedingexampleuntiltherotatefunctionfinishes.
Noticethatthesignaturesofthisrotatefunctionandthepreviousonedifferbythetypeofthelastargument.Inthenextsection,we’llwritethewrapperthatletsthefastestimplementationforagivensituationbeautomaticallyinvoked.
WhenIsanAlgorithmFasterinPractice?Wehaveseenanexamplewhereonealgorithmdoesfewerassignmentsthananotheralgorithm.Doesthatmeanitwillrunfaster?Notnecessarily.Inpractice,theabilitytofitrelevantdataincachecanmakeadramaticdifferenceinthisspeed.Analgorithmthatinvolveslargejumpsinmemory—thatis,onethathaspoorlocalityofreference—mayendupbeingslowerthanonethatrequiresmoreassignmentsbuthasbetterlocalityofreference.
11.5ReverseAnotherfundamentalalgorithmisreverse,which(obviously)reversestheorderoftheelementsofasequence.Moreformally,reversepermutesak-elementlistsuchthatitem0anditemk−1areswapped,item1anditemk−2areswapped,andsoon.
Ifwehavereverse,wecanimplementrotateinjustthreelinesofcode:Clickheretoviewcodeimage
template<BidirectionalIteratorI>
voidthree_reverse_rotate(If,Im,Il){
reverse(f,m);
reverse(m,l);
reverse(f,l);
}
Forexample,supposewewanttoperformourk=2rotationonthesequence012345.Thealgorithmwouldperformthefollowingoperations:Clickheretoviewcodeimage
fml
start012345
reverse(f,m)321045
reverse(m,l)321054
reverse(f,l)450123
Exercise11.11.Howmanyassignmentsdoes3-reverserotateperform?
Thiselegantalgorithm,whoseinventorisunknown,worksforbidirectionaliterators.However,ithasoneproblem:itdoesn’treturnthenewmiddleposition.Tosolvethis,we’regoingtobreakthefinalreversecallintotwoparts.We’llneedanewfunctionthatreverseselementsuntiloneofthetwoiteratorsreachestheend:Clickheretoviewcodeimage
template<BidirectionalIteratorI>
pair<I,I>reverse_until(If,Im,Il){
while(f!=m&&m!=l)swap(*f++,*--l);
return{f,l};
}
Attheendofthisfunction,theiteratorthatdidn’thittheendwillbepointingtothenewmiddle.Nowwecanwriteageneralrotatefunctionforbidirectionaliterators.Whenitgetstowhatwouldhave
beenthethirdreversecall,itdoesreverse_untilinstead,savesthenewmiddleposition,andthenfinishesreversingtherestoftherange:Clickheretoviewcodeimage
template<BidirectionalIteratorI>
Irotate(If,Im,Il,bidirectional_iterator_tag){
reverse(f,m);
reverse(m,l);
pair<I,I>p=reverse_until(f,m,l);
reverse(p.first,p.second);
if(m==p.first)returnp.second;
returnp.first;
}
Wehaveseenthreedifferentimplementationsofrotate,eachoptimizedfordifferenttypesofiterators.However,we’dliketohidethiscomplexityfromtheprogrammerwho’sgoingtobeusingthesefunctions.So,justaswedidwiththedistancefunctionsinSection10.5,we’regoingtowriteasimplerversionthatworksforanytypeofiterator,andusecategorydispatchtoletthecompilerdecidewhichimplementationwillgetexecuted:Clickheretoviewcodeimage
template<ForwardIteratorI>
Irotate(If,Im,Il){
returnrotate(f,m,l,IteratorCategory<I>());
}
Theprogrammerjustneedstocallasinglerotatefunction;thecompilerwillextractthetypeoftheiteratorbeingusedandinvoketheappropriateimplementation.
***
We’vebeenusingareversefunction,buthowmightweimplementit?Forbidirectionaliterators,thecodeisfairlystraightforward;wehaveapointertothebeginningthatmovesforward,andapointertotheendthatmovesbackward,andwekeepswappingtheelementstheypointtountiltheyrunintoeachother:Clickheretoviewcodeimage
template<BidirectionalIteratorI>
voidreverse(If,Il,std::bidirectional_iterator_tag){
while(f!=l&&f!=--l)std::swap(*f++,*l);
}
Exercise11.12.Explainwhyweneedtwotestsperiterationintheprecedingwhileloop.
Itmightappearthataccordingtothelawofusefulreturnweshouldreturnpair<I,I>(f,l).However,thereisnoevidencethatthisinformationisactuallyuseful;thereforethelawdoesnotapply.Ofcourse,ifwealreadyknewinadvancehowmanytimeswehadtoexecutetheloop(thetripcount),
wewouldn’tneedtwocomparisons.Ifwepassthetripcountntoourfunction,wecanimplementitwithonlyn/2tests:Clickheretoviewcodeimage
template<BidirectionalIteratorI,IntegerN>
voidreverse_n(If,Il,Nn){
n>>=1;
while(n-->N(0)){
swap(*f++,*--l);
}
}
Inparticular,ifwehavearandomaccessiterator,wecancomputethetripcountinconstanttime,andimplementreverseusingreverse_n,likethis:Clickheretoviewcodeimage
template<RandomAccessIteratorI>
voidreverse(If,Il,std::random_access_iterator_tag){
reverse_n(f,l,l-f);
}
Whatifwehaveonlyforwarditeratorsandwestillwanttoreverse?We’llusearecursiveauxiliaryfunctionthatkeepspartitioningtherangeinhalf(thehinthecode).Theargumentnkeepstrackofthelengthofthesequencebeingreversed:Clickheretoviewcodeimage
template<ForwardIteratorI,IntegerN>
Ireverse_recursive(If,Nn){
if(n==0)returnf;
if(n==1)return++f;
Nh=n>>1;
Im=reverse_recursive(f,h);
if(odd(n))++m;
Il=reverse_recursive(m,h);
swap_ranges_n(f,m,h);
returnl;
}
Exercise11.13.Usingthesequence{0,1,2,3,4,5,6,7,8}asanexample,tracetheoperationofthereverse_recursivealgorithm.
Thefunctionreturnstheendoftherange,sothefirstrecursivecallreturnsthemidpoint.Thenweadvancethemidpointby1or0dependingonwhetherthelengthisevenorodd.Nowwecanwriteourreversefunctionforforwarditerators:
Clickheretoviewcodeimage
template<ForwardIteratorI>
voidreverse(If,Il,std::forward_iterator_tag){
reverse_recursive(f,distance(f,l));
}
Finally,wecanwritethegenericversionofreversethatworksforanyiteratortype,justaswedidforrotateearlier:Clickheretoviewcodeimage
template<ForwardIteratorI>
voidreverse(If,Il){
reverse(f,l,IteratorCategory<I>());
}
TheLawofInterfaceRefinementWhatisthecorrectinterfaceforrotate?Originally,std::rotatereturnedvoid.Afterseveralyearsofusage,itbecameclearthatreturningthenewmiddle(thepositionwherethefirstelementmoved)madeimplementationofseveralotherSTLalgorithms,suchasin_place_mergeandstable_partition,simpler.Unfortunately,itwasnotimmediatelyobvioushowtoreturnthisvaluewithoutdoinganyextra
work.Onlyafterthisimplementationproblemwassolvedwasitpossibletoredesigntheinterfacetoreturntherequiredvalue.Itthentookmorethan10yearstochangetheC++languagestandard.Thisisagoodillustrationofthelawofinterfacerefinement:
Designinginterfaces,likedesigningprograms,isamulti-passactivity.
Wecan’treallydesignanidealinterfaceuntilwehaveseenhowthealgorithmwillbeused,andnotalltheusesareimmediatelyapparent.Furthermore,wecan’tdesignanidealinterfaceuntilweknowwhichimplementationsarefeasible.
11.6SpaceComplexityWhentalkingaboutconcretealgorithms,programmersneedtothinkaboutwheretheyfallintermsoftimeandspacecomplexity.Therearemanylevelsoftimecomplexity(e.g.,constant,logarithmic,quadratic).However,thetraditionalviewofspacecomplexityputalgorithmsintojusttwocategories:thosethatperformtheircomputationsinplaceandthosethatdonot.
Definition11.6.Analgorithmisin-place(alsocalledpolylogspace)ifforaninputoflengthnitusesO((logn)k)additionalspace,wherekisaconstant.
Initially,in-placealgorithmswereoftendefinedasusingconstantspace,butthiswastoonarrowarestriction.Theideaofbeing“inplace”wassupposedtocapturealgorithmsthatdidn’tneedtomakeacopyoftheirdata.Butmanyofthesenon-data-copyingalgorithms,likequicksort,useadivide-and-conquertechniquethatrequireslogarithmicextraspace.Sothedefinitionwasformalizedinawaythatincludedthesealgorithms.Algorithmsthatarenotin-placeusemorespace—usually,enoughtocreateacopyoftheirdata.
***
Let’suseourreverseproblemtoseehowanon-in-placealgorithmcanbefasterthananin-placealgorithm.First,weneedthishelperfunction,whichcopieselementsinreverseorder,startingattheendofarange:Clickheretoviewcodeimage
template<BidirectionalIteratorI,OutputIteratorO>
Oreverse_copy(If,Il,Oresult){
while(f!=l)*result++=*--l;
returnresult;
}
Nowwecanwriteanon-in-placereversealgorithm.Itcopiesallthedatatoabuffer,thencopiesitbackinreverseorder:Clickheretoviewcodeimage
template<ForwardIteratorI,IntegerN,BidirectionalIteratorB>
Ireverse_n_with_buffer(If,Nn,Bbuffer){
Bbuffer_end=copy_n(f,n,buffer);
returnreverse_copy(buffer,buffer_end,f);
}
Thisfunctiontakesonly2nassignments,insteadofthe3nweneededfortheswap-basedimplementations.
11.7Memory-AdaptiveAlgorithmsInpractice,thedichotomyofin-placeandnon-in-placealgorithmsisnotveryuseful.Whiletheassumptionofunlimitedmemoryisnotrealistic,neitheristheassumptionofonlypolylogextramemory.Usually25%,10%,5%,oratleast1%ofextramemoryisavailableandcanbeexploitedtogetsignificantperformancegains.Algorithmsneedtoadapttohowevermuchmemoryisavailable;theyneedtobememoryadaptive.Let’screateamemory-adaptivealgorithmforreverse.Ittakesabufferthatwecanuseas
temporaryspace,andanargumentbufsizeindicatinghowbigthebufferis.Thealgorithmisrecursive—infact,it’salmostidenticaltothereverse_recursivefunctionintheprevioussection.Buttherecursionhappensonlyonlargechunks,sotheoverheadisacceptable.Theideaisthatforagiveninvocationofthefunction,ifthelengthofthesequencebeingreversedfitsinthebuffer,wedothefastreversewithbuffer.Ifnot,werecurse,splittingthesequenceinhalf:Clickheretoviewcodeimage
template<ForwardIteratorI,IntegerN,BidirectionalIteratorB>
Ireverse_n_adaptive(If,Nn,Bbuffer,Nbufsize){
if(n==N(0))returnf;
if(n==N(1))return++f;
if(n<=bufsize)returnreverse_n_with_buffer(f,n,buffer);
Nh=n>>1;
Im=reverse_n_adaptive(f,h,buffer,bufsize);
advance(m,n&1);
Il=reverse_n_adaptive(m,h,buffer,bufsize);
swap_ranges_n(f,m,h);
returnl;
}
Thecallerofthisfunctionshouldaskthesystemhowmuchmemoryisavailable,andpassthatvalueasbufsize.Unfortunately,suchacallisnotprovidedinmostoperatingsystems.
ASadStoryaboutget_temporary_bufferWhentheC++STLlibrarywasdesignedbythefirstauthorofthisbook,herealizedthatitwouldbehelpfultohaveafunctionget_temporary_bufferthattakesasizenandreturnsthelargestavailabletemporarybufferuptosizenthatfitsintophysicalmemory.Asaplaceholder(sincethecorrectversionneedstohaveknowledgethatonlytheoperatingsystemhas),hewroteasimplisticandimpracticalimplementation,whichrepeatedlycallsmallocaskingforinitiallyhugeandgraduallysmallerchunksofmemoryuntilitreturnsavalidpointer.Heputinaprominentcommentinthecodesayingsomethinglike,“Thisisabogusimplementation,replaceit!”Tohissurprise,hediscoveredyearslaterthatallthemajorvendorsthatprovideSTLimplementationsarestillusingthisterribleimplementation—buttheyremovedhiscomment.
11.8ThoughtsontheChapterOneofthethingswe’veseeninboththischapterandthepreviousoneishowsimplecomputationaltasksofferrichopportunitiestoexploredifferentalgorithmsandlearnfromthem.Theprogrammingprinciplesthatarisefromtheseexamples—thelawsofusefulreturn,separatingtypes,completeness,andinterfacerefinement—carryoverintonearlyeveryprogrammingsituation.Thischapterhasalsopresentedsomegoodexamplesofhowtheoryandpracticecometogetherin
programming.Ourknowledgeofthetheoryofpermutations—itselfbasedongrouptheory—allowedustocomeupwithamoreefficientalgorithmforrotation,onethatexploitedthepropertiesthatourtheoryguaranteed.Atthesametime,theexampleofmemory-adaptivealgorithmsdemonstratedhowpracticalconsiderationssuchastheamountofavailablememorycanhaveaprofoundimpactonthechoiceofalgorithmanditsperformance.Theoryandpracticearetwosidesofthesamecoin;goodprogrammersrelyonknowledgeofboth.
12.ExtensionsofGCD
Iswearbytheevenandtheodd.Quran,SurahAl-Fajr
Programmersoftenassumethatsinceadatastructureoralgorithmisinatextbookorhasbeenusedforyears,itrepresentsthebestsolutiontoaproblem.Surprisingly,thisisoftennotthecase—evenifthealgorithmhasbeenusedforthousandsofyearsandhasbeenworkedonbyeveryonefromEuclidtoGauss.Inthischapterwe’lllookatanexampleofanovelsolutiontoanoldproblem—computingtheGCD.Thenwe’llseehowprovingatheoremfromnumbertheoryresultedinanimportantvariationofthealgorithmstillusedtoday.
12.1HardwareConstraintsandaMoreEfficientAlgorithmIn1961,anIsraeliPh.D.student,Josef“Yossi”Stein,wasworkingonsomethingcalledRacahAlgebraforhisdissertation.Heneededtodorationalarithmetic,whichrequiredreducingfractions,whichusestheGCD.Butbecausehehadonlylimitedtimeonaslowcomputer,hewasmotivatedtofindabetterway.Asheexplains:
Using“RacahAlgebra”meantdoingcalculationswithnumbersoftheforma/b. ,where,a,b,cwereintegers.IwroteaprogramfortheonlyavailablecomputerinIsraelatthattime—theWEIZACattheWeizmanninstitute.Additiontimewas57microseconds,divisiontookabout900microseconds.Shifttooklessthanaddition....Wehadneithercompilernorassembler,andnofloating-pointnumbers,butusedhexadecimalcodefortheprogramming,andhadonly2hoursofcomputer-timeperweekforRacahandhisstudents,andyouseethatIhadtherightconditionsforfindingthatalgorithm.FastGCDmeantsurvival.1
1J.Stein,personalcommunication,2003.
WhatSteinobservedwasthattherewerecertainsituationswheretheGCDcouldbeeasilycomputed,oreasilyexpressedintermsofanotherGCDexpression.HelookedatspecialcasesliketakingtheGCDofanevennumberandanoddnumber,oranumberanditself.Eventually,hecameupwiththefollowingexhaustivelistofcases:
Usingtheseobservations,Steinwrotethefollowingalgorithm:Clickheretoviewcodeimage
template<BinaryIntegerN>
Nstein_gcd(Nm,Nn){
if(m<N(0))m=-m;
if(n<N(0))n=-n;
if(m==N(0))returnn;
if(n==N(0))returnm;
//m>0&&n>0
intd_m=0;
while(even(m)){m>>=1;++d_m;}
intd_n=0;
while(even(n)){n>>=1;++d_n;}
//odd(m)&&odd(n)
while(m!=n){
if(n>m)swap(n,m);
m-=n;
dom>>=1;while(even(m));
}
//m==n
returnm<<min(d_m,d_n);
}
Let’slookatwhatthecodeisdoing.ThefunctiontakestwoBinaryIntegers—thatis,anintegerrepresentationthatsupportsfastshiftandeven/oddtesting,liketypicalcomputerintegers.First,iteliminatestheeasycaseswhereoneoftheargumentsiszero,andinvertsthesignifanargumentisnegative,sothatwearedealingwithtwopositiveintegers.Next,ittakesadvantageoftheidentitieswithevenarguments,removingfactorsof2(byshifting)while
keepingtrackofhowmanytherewere.Wecanuseasimpleintforthecounts,sincewhatwe’recountingisatmostthetotalnumberofbitsintheoriginalarguments.Afterthispart,weareoperatingontwooddnumbers.Nowcomesthemainloop.Werepeatedlysubtractthesmallerfromthelargereachtime(sincewe
knowthedifferenceoftwooddnumbersiseven),andagainuseshiftstoremoveadditionalpowersof2fromtheresult.2Whenwe’redone,ourtwonumberswillbeequal.Sincewe’rehalvingatleastonceeachtimethroughtheloop,weknowwe’lliteratenomorethanlogntimes;thealgorithmisboundedbythenumberof1-bitsweencounter.
2Weusedo-whileratherthanwhilebecausewedon’tneedtorunthethetestthefirsttime;weknowwe’restartingwithanevennumbersoweknowwehavetodoatleastoneshift.
Finally,wereturnourresult,usingashifttomultiplyournumberby2foreachoftheminimumnumberof2swefactoredoutatthebeginning.Wedon’tneedtoworryabout2sinthemainloop,becausebythatpointwe’vereducedtheproblemtotheGCDoftwooddnumbers;thisGCDdoesnothave2asafactor.Here’sanexampleofthealgorithminoperation.SupposewewanttocomputeGCD(196,42).The
computationlookslikethis:
Aswesaw,Steintooksomeobservationsaboutspecialcasesandturnedthemintoafasteralgorithm.Thespecialcaseshadtodowithevenandoddnumbers,andplaceswherewecouldfactorout2,whichiseasyonacomputer;that’swhyStein’salgorithmisfasterinpractice.(Eventoday,whentheremainderfunctioncanbecomputedinhardware,itisstillmuchslowerthansimpleshifts.)Butisthisjustacleverhack,oristheremoreherethanmeetstheeye?Doesitmakesenseonlybecausecomputersusebinaryarithmetic?DoesStein’salgorithmworkjustforintegers,orcanwegeneralizeitjustaswedidwithEuclid’salgorithm?
12.2GeneralizingStein’sAlgorithmToanswerthesequestions,let’sreviewsomeofthehistoricalmilestonesforEuclid’sGCD:
•Positiveintegers:Greeks(5thcenturyBC)•Polynomials:Stevin(ca.1600)•Gaussianintegers:Gauss(ca.1830)•Algebraicintegers:Dirichlet,Dedekind(ca.1860)•Genericversion:Noether,vanderWaerden(ca.1930)
Ittookmorethan2000yearstoextendEuclid’salgorithmfromintegerstopolynomials.Fortunately,ittookmuchlesstimeforStein’salgorithm.Infact,just2yearsafteritspublication,Knuthalreadyknewofaversionforsingle-variablepolynomialsoverafield .Thesurprisinginsightwasthatwecanhavexplaytheroleforpolynomialsthat2playsforintegers.
Thatis,wecanfactoroutpowersofx,andsoon.Carryingtheanalogyfurther,weseethatx2+x(oranythingelsedivisiblebyx)is“even,”x2+x+1(oranythingelsewithazero-ordercoefficient)is“odd,”andx2+x“shifts”tox+1.Justasdivisionby2iseasierthangeneraldivisionforbinaryintegers,sodivisionbyxiseasierthangeneraldivisionforpolynomials—inbothcases,allweneedisashift.(Rememberthatapolynomialisreallyasequenceofcoefficients,sodivisionbyxisliterallyashiftof
thesequence.)Stein’s“specialcases”forpolynomialslooklikethis:
Noticehoweachofthelasttworulescancelsoneofthezero-ordercoefficients,soweconvertthe“odd,odd”casetoan“even,odd”case.TogettheequivalenceexpressedbyEquation12.6,werelyontwofacts.First,ifyouhavetwo
polynomialsuandv,thengcd(u,v)=gcd(u,av),whereaisanonzerocoefficient.Sowecanmultiplythesecondargumentbythecoefficient ,andwe’llhavethesameGCD:
Second,gcd(u,v)=gcd(u,v-u),whichwenotedwhenweintroducedGCDearlyinthebook(Equation3.9).Sowecansubtractournewsecondargumentfromthefirst,andwe’llstillhavethesameGCD:
Finally,wecanusethefactthatifoneoftheGCDargumentsisdivisiblebyxandtheotherisnot,wecandropxbecausetheGCDwillnotcontainitasafactor.Sowe“shift”outthex,whichgives
whichiswhatwewanted.Wealsoseethatineachtransformation,thenorm—inthiscase,thedegreeofthepolynomial—gets
reduced.Here’showthealgorithmwouldcomputegcd(x3–3x–2,x2–4):
Firstweseethattheratiooftheirfreecoefficients(thecanddinEquations12.6and12.7)is1/2,sowewillmultiplynby1/2andsubtractitfromm(showninthefirstlineoftheprecedingtable),resultinginthenewmshownonthesecondline.Thenwe“shift”mbyfactoringoutx,resultinginthethirdline,and
soon.In2000,AndreWeilertfurthergeneralizedStein’salgorithmtoGaussianintegers.Thistime,1+iplays
theroleof2;the“shift”operationisdivisionby1+i.In2003,DamgårdandFrandsenextendedthealgorithmtoEisensteinintegers.In2004,AgarwalandFrandsendemonstratedthatthereisaringthatisnotaEuclideandomain,but
wheretheSteinalgorithmstillworks.Inotherwords,therearecaseswhereStein’salgorithmworksbutEuclid’sdoesnot.IfthedomainoftheSteinalgorithmisnottheEuclideandomain,thenwhatisit?Asofthiswriting,thisisanunsolvedproblem.WhatwedoknowisthatStein’salgorithmdependsonthenotionofevenandodd;wegeneralizeeven
tobedivisiblebyasmallestprime,wherepisasmallestprimeifanyremainderwhendividingbyitiseitherzerooraninvertibleelement.(Wesay“asmallestprime”ratherthan“thesmallestprime”becausetherecouldbemultiplesmallestprimesinaring.Forexample,forGaussianintegers,1+i,1–i,–1+i,and–1–iareallsmallestprimes.)Whydowefactorout2whenwe’recomputingtheGCDofintegers?Becausewhenwerepeatedly
divideby2,weeventuallyget1asaremainder;thatis,wehaveanoddnumber.Oncewehavetwooddnumbers(twonumberswhoseremaindersmodulo2arebothunits),wecanusesubtractiontokeepourGCDalgorithmgoing.Thisabilitytocancelremaindersworksbecause2isthesmallestintegerprime.Similarly,xisthesmallestprimeforpolynomials,andi+1forGaussianintegers.3Divisionbythesmallestprimealwaysgivesaremainderofzerooraunit,becauseaunitisthenumberwiththesmallestnonzeronorm.So2worksforintegersbecauseit’sthesmallestprime,notbecausecomputersusebinaryarithmetic.Thealgorithmispossiblebecauseoffundamentalpropertiesofintegers,notbecauseofthehardwareimplementation,althoughthealgorithmisefficientbecausecomputersusebinaryarithmetic,makingshiftsfast.
3Notethat2isnotprimeintheringofGaussianintegers,sinceitcanbefactoredinto(1+i)(1-i).
Exercise12.1.ComparetheperformanceoftheSteinandEuclidalgorithmsonrandomintegersfromtheranges[0,216),[0,232),and[0,264).
12.3Bézout’sIdentityTounderstandtherelationshipofGCDandringstructures,weneedtointroduceBézout’sidentity,whichalsoleadstoanimportantpracticalalgorithmforcomputingthemultiplicativeinverse.TheidentitysaysthatforanytwovaluesaandbintheEuclideandomain,thereexistcoefficientssuchthatthelinearcombinationgivestheGCDoftheoriginalvalues.
Theorem12.1(Bézout’sIdentity):
Forexample,ifa=196andb=42,thenthissaystherearevaluesxandysuchthat196x+42y=gcd(196,42).Sincegcd(196,42)=14,inthiscasex=–1andy=5.We’llseelaterinthechapterhowtocomputexandyingeneral.Likemanyresultsinmathematics,thisoneisnamedaftersomeoneotherthanitsdiscoverer.Although
18th-centuryFrenchmathematicianÉtienneBézoutdidprovetheresultforpolynomials,itwasactuallyshownfirstforintegersahundredyearsearlier,byClaudeBachet.
ClaudeGasparBachetdeMéziriac(1581–1638)
ClaudeGasparBachetdeMéziriac,generallyknownasBachet,wasaFrenchmathematicianduringtheRenaissance.Althoughhewasascholarinmanyfields,heisbestknownfortwothings.First,hetranslatedDiophantus’ArithmeticfromGreektoLatin,thecommonlanguageofscienceandphilosophyinEuropeatthetime.Itishis1621translationthatmostmathematiciansreliedon,anditwasinacopyofhistranslationthatFermatfamouslywrotethemarginalnotedescribinghislasttheorem.Second,Bachetwrotethefirstbookonrecreationalmathematics,ProblèmesPlaisants,originallypublishedin1612.Throughthisbook,mathematicsbecameapopulartopicamongeducatedpeopleinFrance,somethingtheywoulddiscussandspendtimeonasahobby.ProblèmesPlaisantsintroducedmagicsquares,aswellasprovingwhatisnowknownasBézout’sidentity.BachetwaschosenasoneoftheoriginalmembersofAcadémieFrançaise(FrenchAcademy),
theorganizationcreatedbyCardinalRichelieutobetheultimateauthorityontheFrenchlanguage,andtaskedwiththecreationandmaintenanceoftheofficialFrenchdictionary.
Recallthataringisanalgebraicstructurethatbehavessimilartointegers;ithasbothplus-likeandtimes-likeoperations,butonlyanadditiveinverse.(YoucanreviewthedefinitionofaringinDefinition8.3inSection8.4.)ToproveBézout’sidentity,weneedtoshowthatthecoefficientsxandyalwaysexist.Todothis,we
needtointroduceanewalgebraicstructure,theideal.
Definition12.1.AnidealIisanonemptysubsetofaringRsuchthat
1.∀x,y I:x+y I2.∀x I,∀a R:ax I
Thefirstpropertysaysthattheidealisclosedunderaddition;inotherwords,ifyouaddanytwoelementsoftheideal,theresultisintheideal.Thesecondpropertyisabitmoresubtle;itsaysthattheidealisclosedundermultiplicationwithanyelementofthering,notnecessarilyanelementoftheideal.Anexampleofanidealisthesetofevennumbers,whichareanonemptysubsetoftheringofintegers.
Ifyouaddtwoevennumbers,yougetanevennumber.Ifyoumultiplyanevennumberbyanynumber(not
necessarilyeven),youstillgetanevennumber.Otherexamplesofidealsareunivariatepolynomialswithroot5,andpolynomialswithxandyandfreecoefficient0(e.g.,x2+3y2+xy+x);we’llseeshortlywhythislastcaseisimportant.Notethatjustbecausesomethingisasubringdoesn’tmeanit’sanideal.IntegersareasubringofGaussianintegers,buttheyaren’tanidealofGaussianintegers,becausemultiplyinganintegerbytheimaginarynumberidoesnotproduceaninteger.
Exercise12.2.
1.ProvethatanidealIisclosedundersubtraction.2.ProvethatIcontains0.
Lemma12.1(LinearCombinationIdeal):Inaring,foranytwoelementsaandb,thesetofallelements{xa+yb}formsanideal.
Proof.First,thissetisclosedunderaddition:
(x1a+y1b)+(x2a+y2b)=(x1+x2)a+(y1+y2)b
Next,itisclosedundermultiplicationbyanarbitraryelement:
z(xa+yb)=(zx)a+(zy)b
Therefore,itisanideal.
Exercise12.3.Provethatalltheelementsofalinearcombinationidealaredivisiblebyanyofthecommondivisorsofaandb.
Lemma12.2(IdealsinEuclideanDomains):AnyidealinaEuclideandomainisclosedundertheremainderoperationandunderEuclideanGCD.
Proof.
1.Closedunderremainder:Bydefinition,
remainder(a,b)=a–quotient(a,b)·b
Ifbisintheideal,thenbythesecondaxiomofideals,anythingmultipliedbybisintheideal,soquotient(a,b)·bisintheideal.ByExercise12.2,thedifferenceoftwoelementsofanidealisintheideal.
2.ClosedunderGCD:SincetheGCDalgorithmconsistsofrepeatedlyapplyingremainder,thisimmediatelyfollowsfrom1.
Definition12.2.AnidealIoftheringRiscalledaprincipalidealifthereisanelementa RcalledtheprincipalelementofIsuchthat
x I ∃y R:x=ay
Inotherwords,aprincipalidealisanidealthatcanbegeneratedfromoneelement.Anexampleofaprincipalidealisthesetofevennumbers(2istheprincipalelement).Polynomialswithroot5areanotherprincipalideal.Incontrast,polynomialswithxandyandfreecoefficient0areideals,butnotprincipalideals.Rememberthepolynomialx2+3y2+xy+x,whichwegaveasanexampleofanideal?There’snowaytogenerateitstartingwithjustx(itwouldnevercontainy),andviceversa.
Exercise12.4.Provethatanyelementinaprincipalidealisdivisiblebytheprincipalelement.
Recallthatanintegraldomainisaringwithnozerodivisors(Definition8.7).
Definition12.3.Anintegraldomainiscalledaprincipalidealdomain(PID)ifeveryidealinitisaprincipalideal.
Forexample,theringofintegersisaPID,whiletheringofmultivariatepolynomialsoverintegersisnot.
Theorem12.2:ED PID.EveryEuclideandomainisaprincipalidealdomain.
Proof.AnyidealIinaEuclideandomaincontainsanelementmwithaminimalpositivenorm(a“smallestnonzeroelement”).Consideranarbitraryelementa I;eitheritisamultipleofmorithasaremainderr:
Butwechosemasthesmallestelement,sowecannothaveasmallerremainder—thatwouldbeacontraction.Thereforeourelementacan’thavearemainder;a=qm.Sowecanobtaineveryelementfromoneelement,whichisthedefinitionofaPID.
NowwecanproveBézout’sidentity.Sinceitsaysthatthereisatleastonevalueofxandonevalueofythatsatisfytheequationxa+yb=gcd(a,b),wecanrestateitassayingthatthesetofallpossiblelinearcombinationsxa+ybcontainsthedesiredvalue.
Bézout’sIdentity,Restated:AlinearcombinationidealI={xa+yb}ofaEuclideandomaincontainsgcd(a,b).
Proof.ConsiderthelinearcombinationidealI={xa+yb}.aisinIbecausea=1a+0b.Similarly,bisinIbecauseb=0a+1b.ByLemma12.2,anyidealinaEuclideandomainisclosedundertheGCD;thusgcd(a,b)isinI.
WecanalsouseBézout’sidentitytoprovetheInvertibilityLemma,whichweencounteredinChapter5:
Lemma5.4(InvertibilityLemma):
Proof.ByBézout’sidentity,
∃x,y :xa+yn=gcd(a,n)
Soifgcd(a,n)=1,thenxa+yn=1.Therefore,xa=–yn+1,and
xa=1modn
Exercise12.5.UsingBézout’sidentity,provethatifpisprime,thenany0<a<phasamultiplicativeinversemodulop.
12.4ExtendedGCDTheproofofBézout’sidentitythatwesawinthelastsectionisinteresting.Itshowswhytheresultmustbetrue,butitdoesn’tactuallytellushowtofindthecoefficients.Thisisanexampleofanonconstructiveproof.Foralongtime,therewasadebateinmathematicsaboutwhethernonconstructiveproofswereasvalidasconstructiveproofs.Thosewhoopposedtheuseofnonconstructiveproofswereknownasconstructivistsorintuitionists.Attheturnofthe20thcentury,thetideturnedagainsttheconstructivists,withDavidHilbertandtheGöttingenschoolleadingthecharge.Thelonemajordefenderofconstructivism,HenriPoincaré,lostthebattle,andtodaynonconstructiveproofsareroutinelyused.
HenriPoincaré(1854–1912)
JulesHenriPoincaréwasaFrenchmathematicianandphysicist.Hecamefromadeeplypatrioticfamily,andhiscousinRaymondwasaprimeministerofFrance.Hepublishedmorethan500papersonavarietyofsubjects,includingseveralonspecialrelativitydevelopedindependentlyof,andinsomecasespriorto,Einstein’sworkonthesubject.Althoughheworkedonmanypracticalproblems,suchasestablishingtimezones,hemostlyvaluedscienceasatoolforunderstandingtheuniverse.Ashewrote:Thescientistmustnotdallyinrealizingpracticalaims.Henodoubtwillobtainthem,butmustobtaintheminaddition.Henevermustforgetthatthespecialobjectheisstudyingisonlyapartofthisbigwhole,whichmustbethesolemotiveofhisactivity.Sciencehashadmarvelousapplications,butasciencethatwouldonlyhaveapplicationsinmindwouldnotbescienceanymore,itwouldbeonlycookery.
Poincaréalsowroteseveralimportantbooksaboutthephilosophyofscience,andwaselectedamemberoftheAcadémieFrançaise.Poincarécontributedtoalmosteverybranchofmathematics,originatingseveralsubfields,such
asalgebraictopology.PeopleatthetimedebatedaboutwhetherPoincaréorHilbertwasthegreatestmathematicianintheworld.However,Poincaré’scriticismofsettheoryandtheformalistagendaofHilbertputhimonthewrongsideoftherivalrybetweenFranceandarecentlyunifiedGermany.TherejectionofPoincaré’sintuitionistapproachbytheformalistswasagreatlossfor20th-centurymathematics.Bothpointsofviewcomplementeachother.
Whatevertheviewofformalistmathematicians,fromaprogrammingperspective,itisclearlymoresatisfactorytoactuallyhaveanalgorithmthansimplytoknowthatoneexists.Sointhissection,wewillderiveaconstructiveproofofBézout’sidentity—inotherwords,analgorithmforfindingxandysuchthatxa+yb=gcd(a,b).Tounderstandtheprocess,it’shelpfultoreviewwhathappensinseveraliterationsofEuclid’s
algorithmtocomputetheGCDofaandb.Clickheretoviewcodeimage
template<EuclideanDomainE>
Egcd(Ea,Eb){
while(b!=E(0)){
a=remainder(a,b);
std::swap(a,b);
}
returna;
}
Eachtimethroughthemainloop,wereplaceabytheremainderofaandb,thenswapaandb;ourfinalremainderwillbetheGCD.Sowearecomputingthissequenceofremainders:
Notehowthesecondargumenttotheremainderfunctiononiterationkshiftsovertobecomethefirstargumentoniterationk+1.Sincetheremainderofaandbiswhat’sleftoverfromaafterdividingabyb,wecanwritethe
sequencelikethis:
wheretheqtermsarethecorrespondingquotients.Wecansolveeachequationforthefirsttermontheright—thefirstargumentoftheremainderfunction:
InSection4.7,weshowedhowthelastnonzeroremainderrninthesequenceisequaltotheGCDoftheoriginalarguments.ForBézout’sidentity,we’dliketoshowthatrn=gcd(a,b)=xa+yb.Ifeachoftheequationsintheprecedingseriescouldbewrittenaslinearcombinationsofaandb,thenextonecouldaswell.Thefirstthreeareeasy:
Thelastequationcomesfromouroriginaldefinitionofr1inthefirstremaindersequence(Equation12.8).Thenextonerequiressubstitutingtheexpansionofr1andrearrangingterms:
Next,wehaveaniterativerecurrence.Assumethatwealreadyfiguredouthowtorepresenttwosuccessiveremaindersaslinearcombinations:
ri=xia+yib
ri+1=xi+1a+yi+1b
Thenwecanuseourpreviousobservationabouthowtodefineanarbitraryremainderinthesequenceusingthetwopreviousones(Equation12.9),substitutingthosepreviousvaluesandagaindistributingandrearrangingtogroupallthecoefficientswithaandallthecoefficientswithb:
Wehaveseenhowwecanexpresseveryentryinthesequenceasalinearcombinationofaandb,andourproceduregivesusthecoefficientsxandyateverystep.Furthermore,weobservethatthecoefficientsonaaredefinedintermsofthepreviouscoefficientsona(i.e.,theonlyvariablesarexs),andthecoefficientsonbaredefinedintermsofthepreviouscoefficientsonb(i.e.,theonlyvariablesareys).Inparticular,thecoefficientsoniterationi+2are
Whenwereachtheend,wehavecoefficientsxandysuchthatxa+yb=gcd(a,b),whichwaswhatwewantedallalong.We’veseenthatthattheysdonotdependonthexsandthexsdonotdependontheys.Sinceweknow
xa+yb=gcd(a,b),thenaslongasb≠0,wecanrearrangethesetermstodefineyas
Thismeanswedon’tneedtogothroughthetroubleofcomputingalltheintermediatevaluesofy.
Exercise12.6.Whatarexandyifb=0?
***
NowthatweknowhowtocomputethecoefficientxfromBézout’sidentity,wecanenhanceourGCDalgorithmtoreturnthisvalueaswell.ThenewalgorithmiscalledextendedGCD(alsoknownastheextendedEuclidalgorithm).ItwillstillcomputetheusualseriesofremaindersforGCD,butitwillalsocomputetheseriesofxcoefficientsfromourearlierequations.Aswehaveseen,wedon’tneedtokeepeveryvalueintherecurrence;wejustneedthetwoprevious
ones,whichwe’llcallx0andx1.Ofcourse,weneedtoknowthefirsttwovaluestogettheseriesstarted,butfortunatelywehavethem—theyarethecoefficientsofainthefirsttwolinearcombinations(Equations12.10and12.11),namely1and0.Thenwecanusethexi+2formulawehavejustderived(Equation12.12)tocomputethenewvalueeachtime.Eachnewxcomputationrequiresaquotient,sowe’llneedafunctionforthat.Meanwhilewe’restillcomputingtheGCD,sowestillneedtheremainder.Sinceweneedbothquotientandremainder,we’llusethegenericversionofthequotient_remainderfunctionweintroducedinSection4.6,whichreturnsapairwhosefirstelementisthequotientandwhosesecondelementistheremainder.HereisourextendedGCDcode:Clickheretoviewcodeimage
template<EuclideanDomainE>
std::pair<E,E>extended_gcd(Ea,Eb){
Ex0(1);
Ex1(0);
while(b!=E(0)){
//computenewrandx
std::pair<E,E>qr=quotient_remainder(a,b);
Ex2=x0-qr.first*x1;
//shiftrandx
x0=x1;
x1=x2;
a=b;
b=qr.second;
}
return{x0,a};
}
Attheend,whenbiszero,thefunctionreturnsapairconsistingofthevalueofxthatwewantedforBézout’sidentity,andtheGCDofaandb.
Exercise12.7.DevelopaversionoftheextendedGCDalgorithmbasedonStein’salgorithm.
12.5ApplicationsofGCDTowrapupourdiscussionofGCD,weconsidersomeimportantusesofthealgorithm.Cryptography.Asweshallseeinthenextchapter,moderncryptographicalgorithmsrelyonbeingable
tofindthemultiplicativeinversemodulonforlargenumbers,andtheextendedGCDalgorithmallowsustodothis.WeknowfromBézout’sidentitythat
xa+yb=gcd(a,b)
so
xa=gcd(a,b)–yb
Ifgcd(a,b)=1,then
xa=1–yb
Thusmultiplyingxandagives1plussomemultipleofb,ortoputitanotherway,
xa=1modb
AswelearnedinChapter5,twonumberswhoseproductis1aremultiplicativeinverses.Sincetheextended_gcdalgorithmreturnsxandgcd(a,b),iftheGCDis1,thenxisthemultiplicativeinverseofamodb;wedon’tevenneedy.RationalArithmetic.Rationalarithmeticisveryusefulinmanyareas,anditcan’tbedonewithout
reducingfractionstotheircanonicalform,whichrequirestheGCDalgorithm.SymbolicIntegration.Oneoftheprimarycomponentsofsymbolicintegrationisdecomposinga
rationalfractionintoprimitivefractions,whichusestheGCDofpolynomialsoverrealnumbers.RotationAlgorithms.WesawinChapter11howtheGCDplaysaroleinrotationalgorithm.Infact,the
std::rotatefunctioninC++reliesonthisrelationship.
12.6ThoughtsontheChapterInthischapter,wesawtwoexamplesofhowcontinuedexplorationofanoldalgorithmcanleadtonewinsights.Stein’sobservationsaboutpatternsofoddandevennumberswhencomputingtheGCDallowedhimtocomeupwithamoreefficientalgorithm,onethatexposedsomeimportantmathematicalrelationships.Bachet’sproofofatheoremabouttheGCDgaveustheextendedGCDalgorithm,whichhasmanypracticaluses.Inparticular,thediscoveryoftheSteinalgorithmillustratesafewimportantprogrammingprinciples:1.Everyusefulalgorithmisbasedonsomefundamentalmathematicaltruth.WhenSteinnoticedsomeusefulpatternsincomputingtheGCDofoddandevennumbers,hewasn’tthinkingaboutsmallestprimes.Indeed,it’sverycommonthatthediscovererofanalgorithmmightnotseeitsmostgeneralmathematicalbasis.Thereisoftenalongtimebetweenthefirstdiscoveryofthealgorithmanditsfullunderstanding.Nevertheless,itsunderlyingmathematicaltruthisthere.Forthisreason,everyusefulprogramisaworthyobjectofstudy,andbehindeveryoptimizationthereissolidmathematics.
2.Evenaclassicalproblemstudiedbygreatmathematiciansmayhaveanewsolution.Whensomeonetellsyou,forexample,thatsortingcan’tbedonefasterthannlogn,don’tbelievethem.Thatstatementmightnotbetrueforyourparticularproblem.
3.Performanceconstraintsaregoodforcreativity.ThelimitationsthatSteinfacedusingtheWEIZACcomputerin1961arewhatdrovehimtolookforalternativestothetraditionalapproach.Thesameistrueinmanysituations;necessityreallyisthemotherofinvention.
13.AReal-WorldApplication
Iamfairlyfamiliarwithallformsofsecretwritings,andammyselftheauthorofatriflingmonographuponthesubject,inwhichIanalyzeonehundredandsixtyseparateciphers,but
Iconfessthatthisisentirelynewtome.SherlockHolmes
Throughoutthisbook,we’veseenexamplesofimportantalgorithmsthatcameoutofworkonnumbertheory.We’vealsoseenhowattemptstogeneralizethosemathematicalresultsbroughtaboutthedevelopmentofabstractalgebra,andhowitsideasaboutabstractionleddirectlytotheprinciplesofgenericprogramming.Nowwe’regoingputitalltogether.We’llshowhowourmathematicalresultsandourgeneralizedalgorithmscanbeusedtoimplementareal-worldapplication:aparticularkindofsystemforsecurecommunication,knownasapublic-keycryptosystem.
13.1CryptologyCryptologyisthescienceofsecretcommunication.Cryptographyisconcernedwithdevelopingcodesandciphers;1cryptanalysiswithbreakingthem.
1Technically,acodeisasystemforsecretcommunicationwhereameaningfulconceptsuchasthenameofaperson,place,oreventisreplacedwithsomeothertext,whileacipherisasystemformodifyingtextatthelevelofitsrepresentation(lettersorbits).Butwe’llusethetermsinterchangeably;inparticular,we’lluseencodeanddecodeinformallytomeanencipheranddecipher.
Theideaofsendingsecretmessagesdatesbacktoancienttimes,withexamplesofcryptographyinmanysocietiesincludingSpartaandPersia.JuliusCaesarusedthetechniqueofreplacingletterswiththoseina“rotated”alphabet(nowknownastheCaesarcipher)tosendmilitarymessages.Inthe19thcentury,cryptographyand“cryptograms,”puzzlesthatuseasimplesubstitutioncipher,caughtthepublic’simagination.Inan1839magazinearticle,EdgarAllenPoeclaimedhecoulddecipheranysuchmessageshisreaderssubmitted—andapparentlysucceeded.Afewyearslaterhepublishedashortstorycalled“TheGoldBug,”whichincludesanaccountofhowtobreaksuchacode.Asubstitutioncipheralsofeaturedprominently,severaldecadeslater,inSirArthurConanDoyle’sSherlockHolmesmystery,“TheAdventureoftheDancingMen.”Buttheimportanceofcryptographywentwellbeyondcasualentertainment.Codesandciphersplayed
animportantroleindiplomacy,espionage,andwarfare.Bytheearly20thcentury,creatingbettercryptographicschemeswasatoppriorityforthemilitaryoftheworld’sleadingpowers.Theabilitytobreakthesecodescouldmakethedifferencebetweensuccessandfailureonthebattlefield.
BletchleyParkandtheDevelopmentofComputersInWorldWarII,themainBritishcryptanalysisgroupwasbasedatanestateinthecountrysidecalledBletchleyPark.Atthetime,theGermannavywasusinganenhancedversionofacommercialencryptionmechanismcalledtheEnigmamachine.AnearlierversionoftheEngimahadbeencrackedbyPolishcryptographerMarianRejewski,usinganelectromechanicaldevicethattestedmanypossibleEnigmasettingsinparallel.AtBletchleyPark,abrilliantyoungmathematiciannamedAlanTuring,whosepreviouswork
providedmuchofthefoundationsofwhatbecamecomputerscience,designedamuchmoresophisticatedversionofRejewski’sdevicecalleda“bombe.”ThroughtheworkofTuringandmanyothers,theAllieswereabletodeciphertheEnigmamessages,agreathelpinwinningthewar.AnotherencryptionmechanismusedbytheNaziswascalledtheLorenzmachine.Tobreakthe
Lorenzcipher,theBritishcryptographersrealizedthattheelectromechanicalbombeswerenotfastenough.SoanengineernamedTommyFlowersdesignedamuchmorepowerfulelectronicdeviceusingvacuumtubes,called“Colossus.”Althoughitwasnotageneral-purposemachineandwasonlypartiallyprogrammable,Colossusmayhavebeentheworld’sfirstprogrammableelectronicdigitalcomputer.
Acryptosystemconsistsofalgorithmsforencryptinganddecryptingdata.Theoriginaldataiscalledtheplaintext,andtheencrypteddataiscalledtheciphertext.Asetofkeysdeterminethebehavioroftheencryptionanddecryptionalgorithms:
ciphertext=encryption(key0,plaintext)plaintext=decryption(key1,ciphertext)
Thesystemissymmetricifkey0=key1;otherwise,itisasymmetric.Manyearlycryptosystemsweresymmetricandusedsecretkeys.Theproblemwiththisisthatthe
senderandthereceiverofthemessagebothmusthavethekeysinadvance.Ifthekeyiscompromisedandthesenderwantstoswitchtoanewone,hehastheadditionalproblemoffiguringouthowtosecretlyconveythenewkeytothereceiver.
***
Apublic-keycryptosystemisanencryptionschemethatmakesuseofapairofkeys:apublickeypubforencrypting,andaprivatekeyprvfordecrypting.IfAlicewantstosendamessagetoBob,sheencryptsthemessagewithBob’spublickey.TheciphertextisthenunreadabletoanyonebutBob,whouseshisprivatekeytodecryptthemessage.Tohaveapublic-keycryptosystem,thefollowingrequirementsmustbesatisfied:1.Theencryptionfunctionneedstobeaone-wayfunction:easytocompute,withaninversethatishardtocompute.Here,“hard”hasitstechnicalcomputersciencemeaningoftakingexponentialtime—inthiscase,exponentialinthesizeofthekey.
2.Theinversefunctionhastobeeasytocomputewhenyouhaveaccesstoacertainadditionalpieceofinformation,knownasthetrapdoor.
3.Bothencryptionanddecryptionalgorithmsarepubliclyknown.Thisensurestheconfidenceofall
partiesinthetechniquebeingused.Afunctionmeetingthefirsttworequirementsisknownasatrapdoorone-wayfunction.Perhapsthebest-knownandmostwidelyusedpublic-keycryptosystemistheRSAalgorithm,named
afteritscreators(Rivest,Shamir,andAdleman).Aswe’llseeshortly,RSAdependsonsomemathematicalresultsaboutprimes.
WhoInventedPublic-KeyCryptography?Foryears,itwasbelievedthatStanfordprofessorMartinHellman,togetherwithtwograduatestudents,WhitfieldDiffieandRalphMerkle,inventedpublic-keycryptographyin1976.Theyproposedhowsuchasystemwouldwork,andrealizeditwouldrequireatrapdoorone-wayfunction.Unfortunately,theydidn’tgiveanexampleofsuchafunction—itwasjustahypotheticalconstruct.In1977,MITresearchersRonRivest,AdiShamir,andLenAdlemancameupwithaprocedureforcreatingatrapdoorone-wayfunction,whichbecameknownastheRSAalgorithmaftertheinventors’initials.In1997,theBritishgovernmentdisclosedthatoneoftheirintelligenceresearchers,Clifford
Cocks,hadactuallyinventedaspecialcaseofRSAin1973—butittook20yearsafterthepublicationoftheRSAalgorithmbeforetheydeclassifiedCocks’memo.Afterthat,AdmiralBobbyRayInman,theformerheadoftheU.S.NationalSecurityAgency,claimedthathisagencyhadinventedsomesortofpublic-keycryptographictechniqueevenearlier,inthe1960s,althoughnoevidencewasgiven.Whoknowswhichcountry’sintelligenceagencywillcomeforwardnextwithanearlierclaim?
13.2PrimalityTestingTheproblemofdistinguishingprimenumbersfromcomposite...isknowntobeoneofthemostimportantandusefulinarithmetic.
C.F.Gauss,DisquisitionesArithmeticae
Animportantprobleminmoderncryptographyisdeterminingwhetheranintegerisprime.Gaussbelievedthat(1)decidingwhetheranumberisprimeorcompositeisaveryhardproblem,andsois(2)factoringanumber.Hewaswrongabout#1,asweshallsee.Sofar,heseemstoberightabout#2,whichisagoodthingforus,sincemoderncryptosystemsarebasedonthisassumption.Tofindoutifanumbernisprime,ithelpstohaveapredicatethattellsuswhetherit’sdivisiblebya
givennumberi:Clickheretoviewcodeimage
template<IntegerI>
booldivides(constI&i,constI&n){
returnn%i==I(0);
}
Wecancallthisrepeatedlytofindthesmallestdivisorofagivennumbern.JustaswedidwiththeSieveofEratosthenesinChapter3,ourloopfortestingdivisorswillstartat3,advanceby2,andstopwhenthesquareofthecurrentcandidatereachesn:Clickheretoviewcodeimage
template<IntegerI>
Ismallest_divisor(In){
//precondition:n>0
if(even(n))returnI(2);
for(Ii(3);i*i<=n;i+=I(2)){
if(divides(i,n))returni;
}
returnn;
}
Nowwecancreateasimplefunctiontodeterminewhethernisprime:Clickheretoviewcodeimage
template<IntegerI>
Iis_prime(constI&n){
returnn>I(1)&&smallest_divisor(n)==n;
}
Thisismathematicallycorrect,butit’snotgoingtobefastenough.Itscomplexityis.Thatis,it’sexponentialinthenumberofdigits.Ifwewanttotesta200-digit
number,wemaybewaitingformoretimethanthelifeoftheuniverse.Toovercomethisproblem,we’regoingtoneedadifferentapproach,whichwillrelyontheabilityto
domodularmultiplication.We’lluseafunctionobjectthatprovideswhatweneed:Clickheretoviewcodeimage
template<IntegerI>
structmodulo_multiply{
Imodulus;
modulo_multiply(constI&i):modulus(i){}
Ioperator()(constI&n,constI&m)const{
return(n*m)%modulus;
}
};
We’llalsoneedanidentityelement:Clickheretoviewcodeimage
template<IntegerI>
Iidentity_element(constmodulo_multiply<I>&){
returnI(1);
}
Nowwecancomputeamultiplicativeinversemoduloprimep.ItusestheresultweshowedinChapter5that,asaconsequenceofFermat’sLittleTheorem,theinverseofanintegera,where0<a<p,isap−2(seeSection5.4,rightaftertheproofofFermat’sLittleTheorem).ItalsousesthepowerfunctionwecreatedinChapter7:Clickheretoviewcodeimage
template<IntegerI>
Imultiplicative_inverse_fermat(Ia,Ip){
//precondition:pisprime&a>0
returnpower_monoid(a,p-2,modulo_multiply<I>(p));
}
Withthesepieces,wecannowuseFermat’sLittleTheoremtotestifanumbernisprime.RecallthatFermat’sLittleTheoremsays:
Ifpisprime,thenap−1−1isdivisiblebypforany0<a<p.
Equivalently:
Ifpisprime,thenap–1=1modpforany0<a<p.
Wewanttoknowifnisprime.Sowetakeanarbitrarynumberasmallerthann,raiseittothen−1powerusingmodularmultiplication(modn),andcheckiftheresultis1.(Wecallthenumberawe’reusingawitness.)Iftheresultisnotequalto1,weknowdefinitelybythecontrapositiveofthetheoremthatnisnotprime.Iftheresultisequalto1,weknowthatthere’sagoodchancethatnisprime,andifwedothisforlotsofrandomwitnesses,there’saverygoodchancethatitisprime:Clickheretoviewcodeimage
template<IntegerI>
boolfermat_test(In,Iwitness){
//precondition:0<witness<n
Iremainder(power_semigroup(witness,
n-I(1),
modulo_multiply<I>(n)));
returnremainder==I(1);
}
Thistimeweusepower_semigroupinsteadofpower_monoid,becauseweknowwe’renotgoingtoberaisinganythingtothepower0.TheFermattestisveryfast,becausewehaveafastwaytoraiseanumbertoapower—ourO(logn)generalizedEgyptianmultiplicationalgorithmfromChapter7.
***
WhiletheFermattestworksthevastmajorityofthetime,itturnsoutthattherearesomepathologicalcasesofnumbersthatwillfoolitforallwitnessescoprimeton;theyproduceremaindersof1eventhoughthey’recomposite.ThesearecalledCarmichaelnumbers.
Definition13.1.Acompositenumbern>1isaCarmichaelnumberifandonlyif
∀b>1,coprime(b,n) bn−1=1modn
172081isanexampleofaCarmichaelnumber.Itsprimefactorizationis7·13·31·61.
Exercise13.1.Implementthefunction:
boolis_carmichael(n)
Exercise13.2.FindthefirstsevenCarmichaelnumbersusingyourfunctionfromExercise13.1.
13.3TheMiller-RabinTestToavoidworryingaboutCarmichaelnumbers,we’regoingtouseanimprovedversionofourprimalitytester,calledtheMiller-Rabintest;itwillagainrelyonthespeedofourpoweralgorithm.Weknowthatn−1iseven(itwouldbeawfullysillytorunaprimalitytestonanevenn),sowecan
representn−1astheproduct2k·q.TheMiller-Rabintestusesasequenceofsquaresw20q,w2
1q,...,w2
kq,wherewisarandomnumberlessthantheonewearetesting.Thelastexponentinthissequenceisn−1,thesamevaluetheFermattestuses;we’llseewhythisisimportantshortly.We’realsogoingtorelyontheself-cancelinglaw(Lemma5.3),exceptwe’llwriteitwithnew
variablenamesandassumingmodularmultiplication:
Forany0<x<n∧prime(n),x2=1modn x=1∨x=−1
Rememberthatinmodulararithmetic,−1modnisthesameas(n−1)modn,afactthatwerelyoninthefollowingcode.Ifwefindsomex2=1modnwherexisneither1nor−1,thennisnotprime.Nowwecanmaketwoobservations:(1)Ifx2=1modn,thenthere’snopointinsquaringxagain,
becausetheresultwon’tchange;ifwereach1,we’redone.(2)Ifx2=1modnandxisnot−1,thenweknownisnotprime(sincewealreadyruledoutx=1earlierinthecode).Here’sthecode,whichreturnstrueifnisprobablyprime,andfalseifitdefinitelyisnot:
Clickheretoviewcodeimage
template<IntegerI>
boolmiller_rabin_test(In,Iq,Ik,Iw){
//preconditionn>1∧n–1=2kq∧qisodd
modulo_multiply<I>mmult(n);
Ix=power_semigroup(w,q,mmult);
if(x==I(1)||x==n-I(1))returntrue;
for(Ii(1);i<k;++i){
//invariantx=w2i–1q
x=mmult(x,x);
if(x==n-I(1))returntrue;
if(x==I(1))returnfalse;
}
returnfalse;
}
Notethatwepassinqandkasarguments.Sincewe’regoingtocallthefunctionmanytimeswithdifferentwitnesses,wedon’twanttorefactorn–1everytime.Whycanwereturntrueatthebeginningifthepower_semigroupcallreturns1or–1?Becausewe
knowthatsquaringtheresultwillgive1,andsquaringisequivalenttomultiplyingtheexponentinthepowercalculationbyafactorof2,anddoingthisktimeswillmaketheexponentn–1,thevalueweneedforFermat’sLittleTheoremtohold.Inotherwords,ifwqmodn=1or–1,thenw2kqmodn=wn–1modn=1.Let’slookatanexample.Supposewewanttoknowifn=2793isprime.Wechoosearandomwitness
w=150.Wefactorn–1=2792into22·349,soq=349andk=2.Wecompute
x=wqmodn=150349mod2793=2019
Sincetheresultisneither1nor–1,westartsquaringx:
i=1;x2=15021·349mod2793=1374
i=2;x2=15022·349mod2793=2601
Sincewehaven’treached1or–1yet,andi=k,wecanstopandreturnfalse;2793isnotprime.
LiketheFermattest,theMiller-Rabintestisrightmostofthetime.UnliketheFermattest,theMiller-Rabintesthasaprovableguarantee:itisrightatleast75%ofthetime2forarandomwitnessw.(Inpractice,it’sevenmoreoften.)Randomlychoosing,say,100witnessesmakestheprobabilityoferrorlessthan1in2200.AsKnuthremarked,“Itismuchmorelikelythatourcomputerhasdroppedabit,due...tocosmicradiations.”
2Infact,therequirementthatqmustbeoddisneededforthisprobabilityguarantee.
AKS:ANewTestforPrimalityIn2002,NeerajKayalandNitinSaxena,twoundergraduatestudentsattheIndianInstituteofTechnologyatKanpur,togetherwiththeiradvisor,ManindraAgrawal,cameupwithadeterministicpolynomial-timealgorithmforprimalitytesting,andpublishedtheirresult.Thisisaproblemthatpeopleinnumbertheoryhadbeenworkingonforcenturies.ThereisaveryclearpaperbyAndrewGranvilledescribingthetechnique.Althoughitisa
densemathematicalpaper,itisunderstandabletoasurprisinglywideaudience.Thisisunusual;mostimportantmathematicalresultsbeingpublishedinrecentdecadesrequireyearsofpriormathematicalstudytobeunderstood.Determinedreaderswhoarewillingtoputinseriouseffortareencouragedtoreadit.DespitethefactthattheAKSalgorithmisagreataccomplishment,we’renotgoingtouseithere,
becausetheprobabilisticMiller-Rabinalgorithmisstillconsiderablyfaster.
13.4TheRSAAlgorithm:HowandWhyItWorksTheRSAalgorithmisoneofthemostimportantandwidelyusedcryptosystemsinusetoday.Itisoftenusedforauthentication—toprovethatusers,companies,websites,andotherentitieswithanonlinepresencearewhotheysaytheyare.Itisalsooftenusedtoexchangeprivatekeysthatareusedinaseparate,fastersymmetriccryptosystemusedtoencodedatabeingcommunicated.SomeoftheimportantcommunicationprotocolsthatuseRSAare:
Weusemanyoftheseprotocolsdaily.Forexample,anytimeyouvisita“secure”website(onewhoseURLhasanhttpsprefix),you’rerelyingonSSL/TLS,whichinturnusesRSAor(dependingontheimplementation)asimilarpublic-keycryptosystem.RSAreliesonthemathematicalresultswe’vejustshownforprimalitytesting.RSArequirestwosteps:
keygeneration,whichneedstobedoneonlyrarely,andencoding/decoding,whichisdoneeverytimeamessageissentorreceived.Keygenerationworksasfollows.First,thefollowingvaluesarecomputed:•Tworandomlargeprimes,p1andp2(theMiller-Rabintestmakesthisfeasible)•Theirproduct,n=p1p2•TheEulerfunctionoftheirproduct,whichwecancomputeusingEquation5.5fromChapter5:φ(p1p2)=(p1–1)(p2–1)
•Arandompublickeypub,coprimewithφ(p1p2)•Aprivatekeyprv,themultiplicativeinverseofpubmoduloφ(p1p2)(Tocomputethis,we’llusetheextendedGCDfunctionwederivedinChapter12.)
Whenthesecomputationsarecomplete,p1andp2aredestroyed;pubandnarepublished,andprviskeptsecret.Atthispoint,thereisnofeasiblewaytofactorn,sinceitissuchalargenumberwithsuchlargefactors.Theencodinganddecodingprocessissimpler.Thetextisdividedintoequalsizeblocks,say256bytes
long,whichareinterpretedaslargeintegers.Themessageblocksizesmustbechosensothatn>2s.Toencodeaplaintextblock,weuseourfamiliarpoweralgorithm:Clickheretoviewcodeimage
power_semigroup(plaintext_block,pub,modulo_multiply<I>(n));
Decodinglookslikethis:Clickheretoviewcodeimage
power_semigroup(ciphertext_block,prv,modulo_multiply<I>(n));
Observethatwedoexactlythesameoperationtoencodeanddecode.Theonlydifferenceiswhichtextandwhichkeywepassin.
***
HowdoesRSAwork?Encryptionconsistsofraisingamessagemtoapowerpub;decryptionconsistsofraisingtheresulttothepowerprv.Weneedtoshowthattheresultofapplyingthesetwooperationsistheoriginalmessagem(modulon):
(mpub)prv=mmodn
Proof.Recallthatwespecificallycreatedprvtobethemultiplicativeinverseofpubmoduloφ(p1p2),sobydefinition,theproductofpubandprvissomemultipleqofφ(p1p2)witharemainderof1.Wecanmakethatsubstitutionintheexponentontheright.
NowwecanapplyEuler’stheoremfromChapter5,whichsaysthataφ(n)–1isdivisiblebyn;thatis,aφ(n)=1+vn.Makingthatsubstitution,wehave
=m(1+vn)q
Whenweexpand(1+vn)q,everytermwillbeamultipleofnexcept1,sowecancollapsealloftheseandjustsaywehave1plussomeothermultipleofn:
=m+wn
=mmodn
TheEulertheoremstepdependsonmbeingcoprimewithn=p1p2.Sincethemessagemcouldbeanything,howdoweknowthatitwillbecoprimetop1p2?Sincep1andp2areenormousprimes,thatprobabilityispracticallyindistinguishablefrom1,andpeoplenormallydonotworryaboutit.However,ifyouwanttoaddressthis,youcanaddoneextrabytetotheendofm.Theextrabyteisnotactuallypartofthemessage,butisthereonlytoensurethatwehaveacoprime.Whenwecreatem,wecheckwhetheritiscoprime;ifitisn’t,wesimplyadd1tothisextrabyte.
***
WhydoesRSAwork?Inotherwords,whydowebelieveit’ssecure?Thereasonisthatfactoringishard,andthereforecomputingφisnotfeasible.Perhapsifquantumcomputersprovetoberealizableinthefuture,itwillbepossibletorunanexponentialnumberofdivisortestsinparallel,makingfactoringatractableproblem.Butfornow,wecanrelyonRSAformanysecurecommunicationsapplications.
ProjectExercise13.3.ImplementanRSAkeygenerationlibrary.
Exercise13.4.ImplementanRSAmessageencoder/decoderthattakesastringandthekeyasitsarguments.
Hints:•Ifyourlanguagedoesnotsupportarbitrary-precisionintegers,you’llneedtoinstallapackageforhandlingthem.•RememberthattwonumbersarecoprimeiftheirGCDis1.Thiswillcomeinhandyforoneofthekeygenerationsteps.•You’llneedtheresultsthatwederivedinChapter12.Therearetworelevantfunctions,extended_gcdandmultiplicative_inverse.
Recallthattheextended_gcdfunctionfromChapter12returnsapair(x,y)suchthatax+ny=gcd(a,n).Youcanusethisfunctiontocheckforcoprimes.It’salsopartoftheimplementationofmultiplicative_inverse,afunctionthatreturnsthemultiplicativeinverseofamodulonifitexists,or0ifitdoesnot.Unlikethefunctionmultiplicative_inverse_fermatthatweintroducedinSection13.2,thisoneworksforanyn,notjustprimes:Clickheretoviewcodeimage
template<IntegerI>
Imultiplicative_inverse(Ia,In){
std::pair<I,I>p=extended_gcd(a,n);
if(p.second!=I(1))returnI(0);
if(p.first<I(0))returnp.first+n;
returnp.first;
}
You’llneedthistogettheprivatekeyfromthepublickey.
13.5ThoughtsontheChapterIssuesofidentity,privacy,andsecurityarebecomingincreasinglyimportantasmoreofourpersonaldatalivesonlineandmoreofourpersonalcommunicationtravelsovertheInternet.Aswehaveseen,manyimportantprotocolsforkeepingdataprivateandsecurefromtamperingrelyonRSAorsimilarpublickeycryptosystemsforauthentication,exchangeofkeysusedforencryption,orothersecurityfeatures.Theseimportantpracticalcapabilitiesowetheirexistencetoresultsfromoneofthemosttheoretical
branchesofmathematics,numbertheory.Thereisaperceptionamongprogrammersthatmathematiciansarepeoplewhodon’tknoworcareaboutpracticalconcernsandthatmathematics,particularlyinitsmoreabstractareas,haslittlepracticalvalue.Lookingatthehistory,wecanseethatbothoftheseperceptionsarefalse.Thegreatestmathematiciansenthusiasticallyworkedonextremelypracticalproblems—forexample,Gaussworkedononeofthefirstelectromechanicaltelegraphs,andPoincaréspentyearsdevelopingtimezones.Perhapsmoreimportantly,itisimpossibletoknowwhichtheoreticalideasaregoingtohavepracticalapplications.
14.Conclusions
Thestrongestargumentsprovenothingsolongastheconclusionsarenotverifiedbyexperience.
RogerBacon,OpusTertiumWestartedthisbookbycharacterizinggenericprogrammingasanattitudetowardprogrammingthatfocusesonabstractingalgorithmstotheirmostgeneralsettingwithoutlosingefficiency.Throughoutthebook,we’veseenexamplesofthisabstractionprocessinmathematicsandinprogramming.Wesawhowmathematicians’attemptstofindthemostgeneralsettingforEuclid’sGCDalgorithmledtothedevelopmentofabstractalgebra,anentireareaofmathematicsdevotedtoabstractstructures,whichitselfprovidedthebasisforgenericprogramming.Wealsosawhowtousethosesameprinciplesofabstractiontogeneralizeanancientalgorithmformultiplyingpositiveintegerstoafastpowerfunctiononsemigroups,enablingarangeofapplicationsrangingfromcomputingFibonaccinumberstofindingtheshortestpathinagraphtoencryptingdatainInternetcommunicationprotocols.Thisprocess—startingwithaspecificefficientsolutionand,wheneverpossible,relaxingtherequirements—isattheheartofgenericprogramming.Whiletheideaofabstractioningenericprogrammingcomestousdirectlyfromabstractalgebra,as
programmerswealsocareaboutefficiency.Agenericalgorithmthatrunsmoreslowlythanitstype-specificcounterpartwillnotgetused.That’swhyefficiencyisalsopartofthedefinitionofgenericprogramming.We’veshownexamplesthroughoutthebookofspecifictechniquesforimprovingefficiency,fromrewritingcodetousestrengthreduction,tousingmemory-adaptivealgorithms,toexploitingcompile-timetypedispatchsothecomputercaninvokethefastestavailableimplementationforagivensituation.Moregenerally,wehavefoundthatattemptstofindgenericversionsofalgorithmsoftenleadtosimplerandmoreefficientsolutions.We’vealsoseenhowthecorrectnessofaprogramminginterfacecanbejustasimportantasthe
correctnessoftheprogramitself.Acorrectinterfacecanenableawiderrangeofapplications.Itcanalsobringefficiencybenefits—forexample,byreturningalltherelevantcomputations(thelawofusefulreturn)toavoidaduplicationofeffort.Incontrast,anincorrectinterfacecripplestheapplicationbylimitingwhatitcando.Forexample,wesawhowafindfunctionthatreturnsonlyaBooleanvalueinsteadofthepositionofafounditemmakesitimpossibletoseeifasecondmatchingitemexists.Andjustasyouneedtorewriteaprogramafewtimestogetitright,soyouneedtoredesigntheinterface;thecorrectinterfaceusuallywon’tbeclearuntilyou’vealreadyimplementedanalgorithmandexploreditsusecases.Anotherideathat’sessentialtounderstandinggenericprogrammingisthedistinctionbetweentypeand
concept.Inmuchthesamewaythataxiomsinamathematicaltheoryarerequirementsthattelluswhatitmeanstobeacertainkindofabstractmathematicalentity(suchasagroup),conceptsinprogramingarerequirementsontypes;theytelluswhatitmeanstobeacertainkindofcomputationalentity.Choosingtherightconceptsforanalgorithmordatastructureisessentialtogoodprogramming.Choosingaconceptwithtoomanyrequirementsplacesunnecessarylimitationsontherangeofsituationsinwhichanalgorithmcanbeused.Choosingaconceptwithtoofewrequirementsmakesitimpossibletodefinealgorithmsthatdoanythinguseful.Nexttimeyousetouttowriteaprogram,trytoadoptthegenericprogrammingattitude.Startwith
specificimplementationsofyourfunctions,thenreviseandrefinethemtobemoreefficientandmoregeneral.Asyourefineyourcode,thinkcarefullyabouthowthepiecesfittogetherandhowtoprovidean
interfacethatwillstillbeusefulinthefuture.Chooseconceptsthatprovidejusttherightrequirementsforyourdata,withoutimposingunnecessaryassumptions.Andrememberthatyouaretheinheritorofalongmathematicaltraditionofalgorithmicthought.Infollowingtheprinciplesofgenericprogramming,youarealreadybenefitingfromtheworkofthosewhocamebefore,fromEuclidtoStevintoNoether.Bydesigningbeautiful,generalalgorithms,youareaddingyourownsmallcontributiontotheirwork.
FurtherReading
Readerswhoareinterestedinlearningmoreaboutthetopicsdiscussedinthisbookmaywishtolookatsomeofthereferencesmentionedhere.CompletecitationsareincludedintheBibliography.
Chapter1GenericProgramming.ThelanguageTecton,whichfirstusedgenericprogrammingconcepts,isdescribedin“Tecton:ALanguageforManipulatingGenericObjects”byKapur,Musser,andStepanov(1981).TheAdalibraryisdescribedinthepaper“GenericProgramming”byMusserandStepanov(1988),andC++STLin“TheStandardTemplateLibrary”byStepanovandLee(1994).Allofthesematerialsareavailableonwww.stepanovpapers.com.
Chapter2HistoryofMathematics.Agoodcomprehensivereference,notonlyforthischapterbutalsoformuchofthemathematicalhistoryinthebook,isKatz’sAHistoryofMathematics:AnIntroduction(2009).Thisgeneraltextbookcombinesthoroughnessandmathematicalrigorwithaccessibilitytothelayperson.AnincisivebookthatexplainsthehistoricaldevelopmentofsomemajormathematicalideasisMathematicsandItsHistorybyJohnStillwell(2010).RhindPapyrus.ToseeareproductionoftheRhindPapyrus,togetherwithitstranslation,seeThe
RhindMathematicalPapyrus:AnAncientEgyptianTextbyRobinsandShute(1987).VanderWaerdenincludesadiscussionoftheRhindPapyrusinGeometryandAlgebrainAncientCivilizations(1983).
Chapter3EgyptianandGreekMathematics.InadditiontoKatz,twoexcellentresourcesareVanderWaerden’sScienceAwakening(1963)andthetwo-volumeHistoryofGreekMathematicsbySirThomasHeath(originallypublishedin1921butavailableina1981reprint).Bothareveryaccessibletothegeneralreader.FigurateNumbers.ThebestintroductiontoPythagoreanarithmeticisthebookbyNicomachusof
Gerasa,whichcaneasilybefoundinthe10thvolumeoftheBritannica’sGreatBooksoftheWesternWorld,editedbyMortimerAdler.ThisvolumealsocontainsthecompleteworksofEuclidandArchimedes.BasicNumberTheory.AgoodintroductiontobasicnumbertheoryisinChapterIIIofGeorge
Chrystal’sAlgebra:AnElementaryText-Book.
Chapter4GreatestCommonMeasure.ForgeneralhistoryofGreekmathematics,includingthetopicscoveredinthischapter,thebestreferenceisstillHeath’sAHistoryofGreekMathematics,mentionedinthetopicsforChapter3.ForafascinatingandmathematicallysophisticatedaccountofthemathematicalstudiesinPlato’sAcademy,includingthealgorithmforthegreatestcommonmeasure,seeDavidFowler’sTheMathematicsofPlato’sAcademy,aNewReconstruction.Forthoseinterestedinreadingthesource,Plato’scompleteworksareavailableinaone-volumeedition,withaclearmoderntranslation,editedbyJohnM.Cooper.AgoodexplanationoftheGCDmaybefoundinChapterIIIofGeorgeChrystal’sAlgebra:AnElementaryText-Book.
DeclineofGreekScience.AnimportantaccountoftheriseanddeclineofGreekmathematicsispresentedinthebookbyLucioRusso,TheForgottenRevolution:HowScienceWasBornin300BCandWhyItHadtoBeReborn.HistoryofZero.OuraccountofthehistoryofzeroislargelytakenfromvanderWaarden’sScience
Awakening,pp.56–57.LeonardoPisano(Fibonacci).AshortautobiographyofLeonardoPisanohasbeentranslatedby
RichardGrimm.HisgreatworkLiberAbaciisavailableinanEnglishtranslationbyLaurenceSigler.AbriefbutthoroughdescriptionofLeonardoPisano’snumbertheoretictreatiseisgiveninMcClenon’s1919article“LeonardoofPisaandHisLiberQuadratorum.”ReadersinterestedinhistoryofarithmeticalgorithmsmaywanttoreadLeonardoPisano’soriginalLiberQuadratorum,availableinamodernEnglishtranslationbyL.E.Sigler(seeFibonacciintheBibliography).RemainderandQuotient.ThefulltreatmentoftheextensionofGCDtoremainderandquotientisin
Chapter5ofElementsofProgrammingbyStepanovandMcJones(2009).FloydandKnuth’salgorithmforremainderappearsintheir1990article“AdditionMachines.”
Chapter5Fermat’sandEuler’sNumberTheoryWork.AsourceformuchofthematerialinthischapterisNumberTheory:AnApproachthroughHistoryfromHammurapitoLegendrebyAndréWeil.Whilethisbookdoesnotassumeanyadvancedmath,itisprobablytoodetailedformostcasualreaders.TheclassicnumbertheorytextsbyGauss(DisquisitionesArithmeticae)andDirichlet(LecturesonNumberTheory)arestillofgreatvalue,butwouldbeofinteresttoseriousscholarsonly.Euler’sBooks.OurbiographyofEuleralsomentionshisfoundationalworksoncalculus.Althoughthey
arenotdirectlyrelatedtothetopicsofthisbook,theyarestillworthcarefulreading.Euler’sfirstbookonthesubject,IntroductiontoAnalysisoftheInfiniteisavailableinEnglish.Sadly,onlythefirsthalfofhissecondbook,FoundationsofDifferentialCalculus,hasanEnglishtranslation,andallofhisIntegralCalculusstillawaitsanEnglishtranslation.ThebookLetterstoaGermanPrincessisavailableontheInternet.
Chapter6GroupTheory.AclassicbookongrouptheoryisBurnside’sTheoryofGroupsofFiniteOrder.Thoughfirstpublishedin1897,itstillgivesanunparalleledintroductiontowhatgrouptheoryisaboutandincludesmanymoreexamplesthanmostmodernbooks.ItwasreprintedbyDoverPressin2004.ModelTheory.Sadly,wearenotawareofanintroductiontomodeltheoryaccessibletoalayperson.
Foramoreadvancedreader,agoodintroductionisH.JeromeKeisler’s“FundamentalsofModelTheory,”achapterintheHandbookofMathematicalLogic.
Chapter7RequirementsonTypes.ManytopicsinthischapterarediscussedmoreformallyinElementsofProgrammingbyStepanovandMcJones.Reduction.Iversondiscussesreductioninhispaper“NotationasaToolofThought”(1980).Theidea
isalsodiscussedinBackus’“CanProgrammingBeLiberatedfromtheVonNeumannStyle?”(1978).Usingreductionforparallelcomputationwasdiscussedin“OperatorsandAlgebraicStructures”byKapur,Musser,andStepanov(1981).Dean’suseofreductionisdescribedin“Map-Reduce:SimplifiedDataProcessingonLargeClusters”(2004).
Chapter8SimonStevin.DespiteStevin’sgreatcontributionstoscienceandmathematics,verylittlehasbeenpublishedabouthiswork.AgoodoverviewisSarton’s“SimonStevinofBruges.”PolynomialDivisionandGCD.ForarefresheronpolynomialdivisionandpolynomialGCD,see
Chapters5and6ofChrystal’sAlgebra.OriginsofAbstractAlgebra.AgoodintroductiontoGaussianintegersisChapter6ofStillwell’s
ElementsofNumberTheory.TheclassictextthatintroducedthegeneralnotionofalgebraicintegersisRichardDedekind’sTheoryofAlgebraicIntegers.Stillwell’stranslationincludesanexcellentintroductionthatexplainsmanyoftheideas.LeoCorry’sModernAlgebraandtheRiseofMathematicalStructuresisanexhaustivescholarlytreatmentoftheemergenceofabstractalgebrafromDedekindtoNoetherandlaterdevelopments.AbstractAlgebra.Forthereaderwhowantstotakethenextstepinunderstandingabstractalgebra,a
seriousbutaccessible(andhistoricallyinformed)textisStillwell’sElementsofAlgebra.Rings.Stillwell’sElementsofNumberTheorycoversthesetopicsthoroughlyandisquiteaccessible.
(WhileitstitlemightsuggestthatthistopiciscoveredinStillwell’sElementsofAlgebra,thatbookfocusesonGalois’sworkandthereforedoesnotcoverrings.)
Chapter9SocialNatureofProof.Theideathatproofisasocialprocessisdiscussedin“SocialProcessesandProofsofTheoremsandPrograms,”byDeMillo,Lipton,andPerlis(1979).Euclid.SirThomasHeath’stranslationofEuclid,TheThirteenBooksoftheElements,iswidely
available.Thiseditionincludesveryextensivecommentarybythetranslator.Inaddition,thereisnewreproductionofOliverByrne’s1847uniqueedition,whichdemonstratesalltheproofsthroughvisualillustrations.RobinHartshorne’sGeometry:EuclidandBeyondisatextbookaimedatuniversitymathmajors,butthefirstchapter(describingEuclid’sgeometry)isquiteaccessible.AxiomsofGeometry.Chapters1and2ofHartshorne’sGeometry:EuclidandBeyondprovideagood
introductiontoEuclid’saxiomaticmethodandHilbert’smodernversionofEuclideanaxioms.Hilbert’sFoundationsofGeometryisstillthedefinitivetreatmentofhisaxiomsforGeometry.Non-EuclideanGeometry.AclassictreatmentofthistopicisRobertoBonola’sNon-Euclidean
Geometry:ACriticalandHistoricalStudyofItsDevelopment.Amodern(butstillaccessible)mathematicaltreatmentmaybefoundinChapter7ofHartshorne’sGeometry.PeanoArithmetic.Forreadersinterestedinhowarithmeticcanbebuiltrigorouslyfromthegroundup,
EdmundLandau’sFoundationsofAnalysisdescribeshowtoconstructintegers,rationals,reals,andcomplexnumbersstartingwithPeano-likeaxioms.Peano’smagnumopusFormularioMathematicoactuallycoveredmanyareasofpracticalmathematics,notjusttheaxiomsthatbecamefamous.Unfortunately,thisgreatbookhasneverbeentranslatedfromitsoriginalinventedlanguage.FormoreaboutPeano’swork,seeTwelveArticlesonGiuseppePeanobyHubertKennedy.
Chapter10Aristotle’sOrganizationofKnowledge.AgoodintroductiontoAristotle’slifeandphilosophyisSirDavidRoss’sAristotle.ThereareseveralgoodeditionsofAristotle’scompleteworks,includingthetwo-volumeBollingenFoundationeditionandthemultivolumeLoebClassicalLibraryedition.ReadersinterestedonlyinAristotle’sCategories,theworkdiscussedinthischapter,canchoosetheappropriatevolume.Concepts.Chapter1ofElementsofProgrammingbyStepanovandMcJonescoversthismaterialmore
formallyandinmoredetail.IteratorsandSearch.Chapter6ofElementsofProgrammingcoversthismaterialmoreformallyand
inmoredetail.
Chapter11PermutationsandTranspositions.AgoodintroductiontopermutationsisinChapterXXIIIofChrystal’sAlgebra.Chapter1ofBurnside’sTheoryofGroupsofFiniteOrder,mentionedunderChapter6,givesmoredetailsaboutthesetopics.RotateandReverse.ThealgorithmsinthischapteraredescribedinmoredetailinChapter10of
ElementsofProgramming.
Chapter12Stein’sAlgorithm.Stein’soriginalpaperonthefasterGCDalgorithmis“ComputationalProblemsAssociatedwithRacahAlgebra.”KnuthdescribesthealgorithminSection4.5.2ofTheArtofComputerProgramming,Vol.2.RecreationalMathematics.Manydevelopmentsinmathematicscamefromstudyingseemingly
frivolousproblems,andmanydistinguishedmathematiciansbecameinterestedinmathematicsthroughexposuretomathematicalgames.AclassicbookonthesubjectisMathematicalRecreationsandEssaysbyW.W.RouseBall.
Chapter13Cryptography.AnentertaininghistoryofcryptographyisDavidKahn’sTheCode-breakers:TheComprehensiveHistoryofSecretCommunicationfromAncientTimestotheInternet.Tolearnmoreaboutmethodsusedinmoderncryptography,astandardtextisIntroductiontoModernCryptography:PrinciplesandProtocolsbyKatzandLindell.NumberTheory.Agoodmodernintroductiontonumbertheory,whichincludesadiscussionoftheRSA
algorithm,isJohnStillwell’sElementsofNumberTheory(2003).ItalsoincludessomematerialthatwecoverinChapters5and8.AKSPrimalityTesting.Thedeterministicpolynomial-timealgorithmforprimalitytestingisdescribed
inGranville’spaper“ItIsEasytoDetermineWhetheraGivenIntegerIsPrime”(2005).
AppendixA.Notation
Thefollowingaresymbolsusedinthebookthatmaynotbefamiliartoanonmathematicalreaderandthatarenotexplainedinthemaintext.(Othernewsymbolsareexplainedwhentheyareintroduced.)We’lllistthesymbolsfirst,thengiveafewexamplesoftheiruse.¬pLogicalnegation.Read“notp.”Ifpistrue,then¬pisfalse,andviceversa.
p∨qLogicaldisjunction.Read“porq.”Thestatementp∨qistrueifeitherpistrue,orqistrue,ortheyarebothtrue.
p∧qLogicalconjunction.Read“pandq.”Thestatementp∧qistrueonlywhenbothpandqaretrue.
p qLogicalimplication.Read“pimpliesq”or“ifp,thenq.”Notethatthestatementp qisfalseonlywhenpistrueandqisfalse.Itmaynotbeintuitivethattheexpressionistruewhenpisfalse,butonewaytothinkofitis“youcanmakeanargumentforanythingifyougettostartbyassumingsomethingthatisn’ttrue.”Formoreaboutlogicalimplication,see“ImplicationandtheContrapositive”attheendofthisappendix.
p qLogicalequivalence.Read“pifandonlyifq”andsometimeswritten“piffq.”Thestatementp q
istruewhenbothpandqaretrue,orwhenbothpandqarefalse.Thisisexactlythesameassaying(pq)∧(q p).
x SSetmembership.Read“xisanelementofS”or“xisinS.”
x∉SNegationofsetmembership.Read“xisnotanelementofS”or“xisnotinS.”
∀x SUniversalquantifier.Read“forallxinS”or“foranyxinS.”SometimesmembershipinthesetSisassumedfromthecontext,sowejustwrite∀x.
∃x SExistentialquantifier.Read“thereexistsanxinS”or“thereisanxinS.”SometimesmembershipinthesetSisassumedfromthecontext,sowejustwrite∃x.
S∪TSetunion.Read“theunionofSandT.”AnelementxisintheunionofSandTifitisineitherSorTorboth.
S∩T
Setintersection.Read“theintersectionofSandT.”AnelementxisintheintersectionofSandTifitisinbothSandT.
S={x|...}Setdefinition.Read“Sisthesetofallxsuchthat...”(wherethe“...”couldbeanylistofconditionsaboutx).
Thesetofnaturalnumbers0,1,2,3,...—numbersusedforcounting.(Someauthorsdonotinclude0inthesetofnaturalnumbers.)
Thesetofintegers,whichincludesallthenaturalnumbersand(forallexceptzero)theirnegations.
Theset{0,1,2,...,n–1}ofremaindersmodulon.
Theset ofrationalnumbers(theratiooftwointegers).
Thesetofrealnumbers.
Thesetofcomplexnumbersa+bi,whereaandbarerealnumbersandi2=–1.
Exampleseven(x) ¬odd(x)“xisevenifandonlyifxisnotodd.”
S={x|x ∧even(x)}“Sisthesetofallx’ssuchthatxisinthesetofintegersandxiseven”or,moreconcisely,“Sisthesetofallevenintegers.”
∀x∃y:y=x+1“Foranyx,thereisaysuchthatyequalsxplus1.”
x {S∩T} x {S∪T}“IfxisintheintersectionofSandT,thenxisintheunionofSandT.”
ImplicationandtheContrapositiveTheimplicationp q(alsoknownasaconditional)islogicallyequivalenttoavariantcalledthecontrapositive,whichhasthefollowingform:
¬q ¬p
Considerthisexample:
Ifn=2,thenniseven.
Hereourpropositionpis“n=2”andourpropositionqis“niseven”;thisconditionalhappenstobetrue.Toformthecontrapositive,welogicallynegatebothsidesandreversethedirectionoftheimplication.Sothecontrapositiveoftheprecedingstatementis
Ifnisnoteven,thenn≠2
whichisalsotrue.Sinceaconditionalstatementanditscontrapositivearelogicallyequivalent,weoccasionallyreplace
onewiththeother,insituationswherethelatterformismoreconvenient.Don’tconfusethecontrapositiveofp qwithitsconverse,whichisq p.Justbecausean
implicationistruedoesnotmeanitsconverseistrue;thetwoareindependent.Continuingtheearlierexample,eventhoughouroriginalstatementwastrue,itsconverse
Ifniseven,thenn=2
isclearlyfalse.
AppendixB.CommonProofTechniques
Thereareafewstandardprooftechniquesthatoccurfrequentlyinmathematicsandcomputerscience,andwhichweuseinthisbook.Ifyouarehavingtroubleunderstandingtheproofsinthemaintext,youmaywanttoreviewthissection.
B.1ProofbyContradictionManythingswewanttoprovehavetheform“ifp,thenq”(alsosometimeswritten“p q”),wherepandqaretwopropositions.Wealwaysstartwiththepremisethatpistrue;otherwise,wewouldbesolvingadifferentproblem.Theideaofproofbycontradictionistoassumetheoppositeofwhattheoriginalconjectureconcludes(i.e.,assumethatqisnottrue),andthenshowthatthisassumptionwouldleadtoalogicalcontradiction—inparticular,thatpropositionpwouldbefalse,whichweknowisnotthecase.Thisforcesustoconcludethatpropositionqmustbetrueafterall,whichiswhatwereallywantedallalong.Let’slookatanexample.Supposewewanttoprovethatforallintegersn:
Ifn2isodd,thennisodd.
Here“n2isodd”isourpand“nisodd”isourq.Solet’sassumetheoppositeconclusionistrue,thatnisnotodd—thatniseven.Whatdoesitmeanforanintegerntobeeven?Itmeanswecanwriteitastwicesomeotherintegerm:
n=2m
Whathappensifwesquaren?
n2=2·2·m2
Let’sintroduceanewvariablex,andsetx=2m2.Thenwecansubstitute:
n2=2·2m2=2x
Nowweseethatn2canbeexpressedastwicesomeotherintegerx.Butthat’sthedefinitionofeven,andourpremisewasthatn2isodd.n2can’tbebothevenandodd—that’salogicalcontradiction.Sotheassumptionwemadeatthebeginningthatnisevenmustbefalse;nmustthereforebeodd,andwe’veprovedtheoriginalresult.
B.2ProofbyInductionSomeresultswe’dliketoproveinvolveinfinitesetsofthings.Obviouslyinthesesituationswecan’tenumerateallthecases,butwecanoftenusemathematicalinductiontoobtainourresult.Toprovesomethingbyinduction,youneedtodotwothings:
•Provethatit’strueforthefirstelementintheset.Thisiscalledthebasis.•Provethatifit’strueforanarbitraryelementintheset(theinductionhypothesis),thenit’salsotrueforthesuccessorofthatelement.Thisiscalledtheinductivestep.
Forexample,supposewewanttoprovethatforanypositiveintegern:
Basis:Doestheequationholdifn=1?Inotherwords,isittruethat
Wecanjustdothearithmetic,andseethattheansweris“yes.”Inductivestep:
Assumethattheequationistrueforn=k.Ifthatweretrue,woulditalsobetruefork+1?Thisiswhatitmeanstobetruefork(i.e.,thisisourinductionhypothesis):
Let’saddk+1tobothsides:
Thisisjust ,wheren=k+1.Sowehaveprovedthatiftheequationistruefork,it’struefork+1.
Sincewehaveshownboththebasisandtheinductivestep,wehaveprovedouroriginalstatement.
B.3ThePigeonholePrincipleThepigeonholeprinciple(sometimesknownastheDirichletprinciple)isverysimple:ifyouhavenpigeonholesandmorethannpigeons,thenatleastonepigeonholemustcontainmorethanonepigeon.Therearelotsofexamplesofthisinreallife.Forexample,ifyouhave367people,atleasttwoofthemmusthavethesamebirthday.Buttheideaalsoturnsouttobeusefulinsomemathematicalproofs.Oftenwhenyouseeatheoremthat’stryingtoprovethattwothingswillbethesame,thepigeonholeprincipleisagoodapproach.Here’sanexample:
Provethatanysetof10positiveintegerssmallerthan100willalwayscontaintwosubsetswiththesamesum.
First,let’sconsiderhowmanypossiblesumswecanget.Thesumofany10positiveintegerslessthan100can’tbesmallerthan10orlargerthan990,soitmustbesomewhereintherange[10,990].Thatrangecontains990−10=980values,sothat’sthemaximumnumberofpossiblesums.1Next,let’sseehowmanypossiblesubsetsofthose10integersthereare.Wecanrepresenteachsubsetasabinarynumberwheretheithbitis1iftheithintegerinthesetisinthatsubset,and0otherwise.Thereare10
elementsintheset,andweuseonebitforeachelement,sothereare210=1024possiblesubsets.Sincethereareonly980possiblesums,andthereare1024possiblesubsets,bythepigeonholeprinciple,atleasttwoofthesubsetsmusthavethesamesum.
1Actually,theproblemsaysthatwehaveasetof10integers,andasetmaynotcontainrepeatedelements.Sotheactualnumberofpossiblesumsissmallerthan980,buttheresultoftheproofisthesame.
AppendixC.C++forNon-C++Programmers
ThisbookgenerallyusesasubsetofC++thatshouldbeeasilyunderstandabletomostprogrammerswhohaveusedalanguagelikeCorJava.However,thereareafewimportantfeaturesandidiomsspecifictoC++thatwerelyon.ThesearedescribedinthisAppendix.Exceptwherenoted,weuseonlyfeaturesofC++thathavebeenavailableinthe1998standardversionofthelanguage.ForagoodbriefintroductiontoC++11,seeATourofC++byBjarneStroustrup.Foracompletereference,seeStroustrup’sTheC++ProgrammingLanguage.
C.1TemplateFunctionsOnewayC++supportsthegenericprogrammingparadigmisthroughtheuseoftemplates.Supposeyouhaveafunctionlikethis:
intmy_function(intx){
inty;
...dosomethingcomplicated...
returny;
}
Nowyouwanttodothesamesetofcomputations,butthistimeyouwantthefunctiontotakeandreturnadouble-precisionfloating-pointnumber.C++allowsyoutooverloadthefunctionname,soyoucanwriteawholenewfunctionwiththesamenamethatworkswithdifferenttypes:
doublemy_function(doublex){
doubley;
...dosomethingcomplicated...
returny;
}
Butifeverythingisthesameexceptforthetype,writingawholeseparatefunctioniswasteful.Templatesavoidthisproblem.Withtemplates,youcanwriteasinglefunctiontoworkonanytypethat
satisfiesboththesyntacticandsemanticrequirementsofthecode,likethis:template<typenameT>
Tmy_function(Tx){
Ty;
...dosomethingcomplicated...
returny;
}
NowwehaveafunctionthattakestypeTandreturnstypeT,whereTdependsonhowthefunctioniscalled.Ontheonehand,ifyousay
intx(1);
inty=my_function(x);
thenmy_function()willbecalledwithTsettoint.Ontheotherhand,ifyousaydoublex(1.0);
doubley=my_function(x);
thenmy_function()willbecalledwithTsettodouble.Thisisdoneatcompiletime,sothereisnoperformancepenaltyforusingtemplates.
C.2ConceptsConceptsaretheessentialpartofgenericprogramming,andwediscusstheminsomedetailinSection10.3.Thefollowingdiscussionisintendedtobeaquickreference.Ideally,wewouldliketohaveawaytotelltheprogrammerwhattherequirementsareonagiven
templateargument.Forinstance,we’dliketosaytemplate<NumberN>
andhavethatmeanthatwhatevertypethisfunctiongetscalledwithmustbeanumber.Thismeansthecodeisintendedtoworkforthingsintsanddoublesanduint64_ts,butnot,say,forstrings.Arestrictionlike“Number”isanexampleofaconcept.Unfortunately,asofthiswriting,C++doesnotsupportconceptsasabuilt-inpartofthelanguage—thatis,C++doesnothaveanywaytoenforcerequirementsontemplatetypes.Despitethislimitation,wewillwriteourcodeexamplesasifconceptswerepresentinthelanguage.
Wecanimplementthisbyjustdefiningourfavoriteconceptsasaliasesfortypename:#defineNumbertypename
Sowhenwewritetemplate<NumberN>
asfarasthecompilerisconcerned,it’sasifwewrotetemplate<typenameN>
butthehumanprogrammerwillunderstandtheintendedrestriction.
C.3DeclarationSyntaxandTypedConstantsC++providesmultiplewaystodeclareandinitializeavariable.WhilethetraditionalCsyntax
intx=y;
islegal,thecommonwaytowritethisinC++isintx(y);
ThisismoreconsistentwiththesyntaxusedtoconstructarbitraryC++objects.(Note:ThecurrentversionofC++supportsanadditionalwaytodoinitialization:
intx{y};
However,thisusageisstilllesswidespread,sowedonotuseitinourexamples.)Whenusingnumericconstants,we’llbeverycarefulabouttypes.Forexample,atraditionalCprogram
mightcontainalinelikethis:if(something)return0;
Thisisabitsloppy.Whichtypeisthe0returnedbythefunction?Bydefault,it’sanint,butsupposeourprogramwassupposedtoreturnaspecifickindofinteger,likea16-bitunsignedinteger,oronespecifiedbyatemplateargument.Ratherthanrelyonimplicittypeconversion,we’lltrytobeexplicitaboutwhatwe’rereturningbywritingsomethinglikethis:
if(something)returnuint16_t(0);
or,inthecaseofatemplateargument:if(something)returnT(0);
whereTisthetypespecifiedinthetemplate.
C.4FunctionObjectsOftenwe’dliketohaveafunctionthatrequiressomeinitializationandmaintainssomestate.AcommonwaytoimplementthisinC++isbytheuseofafunctionobject,alsocalledafunctor.Afunctionobjectisanobjectthatencapsulatesasingle(unnamed)function.Let’slookatasimpleexample—afunctionobjectfordoingcurrencyconversion:Clickheretoviewcodeimage
structconverter{
doubleexchange_rate;
converter(doubleex):exchange_rate(ex){}
doubleoperator()(doubleamt){
returnamt*exchange_rate;
}
};
Notetheuseofthesyntaxoperator()todeclaretheunnamedfunctionthatbelongstotheobject.Tousethisfunction,wefirstconstructaninstanceofourconverterobject(whichinitializesthe
exchangerate).InthisexamplewewanttoconverteurostoU.S.dollars,sowe’llnameourinstanceeur_to_usd.Thenwecaninvokethefunctionbyusingthatinstance:Clickheretoviewcodeimage
intmain(){
convertereur_to_usd(1.3043);
doubleeuros;
do{
std::cout<<"EnteramountinEuros:";
std::cin>>euros;
std::cout<<euros<<"eurosis"
<<eur_to_usd(euros)<<"dollars"
<<std::endl;
}while(euros>0.0);
}
Functionobjectshavethebenefitthatwecanpassthemasargumentstofunctions.(C++doesnotallowpassingfunctionsdirectly,onlyfunctionpointers,whichrequirestheaddedcostofanindirectfunctioncall.)Inaddition,functionobjectscancontainstateinformation.
C.5Preconditions,Postconditions,andAssertionsGivenvalidarguments,afunctionperformsacertaincomputation.Anotherwaytoputthisisthatifitspreconditionsaresatisfied,certainpostconditionswillbetrue.Sometimeswewritethesepreconditionsandpostconditionsascommentsinourcode,likethis:Clickheretoviewcodeimage
//precondition:y!=0.0
doublemy_ratio(doublex,doubley){
returnx/y;
}
//postcondition:returnedvalueisx/y
However,thelibraryalsoprovidesamechanismcalledassertforcheckingsomeconditions.Sowecouldwrite:Clickheretoviewcodeimage
doublemy_ratio(doublex,doubley){
assert(y!=0.0);
returnx/y;
}
Iftheassertexpressionevaluatestotrue,nothinghappens.Butifitevaluatestofalse,executionoftheprogramhaltsandanerrormessageisprinted.Inproductioncode,assertionsaretypicallydisabledtoavoidaperformancepenalty.
C.6STLAlgorithmsandDataStructuresTheC++languagecontainsalibraryofstandardsoftwarecomponents,knownastheStandardTemplateLibrary(STL).Thislibraryincludesdatastructures,algorithms,andotherutilitiescommonlyusedbyC++programmers.AllSTLcomponentsbelongtothenamespacestd;wewillexplicitlyusetheprefixstd::whenwerefertotheminourcodeexamples.STLisagenericlibrary,meaningthateachcomponentcanbeusedwithanyappropriatetype.Inthe
caseofdatastructures,thetypesarespecifiedastemplateargumentswhentheobjectisdeclared.Forexample,
std::vector<int>v;
declaresavectorofintegers,whilestd::vector<bool>v;
declaresavectorofBooleans.ThefollowingSTLcomponentsareusedinthisbook:
Functionobjectsforarithmeticoperationsandcomparisons(seeSectionC.4forexplanationoffunctionobjects):
•std::plus—Computesthesumofitsoperator’stwoarguments.•std::multiplies—Computestheproductofitsoperator’stwoarguments.•std::negate—Computesthenegationofitsoperator’sargument.•std::less—Returnstruewhenitsoperator’sfirstargumentislessthanitssecond,falseotherwise.•std:less_equal—Returnstruewhenitsoperator’sfirstargumentislessthanorequaltoitssecond,falseotherwise.
Datastructures:•std::pair—Astructthatstorestwoarbitraryobjects;typicallyusedtoreturntwothingsfromafunction.•std::vector—Acontainerforasequenceofelementsofasingletypethatsupportsconstant-timerandomaccess.
Algorithms:•std::fill—Fillstherangespecifiedbyitsfirsttwoargumentswiththevaluespecifiedbythethirdargument.•std::swap—Exchangesthecontentsofitsarguments.•std::partition_point—Returnsaniteratortothefirstelementinanalreadypartitionedrange(specifiedbythefirsttwoarguments)forwhichthegivenpredicate(thirdargument)isnottrue.SeediscussioninSection10.8.
Otherutilities:•std::advance—Incrementsthepositionofaniterator(itsfirstargument)byadistance(secondargument).
ForamoredetaileddescriptionoftheseandotherSTLcomponents,seePartIVofStroustrup’sTheC++ProgrammingLanguage.
C.7IteratorsandRangesIteratorsareanimportantpartofgenericprogramming,andwediscussthemingreaterdetailinSection10.4.Thefollowingdiscussionisintendedtobeaquickreference.Iteratorsareanabstractionofpointers;aniteratorindicatesapositioninasequence.Theexamplesin
thisbookusethesefourtypesofiterators,eachwithitsowniteratortag.•Inputiteratorssupportone-directionaltraversal,butonlyonce,asisfoundinsingle-passalgorithms.Thecanonicalmodelofaninputiteratoristhepositioninaninputstream.Tag:std::input_iterator_tag•Forwarditeratorsalsosupportonlyone-directionaltraversal,butthistraversalcanberepeatedasneeded,asinmulti-passalgorithms.Thecanonicalmodelofaforwarditeratoristhepositioninasinglylinkedlist.Tag:std::forward_iterator_tag•Bidirectionaliteratorssupportbidirectionaltraversal,repeatedasneeded(i.e.,theyalsocanbeusedinmulti-passalgorithms).Thecanonicalmodelofabidirectionaliteratoristhepositioninadoublylinkedlist.Tag:std::bidirectional_iterator_tag•Random-accessiteratorssupportrandom-accessalgorithms;thatis,theyallowaccesstoanyelementinconstanttime.Thecanonicalmodelisthepositioninanarray.Tag:std::random_access_iterator_tag
Theiteratortagsarespecialtypesthatmaybeusedinfunctionsignaturestoensurethatthecorrectversionofanoverloadedfunctionwillbeinvokedwhenagiveniteratorisused;seeChapter11foranexample.
***
STLfunctionsoftentaketwoiteratorsrepresentingthebeginningandendofarangeofdata.Byconvention,theiteratorfortheendofthedata(oftencalledlast)pointstothepositiondirectlyafterthelastelement.Iteratorsalsohavespecialattributescallediteratortraits.Theonesweuseare:•value_type:thetypeoftheobjectspointedtobytheiterator.
•difference_type:anintegraltypelargeenoughtoexpressthenumberofincrementoperationsneededtogetfromoneiteratortoanother.•iterator_category:theiteratortag,describedearlier.
Thesyntaxtoaccessaniteratortraitforaparticulariteratortypexis(forexample)std::iterator_traits<x>::value_type
Formoreinformationoniterators,seeChapter10.
C.8TypeAliasesandTypeFunctionswithusinginC++11C++11,thecurrentstandardversionofC++,hasafeaturecalledusingthatallowsprogrammerstoprovidealiasesfortypesandotherconstructs.Thisistypicallyusedtoprovideashortwaytowritealongandcomplicatedtype.Forexample:Clickheretoviewcodeimage
usingmyptr=long_complicated_class_name*;
Afterwritingthisstatement,programmerscouldrefertomyptrinthecodewherevertheywouldhavepreviouslywrittenlong_complicated_class_name*.UsersofCandearlierversionsofC++maybefamiliarwithanolderaliasingmechanism,typedef,
butusingismoreflexible.Forexample,theusingfeatureallowsustowritetemplatizedtypefunctionsforiteratortraits.IfwewriteClickheretoviewcodeimage
template<InputIteratorI>
usingIteratorCategory=
typenamestd::iterator_traits<I>::iterator_category;
theneverytimewewanttoknowthecategoryofaniterator,wecansayIteratorCategory<I>
ratherthanClickheretoviewcodeimage
std::iterator_traits<I>::iterator_category
C.9InitializerListsinC++11InCandC++,youcanconvenientlyinitializeanarraybyenclosingthelistofvaluesincurlybraces,likethis:Clickheretoviewcodeimage
charmy_array[5]={'a','e','i','o','u'};
C++11extendsthissyntaxbeyondarrays,sonowyoucanalsowritethingslikethis:Clickheretoviewcodeimage
std::vector<char>={'a','e','i','o','u'};
std::pair<int,double>={42,3.1415};
C.10LambdaFunctionsinC++11C++11includessupportforlambdafunctions.Theseareanonymousfunctionsthatareneededonlyonce,oftenasargumentstoanotherfunction.Supposeforsomeapplication,wewantedtotakeafunctionthatcomputedthecubeofitsargumentand
passittoanotherfunction.Traditionally,we’dhavetoimplementthisasafunctionobject,declareitseparately,instantiateit,andpasstheinstance,likethis:Clickheretoviewcodeimage
structcuber{
cuber(){};//constructor
intoperator()(intx){
returnx*x*x;
}
};
intmain(){
...
cubercube;
inta=some_other_function(cube);
...
}
Butifthisistheonlytimeweeverneedthecubefunction,that’salotofwork.Whybothercreatingthefunctionobject,orevengivingthefunctionaname,whenwe’renevergoingtouseitagain?Instead,wecanwritealambdafunctioninlineandpassthewholeexpressionasanargument:Clickheretoviewcodeimage
intmain(){
...
inta=some_other_function([=](intx){returnx*x*x;});
...
}
Thesyntaxforlambdafunctionsisjustlikethesyntaxforimplementinganyotherfunction,exceptthatthenameofthefunctionisreplacedbytheexpression[=],andthereturntypeusuallydoesnotneedtobespecified;thecompilerfiguresitout.
C.11ANoteaboutinlineTheC++directiveinlinebeforeafunctionisahintthattellsthecompilerthattheprogrammerwouldlikethebodyofthefunctiontobeincludedaspartofthecodeofthecaller,avoidingtheusualfunctioncalloverhead.Inpractice,manyfunctionsinthisbookwouldbenefittodayfrombeingdeclaredinline.Inliningmakessenseonlyforrelativelysmallpiecesofcode.Largerinlinedfunctionscouldendup
increasingthesizeofthecallingcodeenoughtodisruptcodecachingorcauseotherperformanceproblems.Compilerstakethisintoaccountandignoretheinlinerequestinthosecases.Atthesametime,compilersaregettingsmartenoughtoinlinecodeautomaticallywhenitmakessense.Forthesereasons,theinlinedirectivewillsoonbeobsolete,sowedidnotuseitinourexamples.
Bibliography
Adler,MortimerJ.(Ed.).(1991).GreatBooksoftheWesternWorld,Vol.10:Euclid,Archimedes,Nicomachus.Chicago:EncyclopaediaBrittanica.
Aristotle.(1938).Aristotle:Categories,OnInterpretation,PriorAnalytics,Vol.325.TranslatedbyH.P.CookeandHughTredennick.Cambridge,MA:LoebClassicalLibrary.
Aristotle.(1984).TheCompleteWorksofAristotle:TheRevisedOxfordTranslation.EditedbyJonathanBarnes.Princeton,NJ:PrincetonUniversityPress.
Backus,John.(1978).“CanProgrammingBeLiberatedfromtheVonNeumannStyle?:AFunctionalStyleandItsAlgebraofPrograms.”CommunicationsoftheACM21(8),613-641.
Ball,W.W.Rouse,andH.S.M.Coxeter.([1922]2010).MathematicalRecreationsandEssays(10thed.).Reprint,NewYork:DoverPublications.Originaleditionpublished1892.
Bonola,Roberto.([1955]2010).Non-EuclideanGeometry:ACriticalandHistoricalStudyofItsDevelopment.TranslatedbyH.S.Carslaw.Reprint,NewYork:DoverPublications.OriginallypublishedasLaGeometrianon-Euclidea,1912.
Burnside,William.([1911]2004).TheoryofGroupsofFiniteOrder(2nded.).Reprint,Mineola,NY:DoverPublications.
Byrne,Oliver.(2010).TheFirstSixBooksoftheElementsofEuclid.Taschen.Facsimileof1847edition.
Chrystal,George.([1964]1999).Algebra:AnElementaryText-Book(7thed.).Reprint,Providence,RI:AmericanMathematicalSociety.Originaleditionpublished1886.
Cohen,MorrisR.,andI.E.Drabkin.(1948).ASourceBookinGreekScience.Cambridge,MA:HarvardUniversityPress.
Corry,Leo.(2004).ModernAlgebraandtheRiseofMathematicalStructures(2ndreviseded.).Basel,Switzerland:Birkhäuser.
Dean,Jeffrey,andSanjayGhemawat.(2008).“MapReduce:SimplifiedDataProcessingonLargeClusters.”CommunicationsoftheACM51(1),107-113.
Dedekind,Richard.(1996).TheoryofAlgebraicIntegers.TranslatedbyJohnStillwell.Cambridge,UK:CambridgeUniversityPress.OriginallypublishedasÜberdieTheoriederganzenalgebraicschenZahlen,1877.
DeMillo,RichardA.,RichardJ.Lipton,andAlanJ.Perlis.(1979).“SocialProcessesandProofsofTheoremsandPrograms.”CommunicationsoftheACM22(5),271-280.
Dirichlet,P.G.L.(1999).LecturesonNumberTheory.SupplementsbyR.Dedekind.TranslatedbyJohnStillwell.Providence,RI:AmericanMathematicalSociety.OriginallypublishedasVorlesungenüberZahlentheorie,1863.
Euclid.(1956).Euclid:TheThirteenBooksoftheElements.TranslatedbyThomasL.Heath.(2nded.).NewYork:DoverPublications.
Euler,Leonhard.(1988).IntroductiontoAnalysisoftheInfinite,Vol.1and2.TranslatedbyJohnD.Blanton.NewYork:Springer.OriginallypublishedasIntroductioinanalysininfinitorum,1748.
Euler,Leonhard.(2000).FoundationsofDifferentialCalculus.TranslatedbyJohnD.Blanton.New
York:Springer.OriginallypublishedasInstitutionesCalculiDifferentialis,1755.Fibonacci,LeonardoPisano,andL.E.Sigler(Trans.).(1987).TheBookofSquares:AnAnnotatedTranslationintoModernEnglish.Boston:AcademicPress.OriginallypublishedinLatinasLiberQuadratorum,1225.
Floyd,RobertW.,andDonaldE.Knuth.(1990).“AdditionMachines.”SIAMJournalonComputing19(2),329-340.
Fowler,DavidH.(1987).TheMathematicsofPlato’sAcademy:ANewReconstruction.Oxford,UK:ClarendonPress.
Gauss,CarlFriedrich.(1965).DisquisitionesArithmeticae.TranslatedbyArthurA.Clarke,S.J.NewHaven,CT:YaleUniversityPress.OriginalLatinedition,1801.
Granville,Andrew.(2005).“ItIsEasytoDeterminewhetheraGivenIntegerIsPrime.”BulletinoftheAmericanMathematicalSociety42(1),3-38.
Gries,David,andGaryLevin.(1980).“ComputingFibonacciNumbers(andSimilarlyDefinedFunctions)inLogTime.”InformationProcessingLetters11(2),68-69.
Grimm,RichardE.(1973).“TheAutobiographyofLeonardoPisano.”FibonacciQuarterly11(1),99-104.
Hartshorne,Robin.(2000).Geometry:EuclidandBeyond.NewYork:Springer.Heath,Thomas.([1921]1981).AHistoryofGreekMathematics.Reprint,NewYork:DoverPublications.
Hilbert,David.([1971]1999).FoundationsofGeometry(10thed.).TranslatedbyLeoUngerandrevisedbyPaulBernays.Chicago:OpenCourt.OriginallypublishedasGrundlagenderGeometrie,1899.
Iverson,KennethE.(1962).“AProgrammingLanguage.”InProceedingsoftheMay1-3,1962,SpringJointComputerConference,AIEE-IRE’62,pp.345–351.ACM.
Iverson,KennethE.(1980).“NotationAsaToolofThought.”CommunicationsoftheACM35(1-2),2-31.
Kahn,David.(1996).TheCodebreakers:TheComprehensiveHistoryofSecretCommunicationfromAncientTimestotheInternet(Reviseded.).NewYork:Scribner.
Kapur,D.,D.R.Musser,andA.A.Stepanov.(1981a).“OperatorsandAlgebraicStructures.”InProceedingsofthe1981ConferenceonFunctionalProgrammingLanguagesandComputerArchitecture,FPCA’81,NewYork,NY,pp.59-64.ACM.
Kapur,D.,D.R.Musser,andA.A.Stepanov.(1981b).“Tecton:ALanguageforManipulatingGenericObjects.”InProgramSpecification,ProceedingsofaWorkshop,pp.402-414.Springer-Verlag.
Katz,Jonathan,andYehudaLindell.(2008).IntroductiontoModernCryptography.BocaRaton,FL:CRCPress.
Katz,VictorJ.(2009).AHistoryofMathematics:AnIntroduction(3rded.).Boston:Addison-Wesley.Keisler,H.Jerome.(1989).“FundamentalsofModelTheory.”InJ.Barwise(Ed.),HandbookofMathematicalLogic.NorthHolland.
Kennedy,Hubert.(2002).TwelveArticlesonGiuseppePeano.SanFrancisco:PeremptoryPublications.Knuth,DonaldE.(2007).TheArtofComputerProgramming,Vol.2:SeminumericalAlgorithms.
Boston:Addison-Wesley.Landau,Edmund.([1966]2001).FoundationsofAnalysis(3rded.).TranslatedbyF.Steinhardt.Reprint,Providence,RI:Chelsea.
McClenon,R.B.(1919).“LeonardoofPisaandHisLiberQuadratorum.”TheAmericanMathematicalMonthly26(1),1-8.
Musser,DavidR.,andAlexanderA.Stepanov.(1988).“GenericProgramming.”InProceedingsoftheInternationalSymposiumISSAC’88onSymbolicandAlgebraicComputation,pp.13-25.Springer-Verlag.
Peano,Giuseppe.(1960).FormularioMathematico.EdizioniCremonense.Originaleditionpublished1908.
Plato.(1997).Plato:CompleteWorks.EditedbyJ.M.CooperandD.S.Hutchinson.Indianapolis,IN:HackettPublishing.
Robins,Gay,andCharlesShute.(1987).TheRhindMathematicalPapyrus:AnAncientEgyptianText.London:BritishMuseumPublications.
Ross,David.(2004).Aristotle(6thed.).London:Routledge.Russo,Lucio.(2004).TheForgottenRevolution:HowScienceWasBornin300BCandWhyItHadtoBeReborn.TranslatedbySilvioLevy.NewYork:Springer.OriginallypublishedasLarivoluzionedimenticata,1996.
Sarton,George.(1934).“SimonStevinofBruges.”Isis21(2),241-303.Sigler,Laurence.(1987).Fibonacci’sLiberAbaci:ATranslationintoModernEnglishofLeonardoPisano’sBookofCalculation.NewYork:Springer.OriginalLatinedition,1202.
Stein,Josef.(1967).“ComputationalProblemsAssociatedwithRacahAlgebra.”JournalofComputationalPhysics1(3),397-405.
Stepanov,Alexander,andMengLee.(1995).TheStandardTemplateLibrary.Hewlett-PackardLaboratories,TechnicalPublicationsDepartment.
Stepanov,Alexander,andPaulMcJones.(2009).ElementsofProgramming.Boston:Addison-WesleyProfessional.
Stillwell,John.(1994).ElementsofAlgebra.NewYork:Springer.Stillwell,John.(2002).ElementsofNumberTheory.NewYork:Springer.Stillwell,John.(2010).MathematicsandItsHistory.NewYork:Springer.Stroustrup,Bjarne.(2013a).TheC++ProgrammingLanguage(4thed.).Boston:Addison-WesleyProfessional.
Stroustrup,Bjarne.(2013b).ATourofC++.Boston:Addison-WesleyProfessional.vanderWaerden,B.L.(1983).GeometryandAlgebrainAncientCivilizations.Berlin:Springer-Verlag.
vanderWaerden,B.L.(1988).ScienceAwakening:Egyptian,Babylonian,andGreekMathematics.TranslatedbyArnoldDresden.Dordrecht,Netherlands:KluwerAcademicPublishers.
Weil,André.(2007).NumberTheory:AnApproachthroughHistoryfromHammurapitoLegendre.Cambridge,MA:BirkhäuserBoston.
Index
*(operator),mathematicalconventionfor,115+(plussign),mathematicalconventionfor,115α.SeeAliquotsumϕ.SeeEulertotientfunctionσ(sumofdivisors)formula,31
AAbeliangroup,86,108,153Abstractalgebra
birthof,85,140–145Euclideandomains,150–151,153fields,151–153groups,85–88,92–95,108,152ideals,226–228modules,151,154monoids,89,108–109,152,154andprogramming,2,141,249principalideals,227–228rings,142–145,153semigroups,90–91,108–109,152semirings,145–149,153vectorspaces,152,154
AbstractionAristotle,177,180,196inmathematics,84,85–109andprogramming,2,5,249
Academy(Plato’s),41–44,178Addition
associativityof,9,156,174commutativityof,155–156,174–175definition,173
Additionchains,11Additivegroups,86Additivemonoids,89,109Additivesemigroups,90,109Address,181Adleman,Len,239advance,190Agrawal,Manindra,244
Ahmes,8–9,57Ahmesalgorithm.SeeEgyptianmultiplication,EgyptiandivisionAKSprimalitytest,244–245AlexandertheGreat,43,178–179Alexandria,43–44Algebraicintegers,140Algebraicstructures,85.SeealsoAbstractalgebraAlgorithms
inancientEgypt,7–11definition,7domainorsetting,150firstrecorded,8generalizing,111,119–123,126–127,151historyof,7–11in-place,215–216memoryadaptive,216–217polylogspace,215–216spacecomplexity,215–216performanceinpractice,211
Aliases,272Aliquotsum,31Amicablenumbers,63–64AnalyticalMechanics,99APL,124Apology,43Archimedes
onacquiringmathematicalknowledge,176axiomof,47placeinhistory,50
Aristophanes,42Aristotle,17,177–180Aristoxenus,17Arithmétique,132TheArtofComputerProgramming,9Aryabhata,51Aryabhatiya,51Assertions,269Associativebinaryoperation( ),108
ingroups,85–86inmonoids,89insemigroups,90
Associativityaxiom,semigroups,91
Associativityofaddition,9,113definition,174visualproof,156
Associativityofmultiplication,visualproof,157Asymmetrickeys,238Athens,41–43Automorphism,104Averroes.SeeIbnRushdAxiomofArchimedes,47Axiomaticmethod,161–162Axioms
definition,163Euclid’s,162–163Hilbert’s,167Peano’s,170–171
BBachetdeMéziriac,ClaudeGaspar,67,235
profile,225–226Backus,John,124Bacon,Roger,1,249Bartels,Martin,165Bernoulli,Johann,69Bézout’sidentity,225–229Bidirectionaliterators,185Binarysearch,191–196.SeealsoPartitionpointsBinarysearchlemma,194–195BletchleyPark,238Bolyai,Farkas,166Bolyai,János,166Bolzano-CauchyTheorem.SeeIVT(IntermediateValueTheorem)Booleansemirings,148Boundedranges,189,203–204bsearch,192
CC++,3,265–273C++11,57,187,195,265,272–273TheC++ProgrammingLanguage,265,270C++StandardTemplateLibrary.SeeSTLCaesarcipher,237Cancellation
CancellationLaw,74–75definition,72–73inversenumbers,73andmodulararithmetic,72–76Self-CancelingLaw,75–76
CancellationLaw,74–75Carmichaelnumbers,242Cartesiancoordinates,131,138Cataldi,Peter,64Categoricaltheories
vs.STL,104definition,104examplesof,104–106
Categorydispatch,188,190,196,213Cayley’sTheorem,198Chinesemathematics,51Chrystal,George,34Cicero,50Ciphertext,238Closedranges,188Clouds,42Cocks,Clifford,240Codes,definition,237CogitataPhysicoMathematica,64Colossusmachine,238CommonLisp,116,124,190Commonmeasureofsegments,33Commonnotions,Euclid’saxiomaticmethod,162Commutativealgebra,rings,143–144Commutativityofaddition,155–156,174–175Commutativityofmultiplication,visualproof,156Commutativityofpowers,semigroups,91Compile-timedispatch.SeeCategorydispatchCompleteness,lawof,203–204Completeness,theories,102Complexnumbers,137–138Compositenumbers.SeealsoPrimenumbers
definition,21distinguishingfromprime,240–245
Conceptsandabstractalgebra,141definition,181
choosing,250examples,116–117,181namingconventions,183overview,181–184,266–267Regular,183–184requirementsontypes,24,182Semiregular,184typeattributes,182–183typefunctions,182–183
Consistency,theories,102,104Constructivists,229Contradiction,proofby,35,261–262Contrapositive,259Coprime,31,78,80–81,246–247Cosets,97.SeealsoLagrange’sTheoremCountedranges,189,203–204Cryptanalysis,237–238Cryptography,233–234,237Cryptology
asymmetrickeys,238BletchleyPark,238Caesarcipher,237ciphertext,238codes,definition,237Colossusmachine,238cryptanalysis,237–238cryptography,237cryptosystems,238Enigmamachine,238keys,238Lorenzmachine,238plaintext,238public-keycryptosystems,239–240RSAalgorithm,239–240,245–247symmetrickeys,238trapdoorone-wayfunctions,239
Cryptosystems,238Cycles,ofpermutations,200,207–211Cyclicgroups,96,109
generator,96Cyclicsubgroups,96
DDatum,180Decimalfractions,129–131Declarationsyntax,267Dedekind,Richard,140,171Degreeofpolynomials,133Dereferencing,iterators,184–185Descartes,René,64,131Differenceofpowersformula,30Differencetype,iterators,187difference_typeiteratortrait,187DifferentialCalculus,70Diffie,Whitfield,239Diophantus,67,225Dirichlet,PeterGustavLejeune,41,139–140,156Dirichletprinciple.SeePigeonholeprincipleDisme:TheArtofTenths,129–131DisquisitionesArithmeticae(“InvestigationsofArithmetic”),136–137distance,186-188divides,240Dividingpolynomials,133Domainofalgorithm,150Domainofdefinition,113Doublylinkedlists,185
EEgyptiandivision,57Egyptianmultiplication,8–11
requirements,111–118generalizingtopower,120
Elements(ofEuclid),2,21,43–45,161–163Proposition[VII,30],70Proposition[VII,32],21Proposition[IX,36],29,31–32Proposition[X,2],45Proposition[X,3],45–46Proposition[X,117],37
ElementsofProgramming,3,113–114,183,185,208Enigmamachine,238Equationalreasoning,114Equivalence,114
Eratosthenes,22Euclid.SeealsoElements
theaxiomaticmethod,161–163GCM(greatestcommonmeasure)algorithm,45–49incommensurablequantities,45–49onnumbertheory,21profile,44–45
Euclideandomains(ED),150–151,153Euclideangeometry
alternativesto,164–167fifthpostulate,163–164vs.hyperbolicgeometry,164–167vs.non-Euclidean,166–167
Euclideanrings.SeeEuclideandomains(ED)Euclid’salgorithm,45–47Euler,Leonhard,84,85
Euler’stheorem,79–83andLagrange,99perfectnumbers,32,63–64primenumbers,63,68profile,69–70
Eulertotientfunction,80,245Euler’sTheorem,79–83,246
proofusingLagrange’sTheorem,101Evenandoddnumbers,9–10,117
inGCD,219–220,224,234Existenceofzeroaxiom,172ExtendedGCDalgorithm,229–235,245,247extended_gcd,233
FFast-multiplicationalgorithm.SeeEgyptianmultiplicationFermat,Pierrede,63,65–69
profileof,67–68proofs,65–66,68
Fermatprimes,63–68,137Fermat’sLastTheorem,67Fermat’sLittleTheorem
converseof,77–79description,69non-invertibilitylemma,79proofbyLagrange’sTheorem,101
proof,77testingforprimenumbers,241–242restatementusingmodulararithmetic,84
Fermattest,241–242fermat_test,242
Fibonacci.SeeLeonardoPisanoFibonaccinumbers,computing,124–127
Fibonaccisequence,58–59Fields
characteristicof,151definition,151,153extensions,151prime,151,154
FifthpostulateofEuclideangeometry,163–164Figuratenumbers
gnomons,20oblongnumbers,19overview,17,19–20triangularnumbers,19squarenumbers,20
find_if,190–191find_if_n,191Finiteaxiomatizabilityoftheories,102Flowers,Tommy,238Floyd,Robert,58Formalistphilosophyofmathematics,167–169FormularioMathematico,170–172Forwarditerators,185FP,124Functionobjects,123–124,268,270Functors.SeeFunctionobjects
GGalois,Évariste
discoveryofgroups,85–88profileof,88–89
Gauss,CarlFriedrich,31,72,136–140,166,240profileof,136–137
Gaussianintegers,138–139,224GCD(greatestcommondivisor)
applicationsof,234ofpolynomials,134
description,59computing,59Euclid’salgorithm,45–46extendedGCD,229–235,245,247historicalmilestones,222andrationalarithmetic,234andringstructures,225–229rotationalgorithms,234Stein’salgorithm,219–225symbolicintegration,234validating,59–60
gcd,150,230GCM(greatestcommonmeasure)33,41
Euclid’salgorithm,45–49properties,33
Generatorelementsinsubgroups,96Genericprogramming
inC++,265–266,270concepts,181essence,127,249–250history,124,134,141,180andmathematics,84overview,1–2,5
get_temporary_buffer,217Gnomons,20Gödel,Kurt,169Göttingen,Universityof
CarlGauss,136–140DavidHilbert,168–169EmmyNoether,140–145profile,135–136
Granville,Andrew,244Grassman,Hermann,171Greatestcommondivisor(GCD).SeeGCD(greatestcommondivisor)Greatestcommonmeasure(GCM).SeeGCM(greatestcommonmeasure)Gries,David,205Gries-Millsalgorithm,204–208Groups
abelian,86,108,153additive,86binaryoperations,86cyclic,96,109
definition,85discoveryof,85examplesof,86–88identityelements,86inverseoperations,86Kleingroup,106orderofelements,94summarydescription,108,152symmetric,198theoremsabout,92–95
Hhalf,118Heath,Thomas,9,45Hegel,G.W.F.,111Hellman,Martin,239Hilbert,David,141,167–169,229
profile,168–169Hilbertspaces,168–169Hilbert’sproblems,169Hilbert’sprogram,169HistoryofAlgebra,129Horner’srule,132Hyperbolicgeometry,164–167
IIbnRushd,180Ideals.SeealsoRings
definition,226idealsinEuclideandomainslemma,227linearcombinationideallemma,227PID(principalidealdomains),228principalideals,227–228principalelements,227
IdealsinEuclideandomainslemma,227Identityelement,108–109,121
ingroups,86inmonoids,89inrings,143
identity_element,123,241Immutableobjects,181Impossibilityofinfinitedescent,21
Inclusion-exclusionprinciple,82–83Incommensurablequantities,45–49Independence,theories,102Indianmathematics,51Induction,proofby,262–263Inductionaxiom,21,170,172–173Inman,BobbyRay,240Innerproductoftwovectors,145–146In-placealgorithms,215–216Inputiterators,185IntegralCalculus,70Integraldomains,145,153Interfacerefinement,lawof,215Interfaces,designing,215Interlingua,171IntermediateValueTheorem(IVT),131,192IntroductiontoAnalysisoftheInfinite,70IntroductiontoArithmetic,10,19Intuitionistphilosophyofmathematics,229Inversenumbers,73inverse_operation,123Inverseoperation,86,119,121
ingroups,86Invertibilitylemma,229Invertibilityofsuccessoraxiom,173Invertibleelements.SeeUnitsIrrationalnumbers,38–39is_prime,241Isomorphism,models,103–104Iteratorcategories
bidirectional,185forward,185input,185output,186random-access,185
Iteratortraits,187iterator_categoryiteratortrait,187Iterators
inarrays,185bidirectional,185definition,184dereferencing,184–185
differencetype,187findingthedistancebetween,186–187forward,185input,185innoncontiguousdatasegments,186linked,186output,186overview,184–185randomaccess,185segmented,186successors,184
Iverson,Kenneth,124IVT(IntermediateValueTheorem),131,192
JJefferson,Thomas,44,130
KKapur,Deepak,124Kayal,Neeraj,244Keys,cryptography,238Khayyam,Omar,164Kleene,Stephen,115–116Klein,Felix,106–107,141Kleingroup,106–107Knuth,DonaldE.,9,58,197Kovalevskaya,Sofia,141
LLagrange,Joseph-Louis,99–100,192Lagrange’sTheorem,97–99,100–101Lambdaexpressions,195,272–273Laplace,Pierre-Simon,70largest_doubling,54LatinesineFlexione,171Lawofcompleteness,203–204Lawofinterfacerefinement,215Lawofseparatingtypes,202–203Lawofusefulreturn,57–58,201–202,213LecturesonNumberTheory(VorlesungenüberZahlentheorie),140Legendre,Adrien-Marie,155Lehmer,D.H.,192
LeonardoPisanoFibonaccisequence,58–59introductionofzero,52profile,52–53
LetterstoaGermanPrincess,70LiberAbaci,52LiberQuadratorum,52LibraryofAlexandra,43Lincoln,Abraham,44Linearalgebra
innerproduct,145–146matrix-matrixproduct,146matrix-vectorproduct,146review,145–147
Linearcombinationideallemma,227Linearrecurrencefunctions,127Linearrecurrencesequences,127Linearsearch,190–191Linkediterators,186Liu,Hui,51Lobachevsky,Nikolai,164–166Lorenzmachine,238lower_bound,195–196Lyceum,179
MMagmas,91,108mark_sieve,24Mathnotationinthisbook,257–259Matrixmultiplication,145–147Matrix-matrixproduct,146Matrix-vectorproduct,146Mauchly,John,192McJones,Paul,3Measureofasegment,33Memory-adaptivealgorithms,216–217Meno,43Mersenne,Marin,64–65Mersenneprimes,63–68Metaphysics,179Miller-Rabintest,243–245miller_rabin_test,243
Mills,Harlan,205Models.SeealsoTheories
definition,103isomorphism,103–104
ModernAlgebra,142Modulararithmetic,72–74,83–84
Fermat’sLittleTheorem,83–84Wilson’sTheorem,83
Modules,definition,151,154modulo_multiply,241Monoids.SeealsoGroups
additive,89,109,154definition,89examplesof,89multiplicative,89,154summarydescription,108,152
Mouseion,43Multiplication
definition,8,173–174Egyptian,8–11RussianPeasantAlgorithm.9
Multiplicativefunctions,31multiplicative_inverse,121,247multiplicative_inverse_fermat,241Multiplicativemonoids,89Multiplicativesemigroups,90Multiply-accumulatefunction,11–14Musser,DavidR.,124Mutableobjects,181
NNamingconventions,concepts,183Namingprinciple,115–116Naturalnumbers,147,170,172,175,258NicomacheanEthics,179NicomachusofGerasa,10,19NineChaptersontheMathematicalArt,51Noether,Emmy,129,140–145
profile,141–142Non-categoricaltheories,106–107Noncommutativeadditivemonoids,119Noncommutativeadditivesemigroups,115
Noncommutativealgebra,rings,143–144Nonconstructiveproofs,229Noncontiguousdatasegments,iterators,186Non-Euclideangeometry,164–167Non-invertibilitylemma,79Notationinthisbook,257–259Numberline,131Numberofassignmentstheorem,200–201Numbersystems,ancientEgypt,8Numbertheory2,41,43
inancientGreece,17–39Bezout’sidentity,225–229Euler’sTheorem,79–83,101Fermat’sLittleTheorem69–78,101figuratenumbers,17–20,33Gauss,136–137andGCD,140LiberQuadratorum,53modulararithmetic,72–74perfectnumbers,28–32primalitytesting240–245primenumbers,21–2817thand18thcentury,63–72,74–84sieveofEratosthenes,22–23Wilson’sTheorem,76,83
OObjecttypes,definition,181Objects
definition,180immutable,181mutable,181remoteparts,181unrestricted,181
Oblongnumbers,19Octonions,151odd,118Oddnumbers.SeeEvenandoddnumbersOne-to-onecorrespondence,92Openranges,188“OperatorsandAlgebraicStructures,”124Orderofgroupelements,94
Organon,180Outputiterators,186
PPalindromicprimes,28Parallelpostulate.SeeFifthpostulateofEuclideangeometryPartitionpoints,193partition_point,194partition_point_n,193Peano,Giuseppe,169–175
profile,171–172Peanoarithmetic,170–171,173–175Peanoaxioms,170–173Peanocurve,171Perfectnumbers
inancientGreece,28–32,38definition,28–29mathematicians’interestin,63
Permutationofremainderslemma,71–72Permutations,197–201Phaedo,43PhiloofAlexandria,7PID(principalidealdomains),228Pigeonholeprinciple,95,263Pisano,LeonardoSeeLeonardoPisanoPlaintext,238Plato,41–43,177–179
profile,42–43PlatonicQuestions,20Platonicsolids,41,44Playfair’saxiom,163Plussign(+),mathematicalconventionfor,115Plutarch,20Poincaré,JulesHenri,85,229–230,248
profile,229–230pointeriteratortrait,187Politics,179Polylogspace,215–216Polynomials
computingGCDfor,134degreeof,definition,133divisionwithremainder,133
historyof,132–135Horner’srule,132treatingasnumbers,133–135
polynomial_value,132Populationcount,10Postconditions,269Postulates,Euclid’saxiomaticmethod,162,163Poweralgorithm,119–123,249
computingFibonaccinumbers,126computinglinearrecurrence,127useincryptology,241–243,246useingraphapplications,148–149
power_accumulate_semigroup,121power_group,123power_monoid,122power_semigroup,122Primalitytesting,240–245Primefactorization,29,31–32,65,136,139–140Primefields,151,154Primenumbers
inancientGreece,21–28definition,21distinguishingfromcomposite,240–245Fermatprimes,63–68finding.SeesieveofEratosthenesinfinitenumberof,21Mersenneprimes,63–68primalitytesting,240–245
Principalelement,227Principalidealdomains(PID),228Principalideals,227–228ProblèmesPlaisants,225–226Proof
bycontradiction,35,261–262definition,158–159byinduction,262–263nonconstructive,229pigeonholeprinciple,95,263visual,155–159
Properdivisor,32Ptolemy,164Public-keycryptosystems,239–240
Pythagoras,17profile,18–19
Pythagoreanprogram,33–38PythagoreanTheorem,44Pythagoreantriples,50–51
QQuadrivium,18Quaternions,151Quotient,55–57,150,153,202
forpolynomials,133quotient_remainder,57
RRandom-accessiterators,185Ranges
bounded,189,203–204closed,188counted,189,203–204definition,188open,188overview,188–189partitionpoints,193semi-open,188swapping,201–204
Rationalarithmetic,GCDapplications,234Rationalnumbers,151,258Realnumbers,131,258reciprocal,124Recreationalmathematics,225–226Recursiveremainderlemma,48–49Reductionalgorithm,124referenceiteratortrait,187Regius,Hudalricus,64Regularconcepts,183–184Regularfunctions,183Regulartypes,114Rejewski,Marian,238Remainder,47–49,53–55,57–59,150,153,222
Floyd-Knuthalgorithm,58permutationofremainders,71–72inmodulararithmetic,73–75
ofGaussianintegers,138–139ofpolynomials,133–134
remainder,54–55remainder_fibonacci,58Remotepartsofobjects,181Requirementsonalgorithm,111–119reverse,212–215reverse_copy,216reverse_n,214reverse_n_adaptive,217reverse_n_with_buffer,216reverse_recursive,214Reversepermutation,201,212–215Rewritingcode,14–15RhindMathematicalPapyrus,8,57Rings.SeealsoIdeals;Semirings
definition,142–143andtheGCD,225–229integraldomains,145summarydescription,153unitary,143units,144zerodivisors,145
Rivest,Ron,239Rotatealgorithms,204–213rotate,207,210,213rotate_cycle_from,208rotate_transform,210rotate_unguarded,206Rotation,204–207Rotationalgorithms,GCDapplications,234RSAalgorithm,239–240,245–247Russell,Bertrand,171RussianPeasantAlgorithm,9.SeealsoEgyptianmultiplication
SSaccheri,GiovanniGirolamo,164Saxena,Nitin,244Scheme,116TheSchoolofAthens,177–178Searches
binary,191–196linear,190–191
Segmentediterators,186Self-CancelingLaw,75–76Semanticrequirementsforgenericalgorithms,113Semigroups.SeealsoGroups
additive,90,109associativityaxiom,91commutativityofpowers,91definition,90,109examples,90multiplicationalgorithm,115multiplicative,90summarydescription,108,152
Semi-openranges,188Semiregularconcepts,184Semirings.SeealsoRings
Boolean,148description,145–147matrixmultiplication,146shortestpath,148–149summarydescription,153tracingsocialnetworks,147–148transitiveclosures,147–148tropical,149weak,147
Separatingtypes,lawof,202–203Settingofalgorithm,150Shamir,Adi,239Shortestpath,finding,148–149SieveofEratosthenes,22–23
implementation23–28sift,27smallest_divisor,240Socialnetworkconnections,tracing,147–148Socrates,42Socraticmethod,42Sophists,42Spacecomplexity,215–216Squarerootof2,anirrationalnumber,37–38StandardTemplateLibrary.SeeSTLStein,Josef,219–222
Stein’salgorithm,219–225stein_gcd,220Stepanov,AlexanderA.,3,124Stevin,Simon,129–135,192
profile,130–131STL(StandardTemplateLibrary)
algorithms,195–196,215,217applicationofgenericprogramming,1,186containers,190–191conventions,24non-categorical,104
Strengthreduction,26Stroustrup,Bjarne,265Subgroups.SeealsoGroups
cyclic,96definition,95,109generatorelements,96trivial,95
Successors,170,184Sumofoddpowersformula,30swap,199swap_ranges,201–203Symbolicintegration,GCDapplications,234Symmetricgroups,198Symmetrickeys,238Symposium,43Syntacticrequirementsforgenericalgorithms,113
TTail-recursivefunctions,12–14Templatefunctions,265–266ThalesofMiletus,18,159–161Thales’Theorem,160–161Theories.SeealsoModels
categorical,104–106characteristicsof,102completeness,102consistency,102,104definition,102determiningtruthof,167finiteaxiomatizability,102independence,102
non-categorical,106–107univalent,104
Totalityofsuccessoraxiom,172Totientofaninteger,80ATourofC++,265Transfiniteordinals,172–173Transformationgroup,92Transitiveclosures,finding,147–148Transpositionlemma,199Transpositions,197,199–201Trapdoorone-wayfunctions,239Triangularnumbers,19TrichotomyLaw,34Tripcount,204,213Trivialcycles,200,208Trivialsubgroups,95Tropicalsemirings,149Turing,Alan,169,238TusculanDisputations,50Typeattributes,182–183Typedispatch.SeeCategorydispatchTypefunctions,182–183
UUnitaryrings,143Units,rings,144Univalenttheories,104Univariatepolynomials.SeePolynomialsUniversityofGöttingen.SeeGöttingen,UniversityofUnreachablenumbers,172–173Unrestrictedobjects,181upper_bound,195Usefulreturn,lawof,57–58,201–202,213
VValuetypes,definition,180Values,definition,180value_typeiteratortrait,187vanderWaerden,Bartel,129,142Veblen,Oswald,104vectorcontainer,116,Vectorspace,152,154
Visualproofs,155–159VorlesungenüberZahlentheorie(LecturesonNumberTheory),140
WWaring,Edward,76Weaksemirings,147Weilert,Andre,224Well-orderingprinciple,34Whitehead,AlfredNorth,43Wiles,Andrew,67Wilson,John,76Wilson’sTheorem
description,76usingmodulararithmetic,83
Witnesses,primalitytesting,242
ZZero
inEgyptiannumbersystem,8introductionof,52originsof,51–53
Zerodivisors,rings,145,154
PhotoCredits
Page18:Vkara/Fotolia;42,44:AlexeyPavluts/Fotolia;52:Bettman/Corbis;64:Lisztcollection/Alamy;67:Marzolino/Shutterstock;69:GeorgiosKollidas/Fotolia;88:Bettman/Corbis;99:GeorgiosKollidas/Fotolia;130:BilwisseditionLtd.&Co.KG/Alamy;136:GeorgiosKollidas/Fotolia;141:Interfoto/Alamy;159:DesignPics/Newscom;165:RIANovosti/Alamy;168:AKGImages/Newscom;171:RobertGruberRose;178:WorldHistoryArchive/Alamy;178:Midosemsem/Fotolia;225:RMN-GrandPalais/ArtResource,NY;229:AKGImages/Newscom.
CodeSnippets
top related