making high performance embedded instruments with bela...

MakingHigh-PerformanceEmbeddedInstrumentswithBelaandPureData

GiulioMoro1,AstridBin2,RobertH.Jack3,ChristianHeinrichs4,AndrewP.McPherson5

CentreforDigitalMusic,QueenMaryUniversityofLondon,UK

1g.moro@qmul.ac.uk,2a.bin@qmul.ac.uk,3r.h.jack@qmul.ac.uk,4c.heinrichs@qmul.ac.uk,5a.mcpherson@qmul.ac.uk

Abstract.Belaisanembeddedplatformforultra-lowlatencyaudioandsensorprocessing.WepresentherethehardwareandsoftwarefeaturesofBelawithparticularfocusonitsintegrationwithPureData.SensorinputsonBelaaresampledataudiorate,whichopenstothepossibilityofdoingsignalprocessingusingPureData’saudio-rateobjects.

Keywords:embeddedaudio,sonicinteractiondesign,sensors,lowlatency,musicalinstrumentdesign.

IntroductionTheincreasingpowerandavailabilityofmicrocontrollersandsingle-boardcomputershasgivenrisetomanynewplatformsforcreatingmusicalinstrumentsandplatformsforinteractiveaudio.Choosingasuitableplatformcanbeachallenge,involvingtradeoffsbetweencomputingpower,hardwareconnectivity,easeofprogrammingandprice.

Manycurrentapproachestodesigningsoundingobjectscombinetwoormoredevicestogether,forinstanceanArduinowhichhandlesanaloganddigitalsensorinputcommunicatingviaUSB-serialwithaacomputerrunningtheaudioprocessing.Usingaself-containedembeddedplatforminthecreationofDMIsandinteractiveaudiosystemshasseveraladvantagesoversuchasetup.

• ReliabilityUsingasingledeviceislesspronetocommunicationerrorsanditiseasiertoprovideabackupsolutionforasimplersystem.

• PerformanceWithacompositesetupastheonedescribedabove,theserialconnectionisslowandthethroughputislimited.Assuch,thelatency,samplingrateandjitteroftheacquireddataareallaffectednegatively,whichmayinturnaffecttheexpressivenessoftheperformance.MIDIdevicestypicallyperformbetterthanserialones,whilewirelesslinksmaybeaffectedbypacketlossorchannelcongestion(McPherson,Jack,andMoro2016).

• ReproducibilityItiseasierforotherpeopletorecreateadeviceifitdoesnotrelyonmultiplepiecesofsoftwareandhardwaredevicesandspecificrevisionsofeachofthem.

• SustainabilitySimilarly,thedevelopersthemselveswillfinditeasiertomaintainanddevelopasystemthatdoesnothavemultipledependencies,alsototheadvantageofmakingsoftwareversioncontroleasier.

RecentEmbeddedPlatformsforDigitalMusicalInstrumentCreationArduinoandsimilarboardsareanaccessiblewayofprovidinglow-levelconnectivitytoanaloganddigitalsensors,butthelow-poweredAVRmicrocontrollerdoesnotallowaudioon-boardaudioprocessing.Thex-OSCboardprovidesanaloganddigitalI/Osoverawirelesslink(MadgwickandMitchell2013).

Twoaudio-oriented,self-containedplatformsbasedona168MHzCortexM4microcontrollerhitthemarketinthepastfewyears:theOwl1programmabledigitaleffect(Webster,LeNost,andKlang2014),whichsurfacedin2013,andAxoloti2,which

1http://hoxtonowl.com/2http://www.axoloti.com/

cameoutearly2015.TheformercanbeprogrammedthroughaC++APIorcanrunPureDatapatchesusingtheHeavyAudioToolsfromEnzienAudio,3whilethelatterprovidesacustomgraphicalpatcherwhichincludesDSPmodulesandcanbeexpandedwithC++.

RaspberryPiisarguablythemostpopularsingle-board-computerintheworldanditslatestrevision3featuresaquad-core1.2GHz64bitCPU.TheCCRMASatellitedistribution4(BerdahlandJu2011)wasdevelopedtoprovideanefficientaudio-orientedenvironmentfortheRaspberryPi.

CoalaisanaudioprocessingplatformbasedontheBeagleBoneBlack5whichwaspresentedin(Piéchaud2014).ThesoftwareandhardwarearchitectureofCoalaweredevelopedforthespecifictaskofmodalcontrol6,whichrequiresaverytightfeedbackloop.Theplatformisthereforeoptimizedforfastsample-by-sampleprocessinginordertominimizeround-triplatency.

Bela:anembeddedplatformforaudioandsensorprocessingBela7(formerlyknownasBeagleRT)isacombinedhardwareandsoftwareenvironmentthatconsistsofaBeagleBoneBlackwithanexpansion“cape”(McPhersonandZappi2015a).ItwasoriginallydevelopedfortheD-BoxHackableDigitalInstrument(ZappiandMcPherson2014)whichrequiredmultiplelow-latencyhybridanalog-digitalfeedbackloops(McPhersonandZappi2015b).Belacombinestheconnectivityofamicrocontrollerwiththeprocessingcapabilityofasingle-boardcomputer.ThecapeprovidesstereoaudioI/Oincluding1Wspeakeramplifiers,8channelseachof16-bitanalogI/O,and16digitalGPIOpins.Belaisopen-sourcehardwareandsoftware.Sourcecodeanddesignmaterialsarepubliclyavailable.8

TheBelasoftwareusesaDebianLinuxdistributionwiththeXenomai9real-timekernelextensions.TheProgrammableRealtimeUnit(PRU),a200MHzmicrocontrolleronthesamechipastheBeagleBoneBlackCPU,transfersaudioandsensordatadirectlytothehardware,bypassingthekerneldrivers.Theuser’sBelacodethereforerunsatthehighestpriorityofanytaskontheboard,includingtheLinuxkernelitself.Thisallowsaudioblocksizesaslowas2samples,resultinginround-tripaudiolatencyof1ms(orevendownto100usifusingtheanaloginputsandoutputsratherthantheaudioconverters)(McPherson,Jack,andMoro2016).

OnBela,everyanaloganddigitalchannelisautomaticallysampledataudiorates,synchronouslywiththeaudioclock.ThehighsamplingrateoftheanaloganddigitalchannelsareuniquetoBelaandtheirjitter-freealignmentwiththeaudiomakesitidealforinteractive,intuitive,responsiveaudioapplications.

ComparedtoAxolotiandOwl,Belahasmoreprocessingpower,whilestillprovidinghardreal-timeperformances,withtheaddedconvenienceofafullLinuxOSandwhilebeingminimallyaffectedbysystemload.Itismoregeneral-purposethanCoalawhichaddressesthespecificfieldofreal-timecontrol,thoughCoalaiscapableofevenlowerlatenciesthanBela.OnaRaspberryPirunningCCRMASatellite,despitethehighprocessingpoweravailableonboard,audiodependsonthestandardLinuxaudiodrivers,sothatlow-latencyprocessingisdifficultbecauseofthepresenceofotherprocessesontheboard,whichmaycauseunderrunsatsmallaudioblocksizesevenwhentheCPUloadislowonaverage.TheBeagleBoneBlackCPUislesspowerfuloverallbuttheXenomaiextensionsusedintheBelasoftwareallowreliableandconsistentperformancewithsub-millisecondlatency.Additionally,thenumberofI/OsavailableinBelaisgreaterthanthoseoncommonlyavailableRaspberryPihats.

3http://enzienaudio.com/4https://ccrma.stanford.edu/~eberdahl/Satellite/5http://beagleboard.org/black6http://instrum.ircam.fr/smartinstruments/7http://bela.io8http://bela.io/code/9http://xenomai.org/

Bela,providingalargenumberofI/Osforaudioandsensors,poweroutputforloudspeakers,andprovidingenoughprocessingpowertosatisfymostneeds,entirelyfulfillstherequirementsofaself-containeddevice,whichcanbeembeddedinastand-aloneDigitalMusicalInstrumentorsoundingobject.

PureDataonBelaPureData10(Pd)isapopularopensourcegraphicalprogramminglanguagewidelyusedbymusiciansandsounddesignersalike,whichallowsforquickprototypingofsoundandsensormappings.PdpatchesareusuallyrunwithinPditself,orusingthesharedlibrarylibpd11.ThemessagingarchitectureandtheaudioengineofPdwasnotdesignedtobefastandcomputationallyefficientwhichcanleadperformancepenaltiesonplatformswithlimitedcomputationalpower.

�� Figure1:TheBelacape Figure2:UsingaforcesensitiveresistorwithBela

HeavyAudioToolsTheHeavyAudioToolsfromEnzienAudiousePdasafront-endtogenerateoptimisedCcode.ByanalyzingthegraphofconnectionsbetweenobjectsinthePdcode,Heavyiscapableofproducinghigh-performancevectorizedCcodewhichcanoutperformlibpd,makingitparticularlywellsuitedforembeddeddevicesand,moregenerally,hardwarewithlimitedcomputationalpower.Heavyisaproprietary,cloud-basedserviceandthegeneratedcodeislicensedundertheMITnon-commerciallicense.

TheCcodeproducedbyHeavyiswell-suitedtobeintegratedinaXenomaienvironment,asmemoryisallocatedonthestack,thusavoidingsystemcallsduringexecution.AnautomatedscripttakescareofuploadingthePdpatchtoHeavy’sserver,collectthegeneratedCcodeandcompileitontheBelaboard.TheentireprocessgenerallytakeslessthanoneminuteandmostofthetimeisspentcompilingtheCcodeontheBeagleBoneBlack.

libpdMinimalmodificationswererequiredtoportlibpdforBela,theseincludedallowingblocksizesassmallas8samplesperblockandremovingsocketanddiskI/Ofromtheaudiothread.Additionally,thecallstothepthreadfunctionswerewrappedintoXenomaifunctions.TheresultingsharedlibrarycanbelinkedtoaBelaprogramandlibpd_process_float()istheninvokedfromwithinBela’saudiocallback.

DeployingaPdpatchusinglibpdisvirtuallyinstantaneousasitdoesnotrequirecompiling.AssoonasthepatchissavedontheBeagleBone’sfilesystem,theBelaprogramcanberestartedanditwillloadtheupdatedpatch.AnaddedadvantageofusinglibpdisthatitiseasiertoportPdexternalswhentheirsourcecodeisavailable.ThesameprecautionslistedaboveshouldbetakenfornewexternalsinordertomakesurethatnewobjectsdonotintroduceXenomaimodeswitchesintheaudiothread12.

10http://puredata.info11http://libpd.cc/12https://xenomai.org/2014/08/porting-a-linux-application-to-xenomai-dual-kernel/

PerformancecomparisonRunninganexamplepatchcontainingagenerativeaudiocomposition,Heavycodecompiledwiththeclang13compileruses26%oftheCPU.ThesameHeavycode,compiledwithgcc14,occupies43%oftheCPUcycles.Runningthepatchusinglibpduses53%oftheCPU.

Traditionally,thehighest-performanceplatformshavealsoplacedthemosttechnicaldemandsontheprogrammer.Formanyyears,customDSPboardsofferedthebestbalanceofhardreal-timeperformanceandhighprocessingpower,buttheyweregenerallyprogrammedinlow-levellanguagesusingcustomdevelopmentenvironments.High-levelmusicprogramminglanguagesoftencomewithsignificantprocessingoverhead.RunningPureDataonBela,especiallythroughtheHeavyAudioTools,providesaconvenientgraphicalenvironmentwithminimalsacrificeinperformancecomparedtoprogramminginC++.

��(a)Smoothing(b)Re-centering

(c)Differentiating (d)Thresholding

Figure3:UsingPureDataobjectstoprocesssensordata

SensorprocessinginPureDataManyinteractivesystemstakeapproacheswheresensorsaresampledatlowandnon-constantratesandthemostrecentframeofsensordataisusedtomodulateaparticularsonicparameter.Butinactualfact,themeaningofsensordataisoftendeeper,initsbehaviourovertimeoritsfrequencycontent.Ahighsamplingrateyieldsaveryhighbandwidthofinteractionwhichcapturessubtledetailsthatmightbelostatlowersamplerates.Thoughallthesametechniquescouldbeimplementedatcontrolrate,audio-ratesensordatacanhelpreorientthedesigner’sthinkingtobecomemoreawareofthesepossibilities.WhenusingBelawithPd,thisallowstoconvenientlyprocesssensorsignalsusingaudio-rateobjects.Someexamplesinclude:

• SmoothingSomesensorsareinherentlynoisy,forinstanceapotentiometermaygeneratehigh-frequencynoisewhenitisactuated,oraninfra-redopticalsensormaybesubjecttotransientperturbationsfromotheremittingsources.Thenoiseinthesensorreadingsmayleakintotheaudiosignal,dependingonthesignalflow.Aneasyapproachtoremovehigh-frequencynoiseistoapplyalow-passfilterwithanappropriatecut-offfrequency,asinFigure3a.

13http://clang.llvm.org/14https://gcc.gnu.org/

• Re-centeringReadingsfromaccelerometersandothersensorshaveinherentDC-offsetswhichmaybeundesirableforcertainapplications.Aquickwayofremovingthemwhichdoesnotrequirecalibrationisusinganhigh-passfilterwithanappropriatecut-offfrequency,asinFigure3b.

• DifferentiatingSomesound-generatorparametersarebettercontrolledusingthevelocityofasensorreading,ratherthanwiththerawreading.Ahigh-passfilterwithacut-offfrequencyof0,properlyrescaledcanusedforthispurpose,asinFigure3c.

• ThresholdingAmorecomplicatedexampleinFigure3dshowshowtocombinefull-waverectification,smoothing,DCshiftandconstraintothresholdasignal.

ConclusionThereareseveraltradeoffsinvolvedindifferentdigitalmusicalinstrumentdesigntools:processingpower,latency,connectivity,sensorbandwidth,easeofprogrammingandaccessibility.Withanyoftheprogrammingenvironments,BelabringstogethertheconnectivityandCPUpowerofanembeddedLinuxcomputerwiththelowlatencyandprecisesynchronisationofamicrocontrollerandbringsahigh-bandwidthdimensiontosensorprocessing.UsingeithertheHeavyorthelibpdenvironments,Belaisalsosuitableforrapidprototypingusingthewidely-usedPureDatagraphicalprogramminglanguage,withfullaccesstobothaudioandsensors.

Acknowledgements.ThisworkissupportedbygrantsEP/K032046/1,EP/K009559/1andEP/L019981/1fromtheUKEngineeringandPhysicalSciencesResearchCouncilandtheQueenMaryCentreforPublicEngagement.

ReferencesBerdahl,Edgar,andWendyJu.2011.“SatelliteCCRMA:AMusicalInteractionandSoundSynthesisPlatform.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Oslo,Norway,173–178.

Madgwick,Sebastian,andThomasJMitchell.2013.“x-OSC:AversatilewirelessI/Odeviceforcreative/musicapplications.”SMCSoundandMusicComputingConference.Stockholm,Sweden:KTHRoyalInstituteofTechnology.

McPherson,A.P.,andV.Zappi.2015a.“Anenvironmentforsubmillisecond-latencyaudioandsensorprocessingonBeagleBoneBlack.”AudioEngineeringSocietyConvention138.AudioEngineeringSociety.

McPherson,Andrew,andVictorZappi.2015b.“ExposingtheScaffoldingofDigitalInstrumentswithHardware-SoftwareFeedbackLoops.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.BatonRouge,Louisiana,USA:LouisianaStateUniversity,162–167.

McPherson,AndrewP.,RobertH.Jack,andGiulioMoro.2016.“Action-SoundLatency:AreOurToolsFastEnough?”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Brisbane,Australis.

Piéchaud,Robert.2014.“ALightweightC++Real-TimeActiveControlFramework.”16thRealTimeLinuxWorkshop,October12to13,2014attheCCDCongressCenterDusseldorfcollocatedwithLinuxConEuropeinDusseldorf,Germany.

Webster,Thomas,GuillaumeLeNost,andMartinKlang.2014.“TheOWLprogrammablestageeffectspedal:Revisingtheconceptoftheon-stagecomputerforlivemusicperformance.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.London,UnitedKingdom:Goldsmiths,UniversityofLondon,621–624.

Zappi,V.,andA.McPherson.2014.“DesignandUseofaHackableDigitalInstrument.”ProceedingsoftheInternationalConferenceonLiveInterfaces.Lisbon,Portugal,208–219.