Upload File

making high performance embedded instruments with bela...

  • Home
  • Documents
  • Making High Performance Embedded Instruments with Bela …users.sussex.ac.uk/~thm21/ICLI_proceedings/2016/Practical/... · Making High-Performance Embedded Instruments with Bela and
5
Making High-Performance Embedded Instruments with Bela and Pure Data Giulio Moro 1 , Astrid Bin 2 , Robert H. Jack 3 , Christian Heinrichs 4 , Andrew P. McPherson 5 Centre for Digital Music, Queen Mary University of London, UK 1 [email protected], 2 [email protected], 3 [email protected], 4 [email protected], 5 [email protected] Abstract. Bela is an embedded platform for ultra-low latency audio and sensor processing. We present here the hardware and software features of Bela with particular focus on its integration with Pure Data. Sensor inputs on Bela are sampled at audio rate, which opens to the possibility of doing signal processing using Pure Data’s audio-rate objects. Keywords: embedded audio, sonic interaction design, sensors, low latency, musical instrument design. Introduction The increasing power and availability of microcontrollers and single-board computers has given rise to many new platforms for creating musical instruments and platforms for interactive audio. Choosing a suitable platform can be a challenge, involving tradeoffs between computing power, hardware connectivity, ease of programming and price. Many current approaches to designing sounding objects combine two or more devices together, for instance an Arduino which handles analog and digital sensor input communicating via USB-serial with a a computer running the audio processing. Using a self-contained embedded platform in the creation of DMIs and interactive audio systems has several advantages over such a setup. • Reliability Using a single device is less prone to communication errors and it is easier to provide a backup solution for a simpler system. • Performance With a composite setup as the one described above, the serial connection is slow and the throughput is limited. As such, the latency, sampling rate and jitter of the acquired data are all affected negatively, which may in turn affect the expressiveness of the performance. MIDI devices typically perform better than serial ones, while wireless links may be affected by packet loss or channel congestion (McPherson, Jack, and Moro 2016). • Reproducibility It is easier for other people to recreate a device if it does not rely on multiple pieces of software and hardware devices and specific revisions of each of them. • Sustainability Similarly, the developers themselves will find it easier to maintain and develop a system that does not have multiple dependencies, also to the advantage of making software version control easier. Recent Embedded Platforms for Digital Musical Instrument Creation Arduino and similar boards are an accessible way of providing low-level connectivity to analog and digital sensors, but the low-powered AVR microcontroller does not allow audio on-board audio processing. The x-OSC board provides analog and digital I/Os over a wireless link (Madgwick and Mitchell 2013). Two audio-oriented, self-contained platforms based on a 168MHz Cortex M4 microcontroller hit the market in the past few years: the Owl 1 programmable digital effect (Webster, LeNost, and Klang 2014), which surfaced in 2013, and Axoloti 2 , which 1 http://hoxtonowl.com/ 2 http://www.axoloti.com/

Upload: tranliem

Post on 23-Mar-2018

215 views

Category:

Documents


2 download

Report

TRANSCRIPT

Page 1: Making High Performance Embedded Instruments with Bela …users.sussex.ac.uk/~thm21/ICLI_proceedings/2016/Practical/... · Making High-Performance Embedded Instruments with Bela and

MakingHigh-PerformanceEmbeddedInstrumentswithBelaandPureData

GiulioMoro1,AstridBin2,RobertH.Jack3,ChristianHeinrichs4,AndrewP.McPherson5

CentreforDigitalMusic,QueenMaryUniversityofLondon,UK

[email protected],[email protected],[email protected],[email protected],[email protected]

Abstract.Belaisanembeddedplatformforultra-lowlatencyaudioandsensorprocessing.WepresentherethehardwareandsoftwarefeaturesofBelawithparticularfocusonitsintegrationwithPureData.SensorinputsonBelaaresampledataudiorate,whichopenstothepossibilityofdoingsignalprocessingusingPureData’saudio-rateobjects.

Keywords:embeddedaudio,sonicinteractiondesign,sensors,lowlatency,musicalinstrumentdesign.

IntroductionTheincreasingpowerandavailabilityofmicrocontrollersandsingle-boardcomputershasgivenrisetomanynewplatformsforcreatingmusicalinstrumentsandplatformsforinteractiveaudio.Choosingasuitableplatformcanbeachallenge,involvingtradeoffsbetweencomputingpower,hardwareconnectivity,easeofprogrammingandprice.

Manycurrentapproachestodesigningsoundingobjectscombinetwoormoredevicestogether,forinstanceanArduinowhichhandlesanaloganddigitalsensorinputcommunicatingviaUSB-serialwithaacomputerrunningtheaudioprocessing.Usingaself-containedembeddedplatforminthecreationofDMIsandinteractiveaudiosystemshasseveraladvantagesoversuchasetup.

• ReliabilityUsingasingledeviceislesspronetocommunicationerrorsanditiseasiertoprovideabackupsolutionforasimplersystem.

• PerformanceWithacompositesetupastheonedescribedabove,theserialconnectionisslowandthethroughputislimited.Assuch,thelatency,samplingrateandjitteroftheacquireddataareallaffectednegatively,whichmayinturnaffecttheexpressivenessoftheperformance.MIDIdevicestypicallyperformbetterthanserialones,whilewirelesslinksmaybeaffectedbypacketlossorchannelcongestion(McPherson,Jack,andMoro2016).

• ReproducibilityItiseasierforotherpeopletorecreateadeviceifitdoesnotrelyonmultiplepiecesofsoftwareandhardwaredevicesandspecificrevisionsofeachofthem.

• SustainabilitySimilarly,thedevelopersthemselveswillfinditeasiertomaintainanddevelopasystemthatdoesnothavemultipledependencies,alsototheadvantageofmakingsoftwareversioncontroleasier.

RecentEmbeddedPlatformsforDigitalMusicalInstrumentCreationArduinoandsimilarboardsareanaccessiblewayofprovidinglow-levelconnectivitytoanaloganddigitalsensors,butthelow-poweredAVRmicrocontrollerdoesnotallowaudioon-boardaudioprocessing.Thex-OSCboardprovidesanaloganddigitalI/Osoverawirelesslink(MadgwickandMitchell2013).

Twoaudio-oriented,self-containedplatformsbasedona168MHzCortexM4microcontrollerhitthemarketinthepastfewyears:theOwl1programmabledigitaleffect(Webster,LeNost,andKlang2014),whichsurfacedin2013,andAxoloti2,which

1http://hoxtonowl.com/2http://www.axoloti.com/

Page 2: Making High Performance Embedded Instruments with Bela …users.sussex.ac.uk/~thm21/ICLI_proceedings/2016/Practical/... · Making High-Performance Embedded Instruments with Bela and

cameoutearly2015.TheformercanbeprogrammedthroughaC++APIorcanrunPureDatapatchesusingtheHeavyAudioToolsfromEnzienAudio,3whilethelatterprovidesacustomgraphicalpatcherwhichincludesDSPmodulesandcanbeexpandedwithC++.

RaspberryPiisarguablythemostpopularsingle-board-computerintheworldanditslatestrevision3featuresaquad-core1.2GHz64bitCPU.TheCCRMASatellitedistribution4(BerdahlandJu2011)wasdevelopedtoprovideanefficientaudio-orientedenvironmentfortheRaspberryPi.

CoalaisanaudioprocessingplatformbasedontheBeagleBoneBlack5whichwaspresentedin(Piéchaud2014).ThesoftwareandhardwarearchitectureofCoalaweredevelopedforthespecifictaskofmodalcontrol6,whichrequiresaverytightfeedbackloop.Theplatformisthereforeoptimizedforfastsample-by-sampleprocessinginordertominimizeround-triplatency.

Bela:anembeddedplatformforaudioandsensorprocessingBela7(formerlyknownasBeagleRT)isacombinedhardwareandsoftwareenvironmentthatconsistsofaBeagleBoneBlackwithanexpansion“cape”(McPhersonandZappi2015a).ItwasoriginallydevelopedfortheD-BoxHackableDigitalInstrument(ZappiandMcPherson2014)whichrequiredmultiplelow-latencyhybridanalog-digitalfeedbackloops(McPhersonandZappi2015b).Belacombinestheconnectivityofamicrocontrollerwiththeprocessingcapabilityofasingle-boardcomputer.ThecapeprovidesstereoaudioI/Oincluding1Wspeakeramplifiers,8channelseachof16-bitanalogI/O,and16digitalGPIOpins.Belaisopen-sourcehardwareandsoftware.Sourcecodeanddesignmaterialsarepubliclyavailable.8

TheBelasoftwareusesaDebianLinuxdistributionwiththeXenomai9real-timekernelextensions.TheProgrammableRealtimeUnit(PRU),a200MHzmicrocontrolleronthesamechipastheBeagleBoneBlackCPU,transfersaudioandsensordatadirectlytothehardware,bypassingthekerneldrivers.Theuser’sBelacodethereforerunsatthehighestpriorityofanytaskontheboard,includingtheLinuxkernelitself.Thisallowsaudioblocksizesaslowas2samples,resultinginround-tripaudiolatencyof1ms(orevendownto100usifusingtheanaloginputsandoutputsratherthantheaudioconverters)(McPherson,Jack,andMoro2016).

OnBela,everyanaloganddigitalchannelisautomaticallysampledataudiorates,synchronouslywiththeaudioclock.ThehighsamplingrateoftheanaloganddigitalchannelsareuniquetoBelaandtheirjitter-freealignmentwiththeaudiomakesitidealforinteractive,intuitive,responsiveaudioapplications.

ComparedtoAxolotiandOwl,Belahasmoreprocessingpower,whilestillprovidinghardreal-timeperformances,withtheaddedconvenienceofafullLinuxOSandwhilebeingminimallyaffectedbysystemload.Itismoregeneral-purposethanCoalawhichaddressesthespecificfieldofreal-timecontrol,thoughCoalaiscapableofevenlowerlatenciesthanBela.OnaRaspberryPirunningCCRMASatellite,despitethehighprocessingpoweravailableonboard,audiodependsonthestandardLinuxaudiodrivers,sothatlow-latencyprocessingisdifficultbecauseofthepresenceofotherprocessesontheboard,whichmaycauseunderrunsatsmallaudioblocksizesevenwhentheCPUloadislowonaverage.TheBeagleBoneBlackCPUislesspowerfuloverallbuttheXenomaiextensionsusedintheBelasoftwareallowreliableandconsistentperformancewithsub-millisecondlatency.Additionally,thenumberofI/OsavailableinBelaisgreaterthanthoseoncommonlyavailableRaspberryPihats.

3http://enzienaudio.com/4https://ccrma.stanford.edu/~eberdahl/Satellite/5http://beagleboard.org/black6http://instrum.ircam.fr/smartinstruments/7http://bela.io8http://bela.io/code/9http://xenomai.org/

Page 3: Making High Performance Embedded Instruments with Bela …users.sussex.ac.uk/~thm21/ICLI_proceedings/2016/Practical/... · Making High-Performance Embedded Instruments with Bela and

Bela,providingalargenumberofI/Osforaudioandsensors,poweroutputforloudspeakers,andprovidingenoughprocessingpowertosatisfymostneeds,entirelyfulfillstherequirementsofaself-containeddevice,whichcanbeembeddedinastand-aloneDigitalMusicalInstrumentorsoundingobject.

PureDataonBelaPureData10(Pd)isapopularopensourcegraphicalprogramminglanguagewidelyusedbymusiciansandsounddesignersalike,whichallowsforquickprototypingofsoundandsensormappings.PdpatchesareusuallyrunwithinPditself,orusingthesharedlibrarylibpd11.ThemessagingarchitectureandtheaudioengineofPdwasnotdesignedtobefastandcomputationallyefficientwhichcanleadperformancepenaltiesonplatformswithlimitedcomputationalpower.

�� Figure1:TheBelacape Figure2:UsingaforcesensitiveresistorwithBela

HeavyAudioToolsTheHeavyAudioToolsfromEnzienAudiousePdasafront-endtogenerateoptimisedCcode.ByanalyzingthegraphofconnectionsbetweenobjectsinthePdcode,Heavyiscapableofproducinghigh-performancevectorizedCcodewhichcanoutperformlibpd,makingitparticularlywellsuitedforembeddeddevicesand,moregenerally,hardwarewithlimitedcomputationalpower.Heavyisaproprietary,cloud-basedserviceandthegeneratedcodeislicensedundertheMITnon-commerciallicense.

TheCcodeproducedbyHeavyiswell-suitedtobeintegratedinaXenomaienvironment,asmemoryisallocatedonthestack,thusavoidingsystemcallsduringexecution.AnautomatedscripttakescareofuploadingthePdpatchtoHeavy’sserver,collectthegeneratedCcodeandcompileitontheBelaboard.TheentireprocessgenerallytakeslessthanoneminuteandmostofthetimeisspentcompilingtheCcodeontheBeagleBoneBlack.

libpdMinimalmodificationswererequiredtoportlibpdforBela,theseincludedallowingblocksizesassmallas8samplesperblockandremovingsocketanddiskI/Ofromtheaudiothread.Additionally,thecallstothepthreadfunctionswerewrappedintoXenomaifunctions.TheresultingsharedlibrarycanbelinkedtoaBelaprogramandlibpd_process_float()istheninvokedfromwithinBela’saudiocallback.

DeployingaPdpatchusinglibpdisvirtuallyinstantaneousasitdoesnotrequirecompiling.AssoonasthepatchissavedontheBeagleBone’sfilesystem,theBelaprogramcanberestartedanditwillloadtheupdatedpatch.AnaddedadvantageofusinglibpdisthatitiseasiertoportPdexternalswhentheirsourcecodeisavailable.ThesameprecautionslistedaboveshouldbetakenfornewexternalsinordertomakesurethatnewobjectsdonotintroduceXenomaimodeswitchesintheaudiothread12.

10http://puredata.info11http://libpd.cc/12https://xenomai.org/2014/08/porting-a-linux-application-to-xenomai-dual-kernel/

Page 4: Making High Performance Embedded Instruments with Bela …users.sussex.ac.uk/~thm21/ICLI_proceedings/2016/Practical/... · Making High-Performance Embedded Instruments with Bela and

PerformancecomparisonRunninganexamplepatchcontainingagenerativeaudiocomposition,Heavycodecompiledwiththeclang13compileruses26%oftheCPU.ThesameHeavycode,compiledwithgcc14,occupies43%oftheCPUcycles.Runningthepatchusinglibpduses53%oftheCPU.

Traditionally,thehighest-performanceplatformshavealsoplacedthemosttechnicaldemandsontheprogrammer.Formanyyears,customDSPboardsofferedthebestbalanceofhardreal-timeperformanceandhighprocessingpower,buttheyweregenerallyprogrammedinlow-levellanguagesusingcustomdevelopmentenvironments.High-levelmusicprogramminglanguagesoftencomewithsignificantprocessingoverhead.RunningPureDataonBela,especiallythroughtheHeavyAudioTools,providesaconvenientgraphicalenvironmentwithminimalsacrificeinperformancecomparedtoprogramminginC++.

��(a)Smoothing(b)Re-centering

(c)Differentiating (d)Thresholding

Figure3:UsingPureDataobjectstoprocesssensordata

SensorprocessinginPureDataManyinteractivesystemstakeapproacheswheresensorsaresampledatlowandnon-constantratesandthemostrecentframeofsensordataisusedtomodulateaparticularsonicparameter.Butinactualfact,themeaningofsensordataisoftendeeper,initsbehaviourovertimeoritsfrequencycontent.Ahighsamplingrateyieldsaveryhighbandwidthofinteractionwhichcapturessubtledetailsthatmightbelostatlowersamplerates.Thoughallthesametechniquescouldbeimplementedatcontrolrate,audio-ratesensordatacanhelpreorientthedesigner’sthinkingtobecomemoreawareofthesepossibilities.WhenusingBelawithPd,thisallowstoconvenientlyprocesssensorsignalsusingaudio-rateobjects.Someexamplesinclude:

• SmoothingSomesensorsareinherentlynoisy,forinstanceapotentiometermaygeneratehigh-frequencynoisewhenitisactuated,oraninfra-redopticalsensormaybesubjecttotransientperturbationsfromotheremittingsources.Thenoiseinthesensorreadingsmayleakintotheaudiosignal,dependingonthesignalflow.Aneasyapproachtoremovehigh-frequencynoiseistoapplyalow-passfilterwithanappropriatecut-offfrequency,asinFigure3a.

13http://clang.llvm.org/14https://gcc.gnu.org/

Page 5: Making High Performance Embedded Instruments with Bela …users.sussex.ac.uk/~thm21/ICLI_proceedings/2016/Practical/... · Making High-Performance Embedded Instruments with Bela and

• Re-centeringReadingsfromaccelerometersandothersensorshaveinherentDC-offsetswhichmaybeundesirableforcertainapplications.Aquickwayofremovingthemwhichdoesnotrequirecalibrationisusinganhigh-passfilterwithanappropriatecut-offfrequency,asinFigure3b.

• DifferentiatingSomesound-generatorparametersarebettercontrolledusingthevelocityofasensorreading,ratherthanwiththerawreading.Ahigh-passfilterwithacut-offfrequencyof0,properlyrescaledcanusedforthispurpose,asinFigure3c.

• ThresholdingAmorecomplicatedexampleinFigure3dshowshowtocombinefull-waverectification,smoothing,DCshiftandconstraintothresholdasignal.

ConclusionThereareseveraltradeoffsinvolvedindifferentdigitalmusicalinstrumentdesigntools:processingpower,latency,connectivity,sensorbandwidth,easeofprogrammingandaccessibility.Withanyoftheprogrammingenvironments,BelabringstogethertheconnectivityandCPUpowerofanembeddedLinuxcomputerwiththelowlatencyandprecisesynchronisationofamicrocontrollerandbringsahigh-bandwidthdimensiontosensorprocessing.UsingeithertheHeavyorthelibpdenvironments,Belaisalsosuitableforrapidprototypingusingthewidely-usedPureDatagraphicalprogramminglanguage,withfullaccesstobothaudioandsensors.

Acknowledgements.ThisworkissupportedbygrantsEP/K032046/1,EP/K009559/1andEP/L019981/1fromtheUKEngineeringandPhysicalSciencesResearchCouncilandtheQueenMaryCentreforPublicEngagement.

ReferencesBerdahl,Edgar,andWendyJu.2011.“SatelliteCCRMA:AMusicalInteractionandSoundSynthesisPlatform.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Oslo,Norway,173–178.

Madgwick,Sebastian,andThomasJMitchell.2013.“x-OSC:AversatilewirelessI/Odeviceforcreative/musicapplications.”SMCSoundandMusicComputingConference.Stockholm,Sweden:KTHRoyalInstituteofTechnology.

McPherson,A.P.,andV.Zappi.2015a.“Anenvironmentforsubmillisecond-latencyaudioandsensorprocessingonBeagleBoneBlack.”AudioEngineeringSocietyConvention138.AudioEngineeringSociety.

McPherson,Andrew,andVictorZappi.2015b.“ExposingtheScaffoldingofDigitalInstrumentswithHardware-SoftwareFeedbackLoops.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.BatonRouge,Louisiana,USA:LouisianaStateUniversity,162–167.

McPherson,AndrewP.,RobertH.Jack,andGiulioMoro.2016.“Action-SoundLatency:AreOurToolsFastEnough?”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.Brisbane,Australis.

Piéchaud,Robert.2014.“ALightweightC++Real-TimeActiveControlFramework.”16thRealTimeLinuxWorkshop,October12to13,2014attheCCDCongressCenterDusseldorfcollocatedwithLinuxConEuropeinDusseldorf,Germany.

Webster,Thomas,GuillaumeLeNost,andMartinKlang.2014.“TheOWLprogrammablestageeffectspedal:Revisingtheconceptoftheon-stagecomputerforlivemusicperformance.”ProceedingsoftheInternationalConferenceonNewInterfacesforMusicalExpression.London,UnitedKingdom:Goldsmiths,UniversityofLondon,621–624.

Zappi,V.,andA.McPherson.2014.“DesignandUseofaHackableDigitalInstrument.”ProceedingsoftheInternationalConferenceonLiveInterfaces.Lisbon,Portugal,208–219.


BELA-BELA LOCAL MUNICIPALITY Final IDP-Budget-PMS Process Plan.pdf · for Bela-Bela Local Municipality. This process plan is based on the unique character and circumstances of Bela-Bela

Proposed Development of a Pig Farm near Bela-Bela, … delineation and functional assessment for the Proposed Development of a Pig Farm near Bela-Bela, Limpopo Province. HUM-BEL

AFRICA BELA

20015 bela tehnika

Lermontov Bela.doc  · Web viewMikhail Lermontov [Bela] Bela Bela [I] Ай Я [was] воз был [traveling] traveling путешествие [along] элонг вперед [the]

Nitko, Bela Hamvas

Bela Borsodi - cs.brown.edu

Bela Bartok

BELA-BELA LOCAL MUNICIPALITY 2019 -2020 IDP.pdf · Bela-Bela Local Municipality Private Bag X 1609 Bela-Bela 0480 Tel: 014 736 8000 Fax: 014 736 3288 BELA-BELA LOCAL MUNICIPALITY

Bela Borsodi - static.cs.brown.edu

Bela Kezy, URBACT

GEOGRAPHY: PAPER II EXAMINATION NUMBER · Bela-Bela was originally known as Warmbaths or Warmbad. Bela-Bela is a popular health and holiday destination attracting both local and foreign

Bela Drawning

College of Geoinformatics University of West Hungary Bela Markus Bela Markus

Bela Bela Local Municipality Valuation Roll 1 July 2007 Title Valuation Roll... · Bela Bela Local Municipality Valuation Roll 1 July 2007 ... ST124805/2003 Residential Jager Frederik

Zoltan Bela

BELA BELA MUNICIPALITY COVER TOWN WARMBATHS · rem of erf 17 erf warmbaths wolmarans andre arnoldus gilfillan street 559 residential 618 000 10147 1001829 ptn 1 ... bela bela municipality

Bela Knjiga01

Leishmaniasis. haris bela

Microsoft Word - 8. Expenditure Policy Version 5.doc …  · Web view · 2017-01-22expenditure management policy – 2018/2019 . bela bela municipality . expenditure policy . bela

BELA BELA MUNICIPALITY SUPPLEMENTARY VALUATION ROLL … · chokwe bel008770000000000 6002347 418 residential r55,000 . bela bela municipality - supplementary valuation roll 3 period

Bela Bela Local Municipality Collection and... · Web viewAuthor Veronica Baartman Created Date 03/15/2018 03:02:00 Title Bela Bela Local Municipality Last modified by Kulungwana

BELA BELA MUNICIPALITY COVER TOWN PIENAARSRIVIER · bela bela municipality supplementary valuation roll 1 period of valuation roll: 01 july 2012 – 20 june 2016 date of valuation:

Bela Tunika

Bela Jar Bahasa Arab

Lesotho - Mpharane-Bela Bela Road Upgrading Project - PPER.pdf

Bela Criacao FW14

Bela Municipality Valuation Roll Valuation Roll 2007...Bela‐Bela 1175 1 R 21,000.00 300 m2 Residential Stand 1175 Moloelang Shampi Abraham Bela‐Bela 1176 1 R 31,000.00 300 m2 Residential

BELA Bienal

SUMMARY OF THE LIMPOPO REPORT ON THE RATIONALISDATION ... - justice.gov.za · Bela-Bela magisterial district The area of Leesport and Rooiberg are excised from the Bela-Bela magisterial

Biologija 7 Bela

Making High-Performance Embedded Instruments with …andrewm/moro_icli2016.pdf · Making High-Performance Embedded Instruments with Bela and ... //xenomai.org/2014/08/porting-a-linux

Bela Balassa Paper

BELA -BELA LOCAL MUNICIPALITY Document.pdf · bela – bela 0480 tel: 014 – 736 8000 fax: 014 – 736 3288 . ... section g: one (1) year detailed operational plan 88 section h:

Bela g_liptak_instrument_engineers_handbook_v_3339