TheAugmentedAcousmoniumasInterfacePatrickSaint-Denis

UniversityofMontreal,Montreal,Canadapatrick.saint-denis@umontreal.ca

Abstract.ThisarticlediscussestheSwayArrayproject,anaudio-roboticperformancerevolvingaroundaninteractiveelectromechanicalacousmonium.Theperformanceisexaminedfromitstechnicalaspectsnamelythedesignoftheroboticloudspeakerarray,thesoftwarecomponentsdevelopedforitsoperationandtheaudiointeractions.

Keywords:physicalcomputing,robotics,acousmonium,arduino,supercollider,openFrameworks.

IntroductionMotioncanbeoneofthemostexcitingandexpressivetoolsinthephysicalcomputingtoolbox.Bynow,mostpeoplearedesensitizedtomotiononscreen,whetherit’samonitororamoviescreen.Werespondtoit,butonlyasfilteredthroughourconsciousmind.Whenyouseeanobjectthathasitsownphysicalmove,however,ittriggersamorevisceralreaction.(O’SullivanandIgoe2004)

TheSwayArrayproject1isrootedinaninterestforphysicalmovementanditsinteractionwithsound.Atthecentreofthework,amatrixofninerobotic-speakers(seefigure1)interactswithsoundsandmovementsofaperformer.Eachspeakerisabout2.5metershighandalsogeneratessonicactions.Theirindividualpositioningiscoupledwithdifferentaudioprocessesthroughachainofinteractionsinvolvingthebodyandperformativetechnology.

Thefirstsectionofthearticlediscussestheelectromechanicalpartsoftheloudspeakerarraybeforeaddressingitssoftwarecomponents.Physicalandaudiointeractionsarepresentedandthearticleendswithashortexperimentationreport.

PhysicalcomponentsandprotocolsThephysicalmatrixisbasedontheprincipleoftheLesliespeakerwhileextendingitsvocabularyofmovements.TraditionallyassociatedwithHammondorgans,theLesliespeakerisarotaryspeakerwhosefirstapplicationsdatebacktothe1940s.NamedafteritsinventorDonaldLeslie(1911-2004),thisspeakercancreateatremolothroughthedopplereffectcausedbytherotation(Henricksen1981).

Fortheproject,therotatingspeakerisaugmentedwithcomponentsenablingmovementontheverticalaxis(seefigure2).ThesetincludestwoDCgearmotors,tworotaryencodersandamicrocontrollerforclosed-looppositioningonthetwoaxesofmovement.Theencodersareattachedtorotatingpartsandarepulledintomotionbyapulleysystem.Limitswitchesareusedtocalibratethedeviceatstartupandatotalof4096rotationalpositionsandabout1000positionsinelevationarepossibletoachieve.

1https://vimeo.com/143676764

Figure1.Aninteractiveroboticloudspeakerarrayof9units.

Figure2.Acloserlookatasingleunit’sphysicalparts.

Onepieceofaluminumandonesteelpartweretheonlycomponentsthathadtobedesignedspecificallyforthisproject.Thestandsweredesignedusingaluminumextrusions2andallmechanicalparts(gears,motors,couplers,pulleys,pillowblocks,etc.)arepre-manufacturedaluminumassembliesdistributedbyServocity3.Asteelcounterweightattachedtothebackofeachspeakerwascustombuilttocounterbalancetheweightofthehornandallowtheuseofasmallerandquietermotorforverticalmovement.Finally,a“C”shapealuminumframewasalsocustombuilttosecurethedifferentpartsandholdthebearingsoftheverticalarticulation.Atthecenterofthis“C”shapedpart,aslipringof10conductorstransmitspowertotherotatingspeaker,avoidingtheproblemofentanglementofwires.

2Aluminumextrusionsusuallycomeinasetofpartsthatcanbeassembledincomplexstructures,muchlikemeccanomodels.TheextrusionkitusedintheprojectwasdevelopedbytheKATIMteam(www.katim.biz)basednearMontreal,Canada.3Servocity(www.servocity.com)ascompletelineofmechanicalpartsdesignedforhobbyists.Sourcingmanufacturedpartsisessentialtokeepthecostofaphysicalprojectataminimumascustomdesignedpartscanoftenbe20timesormoreexpensivethanmanufacturedones.

Eachspeakerunitisequippedwithapowersupplyandamotorcontroller4formanagingthepolarityandvoltagesuppliedtothemotors.AnArduinoethernet5andateensy6cardcompletethesetandenabletwo-waycommunicationbetweenthespeakersandacentralcomputer.Thecentralcomputercontinuouslysendspositioningtargetsatarateof10timespersecondandreceivestherealpositioningofeveryunitatthesametemporalresolution.Thecommunicationbetweenthecentralcomputerandthespeakersisbuiltontwocommunicationprotocols.First,thepositioningtargetsaresentbyOpenSoundControl(OSC)7fromthemainframetothearduinoethernettowhomareassignedstaticIPaddresses.ThenthedataissentfromtheArduinoboardstotheteensiesovertheserialport.Thedataastotherealpositionofthespeakersgoestheoppositeway,startingfromtheteensytothearduinoovertheserialportbeforereturningtothecentralcomputeroverOSC.

Althoughitispossibletohavethemotorcontrolroutinesdoneonthearduinoboard,theteensyprovedamoreeffectivesolutionbecauseitallowsbetterreceptionofpulsessentbytherotaryencoders.Theprocessorspeedontheteensyandthehighnumberofinputsenablinginterruptroutines8havemadeofthismicrocontrollerboardabetterchoicethanthearduino.Thatsaid,theefficiencywithwhichthearduinoethernetboardallowsIPcommunication9hasensuredthatthiscomponentisalsoessentialtotheproject.

Figure3.Thischartsummarizesallthephysicalcomponentsandcommunicationprotocolsinvolvedintheproject.

SoftwarecomponentsThecommunicationbetweenthecentralcomputerandthespeakerspassesthroughasimulationprogramcodedinopenFrameworks.10Thisprogramallowscompositionofmovementswithoutthephysicaldeviceandfacilitatesthemanagementofpositionswiththeuseofavirtualspeaker.Initially,targetpositionsinelevationandrotationaredeterminedforavirtualloudspeaker.Apositioningdelayisappliedtotherealspeakersasafunctionofthedistancebetweenthemandthevirtualspeaker.Figure4illustratesthisprocesswherethemastervirtualspeakeriswhiteand

4TheSabertoothdual12Amotordriverwaspreferredforitspricingandabilitytobecontrolleddigitallyovertheserialportwithasimpleprotocol.Thecontrollercanachieve127differentspeedsinthetwodirections.5Thearduinoethernetisamicrocontrollerboardthatcomeswithabuild-inethernetportfacilitatingIPcommunicationwiththemicrocontroller.ThisboardisparticularlyhandywhendistancebetweenthemicrocontrollerandthecomputerbecomesanissueorwhenIPlikecastings(multicast,broadcast,etc.)aredesired.6TheteensyisamicrocontrollerboardcompatiblewiththearduinoIDE.It’sprocessorisfasterthanmostarduinosandhasinterruptsoneverypins.7OpenSoundControlisaprotocolfortransmittingdataconceivedforrealtimeapplications.Itismostlyusedinnetworkcontexts.8Aninterruptserviceroutine(ISR)isafunctionthathasahigherprioritythanthemainloop.Itgetstriggeredbyapulsereceivedontheportit’sattachedto.9OnecandostandardOSCcommunicationwithcontributedlibrariessuchasArdOSC(https://github.com/recotana/ArdOSC)10openFrameworks(www.openframeworks.cc)isaC++toolkitforcreativecoding.

therealspeakersareyellow.Movementsareinitiatedonthevirtualspeakerandechoedontherealspeakerswithadelaycalculatedinnumberofframespersecondtimesthedistancefromthevirtualspeaker.11

Figure4.Imagefromthesimulationprogram.Thevirtualspeakerisinwhiteandleadstheotherspeakersinyellowforphysicalmovement.Virtualspeakerismobileandcanitselfbepositionedanywherewithinthematrixlimits.

Aninertialmeasurementunit(IMU)placedontheheadoftheperformerrecoversancillarymovements(seefigure5).TheorientationdatacollectedbytheIMUissentwirelesstothemainframeviatheuseofanXBeeantenna12.Thisdataisthenusedtocontrolthevirtualspeakersothatheadmovementsareeventuallyechoedonthephysicaldevice.

Figure5.AnIMUsendstheorientationoftheheadwhichisechoedontheloudspeakerarray.Performer:SarahAlbu.

AudiointeractionsandcompositionofphysicalmovementsaremadeinSuperCollider(Wilson,Cottle,andCollins2011).Inthisaudioprogrammingenvironment,managementoflocationdata(headpositioningviaIMU,desiredpositioningviathesimulatorandrealpositioningviamicrocontrollers)isassociatedwithdifferentaudioprocesses.Thus,

11https://vimeo.com/12182124612DigiInternational’sXBeeisaradiomodulethatfacilitateswirelessserialcommunication.

interactionsbetweenthephysicalmovementoftheperformerandthoseoftheloudspeakerarrayareexpandedtosound.

Duringtheintroduction(6min15sec)13,aseriesofheadmovementsareinitiatedbytheperformer.Thesemovementsarethenechoedontheroboticmatrixandtheverticalpositioningofeachspeakerissentbacktocontrolalowpassfilter.Laterintheperformance(10min45sec),thisdataisassociatedwithalivetranspositionofthevoice.Inthelastsection(12min45sec),anaudiogranulationprocessisassociatedwitheachspeakerbasedonthefollowingmapping:

• Elevationofspeaker:rateofgrainsandnumberofgrainspersecond.• Rotationofspeaker:grainlength(azimuthspreadovertwocompleterotations).• Distancefromthevirtualspeakerinthesimulator:Graintriggerdelay.

SummarychartThefollowingchartsummarizesthedataflow.UpstreamanIMUissendingdataastothepositioningoftheperformer’shead.ThisdataisreceivedandhandledinSuperColliderbeforebeingsenttothesimulatorbyOSC.PositioningtargetsarethensentbackfromthesimulatortoSuperColliderandtransmitteddirectlytothearduinoethernetofeachspeakerinOSC.Targetsarethenpassedtotheteensiesovertheserialport.Controlcommandsaresenttoamotorcontrolleroveranotherserialportavailableontheteensy.Realpositionsareroutedbacktothecentralcomputerusingthereversechainofprotocols10timespersecond.Finally,whenthedatareturnstoSuperCollider,itismappedtoparametersofaudioprocessesthuscompletingthechainofinteractions.

Figure6.Dataflowdiagram

ConclusionTheloudspeakerarrayallowscompositionofphysicalmovementsbutitsuseisalsoorientedtowardssymboliccontent.Inaway,thisstrangeaugmentedacousmoniumbecomesacharacterinitself.Itsvisualappearancecreatesimagesthatcanevokedifferentnaturalphenomena.Allusionstomassphenomenasuchasbirdflocking,fishschools,swayingwheatandtelescopearraysarealmostinherenttoitsvisualaspect.Theinteractionsbetweentheperformer’sheadandthematrixalsogivestheloudspeakersananthropomorphicqualitywhichaddsareferentiallayertoitsuse.Furtherresearchsurroundingthesymbolicresonanceoftheloudspeakerarrayneedstobedoneduringthenextcreativecycleoftheproject.

Finally,thespeakershaveacertaininertiaandconsequentlysignificantspeedlimitationintermsofmovement.Researchneedstobedoneregardingthisaspectinordertoupgradethemotorization.Strongerandfastermotorsneedtobesourced.Thiswillbeaparticularchallengesinceacompromisebetweenpricing,speedandmotor/gearnoise

13https://vimeo.com/143676764andhttps://vimeo.com/130647157

needstobefound.Thisspeedlimitationparticularlyfavorslargesonicgesturesovershortsoundswhenitcomestodesigningtheaudiointeractionsandcomposition.Inthisway,thisaugmentedacousmoniumcanalsobeseenasamusicalinterfaceinitself,withitsspecificitiesandlimitations,offeringasingularperspectiveonsoundcreation.

ReferencesHenricksen,CliffordA.1981.“UnearthingtheMysteriesoftheLeslieCabinet.”RecordingEngineer/ProducerMagazine,April.

O’Sullivan,Dan,andTomIgoe.2004.PhysicalComputing.CourseTechnology.

Wilson,S.,D.Cottle,andN.Collins.2011.TheSuperColliderBook.MITPress.