Colliding:aSuperColliderenvironmentforsynthesis-orientedlivecoding
GerardRoma
CVSSP,UniversityofSurreyGuildford,[email protected]
Abstract.Oneofthemotivationsforlivecodingisthefreedomandflexibilitythataprogramminglanguageputsinthehandsoftheperformer.Atthesametime,systemswithexplicitconstraintsfacilitatelearningandoftenboostcreativityinunexpectedways.Somesimplifiedlanguagesandenvironmentsformusiclivecodinghavebeendevelopedduringthelastfewyears,mostoftenfocusingonmusicalevents,patternsandsequences.Thispaperdescribesaconstrainedenvironmentaimedatexploringthecreationandmodificationofsoundsynthesisandprocessingnetworksinrealtime,usingasubsetoftheSuperColliderprogramminglanguage.Thesystemhasbeenusedineducationalandconcertsettings,twocommonapplicationsoflivecodingthatbenefitfromthelowercognitiveload.
Keywords:Livecoding,soundsynthesis,liveinterfaces
IntroductionTheworldofspecializedmusiccreationprogramminglanguageshasbeengenerallydominatedbytheMusic-NparadigmpioneeredbyMaxMathews(Mathews1963).ProgramminglanguageslikeCSoundorSuperColliderembracethedivisionofthemusicproductiontaskintwoseparatelevels:asignalprocessinglevel,whichisusedtodefineinstrumentsasnetworksofunitgenerators,andacompositionallevelthatisusedtoassembleandcontroltheinstruments.AnexceptiontothisisFaust,whichisexclusivelyconcernedwithsignalprocessing.MaxandPureDatauseadifferenttypeofconnectionforsignalsandevents,althoughunderasimilarinteractionparadigm.Similarly,Chuckhasspecificfeaturesforconnectingunitgeneratorsandcontrollingthemalongtime.Theavailabilityoflanguagescapableofgeneratingmusicinrealtimehasfosteredthedevelopmentoflivecoding(Collinsetal.2003),whichhastheadvantageofgivingtheaudiencethepossibilitytoreadcomputermusicperformancesinawaythatiscomparabletoimprovisationwithphysicalinstruments.Livecodingisalsohelpfulinclassroomenvironments,allowingstudentstograspthementalprocessinvolvedinusingaprogramminglanguageorcommandline.Onefeaturethathasinspiredthelivecodingpracticeisthefreedomandpowerthataprogramminglanguagegivestotheperformer.Ontheotherhand,restrictedenvironmentsoftenresultinunexpectedcreativeoutcomes.Facingtheinfinitepossibilitiesofferedbycomputers,musiciansandartistscommonlydesigntheirsystemsonthebasisofconstraints(Magnusson2010).Constraintscanbeseenastherulesthatdefineagame,andthusareconsideredbymanytoplayanessentialroleincreativity(Boden2004;Merker2006).
Someconstrainedlanguagesareavailableforlivecoding,mostoftenwithastrongfocusonmusicalevents(Magnusson2011;McLean2014).IntheMusic-Nparadigm,thismeansnotcreatingnewinstrumentsonstage,butimprovisingnewcontrolsequencesforpre-definedinstruments.Thesystempresentedinthispaperexplorestheotherside.Inthissense,theconceptof"synthesis-oriented"livecodingcanbeopposedto"event-oriented"livecoding.Thereis,asamatteroffact,alongtraditioninchallengingthedistinctionbetweencompositionandtimbre,preciselyonthebasisofthepossibilitiesofferedbycomputers(Döbereiner2011).Underthispointofview,musicaleventscanbeseenassignals,andmusiccanbecreatedusingexclusivelysignalprocessingnetworks.
Withthereleaseofversion3(McCartney2002),theSuperColliderlanguagewassplitintotwoseparateprograms:thesynthesisserver(scsynth)andthelanguageinterpreter(sclang).Asubsetofthelanguageisusedtospecifysynthdefinitions,whichthescsynthservercanexecute.ItisnotuncommontofinddiscussionsamongtheSuperCollidercommunityoncreatingmusicstructurespurelyintheserverside.Amongotherreasons,creatingsynthdefinitionscanbelessdemandingwithrespecttodealingwiththefulllanguageandthedistributedarchitecture.Giventheamountofunitgeneratorsavailable,focusingonthesynthesissideofSuperColliderisbothsimpleandpowerful.Intermsofuser
interface,theevolutionofSuperColliderasageneralpurposelanguageledtoastrugglebetweentheneedofa"properIDE"forobject-orientedprogramming,andtheinterestinthedocument-orientedrichtexteditorthatwasavailableonOSX.
Inthiscontext,CollidingwasdesignedasaconstrainedinterfaceforcreatingSuperCollidersynthdefinitions.Apartfrommusiccreationandperformance,thefocusonsynthesisprovidesacompellingenvironmentformusicandsignalprocessingeducationsettings,allowingeasyexperimentationwithawidevarietyofsynthesistechniques.Theideaofcreatingasimplifiedenvironmentemergedwhenobservingengineeringstudentstryingtocreateproceduralprogramswithalltheasynchronouscallsrequiredtostarttheserver,createasynthdefinitionandinstantiateit.Loadingaudiobuffersandnetworkresourcesrequireasynchronouscallsaswell,whichassumeanunderstandingofanonymousfunctions.Boththeclassroomandtheconcertenvironmentsbenefitfromthereducedcomplexityandfocus.Inthecaseofmusicperformance,theconstraintisinpartaesthetic,butstillmanydifferentstylesofmusiccanbeplayed.Therestofthepaperdescribesthefunctionalitiesimplementedsofarandtheuseoftheprograminbotheducationandperformance.
RelatedworkConstrainedenvironmentsforlivecodingarecommoninclassroom-orientedapplications.Twowell-knownexamplesareEarsketch(Freemanetal.2014)andSonicPi(AaronandBlackwell2013).ThefirstisbasedonanApplicationProgrammingInterface(API)whichcanbeusedinJavascriptorPythoninawebenvironment.ThisAPIiscomplementedwithalibraryofaudioloopsthatcanbemanipulatedandpositionedinanaudiosequencertimelinethroughtheprogrammingAPI.ThesecondexposesalsoabasicAPI,inthiscaseasaRubydomain-specificlanguagethatcontrolsSuperCollidersynths.TheprogrammingenvironmentisdesignedtorunonaRaspberry-Piembeddedcomputer.Bothenvironmentsofferlimitedcapabilitiesintermsofsynthesis.Themainideabehindlivecodingmusicenvironmentsintheclassroomistoengagestudentsintoprogrammingbydoingsomethingfunandcreative.
Suchenvironmentsallowexperimentationwithgenerativemusic,butarenotsowellsuitedforsynthesis-orientedmusicorlearning.Whilethesesystemsgenerallyfocusonlearningprogrammingskills,Collidingemphasizessignalprocessing,requiringonlybasicprogrammingconcepts.Inthissense,perhapsamoresimilarapproachwouldbeusingthecompiledlanguageFaustinaninteractivesetting.FaustLive(Denouxetal.2014)isajust-in-timecompileraimedatfacilitatingthiskindofset-up,however,itdoesnotprovideaninterfaceforcoding.FaustisusedlivebyJuliusO.SmithforteachingsignalprocessingusingEmacs1.
Asthenamesuggests,CollidingismainlyinfluencedbyProcessing(ReasandFry2006)anditscousintheArduinoIDE.Bothhavesucceededincreatingsimplifieddevelopmentenvironmentsforactivitiesthattraditionallyrequiredspecializedtraining.ProcessingwasoriginallypresentedasasubsetoftheJavalanguagethatcanbeembeddedinJavaprograms.ArduinooffersasimpleCAPIforembeddedsystems.Similarly,CollidingusesasubsetoftheSuperColliderlanguage.
AnothermajorinfluenceisixiLang(Magnusson2011).Whileheavilyfocusedonmusicalevents,ixiLangstressestheimportanceofreducingcomplexityformusiclivecoding.LikeixiLang,Collidingcanbeseenasa"SuperColliderparasite",inthiscaseforsynthesis-orientedlivecoding.
�Interface
Overview
TheprogramfollowstheinteractionparadigmofprototypingeditorslikeProcessing.Theinterface(Figure1)allowstheusertocreateupto8tabs,eachwithacodeeditingwindowconfiguredwithalargefontsize.Thisencouragesshortsnippetsandfacilitatesreadability.Thenumberoftabs(andalsoofbuffers,asdescribedbelow)isnotcompletely
1https://www.youtube.com/watch?v=2lEt7dsziO0
arbitrary.Itisgenerallyagreedthatsimilarnumbersofelementsarerelatedwithworkingmemorycapacity(Miller1956).Itiscommontofind8-channellimitsinmusicproductionhardware(smallmixers,oldtaperecorders)orsoftware.Themainactionsconsistincompilingthecode(forerrorchecking),runningit(whichresultsinapotentiallyinfinitesoundstream),andstoppingthesound.Theseactionscanberunthroughkeyboardshort-cutsorusingasetofbuttonsbelowthecodewindow.Anothersetofbuttonsinthetoprightcornerexposesproject-leveloperations(addingtabs,gettinghelpforthecurrentlyselectedtext,loadingandsavingprojects,andapanicbuttonthatstopsallrunningprocesses).Textcolorinthecodeeditingwindowisusedtoindicatecompilationstate.Whitemeansthecodehasbeencompiled,greymeansitisbeingedited,andredmeansthereissomeerror.
Figure1:TheCollidinginterface
Collidingofferstwomodesofoperation:"synthediting"and"advanced".Thesyntheditingmodeaffordsthemetaphorofeditingapatchinahardwaresynthesizer,butitcanalsobeusedforlivecoding.Inthiscase,thecodeisassumedtobeasynthdefinition,andtheenvironmentprovidessomepredefinedvariablesandthemeanstotriggernotes,ifdesired.Inadvancedmode,thecodecanbeanything"playable".InSuperColliderthisisachievedbycompilingthecodeandmakingitthesourceofaNodeProxy,whichallowsusingtheinterfacewithJITLib(Rohrhuber,deCampo,andWieser2005)asaback-endforlivecoding.Thisincludesalsoeventpatterns,althoughthispossibilityhasnotbeenexploredorspecificallysupported.OneimportantdifferenceisthatinadvancedmodetheaudiooutputchanneliscontrolledbyJITLib,whileinsynthesiseditingmodeitisspecifiedexplicitly.Thusin"synthediting"modeitissimplertoaddressmultipleoutputs.Forsynthesisorientedlivecoding,i.e.,inSuperCollidertalk,ifonlyservercodeisevaluated,bothmodescanbeused,althoughtheadvancedmodeallowstersercode.
Afeedbackpanelbelowthecodewindowandbuttonsisusedforerrorreporting.Longstacktracescanbeoverwhelmingforbeginners,andarealsoundesirableinlivesituations.Whenthecoderesultsinastacktrace,thesystemselectstherelevantmessageandhighlightstheoffendingline(Figure2).Inordertoprovidesuchfeedbackwithoutdisruptingthecurrentinterpreter,itwasnecessarytoimplementcompilationasaseparateprocess,bywritingthesnippettofileandcallingsclangtoparseit.Allofthishappensunderthehood.
AllcodeisruninSuperCollider’sinternalserver,whichfacilitatesvisualizationoftheoutput.Thebottomendcontainsanoscilloscopethatprovidesvisualfeedbackabouteverythingthatishappeningintheserver.
Synthediting
Additionalfeaturesareprovidedinthesyntheditingmode.Inthismode,thecodeiswrappedtobuildandinstantiateasynthdefinition,withseveralpre-definedparameters.Awindowrepresentingthecomputerkeyboardallowstriggeringnotesusingthesynththathasbeendefinedinthecurrenttab.Thekeyboardusesanisomorphicmapping.Aslidernexttothecodewindowallowscontrollingthegain.Thisresultsinthefollowingpre-definedvariables.Itisuptothecodertomakeuseofthem.
• key:TheMIDInumbercorrespondingtothekeypressedinthecomputerkeyboard.4• freq:ThefrequencyinHzcorrespondingtothekey.• gate:Agateinputforenvelopes.• amp:Thevaluedefinedbytheslider.
Itistrivialtoextendthissystemwithotherwidgets,forexampleadditionalsliderscouldbeattachedtothekeyboard.Forthemoment,thesystemprovidestheminimumtoleveragetheinputdevicesalreadyavailableineverycomputer.ThemouseisaccessedusingthetraditionalSuperColliderunitgenerators.ThisisaconvenientsetupforclassroomPCsandlaptopperformances.ItisalsoeasytoextendtheconcepttoMIDIcontrollers.
Figure2:Errorreporting
Buffersandfiles
Besidessynthdefinitions,anotherareathatrequiresasynchronouscallsisdealingwithaudiofiles.ThisisoftenasourceofconfusioninSuperCollider.Graphicalinterfacesaregenerallyappreciatedforbrowsingthefilesystemandaccommodatingthetimeforfileloading.Collidingprovides8slotsforbuffermanagement.Buffersaresimplyaccessedviatheirbuffernumber(0-7).Aseparatewindowisusedforloading,visualizingandpre-listeningeachbuffer.Empty
bufferscanbeallocatedforrecordingsignals,andsmallbufferscanbeconvertedtowavetableformatforwavetablesynthesis.Inadditiontoloadingsoundfilesfromdisk,thesystemincludesaninterfacefortheFreesoundquark2,sothattheusercanmakebasictextqueriestotheFreesounddatabase(Akkermansetal.2011),anddownloadsounds.Forthemoment,thesearchisrestrictedtowavfiles.However,ifoggsupportiscompiledwiththeSuperColliderbinaries,itshouldbepossibletoaccessthewholeFreesounddatabaseviahigh-qualityoggpreviews3.
Finally,somebasicprojectmanagementfacilitiesareprovided.Aprojectconsistsofthetextineachtabandallthesoundfilescorrespondingtothebufferslots.Thisisespeciallyusefulforlearningenvironments,butcanalsobeusefultopreservethestateofalivecodingsession.Inthefutureitmightbeinterestingtoaddsomesimpleinterfacetoversioncontrol(e.g.git)whichisalsousuallyhardforbeginners.
InitialExperiencesCollidingwasdevelopedwhiletheauthorwasattheSonologydepartmentatEscolaSuperiordeMúsicadeCatalunya(ESMUC).Whilenoformalevaluationwasconducted,itwasusedin4overlappingsemestercoursesbetween2013and2015,coveringbasicandadvancedsynthesistechniques.Duringthepreviousyear,theSuperColliderlanguageandIDEwereused.ThedifferencewithrespecttousingplainSuperColliderwasdramatic.ByfreeingstudentsfromlearningtousetheSuperCollidereditorandlanguage,itwaspossibletofocusonsynthesisandprocessingasopposedtoprogramming.Atthesametime,thankstothelargenumberofunitgeneratorsavailableandthegeneralpowerofSuperCollideritwaspossibletoexploremanydifferentalgorithmsandconcepts,frombasicsubtractiveandmodulationtechniquestophysicalandspectralsynthesis.CollidingwasusedasakindoflabnotebookeditorthatcouldruninclassroomPCsandstudentlaptops.AssignmentsconsistedinCollidingprojectswheredifferentvariantswereexploredineachtab.Asmallcodesnippetwasusuallyseeded.
Sincestudentsweregenerallybeginnersincomputerprogramming,averysimplegrammarwasadoptedwhichworkedforallassignments(ascanbeseeninthefigures):eachline,exceptthelastone,isavariabledefinitionandassignment,wherepartofthesignalprocessinggraphisrepresented.Thissuggeststhat,forthepurposeofdefiningsynthdefinitions,thelanguagecouldbefurthersimplified.
Theonlylimitationwastheprojectmanagementfeature,whichwasnotalwaystrustedbythestudentsbecauseofitssimplicity.Howeverincaseofdoubttheprojectstructurewassimpleenoughandtextsnippetscouldbeopenwithastandardtexteditor.Somesynthesistechniquesthatrequireadditionalprocessing,suchasinthecaseofwavetablesynthesis,weremoredifficulttoaccommodate.Foradvancedtechniques,suchasspectralmodelingorvocoding,itwasgenerallyconvenienttouse"black-box"unitgeneratorsavailablefromsc3-plugins4.
Collidinghasalsobeenusedinseverallivecodingperformancesbytheauthor.Upto8synthesisprocessesmayruninparallelwhileoneofthemisbeingedited.Thebigfont,oscilloscope,andtheconstrainttosynthesis-basedcodingcontributetothereadabilityoftheperformance.Inlivecoding,thesimplifiedgrammarisnotneeded,andinsteadatersesyntaxismorecommon,inlinewiththeuseofTwitterforsharingsmallSuperColliderprograms.
ConclusionsTheSuperCollidersyntaxforbuildingsynthesizersallowsforcreatingsophisticatedpatcheswithveryfewlinesofcode.Suchconcisedescriptionsoftencontrastwiththecomplicationsoneendsdealingwithwhentryingthesamethingswithgraphicalpatchingsystemsandmodularsynthesizermetaphors.However,learninganobject-orientedlanguage
2https://github.com/g-roma/Freesound.sc3Thisdependsontheversionoflibsndfile,whichforhistoricalreasonswasfixedtoanolderversiononOSXbinaries.AtthetimeofthiswritingtheproblemisfixedinSuperCollider3.7binaries,sotherestrictiontowavforFreesoundsearcheswillberemoved.4https://github.com/supercollider/sc3-plugins
enrichedwithfunctionalprogrammingconstructscaneasilyscaretinkerersandmusicianswithlittleprogrammingbackground.TheCollidingenvironmentprovidesasimplifiedinterfacethatisparticularlyusefulwhenfocusingonsoundsynthesisandprocessing,asopposedtoevent-basedmusiccompositionandperformance.Thisfocuscanbeseenasadesignconstraintthatisusefulineducationalenvironmentsandinsynthesis-orientedlivecoding.Whilethecurrentimplementationmainlystressestheconceptandinterface,theseideascanbefurtherextendedwithoutleavingtheSuperColliderlanguage,bydefiningmorehelpervariablesandfunctions.Also,thesystemisamenabletoencapsulationofsynthesisprocessesasunitgenerators,apracticeusedbysomelivecoderswhichallowsgrowingapersonalsonicpalettewithoutlosingsimplicity.Addingsupportforthiskindofencapsulationwillbeinvestigatedinthefuture.ThecodeisavailableasaSuperColliderquark,andcanbedownloadedfromthegithubrepository5.
AcknowledgementsTheauthorwassupportedbyEscolaSuperiordeMusicadeCatalunya(ESMUC)andtheMusicTechnologyGroup(MTG)atUniversitatPompeuFabra(UPF)duringthedevelopmentofthiswork.
ReferencesAaron,Samuel,andAlanFBlackwell.2013.“FromsonicPitoovertone:creativemusicalexperienceswithdomain-specificandfunctionallanguages.”ProceedingsofthefirstACMSIGPLANworkshoponFunctionalart,music,modeling&design.ACM,35–46.
Akkermans,Vincent,FredericFont,JordiFunollet,BramDeJong,GerardRoma,SteliosTogias,andXavierSerra.2011.“Freesound2:Animprovedplatformforsharingaudioclips.”KlapuriA,LeiderC,editors.ISMIR2011:Proceedingsofthe12thInternationalSocietyforMusicInformationRetrievalConference;2011October24-28;Miami,Florida(USA).Miami:UniversityofMiami;2011.InternationalSocietyforMusicInformationRetrieval(ISMIR).
Boden,MargaretA.2004.Thecreativemind:Mythsandmechanisms.PsychologyPress.
Collins,Nick,AlexMcLean,JulianRohrhuber,andAdrianWard.2003.“Livecodinginlaptopperformance.”
Organisedsound8(03):321–330.
Denoux,Sarah,StéphaneLetz,YannOrlarey,andDominiqueFober.2014.“FAUSTLIVE,Just-In-TimeFaust
Compiler...andmuchmore.”TechnicalReport,GRAME.
Döbereiner,Luc.2011.“Modelsofconstructedsound:Nonstandardsynthesisasanaestheticperspective.”ComputerMusicJournal35(3):28–39.
Freeman,Jason,BrianMagerko,TomMcKlin,MikeReilly,JustinPermar,CameronSummers,andEricFruchter.2014.“EngagingunderrepresentedgroupsinhighschoolintroductorycomputingthroughcomputationalremixingwithEarSketch.”Proceedingsofthe45thACMtechnicalsymposiumoncomputerscienceeducation.ACM,85–90.
Magnusson,Thor.2010.“Designingconstraints:Composingandperformingwithdigitalmusicalsystems.”ComputerMusicJournal34(4):62–73.
Magnusson,Thor.2011.“theIxiLang:aSupercolliderParasiteforLiveCoding.”Icmc2011,no.August:503–506.
Mathews,MaxV.1963.“Thedigitalcomputerasamusicalinstrument.”Science142(3592):553–557.
5https://github.com/g-roma/Colliding
McCartney,James.2002.“Rethinkingthecomputermusiclanguage:SuperCollider.”ComputerMusicJournal26(4):61–68.
McLean,Alex.2014.“Makingprogramminglanguagestodanceto:livecodingwithtidal.”Proceedingsofthe2ndACMSIGPLANinternationalworkshoponFunctionalart,music,modeling&design.ACM,63–70.
Merker,BjornH.2006.“Layeredconstraintsonthemultiplecreativitiesofmusic.”MusicalCreativity:MultidisciplinaryResearchinTheoryandPractice,pp.25–41.
Miller,GeorgeA.1956.“Themagicalnumberseven,plusorminustwo:somelimitsonourcapacityforprocessinginformation.”Psychologicalreview63(2):81.
Reas,Casey,andBenFry.2006.“Processing:programmingforthemediaarts.”AI&SOCIETY20(4):526–538.Rohrhuber,Julian,AlbertodeCampo,andRenateWieser.2005.“Algorithmstodaynotesonlanguagedesignforjustintimeprogramming.”InternationalComputerMusicConference.291.
�
