FYP Final Presentation:FYP Final Presentation:Distributed Audio MixingDistributed Audio Mixing

Daire O'Neill, Final Year Electronic Daire O'Neill, Final Year Electronic EngineeringEngineering

Project Supervisor: Dr Peter CorcoranProject Supervisor: Dr Peter Corcoran

Co-Supervisor: Frank CallalyCo-Supervisor: Frank Callaly

Structure of presentationStructure of presentation

Project OverviewProject Overview

How I approached this projectHow I approached this project

Issues encounteredIssues encountered

Implementation of final systemImplementation of final system

Objectives achievedObjectives achieved

ConclusionsConclusions

Project OverviewProject Overview

The ultimate goal of this project was to design The ultimate goal of this project was to design and implement a client-server system on Linux and implement a client-server system on Linux that allowed mixing of audio files from different that allowed mixing of audio files from different sources, over a networksources, over a network

A server was needed that could receive audio A server was needed that could receive audio data from multiple clients and mix the audio, as data from multiple clients and mix the audio, as in a traditional recording studioin a traditional recording studio

This system would allow musicians in different This system would allow musicians in different locations to record together, as if they were in locations to record together, as if they were in the same roomthe same room

GoalsGoals

The goals for this project were as follows:The goals for this project were as follows:– To install Linux on a desktop PC and become To install Linux on a desktop PC and become

familiar with the basics of using Linuxfamiliar with the basics of using Linux– To install and use ALSA to capture and To install and use ALSA to capture and

playback audio on a Linux machineplayback audio on a Linux machine– To implement a server capable of receiving To implement a server capable of receiving

audio data from clients and mixing the audioaudio data from clients and mixing the audio– To implement a method of synchronising To implement a method of synchronising

separate audio filesseparate audio files

How I approached the How I approached the projectproject

My first task was to install Linux on a PC. I My first task was to install Linux on a PC. I experimented with both Fedora and Ubuntu experimented with both Fedora and Ubuntu distributions, and found Ubuntu easier to usedistributions, and found Ubuntu easier to useAfter becoming familiar with using Linux, I After becoming familiar with using Linux, I began to search for a suitable application on began to search for a suitable application on which to base my project, as it was clear that I which to base my project, as it was clear that I could not hope to write an entire multitrack could not hope to write an entire multitrack recording program by myselfrecording program by myselfI needed a multitrack recording application that I I needed a multitrack recording application that I could configure to use files that were received could configure to use files that were received over a networkover a network

How I approached the How I approached the projectproject

I initially downloaded and installed I initially downloaded and installed multitrack programs such as Ardour and multitrack programs such as Ardour and Audacity, hoping to modify the source code Audacity, hoping to modify the source code such that these programs could connect to such that these programs could connect to a server to receive audio dataa server to receive audio data

However these programs are sophisticated However these programs are sophisticated (and therefore complex), so the source (and therefore complex), so the source code was difficult for me to understandcode was difficult for me to understand

How I approached the How I approached the projectproject

To go back to basics, I then concentrated on To go back to basics, I then concentrated on writing Python scripts to capture and playback writing Python scripts to capture and playback audioaudio

I successfully completed this by using Pyaudio, I successfully completed this by using Pyaudio, a Python module that provides Python bindings a Python module that provides Python bindings for the Linux PortAudio applicationfor the Linux PortAudio application

I then gained understanding of the actual code I then gained understanding of the actual code used in the recording and playback of audioused in the recording and playback of audio

However I still needed an application that could However I still needed an application that could provide multitrack playback of audio files...provide multitrack playback of audio files...

EcasoundEcasoundEcasound proved to be the solution to these Ecasound proved to be the solution to these problemsproblems

As it is a command line based recording As it is a command line based recording application, I could control it the way I wanted by application, I could control it the way I wanted by using shell scripts, eliminating the need to modify using shell scripts, eliminating the need to modify the actual source codethe actual source code

Ecasound can easily be automated by writing Ecasound can easily be automated by writing shell scripts to execute audio playback and shell scripts to execute audio playback and recording functionsrecording functions

Many complex functions can be implemented in Many complex functions can be implemented in Ecasound by passing the required arguments Ecasound by passing the required arguments from the command linefrom the command line

FTP ServerFTP Server

My next task was to implement a server that My next task was to implement a server that could receive audio files from multiple clients and could receive audio files from multiple clients and mix themmix them

The server I used was VSFTPD. I chose this The server I used was VSFTPD. I chose this because of it's stability and ease of usebecause of it's stability and ease of use

I then wrote a series of shell scripts that could I then wrote a series of shell scripts that could upload and download audio data from the server, upload and download audio data from the server, and provide recording and mixing functions, thus and provide recording and mixing functions, thus implementing the final systemimplementing the final system

The Finished SystemThe Finished System

By executing the appropriate shell script, By executing the appropriate shell script, the user (client) can:the user (client) can:– Download an audio track (in wave file format) Download an audio track (in wave file format)

from the serverfrom the server– Playback this audio track and mix their own Playback this audio track and mix their own

audio input during playbackaudio input during playback– Record the desired mix of the downloaded Record the desired mix of the downloaded

track and their own track as a wave filetrack and their own track as a wave file– Upload the new mix to the serverUpload the new mix to the server

The system was set up for use by 2 client The system was set up for use by 2 client PCs, both running Linux PCs, both running Linux

System FlowchartSystem Flowchart

The Finished SystemThe Finished SystemThe server retrieves the wave files uploaded by the The server retrieves the wave files uploaded by the clients from their directories, mixes the audio using clients from their directories, mixes the audio using Ecasound and returns the resulting wave file to the Ecasound and returns the resulting wave file to the clients directoriesclients directories

Each client can then download the new audio file (a Each client can then download the new audio file (a mix of both clients audio tracks), and add more mix of both clients audio tracks), and add more tracks as desiredtracks as desired

In this way, audio tracks from multiple clients are In this way, audio tracks from multiple clients are layered to create a finished audio mix, as in a layered to create a finished audio mix, as in a recording studiorecording studio

For example, Client 1 records a vocal track, then For example, Client 1 records a vocal track, then Client 2 mixes a drum track with this, Client 1 then Client 2 mixes a drum track with this, Client 1 then adds a bass track, etcadds a bass track, etc

Using EcasoundUsing Ecasound

All audio files used in this project are All audio files used in this project are recorded using PCM (uncompressed) wave recorded using PCM (uncompressed) wave file format, sampled at 44.1kHz with a file format, sampled at 44.1kHz with a resolution of 16 bitsresolution of 16 bits

Although this is the default for Ecasound, Although this is the default for Ecasound, this can easily be changed for different this can easily be changed for different sampling ratessampling rates

Using EcasoundUsing EcasoundA fundamental principle of Ecasound is the use of A fundamental principle of Ecasound is the use of “chains”“chains”

Chains can be considered as virtual “audio Chains can be considered as virtual “audio cables”, with a specified input and output. They cables”, with a specified input and output. They are used to route audio in Ecasound.are used to route audio in Ecasound.

In the following example syntax, a chain is set up, In the following example syntax, a chain is set up, the input set as a wave file (test.wav) and the the input set as a wave file (test.wav) and the output routed to the soundcard (/dev/dsp) :output routed to the soundcard (/dev/dsp) :– ecasound -a:1 -i test.wav -o ecasound -a:1 -i test.wav -o /dev/dsp/dev/dsp

This results in playback of the audio file This results in playback of the audio file “test.wav”.“test.wav”.

Example scriptsExample scripts

The next slide shows an example of one of the The next slide shows an example of one of the scripts written to control Ecasound for this scripts written to control Ecasound for this project, called “monitor+record”project, called “monitor+record”

When executed, this script starts Ecasound and When executed, this script starts Ecasound and plays back the latest version of the “current mix” plays back the latest version of the “current mix” as downloaded from the server, and also as downloaded from the server, and also records any audio that is input from the records any audio that is input from the soundcardsoundcard

Monitoring the output allows the user to record Monitoring the output allows the user to record in time with the backing track. The mixed audio in time with the backing track. The mixed audio data is written to a wave file (“client2.wav”).data is written to a wave file (“client2.wav”).

““monitor+record”monitor+record”

““servermix” scriptservermix” script

The next slide shows the “servermix” script The next slide shows the “servermix” script used in this projectused in this project

This script copies the uploaded files from This script copies the uploaded files from each client into a common directory, uses each client into a common directory, uses Ecasound to mix the audio files and returns Ecasound to mix the audio files and returns the resulting mix to the directory of each the resulting mix to the directory of each clientclient

The new audio data can then be The new audio data can then be downloaded by the clientsdownloaded by the clients

““servermix”servermix”

Configuring the FTP Configuring the FTP serverserver

Audio files are uploaded and downloaded from Audio files are uploaded and downloaded from the FTP server by using “put” and “get” the FTP server by using “put” and “get” commandscommands

Seperate directories were set up on the server for Seperate directories were set up on the server for both clients, each containing “incoming” and both clients, each containing “incoming” and “outgoing” folders“outgoing” folders

Files are uploaded by each client to their Files are uploaded by each client to their “incoming” folder, and downloaded from their “incoming” folder, and downloaded from their “outgoing” folder“outgoing” folder

This allows the server to mix audio data from both This allows the server to mix audio data from both clientsclients

Objectives AchievedObjectives Achieved

The required objective of implementing a The required objective of implementing a networked recording system has been achievednetworked recording system has been achieved

Using this system, musicians can collaborate to Using this system, musicians can collaborate to produce a finished audio track containing produce a finished audio track containing contributions from many different userscontributions from many different users

Although a solution to the issue of Although a solution to the issue of synchronisation between different audio files synchronisation between different audio files (due to clock timing differences on different (due to clock timing differences on different PCs) has not been implemented, the finished PCs) has not been implemented, the finished system still allows for server-side mixing of system still allows for server-side mixing of independently captured audio sources.independently captured audio sources.

Objectives AchievedObjectives Achieved

I also learned a lot about using Linux, and I also learned a lot about using Linux, and particularly about audio applications in Linuxparticularly about audio applications in Linux

As I have a keen interest in audio technology, I As I have a keen interest in audio technology, I feel that this project has been of great benefit to feel that this project has been of great benefit to me, and am glad to have been given the me, and am glad to have been given the opportunity to undertake such a project.opportunity to undertake such a project.