INOM EXAMENSARBETE TEKNIK,GRUNDNIVÅ, 15 HP

, STOCKHOLM SVERIGE 2017

Quadcopter formation simulated in a choreographed dance to music

JACOB FRIMAN

ROSMARIE LEIJEL

KTHSKOLAN FÖR TEKNIKVETENSKAP

Abstract

The purpose of this project is to simulate a dance formation for a groupof quadcopters, moving in rhythm to music. Focus has lied on getting thedance to move in synchronization to the music. A description of thequadcopter and its dynamics is made. After simplifications was made,the dance was designed by controlling the velocities of the quadcopters.A song was chosen for the music and a beat detection analysis was madeto calculate the number of beats per minute (BPM) of the song. TheBPM was used when creating the dance formation. Velocity restrictionsand the composition of the music was the governing factors of how thedance was designed. A feedback control was implemented into the danceformation. The simulation was performed in MATLAB and a video wasmade of the resulting dance. The idea is that the simulation can be testedon real quadcopters, additions to the project can be made for future workto implement this.

Sammanfattning

Syftet med detta projekt är simulera en dansformation med en gruppquadcopters, som rör sig i takt till musik. Fokus har legat på att få tillrörelserna i takt med musiken. Först gjordes en efterforskning av hurquadcopternas dynamik fungerar. Efter förenklingar så designades dans-koreografin genom att kontrollera hastigheten hos quadcopterna, för attkomma till önskade positioner genom önskad rörelse. En sång för ändamå-let valdes och en beat detection gjordes för att hitta tiden mellan två beat,och lägga in denna i dansen. Såväl hastighetsbegränsningar som musikenskomposition var avvägande faktorer att förhålla sig till vid skapandet avdansen. En reglerteknisk feedback kontroll implementerades till dansen.Själva simuleringen gjordes i MATLAB och en video gjordes senare somvisade upp resultatet av dansen. Iden med simulationen är att den skakunna testas på riktiga quadcopters, tillägg till projekter för framtida ar-bete kan läggas till för att implementera detta.

3

1 Acknowledgements

We would like to thank our supervisors Xiaoming Hu and Han Zhang for guidingand helping us in this project.

4

Contents

1 Acknowledgements 4

2 Introduction 6

3 Theory 7

3.1 Quadcopter dynamics . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Quadcopter measuring instruments and hardware . . . . . . . . . 103.3 High and lower layer controller . . . . . . . . . . . . . . . . . . . 103.4 Music and beat detection methods . . . . . . . . . . . . . . . . . 11

4 Method 12

4.1 Simplifications and Limitations . . . . . . . . . . . . . . . . . . . 124.2 Design of the dance formation . . . . . . . . . . . . . . . . . . . . 134.3 Feedback control . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.4 Beat detection method . . . . . . . . . . . . . . . . . . . . . . . . 15

5 Simulation 18

6 Discussion and conclusion 22

7 References 23

7.1 Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24*

5

2 Introduction

A quadcoptor is an unmanned helicopter with four rotors; it is therefore anUAV (unmanned aerial vehicle). Quadcopters have a simpler mechanical designthan conventional helicopters and because of recent advances in electronics thishas allowed the production of cheap commercial quadcopters. This has madethem quite popular and there are now many hobbyists who fly quadcoptersfor personal use. Due to their small size and flexibility they also have manypractical purposes such as aerial photography, surveillance and reconnaissanceas well as search and rescue missions. They can be controlled by a pilot onthe ground with a joystick, but they can also autonomously control themselvesas a robot. A quadcopter has four rotors with four motors placed in a crosslike pattern. Two of the rotors (that are placed opposite of each other) bothspin clockwise while the other two spin counter clockwise. This is so that thequadcopter doesn’t spin around its own z-axis while applying equal thrust to allrotors. The quadcopter moves by increasing the angular velocity on some rotorsand decreasing on others. This causes the quadcopter to tilt and since some ofthe thrust now is directed sideways the quadcopter starts to move horizontally.It can also spin around its own z-axis (changing the yaw-angle) like a pirouetteby increasing the rotors that spin in one direction while decreasing the othertwo. To ascend the quadcopter naturally increases the angular velocity on allfour rotors. A pilot user or a robot only gives four inputs to the quadcopter;thrust, forward and sideways motion and yaw angular velocity [1].

The purpose of this thesis is to design a formation for a group of quadcopterswhich they will follow in synchronization with music. We will create a dancefor quadcopters. Dance is to move rhythmically to music using prescribed orimprovised steps and gestures. This is something that humans do best andmaking a single quadcopter dance will not look graceful since it is a rigid body.However, if we control several quadcopters at once we can control them so thatthe formation as a whole is dancing to the music. The movement of severalquadcopters can emulate a dance. This gives us more options in how to designdancing maneuvers. To do this we will design a dance formation so that thequadcopters move in synchronization to the beat of the music. We will choosea song for our dance and analyze it and sample it to obtain the beat time. Wewill design trajectories for each quadcopter adapted to the song and specifythe velocities required to follow the trajectories. The velocities designed will bebounded to not exceed maximum velocities. The maximum velocities was chosenfrom the quadcopter model DJI Phantom 4. A picture of this quadcopter canbe seen in figure 1. To ensure that the quadcopters follow the given trajectorieswe will also implement a feedback control to handle disturbances and errors.This will all be simulated using MATLAB.

Section 2 goes into detail on how quadcopters operate and introduce theirdynamics as well as explaining the basic principle of how music is composed andand introduce concepts about beat detection. Section 3 contains simplificationsand limitations that are used in this project and our motivations and proceedingsfor our method. The simulation is presented in section 4 and finally discussionsand necessary additions for real life implementation can be viewed in section 5.

6

Figure 1: A picture of a quadcopter, the model is DJI Phantom 4 [[1]].

3 Theory

3.1 Quadcopter dynamics

The quadcopter’s dynamics will be explained and it will be shown that one cancontrol the quadcopter’s movements by controlling the four rotor blades. Firsttwo coordinate systems need to be defined. One is the Cartesian inertial systemfixed to the ground; the other is the non-inertial body frame of the quadcopter.The quadcopter can be at different positions in space, have different coordinatesX,Y,Z in the inertial frame, but it can also have different orientations. A quad-copter’s orientation or pose is described by its Euler angles. The quadcoptercan rotate in three different ways. If it rotates around its own z-axis it changesits yaw angle (ψ), roll (φ) is the angle around its x-axis and pitch (θ) aroundits y-axis. The quadcopter moves in horizontally by adjusting either it’s pitchor roll angles. It does this by increasing the angular velocity of one rotor whiledecreasing it on the rotor that is 180 degrees away from it [2][3]. It can yaw byincreasing the rotor speed of all rotors spinning in one direction while decreas-ing the other two. Quadcopters can either operate this way or they can be anX-type quadcopter. X-types pitch and roll by increasing rotor speed on the tworotors closest to each other instead. Basically they fly forward by pitching androlling at the same time while the first quadcopter would fly forward by onlypitching or rolling. The X-type quadcopter will not be used in this mathemati-cal model although the dynamics are very similar [4]. The Euler angles can beseen in the figure 2. Forces and torques around the z-axis of a quadcopter canbe seen in figure 3. The rotation matrix to convert coordinates from the bodyframe to the inertial frame is defined as:

R =

cψcθ cψsθsφ − sψcφ cψsθcφ + sψsφ

sψcθ sψsθsφ + cψcφ sψsθcφ − cψsφ

−sθ cθsφ cθcφ

Where cx = cos(x) and sx = sin(x). For example a vector ~a in the body frame

7

Figure 2: A picture showing the three Euler angles yaw, pitch and roll andhow they are defined in the body frame [[2]].

is R~a in the inertial frame. The total angular velocity of the whole quadcopteris given by:

~w =

1 0 −sθ

0 cφ cθsφ

0 −sφ cθsφ

φ

θ

ψ

The angular velocity of the rotor blades create thrust and also torque inthe z-axis in the body frame. The force and the torque from one rotor can bemodeled as:

fi = kω2i , τMi

= bω2i

,where k and b are constants due to the air drag and ωi is the angular velocity

of the rotor blade. The total thrust produced from the quadcopter is the sumof the thrust from all rotors. Thus the total thrust in the body frame is:

~TB =∑

~fi = k

0

0∑

ω2i

The numbers on the rotors are chosen according to figure 3. Roll is achievedwhen decreasing the angular velocity of the 2nd rotor while increasing it on the4th. Pitch is achieved when decreasing the angular velocity of the first rotorwhile increasing it on the third. Yaw is achieved when increasing the speed onrotors two and four and decreasing it on rotors 1 and three. The yaw torquearound the body z-axis is:

τψ = b(−ω21 + ω2

2 − ω23 + ω2

4)

8

Figure 3: The body frame of the quadcopter is shown together with angularvelocities, torques and forces from the rotors [[3]].

The different signs on the angular velocities are due to the rotors spinningin different directions. The roll torque is:

τφ =∑

r × fi = Lk(−ω22 + ω2

4)

Where L is the distance from the center of the quadcopter to the rotor. Andcorrespondingly the pitch torque is:

τθ = Lk(−ω21 + ω2

3)

Thus the total torque in the body frame is:

~τB =

τφ

τθ

τψ

=

Lk(−ω22 + ω2

4)

Lk(−ω21 + ω2

3)

b(−ω21 + ω2

2 − ω23 + ω2

4)

Modeling the quadcopter as two crossed rods, by the symmetry the inertiamatrix becomes diagonal.

I =

Ixx 0 0

0 Iyy 0

0 0 Izz

The dynamical equations can be derived using an Euler-Lagrange approach orusing Newton-Euler equations. Using the latter, the torque can be written as:

I ~w + ~w × I~w = ~τB

This can be rewritten as:

~w = I−1( ~τB − ~w × I~w) (1)

And finally using Newton’s Second Law in the inertial frame one obtains:

m~r = −mg~z + R ~TB (2)

9

~r =

x

y

z

where m is the mass of the quadcopter, g is the gravitational acceleration, ~z isthe unit vector on the z-axis in the inertial frame and ~r is position vector of thequadcopter in the inertial frame. equation 1 and 2 describes the motions of thequadcopter and if one can control the angular velocity of the rotor blades onecan also control the acceleration and angular acceleration of the quadcopter.Thereby one is able to control the quadcopter fully. Air resistance is not takeninto account but it is not necessary for this example[2][3].

3.2 Quadcopter measuring instruments and hardware

A quadcopter has several instruments available to measure and detects its posi-tion, orientation and other obstacles. A barometer measures air pressure whichis used to measure the aircrafts altitude. The pressure sensors are so sensi-tive that they can detect change in air pressure for a few centimeters. Theaccelerometer measures the acceleration of the aircraft. The accelerometer canalso calculate the orientation since the accelerometer has three axis sensors. Thegyroscope measures the angular velocity in a three axis, the rate of change ofthe Euler angles. The angles and acceleration can then be used to calculatethe velocity by integration of data and time. Ultra sonic sensor, cameras anda Global Positioning System (GPS) are also used as a complement. Ultrasonicsensors attached to the quadcopters, measures the distance to its nearby neigh-bors and other obstacles. GPS is unreliable on small scales and the barometeris unreliable in indoor environments [1]. It is useful to have a preprogrammedfixed coordinate system when designing formations of quadcopters. This can beachieved using a capture motion method. Capture motion is a method wherecameras on three different surfaces together combine data to create signals inX,Y,Z directions. To be more independent from light conditions in the room, ac-tive bright markers mounted at each arm of the quadcopter reduce the exposuretime of the cameras. It is more accurate than other methods, making it a goodchoice for implementation for our project. Since the dance formation designedneeds to follow the trajectory accurately and following it on time is essential.The input given from these instruments are given to the inertial measurementunit (IMU). The data is processed in a micro control unit (MCU). Each rotorblade has its own motor and electronic speed controller, the last one translatingsignals from the MCU to the motor. The MCU sends the output to all fourrotors and tells them what angular velocity they should have [1].

3.3 High and lower layer controller

To create a clear separation of hardware-dependent software from hardware-independent software an abstraction layer is placed between the micro controllerhardware and the application software. The top layer architecture is the onlyvisible interface from the application point of view [5]. We divide these into thehigher and lower layer control. Most quadcopters are designed with differentlayers of control. The hardware dependent lower layer controls what the angularvelocity on the rotor blades should be in order to achieve desired movement and

10

position. It also stabilizes the quadcopter. The input given to the lower layercontroller is the linear velocities for each axis X,Y,Z, the angular velocity of thequadcopters yaw angle and the current orientation of the quadcopter. If theorientation required for the desired velocities and the current orientation differ,they will produce an error. The lower layer produces the required angularvelocities on the rotors so that this error converges to zero. Thus the output isthe angular velocity of the rotor blades and therefore also the torque given fromeach rotor. The higher layer controller on the other hand is given input froma user (with a joystick controller) or computer software. Taking the joystickas an example, a user can specify in which direction the quadcopter should flyand how fast. The joystick can also control the angular velocity of the yaw,but it cannot control the pitch and roll. Pitch and roll are determined by thelinear velocities in X,Y,Z directions, the higher the speed the higher the pitchor roll angles are. So a user with a joystick or computer software gives input tothe higher layer controller and it outputs the desired velocities in all three axisdirections as well as the yaw angular velocity. These are then fed to the lowerlayer controller which will attain the desired input[6][5].

3.4 Music and beat detection methods

Sound are vibrations, that propagates as an audible mechanical wave of localdynamical pressure that varies in time, through a transmission medium such asair. It can be measured by a microphone that converts sound into an electricalsignal. The signals are collected by a defined procedure, and the set of datacollection is called “sample”. The elements of a sample are called “sample units”or “sample points”, the samples are proportional to the energy level of thesound. In stereo mode there are samples from two channels, corresponding tothe left and right loudspeaker. The sound frequencies represent the differentaudible tones. The human ear hears frequencies from 20 Hz up to 20000 Hz.The numeric algorithm Fast Fourier transform (FFT) is used nowadays to getsignals from time domain to frequency domain. FFT needs 2M samples togive M number of tones, each tone for each integer frequency. The Nyquist-Shannon sampling theorem says sampling frequency must be greater than twicethe maximum frequency one wishes to reproduce. The sampling rate for musictherefore has to be greater than 40000 Hz since that is twice the frequency 20000Hz. The most common sampling rate is around 44100 samples per second, usedfor MP3 and other audio file formats. The 44100 sampling rate is used becauseit allows a 2050 Hz transition band, that filters the signals [7].

The beat of a music sound is the fundamental frequency. It is also thefrequency which reduplicates itself the most in a music piece, with short timeexpression and short lay-off. Modern songs often have a beat around 60-150beats per minute (BPM), which gives 1 to 2.5 beats per second. The intervalbetween two beats, that we chose to call “tau” in this report, must thereforebe between 0.4 to 1 seconds long [8]. The sound will be heard as a beat if thecurrent sound is louder (higher energy) than the previous average sound or if thefrequencies of the tones are beneath 200 Hz and repeated regularly. Thereforethere are two main ways to detect a beat correctly.

The first way is to compare sound energy level in a small amount of samples,to its previous average energy level. Where the current local sound energy ishigher than the previous average level, then we might have a beat. The second

11

way is to detect the frequencies in a song using FFT for small intervals, thendividing the frequency spectrum into sub bands. Choose the lowest frequenciesand detect its energy and compare the sound energy in each sub band to therecent average energy corresponding to these sub bands. When the energy levelis superior to the average, then we might have a beat. Since drum and bassin musical instruments play the main role in expressing the beat in most typesof modern music, it is enough to only study frequencies beneath 200 Hz. Acombination of these methods is more precise, but if one is precise enough, it isunnecessary to go further with the other one too.

4 Method

4.1 Simplifications and Limitations

The task to fully control the quadcopter from the bottom layer control is adifficult task due to the complex flight dynamics. Controlling the rotor speedsto follow our desired dance trajectories would be a project of a different scopeand level than this one. To simplify we have decided to only control the higherlayer controller. Thus we only control the velocities of the quadcopter and itsyaw velocity. This has the added benefit of being easy to implement on realquadcopters. If we had real quadcopters our simulation code would replace theinputs given from a joystick since all we input are velocities which is what thelower layer needs. Thus we only need to design what velocities the quadcoptersshould have in order to follow our desired trajectory from the dance formation.All the velocities we will design will be of a form:

~V =

vx

vy

vz

ωψ

Where ωψ is the angular velocity of the yaw angle. To make sure that thesimulation is possible to implement on real quadcopters, physical quadcopterlimitations have been taken into account. Since real quadcopters are beingsimulated an upper limit on their velocity is necessary. Real quadcopters cannotfly infinitely fast. Since there are different types and models of quadcopters withdifferent maximum speeds, we needed to choose a specific quadcopter and useits performance metrics for our simulation. Our choice of quadcopter was theDJI Phantom 4. This model was chosen since it is a quality quadcopter thathas a good visual system. We did not want a budget quadcopter with too poorperformance. This model was sufficient for this goal. The maximum velocitiesof this quadcopter can be seen in table 1 [9].

In addition, due to quadcopters being rotor aircraft, they are subject to aproblematic condition called vortex ring state. The vortex ring state is whenthe rotor-craft descends into its own down wash, its own air it pushes downto create lift. A vortex of air is formed around the rotors and the air just re-circulates around in this vortex and further increasing the rotor blade speed onlyaccelerates this vortex. The result being that the aircraft loses lift and startsdescending into the ground. This condition usually occurs when the rotor-craft

12

maximum velocityvertical velocity 6 m/shorizontal velocity 10 m/syaw rate 200 grad/s

Table 1: The maximum velocities of quadcopter DJI Phantom 4

descends too quickly. If the rotor-craft descends slower or if it descends at anangle, not straight down, the vortex ring state can be avoided. This means thatthe maximum vertical descent speed must be lower than maximum ascent speed[11][12]. Since the maximum vertical speed is the same for descent and ascentand we do not know whether the vortex ring state is taken into account in thesenumbers, we decided to lower the maximum descent speed to 4 m/s.

The limitations explained above restrict how the dance formation can bedesigned and must be taken into account when creating the dance formation.The song that was chosen for the simulation was “Tokyo” by Danny Saucedo.The song was chosen since it has a simple, easily recognizable beat and thecomposition of the song is divided into regular parts. The song belongs to thegenre electronic dance music. This genre usually has songs which all have thesame regular composition. This song is divided into parts that have a numberof beats that are divisible by 8. It has different sections of music that are 16, 32or 48 beats long that are repeated throughout the song. The regularity of thesong and its division into these parts means that we can design a dance for eachsection separately. Except being easier to design the dance, this has the addedbenefit that we can recombine the order of these segments so the designed dancecan be used with different songs.

4.2 Design of the dance formation

Knowing the limitations and the composition of the song the dance formationcan be designed. The dance formation created was divided into sections thatmatch the music sections. Dance sections 16, 32 or 48 beats long were created.When a section of music is played with a specific number of beats in it, acorresponding dance formation will be performed with the same length. When anew music section begins, a new dance formation will be performed. Each dancesection was designed independently of the other sections in order to recombinethe order of the dance formations as we saw fit. To keep the formation asdesired in each section, the velocities were designed so that the quadcopters’starting and ending positions of each section were the same. This assures thatthe dance sections can be put in different order, which gives us the option ofchoosing another song with this composition as well. Since more than threedance sections were created, music sections of a specific length can have morethan one dance section attached to it.

The dance formation was created using mostly periodic functions. Specifi-cally sinusoidal functions were used, often to create circular trajectories. Themotivations behind this are purely aesthetic. Since music and dance is periodicwith a regular structure, it was thought fitting to include sinusoidal functions.Sinusoidal functions also have a smooth derivate which means that changes invelocity will be smooth too. The acceleration was not controlled in this thesis,

13

but in order to make the movements smooth and beautiful the changes in speedshould not be discontinuous or too large. Choosing sinusoidal functions as ve-locities means that the acceleration will be bounded and not infinite, causingthe velocity to be continuous and smooth. However due to simplicity of designand time constraints some movements were designed using constant velocitieswith discontinuous changes of velocity. To alleviate this, a short continuouschange of velocity between two different velocities was introduced.

~V = δ(t) ~V2 + (1 − δ(t)) ~V1 (3)

Initially before equation 1 is implemented, for time t < T the quadcopterhave the velocity ~V1(t) and for time t > T ~V2(t). At time T the change in velocity

between ~V1(T ) and ~V2(T ) is discontinuous. Instead for time T −dt < t < T +dt

we use the velocity from equation 1. The change in velocity from ~V1 to ~V2 will becontinuous since the function δ(t) is continuous and δ(T−dt) = 0, δ(T+dt) = 1.A sinusoidal function 0.5+0.5sin( π

2dt (t− (T −dt)−dt)) was chosen as δ(t) with

these properties. However, replacing the desired velocities ~V1 and ~V2 by thistransition will cause the quadcopters’ trajectories to be slightly off the desiredtrajectory. Since the dance formation was designed using the desired velocities~V1 and ~V2 during the time T − dt < t < T + dt, and not with the velocitytransition. This transition was applied due to removing discontinuities andmaking the dance look smoother. Since the velocities from the transition differfrom the desired ones, this means the quadcopters end up at different positions.This is easily fixed if this transition only is enabled together with the feedbackcontrol as described below.

4.3 Feedback control

To control the quadcopters to follow their intended trajectories a feedback con-trol is implemented. A linear proportional feedback control is used for thispurpose. The control equation is:

~X = −k( ~X −~Xdesired) + ~Vdesired (4)

The desired velocities ~Vdesired are the ideal velocities which should be thevelocities of the quadcopter. When the desired velocities for each quadcopterare designed we can integrate them to find out the desired positions ~Xdesired ofall four coordinates including yaw for each time. The desired positions are thenused to determine how far off the quadcopter is from its intended trajectory.This creates an error that will correct the velocity to return the quadcopter toits path. The error will converge to zero if the initial positions of the quad-copters are displaced. However this controller cannot converge to zero witha constant disturbance, only if the average of the disturbance is zero will thecontroller converge. For example with a Gaussian disturbance with expectationvalue zero the error would converge. If the dance is performed in an indoorenvironment the wind disturbance would be minimal and this controller wouldbe satisfactory. When using the feedback control we include the velocity transi-tion above in the desired velocities in equation 3. The desired positions stay thesame, being derived from the desired velocities excluding the velocity transition.The result is that during the time T −dt < t < T +dt the quadcopter will follow

14

the continuous velocity transition but will still return to the desired trajectoryafterwards. A proof that the feedback controller converges can be seen below.

~X = −k( ~X −~Xdesired) + ~Vdesired => ~X −

~Vdesired = −k( ~X −~Xdesired)

Which is the same differential equation as y = −ky, meaning that:

~X −~Xdesired = ( ~X0 −

~X0,desired)e−kt

This means that as time passes the error becomes smaller and smaller. Itconverges to zero exponentially fast.

4.4 Beat detection method

The shortest amount of time the human eardrum can detect a sound in dependson its frequency and its power. If the frequency is below 200 Hz, which itis usually is for beats, this time is 1/200=5 ms. A sound that has a shorterduration than about 5 ms, will not be detected as a low level as a longer sound,because of different neuronal systems in our ears, might be integrating stimulusenergy at shorter and longer durations. To make sure the hearing system detectsevery frequency correct, about 1 second is preferred at a normal music soundlevel (about 60 dB). [27]

Two short audio samples played just 20-30 ms apart, will be heard as a singleentity. The brain spontaneously makes a single object of them. [14] Therefore,a part of music about 1024 samples (=23 ms) is instant for human ears.

Since not only the beat is playing in a song, it can be difficult to see whatenergy peaks correspond to the beat. Therefore it is a good idea to analyze partswhen drums are the major part that is playing, with little disturbance from othersounds. The basic method performed to extract the beat is to compare the localcurrent sound energy to the average sound energy of the previous one second,which is called the average sound energy history by us. While the sampleswe analyze are not values for energy, they are proportional to energy. Theenergy is converted to a voltage which the computer reads and measures, thatmeasurement is what have been called a sample. In music other high energylevel sounds than the ground beat can occur. And therefore it gives betterresults if some constant C strengthens the average sound energy history. Thisconstant will be used. Because a song has many different acoustic energy levels,at least 5 seconds of the song has to be analyzed to give an exact value of thebeat. However because computer memory cannot handle information withoutadding errors when lacking memory, at most 500 000 samples ( 11 s) can beanalyzed[15][16].

First a part of the song where mostly drums were was chosen to be analyzed.The song was sampled and uncompressed in MATLAB. The samples from thesong are from two channels. One channel is the sound played by the rightloudspeaker and the other from the left one. Five to eight seconds was analyzedto obtain the beat. At first it is impossible to see the beat as seen in figure 4.

Then the local current energy blocks were created. They were created bytaking the square of each sample from both channels and then taking the sumof 1024 samples from both channels together. 1024 was chosen since it humans

15

Figure 4: The sampled part of the song read into the computer, showing thatfurther analysis is needed to extract the beat.

cannot distinguish sounds shorter than that as mentioned above. The j:th localenergy level is therefore defined as:

Ej =

1023+j∑

i=j

S2i,left + S2

i,left (5)

Where j = 1024n, n is a positive integer including zero, S2i,left is a sample

measurement from the left loudspeaker channel and vice versa. Then the averagesound energy history is formed by taking the average of 44032 squared samplesfrom both channels. 44032 is chosen since it is 43*1024 samples which is aboutone second, one second was an adequate number for this. Then to create a soundwhich is 1024 samples long, the average was multiplied by 1024 for reasonsexplained above. Each local energy level is associated with a unique averagesound energy history. The current local sound energy has its average soundenergy history formed by taking the average of all the previous 44032 samplesfrom the current local. The sound energy history is defined:

Ehj =1024

44032

j∑

i=j−44032

S2i,left + S2

i,left (6)

Where j is defined the same way as above. Then the constant C is calculatedby calculating the variance from 43 local energy levels and then C is definedas[15]:

C = −1.5 × 10−6V ar(E) + 1.5142857 (7)

V ar(E) =1

43

43∑

j=0

(Average(E) − Ej)2

Average(E) =1

43

43∑

j=0

Ej

16

Figure 5: Local energy level divided with average sound energy history and Cis plotted against time. One can clearly see the beat in this picture.

After that the following equation is analyzed:

Lj = 10log(Ej

EhjC) (8)

If Lj is greater than zero the local energy level is greater than its history,which is required for a beat and Lj and Lj+1 cannot be equal, it should belarger or decrease. The result from this is bands of consecutive local energylevels (which each are 1024 samples long) for which this is true. In each band,the highest local energy level is taken and in each local energy level the maximumsample value is taken. The time between each maximum sample in each bandshould be the beat. Finally the average of all these values of time is taken toobtain the final value for the beat. A figure illustrating the beat can be seen infigure 5.

17

5 Simulation

The result of this thesis is the simulation of the created dance formation. Thesimulation was performed using MATLAB. The simulation was largely doneusing the MATLAB built in function ode45. Ode45 is a function that solves dif-ferential equations, and in our case it was used to integrate the velocities. Ode45integrates different velocities for different times and different quadcopters. Sincethe feedback controller needs the desired positions to operate one ode45 was runwithout feedback control to obtain this. The data points returned from ode45are discrete and to be able to obtain a value of the desired positions for alltimes, a spline interpolation was done. After that an ode45 could be run withthe feedback loop implemented. The number of quadcopters that was used inthe simulation was chosen to be four. Four is an even number which made iteasier to design symmetries in the dance. A higher number was not chosendue to taking longer time to code and taking more computer power. Using anumber lower than four would not have given the needed complexity to designa good dance. The dance formation was conceived with purpose of dancing tomusic. Therefore the visual simulation of the quadcopters moving needed tobe in synchronization to the music. Doing the simulation of the quadcoptersand playing the music simultaneously in MATLAB proved to be problematic.The MATLAB plotting function was unable to synchronize the music and thedance due to not being able to plot in real time. Instead a video was createdto visualize the dance. The video was a stop motion animation, with the timebetween each frame the same as the time difference between two data points.The video shows that the quadcopters move in synchronization to the beat,which was the goal. Trajectories showing the movement of the quadcopters canbe seen in figure 6-9.

18

Figure 6: Picture is showing a circular maneuver in the dance. While not seenhere the quadcopters change direction while in the circle. After the quadcoptersend up in a vertical line the adjust their positions for their next move. Thevelocities are of the form Awsin(k + w(t − d)) where the constants assure thephase is correct for each quadcopter and to make them able to change directions.

Figure 7: After going around in a circle for a time the quadcopters move in acircular pattern to form a vertical line. As an example, for the circular ascending

part the velocities are ~V = [2wsin(w(t−2τq)),−2wcos(w(t−2τq)), 1.5−(q−1)6τ , 0].

Where q is a number depending on which quadcopter it is and w is π4τ . After

the quadcopters end up in a vertical line they adjust their positions for theirnext move.

The beat detection analysis returned a value for the calculated to be:

τbeat = 0.455

τbeat is the time between each beat in the song. In other words the songwas determined to have about 131 beats per minute. Observation of the sim-ulated dance verified that this was a value that was satisfactory for use in thesimulation.

The feedback control proved to be effective as well. If the starting positionswere displaced and a Gaussian disturbance was applied, the quadcopters wereable to correct themselves to the correct path. When using the feedback con-troller in eq (nr), there is a constraint on k. The velocities cannot violate thevelocity restrictions, the quadcopter cannot fly faster than its maximum veloc-ity. Therefore k cannot be too large so that the velocities exceed the bounds.Since the velocity depends also on the error of position, the best way keep withinthe bounds was to only change k to a lower value if the velocity restriction wasexceeded, and otherwise keep k as the same value. A figure showing the trajec-

19

Figure 8: In this part the quadcopters are moving up and down in a wavelikepattern, they are also yawing. The time between two quadcopters reaching thetop of the wave is exactly the time between one beat.

Figure 9: Here the quadcopters move with linear velocities in a square likeformation.

20

Figure 10: The blue trajectory is showing the desired trajectory and red is theactual trajectory of the simulated quadcopter. Here k = 1.

Figure 11: The rate of convergence for different values of k. The distance fromthe quadcopter to its desired position is on the y axis and time on the x-axis.

tories with initial displacement and disturbance can be seen in figure 10 and therate of convergence with respect to different values of k can be seen in figure 11.

21

6 Discussion and conclusion

Since a simulation of quadcopters was done in this project, a relevant questionis whether or not it can be implemented on real quadcopters. Since we havedesigned the dance from the higher layer controller existing in quadcopters,the simulation is possible to implement. The quadcopters would be given inputfrom the higher layer controller from our simulation which the lower layer woulduse to produce the desired velocities. The velocities of the designed dance arewithin the bounds of max velocity for DJI phantom 4. It is possible to useMatlab compilers to use the program code to implement on real quadcopters.

However there are other things that are not taken into account if this sim-ulation would be implemented. Acceleration was not taken into account, onlythe velocities were designed. The dance was designed to be in sync using thevelocities. It was not taken into account how fast the velocities could change.If the velocity in the designed dances changes faster than the maximum allowedacceleration there will be some lag in the dance as they will not be able to followthe designed trajectories perfectly. The simulation would still be able to be im-plemented but the the desired velocities might not be achievable at all points.Still, the feedback controller would be able to correct this so that the quad-copters return to the desired trajectory on time. The quadcopters would lagbehind when the change in the desired velocities are greater than the maximumacceleration, but they would catch up afterwards when the desired velocity isreached. As long as the lag is not too great or too frequent the dance wouldstill look presentable.

In the simulation the music was played along the dance in MATLAB. Thisworks well since the computer can start the music and dance simultaneouslyand in the simulation environment the music and dance run ideally with anydelays. In reality there would probably be some error between the music anddance. The dance itself could still run as intended but the music might benot be in synchronization to where the dance. The quadcopter would have todetect the music in real time and adjust its velocities accordingly. A feedbackcontrol for controlling that the dance would be aligned with the music in timecould be implemented in addition to the position feedback. The time delayfor the quadcopter to detect and sample the music in real time must be takeninto account when designing this. In Matlab these issues do not appear. Othersmall time delays is the time needed for the angular velocity of the rotor bladeto change, in the dynamics it is assumed that there is no angular acceleration ofthe rotor blades. Also the calculation of velocities that quadcopter does severaltimes per second takes a small time too. These usually have a negligible effect,but they might be an issue in applications were time precision is important suchas this.

When designing the dance absolute coordinates were used to design thevelocities, in the feedback loop absolute coordinates was used for the desiredposition and the current position. Often this is not the case in real life appli-cations as the quadcopters only know their positions relative to each other andthe direction of its velocities are given in its body frame. Before this simulationis applied on real quadcopters, a conversion to relative coordinates is needed.Absolute coordinates can be achieved using GPS but this is rather inaccuratefor the purpose of the designing a dance that must be synchronized.

The purpose of this project was to design and simulate a moving formation

22

of quadcopters that will dance to music, move in rhythm to the beat. Thedance was designed by designing the velocities of the quadcopters while quad-copter velocity restrictions was taken into account. A song was chosen as themusic for the dance and the song was analyzed to detect the beat so the dancewould be able to move in synchronization to the music. A feedback control wasimplemented into the dance to control the quadcopters if they would be dis-placed. The dance formation was then simulated and coded using MATLAB.The presentation of the results was made using a video of the simulated dance.

If additional work was to be done on this project in the future, several ad-ditions could be made. Maximum acceleration could also be taken into accountwhen designing velocities. One step further is to design the lower layer directly,designing the rotor blade velocities instead of the quadcopters’ velocity, thus di-rectly taking quadcopter dynamics into account. An extended feedback controlcould be added, an integral and derivate part could be added to the controller,producing a PID controller. Naturally implementing on real quadcopters is themost logical follow up when doing a simulation. The only certain way of testinga whether a simulation is succesful or not is actually implementing it in reality.

7 References

References

[1] Liang Oskar. June 25 2013. Build A Quadcopter From Scratch.https://oscarliang.com/build-a-quadcopter-beginners-tutorial-1/ (viewedMay 31 2017)

[2] Luukkonen Teppo.2011. Modelling and control of quadcopter. Aalto Uni-versity.

[3] Gibiansky Andrew. November 23 2012. Quadcopter Dynamics and Simu-lation. http://andrew.gibiansky.com/blog/physics/quadcopter-dynamics/.(viewed May 2017)

[4] García Carrillo, L.R., Dzul López, A.E., Lozano, R., Pégard, C. 2013. QuadRotorcraft Control. London, Springer

[5] Wolfhard L. 1997 CAN system engineering from theory to practical appli-cations. New York, Springer.

[6] Bresciani Tommaso. October 2008 Modelling, Identification and Control ofa Quadrotor Helicopter. Lund University

[7] Watkinson, John. 2000 The art of Digital Audio. 2:nd edition, Seattle. Ama-zon books.

[8] Roberts, Gareth E. 2016 From Music to Mathematics, exploring the con-nections. Baltimore. John Hopkins University Press.

[9] DJI. 2017. Phantom 4 SPECS.DJI https://www.dji.com/phantom-4/info(viewed May 31 2017)

23

[10] Corrigan, Fintan. 2016. DJI Phantom 4 and 4 pro IMU calibration Howto instructions.Dronezone. https://www.dronezon.com/diy-drone-repair-videos/dji-innovation-drones/how-to-calibrate-dji-phantom-4-and-4-pro-imu/ (viewed May 2017)

[11] Quadcopter 101. August 1 2014. Quadcopter "Wob-ble of Death": VRS Recovery and Avoidance.Youtube.https://www.youtube.com/watch?v=LCret4rv0HE (viewed May 2017)

[12] Dziubinski Adam, Stalewski Wienczyslaw. 2007 Vortex Ring State Simula-tion Using Actuator Disc. Institute of Aviation, Warsaw.

[13] Picton T.W.2010 Human auditory evoked potentials. San Diego: PluralPublishing. E-book

[14] Deruty, Emanuel.2011 How we hear pitch.SOS Sound on Sound.http://soundonsound.com/sound-advice/how-we-hear-pitch (viewed May2017)

[15] Hawkins, Kevin. 2001. Beat detection algorithm.GameDevhttp://archive.gamedev.net/archive/reference/programming/features/beatdetection/(viewed May 2017)

[16] Ziccardi, Marco. 2015 Beat detection algorithms (part2).http://mziccard.me/2015/05/28/beats-detection-algorithms-1 (viewedMay 2017)

7.1 Figures

References

[1] Tested Hands-On with DJI’s Phantom 4 Quadcopter Drone! Youtube https ://i.ytimg.com/vi/LieIde5ByqU/maxresdefault.jpg

[2] Juan C. Mendez Quadcopter dynamics: Reference frames The UAV Nethttp : //theuav.net/assets/articleimages/2014 − 11 − 19 − quadcopter −

dynamics/angles.png

[3] Luukkonen Teppo.2011. Modelling and control of quadcopter. Aalto Univer-sity.

24

www.kth.se