automatic music composition using a tree of · pdf fileautomatic music composition using a...

AUTOMATIC MUSIC COMPOSITION USING A

TREE OF INTERACTING EMERGENT SYSTEMS

Final Report for the Individual Project (CSS4164) submitted in partial fulfilment of the requirements for the Degree of Bachelor of Computer Science (Special) of the

University of Colombo.

Nuwan I. Senaratna (Index Number: 7096; Registration Number: 2002s8423)

[email protected] May 31, 2006

Supervisor: Dr. D. N. Ranasinghe

Declaration

ii

Declaration “I, Nuwan I. Senaratna, certify that this report does not incorporate, without acknowledgement, any material previously submitted for a degree or diploma in any University or higher educational institution in Sri Lanka or abroad and to the best of my knowledge and belief, it does not contain any material previously published or written by another person except where due reference is made in the text.”

Candidate: Mr. Nuwan I. Senaratna

…………………………….

Signature

……………………………. Date

Supervisor: Dr. D. N. Ranasinghe

…………………………….

Signature

……………………………. Date

Acknowledgements

iii

Acknowledgements I would like to thank…

My parents, for all their support, advice and love My supervisor, Dr. Nalin Ranasinghe for his guidance, opinion and time My music teacher Ms. Indranee Hapugalle for her patience, wisdom and understanding All my lecturers at the University of Colombo Ms. Ruvini Gunasekara, for some very valuable and insightful comments on the results of this project My sister, Devini, for helping me prepare the test data for this project

Abstract

iv

Abstract Automatic music composition (or AMC) is a “hot field” in computer science. It has much commercial potential and is of much interest to scientists and industrialists alike. A wide range of approaches and techniques have been explored in attempts to build AMC systems. Of these, emergent-computing techniques have proved very promising. However, most attempts have tended to use a single, centralized emergent technique for the entire AMC process. This has many drawbacks. The author proposes a solution to these drawbacks in the form of a tree-like structure made up of several interacting emergent systems. The author calls this “A Tree of Interacting Emergent Systems” or TIES. TIES is a context-independent framework, applicable to many applications, including AMC. The author demonstrates the latter by using it to implement the application AMCTIES (An application capable of Automatic Music Composition using a Tree of Interacting Emergent Systems). AMCTIES defines a TIES framework for musical composition. Musical entities are represented as emergent entities. These are generated by emergent systems known as generators. Low-level emergent entities are used to generate high-level emergent entities, and hence form a tree of emergent systems. For example, Motifs and Rhythm are used to generate Phrases. TIES allows for complex interactions between its emergent systems and implements several additional design components in order to facilitate this. The TIES framework is independent of the implementation of the actual generators. The idea is to design a scalable, flexible and robust framework that will maximize practical results. To demonstrate how TIES can be used for AMC, we implement several generator emergent systems. These use Fractals, Cellular Automata and Genetic Algorithms. The particular implementations have been optimized specifically for AMC. AMCTIES is delivered as a set of libraries. These have been developed on the Microsoft .NET framework. The outputs produced by AMCTIES are in the form of MIDI files. The libraries can be used to design a host of applications. A set of music files generated by AMCTIES was subjected to the scrutiny of several experts in the music field. This scrutinization process was designed to validate the generation of each of the component musical entities. Expert responses indicate that AMCTIES can indeed generate music with a reasonable level of creativity and novelty. This project would be of much interest to those in both the computer as well as music fields.

List of Contents

v

List of Contents Declaration ....................................................................................................................................................................... II Acknowledgements .........................................................................................................................................................III Abstract ...........................................................................................................................................................................IV List of Contents ................................................................................................................................................................ V List of Figures ............................................................................................................................................................... VII List of Tables ................................................................................................................................................................VIII Acronyms and Abbreviations ........................................................................................................................................IX CHAPTER 1: INTRODUCTION.................................................................................................................................... 1

1.1 PREAMBLE ................................................................................................................................................................ 1 1.2 BRIEF OVERVIEW OF PROJECT .................................................................................................................................. 2 1.3 MOTIVATION............................................................................................................................................................. 2 1.4 GOALS OF THE PROJECT ............................................................................................................................................ 3 1.5 SCOPE OF THE PROJECT............................................................................................................................................. 3 1.6 OVERVIEW OF REPORT.............................................................................................................................................. 3 1.7 A NOTE TO THE READER ........................................................................................................................................... 4 1.8 VERSION HISTORY .................................................................................................................................................... 5

CHAPTER 2: LITERATURE SURVEY AND BACKGROUND ................................................................................ 6 2.1 CONVENTIONAL MUSIC COMPOSITION ..................................................................................................................... 6 2.2 OVERVIEW OF AMC ............................................................................................................................................... 10 2.3 ATTEMPTS AT AMC................................................................................................................................................ 13 2.4 GENERAL DRAWBACKS WITH ATTEMPTS SO FAR ..................................................................................................... 16 2.5 EMERGENCE AND EMERGENT COMPUTING ............................................................................................................. 18 2.6 TECHNIQUES THAT WILL BE USED IN THIS PROJECT ................................................................................................. 19

CHAPTER 3: REQUIREMENTS SPECIFICATION ................................................................................................ 25 3.1 FUNCTIONAL REQUIREMENTS ................................................................................................................................. 25 3.2 NON-FUNCTIONAL REQUIREMENTS......................................................................................................................... 26

CHAPTER 4: DESIGN .................................................................................................................................................. 28 4.1 OVERALL ARCHITECTURE....................................................................................................................................... 28 4.2 TIES (A TREE OF INTERACTING EMERGENT SYSTEMS) .......................................................................................... 28 4.3 DESIGN PHILOSOPHY .............................................................................................................................................. 30 4.4 TIES DESIGN COMPONENTS ................................................................................................................................... 32 4.5 DESIGNING GENERATORS ....................................................................................................................................... 34 4.6 DESIGNING MAPS.................................................................................................................................................... 39 4.7 BASIC DESIGN MODULES ........................................................................................................................................ 40

CHAPTER 5: IMPLEMENTATION - AMCTIES.FRAMEWORK ......................................................................... 41 5.1 REPRESENTING EMERGENT OBJECTS ...................................................................................................................... 41 5.2 REPRESENTING MAPS AND MAP ELEMENTS............................................................................................................ 43 5.3 REPRESENTING THE USER’S ENVIRONMENT ........................................................................................................... 43

CHAPTER 6: IMPLEMENTATION - AMCTIES.DEFAULTGENERATORS...................................................... 47 6.1 FRACTALS ............................................................................................................................................................... 47 6.2 CELLULAR AUTOMATA ........................................................................................................................................... 49 6.3 GENETIC ALGORITHMS ........................................................................................................................................... 50 6.4 NON-EMERGENT IMPLEMENTATIONS ...................................................................................................................... 51 6.5 TUNING THE OPERATORS ........................................................................................................................................ 54

List of Contents

vi

CHAPTER 7: IMPLEMENTING SUPPORT MODULES ........................................................................................ 56 7.1 AMCTIES.MUSICTHEORYSUPPORT....................................................................................................................... 56 7.2 AMCTIES.MATHSUPPORT..................................................................................................................................... 60 7.3 AMCTIES.MIDISUPPORT...................................................................................................................................... 61

CHAPTER 8: TESTING, EVALUATION AND VALIDATION............................................................................... 63 8.1 THE PLAN ............................................................................................................................................................... 63 8.2 SUMMARY OF CODE TESTING ................................................................................................................................. 67 8.3 SUMMARY OF EXPERT EVALUATION OF OUTPUT .................................................................................................... 67 8.4 SUMMARY OF COMPARING OUTPUT WITH OTHER COMPOSITIONS ........................................................................... 68 8.5 SUMMARY OF VALIDATING FUNCTIONAL REQUIREMENTS ..................................................................................... 69 8.6 SUMMARY OF VALIDATING NON-FUNCTIONAL REQUIREMENTS ............................................................................. 70

CHAPTER 9: GENERAL DISCUSSION..................................................................................................................... 71 9.1 ON THE PROPERTIES OF TIES.................................................................................................................................. 71 9.2 ANALOGIES BETWEEN TIES AND HUMAN MUSIC COMPOSITION .............................................................................. 72 9.3 SOME PHILOSOPHICAL QUESTIONS ......................................................................................................................... 73

CHAPTER 10: CONCLUSIONS AND FUTURE WORK ......................................................................................... 77 10.1 PROBLEMS FACED................................................................................................................................................. 77 10.2 LESSONS LEARNED................................................................................................................................................ 77 10.3 DEVIATIONS FROM ORIGINAL PROJECT PLAN......................................................................................................... 78 10.4 DEFICIENCIES IN THE FINAL PRODUCT................................................................................................................... 79 10.5 EXTENSIONS AND FURTHER WORK ....................................................................................................................... 79 10.6 CRITICAL APPRAISAL OF THE SYSTEM .................................................................................................................. 80 10.7 FINAL CONCLUSION .............................................................................................................................................. 81

APPENDIX A: MUSIC THEORY CONCEPTS AND DEFINITIONS..................................................................... 82 A.1. MUSIC ................................................................................................................................................................... 82 A.2. BRIEF OVERVIEW OF MUSICAL NOTATION ............................................................................................................. 82 A.3. HOW IS MUSIC DIFFERENT FROM OTHER SOUNDS?................................................................................................. 84 A.4. MELODY ............................................................................................................................................................... 85 A.5. HARMONY............................................................................................................................................................. 88

APPENDIX B: REQUIRED DEVELOPMENT AND TESTING ENVIRONMENT .............................................. 89 B.1. HARDWARE REQUIREMENTS.................................................................................................................................. 89 B.2. SOFTWARE REQUIREMENTS ................................................................................................................................... 89

APPENDIX C: SOURCE CODE AND DOCUMENTATION ................................................................................... 90 C.1. SOURCE DIRECTORY STRUCTURE.......................................................................................................................... 90 C.2. PACKAGE AMCTIES.FRAMEWORK ...................................................................................................................... 91 C.3. PACKAGE AMCTIES.DEFAULTGENERATORS....................................................................................................... 91 C.4. PACKAGE AMCTIES.MUSICTHEORYSUPPORT..................................................................................................... 92 C.5. PACKAGE AMCTIES.MATHSUPPORT ................................................................................................................... 92 C.6. PACKAGE AMCTIES.MIDISUPPORT .................................................................................................................... 92

APPENDIX D: DEMO APPLICATIONS .................................................................................................................... 93 D.1. AMCTIES COMPOSER .......................................................................................................................................... 93 D.2. COMMAND LINE RHYTHM GENERATOR ................................................................................................................ 94

REFERENCES ............................................................................................................................................................... 95

List of Figures

vii

List of Figures Figure 1. Factors that make a good composer .................................................................................6 Figure 2. A Good Composition........................................................................................................9 Figure 3. Approaches to AMC.......................................................................................................11 Figure 4. Implementing AMC........................................................................................................13 Figure 5. Approaches × Implementations......................................................................................17 Figure 6. An Emergent System......................................................................................................18 Figure 7. Overall Architecture .......................................................................................................28 Figure 8. TIES................................................................................................................................29 Figure 9. Emergent Entities ...........................................................................................................32 Figure 10. Summary of design components ................................................................................34 Figure 11. Random Fractal Motif Generation..............................................................................35 Figure 12. IFS Fractal Motif Generation .....................................................................................35 Figure 13. Escape Time Fractal Motif Generation ......................................................................36 Figure 14. Random Fractal Rhythm Generation..........................................................................36 Figure 15. IFS Fractal Rhythm Generation..................................................................................36 Figure 16. Patterns in “La Bamba” ..............................................................................................39 Figure 17. Wrapping a generator ................................................................................................40 Figure 18. Package Structure .......................................................................................................40 Figure 19. Formal Accompaniment Generation ..........................................................................52 Figure 20. Chord Sequence in Pachelbel’s Canon.......................................................................52 Figure 21. The Translation Process .............................................................................................82 Figure 22. The Musical Notation .................................................................................................83 Figure 23. Pitch............................................................................................................................83 Figure 24. The staff as a two dimensional graph .........................................................................84 Figure 25. Note Sequences ..........................................................................................................84 Figure 26. Vertical Note Structures .............................................................................................85 Figure 27. Melody Hierarchy.......................................................................................................86 Figure 28. Source Directory Structure .........................................................................................90 Figure 29. Library Assemblies.....................................................................................................90 Figure 30. Comprehensive Comments.........................................................................................90 Figure 31. DOT NET style Documentation.................................................................................90 Figure 32. AMCTIES.Framework Source Code..........................................................................91 Figure 33. AMCTIES.DefaultGenerators Source Code ..............................................................91 Figure 34. AMCTIES.MusicTheorySupport Source Code..........................................................92 Figure 35. AMCTIES.MusicTheorySupport implementations....................................................92 Figure 36. AMCTIES.MathSupport Source Code......................................................................92 Figure 37. AMCTIES_MIDISupport Source Code .....................................................................92 Figure 38. AMCTIES Composer Main Interface ........................................................................93 Figure 39. Composing with AMCTIES Composer......................................................................93

List of Tables

viii

List of Tables Table 1. Project Timeline 2 Table 2. Emergent Characteristics of Cellular Automata 20 Table 3. Emergent Characteristics of Fractals 21 Table 4. Emergent Characteristics of Genetic Algorithms 22 Table 5. Finding the optimal population size 54 Table 6. Finding the optimal tournament size 55 Table 7. Members of Class NoteSequence 57 Table 8. Members of Class Test 58 Table 9. Members of Class Modifier 59 Table 10. Members of class MIDIFileWriter 61 Table 11. AMCTIES.Framework implementations 91

Acronyms and Abbreviations

ix

Acronyms and Abbreviations AI Artificial Intelligence AMC Automatic Music Composition AMCTIES Automatic Music Composition using a Tree of Interacting Emergent Systems CA Cellular Automata CS Visual C# .NET CSS Computer Science Special Degree GA Genetic Algorithm GP Genetic Programming HMM Hidden Markov Model IFS Iterated function systems (fractals) IGA Interactive Genetic Algorithm LSTM Long Short-Term Memory MIDI Musical Instrument Digital Interface NN Neural Network TIES Tree of Interacting Emergent Systems

Introduction - Preamble

1

Chapter 1: Introduction 1.1 Preamble “If I were not a physicist, I would probably be a musician. I often think in music. I live my daydreams in music. I see my life in terms of music.” – Albert Einstein You don’t have to be a Nobel Prize winning physicist to realize that music has become a part of every single facet of our lives. Everyday, consciously or unconsciously, we hear music everywhere we go - irrespective of whether it is on the radio, on TV, on our cellphones, at a concert, or whether it is simply a man whistling or a child singing or just tunes ringing in your mind. Music entertains us – whether it is in the form of our favourite singer or whether it be the background theme of our favourite motion picture. Music heals us; even the medical profession has realized the therapeutic powers of music. Music gives us an identity; most of us will experience a sense of strong patriotism when we hear our own national anthem. In fact, music is probably the most supreme of all human inventions. Yet, most of us, most of the time, take most of the music we hear for granted. Even when we are intentionally listening to music, we fail to appreciate that what we are experiencing is a product of a tradition centuries long. We don’t seem to comprehend the work and effort that has gone into these compositions. There is also the other side of things that we don’t seem to realize. Music is one of the world’s biggest and fastest growing industries. Artists sign deals worth billions with record companies. Royalties, intellectual property infringement and music piracy have become major talking points. Many interesting issues concerning creativity and plagiarism are arising – particularly as musicians seem to be producing music that is very similar to music produced by others. Not surprisingly, musicians and scientists alike have been very keen to understand how exactly humans compose music. Automatic Music Composition (AMC) has, hence, quite naturally, become a field of much interest. Many scientists have pursued the Subject in an effort to come up with a human-free, real-time music composing system. Also, due to its obvious commercial potential, industrialists in the cellular phone and video game industry have been keenly monitoring developments in this area. Many computational techniques have been used for AMC. Conventional methods have been tried, but like many applications that attempt to mimic human behaviour, these have not been very effective. Of late, there have been many attempts at AMC using fresh AI related techniques. Of these “Emergent Computing” techniques have shown much potential. This is particularly because it is possible to draw close analogies between “emergence” and human composing technique. However, attempts so far have been marred by several problems. The author believes that this is chiefly because most Emergent AMC systems tend to use a single technique - a lone, centralized emergent system that attempts generating all the material required.

Introduction - Brief Overview of Project

2

1.2 Brief Overview of Project In this project, the author attempts to remedy these problems by replacing the traditional single technique approach with TIES - A Tree of Interacting Emergent Systems. TIES is a tree-like macro-structure made up of several interacting emergent systems. It is a context-independent framework and is applicable to many applications, including AMC. The main deliverable of this project is AMCTIES - an application capable of “Automatic Music Composition using a Tree of Interacting Emergent Systems”. The following timeline gives a snapshot of the project process.

2005 2006

June

July

Aug

ust

Sep

tem

ber

Oct

ober

Nov

embe

r

Dec

embe

r

Janu

ary

Febr

uary

Mar

ch

Apr

il

May

June

July

Project Initiation X

Literature Survey X

Requirement Specification X

Initial Design X X

Final Design X X X

Coding (Implementation) X X X

Testing X X X

Debugging and Improving X X

Expert Evaluation X

��

The project was initiated in September 2005. Before embarking on the design, existing work related to AMC was thoroughly investigated (See Chapter 2). Based on these investigations, the exact requirements of AMCTIES were recognized and specified (See Chapter 3). It was then possible to design the actual system (See Chapters 4) and implement it (See Chapter 5, 6 and 7). Finally, the resultant deliverables and their output was thoroughly tested (See Chapter 8). Although this project consisted of developing a software product, it is essentially a research project. Hence, it does not follow the development pattern of a conventional software development project.

1.3 Motivation Several motivational factors have inclined the author to attempt this project:

• Currently there are many applications that claim to model the human music compositional process. Most of these are rigidly “rule-based” or “example-based” expert-system like applications. These often result in compositions that are boring and banal. Other applications that claim to be interactive and take the users’ likes into consideration are often no more than elaborate music-score editors. Hence, the main reason that the author chose this project is to come up with a framework that models the human music composing process and also solves problems faced by existing applications.

• Today’s music industry is worth billions of dollars. Hence there is a great demand for

composition modelling applications. This has also influenced the author’s choice.

Introduction - Goals of the Project

3

• The author has worked with some music and computer related applications in the past. He has studied music and music theory to a fairly advanced level, holds several professional qualifications on the subject and is very keen on working with music-computer related applications. The author is also generally very interested in AI and AI related applications and hopes to pursue this area in higher studies.

1.4 Goals of the Project The primary goal of this project (as stated in the project proposal) is to build an “emergent behaviour” utilizing application that is capable of modelling the human music composing process. This goal encapsulates several sub-goals:

• To build an application capable of musical composition • To successfully apply “emergent behaviour” to a real-world problem • To utilize “emergent behaviour” to model a human intelligence process • To exploit “emergent” characteristics of several AI techniques in an optimal way

1.5 Scope of the Project

1.5.1 Use of Emergent Behaviour This project only attempts at applying emergent computation techniques in a specific manner to solve a specific problem. This project is not a general investigation of Emergent Techniques.

1.5.2 Use of Techniques Although many techniques such as Cellular Automata, Fractals and Genetic Algorithms are used in this project, it does not intend to individually develop these techniques in a general way.

1.5.3 Nature of the Musical Compositions In this project, we will not consider all musical traditions in general, but restrict ourselves to western music (that is, music that has emerged from the European Music Tradition). Academically, economically and by popularity this is the most important and influential musical tradition at present. Also, the theoretical framework of western music is easier to translate into other forms (but not always vice-versa).

1.6 Overview of Report Chapter 2 (Literature Survey and Background) reviews the background and existing literature related to the project. First, we will take a look at conventional human music composition. Then, we will review the “state of the art” of AMC and AMC techniques. Finally, we will look at emergent systems, with particular emphasis on methods that we hope to use here. Chapter 3 (Requirements Specification) briefly presents the requirements of this project. Since this project is a research project, it does not contain a formal requirements specification that a typical software engineering project would contain. However, to avoid ambiguities, we present a set of functional and non-functional requirements. Chapter 4 (Design) describes the design. It looks at architectural aspects of the initial design and describes the overall architecture as a whole, the design philosophy, the architecture of TIES and

Introduction - A Note to the Reader

4

the basic design components. It also describes in detail how these design components are specifically designed. Finally in overviews the basic design modules in AMCTIES. Chapters 5 (Implementation – AMCTIES.Framework), Chapter 6 (Implementation – AMCTIES.DefaultGenerators) and Chapter 7 (Implementation – Support Modules), describe the actual implementation. Chapter 5 describes the implementation of the TIES framework. This includes how various musical elements are represented and design components implemented. Chapter 6 describes the implementation of the actual emergent systems. This includes how each of the emergent techniques is implemented programmatically and how these and supporting techniques are tuned. Chapter 7 describes the implementation of some support modules that AMCTIES requires. Chapter 8 (Testing, Evaluation and Validation) describes the testing and validation process. After an overview of the testing methods, it describes the validation of AMCTIES by human experts and the results of these tests and gives the results of comparing AMCTIES with other AMC systems. It also verifies that all the requirements specified in Chapter 3 are met. Chapter 9 (General Discussion) discusses certain topics related to the project. These include the properties of TIES, some interesting philosophical questions, and analogies between TIES and the human music composing process. The concluding Chapter 10 (Conclusions and Future Work) begins with a summary of the results. It describes the problems faced during the project, the lessons learnt, deviations from the original project plan, deficiencies in the final product, extensions to the project and a critical appraisal of the system. This is followed by several appendices. Appendix A gives an overview of some musical concepts and definitions related to the project. Appendix B gives a list of other system requirements with respect to Hardware and the Database. Appendix C gives an overview of the source code and documentation. Appendix D describes two demo applications developed using AMCTIES. References cited are listed at the end.

1.7 A Note to the Reader This project closely studies several music theory concepts. Appendix A defines and explains these. Hence, it might be useful to read through Appendix A before reading the main body of this report. Note, however, that Appendix A surveys only areas in music relevant to the project. Particularly, a comprehensive description of music and music composition is far beyond the scope of this report. The reader may consult some of the references (cited at the end of this report) for more comprehensive explanations. The references specifically pertaining to music theory are [WESTRUP, 1989], [JACOBS, 1997] and [SCHOLES, 1998]. Code Segments described are a combination of C# .NET (in regular font) and pseudo code (in italics), not pure C#. The author believes that this is the most practical. As mentioned under the “Scope of the Project”, we restrict our attention to western music. Hence, all notations, terms and theoretical concepts are in that context. For example, when we say “musical

Introduction - Version History

5

notation” or “musical tradition”, we actually mean “western musical notation” or “western musical tradition” respectively.

1.8 Version History

1.8.1 Interim Report An Interim Report containing the first four chapters was submitted earlier (January 9th 2006) to the supervisor, Dr. D. N. Ranasinghe. He suggested several revisions.

1.8.2 Final Report This final document is the complete Final Report for the Individual Project (CSS4164).

Literature Survey and Background - Conventional Music Composition

6

Chapter 2: Literature Survey and Background 2.1 Conventional Music Composition

2.1.1 Factors that make a good composer “Genius is 10% inspiration and 90% perspiration” – Thomas Alva Edison Like many things in life, “inspiration” and “perspiration” have been recognized as the two factors that make a good composer. On the one hand, composers need to be hardworking and perseverant. Musical education is tough and competitive; a career in music even more so. On the other hand, composers often have a talent or “gift” for composing. It is difficult to judge the relative worth of these factors. Many composers have claimed to possess a god-given talent for composing. Others claim that their achievements are solely due to hard work. It is interesting to compare these two factors [WESTRUP, 1989]. Each factor can be thought of as leading to certain characteristics in compositions. These will be referred to as “inspiration-characteristics” and “perspiration-characteristics”. Inspiration-characteristics include:

• The ability to produce novel and imaginative melodies and Rhythms • The ability to come up with new ways of using musical material

Perspiration-characteristics include:

• The ability to work fast • The ability to accurately adhere to musical rules • Inspiration-characteristics are often instant and unexplained, while perspiration-

characteristics are often as a result of years of study and training For a piece of music (and hence its composer) to be considered good, it is often necessary that the perspiration-characteristics are met. However, very often, it is the presence of inspiration-characteristics that class a piece of music (and hence its composer) as great. It is fairly easy to explain the process as to how perspiration-characteristics result, and hence it is easy to write out algorithms for them. Conversely, it is very difficult to explain how inspiration-characteristics come about [WESTRUP, 1989].

� ��

2.1.2 The Composition process It is difficult to pin-point a clear-cut process for composing. Different composers have had different approaches. The academic approach to teaching composition principally concentrates on the


7

“perspiration-characteristics” given above. This is due to the fact that it is difficult or impossible to “teach” inspiration. However, it might be argued that teaching a student the basic fundamentals of music theory will increase the chances of being “inspired”. The interplay of “perspiration” and “inspiration” might be better understood by analyzing the creative processes of some well-known composers. Along these lines we can class composers into four classes: Spontaneous composers, Constructive composers, Traditionalist composers and Revolutionary composers. Note that these classes are purely for abstractly understanding the creative process. Actual human composers may possess qualities of several of the classes. Conversely, no composers are a completely spontaneous composers or a completely constructive composer.

Spontaneous composers Spontaneous composers are those who seem to be creatively inspired spontaneously. They seem to compose in their minds, often finishing complete works of music without having to commit a single note to paper. Some even claim that the creative process is so fast that they find it difficult to commit the results to paper in time. This is the most difficult type of creativity to analyze. The entire creative process is hidden inside the composers mind. Though it is reasonable to suggest that the composer’s education and experience would have influenced their output, it is difficult to say what and how. From a computing point of view, such a creative process is very similar to the output of emergent systems. We can note several similarities and analogies (For a description of Emergent Systems and related terms, please refer the next section): The creative process discussed is not instantaneous, but takes some period of time (however small). During this time the composer “experiences” the coming of an inspiration. Often this is accompanied by emotional and other physiological changes. The composer knows that he is “being inspired”. Hence, human creativity seems “time evolving”. The human brain is made up of millions of simple neurons and interconnections between them. Hence, even the most complex an idea, discovery or insight must have originated as interactions between these. Hence we see that the human brain is a system consisting of several levels. At the lowest level are neurons; simple and trivial on their own. At the highest level, we have complex ideas and compositions; these correspond to “systemic properties” in emergent systems that we will discuss later. Obviously, it is difficult to “deduce” the wonderful creations of the human brain from the nature of the mundane grey matter it is made of; nor is it possible to predict what it might do next. These properties correspond to “irreducibility” and “unpredictability” in emergent systems. Efforts at algorithmically performing tasks that can mimic human inspiration have proved highly complex in terms of both time and memory. This is analogous to “high logical depth” of emergent systems. In addition to external experiences, a composer’s own work has been known to influence his future works, even in cases where the composers does consciously try to relate earlier and later works. This shows that human creativity might involve some “backward causality” – another emergent property [COPLAND, 1937].


8

Constructive composers Constructive composers are also creatively “inspired”; however, such composers use the results of inspiration in a more conservative manner. They are not inspired with entire works of music. Inspiration to them usually takes the forms of short, fundamental musical ideas, like a short theme for a melody or a novel Rhythm pattern. When such inspiration occurs, these composers usually note down the idea (Note that, unlike in the case of spontaneous composers, what is initially committed to paper is not a completed work of music, but small elements that might later be used to build a complete musical work). The composer reflects on these ideas, often modifies them and finally combines them in a constructive manner. This is analogous to architecture. The composer might follow standard accepted musical forms or come up with ones of his own. Unlike in the spontaneous case, the composer is able to explain why he constructed the fundamental elements in the way he did because each of these actions is done consciously, not subconsciously. This process of construction often undergoes many revisions. Usually this involves a lot of painstaking effort. Constructive composers, hence, rely a lot on “perspiration”. Some of the greatest works of classical music have been written by composers of this kind. Although spontaneous composers have produced some extremely beautiful and novel music, their works have tended to lack architecture and meaning. Constructive composers seem to “control” inspiration; hence their works reflect the best characteristics of inspiration and conventional musical training. From a computing point of view, the constructive composer is analogous to a system that first uses “emergent” techniques to develop small scale ideas and then construct larger scale material from these, taking into account conservative constraints. Constructive composers seem to find a good balance between Inspiration-characteristics and Perspiration-characteristics; hence, in developing an automatic music composition system, it might be wise to follow a similar strategy [COPLAND, 1937].

Traditionalist composers Traditionalist composers can be described as a “historical phenomenon”. Styles and techniques of composition evolve through history. New styles emerge, they develop, they mature, they reach their fullest development and are finally replaced by newer styles. Traditionalist composers take a musical style that is established and then develop it as no one has done before. Hence it is this type of composer who takes a musical style to its state of fullest development. Hence, the traditionalist composer’s creativity is restricted to the confines of the musical style. In a way, this can be considered to be the opposite of the constructive composer. The constructive composer fashions his composition out of simple ideas. On the other hand, the traditionalist composer tries to find simple ideas to make up his musical style. From a computing point of view as well, this can be considered the opposite of the constructive approach. In fact, if we consider the constructive strategy as a bottom-up process (where smaller ideas are at the bottom), the traditionalist approach is top-down. We observed earlier that the constructive composer’s strategy might be best for implementing a computer based music composition system. However, the traditionalist composer’s strategy might be equally good.


9

It is interesting to note that in many computer applications (from compiler code parsing to software engineering design) top-down and bottom-up strategies compete for superiority. In many cases a hybrid of top-down and bottom-up is used. Hence, the best options might be to use a combination of traditionalist and constructive strategies [COPLAND, 1937].

Revolutionary composers Revolutionary composers consciously seek to find new (and sometimes radical) modes of music composition. They usually look in directions that have either not been previously travelled, or else have been considered taboo. Revolutionary composers find it difficult to please public audiences with their music because it is often difficult to understand and “hard on the ear”. Hence, proportionally few revolutionary composers succeed. However, often the few who do end up becoming great composers and tend to strongly influence those coming after them. Revolutionary composers can be compared to certain techniques used in computers. For example, mutation in genetic algorithms might be considered a “Revolutionary” operator. Revolutionary composers differ from spontaneous composers in that the process that produces novel material is conscious (not subconscious as in the spontaneous case) [COPLAND, 1937].

2.1.3 Impediments to creativity As with any human process, creative thinking often meets with impediments. Interestingly, impediments very similar to these are also encountered in emergent system. A few examples are noted [BRAUN, 1975]. Fixation: Fixation results from a human doing some wrong thing (instead of doing the right thing) simply because it is the only thing he knows to do. Learned helplessness: Learned helplessness is a human’s reluctance to attempt some task simply because he has failed at it earlier. Lack of motivation: Motivation is vital for any creative process. The same applies for music compositions.

2.1.4 Evaluating musical compositions

� ��

Humans play two roles with respect to music composition: They compose music and they evaluate music. The evaluation role poses the problem on what exactly a “good composition” is. This can be viewed from two angles:

Literature Survey and Background - Overview of AMC

10

Firstly, the composition must have some goal in mind [WESTRUP, 1989]. • The principal goal of musical composition is expression. The composer wishes to “say”

something and he/she does so using her/his music as the medium. The statement might be a social or political comment, or a description of something, someone or some feeling.

• The next goal should be to structure this statement of expression in a way that is aesthetically beautiful.

• Finally, this aesthetically beautiful statement of expression must be conveyed in such a way that it holds the listener’s attention and entertains her/him.

Secondly, it must have certain characteristics that make it sound “good” [WESTRUP, 1989].

• “Good music” has melodic and Rhythmic subtlety, novelty and elegance. It is also interesting and constantly stimulates the listener with variety.

• “Good music” seldom has “too much” variety. There is always balance, symmetry and controlled repetition. The listener’s memory is not burdened by too many melodies or over-complex Rhythms.

• Finally, the best compositions are those that people remember, recognize easily and relate to.

It is difficult to quantify these views. Rules do exists, but are, at best, guidelines. In this project, results will be validated in two stages. First, they will be individually validated by a human expert. Second, they will be subjected to a “Placebo Turing Test”. This is an adaptation of the standard Turing Test, and will be discussed later.

2.2 Overview of AMC

2.2.1 Pre-Computer History of AMC Automatic techniques for melody composition date back to the 11th Century. Guido d’Arezzo composed by assigning pitches to each word in a religious text. Guillaume Dufay and others were known to structure their compositions according to mathematical relations such as the “Golden Section”. The strict rules of counterpoint that developed in the Renaissance and Baroque era indirectly “automated” music composition. For instance, composition could (almost completely) consist of successions of algorithmic modification operators such as ornaments, inversions, augmentations and diminutions of a given theme [JÄRVELÄINEN, 2000]. Automatic techniques for generating music were adopted with renewed enthusiasm again in the 20th Century. Arnold Schönberg introduced the tone-row technique at the beginning of the century. This was further developed into serialism by Anton Webern and others. Iannis Xenakis experimented with stochastic processes and used them for generating music [JÄRVELÄINEN, 2000]. Right from early days, AMC had a very close relationship with mathematics and algorithms. When electronic digital computers emerged in the mid 20th Century, it was only natural that they be exploited for AMC.

2.2.2 Applications of AMC Part of the reason for the interest in AMC is their high economic potential. AMC has several interesting commercial applications [YEE-KING, 2001]. A few are:


11

Computer Games Music is a very important part of computer games. Many modern games attempt to provide the player with multi-strategy-multi-possibility scenarios. It is even possible for these to be non-deterministically assembled from a set of fundamental scenarios. However, the music that accompanies these is often pre-composed. Each time the player plays a different scenario it is the same music he/she hears – this introduces boredom and reduces the “effect” of the game. Hence, a very interesting application of AMC is composing novel, real-time music for such situations. An extension of this is to provide “context sensitive” music for computer games.

Rapid Music Production Many television producers are often faced with the problem of finding suitable music to accompany documentaries, teledramas, feature films, etc.. Often, this is a long process that takes much time and money. Either the producer has to hire an expensive composer, or else he must use existing recorded music. In addition to being boring, this often results in editing problems. Ideally, the music must be written to suit the screen-script, not the other way around. Strict deadlines make things worse. Rapid Music Production applications allow people who are not necessarily experts on music, to produce reasonable musical pieces, in a required style, quickly. This effectively solves the problem described above and is probably the most profitable future application of AMC.

Modern Art Many professional composers have been looking at AMC as a strictly aesthetic tool, producing “art for art’s sake”. Many have used AMC with visual art techniques. “Real-Time” art exhibitions, concerts and theme parks are still not common, but are bound to be in the future.

2.2.3 Approaches to AMC

� ��

Statistical Approaches A statistical approach to music composition consists of analyzing previous musical works (and also musical rules derived from them) and using the knowledge gained to compose music. Most statistical models used are context models like Markov chains and finite state models. The main advantage of statistical approaches to music composition is that they can (often) be implemented with low complexity. Hence, they are ideal for real time systems and many other commercial applications. They are also non-deterministic and, hence, can generate music with some “originality”. The main disadvantage in this approach is that someone has to analyze a large number of musical works to gain the necessary starting data. Also, it is difficult to statistically quantify many abstract characteristics of music. Also, statistical insight gained from analyzing pieces often has many ambiguities. For example, most “rules” inferred from statistical analysis tend to have too many exceptions [PAPADOPOULOS, 1999].


12

Rule-based Approaches The rule-based approach to automatic composition is, in a way, a specialization of the statistical approach. It is also the method closest to that of academic music-composition-training. Musical characteristics that have been observed over many years (possibly centuries) eventually become accepted as “rules”. Although the study of music over years does not exactly follow strict mathematical or statistical techniques, they are (in a way) a summarization of statistical inferences. However, unlike statistical methods, rule-based approaches attempt at defining “well-defined” domains for music composition. This is the main advantage of this technique. There is a flow of explicit reasoning; each choice of action is understood and justified. Rules can be either inferred from analysing statistical data on music, or else argued directly from existing music theory laws. Rule-based techniques, however, have several disadvantages. Defining subjective rules is difficult and very time consuming. As mentioned before, most musical rules have many exceptions. Incorporating these is very complex and cumbersome. Also, the merit of the system depends, highly, on the ability and the musical knowledge of the programmer, who, in a sense, must be an expert on music as well [PAPADOPOULOS, 1999].

Linguistic Approaches Music can be considered a language and linguistic techniques can be applied to compose music. This is not an unreasonable approach since music has many language-like characteristics. As seen earlier, music has grammar-like components. However, this approach has many disadvantages. Not all types of music conform to the hierarchical structure found in grammars (though some do). Unlike language, it is difficult to define rigid semantic rules for music; breaking rules often, even results in better music. Conversely, obeying grammatical rules correctly, often results in boring or incorrect sounding music. Finally, from a computational point of view, implementing a “music parser” is very expensive [PAPADOPOULOS, 1999].

Theory-independent Approaches The three methods considered so far, progressively specialize towards a “high-level” approach to music composition. Conversely, a totally different approach could be taken. Rules and other patterns could be disregarded and music could be generated using methods that do not consider conventional music theory. Theory-independent techniques have several advantages. They are the only way of generating truly “original” music. They are often simple and efficient to implement. Most importantly, while theory-dependent techniques tend to lead to “perspiration-characteristics”, theory-independent techniques lead to “inspiration-characteristics”. The main disadvantage in theory-independent methods is that at some point the output generated needs to be validated. This has to be done by a human moderator, or else using an automated validation system, often based on rules. This often results in an efficiency bottleneck. Also, for obvious reasons, it is not practical to use theory-independent methods on their own. Aspects of theory-dependent approaches have to be incorporated into applications using theory-independent approaches to avoid random or meaningless music.

Literature Survey and Background - Attempts at AMC

13

2.2.4 Implementing the Approaches

� �� !� � ��

Conventional Implementations Each of the above approaches for AMC might be implemented using conventional direct procedural algorithmic methods.

Learning Systems Learning systems do not have an initial knowledge (e.g. rules, patterns, constraints) of the domain, but instead learn by examples. Learning systems may be subdivided into two subclasses according to the way they store information: Systems that store information in a sub-symbolic or distributed way (e.g. neural networks) and systems that store information in a symbolic way (e.g. Machine Learning). Most learning systems used for AMC are neural networks or neural network derivations [PAPADOPOULOS, 1999].

Evolving Systems Evolving systems are computer programs inspired by the natural process of evolution. The idea is to represent a set of possible solutions to a problem using some artificial "genetic code" and then let the system evolve better and better solutions. Examples of evolving systems include genetic algorithms (GA) and genetic programming (GP) [PAPADOPOULOS, 1999].

Other Emergent Systems Most learning systems and evolving systems have emergent characteristics. In addition to these, other emergent techniques such as fractals and cellular automata (CA) can be used for AMC.

Hybrid Systems Hybrid systems are ones which use a combination of techniques.

2.3 Attempts at AMC

2.3.1 Statistical Approaches using Conventional Implementations

Random Walks The simplest statistical approach to AMC is random walks. It consists of splitting music into discrete events, analysing the distribution of these events and then generating sequences of events at random using the knowledge of the distribution. Random walks are fast. Since they are non-deterministic, they have been known to provide “original” music. However, since events are generated at random, each event is independent of its neighbouring events. Hence, although this system might be good for composing short musical fragments, it is not suitable for long musical pieces [CONKLIN, 2003].

Hidden Markov Models (HMM) HMMs are an extension on random walks. Like random walks, HMMs often produce “original” music. Unlike, random walks, there is some correlation between neighbouring events. However, as


14

the number of transitions considered increases, the amount of memory and processing required also increases sharply. This is a serious problem in using this technique to model larger musical forms. Allan users this method for choral melody harmonization. Farbood and Schoner describe a technique for using the method to generate a complementing musical line to a given one in a particular style [CONKLIN, 2003].

Pattern-based Modelling As mentioned, a problem faced by both random walks and HMMs is how to model interactions between larger musical forms. Another statistical approach to solve this problem is to apply statistical analysis techniques to high-level event patterns instead of the musical events themselves. This technique consists of first applying a conventional pattern detection algorithm to the event data and then analysing the patterns [CONKLIN, 2003].

2.3.2 Statistical Approaches using Learning Systems

Neural Networks Most neural network methods implemented so far attempt at mimicking Markov transition tables. However, unlike conventional implementations that effectively “hard code” statistical information, neural implementations have to “learn” the information. The learning process has several consequences. Neural networks tend to be more efficient at storing the “learnt” information and tend to reproduce statistical pattern more efficiently than conventional approaches. Also, neural network generated music sounds more original. However, the learning process takes time (sometimes even several days) and is not suitable for direct real-time implementations. In addition, neural networks cannot store long statistical patterns since it has limited “short-term memory”. They cannot keep track of temporally distant events that indicate global music structure; hence pieces composed lack coherence between global musical ideas. Mozer implemented a system known as CONCERT using a recurrent auto-predictive connectionist network [MOZER, 2003].

Long Short-Term Memory As mentioned, music composed by standard recurrent neural networks lacks global coherence. Techniques such as LSTM or Long Short-Term Memory have been used to counter this problem. LSTMs seem to be able to learn global patterns in music, and also seem to be able to stick to “good” patterns once they have learnt them.

2.3.3 Rule-based Approaches using Conventional Implementations

Harmonization with Constraints Harmony is probably the area in music theory where the richest set of “rules” has developed. Hence, it is not surprising that most rule-based techniques at AMC attempt harmonization. Attempts so far broadly fall into four categories: The first consists of adding the remaining three lower parts to a given melody part of traditional four part choral harmony. The second is very similar, except that instead of the melody part, the lowest (bass) part is given and the upper three parts (including the melody part) have to be added. The third category is a specialization of the second; here in addition to the bass line itself, additional constraints as to the nature of the harmonies are given. The fourth category (perhaps the simplest) consists of adding a bass line to a given melody line [PACHET, 2001].


15

Unlike many other aspects of AMC, harmonization is one area in which there actually exist computer applications capable of high quality output.

2.3.4 Rule-based Approaches using Evolving Systems

Harmonization with Genetic Algorithms (GAs) To overcome some of the problems of conventional implementations, there have been many attempts at less rigid alternatives. GAs have been some of the more successful. Most consist of incorporating the rules in the fitness function. As before, some of the more successful attempts have been in harmonization [SANTOS, 2000].

2.3.5 Linguistic Approaches using Conventional Implementations

Generative Grammars A generative grammar is an algorithm for specifying (or generating) all and only the grammatical sentences in a language. There have been many efforts to define such algorithms for music. Generative grammars have been known to be useful in musical style analysis. These techniques might, conversely, be used to generate music [BUZZANCA, 2001]. Conventional implementations usually consist of building pieces of music top-down. Starting with a broad framework for high-level forms, the underlying low-level structures are step-wise built. Although, this approach seems over-conservative, proper utilization of material between grammatical levels can result in works of music that show surprising creativity and novelty. However, direct application of conventional algorithms does not often result in satisfactory pieces. Hence, more recently, researchers have been looking at non-conventional approaches. Baroni, Dalmonte and Jacoboni define a grammar for melodic Phrases, in which they grammatically define a melody as <melody> � <anacrusis> <central section> <ending>. Material is defined in terms of a “kernel” melody and musical “figures” [BUZZANCA, 2001].

2.3.6 Linguistic Approaches using Evolving Systems

Grammatical Evolution Evolutionary methods to grammatical AMC define grammars very similar to those used in conventional methods. However, unlike most conventional methods (where the exact generation process is based on the grammar), evolving systems usually use the grammar only partially in the generation process. The full use of the grammar comes in the fitness evaluation process. The actual level of use of the grammar in the generation process is an important design decision [PUENTE, 2002].

2.3.7 Theory-independent Approaches using Evolving Systems

Interactive Genetic Algorithms (IGAs) An Interactive Genetic Algorithm is a genetic algorithm where a human moderator serves as the fitness function evaluator. In AMC applications based on IGAs, a human listener evaluates the musical material evolved. The human listener’s opinion (e.g. Good, Mediocre or Bad) is transformed into some fitness value. The presence of a human is obviously a bottleneck. Also, other human constraints such as fatigue and prejudices on the part of the listener must be taken into consideration. Few systems used for AMC are “completely” dependent on the human listener’s opinion. Most take prior musical knowledge, usually in the form of rules, into consideration [PAPADOPOULOS, 2000].

Literature Survey and Background - General drawbacks with attempts so far

16

There have been similar attempts with Genetic Programming (GP) [JOHANSON, 1998] and Genetic Evolutionary Algorithms [MARQUES, 1998].

Neural Network Moderated GAs To overcome the listener bottleneck there have been many attempts to automate this role. Some of the more successful attempts use a neural network based fitness evaluator [SANTOS, 2000].

2.3.8 Theory-independent Approaches using Other Emergent Systems

Cellular Automata-based Melodic Composition There have been attempts at AMC by applying cellular automata to represent melodies. Typically, the states of the cells represent notes (pitch or duration or both). The emerging patterns in the cellular automata grid form the final output. These have been fairly good at providing short melodic Motifs [DORIN, 2002]. The CAMUS 3D applies cellular automata [BURRASTON, 2004].

Cellular Automata Harmonization Musical harmony defines relationships between simultaneously played notes (See Appendix A for details). Hence, the note values of notes are dependent on those of its neighbours. As a melody unfolds (and as its melody note value changes) so must the underlying harmonies. This is, in many ways, similar to the evolution of cellular automata.

Fractal-based Melodic Composition Music has often been compared to architecture. In fact, very often the evolution of music tradition through the years has often been related to contemporary architecture. Hence, many architectural concepts such as structure, form and shape have been incorporated into music. The “shape” of music has been compared to geometric shapes; hence, it is not surprising that fractals have been used to generate melodies.

2.4 General drawbacks with attempts so far

2.4.1 Lack of domain knowledge The chief problem with attempts at AMC so far, has been the lack of incorporation of domain specific knowledge on the part of the application designers. In this case the “domain” is music and its composition process. Many of the AMC techniques that have been recognized as “good” are often very domain specific. Using domain knowledge in implementing systems is vital.

2.4.2 Technique specific applications Many attempts at AMC seem to have concentrated on the technique used rather than the problem at hand. For example, many neural network based efforts seem to concentrate on improving existing generic neural network designs rather than devising better AMC schemes. Although such attempts at technique enhancement have resulted in interesting and useful findings, they are not helpful in the context of this project.

2.4.3 Separation of Music from Computing As a result of the previously mentioned “technique specific-ness”, the nature of the musical output is tightly coupled to the computer generation technique that produces it. Ideally, this should not be the case. Suppose a user makes a request to the AMC systems to produce some musical element (for example a Rhythm). The AMC system should find an appropriate generator (out of possibly several generators) that could service the request, and then ask it to so service it.

Literature Survey and Background - General drawbacks with attempts so far

17

This would also make it possible for a more “distributed” implementation of AMC; rather like a group of composers cooperating to produce a piece of music. All the composers need not be available at a given instance. Nor does the user need to known about their existence.

Statistical Rule-Based Linguistic Theory-Independent

Conventional

Random Walks

Hidden Markov Models

Pattern-based

Modelling

Harmonization with Constraints

Generative Grammars -

Learning Neural Networks

Long Short-Term

Memory

- - -

Evolving - Harmonization with Genetic Algorithms

Grammatical Evolution

Interactive Genetic Algorithms

Neural Network moderated

Genetic Algorithms

Other Emergent - - -

Cellular Automata based Harmonization

Cellular Automata based

Melodic Composition

Fractal Based Melodic Composition

� �� " � � � � � � � ��# �!� � ��

2.4.4 Imitating and not creating Many AMC applications seem to be only good at boring, imitations of existing compositions. Many simply provide variations on existing works, rather than generating new and creative works. This problem can be attributed to the over reliance on rule-based strategies and too conventional implementation techniques.

2.4.5 Boring and predictable results Another bad result of over reliance on rule-based strategies and too conventional implementation techniques is that the resultant output is highly predictable. Unpredictability is an essential part of creativity.

2.4.6 Lack of control over the nature of the output At the other extreme of things, techniques using “pure” emergent techniques (for example pure fractal based melodic composition or purely cellular automata based music composition), the resultant musical output tends to be ad hoc and meaningless. The user (or initiator) of the process has little control results. In practical situations this is a serious problem; what is the use of an AMC system if it cannot produce what the user wants?

Literature Survey and Background - Emergence and Emergent Computing

18

2.4.7 Lack of Flexibility As mentioned, most attempts at AMC use a single method. Though some applications have fairly elaborate musical options, few allow the flexibility to change the generation technique. It is reasonable to expect new and better techniques of generation coming into being as time goes on. A good AMC system must be flexible enough to incorporate these.

2.4.8 Lack of Scalability An extension of the lack of flexibility is the lack of scalability. Musical compositions vary considerably in scale and variety. Some compositions might be very simple, while others are complex and elaborate. A good AMC system must be scalable. It must be minimally possible to combine two systems to form a composite system. Current attempts at AMC do not allow this.

2.5 Emergence and Emergent Computing

2.5.1 What is Emergence?

� �� $ � � � �% � �� & ' ��

Generally, “emergence” is a process of complex entity formation from simpler entities. Usually, emergence is considered in the context of a system. An “emergent system” is a complex system that has features that exhibit emergence. Not all complex systems are emergent. Listed below are some typical properties of emergent systems.

2.5.2 Properties of Emergent systems

Time Evolving Emergent systems are essentially dynamically evolving systems. Emergent properties usually “emerge” out of a Dynamic process that evolves over a period. For example, genetic algorithms converge to an optimal (or near optimal) solution by evolving over several generations.

Different Levels Emergent systems, structurally, consists of several levels of different size scales. The actual emergence actually happens across these size scales. For example, in a genetic algorithm, the final solution is a property of the entire population; but, this emerges out of interaction between genes, which are entities at a much lower level.

Literature Survey and Background - Techniques that will be used in this project

19

Systemic Properties Emergent systems have properties that its low-level-component-entities do not have. Such properties are termed systemic properties. The time evolving property, the existence of structural levels and the existence of systemic properties are often considered minimal requirements for a system to be classed “emergent”. Systems that conform to these are termed “weakly emergent”. Systems that possess systemic properties with the properties of irreducibility or unpredictability or both are termed “strongly emergent”. These are described next.

Irreducibility A systemic property is considered irreducible if it cannot be “deduced” from the properties of the system’s low-level-component-entities (even with complete knowledge of the structural arrangement and the individual properties of these). Informally this says that, “the whole must be more than the sum of its parts”.

Unpredictability A systemic property is considered unpredictable if it is, in theory (that is, even with “perfect knowledge”), impossible to predict its appearance in the future. Generally, most emergent systems also possess High Logical Depth and Backward Causal Power.

High Logical Depth ‘Logical depth’ is an information-theoretic measure of the time a computer takes to perform a task. Emergent systems usually have high logical depths. An algorithm that can recreate the effect of an emergent system (when one is known) is usually very complex, both in terms of time and memory.

Backward Causal Power The fact that low-level entities influence the characteristics and behaviour of high-level entities is generally assumed in most complex system. It has been suggested that in emergent systems, high-level entities influence the behaviour and interaction between low-level entities. This is known as “Backward Causal Power”. It has not only given much new insight into the behaviour of emergent systems, but also has been attributed with the power of emergent systems.

2.6 Techniques that will be used in this project

2.6.1 Cellular Automata (CA) A cellular automaton is a discrete model studied in computer science, mathematics and theoretical biology. It consists of an infinite, regular grid of cells, each in one of a finite number of states. The grid can be in any finite number of dimensions. Time is also discrete, and the state of a cell at time t is a function of the state of a finite number of cells called the neighbourhood at time t-1. These neighbours are a selection of cells relative to the cell, and do not change. Every cell has some rule for updating (evolution function), based on the values in its neighbourhood. Each time the rules are applied to the whole grid a new generation is produced. The CA grid can be one dimensional (line), two dimensional (plane) or multi-dimensional. One dimensional CA with two states per cell have been the most widely studied CA.


20

It is possible to have a single common evolution function for all the cells in a CA. Such a CA is known as a Uniform CA. It is also possible to vary the function from cell to cell. These are known as Hybrid CA. Usually, the evolution function is deterministic. However, it is also possible to use probabilistic or fuzzy functions. As mentioned, the output of evolution function is generally dependent on the previous state. However, it is also possible to have time-dependent fitness functions. It is also possible to have cyclic-time-dependent fitness functions (A simple case would be to alternate the fitness function every step) [GANGULY, 2004]. Emergent characteristics of Cellular Automata are summarised below:

The System The grid of cells as a whole forms the emergent system. High-level entity or Emergent Feature

The emergent feature is the pattern formed (as a whole) by the states of the individual cells on the grid.

Low-level entity Each individual cell. Time Evolving The final emergent pattern arises over several generations of the

grid. Systemic Properties

The grid patterns that evolve are often complex. An infinite (or even finite) grid is capable of having a wide range of intricate and elaborate patterns. These can also be mapped into other high-level characteristics (for example, colours or sound). Hence, the grid is capable of having a range of complex Systemic Properties. �� % � �� (��

CA can be categorised according to their long-term behaviour (global Dynamics). In [WOLFRAM, 1984], Steven Wolfram givens four categories:

• Class 1: CA which evolve to a homogeneous state • Class 2: CA that display simple separated periodic structures • Class 3: CA that exhibit chaotic or pseudo-random behaviour • Class 4: CA that yield complex patterns of localized structures

Design issues when implementing Cellular Automata for AMC The first design decision is to decide on what type of musical material can be generated using CA. This is fairly straightforward. As seen before, CA are ideal for small musical units – typically, Motifs, but cannot be used for larger forms since they lack global coherency. CA can also be used for harmonization. The next design decision is how to represent music using CA. Each cell has a state and also a position in the grid. Usually, characteristics such as pitch, instrument and other non-temporal information are represented in the state, while temporal information such as duration and start time of the note are represented in the position of the cell in the grid. However, these options can be reversed or otherwise modified. A decision also has to be made on the evolution function. This is context dependent. In this context it depends on the types of music to be generated using the CA or (in other words) the nature of Motifs. An evolution function that will enable the generation of successful Motifs has to be defined.


21

Finally, we must validate these results along with the long-term behaviour that we expect. Usually we would expect Class 4 behaviour. The exact design decisions will be discussed later under implementation.

2.6.2 Fractals A fractal is a geometric object which is rough or irregular on all scales of length and therefore appears to be 'broken up' in a radical way. Fractals can be most simply defined as images that can be divided into parts, each of which is similar to the original object. Fractals are said to possess infinite detail, and some of them have a self-similar structure that occurs at different levels of magnification. In many cases, a fractal can be generated by a repeating pattern, in a typically recursive or iterative process. Fractals are usually generated in one of the following ways: Iterated function systems (IFS fractal), which have a fixed geometric replacement rule; Escape-time fractals, which are defined by a recurrence relation at each point in a space; and Random fractals, which are generated by stochastic rather than deterministic processes. Emergent characteristics of Fractals are summarised below:

The System The fractal as a whole. High-level entity or Emergent Feature

The geometric shape of the fractal.

Low-level entity At the heart of any fractal generation process is some “fundamental process” or function that defines the generation process. Depending on the generation method, this might be a contractive function, a recurrence relation or some stochastic rule. It is how the geometric space behaves in the face of this “fundamental process” that defines the fractal. This fundamental process usually interacts with simple geometric entities (line points in space or simple line fragments). Hence, it is these simple geometric entities that serve as the low- level entities.

Time Evolving Although in theory a fractal can be thought of as a constant and complete product, in practice fractals are generated by some process that takes a non-zero time to execute. It is, hence, time evolving. This Dynamic view is sometimes better for understanding the actual nature of the fractal.

Systemic Properties

The geometric shape of the fractal is a macro-geometric characteristic that micro-geometric low-level, component entities do not possess.

�� % � �� (��

Design issues when implementing Fractals for AMC As before, the first design decision is to decide on what type of musical material can be generated using fractals. Like CA, fractals are ideal for small musical units. However, unlike CA, fractals naturally have global dependencies between entities (which apparently form symmetries in shape) and hence might be used for slightly larger musical units. However, at too larger scales, they are difficult to control. Also, fractals have geometric properties like “shape”. This makes them ideal for generating musical Phrases.


22

Next, a decision on the representation of the fractal must be made. Note pitches could be represented as local fractal entities (analogous to colour, for example) and temporal information represented by the geometric position of the fractal entity. Completed Motifs could also be (more adventurously) coded as fractal entities. As before, the exact design decisions will be discussed later under implementation.

2.6.3 Genetic Algorithms (GAs) Genetic algorithms are a particular class of evolutionary algorithms that use techniques inspired by evolutionary biology such as inheritance, mutation, natural selection, and recombination. Genetic algorithms are typically implemented as a computer simulation in which, a population of abstract representations (called chromosomes) of candidate solutions (called individuals) to an optimization problem, evolves toward better solutions. Traditionally, solutions are represented in binary as strings of 0s and 1s (though different encodings are also possible). The evolution starts from a population of completely random individuals and happens in generations. In each generation, the fitness of the whole population is evaluated, multiple individuals are stochastically selected from the current population (based on their fitness), modified (mutated or recombined) to form a new population, which becomes current in the next iteration of the algorithm [MITCHELL, 1996]. Emergent characteristics of Genetic Algorithms are summarised below:

The System The population as a whole. High-level entity or Emergent Feature

The optimal solution.

Low-level entity The genes. Time Evolving Genetic algorithms evolve over several generations, in which time,

genes interact in such a way that the average fitness of the population increases, paving the way for the emergence of an optimal (or near optimal) solution.

Systemic Properties

Each organism in the genetic algorithm consists of several genes that map to various characteristics. On their own, the genes are trivial, often being mere symbolic codes and do not possess any meaningful characteristics. The various characteristics of the organism could be described as the systemic properties. �� % � �� (��

Design issues when implementing GAs for AMC GAs will be used for generating high-level entities by combining low-level ones. The input into the GA will be a collection of low-level musical entities (for example, say, Motifs, Rhythms etc.). However, these will not form the actual GA population. The actual GA population will be formed by combining these initial low-level entities. The combination process is also a design decision. However, this is essentially GA independent and will be done according to a subset of musical rules. The reason for selecting a subset is to ensure that the generated output is not over-constrained. The actual validation of these will be done according to the GA fitness function. The nature of the fitness function is an important design decision. There are several options:


23

• Deterministic evaluation methods: The use a mathematical function to give the fitness of the individual musical event.

• Formalistic rule-based methods: These make use of stylistic features of existing music that are codified into a set of rules.

• User-determined evaluation methods call for subjective input from the user of the algorithm. The user is required to supply the fitness measures for individuals. Interactive Genetic Algorithms fall into this category.

• Neural methods: These recognize the good individuals according to its training data. Neural networks are also used to make user-determined evaluation methods more efficient.

A decision also has to be made on how to implement the mutation and crossover. It is important that these are implemented in a context dependent way. Mutation and crossover operators can typically mimic the fundamental operators of musical development. The nature of GA parameters such as mutation and crossover rates are also important design decisions. A decision will have to be made on how much variety and balance is required. Population size and convergence control measures must also be considered. However, like most GA parameters, optimal values cannot be calculated in advance. Several selection methods can be used in GAs. However, in this project we will use a customized variant of tournament selection. Unlike conventional “Roulette Wheel” and rank selection, tournament selection does not require that the population set is sorted. Nor does it required that the population be parsed (one or more times) before selection. Tournament selection is computationally efficient and also has selection pressure similar to alternate techniques. Fundamentally, tournament selection consists of picking organisms from the population at random and making them compete with each other. For added efficiency, in this project, we combine the tournament process into mutation and crossover, in addition to selection. In the case of mutation, an organism is selected at random, and a copy of it is mutated. If the organism is less fit than its mutated copy, it is replaced by it. For crossover, two organisms are selected at random and a third organism is generated by crossing over the former two. If the new organism is fitter than any one of the former two, it replaces the less fit one. Selection is done exactly as in conventional tournament selection with the exception that the fitter organism replaces the weaker one with probability 1. Note that this approach promotes a degree of elitism and is a kin to steady state selection [MITCHELL, 1996].

2.6.4 Reasons for selecting Emergent Techniques for AMC

Modelling the human compositional process As explained, a musical composition, as a whole, consists of many characteristic elements. These characteristic elements interact in complex and not-always-obvious ways. By themselves, the elements are often very short and trivial, with no apparent musical meaning. It could be said that, when a human composes a piece of music, what he actually does is to arrange these characteristic elements such that his aesthetic and expressive goals are achieved. Hence, at least on the surface, the human composition process itself seems to be an emergent system. Therefore, it is logical that emergent techniques are used to artificially model it.


24

Nature offers many examples Someone might argue that the previously expressed connection between the human music composition process and emergent techniques is only superficial (or even non-existent). Such an argument is, in a way, not unreasonable as the human brain has not been understood to a degree to logically justify such a claim. However, we have much indirect evidence to justify the use of emergence. Much of this comes from nature. For example, the branching of blood vessels in our circulatory system, the emergence of tree rings on the barks of trees; even weather patterns have been shown to follow the behaviour of emergent systems.

Flexibility As explained, there are many different ways in which AMC can be approached, and many different ways in which each approach can be implemented. In spite of drawbacks, each approach has good features as well. An advantage of using emergent techniques is that they can be adapted to incorporate good features of other techniques. For example: It is possible to incorporate statistical (especially stochastic) features into CA and fractals. GA fitness functions can easily encode Rule-based knowledge and Grammars; hence exploit their advantages, while at the same time avoiding disadvantages.

Robustness and Speed Emergent systems have no single-point of failure; if a single unit fails, the system still works. Emergent techniques tend to find a reasonable solution quickly and then optimise. Traditional computer algorithms tend not to produce a useful result until they are complete.

Interplay between local randomness and global determinism Creativity can be “generated” in two ways in a computer. Firstly, the generation of random musical entities can be considered “creative”; after all, random data can be said to have “never been seen before”. Secondly, if these random entities were composited to form some sort of higher level entities, these higher level entities could also be “creative”. Note that in the second case the results might cease to “appear” to be random. Generally these two kinds of creativity can be referred to as creativity resulting from local randomness and creativity resulting from global determinism. Emergent systems combine both these techniques in varying degrees; hence, allow a system capable of producing maximal “creativity”.

Stable Structure and error tolerance A disadvantage of using conservative techniques is that these do not allow flexibility in terms of the material generated. Any attempt at flexibility is resisted or else results in too many errors. Conversely, emergent systems have some degree of error tolerance and as a result give the system some extra stability.

Requirements Specification - Functional Requirements

25

Chapter 3: Requirements Specification 3.1 Functional Requirements As mentioned earlier, the principal goal of this project is to build an application which utilizes “emergent behaviour”, capable of modelling the human music composing process. Three types of functional requirements can be defined based on the sub-goals that this principal goal was split into. These are: Application requirements, framework requirements and technique utilization requirements. Of these, the application requirements will be defined in detail. However, the latter two will only be specified briefly, at a high-level since many functional characteristics tend to emerge later on when theories have been implemented. Since the project is of a research nature, this is assumed to be not much of an issue.

3.1.1 Application Requirements

Requirements on the type of music The output of the application is music. Since music composition is a potentially inexhaustible field, for practicality and clarity, this AMC application has been constrained by the following:

Tonal Music: As mentioned earlier, attention is restricted to composing western music. Out of western music, “tonal” music will be considered. Tonal music is music that is generally based on “melody”. Other types of music include “beat” music, “dance” music, etc.; these will not be considered initially, but can be incorporated later once the system is sufficiently designed. “Classical” Music: We will attempt to build a music compositional model according to “Classical” western music principals. Classical music is clearer in terms of theoretical concepts and more objective in terms of general listener opinion. As the project progresses, these attempts can be easily extended to consider other genres as well. Melodic Composition: We will chiefly consider melodic composition. There are two principal reasons for this: Firstly, there are several applications capable of high-quality harmonic composition. Secondly, the chosen technique (emergent behaviour) is more suitable for melodic composition. Note, however, that the harmonic aspect can be easily incorporated into the system once the melodic part is done.

Requirements on Basic Deliverables The application must be delivered as a set of reusable libraries. These must be capable of generating pieces of “tonal-western-classical-music” of reasonable creativity and novelty. Non-functional requirements of these deliverables have been defined later.

Requirements on Output Format The generated music must be in an accepted format that can easily be translated into standard western music notation. It must also be possible to playback generated music using standard techniques.

Requirements Specification - Non-functional Requirements

26

Requirements on User Involvements Human involvement must be minimal in decision making during the compositional process. However, it must be possible for a human to validate the music, and express her/his opinions about it. Hence, it is vital that the music produced is in a form playable by standard media software.

3.1.2 Framework Requirements To successfully apply “emergent behaviour” to a real-world problem, a suitable framework must be developed. This framework itself is a theoretical deliverable which can be later re-used. Hence it must incorporate advantageous characteristics of emergent techniques to full effect, while minimizing disadvantageous characteristics at the same time.

3.1.3 Technique Utilization Requirements When modelling any human intelligence process, the actual techniques utilized must justify their use. They must be validated using accepted techniques and shown to be comparable or superior to contemporary implementations. Like the framework, techniques are also theoretical deliverables which can be later re-used.

3.2 Non-functional Requirements The following non-functional requirements have been listed to counter certain issues that may arise.

3.2.1 Product Requirements

Usability Requirements Any Human-Computer interfaces (for example, those used for moderation and listening) should conform to reasonable usability metrics.

Efficiency, Reliability and Robustness Requirements Once the system is complete, it must satisfy efficiency, reliability and robustness constraints imposed by the domain. For example, if the system is implemented as a backend for a real-time application, it must conform to real-time constraints and standards.

Portability Requirements (Optional) The system is scheduled to be developed on Windows. However, since many potential applications run on other platforms, it would be convenient if migration is possible.

3.2.2 Organizational Requirements

Delivery Requirements Deadlines: This Individual Project (CSS4164) needs to be finished before June 2006. The deadline applies to both the application and associated documentation.

Implementation Requirements There are no direct requirements with regards to implementation software, hardware, etc. The selected implementation platform and components are acceptable.

Standards The project must conform to general accepted standards for coding, documentation, etc.

Requirements Specification - Non-functional Requirements

27

3.2.3 External Requirements

Interoperability Requirements As for Portability.

Ethical and Legislative Requirements Composers have intellectual property rights to their compositions. If any outside material is used, the legality of use must be verified. The legality of development and application software must also be verified.

Design - Overall Architecture

28

Chapter 4: Design 4.1 Overall Architecture

� �� ) � * + ��

At the core of the AMCTIES architecture, is a tree of interacting emergent systems (or TIES for short). TIES consists of a framework for implementing emergent systems and the implemented emergent system themselves (which we will refer to as generators). In additional to this, TIES needs several support modules. Since AMCTIES is a music composing application, music related constructs and algorithms need to be defined and implemented. Also, since the output of AMCTIES is for the MIDI format, we need support for writing musical compositions to MIDI files as well. Finally, we all these modules implement several mathematical functions. Support for these must also be provided.

Hence, the overall AMCTIES architecture consists the “framework” and the “generators” in TIES and support modules for Music Theory, MIDI and Mathematics.

4.2 TIES (A Tree of Interacting Emergent Systems) Generically, TIES (a Tree of Interacting Emergent Systems) consists of a cascading tree of emergent systems. The systemic properties of the root node emergent system are the actual output of the system. The low-level entities of each (non-leaf) node emergent system consist of its children’s systemic properties. For example, for this AMC application, the TIES root node will be an emergent system that produces the final piece of music. The low-level entities out of which this emerges are the systemic properties of child emergent systems that produce Subjects. These, in turn, have child emergent systems that produce Phrases. Finally, at the bottom of the tree are the lowest-level emergent systems that produce Motifs, Rhythms etc.. The user perceives the system as one emergent system that generates movements of music. The TIES framework, though ideal for the kind of AMC application pursued in this project, is not limited to AMC. It might be used in other areas of computer science as well.

Design - TIES (A Tree of Interacting Emergent Systems)

29

� �� , � �!% & �

TIES is, as mentioned, itself an emergent system. However, it has several advantages over monolithic emergent systems – hence its use in this project. Some of these are as follows:

Context Sensitivity A general problem with all techniques used in AMC applications (including emergent systems) was the lack of context sensitivity. Although a given technique might be good for implementing certain parts of an application, it is generally not suitable for implementing all application aspects. As seen, many AMC applications use a single technique to accomplish “too many” tasks. It is more the pity since many techniques are actually quite good for specific tasks. TIES solves this problem by having different systems with multiple capabilities hidden inside its structure.

Controllability Most complex systems (including emergent systems) are difficult to control. For example, in a conventional emergent system, it is difficult to control how exactly low-level entities influence high-level systemic properties. In TIES, it is possible to control interactions between the component node emergent systems (even though it might not be, in general, possible to control interactions within them). This gives better

Design - Design Philosophy

30

controllability, especially of design parameters that, often, directly influence the efficiency of the application.

Higher irreducibility The fact that TIES is more controllable does not in any way make it more reducible. TIES is a function built upon the nesting of several irreducible functions. Since, the interactions within it do not, in general, force trivialization, TIES itself is highly irreducible. The concept of “irreducibility” in emergent and other complex systems has been coupled with the concept of “non-linearity” (the fact that small changes can lead to big effects). The fact that it is made up of a hierarchy of interconnecting systems that each display non-linearity, multiplies the effect of non-linearity in TIES itself.

Higher unpredictability As was the case with irreducibility, TIES is also relatively more unpredictable. This is particularly important in the case of AMC since there is a very real need for creativity and originality.

Flexibility All that is required of each individual node in TIES is to play some specific role. Since, it is the role that is important, the actual implementation might be changed. TIES serves as a flexible framework that defines interactions, not specific implementations. Hence, it is possible to change actual implementations at will, when required.

Scalability A direct implication of flexibility is scalability. Although this does not seem directly relevant to AMC systems, future applications might benefit from this.

Parallel Implementability Since TIES has a distributed node-based structure, it has a high potential for parallelization. This might be important for real-time AMC applications.

Reusability As mentioned, TIES is a collection of nodes that play several different specific roles. These roles are, in general, specific parts of an application whole. The framework of TIES allows nodes to be implemented in independent modules, allowing free code reuse.

4.3 Design Philosophy

4.3.1 Mimic a process not a living thing AMCTIES aims at developing a system that mimics the human music compositional process. It does not attempt a system that mimics general human behaviour. Certain human characteristics (for example emotional effects) have not directly been incorporated into AMCTIES.

4.3.2 Not to re-invent the wheel The design philosophy (with respect to technique use) is not to re-invent the wheel by looking for new and novel techniques to implement Genetic Algorithms, Cellular Automata or Fractals by themselves. The design philosophy is to find the optimal way to combine these techniques in TIES. Optimal techniques have been selected (according to the previously explained criteria) and implemented as described later on.

Design - Design Philosophy

31

4.3.3 Maximize Modularity The emergent systems that make up TIES form modules. We will try to maximize the number of independent modules. If we concentrate functionality in one place (centralized system), the idea behind using TIES is lost.

4.3.4 Promote Controlled Emergence The creative force behind TIES is emergence. Hence it must naturally be promoted as much as possible. Hence, we use as many emergent techniques as possible. However, this must be controlled (to a certain extent) in order to produce coherent and non-ad hoc pieces of music. Hence, we hope to strike a balance between emergence and control. In the lower level entities we will use pure emergence freely. However, in higher levels we will control it with respect to conventional musical rules and conventions.

4.3.5 Ensure loose coupling between musical output and generation technique What a human user of AMC expects is a system that generates musical compositions. He/She will not be interested in the underlying techniques. Also, it must be possible to have several techniques that might be capable of generating musical output; the choice of which techniques to use should be shielded from the human user. We will attempt to satisfy these requirements by having a loose coupling between the musical output and the computer generation techniques.

4.3.6 User involvement Music is a very subjective field. It is vital that a music composing application involves the human user. However, this involvement must not be at the expense of the efficiency of the system or the freedom of the emergent techniques used. Hence, we will exclude direct user involvement from where it might be a potential bottleneck (for example, we will not use interactive genetic algorithms). Instead, we will try to make up for this by providing as many initial options for the user.

Design - TIES Design Components

32

4.4 TIES Design Components

4.4.1 Generators and Emergent Objects

� �� - � % � �� % � ��

In applying TIES to AMC, the “emergent systems” are a set of algorithms capable of generating musical entities (which are the “emergent objects” or “Emergent entities”). Emergent entities belong to a certain Emergent Object Type. Specifically, the emergent object types in AMCTIES are Motifs, Rhythms, Chords, Dynamics, Timbres, Phrases, Subjects and Movements. The emergent systems will be Motif Generators, Rhythm Generators and so on. Note: We often refer to an object of a type by the name of a type. For example, when we say a Motif Generator generates a Motif, we mean that it generates an instance of type Motif. It is hoped that the difference is inferred from the context. Generators and emergent entities are the two fundamental design components of TIES. In trivial cases, these two alone will be sufficient to implement TIES; however, this is not so for more complex ones. To see this let us first consider a simple case where we have a single Motif Generator, a single Rhythm Generator and so on. Also suppose that generating high-level emergent objects requires exactly one of each of its immediate low-level objects. For example, suppose the Phrase generator needs exactly one Motif, exactly one Rhythm and so on. This can be very easily implemented. We generate a Motif using the Motif Generator and store it in some temporary variable. We do the same for the Rhythm, Chords etc. Finally, we retrieve the stored low-level objects and pass it to the Phrase Generator to generate the Phrase. The process is continued to generate the Subject and the Movement.

Design - TIES Design Components

33

Note: From here on when speaking in the context of TIES, by “generating high-level emergent objects from low-level objects” the author means “generating high-level emergent objects from the low-level objects in the immediate lower level”.

4.4.2 Emergent Object Collections In most cases several low-level objects would go into generating a single high-level object. For example, we need several Motifs to generate a single Phrase. To complicate matters further, several low-level objects might be used to generate several high-level objects. For example, several Phrases might be based on several Motifs. However, not all low-level objects might be suitable for generating a particular high-level object. This poses two design requirements: Firstly, we need some mechanism of temporarily storing low-level objects. Secondly, we need to specify what low-level objects should be used to generate each high-level object. We satisfy the first requirement by implementing design components known as Emergent Object Collections. An Emergent Object Collection is simply an array storing pre-generated emergent objects. The objects might be accessed via the array index. For each emergent entity type, we have exactly one Emergent Object Collections. For example, a TIES implementation will have exactly one Motif Collection, exactly one Rhythm Collection and so on.

4.4.3 Maps and Map Elements The second requirement is satisfied by implementing design components known as maps. A map simply “maps” low-level objects to high-level objects. For each high-level object that need be generated, a map specifies what low-level objects should go into the generation process. The low-level object is specified via its array index in its respective collection. Since a high-level object usually corresponds to several low-level objects, the indices of these low-level objects are stored in as a single structure known as a Map Element. For example, a Phrase Map Element will consist of an index of an element in a Motif Collection, an index of an element in a Rhythm Collection and so on.

4.4.4 Generator Collections To produce a variety of musical styles we would have to use several generators for each type of emergent object. For example, we might have a Fractal Chord Generator and a Cellular Automata Chord Generator. For this reason, we need to also specify what generators should be used in generating each high-level object. This too can (and is) specified in the map. We can store all the generators in an array, which we will call a Generator Collection and access the element generators via its array index. Hence, (finally) a map contains an index to a generator array and a set of map elements.

4.4.5 Map Collections We group all the map elements for each emergent entity type in a single array which we will refer to as a Map Collection. For each emergent entity type, we have exactly one Map Collection. For symmetry, we can implement the Map Collection such that a map pertaining to a particular object is indexed by the objects index in its respective Emergent Object Collections.

Design - Designing Generators

34

4.4.6 Lowest level emergent objects Above, we have partially assumed that all emergent objects are generated using lower-level emergent objects. This is not true for the lowest-level emergent objects (for example, Motifs, Rhythms etc.). In fact in the case of AMCTIES most of the emergent object types are at the lowest level of the tree. The design components have to be slightly modified to accommodate these. Lowest-level emergent objects have no map elements. Corresponding maps will only specify the generator index. Hence, such maps might be specified as an integer array.

4.4.7 Map Generators Maps are generated via design components known as Map Generators.

4.4.8 Summary of design components

� �� . � & � � � �' �� (��

4.5 Designing Generators

4.5.1 Motif Generation A Motif is simply a sequence of pitches. The only constraints that must be adhered to when generating Motifs is that certain pitch relationships between pitches must be respected.

Random Fractals Random Motif Generation can be implemented in two ways. First, we could generate a random sequence of pitches, and then modify certain notes that may not respect any undesirable pitch relationships. There are two types of such undesirable relationships: One, pitches that do not correspond to the scale of the Motif. Two, notes that have either too high or too low pitches. This can be done by using methods ModifyToChord and NormalizeRange in class Modify.


35

� (modify)

� �� / �� (��

Second, we could generate a sequence of pitches at random, but from a set of pitches that are guaranteed of satisfying the above pitch relationships. Random Motif Generation is very simple to implement and is very efficient. The randomness gives a natural creativity and emergence. It could be argued that this method is “ad hoc” and no rational pattern could emerge out of this. This argument might be countered with two points: One, Motifs are very short entities and it is difficult to spot rational patterns in them. Two, qualities that might emerge out of a process of generating Motifs have less to do with structure and more to do with melody. However, this view might be considered subjective; hence, we will look at some other methods of generating Motifs as well.

Iterated Function System (Fractal) This will exploit the fact that within Motifs certain symmetries exist.

Initial State

(Generation 0)

�

Generation 1

�

Generation 2

� �� !� & �� (��

This fractal implementation is intuitive as a pitch is dependent on its predecessor and successor. The generation function can be implemented in several ways: We can use a deterministic table which maps (Predecessor, Successor) pairs to generator point values. We can also make this same process stochastic by making the table probabilistic (For example, use a second order Markov Chain). The building of the table can be done either according to known musical relationships or using some other process (We could even incorporate this into TIES by using an emergent technique. If the initial state is the same and the number of generations is constant, the deterministic technique will always return the same result. Hence, it is vital (for variety) that these variables (initial state and number of generations) are changed.

Escape Time Fractal. Compared to the IFS technique, using Escape Time Fractal will give more variety since generation depends on only one factor (the Predecessor). As before, the generation function can be either deterministic or stochastic (The same design decisions discussed also apply). Additionally, however, to prevent the fractal from growing out of range, it must be bounded.


36

There are two implementation problems in Escape time fractals. Firstly, they might not be suitable for relatively short Motifs. This is particularly problematic as we will be implementing Motifs as lowest-level entities in TIES. Secondly, the functionality of Escape Time Fractals ( at least in this context) can be mimicked to an extent by Random Fractals.

Initial State

(Generation 0)

�

Generation 1

�

Generation 2

� �� % � �� (��

4.5.2 Rhythm Generation

Random Fractals As with Motifs, the random technique has also been used for Rhythms. The technique used here is as follows. Initially, the Rhythm consists of a single pulse with duration equal to the duration of the entire Rhythm. The pulses in the Rhythm are iteratively selected and split randomly.

Generation 0

�

Generation 1

�

Generation 2

�

Generation 3

� �� / �� / � ' ��

Iterated Function System (Fractal) An Iterated Function System based technique has been used to generate Rhythms. As in the case of Motif note sequences, it is essential that Rhythms form some coherent structural shape.

�

� �� " � !� & �� / � ' ��

Beat Rhythm Generation In many musical works, Rhythms follow standard well-known patterns. In such cases, the Rhythms can be hard-coded, and it is not necessary to use fractals or a similar generation method. A single piece of music might have a well-known Rhythm pattern intermingled with more novel patterns; hence, it is important that both strategies of Rhythm generation co-exist.

4.5.3 Timbre Generation

Fixed Timbre Groups Usually, the Timbre of music is fairly fixed. In cases where there are more than one instrument playing simultaneously, these are assigned specific parts. Hence, we will not use emergence for


37

generating sequences of Timbre possibilities. We will simply assign constant instruments for each MIDI channel.

4.5.4 Dynamic Generation

Fractal based Dynamic Generation An Iterated Function System based technique is used to generate a Dynamic pattern. A Dynamic pattern in a piece of music is a sort of time series. On the other hand, it can also be considered a sort of architectural shape, with peaks and valleys. Iterated Function System Fractals are ideal for generating this behaviour. However, since AMCTIES is essentially a music composing application (and not a sound generating one), it might be argued that elaborate dynamic directions are superfluous. We might be better off with a simpler design and leave the dynamics to the discretion of the performer.

Constant Dynamic Generation Unlike Motifs, Rhythms, Chords and Timbres, in works of music, the Dynamic variety is usually left to the discretion of the performer. Hence, it is not vital that the exact Dynamic directions are specified. Also, modifying the Dynamics of a piece would not (usually) change its style or character that much. Hence assigning a constant value for Dynamics would not impair the compositional process too much, and would also add to practicality and simplicity.

4.5.5 Chord Generation

Fractal based Chord Generation An Iterated Function System based fractal technique provides a very intuitive strategy for generating sequences of Chords. A Chord is dependent on its predecessor and successor. Conversely, given a predecessor and successor it is possible to deterministically (or probabilistically) say what the Chord should be. This technique is very similar to the IFS fractal based technique for generating notes. However, unlike in the case for generating note sequences, the nature of Chords is governed by harmony. Hence, in this case, we are constrained by harmonic rules. So we have less freedom in specifying deterministic (or probabilistic) rules. This results in reduced creativity. Conversely, we have the option of ignoring music theory constraints in order to give emergence a freer hand. However, this could result in some “unmusical” sounding output.

CA based Chord Generation The principal behind using CA for Chord generation is the same as that in using fractals. We use the fact that a Chord is dependent on its predecessors and successors. However, in the case of CA we can explore these dependencies more deeply. For example, we can use an evolution function that considers a neighbourhood that goes beyond immediate predecessors and successors. We can also consider relationships between individual notes within Chords, by using a two dimensional CA. As in the case of fractal based generation, we are contained by music theory rules.


38

Standard Chord Generation As in the case of Rhythm, many “standard” Chord progressions are found in pieces of music. Certain kinds of pieces involve the repetitions of the same Chord sequence over and over again. Hence, it is important that this is implemented as well.

4.5.6 Phrase Generation

GA based Phrase Generation The emergent entities considered thus far were lowest level entities; that is, they are not generated using other emergent entities. A Phrase is generated using a set of Motifs, Rhythms, Chords, Timbre and Dynamics. In our TIES design these have been generated earlier and stored as emergent object collections. The exact low-level entities to be used in the generation of the Phrase are defined in a PhraseMap. Hence, part of the “generation” of the Phrase is defined in the Phrase map – however, the Map generation process is fairly simple and unimportant in this context. Generating a Phrase does not consist of simply merging the low-level material together. It must be verified that these are compatible. For example, we must check if a certain Motif is compatible with a Chord. If not the Motif must be modified to fit the Chord. Also, Motifs, Chords, Dynamics and Timbre might not be compatible with the Rhythms. For example, the Motif might contain more (or less) pitches than the Rhythm. If this is the case, the Motif must be cropped (or extended artificially). Finally, when several Motifs are concatenated to form the Phrase, we must make sure that the end of one Motif is compatible with the beginning of the next Motif. The above “verification of compatibility” is not a trivial task. GA based Phrase Generation consists of performing the “compatibility assuring” tasks using GA techniques. For specifically the GA tasks are as follows:

• Given a Rhythm and Motif, we evolve the musical material to find some coherent NoteSequence (Actually this should be “Given a Rhythm, Motif. Timbre and Dynamic”. However Dynamics and Timbre are fairly independent of fellow low-level entities and can be exempted from this process).

• Given the above evolved NoteSequence and a Chord, we evolve the NoteSequence to

suitably fit the Chord.

• Given a set of NoteSequences, we evolve them so that their concatenation will form a coherent Phrase.

4.5.7 Subject Generation and Movement Generation

Fixed Subject Generation and Fixed Movement Generation. Subjects and Movements are high-level musical entities. This has some pertinent consequences. Since they are temporarily fairly long, a single piece of music has relatively few of them. For example, a movement in what is known as “Simple Rondo Form” consists of just five Subjects A1+B+A2+C+A3 (Here A1, A2 and A3 might be identical or slightly varied). For this reason, it is difficult to give creativity or novelty to the piece by controlling the generation process. Subjects and Movements are essentially structural entities and are there to give the piece an architectural foundation, not variety or colour. Conversely, following fairly set patterns in music theory in generating Subjects and Movements would not reduce the creative nature of the piece.

Design - Designing Maps

39

Hence, we follow a “fixed” approach to Subject and Movement generation. That is when combining Phrases to form Subjects we will simply follow standard musical rules; we will not use any emergent techniques. The same applies to combining Subjects to form Movements.

4.6 Designing Maps

4.6.1 Random Map Generation As mentioned earlier, each emergent system generates a set of outputs (Emergent Object Collection). High-level emergent systems input and process the output of low-level emergent systems. A map lists the collection index of the relevant low-level emergent object that must go into generating the high-level emergent object. For example, suppose we have generated 12 Phrase (Phrase 0, Phrase 0,…, Phrase 11). Also suppose we are to generate a Subject with four Phrases. A SubjectMap might specify that the first Phrase in the Subject should be Phrase 1, the second should be Phrase 10, etc.. The simplest method of performing this mapping is to Map at Random. In the previous example, the four Phases could be assigned with a Random Phrase with an index in 0,1,…,11. This has the obvious advantage of being fast. However, it is somewhat ad hoc and results in somewhat incoherent results. However, it is good when the number of objects in the Emergent Object Collection is small.

4.6.2 Ordered Map Generation Another strategy equally simple as Random Map Generation is to do the exact opposite. That is to Map objects in a strict order. Under this scheme, in the previous example, the four Phases would be assigned with Phrases Phrase 0, Phrase 1, Phrase 2 and Phrase 3. Although, this might seem rigid and mundane, it does have some advantages. In many cases, Rhythms and Chords follow certain sequences. For example, in much dance music, a single Rhythmic-Chordal sequence is repeated over and over. For example, in “La Bamba” popularized by Triny Lopez, we see a 4-3-5-4 Rhythm and a C-F-G-G Chords repeating.

� �� $ � �� 01 ��2 �� 3�

Ordered Map Generation allows a simple way of reproducing such Rhythm patterns. It does, however, require an additional wrapping around the original emergent object generator. However, this is easily accomplished. It simply consists of generating Rhythms in an index specific way. For

Design - Basic Design Modules

40

example, if we want a 4-3-5-4 Rhythm repetition, we generate the Rhythms so that the first has a duration of 4 pulses, the second 3 pulses and so on. Note that there is no change to the actual generation method (e.g. Fractal).

� �� ) � 4 ��

4.7 Basic Design Modules

� �� , � � � �� & ��

AMCTIES is consists of five packages: AMCTIES.Framework, AMCTIES.DefaultGenerators, AMCTIES.MusicTheorySupport, AMCTIES.MathSupport and AMCTIES.MIDISupport. The latter three are support libraries that can be used for other applications as well. AMCTIES.Framework defines the representation of the design components. It does not implement any emergent systems, but defines how they are to interact in order to facilitate automatic music composition. The actual emergent systems are implemented in AMCTIES.DefaultGenerators. Note that AMCTIES.Framework might be used with other generators as well. AMCTIES.DefaultGenerators simply consists of a set of generator designed for this project. The implementation of AMCTIES.Framework and AMCTIES.DefaultGenerators are described in the next two chapters. The implementation of AMCTIES.MusicTheorySupport, AMCTIES.MathSupport and AMCTIES.MIDISupport is discussed in the chapter thereafter.

Implementation - AMCTIES.Framework - Representing Emergent Objects

41

Chapter 5: Implementation - AMCTIES.Framework 5.1 Representing Emergent Objects The representation of Emergent Objects is defined in package AMCTIES.Framework.

5.1.1 Motifs Motifs will be viewed as a sequence of pitches (independent of Rhythm, Dynamics and Timbre). Hence, a Motif will be represented as an array of integers.

int[] Motif;

Interface IMotifGenerator is implemented by classes that may generate Motifs.

public interface IMotifGenerator { int[] GenerateMotif(); }

5.1.2 Rhythms Rhythms are essentially a sequence of note durations and are also represented as an array of integers.

int[] Rhythm;

Interface IRhythmGenerator is implemented by classes that may generate Motifs.

public interface IRhythmGenerator { int[]GenerateRhythm(); }

5.1.3 Timbre Timbre will be viewed as a “sequence of Timbres” corresponding to a sequence of notes. Hence, Timbre too will be represented as an array of integers.

int[] Timbre;

Interface ITimbreGenerator is implemented by classes that may generate Motifs.

public interface ITimbreGenerator { int[]GenerateTimbre(); }

5.1.4 Dynamics Dynamics is similar to Timbre.

int[] Dynamics;

Interface IDynamicGenerator is implemented by classes that may generate Motifs.

Implementation - AMCTIES.Framework - Representing Emergent Objects

42

public interface IDynamicGenerator

{ int[]GenerateDynamic();

}

5.1.5 Chords A Chord is a set of pitches. Chords will be viewed as a sequence of Chords that correspond to a sequence of notes. Hence, Chords will be represented as an array of integer arrays.

int[][] Chords;

Interface IChordGenerator is implemented by classes that may generate Motifs.

public interface IChordGenerator { int[][] GenerateChord(); }

5.1.6 Phrases, Subjects and Movements Phrases, Subjects and Movements are compound collections of the previous entities. These will be represented as NoteSequence objects. NoteSequence Phrase; etc...

Interface IPhraseGenerator is implemented by classes that may generate Motifs. Note that the parameters contain: A map (PhraseMap) and a set of lower level Emergent Object Collections (Motifs, Rhythms, etc.). This pattern will be followed throughput TIES.

public interface IPhraseGenerator { NoteSequence GeneratePhrase( PhraseMapElement[] PhraseMap,

int[][]Motifs, int[][]Rhythms, int[][]Timbres, int[][]Chords, int[][]Dynamics, Scale[] scales ); }

Interface ISubjectGenerator is implemented by classes that may generate Motifs.

public interface ISubjectGenerator { NoteSequence GenerateSubject( SubjectMapElement[] SubjectMap, NoteSequence[]Phrases, ); }

Implementation - AMCTIES.Framework - Representing Maps and Map Elements

43

5.2 Representing Maps and Map Elements The representation of Maps and Map Elements is defined in package AMCTIES.Framework.

5.2.1 Motif, Rhythm, Chord, Dynamic and Timbre Maps Motif, Rhythm, Chord, Dynamic and Timbre are not made up of further low-level entities. Hence, maps corresponding to these simply store the indexes of their generators, and can be implemented as integer arrays.

5.2.2 Phrase Maps Phrase Maps and Phrase Map Elements are represented as follows. . public class PhraseMapElement { public int Motif_index; public int Rhythm_index; public int Timbre_index; public int Dynamic_index; public int Chord_index;... }

public class PhraseMap { public int PhraseGenerator_index; private PhraseMapElement[] elements; ... }

5.2.3 Subject Map Subject Maps and Subject Map Elements are represented as follows. .

public class SubjectMapElement { public int Phrase_index; } public class SubjectMap { public int SubjectGenerator_index; private SubjectMapElement[] elements; }

5.2.4 Movement Map Movements consist of a direct collection of Subjects. Hence, the mapping can be stored as a simple collection of indices (as an integer array).

5.3 Representing the User’s Environment AMCTIES is a framework that contains several techniques for generating a range of different types of music. As it attempts to mimic human musical composition, certain aspects of a human’s environment need to be represented within the framework. This can be done by the user selecting what generators to use and other options.

Class Global defines an abstract representation of the User’s Environment. This consists of several option variables, public method Generate and several supporting private methods. Class Global

Implementation - AMCTIES.Framework - Representing the User’s Environment

44

is by far the most important single class in AMCTIES. This is where the whole system is “held together”. Firstly, class Global defines and stores all the map generators and generator collection as variables. The map generators and generator collections are initialised by the user according to his/her requirements. What map generators and generator collections are set defines the “ability” and “experience” of the composing application.

protected IMotifMapGenerator MotifMapGenerator; ... protected IMovementMapGenerator movementMapGenerator;

protected IMotifGenerator[] MotifGenerators; ...

protected ISubjectGenerator[] SubjectGenerators;

Additionally, class Global defines the scale of emergent entity generation, by specifying how many entities must be produced by generators.

protected int MotifCount; ... protected int SubjectCount;

It also specified various MIDI options. These are directly passed to method NoteSequenceToMIDIFile in class AMCTIES.MIDISupport. MIDISupport when the results are finally output to MIDI files. Method NoteSequenceToMIDIFile will be discussed later.

: public byte[] channels; public int[]pitchOffsetPerChannel; public int keyOffset;

The TIES framework is exposed by way of the public method Generate. This interacts with several private methods that implement various parts and functions of the TIES tree. Emergent Object Collections are stored as variables in these methods. Emergent entities are generated by using the generators defined in the map generators and generator collections. The methods in class Global are as follows. public void Generate(int melodyCount, out NoteSequence[] melodies) {

For melodyCount iterations {

Generate a highest-level entity (movement) by method GenerateGlobalMovement iteratively

Add this to melodies }

}

public NoteSequence GenerateGlobalMovement(All the map generators) {

Generate low-level entities by calling method GenerateGlobalMovementLowLevel


45

Generate non-low-level entity maps by calling method GenerateGlobalMovementMiddleLevel

Generate non-low-level entities by calling method GenerateGlobalMovementHighLevel

return the highest level entity (movement); }

public void GenerateGlobalMovementLowLevel(

Map Generators for lowest level emergent object types , Lowest level emergent object collections (out))

{ Generate Map Collections (for Motifs, Rhythms,...,Timbre)

using the Map Generator Generate the Lowest emergent object collections }

public void GenerateGlobalMovementMiddleLevel(

Map Generators for non-lowest level emergent emergent object types , Map collections for non-lowest level emergent emergent object types (out))

{ Generate Map Collections (for Phrases, Subjects and movements)

using the Map Generators }

public void GenerateGlobalMovementMiddleLevel(

Lowest level emergent object collections, Map collections for non-lowest level emergent emergent object types, Movement(out))

{ Successively generate phrases, subjects and Movement

}

When using AMCTIES libraries to build a custom automatic music composing application, class Global must be extended and the users specific options and specification must be initialised in the class constructor. The methods of the class should be used without change. For example, the demo application AMCTIES Composer (described in Appendix D) implements class ComposerGlobal extending class Global. It has the following constructor:

public ComposerGlobal( Custom constructor parameters... int duration, int tempo, int maxChannels, int[] variety, int[] quotas) { Specify MIDI options... keyOffset =0; this.tempo = 500000*tempo/Duration.Crotchet; ... etc.


46

Define the scale of emergent entity generation... motifCount= (int)(Math.Pow(2,melodicVariety)); rhythmCount = (int)(Math.Pow(2,rhythmicVariety)); dynamicCount = 1; ... etc. Specify (emergent entity) generators... motifGenerators = new IMotifGenerator[]{ new RandomMotifGenerator(scale) }; rhythmGenerators = new IRhythmGenerator[]{ new RandomRhythmGenerator(Duration.Crotchet,0.3, 4),

new BeatRhythmGenerator(Duration.Crotchet,0.3, 3, 3, 2) }; ... etc. Specify Map generators... motifMapGenerator = randomMapGenerator; rhythmMapGenerator = randomMapGenerator; ... etc. }

Implementation - AMCTIES.DefaultGenerators - Fractals

47

Chapter 6: Implementation - AMCTIES.DefaultGenerators 6.1 Fractals

6.1.1 Random Motif Generation The random fractal Motif generation has been implemented in class RandomMotifGenerator. As mentioned before, there must be some control over the pitches allowed; hence, all the pitches will be derived from a given scale. Initially a random pitch is selected (from a given scale). The notes of the Motif are formed by successively adding (or subtracting) a random number of pitch steps from each successive note. For example, the first note is the initial pitch plus (or minus) a random amount; the second note is the first note plus (or minus) a random amount. The implementation is as follows:

public int[] GenerateMotif() { Generate a random integer, pitchIndex for(int i=0; i<LENGTH; i++) where LENGTH is the number of notes in the Motif { Add or subtract a random amount from pitchIndex Let pitch = the pitchIndexth note in some given scale

Make sure that pitch is within a reasonable range (it Cannot be too high or too low)

Add pitch to the Motif } return Motif; }

This technique is similar to an escape time fractal in that each note is dependent on its previous note. However, this relationship is non-deterministic. Any observable relationship is due to change. However, we are likely to observer global “emergent” patterns.

6.1.2 Random Rhythm Generation The random fractal Rhythm generation has been implemented in class RandomRhythmGenerator. The implementation is as follows:

public int[]GenerateRhythm() {

Integer array pulseArray stores whether or not a beat falls on a given pulse. Initially, pulseArray[j] = 0 for all j.

For each pulse, j For k = 8, 4, 2, 1

If it is possible to have a note of duration, k pulses at pulse j

Set pulseArray[j] = 1 Convert pulseArray into integer array Rhythm that contains

durations that make up the Rhythm, return Rhythm }

In addition to musical constraints, “If it is possible to have a note of duration” considers a stochastic factor as well. By increasing the probability, it is possible to generate Rhythms with many short notes; reducing it will result in a Rhythm with a few long notes. It is this that encodes the “randomness”.

Implementation - AMCTIES.DefaultGenerators - Fractals

48

6.1.3 Fractal Rhythm Generation Iterated System Fractal Rhythm generation has been implemented in class FractalRhythmGenerator. The implementation is as follows:

The fractal seed is stored in double array normalizedSeed

public int[]GenerateRhythm() {

Initialize integer array fractal to store a single pulse For a given number of generations {

Evolve fractal to give a new fractal by method EvolveFractal }

} private int[]EvolveFractal(int[] fractal) { Split each element in fractal according to normalizedSeed }

6.1.4 Fractal Chord Generation Iterated System Fractal Chord generation is implemented in class FractalChordGenerator. The generation of Chords has been restricted to major and minor Chords. There are two reasons for this: First, the table storing the stochastic relationships between Chords has memory complexity O(n2) with respect to the number of Chords. Second, most conventional music can be represented by solely major and minor Chords. The implementation is as follows:

Relationships between Chords are stored in array fractal_data public int[] GenerateChord() {

Integer array Chord_fractal stores the generated Chord sequence in a coded form

Set Chord_fractal[first index] and Chord_fractal[last index] to some given startChord and endChord respectively.

Call SetMid that will recursively generate the Chords Convert Chord_fractal into integer array Chord that will

represent the actual Chord. }

public static void SetMid(int[]Chord_fractal, int start_index, int end_index) {

Calculate the mid point of Chord positions start_index and end_index, mid.

If mid has already been assigned a Chord return. Assign mid a Chord, based on fractal_data, and the Chords as

start_index and end_index Recursively call SetMid(Chord_fractal, start_index, mid) and SetMid(Chord_fractal, mid, end_index)

}

Array fractal_data is initialized when the class is constructed. This is random, but done according to accepted musical rules.

Implementation - AMCTIES.DefaultGenerators - Cellular Automata

49

6.2 Cellular Automata

6.2.1 CA Chord Generation Cellular Automata Chord generation is implemented in class CAChordGenerator. The Chord is represented by a cellular automata consisting of 12 cells, each representing a note label (C, C Sharp, D...A Sharp, B). The cells take binary values (zero indicating that the note is not contained in the Chord). The next state of each cell is dependent on all 12 cells (including it). Since the relationships between notes are cyclic, the next state dependency depends only on the relative positions of the note labels (For example, C and E are four semitones apart, so are G and B. Hence, B depends on G exactly the same way E depends on C). Hence, the evolution function for the cellular automata can be encoded in Boolean array (function) with 212 elements. For example, if the current population is (0,0,0,0,0,0,1,1,0,0,1,0) and function[25] = 1, the next state will have the note label C Sharp in it. (Note: C Sharp is represented in cell with index 1 in the cellular automata; 000000110010 circularly shifted one position gives 0000000110012 = 2510) It is possible to define function according to musical rules. However, we have defined it at random to see what results might emerge. Although the cells of the cellular automata and the elements of function are binary, they have been implemented as integers. However, they still store binary values (one or zero).

public CAChordGenerator() { Initialize function randomly Generate a specified number of Chords and store in Chords }

public int[][] GetCAChords(int ChordCount) { Create an initial cellular automata, and store it in integer

array CA For the number of Chords required (ChordCount) {

For gen number of generations... { Evolve the cellular automata }

Add the Chord derived from the cellular automata to Chords

} return Chords; }

public int[] GenerateChord() { return the next Chord in Chords and increment the counter

count }

All the Chords are generated when the instance object is constructed and then cyclically returned.

Implementation - AMCTIES.DefaultGenerators - Genetic Algorithms

50

6.3 Genetic Algorithms

6.3.1 GA based Phrase Generation The generation of Phrases consists of combining several low-level entities. Since Rhythm, Dynamics and Timbre are mutually independent they can be directly combined. However, as mentioned, certain dependencies exists between Motifs and Chords. The notes in the Motif need to be “evolved” to fit the Chords. This task can be done using Genetic Algorithms, and is implemented in class GATransposer. The Genetic Algorithm uses tournament selection. Its selection has bee justified earlier. The process is initiated by calling method ModifyToChord. Certain modifications to pitches need to be done relative to a scale. Also, since a Motif is dependent on Motif heard previously, the last note of the latter needs to be specified (last_pitch). public static NoteSequence ModifyToChord(

NoteSequence noteSequence, Scale scale, int[] Chord, int last_pitch) {

Initialize population: All organisms are set equal to copies of noteSequence

For GENERATIONS number of generations... { For TOURNEMENT_SIZE number of iterations... { Select a random organism

Make a copy of this and mutate it by method Mutate (The mutation probability is implemented within this method)

If the copy is fitter than the original, replace the original with the copy. Fitness is measure by method GetFitness

} For TOURNEMENT_SIZE number of iterations... { With a specified CROSSOVER_PROBABILITY { Select two organisms at random

Create a new organism by performing crossover on these, using method crossover.

If the new organism is fitter than any one of its parents, replace the weaker parent with the child.

} } For TOURNEMENT_SIZE number of iterations... { Select two organisms at random Replace the more fitter by the less fitter } }

Select and return the fittest organism }

Implementation - AMCTIES.DefaultGenerators - Non-emergent Implementations

51

public static NoteSequence Crossover(NoteSequence noteSequence1, NoteSequence

noteSequence2) {

Generate a new NoteSequence, with the same number of notes as noteSequence1 and noteSequence2, and filling each note position with notes from either noteSequence1 or noteSequence2 (with equal probabilities). Note, this corresponds to uniform crossover.

Return the new NoteSequence }

public static NoteSequence Mutate(NoteSequence noteSequence) {

A copy of noteSequence is mutated using four techniques implemented in class Modifier. These are ScaleTranspose, ReversePitches, Invert and Mutate. Each are done with a certain probability. These combine to give a composite mutation probability.

Return the mutated NoteSequence. }

public static double Fitness(NoteSequence noteSequence) {

Return a fitness measurement based on how well the note sequence matches the Chord. This uses method MatchChordProb is class Tests. It also considers last_pitch

}

This implementation also has several parameters that need to be set.

private const int POPULATION_SIZE = 64; private const int TOURNEMENT_SIZE = 128; private const int GENERATIONS = 32;

We need to specify the population size and tournament size corresponding to the running of the Genetic Algorithm. We also need to specify for how many generations the population evolves. We could wait until a certain average fitness is reached, but this may result in the algorithm not terminating (especially if a sufficient solution does not exist). In addition to these we need to specify the crossover rate (probability) and the mutation rate. Since the mutation is compounded into four independent mutation operators, the individual rates must be specified optimally.

6.4 Non-emergent Implementations

6.4.1 Beat Rhythm Generation and Formal Accompaniment Generation Beat Rhythm generation consists of generating well-known beat Rhythms. Beat Rhythm generation is implemented in class BeatRhythmGenerator and simply consists of superimposing a series of random fractal Rhythms onto a given set of beats.


52

public int[]GenerateRhythm() { For each beat...

{ Generate a random fractal Rhythm with the duration of

the beat Add this to Rhythm

} return Rhythm; }

Formal accompaniment generation is the Motif equivalent for beat Rhythm generation. It consists of generating a Motif that is based on a given Motif pattern. Certain well known Motific sequences are used to accompany music and it is important that these might be generated.

Might be used to generate

� �� - � � � � ��

6.4.2 Standard Chord Generation Like beat Rhythms and formal accompaniments, certain standard Chord sequences are used in music. This functionality is implemented in class StandardChordGenerator. These can be generated directly. For example to generate a major Chord:

public static int[]GetMajorChord(int root) { return new int[]{root,root+4,root+7}; }

More elaborate Chord sequences can be generated by combining simple operators.

public static int[][]Chords_pachelbelCanon = new int[][]{GetMajorChord(Pitch.C), GetMajorChord(Pitch.G), GetMinorChord(Pitch.A), GetMinorChord(Pitch.E), GetMajorChord(Pitch.F), GetMajorChord(Pitch.C), GetMajorChord(Pitch.F), GetMajorChord(Pitch.G) };

This Chord sequence is based on the 17th Century composer Pachelbel’s Canon in D. The Chords are (transposed from D major to C major ):

� �� . � �� & �5 � �� 6��


53

Standard Chords are implemented as static variables of StandardChordGenerator. Construction of a class instance consists loading one of these into an instance variable that is later use to return Chords to the user.

6.4.3 Classical Form Generation Classical Form Generation is implemented in class ClassicalFormGenerator. In our context, the use of classical forms applies to mapping Subject to for a movement. For example, in a movement in “Ternary Form”, two Subjects A and B form the movement in the form ABA. This might be implemented as:

public int[] GenerateGlobalMovementMap() {

return new int[]{0,1,0}; }

Here, Subjects A and B must correspond to the entities with indices 0 and 1 in the Subject collection.

6.4.4 Ordered Map Generation, Random Map Generation and Partitioned Map Generation Ordered maps and partitioned maps provide a way of controlling the nature of the entire piece of music, merely by controlling the way low-level entities are generated. Ordered maps simply consist of mapping low-level entities in ascending order. For example, if we want a movement of form ABACA, we could specify that the movement is made up of Subjects with indices in the Subject collection 0, 1, 2, 3, 4 and 5; then, we generate Subjects in such a way that identical Subjects occupy positions indexed with 0, 2 and 4. Suppose next that we want a movement with form AAABB; we can achieve this by not modifying the movement map, but by simply modifying the Subject generation process such that Subjects corresponding to indices 0, 1, 2 and Subjects corresponding to indices 3, 4 are respectively identical. Random map generation simply consist of mapping low-level entities at random. Partitioned map generation partitions all entities into a fixed number of partitions (partitionCount), and allows high-level entities only to be made up of low-level entities that belong to its own partition. For example, if we have 16 Motifs and four Phrases, and if we partitionCount of 4, then only Motifs 0,1,2,3 will be used to generate Phrases 0, Motifs 4,5,6,7 for Phrase 1 etc. This allows us to control the style of the high-level entity by controlling the styles of low-level entities. These are implemented in OrderedMapGenerator, RandomMapGenerator and

PartitionedMapGenerator respectively.

6.4.5 Constant Content Generation and Cyclic Content Generation ConstantContentGenerator implements the trivial function of generating constant entities. For example, if an entire piece of music is to be played by one and only one instrument, the Timbre is constant. This sort of scenario is implemented here. Cyclic content generation is slightly less trivial than constant content generation. If a piece of music contains parts for several instruments (hence has several Timbres), content for each instrument will

Implementation - AMCTIES.DefaultGenerators - Tuning the Operators

54

be generated one instrument at a time. The user requires that the Timbre is held constant for the time during which content is generated for that instrument, and then changed to the Timbre of the next instrument. This functionality is implemented in class CyclicContentGenerator.

6.4.6 Simplistic Content Generation In some cases, generating a high-level entity simply consists of “putting together” of low-level entities according the map (without further processing). This is implemented in SimplisticContentGenerator.

6.5 Tuning the Operators

6.5.1 Population Size, Tournament Size and Number of Generations in Genetic Algorithms In theory, a large population combined with a large tournament size and a large number of generations would result in reaching a solution. However, this would obviously result in an efficiency bottleneck. The technique we used to find the optimal parameters for population size, tournament size and generation count, was to start with an algorithm with large enough parameter values to give good results, and then reduce the parameter sizes while making sure that the standard of the results was not diminished. The initial parameters used were 256, 256 and 32 respectively for Population Size, Tournament Size and Number of Generations. Relatively small population and tournament sizes is justified, since the variance among organisms is small. For this same reason, the population can be expected to converge relatively quickly, hence the algorithm need not be run for too many generations. If the population does not converge relatively quickly, it is usually because a good solution does not exist; hence, increasing the population in the hope of broadening the search space is not going to be successful. A range of pieces with varying styles and durations were tested, and the optimal rates proved to be similar. To demonstrate the tuning process, we described results from a “Canon” generated by AMCTIES approximately 48 seconds long. The processing time is the average time over several runs of the algorithm. Starting with the initial parameters, we reduced the population size as 256, 128, 64, 32, 16. The results were as follows. This showed that the optimal population size was around 32. Note: “C.O.V.” denotes “Coefficient of variance”. Population size (Number of Organisms) 256 128 64 32 16 Mean Processing time (In Seconds) 33.60 22.59 17.09 12.60 10.62 C.O.V. for Processing time 2.47% 1.21% 6.28% 3.73% 6.12% Acceptable (Quality of result) Yes Yes Yes Yes No

��" � � �� 7 ��

Next, while holding the population size at 32, the tournament size was reduced as 256, 128, 64, 32, 16. The optimal tournament size was also around 32.

Implementation - AMCTIES.DefaultGenerators - Tuning the Operators

55

Tournament size 256 128 64 32 16 Mean Processing time (In Seconds) 12.60 7.84 5.55 4.21 2.16 C.O.V. for Processing time 3.73% 5.51% 6.53% 2.47% 2.46% Acceptable (Quality of result) Yes Yes Yes Yes No

��$ � � �� 7 ��

Finally reducing the generation count to 16 adversely affected the quality of the results. So we held it at 32. Hence, the selected parameter values for Population Size, Tournament Size and Number of Generations were 32, 32 and 32 respectively.

6.5.2 Mutation Rates and Crossover Rates in Genetic Algorithms Unlike conventional genetic algorithms (that usually use a random initial population), here we initialize the population with homogenous organisms. Hence, we need to use much higher mutation and crossover rates than conventional genetic algorithms. A process of trial and error showed that 0.5 for the crossover probability and 0.5 for each of the four mutation operators (ScaleTranspose, ReversePitches, Invert and Mutate operators) is good. Since the population space is relatively small, there was no need to reduce the rates as the population evolved.

6.5.3 Tuning the fractals and cellular automata The only variables that might need potential tuning are the stochastic functions that define the behaviour of the emergent systems. As mentioned before, we have the choice of either initializing these at random, or else explicitly specifying them. Random initializations seems to result in more “creative” sounding results, as compared with explicit specification. Hence, the former technique has been used in the case of cellular automata and fractals.

Implementing Support Modules - AMCTIES.MusicTheorySupport

56

Chapter 7: Implementing Support Modules 7.1 AMCTIES.MusicTheorySupport Elements of Music Theory required for the design, but which are independent of TIES, are implemented under package AMCTIES.MusicTheorySupport. This consists of fundamental musical entities (Classes Note, NoteSequence and Scale), classes that implement methods for performing various musical operations on musical entities (Class Modifier) and classes that implement methods that perform various musical tests on musical entities (Class Test).

7.1.1 Class Note and supporting classes Class Note represents a musical Note. It has four attributes: pitch, duration, Timbre and Dynamic.

public class Note { private int pitch; ... private int duration; ... private int Timbre; ... private int Dynamic; ... }

The values of pitch correspond to those defined in the MIDI standard (A value in 0, 1…127). Some constants and methods pertaining to pitch are defined in support class Pitch. public class Pitch { public const int C = 0; public const int CSharp = 1; ...

public const int B = 11; public const int C0 = 0; ... public const int C11 = 132; public static int GetInterval(int pitch1, int pitch2)... public static int GetNoteLabel(int pitch)... }

The values of duration also correspond to the MIDI standard (A value in 0, 1…127). The support class Duration is analogous to Pitch. Duration is measured in MIDI ticks. The duration of tick can be preset to a given number of Milliseconds.

public class Duration { public const int Crotchet = 120; public const int Quaver = 60; public const int SemiQuaver = 30;


57

public const int DemiSemiQuaver = 15; public const int Minim = 240; public const int SemiBreve = 480; public const int Breve = 960; }

The values of Timbre are mapped to one of the 16 MIDI channels. The values of Dynamic also correspond to the MIDI standard (A value in 0, 1…127).

7.1.2 Class NoteSequence Class NoteSequence represents a linear sequence of Note objects. The Notes are stored in a System.Collections.SortedList (Class SortedList represents a collection of key-and-value pairs that are sorted by the keys and are accessible by key and by index). The object Note forms the “value” and its position in the NoteSequence (the number of ticks elapsed from the beginning of the NoteSequence at the point where the Note appears) is the “key”.

public class NoteSequence {

private SortedList notes; }

The class has several more methods and properties that manipulate the content of the NoteSequence. These are as follows: Method interface Description public NoteSequence()

Constructor

public NoteSequence(NoteSequence noteSequence) "Copy" constructor public int Length Length of the sequence public int Duration Total duration of the sequence public int StartNotePosition Position of the starting note of

the sequence public int EndNotePosition Position of the ending note of

the sequence public Note StartNote The starting note of the

sequence public Note EndNote The ending note of the sequence public ICollection NotePositions All the note positions public ICollection Notes All the notes public int GetNotePosition(int index) Gets the position of a note with

the given index public void AddNote(Note note, int notePosition) Adds a note to a given position public static NoteSequence operator+(NoteSequence noteSequenceA, NoteSequence noteSequenceB)

Concatenates two note sequences

��) � � �� (�Class�NoteSequence�


58

7.1.3 Class Scale Class Scale represents the musical entity scale. The note pattern in the scale is represented by integer “tonic” that stores the pitch of the lowest (defining) note of the scale, and the integer array “pattern” that stores the pitches of the other scale notes relative to the “tonic”.

public class Scale {

private int tonic; ... private int[] pattern;

}

Class Scale represents a generic scale. For specific scales are represented in other classes. For example: public class TriScale: Scale public class ChromaticScale: Scale public class PerfectScale: Scale public class MajorScale: Scale public class MinorScale: Scale public class PentatonicScale: Scale public class BluesScale: Scale

...

7.1.4 Class Test Class Tests has methods that perform some musical tests.

Method interface Description public static bool MatchChord(NoteSequence noteSequence, int[]Chord)

Checks if the notes in a given NoteSequence match a given Chord

public static double MatchChordProb(NoteSequence noteSequence, int[]Chord)

Returns the 'confidence' with which a given NoteSequence might match a given Chord.

��, � � �� (�Class��


59

7.1.5 Class Modifier Class Modifier implements methods that are used to modify NoteSequences.

Method interface Description public static NoteSequence ScaleTranspose (NoteSequence noteSequence, Scale scale, int steps)

Transposes a NoteSequence along a scale by a given number of steps

�

public static NoteSequence NormalizeRange (NoteSequence noteSequence, Scale scale)

Normalizes the pitches of a NoteSequence so that they all fit within a certain range of a given scale

�

public static NoteSequence ReversePitches (NoteSequence noteSequence)

Reverses the pitches of a given NoteSequence

�

public static NoteSequence Invert (NoteSequence noteSequence, Scale scale)

Inverts the Pitches of a given NoteSequence with respect to

�

public static NoteSequence Mutate(NoteSequence noteSequence, Scale scale)

Randomly mutate the pitches of a NoteSequence

�

public static NoteSequence ModifyToChord (NoteSequence noteSequence, Scale scale, int[] Chord, int last_pitch)

Modifies a NoteSequence so that its pitches satisfy a given Chord. Since this is used to combine Motifs to form Phrases, an additional constraint forces the starting note of the NoteSequence to have a pitch neighbouring a given pitch 'last pitch'.

��- � � �� (�Class�Modifier�

Implementing Support Modules - AMCTIES.MathSupport

60

7.2 AMCTIES.MathSupport AMCTIES.MathSupport implements a set of mathematical functions that are used in AMCTIES. These functions are described below. It consists of a single class NMath. Method interface Description public static int RandomInt(int i)

For a given i, return a random integer x, s.t. x is in [0,x[

public static bool Prob(double p)

Return true with probability p.

public static int[] RandomIntArray(int length, int range, int offset)

Return an int[] of a given length, with integers in [offset, offset+range[ generate at random.

public static int[] OrderedIntArray(int length, int range)

Return an int[] of a given length, where the value of the (index)th term is index%range.

public static int PartitionedInt(int total, int i, int iTotal, int partitionCount, int index)

Partitions "iTotal" and "total" into "partitionCount" partitions (Partitioning consist of splitting into equal parts), and returns "index"th value from the jth partition in “total”, where "i" belongs to the jth partition in "iTotal". The partition is treated as a cyclic array, hence there is no overflow in taking the "index"th value.

public static double[] Normalize(int[]data)

Normalizes integers in a given int[] data, so that their sum is unity.

public static byte[] int2variable(int x)

Convert a given integer x, to a byte array, allowing the byte array to have a variable size depending on x, as defined in the MIDI file format.

public static int RandomIntZ(int i)

For a given i, return a random integer x, s.t. x is in [-i,i]

public static int[] RandomIntArray(int length, int range)

Return an int[] of a given length, with integers in [0,range[ generate at random.

public static int[] ConstantIntArray(int length)

Return an int[] of a given length consisting of zeros

public static int[] PartitionedIntArray(int length, int range, int partitionCount)

Return an int[] of a given "length", where its elements are in "partitionCount" taking values from the "partitionCount"th partition of "range". Partitioning consist of splitting into equal intervals. This uses method PartitionedIntArray.

public static int PartitionedRandomInt(int total, int i, int iTotal, int partitionCount)

Partitions "iTotal" and "total" into "partitionCount" partitions (Partitioning consist of splitting into equal intervals), and returns a random value from the jth partition in “total”, where "i" belongs to the jth partition in “iTotal".

public static byte[] int2bytes(int x)

Convert a given integer x, to a byte array

Implementing Support Modules - AMCTIES.MIDISupport

61

7.3 AMCTIES.MIDISupport Package MIDISupport has some classes that implement certain MIDI related functions. Details about the MIDI format have not been included. For details refer [MIDI].

7.3.1 Class MIDIFileWriter Class MIDIFileWriter implements methods that will enable the writing of MIDI files. These are summarized below.

Method interface Description public static void write_Header(FileStream fs). Writes a MIDI header to a given file

stream public static void write_TrackHeader(FileStream fs,int length)...

Writes a MIDI track header to a given file stream

public static void write_NoteOn(FileStream fs,int delta,byte channel,byte pitch,byte velocity)...

Writes a "NoteOn" event to a given file stream

public static void write_NoteOff(FileStream fs,int delta,byte channel,byte pitch,byte velocity)...

Writes a "NoteOff" event to a given file stream

public static void write_RunningStatus(FileStream fs,byte status)...

Writes running status to a given file stream

public static void write_end(FileStream fs)... Writes a "end of MIDI file" to a given file stream

public static void write_Tempo(FileStream fs, int temp)...

Writes a default MIDI Tempo to a given file stream

public static void write_KeySignature(FileStream fs)... Writes a default MIDI Key Signature to a given file stream

public static void write_Channel(FileStream fs,byte channel,byte instrument)...

Writes the setting of a given instrument to a given channel, to a given file stream

��. � � �� (� ��MIDIFileWriter�

7.3.2 Class MIDISong Class Song represents a single MIDI song (or file). Method AddNote adds a note with given pitch, velocity (Dynamic), start time and duration, to a given channel.

public void AddNote(int channel,int pitch,int velocity,int start,int duration)

Method WriteToFile serializes an instance of class Song as a standard MIDI file.

public void WriteToFile(string fileName)

7.3.3 Class MIDISupport Class MIDISupport acts as an intermediary for AMCTIES to interact with MIDI. Class MIDISong and Class MIDIFileWriter have been designed to be "standalone" from AMCTIES, and could be used by other applications. Conversely, AMCTIES might be used with other class libraries that implements MIDI.

Method NoteSequenceToMIDIFile serializes an instance of class NoteSequence as a standard MIDI file.

Implementing Support Modules - AMCTIES.MIDISupport

62

public static void NoteSequenceToMIDIFile( string fileName, NoteSequence[] noteSequences, byte[]channels, int[]pitchOffsetPerChannel, int keyOffset, int tempo) { ... }

Parameter filename is the name of the midi file; noteSequences are the NoteSequences that must be written; channels maps MIDI patches to each of the 16 MIDI channels; tempo is the speed of MIDI playback; finally pitchOffsetPerChannel and keyoffset are the key offsets of each channel and the global key offset respectively.

Testing, Evaluation and Validation - The Plan

63

Chapter 8: Testing, Evaluation and Validation 8.1 The Plan Testing, Evaluation and Validation of the system was carried out in four stages.

• First, the implemented deliverables were tested (Code Testing). • Next, musical output generated by the system was evaluated by experts in the music

profession (Expert Evaluation of Output). • After that, the same output was compared with other musical compositions (Comparing

output with other compositions). • Finally, the system’s requirements (as specified in Chapter 3) were validated.

The content of the first three stages are described in detail, next.

8.1.1 Code Testing Since this project is not a conventional software development project, a conventional testing procedure need not be followed. Most of the implemented code follows directly from the design. Hence, the likelihood of semantic and logical errors is low. Almost all syntax can be directly detected by the development IDE (Microsoft Visual Studio .NET 2003). Nevertheless, we will be performing the below described tests to verify that the components can be integrated, functionality is met and non-functional requirements are satisfied. First, the following white-box tests will be performed.

• Test control structures and paths for correct implementation • Test the interfaces of functions • Test the dependencies between packages

Second, these black-box tests will be performed.

• Test each generator for functionality (to make sure it produces the required output) • Test the performance of the algorithms (principally speed) • Test the input options that the user can provide

8.1.2 Expert Evaluation of Output The fact of whether or not AMCTIES has the ability to produce novel and imaginative melodies and Rhythms is subjective. So is the issue of its ability to come up with new ways of using musical material. Hence, these “inspiration characteristics” can only be evaluated indirectly through the human reaction to music produced. Even “perspiration characteristics” like the ability to accurately adhere to musical rules can be (accurately) evaluated by a human expert. A few characteristics can be evaluated without human opinion. However, from a musical point of view, these are not very important (For example, how fast the output is produced). They are none-the-less important from a computing point of view. Different validation criteria have been used for various aspects of the musical piece. These have been broken down into three areas: local structural features (Motifs, Rhythms, Chords, Dynamics and Timbre), global structural feature (Phrases, Subjects and the Movement as a whole) and stylistic features (historical relevance, expressiveness, creativity etc.)


64

As stated in this project’s proposal, the results were to be validated by experts in the music field. Although we had the relevant criteria it was difficult to devise a specific mode of validating these criteria. We did have several options, however each had its own problems. The options were as follows:

• For each of the criteria we could get the expert to assign a score (say from 0 to 100). This is the established practice in music examinations. However, we have several problems in implementing the same strategy here. First, established music examinations have a historical precedence. If some piece is assigned a score of 50, we known that it will be (approximately) comparable to other pieces with similar scores. In our case we do not have a precedent; we do not know how to interpret the results in an absolute manner. Secondly, we need to evaluate several criteria not evaluated in usual music examinations (For example, local structural features). Thirdly, in music examinations the score is accompanied by the examiner’s written comments. This carries as much importance as the score. It is difficult to quantify comments, and difficult to individually analyze.

• Instead of assigning a score, we could ask the expert to give a more discrete opinion such as

“Very good”, “Good”, “Average” etc.. This is a sort of simplification of scoring, but is easier to analyze. However, here too it is difficult to make absolute comparisons.

• We could go a step further and simplify the discrete opinion to a simple yes/no question. We

could Phrase the question on the lines of “Do you think whether such and such an aspect of musical composition has been successfully achieved at a level that is comparable to a human musician with a fairly advance musical talent and education?” This technique has the obvious advantage of being very simple to analyze. It also takes some of the subjectivity out of the picture. For example, the expert cannot say that piece A is better than piece B. All he can say is whether or not piece A or piece B is of a certain standard.

None of the strategies are ideal. However, it is noteworthy that the third option has some reasonable advantages. Interestingly, it also follows along the lines of the Turing Test, the difference being the human generated version that is used for the “comparison” resides in the expert’s mind. Also, the simplicity of the test by no means makes it less valid. The downside of using the first two options is that the significance of the test is reduced. For example, if piece A gets a score of 70 and piece B gets a score of 65, can we say (with reasonable confidence) that piece A is better than piece B? Hence it was decided to use the third option. The tests consisted of a series of yes/no questions asking for a comparison between the output produced by AMCTIES and “a human musician with a fairly advance musical talent and education”. The questions were adapted from a set of guidelines given to examiners marking examination questions in music composition.

Validating local structural features Motifs Was the level of variety within each Motif sufficient?

Was there sufficient effect and colour? Were the Motifs interesting to the listener?

Rhythm Did the piece have sufficient Rhythmic variety? Were the Rhythmic patterns, accentuation etc. regular and coherent? Was there sufficient independence of Rhythm (rubato, etc.)?


65

Chords Were standard rules of harmony followed with respect to the Chords in the piece? Was there a sense of coherence between consecutive Chords? Did the harmony in general follow some order or system?

Timbre Were the selected Timbres appropriate for the piece? Was the amount of variation in Timbre suitable?

Dynamics Is there sufficient Dynamic variation? Do the Dynamics compliment the style of the piece? Was there sufficient drama and climaxes in the music?

Validating global structural features

Phrases

Is there a balance and symmetry between the materials that have gone into Phrase construction? Does the structuring of the Subjects have form and balance? Was the Motific development appropriate?

Subjects

Is there sufficient variety within each Subject? Is there sufficient variety among different Subjects?

Complete Movement

Is the movement coherent as a whole? How would the compositional effort match that of a human composer? Does the movement exhibit sufficient stylistic variety? Is the music expressive? Can the music hold the listeners attention?

Validating stylistic features

Historical relevance

Is the content of the piece relevant to some historical period of time?

Expressiveness

Does the piece express some idea (unknown as it might be) to a level a human composer might be capable of?

Creativity

Does the piece of music have creativity? Are the general musical devices sufficiently novel?

8.1.3 Comparing output with other compositions

Can we use a Turing Test like test? The Turing Test was originally proposed by Alan Turing in 1950, to test a machine's ability to carry out conversations in a human-like manner. The test follows some very simple steps [WIKIPEDIA]:


66

• A human adjudicator participates in a natural language dialogue with two other entities: a human and a machine.

• If the human adjudicator cannot reliably tell which is which, then the machine is said to pass the test.

Our goal is to “test a machine's ability to compose music in a human-like manner”. However, we cannot directly use the Turing Test for Several reasons. Firstly, we need to find a human capable of composing in a wide range of styles; this is difficult (human composers usually specialize in relatively narrow areas). Secondly, the human must compose in real-time and at will to order; this is near impossible. We might get around this problem by outputting music already composed by humans as the “human product”. This results in a different problem. Suppose we create a machine capable of generating plays similar to those produced by Shakespeare. Now, suppose we do a Turing Test using the plays of Shakespeare as the “human product”. However, the human adjudicator might have at least a superficial knowledge of the actual Shakespeare plays; hence, his opinion will be biased. The same problem will be encountered if we use existing works of music to test AMCTIES. Hence, it appears that it is not possible to use the Turing Test directly to validate AMCTIES output.

Alternative strategy A way out of the above problem might be to use human compositions that are not well known. This somewhat goes against the ideal thinking behind the Turing Test. However, there seems to be no other practical way out. An extreme alternative might be to use a human adjudicator with no previous musical knowledge at all; but this is totally unacceptable. Another problem in using human compositions is that they need to be in MIDI format (Since AMCTIES output are in MIDI, this is essential for a fair comparison). MIDI versions do not have the quality of recordings on real musical instruments. On the other hand, this “distortion” might be considered an advantage, as it will hinder the human adjudicator as recognizing it (from previous experience) as a human composition. This strategy, despite its drawbacks, does have one good point. We can use the same strategy to compare output produced by AMCTIES with output produced by other AMC systems.

Carrying out the test The actual test can be carried out along the lines of the Turing Test. We have a human adjudicator. We put AMCTIES generated results into one set and material generated by other AMC systems (or human compositions that are not well known) into the other set. A set is selected at random and a random item of the selected set is presented to the human. Based on the results we could specify a reasonable confidence level at which we could say how similar or dissimilar the results are.

Testing, Evaluation and Validation - Summary of Code Testing

67

8.2 Summary of Code Testing

8.2.1 Summary of the test process Before embarking on the actual code testing, the full source code was thoroughly inspected for errors. A few errors were uncovered. The inspection also provided the opportunity to validate coding conventions and comments etc. The white box testing process was based on the design of the code implementation. Path testing, control structure testing was done while running the software in debug mode. To enable testing specific sets of input data, simple test drivers were implemented. Dependencies between packages were tested in a similar manner. Black-box testing was done incrementally in a bottom-up manner, supported by drivers.

8.2.2 Summary of the test results As expected the white-box testing uncovered no serious errors or bugs. Control structures and paths were free of error. Interfaces between functions were error-free. There were no errors emerging from dependencies between packages. The generators performed according to specification. Similarly, the input options resulted in the expected output results. The speed and efficiency of the system was good.

8.3 Summary of Expert Evaluation of Output

8.3.1 Summary of the process The test cases consisted of a collection of varied pieces of music generated by TIES. Various parameters were given to generate varied pieces (e.g. pieces in different styles). However, after generation no selection or pre-judging was done on the test cases. Hence, all the test cases were as AMCTIES produced them and free from any tampering. Also, there was no attempt to improve on the pieces by other means (for example, by editing them on a MIDI editor) after they were generated.

The test set consisted of 46 samples, each around 30 seconds to 5 minutes long. The upper limit was due to practical constraints (AMCTIES is capable of generating arbitrarily long pieces of music). An expert session consisted of the expert validating 30 randomly selected test samples (out of the total 46). A session typically lasted around 40 minutes, with 30 minutes for listening and 10 minutes for taking down responses. The “experts” were professionals in the music field with training and professional qualifications in both performance and teaching.

8.3.2 Summary of the results As mentioned, human evaluation of music is highly subjective. Hence, I have tried to disconnect the expert comments from any personnel niceties and tried to concentrate on the purely musical details.

Testing, Evaluation and Validation - Summary of Comparing output with other compositions

68

Motifs The experts agreed that there was considerable variety in the range and style of Motifs. There was a range of colours and effect. The Motifs were very interesting, seldom dull.

Rhythms As with the Motifs, there was considerable variety in Rhythm. The accentuation was regular and coherent. Some Rhythms were sometimes indeed unorthodox, but this often added to the novelty of the music.

Chords Most of the harmony was regular and correct. However, there were certain unorthodox Chords that seem to give the impression of “wrong notes”. However, even the “wrong notes” seemed to be put there for “right reasons”, in that they seemed to form a coherent whole.

Dynamics and Timbre The actual emergent techniques used in AMCTIES did not attempt at controlling Dynamics and Timbre, actively, since it is essentially a music composing application, not a sound generating one. The Dynamics and Timbre were essentially added artificially. However, it was essential that the other musical entities (Motifs, Rhythms, etc.) complement these. The general expert opinion was that this was the case. They also provided variety and effect.

Phrases and Subjects The structure and arrangement of Phrases and Subjects enable satisfactory Motif development. The developmental lines were sometime blurred. However, this was recognized only on close scrutiny of the score; as far as listening was concerned, the form and structure was good.

Movements as a whole The complete musical works produced could be compared to a human composer with a fair composing ability and in certain sections “very clever” devises were used. The music was ideal at creating atmosphere and emotion specific sounds. The flow of some of the music was very soothing. The system proved capable of generating music in several contrasting styles. The music was generally expressive and was sufficiently interesting enough to hold the listener’s attention.

8.4 Summary of Comparing output with other compositions

8.4.1 Summary of the process The AMCTIES test set consisted of the 46 samples described earlier. The “other set” consisted of 74 samples of music which consisted of samples generated by other AMC systems and MIDI versions by some not too well known compositions by human composers. Random samples were picked from each set (with each set having an equal probability of being selected). A programme consisting of these samples were presented independently to experts.

8.4.2 Summary of the results As in the previous case, comparisons by a human adjudicator are subjective. For clarity, the test results have been summarised below.

Motifs There was no considerable difference between the output of AMCTIES and other systems (in general) in this respect. However, AMCTIES did have considerably more variety than certain AMC

Testing, Evaluation and Validation - Summary of Validating Functional Requirements

69

systems (especially those based on fractals). These other systems produced output that tended to be too repetitive, while AMCTIES reached a better balance between repetition and variety.

Rhythm Similar observations (to Motifs) can be made here. However, again, in general AMCTIES produced more controlled output.

Chords Most other systems completely lacked harmony. Most had totally random harmony (that is with no actually conscious attention to harmonization at all). Others had one or two Chords that repeated in a highly monotonous manner. Those that had few parallel parts sounded better than the others for this reason. AMCTIES was far superior. It had instances of both conventional and unconventional harmony. The important point in both these cases was that AMCTIES produced coherent Chords.

Timbre and Dynamics AMCTIES was similar in this aspect to most other applications. Not many deviations were observed. Also, since Timbre and Dynamics can easily be added artificially, making comparisons on these grounds might not be valid or fair.

Phrases and Subjects Most other applications were simple in form and structure. Conversely, AMCTIES is capable of elaborate forms and structures. As the expert evaluation revealed earlier, large forms can be achieved with coherence.

Movements as a whole In general it is evident that AMCTIES is capable of producing a very large range of musical styles. In contrast, most other systems are restricted in what they can produce. Though these systems might produce a limited range of styles well, they are incapable of producing other styles. Hence, AMCTIES has the functionality of several other systems combined.

8.5 Summary of Validating Functional Requirements

8.5.1 Application Requirements

Requirements on the type of music As was apparent from the expert validation, AMCTIES is capable of generating tonal music. However, it was found that AMCTIES is also capable of generating other types of music (notably dance music). It was also apparent that AMCTIES was capable of generating music in the classical tradition. However, again it was noted that other types of music was also possible. Among these was music mimicking Chinese and other oriental styles. AMCTIES performed very well at melodic composition as was required. In addition several harmonization techniques were also tried out and these too performed well. In general, it is fair to conclude that AMCTIES has performed beyond expectation with respect to requirements pertaining to the type of music it should produce.

Requirements on Basic Deliverables AMCTIES was implemented as a set of libraries that implement a music composition system. The requirements set out for each of the incremental stages have been met.

Testing, Evaluation and Validation - Summary of Validating Non-functional Requirements

70

Requirements on Output Format The output format was MIDI. This can be easily translated into standard western music notation as required. It can also be played back on standard music software. An additional by-product of this project is a library that can be used for writing MIDI files.

Requirements on User Involvements Involving the user, directly during the composing process was considered somewhat a bottleneck (for example, in validating genetic algorithm fitness). Hence, maximal user involvement was achieved by allowing the user as many initial options as possible. The results generated by the system indicate that this strategy has been successful.

8.5.2 Framework Requirements and Technique Utilization Requirements As we saw, TIES is an excellent framework for implementing an AMC application. It is component based and most of these are reusable. Several emergent techniques have been used and their use justified.

8.6 Summary of Validating Non-functional Requirements

8.6.1 Product Requirements The final deliverable is a set of class libraries. The source has been well commented and documented. These libraries can be robustly and reliably integrated with other applications, either as a processing backend or as support libraries. AMCTIES is efficient enough for real-time music composition.

8.6.2 Organizational Requirements The final deliverable, together with its associated documentation has been delivered on time. Standard coding and documentation best practices have been followed.

8.6.3 External Requirements All external material used for this project have been so used without violating any legal or ethical laws. All previous work referenced have been cited.

General Discussion - On the Properties of TIES

71

Chapter 9: General Discussion 9.1 On the Properties of TIES

9.1.1 Why this approach is better than other attempts The comparison of AMCTIES with other methods shows that it is generally superior to other techniques. Although TIES might have used a specific technique A to generate musical entity B, the design philosophy was to try and find a way of generating entity B using some arbitrary technique. The design philosophy was to find some way of generating B. Hence, given B, an attempt was made to find a suitable A. Most other attempts try to do the opposite (for example, a typical attempt might be titled “Rhythm generation using fractals”). TIES is independent of any technique. Although it might contain a generator that generates Rhythms using fractals, it cannot be referred to as a “Fractal based technique” or a “Cellular automata based technique” in general.

9.1.2 Interaction Entities Part of the power of emergent systems is derived from interactions between high-level and low-level entities. In TIES this is apparent in both directions: High-level entities influence low-level entities and also vice versa. The manner in which low-level entities (Motifs, Rhythms, Chords, Dynamics and Timbre) influence high-level entities in obvious from the fact that high-level entities are derived from combining low- level entities. The way high-level entities influence low-level entities is just as potent, but less apparent. The combination process involved in deriving high-level entities from low-level entities is not indiscriminate. An intermediate process “selects” the appropriate low-level entities, some times modifying them if necessary. For example, in generating a Phrase (in this case the high-level entity), notes and Chords (low-level entities) are selected by a Genetic Algorithm such that the notes match the Chords. Hence, the high-level entity “decides” on what low-level entities it is to be made up of. These simultaneous “top-down” and “bottom-up” interactions between entities combine to give TIES some very positive features.

9.1.3 Hybrid Approach TIES provides a framework for building a system consisting of interacting entities. TIES utilizes “hybrid-ness” in two ways. Firstly, each specific type of musical entity is generated by a generator entity that is optimized for the purpose. The work is delegated to specialized agents. Secondly, a single type of musical entity (for example, a Rhythm) might have several generator entities capable of generating it (for example, FractalRhythmGenerator, BeatRhythmGenerator etc. for Rhythm). The generation of the entity might thus be divided among several generation entities. Hence, we have a further level of “hybridization”.

General Discussion - Analogies between TIES and human music composition

72

9.1.4 Using emergent methods and conventional methods side by side As would have been noticed, AMCTIES utilizes several techniques that are not “emergent”. The use of conventional methods is essential for several reasons. Nearly, all the low-level entities (Motifs, Rhythms, etc.) have been generated using emergent techniques. Emergent techniques (For example, Genetic Algorithms) have been used in combining these to form the next level of entities (namely, Phrases). However, these methods have been coupled by some non-emergent methods. This is essentially to prevent chaos (“too much” creativity can be said to equate to chaos). At the highest level of entities (namely, Subjects and Movements), the nature of the entity is largely dictated by high-level human concepts like form and architecture. Hence it is difficult to directly apply emergent techniques. It should be noted that the term “emergence” is relative. It is also a manner of explaining (or describing) the behaviour of complex systems and it is difficult to say that “that is what exactly happens within a complex system”.

9.1.5 Why AMCTIES is not a mere “Music Synthesizer” It is possible to mimic all the functionality if a standard music synthesizer on AMCTIES. However, the converse is not true. Standard music synthesizers are not capable of producing the more complex and higher level musical forms. Even in the case of musical output that has been constrained (for example, Phrase with a fixed Rhythmic pattern and Chords), AMCTIES exhibits far more creativity than a standard music synthesizer would have. The latter would have a fixed set of Phrases hard-coded, while AMCTIES would generate a novel Phrase each time.

9.2 Analogies between TIES and human music composition Very importantly many aspects of the TIES process can be said to mimic the human compositional process. Earlier we discussed the human music composing process along with several “types” of composers and various related issues. Let us now see how TIES corresponds to these.

9.2.1 TIES as a Spontaneous composer If we consider TIES as a black-box (without paying special attention to its internal workings), it is very much like the spontaneous composer. TIES can be said to be creatively inspired spontaneously. It generates complete works of music, without any indication of intermediate results. The entire creative process can be said to be hidden inside TIES.

9.2.2 TIES as a Constructive composer If we look at TIES from a white-box point of view, what we see is that TIES is an almost exact representation of the constructive composer. As mentioned before, constructive composers are also creatively “inspired”; however, such composers use the results of inspiration in a more conservative manner. Similarly, TIES generates “creatively inspired” low-level entities and combined these to form higher level entities in a more conservative manner. Like constructive composers (who are not inspired with entire works of music), TIES does not use pure emergence to produce the entire piece music. It uses such techniques only for lower level entities. Just as composers note down the ideas, TIES will store low-level in temporary store until they are required. The composer’s notebook is a logistic requirement and not a direct part of the

General Discussion - Some Philosophical Questions

73

composing process; so is temporary memory storage for TIES. Just as a composer might follow standard accepted musical forms, it is possible to incorporate these into TIES. This process of construction often undergoes many revisions. TIES will use techniques such as genetic algorithm (that in effect try out a large number of organisms until a suitable one is reached) for this revision process. The entire TIES process follows the “Bottom-up” process followed by the constructive composer.

9.2.3 TIES as a Traditionalist composer “Traditionalist” aspects of musical composition can be easily incorporated into TIES. This can be done binding generation techniques to constraints. For example, if the music required must follow a certain given Rhythm pattern or overall form, this can be incorporated. As mentioned earlier, the Traditionalist composer follows a “Top-down” approach. The same effect can be said to manifest in the “Backward Causal Power” found in emergent systems.

9.2.4 TIES as a Revolutionary composer It is difficult to compare a computer system to a human-being with “revolutionary ideas”. Indeed, at a high level, TIES does not implement any “revolutionary” techniques or devices. However, it does use some techniques that might result in “un-conventional” music. A good example is mutation in Genetic Algorithms.

9.2.5 TIES and human-style impediments In conventional music composition systems there can be some fixation since there is little flexibility in technique. However, TIES is far more flexible and error tolerant, resulting in little or no fixation. Unlike, humans TIES has no “learned memory”. This might be considered a disadvantage in certain cases. However this also has the advantage that as a result that TIES will not suffer from “learned helplessness”. Finally, the problem of motivation does not arise since the functionality of the computer is not emotion dependent.

9.3 Some Philosophical Questions

9.3.1 Serendipitous Randomness Einstein once said, “I am convinced the Old One (that is, God) does not play dice”. This reflects the conventional school of thought that chaos (and randomness) is bad. This thinking has been challenged by some modern discoveries. Many emergent phenomena ranging from geographical landscape features to variations in global economic trends are based on interrelationships founded on randomness. In the best-seller A New Kind of Science, Wolfram says that randomness evolves order and order evolves randomness. He demonstrates this using Cellular Automata. As in the case of Einstein’s dice, what we see here is a complex relationship between local randomness and global determinism. The outcome of the dice is random; however, over time the number of outcomes would converge to values determined by the way the dice is loaded. In the case of AMCTIES, the nature of low-level entities is ad hoc and random. However, complete musical movements have a global determinism (even though this might not have been predicted from the local randomness).


74

Emergence is ad hoc. There is no “rational” logical process that can explain results that are generated by an emergent system. So why should an emergent system produce good results? Philosophers and theologists have described this phenomenon as “Serendipitous creativity”. Many theistic religions (Including Hinduism, Islam and Christianity) regard this as the “manifestation of God” [CARR, 2004].

9.3.2 Competition and Cooperation - The fundamental forces in nature Two interesting phenomena related to many systems (particularly natural systems) are the forces of competition and cooperation. In emergent techniques such as cellular automata, fractals and genetic algorithms, we see both forces acting simultaneously. The actual human composition process can also be said to be influenced by cooperative and competitive forces. Hence, it is interesting to analyze these.

Genetic Algorithms Genetic algorithms can be said to be a technique that is founded on competition between organisms. In the end, the outcome of the genetic algorithm is the organism (in the GA population) with the highest fitness. It can said to have competed with all other organisms and won. GAs have three principal operators: Selection, Crossover and Mutation. Of these, competition is embodied strongest in selection. Selection ensures that the weaker organisms are removed from the population while fitter ones remain. Note, however, that selection can create competition only as long as there is scope for competition in the population. For example, if the population is homogenous (that is, all the organisms have roughly the same fitness levels), then selection will have little effect on the population. This is where crossover and mutation come in. Mutation naturally creates diversity in the population. Hence, it can be said to make certain individuals fitter and others weaker. This added diversity results in added competition. The role of mutation in competition is indirect; it is selection that actually facilitates the competition. However, without the diversity created by mutation, the competitive effect of selection would be much less (or even zilch). Some have argued that selection (an operator that facilitates competition) and mutation (an operator that creates the diversity needed for efficient competition) alone might be sufficient for implementing genetic algorithms. This approach to genetic algorithms (that is, genetic algorithms without crossover) is known as “Naïve” evolution. However, this fails to consider the role of cooperation in genetic algorithms. Crossover actually facilitates both competition and cooperation. The act of crossover destroys existing patterns encoded in an organism, and replaces these with new ones. This (like mutation) creates diversity. Note, however, the ability of crossover to create this diversity is dependent on the population. Crossover cannot create diversity in a homogeneous population (for example, performing crossover on two identical organisms will result in offspring that are identical to the parents). The more important role of crossover is to facilitate cooperation. Crossover allows organisms to exchange genes. Hence (in theory at least) an organism can obtain genes that it needs, from another organism, in exchange for genes that the other needs. Some might argue that if an organism needs certain genes, it is possible that in the course of several mutations, the existing genes will mutate into the needed genes. This is true. However, crossover facilitates gene changes over several genes


75

simultaneously and might be considered more efficient. Also, crossover results in the putting together of gene sets that have been independently optimized over time by successive selection.

Cellular Automata and Fractals In the case of Cellular Automata, defining competition and cooperation is not as “clear cut” as in genetic algorithms. We might consider competition between individual cells for survival. However, in cellular automata, individual cells are low-level entities; it is the overall global state that actually “represents” the cellular automata. Hence, it is more relevant competition between states. What is meant by “competition between states”? A cellular automaton evolves through a series of states. After some time the cellular automaton might converge to some kind of behaviour that is restricted to certain states (Note, this is not strictly true for all cellular automata). Hence these states can be said to have successfully competed against all other states that failed to be expressed in the behaviour of the cellular automata. Ultimately, it is the evolution function of the cellular automata that defines what states will succeed. However, the emergent nature of cellular automata makes predicting this difficult (or impossible). However, observations indicate that competitive forces do exist. In the case of cooperation, analysis is even more difficult. Nevertheless, it too can be said to exist. In the case of fractals as well, we might hypothesize that the nature of the technique results in competitive and cooperative forces emerging. For example, various points in the fractal space can be said to cooperate to form some final shape. Simultaneously, we could say that these same points are interacting in a way to prevent certain shapes emerging, while promoting others; that is facilitating competition. In genetic algorithms, competition and cooperation is visible at a high level. However, in cellular automata and fractals, the manifestation is at a low level.

Competition and cooperation in the human music composing process The human music composing process (like any human thinking process) involves certain basic types of actions. Thinking should generate new ideas (or ideas that the human was not consciously aware of previously). Thinking should see potential connections between known ideas and combine them to form newer (and more complete) ideas. The human mind also generates a continuous stream of irrelevant (and sometimes meaningless) ideas as well. Hence, thinking should also learn to exclude these bad ideas and select only the good ideas. The putting together of ideas is essentially a cooperative process. Good ideas competing with bad ideas is a manifestation of competition. For both these, it is essential that there is a steady flow of diversity of new ideas (Good or bad). Hence, we see several parallels between the “competition and cooperation” observed in genetic algorithms, cellular automata, fractals, the human music composing process and AMCTIES. Initially we generate new ideas in the form of Motifs, Rhythms, etc.. We use essentially emergent systems here, hence, we might say that the system is not “consciously aware” of probable output previously. We “see” potential connections between known ideas (for example, via a GA fitness functions) and combine them to form newer ideas (for example, a Phrase). Obviously, fractals and cellular automata generate a lot of irrelevant and meaningless material. These are excluded either


76

explicitly by selection in Gas or implicitly by the structure of TIES or the customizations of the generation process (for example, specifying that all Motifs must be based on a certain musical scale). Combining musical entities is essentially a cooperative process. For example, in attempting to match Chord, Motifs might exchange notes with each other (this is facilitated by GA crossover). “Bad musical material” eventually loose out on competition. Competition and cooperation are not perfect processes. We see their drawbacks manifesting in TIES in several cases. For example, sometimes “bad organisms” are able to “cheat” competition (for example, due to some flaw in the GA fitness function). Hence, from time to time we see some “wrong notes” appearing in the output. In other cases, the mechanisms for cooperation might not be sufficient to allow a favourable manifestation of it. For example, two GA organisms might, between themselves, have all the genetic material to form an optimal organism. However, the positioning of the genes might not allow it. Hence, good musical ideas might escape us.

Conclusions and Future Work - Problems Faced

77

Chapter 10: Conclusions and Future Work 10.1 Problems Faced

10.1.1 Coming up with the final design The idea behind TIES is highly abstract and high-level. Although the idea of a tree of emergent systems sounds fairly simple, implementing a set of emergent systems as nodes in a tree was a interesting but difficult problem. The most difficult part was to make the tree perfectly flexible, that is, make the TIES framework totally independent of the emergent systems that it would contain. This was vital for scalability. If TIES was dependent on a set of specific emergent systems, then it would probably not work well with others. The design had to be changed several times because of this. Since music composition is not a standard problem solved in computer science (unlike, say, function approximation, for example), foreseeing design problems was also difficult. Often a problem actually emerged only when the design was implemented and then it was found that certain compositions were impossible. However, by trial and error we have reached the final design described in this report. It seems now to have the flexibility and independence that we expected. This fact has not, however, been formally proved, but its ability to incorporate a range of different emergent systems to produce a range of music in different styles is impressive.

10.1.2 Evaluation The other major problem faced in this project was how to evaluate the musical output. Firstly, designing a valid test was difficult. For example, we had to resort to expert evaluation to validate the musical output and were unable to use Turing like tests that are usually used to evaluate human intelligence mimicking applications. Secondly, finding test data was a problem. For example, it was difficult to find appropriate MIDI files generated by other systems to compare with the output of AMCTIES.

10.2 Lessons learned

10.2.1 Understanding emergent systems Emergent systems are not “conventional” computer scientific phenomena. And, like so many “unconventional” concepts, can only be understood by first hand experience. The author believes that this project enabled him to gain this. It enabled him to understand the workings of these, how they behave when various parameters are changed and how they might be used in practical situations.

10.2.2 Understanding artificial music composition Although the author had previously studied both music and computer science, and although he was familiar with several AMC applications, this was the first time he got to work directly with such an application. This enabled him to acquire a more realistic understanding of AMC systems, their limitations and how they can be improved. This project is the chief outcome of this understanding.

Conclusions and Future Work - Deviations from original project plan

78

10.2.3 Learning new technologies The author had the opportunity to work with several new computer technologies and related concepts. For example, he gained much knowledge about the Musical Instrument Digital Interface (MIDI) and its associated file format. He had the opportunity of learning many new concepts related to the .NET framework.

10.2.4 Literature Reviewing The actual design and implementation was preceded by a very comprehensive literature review that consisted of reviewing a large body of information. This process enabled the author to acquire skills at reading through material quickly, categorizing information and appropriately documenting them.

10.3 Deviations from original project plan

10.3.1 The name of the project The name of the project was changed from “Modelling the human music composing process with emergent behaviour” to “Automatic Music Composition using a Tree of Interacting Emergent Systems”. The new name is more representative of the unique design of the system and the name as a whole more commercially relevant. “Automatic Music Composition using a Tree of Interacting Emergent Systems” also forms the convenient acronym “AMCTIES”. This is important for an application with potential commercial value.

10.3.2 Use of techniques The original project proposal indicated that emergent behaviour would be used to generate musical entities. We have indeed used emergent behaviour; however, in an attempt to optimize the application, other non-emergent techniques have also been used hand-in-hand with the emergent techniques. Also, the original project proposal stated that Cellular Automata, Genetic Algorithms and Swarms will be used in the system. Cellular Automata and Genetic Algorithms are indeed used, but not Swarms. In its place Fractals are used.

10.3.3 Interactive Validation The original project proposal stated that the generation process of a musical piece will be interactively validated by a human. However, we found this disadvantageous for several reasons. Firstly, it is a serious performance bottleneck (the user needs to listen to the music and give his/her feed back, possibly several times for a single piece of music). Secondly, it hinders emergence (the output will be effectively generated by the human and not the emergent system). Finally, it is highly subjective. Hence, it was decided to exclude humans from interactively validating the system. However, human users are allowed to control the system if necessary by specifying a set of input options. These are specially designed not to hinder emergence. Also, the performance bottleneck is avoided.

Conclusions and Future Work - Deficiencies in the final product

79

10.4 Deficiencies in the final product Although the final product meets the specified requirements satisfactorily, it might be said to be slightly deficient in a few areas.

10.4.1 Demo Examples Although the specified final deliverables (the AMCTIES libraries) are thoroughly documented with source code well commented, only two demo applications were developed. This was due to time constraints. More demo applications would allow developers new to AMCTIES understand it better and quicker. Note, however, that neither the original project proposal nor the requirement specification specify the demo applications as requirements. They are purely extras.

10.4.2 Platform Dependence AMCTIES has been developed for the Microsoft .NET framework. This makes it deficient in terms of platform independence. However, this problem can easily be avoided by re-writing the code in some other more platform-independence-friendly language such as Java.

10.4.3 Layman Evaluation AMCTIES has only been evaluated by “experts” in the musical field, and not by “ordinary listeners of music”. Hence, although the expert evaluation confirms that AMCTIES is sound according to accepted music theory, we cannot categorically state that its results will be universally popular with layman listeners.

10.5 Extensions and Further Work

10.5.1 Further Validation As mentioned, the only real problem faced during this project was how best to validate the system. Further testing possibly along the lines of the Turing Test might be attempted. However, this will need much time and resources. Designing the tests will also be difficult.

10.5.2 Incorporating more aspects of MIDI AMCTIES uses MIDI more as an output medium, rather than a medium of expression (since it is essentially a music composing application, not a sound generating one). Hence, several functions and features of MIDI have not been incorporated (for example, more elaborate “control” features). These can either be incorporated into the TIES or else can be built as a layer above it.

10.5.3 Incorporating other musical elements In a sense, we have trivialized musical elements in order to build a “model of human musical composition”. We could extend this model in several ways. We could split current low-level entities into further entities, effectively extending TIES vertically. We could also extend TIES horizontally by adding further entities to each level. However, this expansion might come at a cost. The more specialized the emergent system is, the simpler its output. This effectively transfers the responsibility of producing effective output from the emergent systems themselves to the maps that combine them; the whole idea behind TIES is lost.

Conclusions and Future Work - Critical Appraisal of the System

80

10.5.4 Further musical element generators Several musical element generators have been built as a part of AMCTIES. However, AMCTIES has unlimited scalability. More generators can be added. This will enable composing music in a range of new and varied styles.

10.5.5 Other applications that might use TIES TIES might be also used for application generating other types of creative works. Two very interesting applications include visual art and sound wave synthesizing. Fractals and other emergent systems have been used (for quite a long time now) for generating visual artistic images. Though many of these are very beautiful, they are faced with many of the problems faced by attempts at AMC using pure emergent systems. For example, it is difficult to generate images to customized specifications. TIES might pave the way for more controlled visual art using emergent systems. Emergent systems have been used to generate some very interesting types of sounds. Here again the problems of controllability come up. TIES can give this controllability. A probable application might be used in electronic organs, on MIDI interface software, etc..

10.5.6 GUI style interfaced applications AMCTIES provides a comprehensive library for AMC. It can be made more accessible for users by building an application with a graphical user interface. However, care must be taken to preserve the inherent flexibility in AMCTIES. For example, AMCTIES tries to give the user as many initial options. The user has the ability to build his own composing environment to suit the style of music he requires. This would need some very careful designating. The complex nature of a potential user interface was one reason why it was not attempted as part of this project. On the other hand, TIES shields the user from implementation details. This must also be considered when building an application.

10.6 Critical Appraisal of the System

10.6.1 Does the system satisfy its specified requirements? AMCTIES has satisfied all its functional and non-functional requirements very well. Hence, this project has reached its goal of a system capable of mimicking the human music composing process.

10.6.2 Is the design of a high quality? AMCTIES is built upon a sound design. The TIES framework is highly versatile and flexible and is ideal for AMC. The emergent systems used in the generators incorporate several emergent techniques. Their use has been well justified.

10.6.3 Will the final deliverables be of practical use? The primary goal of this project was to build an “emergent behaviour” utilizing application that is capable of modelling the human music composing process. This has been successfully delivered in the form of a music composing class library written in C# .NET. The library consists of five packages: AMCTIES.Framework, AMCTIES.DefaultGenerators, AMCTIES.MusicTheorySupport,

Conclusions and Future Work - Final Conclusion

81

AMCTIES.MathSupport and AMCTIES.MIDISupport. These packages can be used to implement a range of AMC applications that would practically be very useful. Additionally, AMCTIES.MIDISupport and AMCTIES.MusicTheorySupport might be used for other applications as well. AMCTIES.MIDISupport implements a selection of methods that can be used to interact with MIDI files. This package can be used as a separate package for other music composing systems that also wish to utilize MIDI. AMCTIES.MusicTheorySupport represents common music theory concepts as object oriented classes. This, too, might be used independently.

10.6.4 Is the coding and documentation satisfactory? AMCTIES libraries have been coded according to best-practices in coding. All source code contains comprehensive XML comments. These are also accompanied by .NET style documentation. All codes have been thoroughly tested.

10.6.5 How do experts critically appraise the system? AMCTIES has been evaluated by experts in the musical field. They are confident about the musical merits of its output and rate its computational ability favourably. Music produced by AMCTIES was found to be expressive, naturally structured and interesting.

10.7 Final Conclusion The author is confident that AMCTIES or Automatic Music Composition using a Tree of Emergent Systems has succeeded in achieving its goal as a system that mimics the human music composition process. It has several interesting features and implements many novel techniques. It can also be considered as an excellent foundation for developing bigger and better music composition applications in the future. The author hopes that it will prove useful for professionals in both the music and the computer fields alike.

Music Theory Concepts and Definitions

82

Appendix A: Music Theory Concepts and Definitions A.1. Music Sound has been defined as the “sensation caused by the application of sound waves on the ear drum” [WESTRUP, 1989]. Any sound of finite length can be represented by a sound wave on a two dimensional graph – time and amplitude. Music can be defined as “sound with certain psycho-acoustic characteristics” [WESTRUP, 1989]. (For example, all musical sounds usually have a melody and a Rhythm.) This is a problematic definition: Most of the “psycho-acoustic characteristics” referred to cannot be directly recognized in the two dimensional representation of a sound wave (complex calculations involving Fourier transformations and other techniques are capable of some melodic recognition, but these are only vague approximations). What’s more, the term “certain psycho-acoustic characteristics” is very subjective. Instead of defining music in terms of its ensuing effect (in a sense Bottom-up), music could be alternatively defined in terms of its causal process (Top-down). Music can be defined as the output of the process of musical composition and performance. Musical Composition can be generally defined as the process of combining musical entities to produce a beautiful, satisfying and creative whole. Most musicians communicate their compositions to the outside world by writing them down in special musical notation. Musical “Performers” can be considered high-level translators of this musical notation into sound.

� ��

Hence, a better definition of music can be “sound that is resultant of a music compositional and performance process”.

A.2. Brief overview of musical notation At the core of musical notation is a structure of lines and spaces (known as a staff) on which music is written. Notes are indicated as rounded shapes (with or without tails that indicate the note’s duration). The pitch (frequency) of the note corresponds to its position on the staff. By default, the pitches of the notes are respectively C, D, E, F, G, A, B and the next C, D and so on. These might be changed by adding accidentals: Sharps or Flats that increase or decrease the pitches of the notes. Alternatively, what notes should be sharpened or flattened could be explicitly defined at the beginning of the piece by the key signature The duration of the note is a multiple or a division of some fixed duration quantum. The size of this quantum is indicated implicitly (as in fast, slow, etc.) or explicitly (as in 120 crotchets per minute), usually at the beginning of a piece. The music is divided into Rhythmic units known as bars. The time signature indicates the number of pulses (or beats) per bar, and the duration (in time quanta) of each pulse.


83

� �� 8 � ��

The Dynamics (amplitude, loudness or volume) of a note is indicated by directions such as f (forte, Italian for loud) and p (piano, soft). Changes in Dynamics can also be indicated by direction such as “Gradually getting louder”. There are many other types of directions for the performer, which are indicated textually in relevant parts of the music and/or at the beginning of the piece. The Timbre of a note is its “tone quality”. A note is made of several frequencies. The lowest of these is known as the fundamental and it is this frequency that corresponds to the notes pitch. The remaining frequencies are multiple of the fundamental and are known as overtones. The number and amplitude of the overtones define a notes Timbre. If the musical notation above was translated into a pitch-time graph it would look something like this.

. � ��


84

A.3. How is music different from other sounds?

� �� ((��9 � ��

A staff can be described as a two dimensional graph representing sounds. Here each note is a block. (Each of these blocks have many other characteristics such as Dynamics, tone quality, texture, etc., but for the moment these will be disregarded).

If note patterns are analysed, most notes are seen as form strands or sequences. These are not mere mathematically apparent sequences, but are actually recognizable to the ear. Almost, every sound fragment of music has a “principal note sequence” that is recognized as the melody or tune. A sound fragment may have several note sequences playing simultaneously at a given time. In such cases, often (but not always) it is the note sequence with the highest average pitch that is recognised as the melody.

� �� " � 8 � ��& �5 � ��

Not all the notes heard at a given time belong to a specific melodic note sequence. Some notes act as simply “filling material”.

In the above, notes are looked at for horizontal patterns. Conversely, notes could be looked at as vertical structures. In a musical sound fragment, notes often overlap to form vertical structures. If they are to sound pleasant to the ear, notes that overlap must be related according to certain pitch relationships. Also, a given vertical note structure, may only be succeeded and preceded by certain vertical note structures. These relationships are known collectively as harmony. Simple vertical note structures are referred to as Chords [SCHOLES, 1998]. Melody and harmony are the fundamental characteristics of music. They are perhaps the most important “psycho-acoustic characteristics” that make musical sound different from sound in general. Generally, all sounds have a volume, tone quality and texture, but only musical sounds have a melody and harmony. Also, melody and harmony are essentially abstract entities in that they cannot be directly (and accurately) be inferred as a sound fragment.


85

� �� $ � : �� 8 � ��& ��

Although music composition deals with many other aspects of sound, it is principally melody and harmony that is important. For simplicity, musical composition will be restricted to these aspects: Melodic Composition (or the composition of melodies) and Harmonic Composition (or the putting together of notes in harmony).

A.4. Melody A.4.1. Notes

As mentioned, a melody is a note sequence. A note is a single sound of a given pitch and duration. Hence, mathematically, a note may be represented by a duple (pitch, duration). Mathematically, a melody can be described as an ordered array of notes indexed by time. The pitch of a note cannot be an arbitrary frequency. Mathematically, the frequencies of the possible notes are of the form:

122.p

a ...Equation (1) The constant a defines the right tuning of musical instruments and is about 8.18. In theory, Zp ∈ , but in practice due to limitations in human hearing and instrumental ranges, it is constrained to a finite range. In the MIDI framework, pitches are defined in the range }127,...,1,0{∈p .

A.4.2. Key

In most pieces of music, all the notes bare a strong allegiance to a certain note. This is known as the key note or tonic. The music must return to this note for it to sound “complete” and notes that are unrelated to the tonic usually will sound unpleasant [SCHOLES, 1998].

A.4.3. Intervals

Since, musical notes are represented on an exponential scale, relationships between notes are measured in terms of their frequency ratios. These are known as intervals. The smallest interval used in music is the semitone. This corresponds to the ratio,

121

12

12)1(

22.

2. =+

p

p

a

a ...Equation (2)

Any interval between two notes consists of a whole number of semitones. The simpler the frequency ratio of the interval, the more related the notes appear to sound. The simplest relationship is that between a note and itself (known as a Unison). The next simplest relationship is that between


86

two notes, where the frequency of one is double that of the other. This is known as an octave [SCHOLES, 1998]. Notes that are an octave apart sound very similar (almost the same). For this reason, pitches might be notated as <Label><Octave>. Here, p is split into p = 12.x + y. x is known as the pitches octave and the possible values for y (0, 1,…11) are labelled: C, C sharp (or D flat), D, D sharp (or E flat), E, F, F sharp (or G flat), G, G sharp (or A flat), A, A sharp (or B flat) and B respectively. For example “Middle C” may be labelled, “C5”.

A.4.4. Scale

In general, a scale is a progression of notes, in strictly increasing order of pitch. More practically, the term scale has been used to refer to those scales (in the general sense), that contain the chief notes in particular types or traditions of compositions [WESTRUP, 1989]. The starting note of a scale is generally of no importance; it is the intervals between notes that actually define the scale. Hence, more mathematically, a scale can be defined as some arbitrary note and a set of other notes that are related to the former note by a set of intervals, increasing in size. Two of the most important scales found in western music are the major and minor scales.

A.4.5. Melodic Hierarchy

A piece of music can be considered as having one single principal melody, spanning the length of the whole piece. Alternatively, the melody can be broken down into component parts that form a hierarchy, similar to language [WESTRUP, 1989].

� �� ) � � �� ' �; �� ' �

A.4.6. Motif

Motifs are the smallest musical entities that can have a meaning. They are like words in language. They often consist of a few notes. Like individual letters that make up a word, the notes that make up a Motif, on their own, have little meaning. Usually, a single Motif belongs to a single key (or scale). Since it is very short, melodic and Rhythmic patterns cannot (usually) be identified. However, Motifs must be interesting and must have variety in melody, Rhythm or both [JACOBS, 1997]. Usually it is only the notes that make up a Motif that uniquely identify it. Other characteristics such as Rhythms, Dynamics, Timbre and key might be varied. Hence, alternately, we might define a Motif as a sequence of pitches, independent of other characteristics.


87

A.4.7. Phrases and Sentences

Musical Phrases are like the sections of a language sentence that are delimited with commas. Phrases consist of a repeated and/or developed sequence of Motifs. However, it is also important that a Phrase does not have “too many” Motifs and is balanced. Phrases usually have a melodic direction (for example, rising or falling) and its Rhythm must reflect this (for example, by having longer notes towards the end of a Phrase) [JACOBS, 1997]. Unlike Motifs, Phrases have a unique Rhythm, Dynamics, Timbre and key. A Phrase might thus be considered a collection of superimposed sets of Motifs, Rhythms, Chords, Dynamics and Timbre. A musical sentence is rather like a sentence in language. It expresses a single complete idea. Like a language sentence which ends with a full stop, a musical sentence ends on a note structure designed to give an effect of completion. Sentences are typically collections of two or four Phrases (though exceptions are abundant). These Phrases are in some characteristic order (for example, two Phrases ascending and descending respectively, or a sequence of similar Phrases progressively rising or falling). Component Phrases should be contrasting, but not to the extent of being incoherent. Also, Phrases within a sentence might modulate (that is, change in key). Modulations provide variety and make the music interesting. However, they must also adhere to certain rules [JACOBS, 1997]. Alternatively, a Sentence might be considered an “arbitrary” sequence of Phrases. This is because as far as music theory and composition technique is concerned, a Sentence is creatively somewhat redundant. Hence, we might even ignore the concept of a sentence, and consider higher formal structures as built from Phrases. For example, a Subject (which will be defined next) can be considered a sequence of Phrases.

A.4.8. Subject

Subjects are like paragraphs in an essay. A Subject is a subsection of a melody that can be considered to be in a particular mood. A single movement usually has several Subjects of contrasting nature. Subjects within a movement usually contrast in key, general mood and texture. However, these contrasts are in the context of a movement (For example, all Subjects in a fast movement are more or less fast.) [JACOBS, 1997].

A.4.9. Movement

A movement can be described as the smallest musical entity that may be considered complete in itself. Larger pieces of music often contain several movements. A movement often expresses some individual theme or idea and has its own specific speed (for example, fast or slow). It can be compared to an “essay” on a specific topic. The movements are either programmatic (the music attempts to describe a predefined story or image) or absolute (not programmatic). Absolute movements usually follow academically preset forms defined by the type of the piece (for example, the first movement of a classical sonata (usually) follows, what is known as, “Sonata Form”). Programmatic movements are usually freer in form. Both types roughly consist of a set of Subjects of varying character [JACOBS, 1997].

A.4.10. Musical Development

Generally, musical development can be defined as “exploiting further possibilities of thematic material by altering it in various ways” [JACOBS, 1997]. Since much of musical composition is spontaneous, it is difficult to say whether the process of using development in musical composition is conscious or sub-conscious. However, upon analyzing finished pieces of music, most “good”


88

attempts at development could be described as multiple (ingenious) applications of fairly simple musical operators such as ornamentation, augmentation and inversion [JACOBS, 1997].

A.5. Harmony Harmony is the study of notes played simultaneously together. It is principally based on the concepts of concords and discords. Concords are collections of notes that when played simultaneously together sound “settled” or “at rest”. Discords are collections of notes that when played simultaneously together sound “unsettled” or “incomplete”. Discords are expected to move (or resolve) into a concord [WESTRUP, 1989]. The study and practice of harmony is based on a complex and elaborate set of rules. These rules are results of years of observation, trial and error. Most rules have many exceptions. Fundamentally, harmony consists of a complex set of interactions between the various musical entities described earlier (notes, keys, intervals, etc.). For example, a note followed by two notes respectively 4 and 7 semitones above it, is known as a major triad. A major triad is a concord and is usually considered to sound “happy”. As exactly how and why this is, is not known. The concept of a “major triad” is one of the simplest concepts in harmony. This and countless other entities and concepts form a giant network of rules and practices. A detailed description of these is far beyond the scope of this report. Please refer [WESTRUP, 1989], [JACOBS, 1997] and [SCHOLES, 1998] for more details.

Required development and testing environment

89

Appendix B: Required development and testing environment B.1. Hardware requirements

• Personal Computer with 2.0 GHz Pentium IV processor, with 512 MB RAM and 40GB is required to develop, build and run the application.

• Multimedia capable for sound input and output is also required.

B.2. Software requirements • Microsoft .NET 2003 framework 1.0 for development. • MIDI editors for preparing test data. For example, Cakewalk® Proaudio™ 9.0.

Source Code and Documentation

90

Appendix C: Source Code and Documentation C.1. Source Directory Structure This project has been developed as a Microsoft .NET solution, on C# .NET. The full documented source code is over 100 pages long and has not been included in this report.

� �� , � & � � ��<�� ' �& �� - � 1 ��' ��

AMCTIES has been developed as a set of .NET projects under a single .NET solution. Five of the projects correspond to the five libraries delivered (AMCTIES_DefaultGenerators, AMCTIES_Framework, AMCTIES_MathSupport, AMCTIES_MIDISupport and AMCTIES_MusicTheorySupport). One (AMCTIES_Testing) is a general test stub for implementing the libraries for testing purposes. The other two (AMCTIES_Demo_Composer and AMCTIES_Demo_RhythmGenerator) are two demo applications for demonstrating the libraries. All the source code have been thoroughly commented and documented as DOT NET style documentation.

� �� . � �� + �� <* ��8 % ��' ��<� � � ��


91

C.2. Package AMCTIES.Framework

File Interfaces defined

Motif.cs

public interface IMotifGenerator public interface ImotifMapGenerator

Rhythm.cs

public interface IrhythmGenerator public interface IRhythmMapGenerator

Chord.cs

public interface IchordGenerator public interface IchordMapGenerator

Dynamic.cs

public interface IDynamicGenerator public interface IDynamicMapGenerator

Timbre.cs

public interface ItimbreGenerator public interface ItimbreMapGenerator

Phrase.cs

public interface IPhraseGenerator public interface IphraseMapGenerator

Subject.cs

public interface ISubjectGenerator public interface ISubjectMapGenerator

Movement.cs public interface IMovementMapGenerator

� �� !% & �� 9 � � �& � � ��

�� !% & �� 9 � � ��

The package is delivered as AMCTIES_Framework.dll. The interfaces defined in each file are described in the table. Additionally, Phrase.cs and Subject.cs implement maps and associated structures. For example, Phrase.cs implements classes PhraseMap and PhraseMapElement.

C.3. Package AMCTIES.DefaultGenerators

� �� !% & �<�(�� & � � ��

The package is delivered as AMCTIES_DefaultGenerators.dll. Each file contains the implementation of exactly one class. For example class CAChordGenerator is coded in file CAChordGenerator.cs. Note that both emergent object generators and map generators are implemented here.


92

C.4. Package AMCTIES.MusicTheorySupport

� �� !% & �� ' & �� & � � ��

File Classes implemented

Note.cs

public class Note public class Pitch public class Channel public class Duration

NoteSequence.cs

public class NoteSequence

Scale.cs public class Scale public class TriScale: Scale public class ChromaticScale: Scale public class PerfectScale: Scale public class MajorScale: Scale public class PentatonicScale: Scale public class BluesScale: Scale

Modifier.cs public class Modifier

Tests.cs public class Tests

� �� " � � � ��!% & �� ' & � � � � ��

The package is delivered as AMCTIES_MusicTheorySupport.dll. The classes implemented in each file are described in the table.

C.5. Package AMCTIES.MathSupport

� �� $ � � � ��!% & �� & � � � � ��& � � ��

AMCTIES.MathSupport package is contains a single class NMath that implements all its functionality. The package is delivered as AMCTIES_MathSupport.dll.

C.6. Package AMCTIES.MIDISupport

� �� ) � � � ��!% & = � !<!& � � � � ��& � � ��

Package AMCTIES.MIDISupport consists of three classes (coded in files with the same name) MIDIFileWriter, MIDISong and MIDISupport. The package is delivered as AMCTIES_MIDISupport.dll.

Demo Applications

93

Appendix D: Demo Applications The requirement of this project was to develop a set of libraries that would enable AMC. Extending these to user applications is, strictly, beyond the scope of this project. However, to give an idea of what sort of applications might be developed using the AMCTIES libraries, we quote two such.

D.1. AMCTIES Composer AMCTIES Composer is an application that will enable the user to compose pieces of music given minimal use input. The user simply selects what instruments must be used, the approximate speed of the music and the level of variety expected.

� �� , � � � ��!% & �� !� ��(� ��

� �

�

� �� - � �� 9 �� !% & ��

The application has some additional features as well. After the user generates the MIDI file of his/her composition, he/she can directly play back output on Windows Media. He/She can also edit the MIDI file on Cakewalk ProAudio. Both Windows Media Player and Cakewalk ProAudio have been integrated with the application. AMCTIES Composer is ideal for composing short bursts of atmospheric music, ideal for documentaries and films.

Demo Applications

94

D.2. Command Line Rhythm Generator

The Command Line Rhythm Generator (rgen.exe) is a simple console application that generates rhythms according to the user’s specification. It users the BeatRhythmGenerator. The user specifies the output MIDI file, output MIDI instrument, the duration of the output (in bars) and the beat pattern.

References

95

References [BRAUN, 1975] Jay Braun and Darwyn E. Linder

Psychology Today Random House, 1975

[BURRASTON, 2004] Dave Burraston, Ernest Edmonds, Dan Livingstone and Eduardo Reck Miranda Cellular Automata in MIDI based Computer Music. Proceedings of the 2004 International Computer Music Conference, 2004.

[BUZZANCA, 2001] Giuseppe Buzzanca A Rule-Based Expert System for Musical Style Recognition. Proceedings of the 1st International Conference “Understanding and Creating Music”, UCM 2001, Caserta, 2001.

[CARR, 2004] Paul H. Carr Does God Play Dice? Insights from the Fractal Geometry of Nature. Zygon, vol. 39, no. 4, 2004

[CONKLIN, 2003] Darrell Conklin Music Generation from Statistical Models. Proceedings of the AISB 2003 Symposium on Artificial Intelligence and Creativity in the Arts and Sciences, Aberystwyth, Wales, 2003.

[COPLAND, 1937] Aaron Copland What to listen for in Music McGraw-Hill, 1937

[DORIN, 2002] Alan Dorin LIQUIPRISM: Generating Polyrhythms with Cellular Automata. Proceedings of the 2002 International Conference on Auditory Display, Kyoto, Japan, July 2-5, 2002.

[GANGULY, 2004] Niloy Ganguly, Biplab K. Sikdar, Andreas Deutsch, Geoffrey Canright and P Pal Chaudhuri A Survey on Cellular Automata.

[JACOBS, 1997] Arthur Jacobs Penguin Dictionary of Music. Penguin Books, 1997.

References

96

[JÄRVELÄINEN, 2000] Hanna Järveläinen Algorithmic Musical Composition. Seminar on content creation, Telecommunications software and multimedia laboratory, Helsinki University of Technology.April, 2000.

[JOHANSON, 1998] Brad Johanson and Riccardo Poli GP-Music: An Interactive Genetic Programming System for Music Generation with Automated Fitness Raters. Genetic Programming 1998: Proceedings of the Third Annual Conference, 1998.

[MARQUES, 1998] M. Marques, V. Oliveira, S. Vieira and A. C. Rosa Music Composition using Genetic Evolutionary Algorithms, 1998.

[MIDI] MIDI Manufacturers Association Website www.midi.org

[MITCHELL, 1996] Melanie Mitchell An Introduction to Genetic Algorithms Prentice Hall, 1996

[MOZER, 2003] Michael C. Mozer Neural network music composition by prediction: Exploring the benefits of psychoacoustic constraints and multiscale process. Connection Science, 1994.

[MSDN] MSDN, Microsoft Developer Network Documentation, July 2004. Microsoft Cooperation 2004.

[PUENTE, 2002] Alfonso Ortega de la Puente, Rafael Sánchez Alfonso and Manuel Alfonseca Moreno Automatic composition of music by means of Grammatical Evolution. APL'2002 Madrid Proceedings, 2002.

[PACHET, 2001] Francois Pachet and Pierre Roy Musical Harmonization with Constraints: A Survey. Constraints 2001

References

97

[PAPADOPOULOS, 1999] George Papadopoulos and Geraint Wiggins AI Methods for Algorithmic Composition: A Survey, a Critical View and Future Prospects. AISB’99 Symposium on Musical Creativity, Edinburgh, Scotland, 1999.

[PAPADOPOULOS, 2000] George Papadopoulos and Geraint Wiggins A Genetic Algorithm for the Generation of Jazz Melodies. STeP, 2000.

[SANTOS, 2000] Antonino Santos, Bernardino Arcay, Julián Dorado, Juan Romero and Jose Rodriguez Evolutionary Computation Systems for Musical Composition. Mathematics and Computers in Modern Science, World Scientific and Engineering Society Press, 2000.

[SCHOLES, 1998] Percy Scholes The Oxford Companion to Music. Oxford University Press,1998.

[YEE-KING, 2001] Matthew Yee-King Algorithmic Composition and the State of the Art., 2001.

[WESTRUP, 1989] Sir Jack Westrup and F.Ll.Harrison Collins Encyclopaedia of Music. Collins, 1989.

[WIKIPEDIA] Wikipedia Website www.wikipedia.org

[WOLFRAM, 1984] Steven Wolfram Universality and Complexity in Cellular Automata. Physica D, 10:1-35, 1984.

[WOLFRAM, 2002] Steven Wolfram A New Kind of Science, 2002.

automatic music composition using a tree of · pdf fileautomatic music composition using a...

Documents