a creative improvisational companion based on idiomatic ...keller/jazz/improvisor/iccc2012... · a...
TRANSCRIPT
![Page 1: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/1.jpg)
A Creative Improvisational Companionbased on
Idiomatic Harmonic Bricks
Robert M. Keller1
August Toman-Yih1
Alexandra Schofield1
Zachary Merritt2
1 Harvey Mudd College2 University of Central Florida
Funded in part by NSF CNS REU 0753306
Third International Conference on ComputationalCreativity 2012, Dublin
![Page 2: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/2.jpg)
Motivation
• Educating (jazz) improvisers
• Provide feedback (visual and aural) onsolos over chord changes:• Through-composed solos• Contrafacts (new melodies over chord
changes of existing tunes)• Improvised solos (through MIDI input)
![Page 3: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/3.jpg)
Aside on the Origin of “jazz”
• Said to be derived from Irish: “teas”,pronounced “jass” or “chass”
• Meaning:• Heat, passion
• Reference:www.counterpunch.org/2006/07/14/how-the-irish-invented-jazz/
![Page 4: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/4.jpg)
Impro-Visor(Improvisation Advisor)
• Free, open-source, software
• Developed at Harvey Mudd College since 2005
• Over 7000 users registered in community forum
• Classroom use for 5 years
![Page 5: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/5.jpg)
“Leadsheet”
• Commonly used by jazz and othermusicians
• A leadsheet abstracts a tune:• Melody• Chord progression• Minimal or no arrangement
![Page 6: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/6.jpg)
Sample Impro-Visor Leadsheet
![Page 7: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/7.jpg)
Chords Only
![Page 8: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/8.jpg)
Human-Composed Solo
![Page 9: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/9.jpg)
Impro-Visor Creativity
• Creates background (e.g. bass, piano, drums)from• Chord progression• Style specification
• Creates demonstrative improvised melodies• Based on probabilistic grammar
& melody abstraction/induction• Tries to express various player styles
![Page 10: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/10.jpg)
Induced Solo
![Page 11: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/11.jpg)
Melody Induction
Abstract Melody1
Abstract Melody2
Abstract Melodyn
.
.
.
Grammar
Generation Instantiation
Melody1
Melody2
Melodyn
.
.
.
Chord Sequence
![Page 12: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/12.jpg)
What can be improved?
• Coherence of generated melodic line
• “Bricks” provide an approach toenhancing coherence.
![Page 13: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/13.jpg)
Bricks
• After Conrad Cork (1988, …, 2008):“Jazz Harmony using LEGO Bricks”
• Reverse-engineering chord progression intoidiomatic sub-progressions (“bricks”).
• Explain the tune by providing an abstractionof a tune’s harmony, coarser-grained than asequence of chords.
![Page 14: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/14.jpg)
Chord Sequence for“In Dublin’s Fair City”
G | Em | Am | D7 | G | Em | Am | D7 | G | Em | Am | D7 | G | Em | Am / D7 | G / D7 |
![Page 15: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/15.jpg)
Brick Analysis = “Roadmap” of Tune4 of same brick + 1
POT = “Plain Old Turnaround”
![Page 16: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/16.jpg)
Roadmap of a Jazz Tune4 distinct brick types
![Page 17: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/17.jpg)
Textual Input for the Jazz Tune
Cm69 | / | Fm7 | / | Dm7b5 | G7alt | Cm69 | / |
Ebm7 | Ab7 | DbM7 | / |
Dm7b5 | G7alt | Cm69 | Dm7b5 G7alt |
![Page 18: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/18.jpg)
Brick Analysis
• Impro-Visor automates analysis ofchord progression into bricks.
• Based on “brick dictionary”, anempirically-derived grammar.
• (Analysis algorithms are described inanother paper, currently under review.)
![Page 19: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/19.jpg)
Brick Dictionary
• About 100 brick types
• John Elliott: “Insights in Jazz” (2009)
• About 500 rules for brick expansion
• For more examples, google“The Roadmap Garden”:http://www.cs.hmc.edu/~keller/jazz/improvisor/RoadmapGarden508a.htm
![Page 20: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/20.jpg)
Interaction
![Page 21: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/21.jpg)
Interactive Companions for Jazz
• Al Biles: GenJam (1994)
• William Walker (1997)
• Belinda Thom: BoB (Band out of a Box, 2000)
• The above are either proprietary or notgenerally available.
![Page 22: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/22.jpg)
Using Bricks for anInteractive Companion
• Play-along: point at brick to play / loop
• Auto-improvisation based on bricks
• Trading based on bricks(work in progress)
![Page 23: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/23.jpg)
Impro-Visor Grammars
• Currently productions expand to fill space,without discriminating on specific chordpatterns.
• To add coherence, abstract window units(e.g. 1-bar in length) are sequenced byembedding Markov chains in grammar.
![Page 24: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/24.jpg)
Brick-Based Grammars
• Intended to achieve greater coherenceof generated melodic lines.
• Brick names used as non-terminals.
• Presence of brick in the roadmapdetermines possible melodicexpansions.
![Page 25: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/25.jpg)
Learning Aspects
![Page 26: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/26.jpg)
Learning Creativity: Degrees
• No learning
• Scripted learning
• Autonomous learning
Impro-Visor
![Page 27: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/27.jpg)
Grammar Learning: Current
• Gillick, Tang, and Keller (CMJ, 2010)Grammar learning based on abstractwindow units that don’t necessarily relateto harmonic phrasing.
![Page 28: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/28.jpg)
Learning = Grammar Inference
Abstract Melody1
Abstract Melody2
Abstract Melodyn
.
.
.
Grammar
GenerationLearning Instantiation
Melody1
Melody2
Melodym
.
.
.
Melody1
Melody2
Melodyn
.
.
.
Chord SequenceChord Sequence(not necessarily related)
![Page 29: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/29.jpg)
Grammar Learning Direction
• Brick-based scheme
• Use bricks as the grammatical unit.
• Use the existing scheme to fill in any gaps in brickanalysis.
![Page 30: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/30.jpg)
Brick-Based Grammar Learning
• Transcribed solo melody is segmentedaccording to bricks, rather than fixed-lengthwindows.
• Segmented melodies are mapped toabstract melodies, as in current Impro-Visor.
• Productions are derived:• Left-hand sides: Brick names + duration argument• Right-hand sides: Abstract melodies
![Page 31: A Creative Improvisational Companion based on Idiomatic ...keller/jazz/improvisor/iccc2012... · A Creative Improvisational Companion based on Idiomatic Harmonic Bricks ... (e.g](https://reader031.vdocument.in/reader031/viewer/2022021503/5a78f4f07f8b9a83238ef515/html5/thumbnails/31.jpg)
Remaining Work
• Implementation of brick-based grammarsand learning.(Technology exists, integration required.)
• Evaluation of brick-based method.