PyClarion / Mockingbird

Michael F. Lynch

What is Clarion?

• A hybrid cognitive architecture developed at the Department of Cognitive Science at RPI by Ron Sun and colleagues.

• Four major subsystems, each supporting an upper, symbolic layer and lower, sub-symbolic layer.

• Models, among other things, motivation (drives and goals) and metacognition

• Also models (in Wilson’s version) personality.

Action-CenteredExplicit

Representation

Action-CenteredImplicit

Representation

Non-Action-CenteredExplicit

Representation

Non-Action-CenteredImplicit

Representation

Filtering/Selection/Regulation

Goal Setting

Reinforcement

Action

Perception

Goal Structure

Drives

Clarion

ACS

MS

NACS

MCS

What is PyClarion?

• Another rewrite, necessitated by limitations in the C# edition, this time in Python.

• Uses the PyBrain library by Schmidhuber et al for its neural net implementations.

• This is turn will form the basis for further work into episodic memory.

• Also the basis for Mockingbird, an interactive musical agent (IMA).

Mockingbird

What is Mockingbird?

• … an attempt to build an Intelligent musical agent (IMA) based on Ron Sun’s Clarion cognitive architecture and Doug Van Nort’s FILTER expanded musical instrument (aka “hyperinstrument”).

Mockingbird – What it Does

• Mockingbird interacts with a live performer in a free improvisation performance.

• Mockingbird analyzes, via FILTER, the human performer’s auditory live audio signal, and makes decisions in real time to create its own musical accompaniments.

Positioning Mockingbird

• “Tape delay” (and later digital delay) music has a long tradition, dating back to the 1950s.

• Builds on the musique concrète tradition of Pierre Schaeffer.

• Among the early pioneers: Pauline Oliveros, right here at RPI.

Expanded Instruments

• Oliveros’s EIS• Van Nort’s GREIS (inspired by EIS)• Van Nort’s FILTER• CAIRA (Creative, Artificially-Intuitive and

Reasoning Agent)• Mockingbird

Mockingbird – Design Goals

• Construct a musical “agent” (a musical accompanist and improviser)– Our term: Intelligent Musical Agent (IMA)

• Identify and develop cognitive models of musical performance needed in such an agent.

• Implement these models in an actual agent, using the Clarion cognitive architecture as the “substrate.”

Mockingbird – Initial Goals

• Be able to listen to a single live performer• “Learn” the contours of the performance

during the actual performance• From that, accompany the performer in the

manner of free improvisation, one-on-one, playing back the performer’s own materials, time-delayed and transformed.

• That’s why it’s called Mockingbird.

What is Needed…

• Python programmers… especially if you are really good with multiprocessing programming techniques.

• People interested in neural nets and working with the PyBrain library.

• People interested in cognitive architectures and working on Clarion.

• People interested in these sorts of issues in experimental music.

• lynchm2@rpi.edu