Improvisation(Multimodal Music Performance)

When we talk about improvisation with computer, we distinguish between two main approaches:

1. Computer as a special instrument that can produce sounds by direct manipulation of sounds in ways that acoustic instruments can no do

2. Computer as a musician, or in other words we expect the machine to take more of a creative role, something a human musician would do. A computer can be used then as a musical companion and not so much as an instrument.

So In order to improvise with a computer we need two main elements: an input device that can control the computer operation, and an algorithm to synthesize, modify existing or create original music

Controllers

• Midi Controller with software synth/Pd
• Midi Controller with Ableton Live
• Kinect and Ableton Live/Pd(Synapse, TUIO)
  • For PC, go to this site and follow the instructions. OpenNI/NITE Installation on Windows
    • Since the official site for openni is down, have to get the source code from here.
  • Need OSCeleton to bridge between Kinect and Pd. OSCeleton
• Mobile Devices as controller(OSC, PyGame & RaspberryPi)
• Littlebits
• Software(PyOracle Improviser)

Task

• Tryout some of the things mentioned above, make some noise.
• What`s the strategy when improvising alone? with a partner? within a group?
  • Expectation/surprise
  • Synchronization
  • Form?
• How these technologies used effect the above questions?
• Collaborate and make some music!

Algorithmic Composition

• History of Algorithmic Composition
• Random Processes as Music

Mozart Dice Game Music

Lexikon Sonata

• Chaos, Genetic Algorithms and Cellular Automata

Improvisation

• Grammars to represent Music
• Statistical Learning Methods

Improvisation by computer requires the machine to be able to "understand" aspects of music. We have seen that even as humans we have difficulty understanding the complexity and variety of musical events. The first step in trying to understand sound by computer is to extract some useful information from the sound. Explain the "computer audition" paradigm:

• Listening to Sound
• Feature Extraction
• Modeling Time Evolution / Musical Form
• Understanding Information Contents: Surprise, Complexity

Introduce Sonic Visualizer and / or some feature extraction Libraries Chord Estimation in Audacity with Chordino Python feature extraction libraries: Bregman Toolkit, Essentia

• Advanced / Potential Project:

Improvisation with PyOracle

Chord Estimation with HMM