In this class students develop interactive music projects: pieces of music that are not linear, but rather offer multiple dimensions for listeners to explore ––on their phones in a crowded subway, at an abandoned factory in Palermo, back on their couches after a long day, or at a classical concert hall.

Students will take a project from concept to execution over several iterations, applying music production, interaction design, and creative coding tools and techniques. During the first half of the semester, they will gather references and create musical user paths and interactive studies to explore specific elements of their composition. This work will lead to the implementation of the midterm project: a functional, high-fidelity prototype. For their final projects, students will evaluate their midterm pieces from the perspectives of music, visual / tactile and interaction design, and refine them to produce an engaging piece of interactive music.

ICM or equivalent experience is required. Some experience in making or producing music will be useful, but is not required.


Spring 2019



Luisa Pereira |


12:10 - 2:40pm

Office Hours

Thursdays, 2-4pm. Sign up here

Syllabus Overview

Jan 30
Class context, goals and process. Applications.
Feb 6
Rhythm and melody.
Music teaching tools, Music making tools.
Feb 13
Harmony and timbre.
Music teaching tools, Music making tools.
Feb 20
Interaction design frameworks.
Dimension spaces for musical devices.
Feb 27
Design principles for musical devices.
The problem of mapping.
Mar 6
Tools: Overview + references for self-study.
MIDI, OSC, and Live API control.
Creative coding frameworks and libraries.
Art Installations. Guest: Lea Bertucci.
Mar 27
Evaluation criteria. Project planning.
Case Study: Instrument for Live Performance.
Apr 3
Museum Exhibitions. Guests: Andreas Matt and Valentina Camacho.
Focused ideation workshop: La la lab, an exhibition on music and math.
Apr 10
Max for Live tutorial / workshop
Apr 17
Tone.js tutorial / workshop
Games. Guests: Ross & Cody
Apr 24
User Testing

Class Notes and Assignments

This section will be updated weekly with references, code examples and assignment details.


Class Meeting

Introductions + music pieces

Class topics, context and approach


  • Museum Exhibitions
  • Interactive Albums
  • Art Installations
  • Participative Performances
  • Music Teaching Tools
  • Music Making Tools
  • Games

Group activity: design an interactive musical experience


1. Document your concept on a blog post containing:

  • A brief description of the concept that includes what it does, who it is for, and where it lives (not more than a couple of sentences)
  • A drawn sketch (or sketches) that indicates form, materials, scale, and interaction
  • The song you started from
  • The oblique strategy you got
Submit your blog post link here.

2. Write a project prompt for yourself. You will use it to frame subsequent assignments, but it can evolve/change later. Submit it here.

Music I

Class Meeting

Group activity presentations

Individual project prompts

Lecture: The elements of rhythm. The elements of melody

Music Teaching Tools: Rhythm and Melody. Try, compare, discuss

Music Making Tools: a brief introduction to Ableton Live


1. Fill in this poll re: software licenses (thank you!)

2. Teach yourself Ableton Live (if you are not already a user). I recommend following these lessons from Help / Help View:

  • A tour of Live
  • Recording Audio
  • Creating Beats
  • Playing Software Instruments

3. Start working on your project:

  • Refine your prompt: who is your project for? Are you a part of this group of people? Is someone close to you (a classmate, friend or family member)? Can you find someone to play test your project and give you feedback?
  • Make an aural mood board: a collection of reference audio clips. This should be in a format that you can easily share with the class in under two minutes (include short clips rather than full pieces or recordings). You can annotate them to indicate why you included each clip.
  • Create a musical user path: what does it sound like to experience your project, to one user, during one particular interaction session? If you like, you can add a second path to illustrate a range of possible experiences. Whatever process you use to create each path, the result should be a rendered audio file. Remember this is a first draft: the idea is that you start considering duration, musical elements, arcs, sections (if any), and so on. There is no expectation of production polish at this point.
Submit a blog post including the above here.

Music II

Class Meeting

Music Making Tools: demos by Billy - Thingamagoop, Op-1

Project updates, part I:
target audiences, aural mood boards and musical user paths.

Lecture: Harmony, Timbre: Synthesis.

Music Teaching Tools: Synthesis. Spectrogram, Harmonics Explorer


1. Music Teaching Tools: Harmony. Make some music using the following:

Write a short response discussing your experience.

2. Do a second pass on your musical user path exploring tools for manipulating harmony and timbre. For example:

  • Harmony: MIDI effects: Chord, Arpeggiator
  • Synthesis: Analog, Operator
  • Sampling: Simpler, Sampler, Granulator

3. Create additional, alternative user paths to illustrate the range of interaction experiences your project might offer. Start thinking of interactions: what user actions will determine which musical outcomes? What aspects of the music will evolve independently of user actions (if any)?

Submit your blog post here.

Interaction I

Class Meeting

Lecture: Additive, Subtractive, FM Synthesis

Music Making Tools: demo by Aaron - Ableton's Operator

Lecture + Activities: Interaction Design.

Project updates, part II

Creative Coding Tools: demo by Matthew - capacitive sensing with Teensy > Granulator, via MIDI


1. Apply Verplank's IxD framework to your project - a second pass, after getting feedback from your partner in class. Remember you are not limited to just one metaphor, scenario, control, etc.

2. Place your project on the 7-axis dimension space diagram. What features are a priority to you? To your target audience? Feel free to change the number of axes and the dimension they represent.

3. Describe at least three feel/know/do cycles for your project. Be specific. Prototype at least two, and have someone else try it. Your prototype can take any form, including you acting as Wizard of Oz. Document your test and findings.

Submit your blog post here.

Interaction II

Class Meeting

Class updates: guest speakers + music & math exhibition

Project updates: Billy, Aaron, Dana, Maya, Raaziq, Marcela

Music Making Tools: demos by Dana - Little Bits and Pocket Operator

Lecture + Discussion: Design Principles and the problem of Mapping.

Video: Tom Zé performance with power tools, hack saws, and custom sampler.


1. Read the paper on mapping by Hunt et al

2. Use the worksheet to generate many potential mapping ideas for your project. A few notes:

  • Consider cross-couplings and layered mapping (see the paper)
  • "button" stands for any discrete control; "slider" stands for any continuous control. You can replace them with any sensors.
  • You can have different ranges of variation for each 'control' column
  • The musical elements on the left column are meant as a guide, and follow our class discussions from weeks 1 and 2. Feel free to adapt the column so that it applies to your project more directly.

3. Prototype three of the mappings from the worksheet (at least). You can use a MIDI Controller with Live, either a pre-built one from the ER or a custom-made one (with Arduino or Teensy). Have other people try your prototypes.

4. Optional: re-create the two-slider experiment described on the paper.

Submit a blog post including assignments 2, 3 and 4 here.

5. Fill in this form about the technology you are using for your midterm. You can also just drop the paper handout at my office.

Tools I: Tools Overview. Applications I: Art Installations

Class Meeting

Project updates: Tawania, Matthew, Yves, Kemi, Xiaotong, Nianqi, Chenhe

Discussion: a map of available tools

Applications: Art Installations. Guest speaker: Lea Bertucci



1. Midterm project: build a functional prototype of your project

2. Prepare a 5 - 8 minute presentation, including:

  • Concept. In one or two sentences, what is your project? Why did you choose this project? Is it expressive / needed / entertaining / intriguing / provocative?
  • Demo. Perform your piece for the class, or have a classmate perform it. Do this live, but have a backup video.
  • Technology. Briefly, how does it work? Mention the main techniques and components you used.
  • Future work / next steps

3. Post your presentation here (link to spreadsheet).

Midterm Project Presentations


Interaction Design: Evaluation

Class Meeting

Survey: this class so far

Activity: final project planning

Activity: evaluation criteria

Project case: Little Dragon + Samsung

Tools: introduction to Max for Live


1. Review your midterm project presentation (see the "Midterm Project Documentation" list above), and make sure it is clear enough for your partner to read/watch through and evaluate. If you would like me to replace the presentation with a blog post, please send me an email with a link to it.

2. Write a critique of your partner's midterm project based on the criteria you came up with together. You can find their documentation under "Midterm Project Presentations" on this syllabus. Post your structured critique here.

3. Finish your Midterm to Final worksheets and post here.

Applications: Museum Exhibitions

Class Meeting

Guest Speaker: Valentina Camacho, Senior UX Designer, Gallagher Design

Guest Speaker: Andreas Matt, Managing Director, IMAGINARY gGmbH

Activity: ideation workshop for La la lab, an Open Source exhibition on Music and Math


1. Submit your idea posters here.

2. Work on your final project and write a blog post on your progress. Submit here

Tools: Max for Live

Class Meeting

Project Updates I

Project Case: Generative Music

Max for Live Basics + Exercises


Work on your final project and write a blog post on your progress. Submit here.

Tools: p5.js + Tone.js

Class Meeting

Project Updates II

p5.js + Tone.js tutorial: create a step sequencer, step by step.

Applications: Games. Guest speakers: Ross Wariner and Cody Uhler from Two Dots


1. Work on your final project and prepare for user testing - please read the guidelines below.

2. Sign up for a day to present your final project here.

3. Revisit our shared evaluation criteria spreadsheet. In the first sheet, under your name, mark the five criteria that you consider to be most important for the success of your project. You can rank them from 1 to 5 if you like. During your final presentation, your classmates will refer to the criteria you choose to give you feedback on your project.

Interaction Design: User Testing

Class Meeting

Each of you will be a presenter for half of the class, and a tester for the other half. Please come prepared and follow these guidelines:

As a presenter:

  • Try to emulate the way people would encounter your project in its intended context. In most cases this means you will not be there to explain the concept or interface. If the context is a gallery space and there is a plaque, write the plaque. If the context is a website and there will be instructions, write the instructions on a piece of paper.
  • It is okay if some parts of your project are not implemented yet. Come prepared to illustrate them: there's no sound when you click yet? Play it yourself, perhaps on your phone. There is a physical device but you haven't made it yet? Bring a similar object or a cardboard mockup.
  • Bring specific questions to ask your testers during feedback.

As a tester:

  • While testing, think out loud so your classmate can better understand your experience ––what's confusing? what's delightful? what's jarring? For example: "I'm going to move this slider because I think it will give me the sound texture I want. Wait, I did not expect that to happen".
  • While giving feedback, be constructive. Start/stop/continue can be a useful framework: what should the project do that it's not doing yet? What should it not do? What should it continue doing?


1. Document your user testing findings. Submit your blog post here.

2. Finish your final project and prepare your presentation. Please post your slides here before presentation time.

Final Projects

Student Projects

  • Marcela Mancino
  • Kemi Sijuwade
  • Matthew Ross
  • Dana Elkis
  • Chenhe Zhang
  • Xiaotong Ma
  • Yves Pokakunkanon
  • Nianqi Zhang
  • Ian Macdougald
  • Guillermo Montecinos
  • Aaron Ayala
  • Maya Pruitt
  • Raaziq Brown
  • Tawania Reggler
  • Billy Bennet


Write a final blog post documenting your project for posterity (this will be linked above). Submit here.


Books and Papers

In no particular order:

  • An Alternative to a Standard Taxonomy for Electronic and Computer Instruments, by Laurie Spiegel
  • The Rest is Noise, by Alex Ross
  • Deep Listening, by Pauline Oliveros
  • Making Music: Creative Strategies for Electronic Music Producers, by Dennis DeSantis
  • Music as Mirror of Mind, by Laurie Spiegel
  • A Geometry of Music, by Dmitri Tymoczko
  • The Computer Music Tutorial, by Curtis Roads
  • Make:Analog Synthesizers, by Ray Wilson
  • The Design of Everyday Things, by Don Norman
  • Designing Interactions, by Bill Moggridge