This course explores the elements of music through the lenses of computation and interactive design.

Each of the four course units focuses on one element of music (rhythm, melody, timbre, and harmony), approaching it from the perspectives of music history and theory, interactive design, and computation. For each element, students listen to examples from different periods and styles, represent and manipulate the element in code, and create an interactive study around it. Algorithmic composition techniques such as Markov Chains, Neural Networks and L-Systems are covered throughout the semester. The final two weeks are dedicated to the development of a final project. The final project can consist of: a) The studies created for the class, refined and presented as a series; b) A continuation or combination of previous studies or c) A new generative/interactive music piece or instrument.

In-class coding and assignments will be done in P5.js and Tone.js, but students will be free to use other languages and frameworks for their final projects. ICM or equivalent programming experience is required.

Weekly Assignments
Assignments will range from design and programming exercises to readings. In addition, every two weeks you will create a project exploring a different element of music. Assignments and projects are are required and you should be prepared to show/talk about them in class. You will be expected to post them to your blog the night before class.

Final Project
Prompt: create a generative / interactive musical piece. It can be web-based, a physical device, or in space. In your presentation, discuss musical, design, and programmatic aspects of your project.

Syllabus Overview

Sept 19
Oct 10
Oct 31
Nov 21
Final Projects
[Thanksgiving Recess]
Nov 28
Final Project Workshop
Dec 5
Present to Guests
Dec 12


Fall 2019, NYU ITP


Luisa Pereira |


Thursdays, 12:25 – 2:55pm

Office Hours

Class Notes and Assignments

This section will be updated weekly with references, code examples and assignment details.


Class Meeting

Introductions + music pieces

Syllabus overview

Space + applications

Activity: do a catalog of attributes of one of your songs. Design an interactive music experience based on it.


  1. Design: finish your experience concept and document it. Include a title, a one-paragraph description, a sketch/text description of the interaction, and anything else that would help communicate your concept quickly. Submit it using this form.
  2. Listen: find a piece of music you find rhythmically engaging. Submit it on this form.
  3. Read: This is your brain on music, Chapters 1 and 2.


Creative Coding Review

Please make sure to review the following concepts:
  • Variables
  • If Statements
  • For loops
  • Arrays
  • Functions
  • Callback functions

Rhythm: Listen + Interact

Class Meeting

Present assignments + discuss reading

Listen: beats, tempo, subdivision, accents, syncopation, cross-rhythms

A dimension space for musical devices, as proposed by D. Birnbaum, R. Fiebrink et al.

Group notes



Pick two rhythm interfaces and interact with each for at least three minutes. These can be digital or physical - here are a some web-based examples:

Which elements of rhythm does each interface represent and how? What kinds of inputs/gestures are available to control them? What feedback modes are present or missing? Does the device give the user control over the musical process? What degree of expertise does each interface require? Is the interaction expressive/didactic/something else?

Place each interface on a dimension space - either D. Birnbaum's, or your own dimension space, defined by any axes you deem relevant. Here is a printable version of the original, labeled dimension space diagram.

Submit your diagram/text responses here: rhythm interface analysis.

Rhythm: Code + Design

Class Meeting

Code: Keeping time. Play a sound over and over

Code: The elements of rhythm

Discuss: What possibilities does representing these elements in code open up? Consider interactivity: input gestures and control, visual/tactile representation of each element, mapping, and feedback modes.

Design: Sketch three ideas for a rhythmic interface.

Code: Generative rhythm

Tutorial: Create a drum machine

  • Make a drum pad (click to trigger kick).
    Include Tone.js, import a sound, play it back when the mouse is pressed. Try making a whole drum kit.
  • Make a beat (four on the floor)
    Create a loop to play the kick drum twice per second. Try making it faster and slower.
  • Make a drum pattern
    Add more sounds and play them back on specific steps using modullo. Try creating different patterns.
  • Make a drum machine
    Replace steps by modullo with a pattern matrix. Try different patterns by editing the matrix.
  • Draw your drum machine
    Draw the drum pattern. Try making it a variable size. Group beats in bars. Add colors and margins. Try your own palette, shapes, images, positions, animations. Make a circular sequencer.
  • Make it interactive. Click to turn steps on and off.
  • Add a full drum kit and interface
    Exercise: Add sliders and buttons to change the tempo, number of measures, time signature, accents, swing, and any other parameter you find relevant. These can be DOM elements or physical controllers.


1. Post between 3 and 10 pen-and-paper sketches for possible interfaces, and submit them here.

2. Optional: follow the drum machine tutorial above. Try the prompts and do the suggested exercise at the end. I highly recommend doing this if you are relatively new to programming, javascript, or the libraries we are using.

3. Design + Code: Create an interactive sketch to explore rhythm. Record a 20-second composition using it. Post your documentation and code here, and come prepared to do a 5-minute presentation in class.



Web Audio

Neural networks

Rhythm: Present Projects

Interactive Rhythm Studies

Melody: Listen + Interact

Class Meeting

Listen: pitches, notes, modes and scales, intervals

Discuss: The Do/Feel/Know cycle, as descibed in B. Verplank's Interaction Design Sketchbook.


1. Listen: Find a piece of music you find melodically engaging. If possible, determine the mode/scale it is in. Draw the melody, denoting the contour created by its intervals.

2. Ideate: Based on the song you picked and one random oblique strategies, sketch an interfaces to explore the elements of melody. Submit song and sketch here (this a pen and paper sketch, not a p5 sketch!).

3. Interact: Pick two melody interfaces and interact with each for at least three minutes. Here are a some screen-based examples:

And some physical examples (come ask me):

  • Akai LPK25 arpeggiator
  • Pocket operator
  • A custom, slider-based theremin

Analyze each interface in terms of the Know/Feel/Do cycle we discussed in class. Submit your analyses here.

4. Code (optional): Follow this tutorial to turn the drum machine into a melody sequencer. Steps:

  • Replace drum samples with notes. Things to try:
    • Different scales or modes. Include a variable step. Make a microtonal sequencer.
    • Different sounds (a cat meowing? a person talking?)
  • Extend your grid. Add octaves and try different grid sizes.
  • Add controls. Add DOM controls for picking the scale, range of notes (octaves), and any other variable you come up with.

Share your sketch / sketches here.

Melody: Code + Design

Class Meeting

Code: The elements of melody


  • Modes and scales. Create a random melody in different modes. Add a high likelihood for the root note to play. >> Here is Vibert's response, "guess the mode"
  • Intervals. Create a random melody by picking a sequence of intervals. Apply this melody to differrent scales.

Discuss: Mapping

Activity: Think through different mappings between the elements of melody and a set of button/slider combinations. Consider cross-couplings. Quickly prototype one of these mappings (here is a random example).

Code: Generative melody


Design + Code: Create an interactive sketch to explore melody. Record a 20-second composition using it *. Post your documentation and code here, and come prepared to do a 5-minute presentation in class.

* here is a tutorial Jessica wrote on how to record your internal audio, in case you are using Quicktime.




Melody: Present Projects

Interactive Melody Studies

Timbre: Listen + Interact. Sound Synthesis

Class Meeting


  • Pitched sounds vs percussive sounds
  • Synthesis techniques: additive, subtractive and FM synthesis
  • Synthesizer components: oscillators, amplifiers, filters, envelopes and LFO's


  • Spectrogram with examples of different timbres
  • A tool to explore harmonics
  • Synthesizers:
    • Digital, custom: Max patches for each synthesis technique.
    • Digital, off the shelf: Ableton's Analog (subtractive) and Operator (a combination of subtractive, additive, and FM).
    • Analog, semi-modular: Moog Mother32 (subtractive).


Using a synthesizer of your choice, create a 20 - 60'' composition focusing on sound texture. This can be any of the synthesizers/patches above*, or any other you are interested in exploring. The only requirement is that it have controls for some of the components we discussed in class. Post your composition here.

Read The Future of music: Credo, by John Cage

Create a found sound library: go out into the city, listen, and record a range of sounds to use in composition - percussive, pitched, different timbres.

* You can find the Max patches from class in our shared folder. Both Max and Ableton Live have free 30-day demos available for download. You can also borrow a production laptop from the ER, with Ableton Live installed. If you are interested in trying the Moog Mother32, sign up for office hours.


Interactive tutorial on synthesis


Timbre: Code + Design. Synthesis and Sampling

Class Meeting

Listen: Assignments

Code: Synthesis

Listen: Sampling

Code: Sampling

Code: Mic input + analysis and visualization

Interact: A recorder+looper. A keyboard sampler.


Design + Code: Create an interactive sketch that explores timbre. It can be synthesizer-based, sample-based, or a combination of both. Record a 20-second composition using it. Post your documentation and code here, and come prepared to do a 5-minute presentation in class.


Free Sample libraries:

  • Aporee
  • London Philharmonic Orchestra Samples
  • Spitfire audio
  • Macaulay library
  • The computer music tutorial, by Curtis Road. Chapter 4: Sampling Synthesis.

    Timbre: Present Projects

    Interactive Timbre Studies

    • Antonio Guimaraes
      read | play
    • Jung Soo Ha.
      read | play
    • Lucas White.
    • Jessica Chase.
      read | play
    • Sylvan Zheng
      read | play
    • Zeyao Li
      read | play

    Harmony: Listen + Interact

    Class Meeting

    Timbre projects: Antônio, Sam, Zeyao

    Listen: consonance/dissonance, drones, parallel voices, triads

    Interact & Discuss, following the interaction design framework in B. Verplank's IxD Sketch Book:


    Apply the Idea/Metaphor/Model/Display/Error/Scenario/Task/Control framework to

    • analyze your previous studies, or
    • start defining/sketching/designing your harmony project (or your final project).
    Post your analysis / design here.


    • Making Music, by Dennis DeSantis. Parallel harmony in deep house, post-dubstep and related styles.
    • A Geometry of Music, by Dmitri Tymoczko, Chapter 1.
    • Designing Interactions, by Bill Moggridge, final chapter.

    Harmony: Code + Design

    Class Meeting


    Listen: Chord Progressions

    Interact & Discuss: Hook Theory / Nova Synth

    Code: Chord progressions


    1. Harmony project

    Design: sketch 3 alternative interfaces to engage with harmony, each following a different paradigm, as described on the IxD Sketch Book, p14 to p22.

    • Computer as Tool (for example, an instrument)
    • Computer as Media (for example, an interactive music video or installation)
    • Computer as Brain (for example, a composition collaborator)
    Code (optional): Implement one of your ideas, and create a 20'' composition using it.

    Post sketches and implementation here.

    2. Final project proposal

    Define and make progress on your final project

    • If you are presenting a series of studies, describe how you are going to improve/refine/iterate on each of them, in terms of interaction design, sound and music design, and visual/tactile design.
    • If you are combining previous projects or creating a new one, come to class with a proposal (inspiration references, concept sketches, mapping diagrams, etc), as well as a working prototype.

    Post proposal and progress here

    3. One-on-one meeting

    Schedule an appointment with me. If none of these slots work for you, please email me so we can find another time.




    • Assignments and Bi-weekly Projects: 50%
    • Participation and Attendance: 40%
    • Final Project: 10%

    Please see ITP's statement on Pass/Fail which states that a "Pass" is equivalent to an "A" or a "B" while anything less would be considered a "Fail".

    Attendance is mandatory. Please inform me via email if you are going to miss a class. Two unexcused absences is cause for failing the class. (An unexcused lateness of 10 minutes or more is equivalent to 1/2 an absence.)

    This class will be participatory, you are expected to participate in discussions and give feedback to other students in class and also participate with their projects. This (along with attendance) is 40% of your grade.

    Class will culminate with final projects. You are expected to push your abilities to produce something that utilizes what you have learned in the class that is useful in some manner to yourself or the world. This will comprise 10% of your grade.