This course explores music through the lenses of computation and interactivity.
The first part of the semester consists of a structured exploration of rhythm, melody, timbre, and harmony, from the perspectives of code, design, and music theory. For each musical element, we will hold listening sessions, represent and manipulate the element in code, and create an interactive study around it.
The second half of the semester we will explore algorithmic composition techniques such as Markov Chains, Neural Networks and L-systems. The last three weeks of class will be dedicated to the development of a final project ––an interactive/generative musical piece.
In-class coding and assignments will be done in p5.js and Tone.js. Introduction to Computational Media (ICM) or equivalent programming experience is required.
This class is a good fit for students who are interested in:
Term
Spring 2021
Interactive Media Arts - NYU Tisch
Instructor
Luisa Pereira | luisa.ph@nyu.edu
GA
Yona Ngo | phuong.ngo@nyu.edu
Meetings
Mondays and Wednesdays
3:30 - 5pm EST
Office Hours
Luisa: Sign up here
Yona: Sign up here
Assignments will range from readings to design and programming exercises. They are required and you should be prepared to show/talk about them in class. You will be expected to post them to your blog the night before they are due.
Midterm and Final ProjectsPrompt: Create an interactive/generative musical piece or instrument.
Prepare a short presentation describing your musical, design, and programmatic decisions and process.
Assignments are due the day before class. Please post all assignments using this form. You can find your classmates' posts on this spreadsheet.
This section will be updated weekly with references, code examples and assignment details.
Introductions + music pieces (you can find all pieces here)
Class approach and syllabus overview
Activity: Inspiration Projects - Interact + Discuss.
1. In pairs: get ready to share the project you experienced + discussion highlights with the class on Wednesday.
2. If you don't have one already, set up a blog or web page where you can post your assignments for this class ––see resources below. If you need any help, please contact Yona (see office hours and contact info on the top right of this page).
3. If you are a beginner programmer or are not yet familiar with p5.js, take some time to do a personal Creative Coding review this week. Make sure to review the following concepts in p5.js/Javascript (see resources below):
Creating a website
Creative Coding Review
Inspiration Projects
Audio-visual instruments / compositions
Playback and progress
Playback and visualization
W1.1. Audio-visual instrument
Submit your blog post by Sunday night
AV Instruments
Activity: catalog the elements of the musical piece you brought in
Activity: based on the catalog, sketch an interactive experience
W2.1. Post your catalog and experience concept to your blog
W2.2. Keeping in mind the examples from class, reflect on the possibilities that open up by representing and manipulating musical tracks and loops in code. Then:
A few suggestions:
W2.3. Find a song / musical piece you find rhythmically interesting. Submit it via our form.
All assignments are due on Tuesday night (Feb 16). They can be done individually or in pairs.
Tone.js Quick Start | API Reference | Examples
ffmpeg, a command-line tool you can use to convert wav files to mp3 in batches. Here is the command I used:
$ for f in *.wav ; do ffmpeg -i "$f" -acodec libmp3lame -q:a 2 "${f%.*}.mp3"; done
Listen: Elements and Examples
Discuss: What possibilities does representing these elements in code open up? Consider interactivity: input gestures and control, visual/tactile representation of each element, mapping, and feedback modes.
Design Activity: in pairs, sketch two ideas for interactives to explore rhythm
W3.1. Finish your concept sketches and post them to your blog
Tutorial: Create a Drum Machine, Step by Step
Present + Discuss: students' rhythm interactive sketches
Code: make the drum machine code your own (suggestions in prompts above)
W3.2. Interactive Rhythm Study. Design and implement an interactive exploration of rhythm. Once you're done, record a 20-second composition/improvisation using it. On your blog post, you can include some text (your inspiration/motivation), sketches, a brief description of your sound, visual, and interactive design decisions, as well as any challenges and questions that came up.
W3.3. Find a song / musical piece you find melodically interesting. Submit it via our form.
Interact: Four web interfaces
Listen: Elements and examples
Code:
Activity: Listen to the melodically engaging piece you posted. Draw its melody, using your own visual language
W4.0. Finish your melody drawing, and post it to your blog, along with a link to the piece itself.
W4.1. Sketch two ideas for interactives to explore melody.
Video: Learn music theory in half an hour (a short introduction to chord progressions and melodies, recommended by a student)
Code: From drum machine to melody sequencer
Activity: Share your interactive melody ideas with a partner. Together, pick one for them to share with the rest of the class. Switch roles.
Activity: Starting from the first Sequencer sketch above, pick one or two "things to try" suggestions and implement them. Once you are done, move on to the next sketch and do the same.
From class: Finish your sequencer exercise (or exercises) and post it to this document.
W4.2 Melody Study. Design and implement an interactive exploration of melody. Record a 20-second composition/improvisation using it. Post the running sketch, code, and documentation to your blog.
W4.3 Listen: find a piece of music you find interesting in terms of timbre, and post it using our assignment form.
Quantization in Tone.js
Monday, March 15: no assignments
Wednesday, March 17:
The computer music tutorial, by Curtis Road. Part II: Synthesis (available at NYU library); Chapter 4: Sampling Synthesis.
Code: Sampling
Code: Mic input + analysis and visualization
Finish assignments 5.1., 5.2, 5.3. The sketches and interactive study can focus either on Synthesis or on Sampling
Free Sample libraries:
Elements and examples
Five web interfaces:
Code:
Wednesday Workshop: worksheet exercises
1. Finish at least three of the exercises in the worksheet from class. Post them by linking to your worksheet.
2.
Student Projects
Demo: Refactoring Code
Review and iterate on your project's code, or swap with a classmate. Start by adding comments, then look for repeated code, variable and function names that could be clearer, code that can be organized better (by creating functions, for example), and so on. The goal is that the code is better organized / more succinct, and the sketch does exactly what it did before.
Post documentation to your blog, including a link to the original code, a link to the 'refactored' code, and a description (can be bullet points) of the changes you did.
Musical systems - a brief history.
Randomness, Probability, Algorithms, Interactivity.
Markov Chains, musical applications
The Computer Music Tutorial, by Curtis Roads (available at NYU library)
Code:
Grammars and L-Systems, musical applications
Due Monday, May 3:
Neural Networks, musical applications
Magenta Project
Continue sequences, generate from scratch
Also see ImprovRNN and PerformanceRNN
See demos here (all have code and documentation).
Due Monday, May 10:
Code:
Workshop: How to Host your Project on Github, by Yona Ngo.
Finish your final project (either a Neural Networks study or an extension of a previous project). When you are done, record a 20-second composition using it. Come prepared to do a 5-minute presentation in class. You can include:
The live demo is the main part of your presentation: the other sections provide context, but are secondary and can be brief. In preparing, keep in mind you only have 5 minutes to present.
Post your code, video and documentation to your blog (you can follow the same outline as your class presentation); submit using our usual form.
You will need a modern laptop (4 years old or younger is a good rule of thumb). Limited numbers are available for checkout from the department. Commercial software required in the course will be made available on the laptops available for checkout. Other required software is freely available.
Extensive online resources will be available on this course website
Grades for the course will follow the standard A through F letter grading system and will be determined by the following breakdown:
At most two (2) unexcused absences will be tolerated without effect to your grade. Any more than two (2) unexcused absences will result a lowering of your final grade by one whole grade for each unexcused absence. For example, three (3) unexcused absences will result in your highest possible grade being a B instead of an A. Four (4) unexcused absences will result in your highest possible grade being a C and so on. Six (6) unexcused absences would result in an automatic F for the course. Two (2) late arrivals will count for one (1) absence.
This class will be highly participatory. You are expected to contribute to discussions, engage in group work, give feedback to your peers, and otherwise fully participate in class.
Recreational use of phones, music players, video game systems and other devices are forbidden. Laptops and tablets are allowed for note taking, in class work, as well as relevant research only. Activities not related to the class, such as recreational use of the internet, including all social media websites, email and instant messaging, game playing, and work for other classes, will not be permitted. Such activities are disrespectful to the instructor and distracting to others. Your laptop should always be closed whenever a fellow student is presenting.
Classes will be a mixture of lecture, discussion, hands-on tutorials, homework review, presentations, and group work. You will need to come to class prepared with a laptop and any other supplies specified for that class.
The course will be two (2) times per week for one hour and thirty minutes (1:30) for a total of 14 weeks.
There will be regular assignments that are relevant the class material. These assignments must be documented (written description, photos, screenshots, screen recording, code, and video all qualify based on the assignment) on the class website. Each assignment is due by the next class unless stated otherwise.
It is expected that you will spend 6 to 8 hours a week on the class outside of class itself. This will include reviewing material, reading, watching video, completing assignments and so on. Please budget your time accordingly.
Each assignment will be marked as complete (full credit), partially complete (half credit), or incomplete (no credit). To be complete an assignment should meet the criteria specified in the syllabus including documentation. If significant portions are simply not attempted or the assignment is turned in late (up to 1 week) then it may be marked partially complete. If it is more than a week late, not turned in, or an attempt isn’t made to meet the criteria specified it will be marked incomplete.
Responses to reading and other written assignments are also due in class one week after they are assigned and must also be submitted via the class website. Written assignments are expected to be 200 to 500 words in length unless otherwise specified. Grading will follow the same guidelines as above; on time and meeting the criteria specified will be marked as complete. Late (up to 1 week) or partially completed work will be given half credit. Work that is more than a week late, not turned in, or fails to meet the criteria specified will be given no credit.
Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else. More information can be found on Tisch’s page regarding Academic Integrity.
Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.
Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYUWellness Exchange 212-443-9999.