The Code of Music

Fall 2019, NYU ITP

Luisa Pereira | luisa.ph@nyu.edu

Thursdays, 12:25 – 2:55pm

Sign up here

Class Meeting

Introductions + music pieces

Syllabus overview

Space + applications

Activity: do a catalog of attributes of one of your songs. Design an interactive music experience based on it.

Assignments

1. Design: finish your experience concept and document it. Include a title, a one-paragraph description, a sketch/text description of the interaction, and anything else that would help communicate your concept quickly. Submit it using this form.
2. Listen: find a piece of music you find rhythmically engaging. Submit it on this form.
3. Read: This is your brain on music, Chapters 1 and 2.

Resources

Creative Coding Review

• p5 reference
• Daniel Shiffman's video tutorials
• Getting Started with p5.js

• Variables
• If Statements
• For loops
• Arrays
• Functions
• Callback functions

Class Meeting

Present assignments + discuss reading

Listen: beats, tempo, subdivision, accents, syncopation, cross-rhythms

A dimension space for musical devices, as proposed by D. Birnbaum, R. Fiebrink et al.

Group notes

Assignments

Interact:

Pick two rhythm interfaces and interact with each for at least three minutes. These can be digital or physical - here are a some web-based examples:

• TR-808 re-creations (one, two), Type Drummer
• Groove Pizza, Rhythm toy
• Beat Blender, Neural Drum Machine

Place each interface on a dimension space - either D. Birnbaum's, or your own dimension space, defined by any axes you deem relevant. Here is a printable version of the original, labeled dimension space diagram.

Submit your diagram/text responses here: rhythm interface analysis.

Class Meeting

Code: Keeping time. Play a sound over and over

• Use the draw loop
• Create a separate loop
• Clock problem --> Tone.js

Code: The elements of rhythm

• Beats
• Tempo | + slider
• Meter / Time Signature
• Accents
• Syncopation
• Cross-rhythms

Discuss: What possibilities does representing these elements in code open up? Consider interactivity: input gestures and control, visual/tactile representation of each element, mapping, and feedback modes.

Design: Sketch three ideas for a rhythmic interface.

Code: Generative rhythm

• Markov Chains:
  • Using Tone's CtrlMarkov
  • Create a markov matrix from scratch, based on an existing sequence
• Neural Networks: DrumsRNN example

Tutorial: Create a drum machine

• Make a drum pad (click to trigger kick).
  Include Tone.js, import a sound, play it back when the mouse is pressed. Try making a whole drum kit.
  
• Make a beat (four on the floor)
  Create a loop to play the kick drum twice per second. Try making it faster and slower.
  
• Make a drum pattern
  Add more sounds and play them back on specific steps using modullo. Try creating different patterns.
  
• Make a drum machine
  Replace steps by modullo with a pattern matrix. Try different patterns by editing the matrix.
  
• Draw your drum machine
  Draw the drum pattern. Try making it a variable size. Group beats in bars. Add colors and margins. Try your own palette, shapes, images, positions, animations. Make a circular sequencer.
  
• Make it interactive. Click to turn steps on and off.
• Add a full drum kit and interface
  Exercise: Add sliders and buttons to change the tempo, number of measures, time signature, accents, swing, and any other parameter you find relevant. These can be DOM elements or physical controllers.

Assignments

1. Post between 3 and 10 pen-and-paper sketches for possible interfaces, and submit them here.

2. Optional: follow the drum machine tutorial above. Try the prompts and do the suggested exercise at the end. I highly recommend doing this if you are relatively new to programming, javascript, or the libraries we are using.

3. Design + Code: Create an interactive sketch to explore rhythm. Record a 20-second composition using it. Post your documentation and code here, and come prepared to do a 5-minute presentation in class.

Resources

Tone.js:

• Documentation
• Examples
• Transport, Time

Web Audio

• Mozilla's introduction to the Web Audio API
• If you want to dig deeper into scheduling in Web Audio: A Tale of Two Clocks

Neural networks

• Neural Nets for Generating Music
• Tutorial on making music with magenta.js

Interactive Rhythm Studies

Class Meeting

Listen: pitches, notes, modes and scales, intervals

Discuss: The Do/Feel/Know cycle, as descibed in B. Verplank's Interaction Design Sketchbook.

Assignments

1. Listen: Find a piece of music you find melodically engaging. If possible, determine the mode/scale it is in. Draw the melody, denoting the contour created by its intervals.

2. Ideate: Based on the song you picked and one random oblique strategies, sketch an interfaces to explore the elements of melody. Submit song and sketch here (this a pen and paper sketch, not a p5 sketch!).

3. Interact: Pick two melody interfaces and interact with each for at least three minutes. Here are a some screen-based examples:

• Figure (free iOS app)
• Music Lab's Melody Maker, Kandinsky
• Learning Music: Basslines, Melodies

And some physical examples (come ask me):

• Akai LPK25 arpeggiator
• Pocket operator
• A custom, slider-based theremin

Analyze each interface in terms of the Know/Feel/Do cycle we discussed in class. Submit your analyses here.

4. Code (optional): Follow this tutorial to turn the drum machine into a melody sequencer. Steps:

• Replace drum samples with notes. Things to try:
  • Different scales or modes. Include a variable step. Make a microtonal sequencer.
  • Different sounds (a cat meowing? a person talking?)
• Extend your grid. Add octaves and try different grid sizes.
• Add controls. Add DOM controls for picking the scale, range of notes (octaves), and any other variable you come up with.

Share your sketch / sketches here.

Class Meeting

Code: The elements of melody

• Pitches and notes
• Modes and scales

Activity:

• Modes and scales. Create a random melody in different modes. Add a high likelihood for the root note to play. >> Here is Vibert's response, "guess the mode"
• Intervals. Create a random melody by picking a sequence of intervals. Apply this melody to differrent scales.

Discuss: Mapping

Activity: Think through different mappings between the elements of melody and a set of button/slider combinations. Consider cross-couplings. Quickly prototype one of these mappings (here is a random example).

Code: Generative melody

• Markov Chains:
• Using Tone's CtrlMarkov
• From scratch: learn from a given sequence
• From scratch: generate from an existing score
• L-systems:
  • Algae melody
  • Turtle melody
  • OOP example from The Nature of Code
• Neural Networks:
  • MusicRNN format + Tone.Part
  • MusicRNN continue example

Assignments

Design + Code: Create an interactive sketch to explore melody. Record a 20-second composition using it *. Post your documentation and code here, and come prepared to do a 5-minute presentation in class.

* here is a tutorial Jessica wrote on how to record your internal audio, in case you are using Quicktime.

Resources

Readings:

• Don't Forget the Laptop: Using Native Input Capabilities for Expressive Musical Control, by Rebecca Fiebrink et al.
• Grammars as Representations for Music, by Curtis Roads
• The Nature of Code, Chapter 8, Fractals, by Daniel Shiffman

Code:

• Scheduling examples
  • Event example | documentation
  • Loop example | documentation
  • Part example | documentation
• Tool to convert MIDI into Tone.js-friendly JSON (used to generate the scores in the Markov example)

Interactive Melody Studies

Class Meeting

Listen:

• Pitched sounds vs percussive sounds
• Synthesis techniques: additive, subtractive and FM synthesis
• Synthesizer components: oscillators, amplifiers, filters, envelopes and LFO's

Interact:

• Spectrogram with examples of different timbres
• A tool to explore harmonics
• Synthesizers:
  • Digital, custom: Max patches for each synthesis technique.
  • Digital, off the shelf: Ableton's Analog (subtractive) and Operator (a combination of subtractive, additive, and FM).
  • Analog, semi-modular: Moog Mother32 (subtractive).

Assignments

Using a synthesizer of your choice, create a 20 - 60'' composition focusing on sound texture. This can be any of the synthesizers/patches above*, or any other you are interested in exploring. The only requirement is that it have controls for some of the components we discussed in class. Post your composition here.

Read The Future of music: Credo, by John Cage

Create a found sound library: go out into the city, listen, and record a range of sounds to use in composition - percussive, pitched, different timbres.

* You can find the Max patches from class in our shared folder. Both Max and Ableton Live have free 30-day demos available for download. You can also borrow a production laptop from the ER, with Ableton Live installed. If you are interested in trying the Moog Mother32, sign up for office hours.

Resources

Interactive tutorial on synthesis

Readings:

• The computer music tutorial, by Curtis Road. Part II: Synthesis (available at NYU library)

Class Meeting

Listen: Assignments

Code: Synthesis

• Noise + Volume Envelope. Also see NoiseSynth and MembraneSynth
• Tone + Volume Envelopes. Also see Synth
• Additive Synthesis
  • Two Oscillators
  • + Volume Envelopes
  • Six Oscillators + Envelopes. If adding frequencies from the harmonic series, you can simply partials (harmonics) to one Oscillator (look into oscillator.partials)
• Subtractive synthesis
  • Oscillators
  • Oscillator + ADSR + Filter
  • + Filter Envelope
  • + LFO
  • Also see MonoSynth, which has these features built in. For LFO effects, see Tremolo (LFO affecting volume), Vibrato (LFO affecting pitch), and AutoFilter (LFO affecting the filter)
• FM Synthesis: try creating different timbres using FMSynth

Listen: Sampling

Code: Sampling

• Player
• Sampler, + repitch, + more samples
• Granular Synth
• Effects

Code: Mic input + analysis and visualization

• Mic + Waveform
• Mic + FFT
• Mic + Meter

Interact: A recorder+looper. A keyboard sampler.

Assignments

Design + Code: Create an interactive sketch that explores timbre. It can be synthesizer-based, sample-based, or a combination of both. Record a 20-second composition using it. Post your documentation and code here, and come prepared to do a 5-minute presentation in class.

Resources

Free Sample libraries:

The computer music tutorial, by Curtis Road. Chapter 4: Sampling Synthesis.

Interactive Timbre Studies

Class Meeting

Timbre projects: Antônio, Sam, Zeyao

Listen: consonance/dissonance, drones, parallel voices, triads

Interact & Discuss, following the interaction design framework in B. Verplank's IxD Sketch Book:

• Chords: on a grid, on a piano keyboard, on the circle of fifths, on a tonnetz diagram.

Assignments

Apply the Idea/Metaphor/Model/Display/Error/Scenario/Task/Control framework to

• analyze your previous studies, or
• start defining/sketching/designing your harmony project (or your final project).

Resources

• Making Music, by Dennis DeSantis. Parallel harmony in deep house, post-dubstep and related styles.
• A Geometry of Music, by Dmitri Tymoczko, Chapter 1.
• Designing Interactions, by Bill Moggridge, final chapter.

Class Meeting

Code:

• Consonance / Dissonance
  • Just intonation + Lissajous
  • Add mic input
• Drones
  • Over random melody
  • Add silences
• Parallel harmony
  • Fixed
  • Variable interval
• Three-note chords (Triads)
  • Arbitrary intervals, parallel
  • Diatonic: major and minor chords
  • Towards abstraction (a first step) - > libraries (see Resources)
• Four-note chords
  • Diatonic: major and minor seventh, diminished chords.

Listen: Chord Progressions

Interact & Discuss: Hook Theory / Nova Synth

Code: Chord progressions

• Passamezzo moderno
• Same, arpeggiated

Assignments

1. Harmony project

Design: sketch 3 alternative interfaces to engage with harmony, each following a different paradigm, as described on the IxD Sketch Book, p14 to p22.

• Computer as Tool (for example, an instrument)
• Computer as Media (for example, an interactive music video or installation)
• Computer as Brain (for example, a composition collaborator)

Post sketches and implementation here.

2. Final project proposal

Define and make progress on your final project

• If you are presenting a series of studies, describe how you are going to improve/refine/iterate on each of them, in terms of interaction design, sound and music design, and visual/tactile design.
• If you are combining previous projects or creating a new one, come to class with a proposal (inspiration references, concept sketches, mapping diagrams, etc), as well as a working prototype.

Post proposal and progress here

3. One-on-one meeting

Schedule an appointment with me. If none of these slots work for you, please email me so we can find another time.

Resources

• Theory Libraries
  • Tonal
  • Teoria.js
• On Chord Progressions
  • Interactive: song walk-throughs
  • Video: Giant Steps by John Coltrane
  • Video: Modal interchange