Studying Music Performance: A Probabilistic Approach David Temperley Eastman School of Music University of Rochester Four goals of music performance: 1. 2. 3. 4. Render the notes (and other score information) correctly Convey the emotional content of the piece Clarify the structure of the piece Modulate the flow of information Four goals of music performance: 1. 2. 3. 4. Render the notes (and other score information) correctly Convey the emotional content of the piece Clarify the structure of the piece Modulate the flow of information The topic of today’s talk Part I: Clarification of structure • Three examples • The circularity problem (performance perception or perception performance?) • Ambiguity avoidance Clarification of structure in music performance Many authors have observed that one goal of a good performance is to convey the structure of the piece (Sloboda 1985, Clarke 1988, Gabrielson 1999). (“Structure” might include many things—meter, harmony, phrase structure, form, motivic connections, etc.) Sometimes this point is made indirectly—e.g. when it is said simply that a good performer must be aware of the structure of the piece. (Why is it beneficial to be aware of this? Presumably so that it can be conveyed to the listener.) Clarification of structure – Example 1: Accentuation of metrically strong events The perception of meter is a complex process, involving a variety of different cues or “phenomenal accents” (Lerdahl & Jackendoff, 1983): length, loudness, articulation, harmony, parallelism... Performers can facilitate this process by aligning phenomenal accents with strong beats—that is, by emphasizing metrically strong events. Clarification of meter, cont’d. The same notes (pitches and rhythms) in different metrical contexts will be played quite differently: metrically strong notes are louder, longer, more legato (Sloboda 1983, 1985). …And listeners can identify which melody is being performed. Clarification of meter: Dynamics Performers tend to play metrically strong notes somewhat louder than others, though the difference is relatively small. Data from Drake & Palmer (1993) for piano performance, showing average “hammer velocity” values for notes on beats 1-4 of a 4/4 measure. (Note: range for entire piece was 30 to 600 units!) Clarification of structure – Example 2: Phrase-final lengthening Performers tend to decrease tempo at phrase boundaries, and more markedly at larger sectional boundaries (Todd 1985; Palmer 1996; Windsor & Clarke 1997). (smooth curve) (jagged line) phrase boundaries Data from Windsor & Clarke 1997 Clarification of structure – Example 3: Individuation of melodic lines The perception of polyphonic music requires the identification of individual lines—voice separation or “contrapuntal analysis” (Temperley 2001). Performance teachers (of polyphonic instruments) often urge students to individuate different lines of the texture in some way—for example, by emphasizing the melody. A specific case in point is melodic lead—the tendency of the melody notes to anticipate other notes (Palmer 1996). This could be an strategy to individuate melodic notes by making them asynchronous with other notes. According to the “clarification of structure” view, these phenomena (accentuation of metrically strong events, phrase-final lengthening, melodic lead) arose, at least in part, as ways of clarifying musical structure for the listener. A possible circularity in this argument: therefore Listeners expect metrically strong notes to be louder / longer Performers play metrically strong notes louder / longer A possible circularity in this argument: therefore Listeners expect metrically strong notes to be louder / longer therefore Performers play metrically strong notes louder / longer How to address the circularity problem? Three responses... 1. Perhaps the process really is circular. Performance tendencies gave rise to perceptual tendencies, but were then reinforced by them. 2. Perceptual tendencies may have come first and may have originated outside music, e.g. in language: - Stressed syllables are normally longer and louder (Fry 1955). - Phrase-final lengthening is a cue to syntactic boundaries in speech (Lehiste 1973). 3. The case for perception influencing performance is strengthened if it can be shown that the performance tendencies are strongest in cases that would otherwise be ambiguous perceptually. 3. The case for perception influencing performance is strengthened if it can be shown that the performance tendencies are strongest in cases that would otherwise be ambiguous perceptually. Ambiguity avoidance is well-documented in language: For example, speakers exaggerate prosodic marking of phrase structure (e.g. lengthening at syntactic boundaries) when a sentence is ambiguous in context (Snedeker & Trueswell 2003). Ambiguity Avoidance in Music – Example 1: Organ performance On instruments where dynamic accentuation is not possible, do performers increase other kinds of accentuation on metrically strong notes? In organ pedagogy, performers are taught to emphasize metrically strong notes through duration (i.e. with longer inter-onset intervals to the following note—[A]) or articulation (playing them slightly more legato— [B]) (e.g. Hurford 1988). : . : . A B Ambiguity Avoidance – Example 2: Phrase-final lengthening at ambiguous points Do performers exaggerate phrase-final lengthening in cases where the phrase boundary would otherwise be unclear or ambiguous? Brahms, Intermezzo Op. 116 No. 6, mm. 1-8 Rubinstein’s performance (1959) Question: Why does Brahms mark sostenuto and a fermata at the phrase ending in mm. 7-8 – when the performer would normally slow down at a phrase boundary anyway? Answer: Given the unusual harmonic context (the sudden move to C# major), the boundary might not easily be noticed here unless especially reinforced by the performer. Schumann, “Von fremden Ländern und Menschen” from Kinderszenen Question: Why does Schumann mark ritardano before the return of the opening theme? Answer: The return happens at an unexpected place in the phrase structure—after only a six-measure phrase—and with no clear change of harmony; thus the structural boundary might be missed if it were not especially marked by phrase-final lengthening. Ambiguity Avoidance – Example 3: The rise of “swing tempo” in jazz Swing tempo: The lengthening of the first half of the beat in relation to the second. Maple Leaf Rag, as performed by Scott Joplin (1918) and Jelly Roll Morton (1938) (my transcriptions) Excerpt 1 (Joplin) (Morton) “Maple Leaf Rag”, Excerpt 2 (Joplin) (Morton) Morton’s performance features a great deal more syncopation than Joplin’s—especially left-hand events on weak eighth-note beats; this potentially obscures the quarternote level of the meter. But it also features a swing tempo, which clarifies the quarter-note level of the meter (by making strong-beat notes longer than weak-beat ones). And in general, the transition from ragtime to jazz features... • a shift from the “even-note” rhythms of ragtime to the “swing” feel of jazz (Schuller, 1967) • an increase in the degree and complexity of syncopation (Waterman, 1974; Sales, 1984) In each of these cases of ambiguity avoidance... - Accentuation of metrically strong notes through length and articulation in organ performance - “Disambiguating” ritards marked at potentially ambiguous phrase boundaries - The rise of swing tempo in jazz ...we see increased use of “clarifying” performance cues in situations that would otherwise be ambiguous—supporting the idea that these cues arose, in part, as a way of clarifying structure. Part II: Modulation of Information Flow Information is the negative log of probability. So in a sequence of events, the lower the probability of an event, the more information it conveys. (Also known as surprisal...) Information 6 5 4 3 2 1 0 Probability 1.0 The Uniform Information Density Hypothesis (Levy & Jaeger, 2007): A certain rate of information flow is optimal in language perception, and language production tends to be adjusted to maintain this rate. Uniform Information Density in Language (Levy & Jaeger, 2007) In prosody, words that are more probable are pronounced more quickly: - More frequent words are shortened (Jurafsky et al., 2001) - Words that are more probable in local context (e.g. P(wn | wn–1)) are shortened (Jurafsky et al. 2001; Aylett & Turk, 2004) - Words that are more probable given larger context (e.g. that have already occurred in the discourse) are shortened (Fowler & Jousum, 1987) Modulation of information flow: What does this predict for music performance? One prediction: Less probable events should be performed more slowly. Modulation of information flow: What does this predict for music performance? One prediction: Less probable events should be performed more slowly. What about unexpected harmonies, e.g. chromatic harmonies? Sundberg’s (1988) rules for good expressive performance include the “Harmonic Charge Rule”: Slow down on chords whose pitches are far from the tonic on the circle of fifths. (Never really tested with regard to performance.) Expressive performance as modulation of information flow: Anecdotal evidence Brahms, Intermezzo Op. 116 No. 6, mm. 1-8 (again) Question: Why does Brahms mark sostenuto and a fermata at the phrase ending in mm. 7-8? Possible Answer: Given the modulation to a distant key and the rapid harmonic rhythm, this passage is high in information content; slowing down smooths out the information flow. How does harmony affect expressive timing? Do performers slow down on unexpected harmonies? Chris Bartlette’s dissertation (Eastman, 2007) examined this experimentally. Skilled pianists were given pairs of passages to practice and perform. Within each pair, the passages were identical except for one chord, which was diatonic (within the key) in one case and chromatic (outside the key) in the other case. Effect of harmony on performance expression was examined (timing, dynamics, asynchrony). Examples (Barlette, 2007) [Title] œœœ œ #### 6 œ. œ œ œœœœœœ œ œ œœœ œ œ(above) and a chromatic The second & half of 8 a diatonic passage J J passage (below). (MIDI files generated from experimental trial, œ. œ œ œ œ. of performance.) œ œ œ œ reflecting ?actual timing and dynamics # œ ## # 6 œ œœ œ. œ œ œ 8 œ [Title] œœœœœ œ œ œ œ œœ œœœœ œ œ œœœ ###### 6 œ. œ œœ œ œ œ œ œ œ Ý. J # 8œ œ œ œ œ œ œ œ & # J & œ. œ œ œ œ . œ œœ œ œ . œ ?? ###### 6œ.œœ œ. œœ œœ œ œ . . . œ œ œ œ œ œ. œ. ##8 E: ii6 #### œ œ œ œ œ œ œ œnœnœ œ œ œ œ œ Ý. œ œ œ & ? #### œ. E: œ. œ. œ. bII6 œ. œ. œœ œ œ. Result: Both the “chord of interest” and the previous chord are lengthened when the chord of interest is chromatic. Cn-1 Cn (“chord of interest”) Cn+1 Why? To allow more time for the chord to be processed... “smoothing the information flow.” Various sources of evidence... - Sundberg’s rules - Bartlette’s dissertation - anecdotal evidence from score markings ...suggest a preference in performance to slow down at unexpected harmonies. This can be seen as an example of modulation of information flow. Part III. The Syncopation-Rubato Trade-Off Syncopation - non-alignment between beats and phenomenal accents Rubato - irregularity or fluctuation in tempo for expressive purposes The Syncopation-Rubato Trade-Off: Music with more syncopation seems to permit less rubato. Rhythm in traditional sub-Saharan African music (compared to Western classical music) - Greater use of syncopation (Jones 1959, Chernoff 1977, other authors) - Greater strictness of tempo (less rubato): “When we Europeans imagine we are beating strict time, the African will merely smile at the ‘roughness’ of our beating.” (Jones 1959, p. 38) Also: rock and jazz - a high degree of syncopation (compared to classical Western music); very little tempo fluctuation. An Experimental Study of the Syncopation-Rubato Trade-Off (with Kelly Francis) Subjects: Doctoral students in piano, Eastman School of Music Procedure: Subjects were given short passages and asked to perform them (after practicing). The variance in tempo within each performance was then analyzed. Materials: Four pairs of musical phrases, with the same pitches in the same approximate positions; one pattern in each pair is unsyncopated (1A), the other is highly syncopated (1B). (Deadpan MIDI performances) Results Pianists played unsyncopated excerpts with significantly more variance in tempo than syncopated excerpts (F(1, 9)=8.17, p=.02). The effect varied greatly across excerpt pairs: 0.045 Tempo variance 0.04 0.035 0.03 0.025 U ns ync opated Sync opated 0.02 0.015 0.01 0.005 0 1 2 3 Excerpt pair 4 THE BIG QUESTION: The syncopation-rubato trade-off—is it clarification of structure, OR modulation of information flow? THE BIG QUESTION: The syncopation-rubato trade-off—is it clarification of structure, OR modulation of information flow? Clarification of structure: Could we argue that the combination of rubato and syncopation obscures the metrical structure more than either one alone? The syncopation rubato trade-off as clarification of structure A. A perfectly regular pattern B. Syncopation: one event occurs slightly before the beat C. Rubato: intervals between beats are slightly irregular D. Syncopation and rubato combined: the last beat is slightly late, the last event slightly anticipates it (now the onset pattern is the same as in A!) 600 ms 600 ms 600 ms 600 ms 600 ms 600 ms 450 ms 600 ms 600 ms 800 ms 600 ms 600 ms 800 ms 600 ms intended rhythm beats note onsets The syncopation rubato trade-off as clarification of structure A. A perfectly regular pattern B. Syncopation: one event occurs slightly before the beat C. Rubato: intervals between beats are slightly irregular D. Syncopation and rubato combined: the last beat is slightly late, the last event slightly anticipates it (now the onset pattern is the same as in A!) 600 ms 600 ms 600 ms 600 ms 600 ms 600 ms intended rhythm beats note onsets 450 ms 600 ms 600 ms 600 ms 600 ms 800 ms 800 ms 600 ms So...syncopation and rubato in combination obscures the meter. While there are cases where the combination of syncopation and rubato obscures the meter, it is difficult to make a general argument to this effect. What about modulation of information flow? The Syncopation-Rubato Trade-Off as Modulation of Information Flow Imagine a one-second segment of a melody. The listener must predict when the next note will occur. q q P The Syncopation-Rubato Trade-Off as Modulation of Information Flow Imagine a one-second segment of a melody. The listener must predict when the next note will occur. q P q P High information—all locations are equally likely P Low information—one location is much more likely than all others In a style with no syncopation or rubato, the location of each note is highly predictable. (Low information) q q In a style with rubato, the beat location itself is unpredictable so the note location is as well. (Moderate information) P q q In a style with syncopation, note locations become less predictable because a note may occur on a weak beat. (Moderate information) P q q In a style with rubato and syncopation, the possibility of notes on weak beats combined with the variation in tempo creates high uncertainty. (High information) q q P In a style with rubato and syncopation, the possibility of notes on weak beats combined with the variation in tempo creates high uncertainty. (High information) q q Perhaps the combination of syncopation and rubato is avoided to keep information flow at a moderate level. P Conclusions Two goals of music performance: - Clarification of structure - Modulation of information flow The syncopation-rubato trade-off: Could be clarification of structure, could be modulation of information flow; more likely the latter. Thank you! References Aylett, M. and Turk, A. 2004. The Smooth Signal Redundancy Hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):31–56. Bartlette, C. 2007. "A Study of Harmonic Distance and Its Role in Musical Performance." Ph.D dissertation, Eastman School of Music, University of Rochester. Chernoff, J. M. 1979. African Rhythm and African Sensibility. Chicago: Chicago University Press. Clarke, E. 1988. Generative principles in music performance. In J. A. Sloboda, (Ed.), Generative processes in music: The psychology of performance, improvisation, and composition (Oxford: Clarendon), pp. 1–26. Drake, C., & Palmer, C. 1993. Accent structures in music performance. Music Perception 10, 343-78. Fowler, C., & Housum, J. 1987. Talkers’ signaling of ‘new’ and ‘old’ words in speech and listeners’ perception and use of the distinction. Journal of Memory and Language, 26, 489-504. Fry, D. 1955. Duration and intensity as physical correlates of linguistic stress. Journal of the Acoustical Society of America 27, 765-768. Gabrielsson, A. 1999. “The Performance of Music”, in D. Deutsch, Ed., The Psychology of Music (San Diego: Academic Press), pp. 501602. Hurford, P. 1988. Making music on the organ. Oxford: Oxford University Press, 1988. Jones, A. M. 1959. Studies in African Music. London: Oxford University Press. Jurafsky, D., Bell, A., Gregory, M., & Raymond, W. 2001. Probabilistic relations between words: Evidence from reduction in lexical production. In J. Bybee & P. Hopper (Eds.), Frequency and the Emergence of Linguistic Structure (Amsterdam: John Benjamins), pp. 22954. Lehiste, I. 1973. Phonetic disambiguation of syntactic ambiguity. Glossa, 7, 107-21. Lerdahl, F., & Jackendoff, R. 1983. A Generative Theory of Tonal Music. Cambridge, MA: MIT Press. (cont’d on next slide) References (cont’d.) Levy, R., & Jaeger, F. 2007. Speakers optimize information density through syntactic reduction. Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems. Palmer, C. 1996. On the assignment of structure in music performance. Music Perception 14, 21-54. Repp, B. 1996. Patterns of note onset asynchronies in expressive piano performance. Journal of the Acoustical Society of America, 100, 3917-3932. Roederer, J. 1975. Introduction to the physics and psychophysics of music. New York: Springer. Sales, G. 1984. Jazz: America's Classical Music. Englewood Cliffs, N.J.: Prentice-Hall. Schuller, G. 1968. Early Jazz. New York: Oxford University Press. Sloboda, J. 1983. The communication of musical metre in piano performance. Quarterly Journal of Experimental Psychology 35, 377-96. Sloboda, J. 1985. The Musical Mind. Oxford: Clarendon Press. Snedeker, J., & Trueswell, J. 2003. Using prosody to avoid ambiguity: Effect of speaker awareness and referential context. Journal of Memory and Language 48, 103-30. Sundberg, J. 1988. Computer synthesis of music performance. In J.A. Sloboda (Ed.), Generative Processes in Music, 52–69. Oxford: Clarendon Press. Temperley, D. 2001. The Cognition of Basic Musical Structures. Cambridge, MA: MIT Press. Temperley, D. 2007. Music and Probability. Cambridge, MA: MIT Press. Todd, N. P. M. 1985. A model of expressive timing intonal music. Music Perception 3, 33-58. Waterman, G. 1974. Ragtime. In N. Hentoff and A. J. McCarthy (eds.), Jazz: New Perspectives on the History of Jazz by Twelve of the World’s Foremost Jazz Critics and Scholars, pp. 43–57. New York: Da Capo Press. Windsor, L., & Clarke, E. 1997. Expressive timing and dynamics in real and artificial musical performances: Using an algorithm as an analytical tool. Music Perception, 15, 127-152.