Using Semantics to Interpret Music

Using Semantics to Interpret Music

09.06.2017, by
Conductor Ryan McAdams directing music for the film "The Tree of Life" by Terrence Malick (New York, November 16, 2016).
Music cannot be reduced to a combination of notes and rhythms that can trigger emotions. It is also a form of language likely to convey information. A new discipline, music semantics, focuses on this phenomenon.

Between our carefully chosen playlists and the pieces that we hear in ads, radio shows and films, music is part of our lives. But what does it tell us? "Music transmits information that is highly abstract, often interpreted in terms of emotions," says Philippe Schlenker, a senior researcher at the Institut Jean Nicod1 and a global distinguished professor at New York University. "Yet there’s more to it than that. For example, a lower volume and slower tempo indicate that a piece is drawing to a close. Is this merely a matter of convention, which may vary from one culture to another, or does music contain messages that are interpreted at a deeper, more biological level?" By transposing the methods of linguistics to the study of musicology, the scientist seeks no less than to develop a semantics of music.2

The music is conceived as an "animated auditory scene," from which the listener will elicit extramusical information.

While theory and harmony can be considered the elements of its "syntax," semantics is the study of meaning. It focuses on the non-musical information conveyed by musical sounds, the information that makes listening to music more than a matter of simply experiencing its notes and rhythms. Presented at conferences held at the IRCAM3 in Paris and the Barenboim-Said Academy in Berlin, Schlenker’s work has also been published in the journal Music Perception, and an extended version of his paper is set to appear in Review of Philosophy & Psychology.

Musical inferences

Schlenker did not base his work on the best-known discipline in the field, namely traditional linguistic semantics, but rather on iconic semantics, which concerns images, gestures and certain properties of sign languages. He conceives of music as an "animated auditory scene," and looks for the inferences that it generates, i.e. the extramusical information that it transmits. For example, a  drop in volume leads the listener to conclude that the source is losing energy, and the piece or movement is coming to an end.
"However, when we listen to music, we are not necessarily interested in its actual source, in other words the different instrumentalists," Schlenker adds. "On the contrary, we postulate something more abstract, an underlying source that explains the motion of the music. It can be a single source if there is only one voice, but music often comprises several melodies, each forming a voice."

The Carnival of the Animals by Camille Saint-Saëns, in which each movement is associated with an animal, is another example. None of the voices has a kangaroo or a hen as its actual source, but the music clearly evokes various creatures. The inference is based on virtual sources. In extramusical listening, the low frequencies are usually produced by larger objects, for which reason the very low sound of the double bass can evoke the elephant mentioned in the title of the movement.

With the help of the composer Arthur Bonetto, Schlenker was able to adapt a method commonly used in linguistics and psychology: minimal pair analysis, which consists in comparing the impact of a slight modification between two otherwise identical elements. Using the MIDI format, which enables a wide range of musical manipulations, they succeeded in transforming the selected phrases as desired. Without violating the rules of harmony, they changed the pitch of the phrases as well as certain chords. Three octaves higher, the sound of the double bass no longer evokes an elephant at all. 

Extrait du Carnaval des animaux, de Camille Saint-Saëns (mouvement de l’éléphant)

Année de production: 

The origin of the inferencesFermerAn inference is an operation that consists in admitting a proposition because of its link with a prior proposition held to be true. can also be tonal in nature. For the tortoises movement in The Carnival of the Animals, Saint-Saëns used a slowed-down, dissonant version of the "French Cancan," originally the "Infernal Galop" from Offenbach’s Orpheus In The Underworld, a choice that intensifies the impression of clumsiness associated with tortoises.

While the notion of dissonance seems obvious in the field of musicology, it has no equivalent in nature. Yet it has a meaning and therefore enters into the study of musical semantics. Research could help interpret the rules of traditional musicology for the purposes of psychoacoustics.

From practice to theory 

How do those concerned, in other words composers, feel about it? The Hollywood-based French composer and orchestrator Sacha Chaban has worked on box office hits such as Get Out and Guardians of the Galaxy Vol. 2. His techniques are derived not only from his extensive classical training but also from empirical "tricks of the trade" shared among professionals.

Film music lends itself particularly well to semantic analysis, since elements of the script are translated directly into music. Here, the composer Sacha Chaban at a studio rehearsal.
Film music lends itself particularly well to semantic analysis, since elements of the script are translated directly into music. Here, the composer Sacha Chaban at a studio rehearsal.

"Music is a sonic creation of space, time and emotions," he says. "A composer can apply musical theories, draw inspiration from them, or go beyond them—as well as control the space: the high-pitched sounds seem to come from above, and low sounds from below. With stereo, reverb and the placement of the musicians and sounds, you get all three dimensions. The tempo, and therefore time, is in the hands of the composer as well."

With stereo, reverb and the placement of the musicians and sounds, you get all three dimensions.

Whether instrumental, vocal, synthetic or concrete (recorded), the extensive range of techniques available to the composer allows them to determine everything down to the texture of the sound. For the soundtrack of his most recent film, Breaking & Exiting, Chaban used and transformed the sound of a crowbar to create the main percussion tracks for the break-in sequences.

He also points out that music has psychoacoustic effects on the viewer/listener. The association of melodies with certain characters and the use of specific instruments, types of composition or effects can underline the setting—the time and place—of each scene. In this way, aspects of the script are represented directly in the music, reinforcing the sonic and emotional identity of the film.

"Composers, in particular those who write film soundtracks, have a whole array for eliciting emotions," Schlenker says. "But in the same way that simply mastering a language is not enough to formulate a linguistic theory, their rich, vast knowledge is above all intuitive." The development of musical semantics will make it possible to better understand and analyze these intuitions. 

  • 1. CNRS / EHESS / ENS.
  • 2. This research was funded by the ERC grant New Frontiers of Formal Semantics.
  • 3. Institut de Recherche et Coordination Acoustique / Musique, the host institution of the joint research unit Sciences et Technologies de la Musique et du Son (CNRS / IRCAM / UPMC).
Go further


Martin Koppe

A graduate from the School of Journalism in Lille, Martin Koppe has worked for a number of publications including Dossiers d’archéologie, Science et Vie Junior and La Recherche, as well the website He also holds degrees in art history, archaeometry, and epistemology. 



0 comment
To comment on this article,
Log in, join the CNRS News community