The Pianist’s Gestures

Motion capture suits, infrared cameras, and an array of specialized microphones. It’s the opening night of an unlikely recital in Poitiers. Scientists and musicians are warming up their instruments to test a widespread belief: that the sound of music is as dependent on the artist as it is on the instrument.

ITV Alain Villard

In going to concerts, I think most people have observed that each pianist is recognizable, that each has their particular sound. We notice these difference when they play on the same instruments. But the conditions are almost always different.

What interests me in particular is to take one location, under the exact same conditions, and bring four different pianists to observe them from biomechanical and acoustic perspectives. To really watch for the small things that make them unique.

Researchers and engineers from the Pprime Institute laboratory focus on acoustic and biomechanical theory. Two of their teams have come together for this project. As the team unpacks the motion capture equipment, Jean-Christophe Valière takes the room’s dimensions and acoustic response.

ITV Jean-Christophe Valiere

In this experiment we aim to verify the hypothesis that if we take one piece of music, on the same piano in the exact same environment, two pianists will create different sounds despite the fact that they’re made by a mechanical device. In other words the posture of the pianist and the way they play the keys will actually change the sound.

This is hard to verify because you have to capture both the pianist’s movements as well as the sounds created, then find a correlation between them.

ITV Alain Villard

I think it’s incredibly useful for my own research as a musician and teacher, finding certain answers, and being able to analyse them for certain criteria.

The biotechnicians assemble a framework equipped with 16 infrared cameras. They can record high-speed images for a detailed study of the pianists’ movements. The structure is designed to envelop the artist in order to capture their movements from every angle.

ITV Floren Colloud

We’re in the midst of installing the motion capture system. These are the digital cameras that we use with a standard lens and infrared filter. These cameras are incredibly sensitive. The infrared light shed by these LEDs will illuminate these small markers, then be digitally tracked and reconstructed.

So we’ll put the markers on the pianist.

We put the cameras around the piano and the markers need to be visible to at least two cameras in order to capture their three dimensional trajectory. Once we have the trajectories of each unique point in relation to the others, we can calculate the overall angles and in the end obtain the general movement of the pianist. At the same time we’re recording the music, which we can then overlay in high fidelity.

The sound is captured in two different ways: the first is through close proximity, to create a very precise analysis of the sound produced by the piano.

ITV Danae

I’m positioning two microphones to capture the sound directly from the piano as well as the room’s reverberations in order to find a proper balance between the two. After that I’ll place microphones even closer to ensure we record the sound precisely. The idea is to be able to recreate the sound and synchronize it with the cameras.

The second measurement is carried out through artificial heads equipped with microphones installed where their ears would be, to mimic human hearing. This creates what is called binaural sound.

ITV Jean-Christophe Valiere

This second kind of recording allows us to perform a blind test, where different people will be able to listen from the perspective of the artificial head and not know which pianist is playing. We’ll see if they’re able to differentiate between the artists and the difference of quality in how they play.

ITV Alain Villard

In reality we play with our entire body, not just our two hands and fingers. Demonstrating that is what the experiment is all about.

ITV Jean-Christophe Valiere

This experiment has lots of potential uses. It also fits within training for arts medicine. There are aesthetic aspects, health aspects, all within the realm of mechanical posture, but also within the realms of sound and hearing. It’s comprehensive. Starting with this information, we can apply what we discover to a broad number of subjects.

With the equipment in place, each of the four pianists will play the same pieces. The recordings will be synchronized across audio and video.

Thanks to this experiment, these researchers will determine which of the musicians’ movements affect the sound they create. In doing so, they might clear up the mystery surrounding pianists and their signature sound.