Listening to music with your eyes
Recent discussions in the School of Music have touched on the question of the effect of sight on music listening. We just had music theorist Elizabeth West Marvin visiting from Eastman School of Music, and one of the topics she discussed was the impact of following a score on listening to music. It is my understanding (second-hand, because I was teaching during her talks) that she advocates listening to a piece of music first before working with the score and analyzing it. I am a fan of that approach, and fully agree that treating the aural experience of the music as secondary to the visual experience of the score cannot help but impact the way one hears the music. Because of time considerations, I do not always approach the music that way when we discuss it class, but I will have to continue thinking about ways to ensure that students listen to the music first before listening with score.
This general topic of vision and music listening got me interested into what research has been done, and led to the discovery that research on the eyeβs role in musical listening and response has blossomed in recent years. The use of eye-tracking technology in particular has become a new frontier in music cognition research.
As even a cursory exploration shows, researchers are increasingly looking at the eyes as a window into understanding musical expression. One common avenue of inquiry is the tracking of eye movement and pupil dilation to measure emotional response to music (see for example, this 2015 study and this 2016 study). In August 2017, a two-day conference on music and eye-tracking research convened in Frankfurt, Germany, and the conference literature noted that βDuring the last few years there seems to be growing enthusiasm to apply eye-tracking methodology to capture the full complexity of musical processing, extending to music perception, cognition, learning, skill-acquisition, expertise, performance, social interactions, rhythmic entrainment, emotion processing, induced states of altered awareness, and cross-modal interactions between the auditory, visual, or motoric systems, to name just a few examples.β
Creators of musical performance interfaces that incorporate eye movement have been working on this for longer. Some of the more well-known examples include innovations designed by Tod Machover and co. at the MIT Media Lab, the EyeHarp project based in Barcelona, and EyeMusic, a design out of the University of Oregon.
Meanwhile, some research is examining how musicians use vision in performance. For example, see (below) the March 2017 video featuring Daniel Beliavsky and his students playing with eye-tracking headsets.
Of course, the fascination with the relationship between visual perception and musical perception goes back to antiquity. With technological and scientific advances in our understanding of the brain, it will be interesting to see how insights about the connection between vision and music will impact music pedagogy in the coming decades.
