Researchers use AI to explain how music affects our minds, bodies, and emotions

There's a song for every mood and occasion, isn't it? But, why it's that we feel what we feel when listening to certain music or song? Why some songs make us happy, emotional, and so forth? Well, we may have finally an answer to these questions, courtesy Artificial Intelligence

Researchers at the Viterbi School of Engineering of USC used Artificial Intelligence to determine how music affects our minds, bodies, and even emotions.

The USC researchers discovered that "different kinds of audio musical features were predictive of different response types (neural, physiological and self-reported judgments)."

"This finding gives credence to using a multi-pronged, multi-perspective approach to studying music listening experiences; in other words, different aspects of music affect our brain, body, and behavior, and affect the measured responses in different ways," said Tim Greer, lead author of the study and computer science PhD student at the University of Southern California and a member of the USC Signal Analysis and Interpretation Laboratory (SAIL).

Another interesting finding was that listeners getting excited at the onset of exciting musical moments increase the spike in the skin conductance response. One example of this could be at EDM concerts before the DJ drops the beat.

How the study was conducted?

The study as explained by Greer, was conducted by selecting 120 pieces with the words "happy" or "sad" and which were listened to by eight human judges. The pieces in which at least 75% of the human judges agreed on were included in another survey among 82 adult participants via Amazon's Mechanical Turk. Finally, a happy song, a sad short song and a sad long song was chosen.

These three songs were then listened to by 40 adult participants and their brains were scanned through an fMRI scanner through this process. For a physiological response, the same process was followed among 60 different participants with their heart activity and electrodermal (activity) skin conductance collected. Each song was listened to twice, once for rating emotion and the other for enjoyment.

"We also extracted from each song seventy-four audio features that capture music dynamics, timbre, harmony, rhythm, and register and used machine learning models to predict the responses recorded from the subjects using these auditory musical features," Greer explained.

Why the study is important?

According to Shrikanth who is a professor and holds the Nikais Chair in engineering at USC, this study can be applied to areas like scientific music studies, understanding different music emotions and music therapy. It could also help the music industry for composers to understand how they can get a better response from the audience. It can also be used for film scoring and media production.

Science of popular music

Popular music these days sounds almost the same across different genres. According to Greer, music has evolved over the years and creators find new ways of capturing audiences. Greer also shared how a related study found that many Billboard hits like Old Town Road "enjoyed success by having lyrics that were steeped in one tradition (like hip hop), while the music of the song was from another genre (like country, or rock)."