Sahil's Capstone Research

This week I completed the deinstallation process at Mana Contemporary and also compiled together all of my project's assets and information for the web dev team to use on the Capstone website.

The deinstallation process was pretty simple for me! The physical app mockups peeled right off the flipped podium and my flash-drive was easy to remove from the TV! Removing the headphone stand was a little difficult and caused slight damage to the wall which I was able to fix with wall filler/putty.

For the Capstone website, I decided to go with Web Dev's template option since designing a full page for the project would be difficult for me to complete in time. I compiled all of the project's info and graphics (ex. below), along with installation photos, into a shared OneDrive folder shared with the Web Dev team.

This week I worked on my installation and updating the video animation. I wanted to make the video backgrounds more responsive to the music to represent the theme of the app. I explored different tools on after effects and was able to come up with a few interesting animations that correspond to what's happing in the video.

For the installation, after setting up on Monday, I realized the project description would look better on a foam board. I picked up a board the next day, cut it to size, and then glued it to the printed description.

For this week, before making changes to the animation, I tasked myself with completing the UI designs for the app, making refinements and iterations based on feedback I received from the last milestone critic.

^ Before ^ After

For the recommended screen, I altered the live heart-rate visualizer to make the design more functional. I also made it so the filters selected on the generate screen appear at the top with the genre filters. Throughout all of the screens, I changed the body and caption text sizes to align more with existing streaming services. I also altered the padding in-between elements throughout, giving the app a more cohesive feel.

In the generate screen, along with making slight adjustments to the font sizes and padding, I changed the saved emotions to appear as icons that can be expanded. This is helpful when a user has many emotions saved, since it would be annoying to scroll past them all each time a user generates music.

This week I worked on my project description. I'm planning to have it on the wall alongside the monitor and phone prototype in my installation.

This week I finished prototyping on Figma, polished the app designs further, created animations of the app in After Effects, and designed the sound for the video in FL Studio.

Creating the video was a lengthy and time consuming process. I started out by learning how to use ProtoPie, which is a software for UI prototyping and animations. ProtoPie didn't end up working out though - I was having trouble using the software and concluded that Adobe After Effects could be sufficient!

After transitioning to After Effects, I continued to stumble into issues... I wanted to originally make the phone 3D but I wasn't able to without specific expensive plugins and mockups. I ended up animating the phone in 2D and played around with the movement to make the video more visually interesting and match the music. After I was done animating in After Effects, I brought the rendered video into FL Studio to create UI sounds and other FX.

Over the course of week 8 (+ spring break), my goal was to finish the app designs for HeartBeat. I started the process by redesigning and finishing the Emotion Library, which I renamed to just "Emotions." Although the new version looks a little emptier and simpler, this is because I worked on iterating the designs to be more intuitive.

I removed the emotions timeline from this screen since I thought it would be better placed on a separate page so it can be more interactive.

I added a search bar with multiple functionalities, where searching allows the user to find or add emotions and songs. Clicking the search bar also triggers the "explore more emotions" prompts, rather than having them below the saved emotions. This is beneficial for when users have many emotions saved, since it would be annoying to scroll past them all to find or add emotions and songs.

When the user clicks an emotion, they are brought to the emotion's page (essentially HeartBeat's version of a playlist). I updated the styling to match the current UI and added more features, like filtering and rating songs, heartbeat shuffle mode, interactive slider buttons for each song, and an indication of whether a song is a HeartBeat recommendation or from the user's Spotify library.

I also designed the music player, studying other streaming services and how they lay out buttons to better understand listening UX.

In addition to the app designs, I also created HeartBeat's app icon & logo. I tried to incorporate a heart-rate icon into a music note and I think it looks pretty nice!

After deciding to change the app to a light theme, I spent a good portion of time this week updating the recommended screen to match the latest designs. During the process, I made a few more changes to the recommended screen as well.

Previous ^

Current ^

Other than changing the theme to light, I changed the persistent heart rate monitor functionality because I wanted to incorporate a persistent signifier for when the recommended list is eventually expanded with more sections. This could also be used in other pages to create more consistency throughout the app. I also researched modern apps and many of them display section headers at the top of the page and transition to a persistent overlay when the user scrolls past the header. With the addition of a persistent section header, I decided to remove the persistent heart rate monitor, since there would be too many fixed elements on screen and I think it would be better placed in the music player screen.

I created a styling sheet for the app with specific color branding and font guidelines to produce more consistent designs. I slightly tweaked all of the app's text and colors where it was necessary.

I also changed the Mixes by HeartBeat section to display the mix titles below the mix covers; the titles can be difficult to read depending on the cover.

Gallery Hopping Trip

During the gallery hopping trip last Wednesday in Chelsea, I came across a few pieces that inspired me.

This is a work from Camille Henrot’s immersive exhibition ‘A Number of Things.’ I'm intrigued by how the smaller objects surrounding the large cube give the piece a more dynamic and immersive experience. Since immersion has been a central theme in my project, I think it would be interesting to incorporate other elements in the exhibition that immerse the viewer as they engage with my project.

Studying the presentation and installation of numerous works, I observed the average heights and scales to think about popular and effective placement choices. For my project, I'm currently thinking about having the tv monitor tilted slightly down above the phone. I think this can work depending on the tv size.

Reflection

This week my goal was to refine the filters screen based on new thoughts and feedback.

Previous ^ Current ^

During the process of working on this screen, I came to the realization that it makes more sense for the app to be made in light mode by default - many activities where the app is most useful are outside during the day.

I changed the design of the emotion tags, adding an icon and colored stroke that fills in when selected. I'm planning on using these designs for the emotions in the emotion library too.

I also added a container at the bottom of the section next to the Generate button that displays the filters selected and a clear filters button.

Reflection

This week I completed iterating the recommended screen, further implementing the emotion side of the app into the user experience.

Previous ^ Current ^

I updated many aspects of the UI styling after spending time researching the designs of other music player apps.

I created more contrast between the recommended filters and the liked songs mini section by applying a black background within the header container.

I removed many lines from this section since they weren't really necessary and it saved screen space.

I added an indicator of the filters selected from the onboarding screen. Attached to the indicator is an edit button that takes the user back to the onboarding screen.

I also added a shuffle button in the Liked Songs mini section, which will shuffle through the recommended songs based on the user's active heart-rate.

When a song's drop down arrow is clicked, information about the song will appear, such as its associated emotions, activities, and recommended percentage. The user can click on the edit icon to change the song's associated emotions and activities.

Reflection

This week I further developed my app based on feedback received during my milestone critic.

Emotion Library

I created an emotions library page where the user can see their most frequent emotions and then click on an emotion to see which songs are associated. The user will be able to edit the recommended lists and maybe even filter by bpm. Adding the emotion library and more customizability allows the user to apply a more intentional and personal touch into their recommended music.

I added a chart at the top of the emotion library called "Frequent Emotions." The chart shows the user's hours spent listening to music with emotions over the last 7 days, last 28 days, or last 12 months.

Other Changes

This week I also cleaned up the UI on the home page, condensing the heartbeat bar to allow for more screen space - this is ideal when the music player controls appear. I decided to make the heartbeat bar a more neutral blurred gray because I want to have the song's audio spectrum overlayed on the user's heart rate, so the user can see in real time how they are reacting to the music.

Reflection

This week I engaged in user research, iterated the apps navigational structure, and redesigned the main screens in a prototype flow. I refined the apps branding design as well.

User Research:

Each round of research highlighted interesting perspectives of my concept. One thing in my research that stood out to me was the idea of 'Community-based taste' and creating more intentionality behind music recommendations.

New App Mockups:

Reflection:

This week I created user research questions that I plan to test throughout next week. These questions are focused on market research, aiming to discover the problems that exist when listening to music during various activities.

User Research Questions

0a) Age Group 0b) Do you own a smartwatch? 1a) When your heart is racing from stress inducing activities, what methods do you use to calm down? 1b) How often do you listen to music? 1c) What genres of music do you find most effective for reducing stress? 1d) Do you actively seek music that matches your current mood, or do you prefer music that shifts your mood to a desired state? 1e) Do you find listening to music successful in reducing stress? If not, why? 2a) Do you listen to music while engaging in other activities? 2b) Are there any issues in the music listening process that occur during these activities? 2c) What solutions to these issues (e.g. undesired music recommendations) have you tried? 2d) What about these solutions do you not like?

Although I will finish testing these questions next week, I started developing the wireframe for the home screen based on further development of my concept and early user research.

Reflection

Research

ResearchGate: “MusicalHeart: A Hearty Way of Listening to Music”

This is a similar music recommendation system called ‘MusicalHeart.’ The Septimu platform, which consists of a pair of sensor-equipped earphones that communicate to the smartphone via the audio jack, enables the MusicalHeart application to continuously monitor the heart rate and activity level of the user while listening to music.

The app “provides dynamic music suggestions to help the user maintain a target heart rate.”

“MusicalHeart helps the user achieve a desired heart rate intensity with an average error of less than 12:2%, and its quality of recommendation improves over time.”

I could add an option that lets user’s set a desired heart rate

Examining MusicalHeart, it seems like the user’s immediate activity is an important aspect in generating accurate recommendations.

Medium: Groove is in the Heart: Matching Beats Per Minute to Heart Rate

“Songs of various tempos can influence exercise, anxiety, post-surgical recovery, and many other outcomes”

Similarly to heart rates, “most songs also fall somewhere in the span of 60 and 200 beats per minute. However, when you graph the frequency of bpms for the top 10,000 streamed songs, the most common tempos are not in the resting heart rate zone, when you’d expect most people are listening to music, but slightly higher, between 120 to 130 bpm.”

Research Reflection

For my project this week, I did more research on my concept and found examples of other heartbeat based music recommendations. I think my idea can still work and I am figuring it out more as I do more research and build out the app’s structure.

Citations:

(PDF) MusicalHeart: A Hearty Way of Listening to Music, www.researchgate.net/publication/255568687_MusicalHeart_A_Hearty_Way_of_Listening_to_Music. Accessed 29 Jan. 2025.

Accessible Audio / Speculation

Source #1: Relationship between heart rate and preference for tempo of music

Source #2: Reactive Music: When User Behavior affects Sounds in Real-Time

Source #3: panGenerator: The Wall of Sound

Research Reflection:

My research this week was centered around the concept of reactive audio, where audio, or music, is able to react to the listener and their environment in real-time. I was able to learn a lot more about this concept and find it really interesting. I'm going to look more into reactive/interactive audio and also see which sounds correlate with real-life subjects to learn more for my next prototype!

Prototype:

Works Cited:

Iwanaga, M. “Relationship between Heart Rate and Preference for Tempo of Music.” Pub Med, pubmed.ncbi.nlm.nih.gov/8570336/. Accessed 20 Nov. 2024.

Waldner, Florian. “Reactive Music: When User Behavior Affects Sounds in Real-Time.” ResearchGate, Apr. 2013, www.researchgate.net/publication/235788265_Reactive_Music_When_User_Behavior_affects_Sounds_in_Real-Time.

Accessible Audio / Speculation

Source #1: Rafael Lozano-Hemmer: Pulse Topology

Source #2: Music in Advertisements: A Tool for Emotional Connection

Source #3: Family Guy Scene

Source #4: Mercedes-Benz and Will.i.am Turn Electric Cars Into Musical Instruments with 'Sound Drive'

Research Reflection:

I was really excited by my research this week and I think it will really help me develop my capstone project further. I am interested by the methods in which sound can be reactive. I think by exploring this concept, I can learn how to make audio more interactive!

Works Cited:

Lozano-Hemmer, Rafael. “Pulse Topology.” Rafael Lozano-Hemmer, 2021, www.lozano-hemmer.com/pulse_topology.php.

“Music in Advertisements: A Tool for Emotional Connection.” Novecore Blog, 2023, blog.novecore.com/music-in-advertisements-a-tool-for-emotional-connection/. 

MacFarlane, Seth. “Family Guy Viewer Mail #1 (Season 3, Episode 21).” Family Guy, Fox, 14 Feb. 2002, www.youtube.com/watch?v=XPM1o9QKw1Q. 

Accessible Audio / Speculation

Source #1: WWDC23: Explore immersive sound design | Apple

Source #2: Sonic Reflections: Observations of the Changing Soundscapes of the COVID-19 Pandemic

Source #3: Forbes: Using Technology To Help Visually-Impaired People Navigate Cities

Source #4: BBC: Tube app: Guide for blind people on London Underground

Research Reflection:

My research this week went very well! I focused on learning more about spatial audio, soundscape characteristics, and various methods that visually impaired people use to navigate noisy and bustling city environments.

I found a lot of interesting information in my research, like the effects that soundscapes can have on people addressed in the Sonic Reflections' article. Thinking about how a soundscapes effect a person's emotional state, I wonder how the user experience of soundscapes could be perfected to allow room for pushing the boundaries of accessible audio in society.

Prototype:

In this prototype, I added audio feedback into the soundscape of a playground to visually speculate the creative and interactive potential of sound-driven playground equipment. Designing sounds with a focus on the interplay of frequencies and melodies can encourage cooperation and creativity among children. A sound-based playground could also beneficial by unconsciously embedding basic music knowledge into the minds of kids, encouraging them to discover their love for music at a younger age.

Works Cited:

Apple Developer. “WWDC23: Explore Immersive Sound Design | Apple.” YouTube, Apple, 17 May 2024, www.youtube.com/watch?v=BDO6NTvoTDU.

Guzy, Marinna. “Sonic Reflections: Observations of the Changing Soundscapes of the COVID-19 Pandemic.” Smithsonian Center for Folklife and Cultural Heritage, Smithsonian, 27 July 2020, folklife.si.edu/magazine/crisis-changing-soundscapes-covid-19-pandemic.

Winkless, Laurie. “Using Technology To Help Visually-Impaired People Navigate Cities.” Forbes, Forbes Magazine, 29 Mar. 2017, www.forbes.com/sites/lauriewinkless/2017/03/28/using-technology-to-help-visually-impaired-people-navigate-cities/.

Accessible Audio / Speculation

Source #1: PercussionPlay

Source #2: Hansgrohe Group

Source #3: Adi Mor: Why IOS notification center made me feel overwhelmed and what I did about it

Research Reflection:

My research this week went well! I thought more about sound instillation ideas, looking into various environments that would be cool to speculate the utilization of audio. I like the idea of changing the sound design of a playground, with a focus on encouraging creativity and collaboration among kids, and may look further into it.

Works Cited:

“A Sound Playground: 7 Easy Music Solutions For Your Outdoor Area.” Percussion Play, www.percussionplay.com/a-sound-playground-7-easy-music-solutions-for-your-outdoor-area/. Accessed 1 Nov. 2024.

“Sound Quality in the Bathroom: The Perfect Water Sound.” Hansgrohe Group, www.hansgrohe-group.com/en/stories/sound-of-water. Accessed 1 Nov. 2024.