Team Emoti

Monday night, the Emoti team came together to draft together things we had learned from launch week and create the second version of our product roadmap. Here is what we learned from launch week: 

Interactivity: interactivity needs to be a major focus of the next round of product development, as that was the component that most users requested for, and what we saw as the main exciting aspect of VR. 

Modularization: our experience can’t be consistent for all our users, but needs to have different components for each end user. We’re going to work on making sure that we can customize the experience for each school. 

Real world context: the practice modules were effective at inducing stress, but they seemed very artificial- our team was skeptical whether the learnings would apply to real world situations. 

Wearables: our hardware is having some problems, and we’re going to work on figuring out how to make it better. 

On April 29, Emoti went to two schools (Alpha and Synapse) and showed our product to 10 students (5 at each school) and got some sizable feedback. We started at 9 am in the morning with Alpha and finished around 4 PM at Synapse. It was a long and arduous day, but we finished strong and are more pumped than ever. 

Some highlights: interacting with students and seeing them try on VR for the same time, some hilarious reactions to putting on VR, having students expand their SEL skills because of our software, and watching students apply the techniques to practice when they got stressed. Definitely a huge perk and a huge motivator for the team- we’re more amped than ever to make Emoti the coolest product it can possibly be! 

We’re going to upload some pictures below of images from our testing sessions, but feel free to read our documentation of our debriefs from our notes here (Alpha here, Synapse here), and expect a more formal analysis of the debrief and new product roadmap coming up in the next few days. The team is now going to take a break to digest all that it’s learned so it can be amped up to tackle the next stage of dev :)

Alpha: 

Synapse: 

Hey everyone! Over the past few weeks, we’ve been busy working away on our v1. Essentially, that involved a few things: one is the VR development and the other is the ML backend (aka the server that takes in readings from the Empatica wristband, and then makes the appropriate classification of whether the person is stressed or not). The ML backend also involves creating an iOS app that we use to setup a new user session. So, we have to do work on all those ends. This is what we have been working on so far: 

We have gone ahead and split the VR dev into a few different components: the first is the navigational sequence, the second is the instructional sequence, and the third is the assortment of practice modules. I’ve attached some screenshots of some of the components, but this involved a lot of intense VR dev to get the effects right and some time with Yale CEI to develop a video about instructing kids to learn about stress and techniques for calming themselves down. 

The ML backend involved conducting another data collection session, and running more ML models using sci-kit and figuring out which model works best. We’re still having issues with our data and getting this to work 100%, so we’re relying on existing thresholds in physiological signs from psych research to support our backend. We rewrote the server to be a cleaner, less clunkier design, and a completely redesigned iOS app that makes it dead simple to connect all the components and setup demo sessions. Screenshots from that app are also attached. 

Now that we have all this equipment, we’re ready to launch and put our product in front of kids, so what are you waiting for? 

Photos of the demo and updated iOS App: 

Hi everyone, and sorry for the delay! Here is what Emoti has been upto in the last 1.5 weeks: 

As mentioned earlier, we spec’ed out our v1 for Emoti. We’re focusing on creating an experience that can teach students about the emotions of calmness/serenity. So, to do that we need to a) work on the dev for creating the actual VR module and b) work on improving/testing the ML backend. Here’s what we have accomplished so far: 

We have a rough version of the deep breathing module working (i.e. the part of the simulation that actually teaches students deep breathing techniques). We’re finalizing some kinks, but the prototype is rough and ready enough for user testing in about 2 weeks. 

We have finished collecting data about the emotions of calmness/stress. We still have to run the data through the classifier, but we’ll be working on finalizing and hooking up the backend this week. 

We’ve set up user testing sessions. Alpha and Synapse are particularly on board, and we’re just working on finalizing the dates for testing wit them. For other schools, we’re still waiting to hear back, but we should have a third school that can give us concrete feedback as well. 

We know we posted a tentative roadmap for the quarter last week, but after speaking with our professor and getting some more advice, we’ve decided to scrap needfinding altogether and start building building building. So, taking his advice into account, we decided to modify our old roadmap and come up with a new one, which you can find here 

Essentially, the highlights are: 

We’re focusing on one emotion (serenity/calmness) since our instructor implored us to focus on a positive emotion, as it will be an easier sell to schools themselves. 

We’ve come up with a scenario that we want to test that allows for both practice and test using our physiological sensing testing system. 

We have a plan to get this MVP built in the next 4 weeks. 

Here is our plan for the rest of spring quarter: 

Weeks 1-2: prototyping, specifically, working on parallel prototyping which emotion we want to start off with and what experiences we want to use to elicit different kinds of emotions.

Weeks 3-5: Have ML part working and/or BinaryVR part working, and collect lots of training data. Finish VR content creation.

Week 6: user testing by going into schools and testing out VR content

Week 7-8: Build v2 based on feedback from user testing.

Sorry for the lack of documentation everyone, Team Emoti was a little busy conquering London :) But now that we’re back, here are some quick updates: 

We finished putting together our demo and presented at the CS210 end of quarter fair, as well as presenting at FB London. You can find our pitch deck here. The overall feedback was good, and people really liked the idea of Emoti, but wanted to see Emoti more in action at schools before they could give actionable feedback. We could have also done a better job of picking a better scenario in order to elicit sadness, and this is great feedback that we’ll definitely use when creating our product. We also got a tour of the FB MPK office, and enjoyed some delicious pizza and ice cream :)

Next,we took a plane to London and got there just in time for our demo at MPK! We were slightly late due to the traffic, but got to give our presentation and got some really interesting questions. One piece of feedback we got was we needed to do a better job giving more background since many of the folks there weren’t familiar with SEL, so our value prop wasn’t clear to them. Great feedback, and something we’ll definitely take into account! 

In London, we also meetup with Edward Miller (founder of Medical Realities and a VR/AR imagery specialist) and a startup called Improbable.io. While neither of these meetings was directly useful to our project, we got some great insights into the VR/AR scene in London, as well as the startup scene in London in general. We also got some cool swag, which hey, never hurts :) 

While in London, we did some sightseeing, teambonding, and lots and lots of eating. We also did a team feedback session where we went around and gave feedback to each other about pros/cons, and we did a quick roadmapping for the next quarter. Both of these were a great way to wrap up the quarter, and we’re looking forward to more of these sessions in the future. 

Additionally, before we left for London, we spoke with BinaryVR, a company that maps facial expressions in real world to expressions in VR (notes from the call found here). We spoke with their CEO/COO, and they’re definitely interested in working with us. We’re going to be discussing the fruitfulness of this partnership, and then accordingly continuing to work with them! 

Finally, post-London, one of our teammates (Steve) actually flew out to Yale to speak with our liaisons at the Center for Emotional Intelligence, where we got some feedback on our idea and some help for the next direction of our project. You can find meeting notes from the meeting here. 

This past week, we visited AltSchool, a startup school in SF where SEL is definitely a priority. Notes from our visit can be found here: https://docs.google.com/document/d/1xHKDgUju7Vv5lNtRTGM3IQbIW-N_FqLPsPhh2HJ9xhY/edit

Here are some key takeaways: 

Emotionally intelligent teachers can expedite learning a lot. We’ve talked a lot about this with our earlier visits, but we really saw the effects of this in AltSchool. AltSchool has the culture (such as supervised group conversation with teachers in which students can express their feelings, and teachers open and honest about their emotions) where students are openly encouraged and do  express their feelings. This offloading of emotions allows students to focus on learning emotions, rather than trying to offload emotions. Therefore, AltSchool is very effective at what it does. 

The students we met were super adept with technology. This surprised us (although not really, given Zuck’s investment in AltSchool), and we were surprised at how much technology was integrated into everyday learning (although not with SEL)  

AltSchool’s model of teaching mostly fixated around using patterns that allowed students to express their feelings, such as “I feel X because Y and I need Z.” These patterns were really useful, and the fact that teachers were using them made them even more effective! 

So, other than needfinding, what else has Emoti been upto in the past few weeks? Well, the truth is, we’ve been hard at work coding up our demo, which is something we talked about in the last few weeks. So, as a refresher, our eng roadmap is broken down in three parts: 1) ML classifier for emotions 2) Unreal Engine plugin so that the VR simulation can grab physiological data from sensors 3) VR demo itself that plays video and shows the weather changing due to changing physiological state of the user. 

re: 1, we’ve been having some trouble getting the linear regression to work. We’ve tried SKLearn library in which we threw a bunch of ML models and saw how it worked, but unfortunately that hasn’t yielded the most positive results. However, we’re trying to filter out our data and see if by simplifying the problem (esp after talking with some grad students that have done similar research) we can simplify the problem from dealing with a continuous scale of valence-arousal scale to focusing on predicting a single emotion (thereby making it discrete). We’re working on the modeling right now, and if that doesn’t work, we’re going to recollect our data but only focus on sadness/excited emotions. 

re: 2, we’re stuck because the Empatica wristband that has an API that plays nicely is stuck in Italy. Therefore, for the purposes of the demo, we might have to do something janky, like using an app to record people’s physiological signs and connecting the app to unreal. TBD.