My Research Experience

After discovering that the other security group working on the PC was getting a better accuracy percentage when detecting the P300 with the Neurosky, our graduate student encouraged us to go back and try testing with the Mindwave Mobile further while we awaited the dongle for the Emotiv. We are still unsure about how we would develop for the Emotiv on Android and after emailing and messaging the Emotiv team several times we still have not gotten very much feedback on how it could be done. As a result, we began testing again with the Neurosky. 

Sadly, we found that we could not find the most recent code we used for the Neurosky and spent a large chunk of our time re-creating what we had previously done. We also changed the numbers presented to the user so that it printed digits to the screen rather than words. 

Unfortunately, this week was not particularly productive. Since discovering that the Neurosky was not detecting the P300 reliably, we started working on trying to use the Emotiv. However, we discovered that not only does the Emotiv have terrible customer service, it also doesn’t have an Android development kit available for use yet. We found some apps that still use the Emotiv, however they require a dongle which we have to wait to receive. 

While attempting to find more information about how we could develop with the Emotiv, we continued fleshing out the dating sim game idea. We were able to come up with specific dialogue for the characters that might be used to get a person’s personal information. 

This week, to our dismay, we discovered that the Neurosky does not appear to detect a person’s P300. In order to have better visualization of the data my partner needed to adjust the python code so that we could see the number of P300′s for each variable. We then performed a vast amount of tests from each of us and had a very low percentage of success. 

We also ran into issues while testing when we realized that the data we were receiving looked wrong. It appeared as though we weren’t getting a constant signal from the Neurosky headset. After we discovered we had a hard time pairing with the device, we realized that we might have been accidentally pairing to the other teams headset. In order to prevent this problem we made sure to do our testing further away from the lab in an appropriately quiet area. 

Even after taking these steps we found that the we rarely were able to see the thing we were thinking of during the tests. We told this to our graduate student and he suggested we work with the emotiv headset to see if the issue is with our python code or if it is a problem of the headset itself (I suspect it is the headset itself, seeing as how the Neurosky only has the forehead sensor). 

Next he wanted us to start brainstorming on a malicious game that could potentially be used to steal a person’s information. We thought about what kind of information a person could potentially want, including a person’s address, name, date of birth, bank information, pin number, etc. Then we had to consider in what ways could we possibly extract a P300 to learn these pieces of information. 

Our initial idea is to make a dating simulator, where the user must control their engagement and anxiety levels in order to successfully get through dates. We assume that we can detect the P300 level during conversations between the user and the characters in the game. 

After fiddling endlessly with OpenViBE and EEGlab throughout the last two weeks, we were able to analyze some of the waveforms and discovered that the GDF file writer we were utilizing did strange things with the signal display of brain waves. We also were getting tired of having to go through so many hoops simply to filter the data, so we found code in python that could filter and graph waveforms so that we could automate some of the processes without having to use OpenViBE and EEGlab. I also quickly created a python program that could check to see if a P300 or N400 exists so that this process could be automated as well. We merged these two programs and then began to test. So far we have not reached any conclusive results, but we are detecting P300s.

Also this week the dean of computer science at UNT came by to talk to all of us about graduate school. We were all able to ask questions about what preparations we would need to take if graduate school was something we were interested in, and to learn some of their own personal tips and words of wisdom. One of the professors here for computational epidemiology spoke as well and I was very intrigued by some of the things he gets to research. I haven’t quite decided if I want to go straight into graduate school, but they certainly make a good case for it! 

Hooray! We were finally introduced to the graduate student that we should be working under during this summer. He introduced us to the project we’ll be working on, which involves trying to detect the P300 and N400 using the Neurosky headset. For instance, we want to see if it is possible to find out what a person is thinking about by looking for certain features in their brainwaves. 

So we started off our adventure by creating an android application that would display either a list of months, visa cards, or numbers. This way we could record our raw EEG activity while executing the application and export that data into a csv file for further analysis. For example during a test we would think of the month we were born, start the application that lists the random order of months, and simply record the raw EEGs as the months are displayed. After we had completed this we needed to filter the data appropriately to get rid of noise and artifacts and to get the appropriate frequency band in order to find the P300. We initially attempted doing this in java, but found few resources on how to implement the filters, and then decided to switch to using OpenVibe and EEGlab. 

The process with OpenVibe and EEGlab proved to be very tedious because we initially could not get the right file format to modify the waveforms in OpenVibe. As a result we worked a lot with copying the data from the excel sheets and importing it into EEGlab, only to export it into a different file format to use in OpenVibe. From there we could finally filter the data and view a display of the signal. Since this whole process was extremely long winded, one of our group worked on seeing if we could filter the data straight from excel. 

During this week Dr. Takabi gave us free reign to work with the Neurosky, another brand of BCI headset. We spent much of our time playing with it, determining how to use bluetooth to pair it with our devices and figuring out how to use it properly. We were told to read all the documentation for the headset and also explore some of the apps created for it. Because the Neurosky only has one sensor, it has limited capabilities. Most of the apps that use the Neurosky are for entertainment purposes and are mainly derived from the headset’s primary functionality, which consists of determining a person’s level of meditation and level of attention. For instance, in one app the user had to control their focus in order to burn a barrel or use meditation to lift a ball. Many other apps followed this same formula, with slight tweaks. There were also many applications that were used for practical purposes, like assisting those with disabilities by using a person’s blink to select certain text and phrases. 

Dr. Takabi then informed us that we were to create an app using the Neurosky. Our group was split up into people working on an Android application or a PC application. I chose to work with Android, and we set off to determine what we should create. 

After some brainstorming we decided to create a simple cursor that could be controlled with one’s meditation and attention. For example, when a person’s meditation is high, the cursor would move to the right across the screen, if low, it would move left. If one’s attention was high then the cursor would move downward on the y axis, and if low it would move upward along the y axis. We fiddled with getting Eclipse to work, and began attempting to implement our idea. The actual process was not too difficult since there was a decent amount of code provided for the Neurosky. 

One of the most challenging aspects of this application was when we attempted to allow the cursor to “select” something on the screen when a user blinked. We were able to finally accomplish this and added a webview to the app so that a user could traverse the web and select buttons by blinking when the cursor hovered over them. Unfortunately, we were not able to make it so that the cursor could use the keyboard if the user selected a text box. 

Although implementing this app was cool, I found that actually controlling ones meditation and attention was extremely difficult. Eventually it got somewhat easier to control my focus to move the cursor upwards and downwards as I discovered different methods (singing a song in my head, thinking of a single word over and over), but I had virtually no way to control the left and right movements as there were fewer methods to control my meditation. I thought this caused an issue with the overall app and made it much less practical. 

The last day of this week we wanted to try doing something slightly different, but expanding off of our initial idea. We decided we’d once again use meditation and attention, but instead have it so that it could control a toy car. To do this we used a Raspberry Pi and found some code that interfaced with the Mindwave Mobile. After brainstorming the idea that morning, we were able to successfully control the toy car with our mind after just one day. 

The first week of my research experience was extremely exciting and consisted mainly of getting to know the other research participants as well as determining whether or not we wanted to work with security or software testing. The first two days alone were spent introducing ourselves and discussing what was expected of us during our stay this summer. We were encouraged to socialize and determine who we would work best with to help us decide on what projects we would want to dedicate our time to. 

After the projects were introduced and after reading many of the research papers we received, I found myself fascinated by the Brain-Computer Interface projects (BCI) for security. Even though the research participants might have initially been more interested in something different (for instance I assumed I would be more inclined to do software testing), we were encouraged to change our mind if we found ourselves more interested in something else or if we found someone in particular we wanted to work with.  After learning more about what each of the projects involved, I knew for sure I wanted to work with BCI.  

We were then split up into our different groups. For security, Dr. Takabi provided a few more research papers that we were to read over and discuss with one another. We analyzed the papers and talked about what we believed to be some of the issues regarding the experiments and also discussed how we could possibly replicate said experiments with the BCI. 

My name is Cree White and I am a UNT student participating in the 2015 Bug Wars REU this summer. I’ll be giving updates on the progress I’ve made during my research and hope to give future participants an idea of what the research experience has been like during the next 9 weeks. 

I’m currently a senior majoring in Computer Science and hope to graduate this December. I have lived in Denton for the past 2 years, but originally I was from the San Antonio area. I have little experience outside of school projects which include a couple of Android applications and a small game created for Windows.