It’s my last week of my DREU experience here at the University of Minnesota! I’ve learned a lot about HCI, academic research, and life in Minneapolis. In addition, I’ve made new friends who have made my time here very memorable. I have a lot in store for this week, community orchestra concert, Symposium on Thursday, and packing!
In research lab, I’ve mostly been working on editing the follow up surveys. In addition, we ran one more pilot study. Which has helped us continue to gauge how effective our operation of the chatbot is. Playing in the Minnesota summer orchestra concert was a blast! We performed a rockin’ program of Tchaikovsky, Wagner and Mozart. Even though I accidently my music off the stand while turning the page during the finale, I’d say it was a success. I’m so thankful I had the opportunity to perform with my violin while at UofM! The next day, my team and I presented at the Summer Undergraduate Research Expo. The expo was located at the Alumni center which is an incredibly beautiful building! We setup our poster and table to let people interact with MLBot. I was a little nervous to present to other people, but it ended up being really fun, and our designated hour of presentation went by really fast! Presenting the poster made me realize how much my team and I accomplished over the 10 week experience. It made me proud of all of our hard work. I’ll be really sad to leave my DREU experience here at UMN. I had so much fun exploring new topics in computer science through research. It feels surreal that I was able to have this incredible opportunity. I look forward to see how I can apply all that I’ve learned in my future endeavors. Thanks for following me on my journey!
0 Comments
I’ve reached the final stretch of my DREU experience. There are only 2 weeks left which is wild! I’ve quickly adapted to my life here at UMN and it feels like another home. This week has been really slow for our research. However! We have solidified our experimental design to be 2x2 +1 +1, comparing interventions of structure and opinion for the active bot to a passive bot and control group. My main task for the week was creating the follow-up surveys using Qualtrics. We also ran another pilot study on the Structure/Opinion bot.
In addition, we started working on our poster for the summer undergraduate symposium for next week. The symposium will be on Thursday on August 10th. Also it has been my second week of rehearsals for the Minnesota Community Orchestra. I’m playing violin in the 1st violin section. I brought my violin to Minnesota to practice my solo repertoire for the fall semester, but I’m glad I can play in an ensemble too! On Saturday I dropped off my friend, who was also in the DREU program, at the the airport. It was a bittersweet goodbye, but I have faith that I’ll see her and everyone else again in the future. Then on Sunday I checked out the Uptown Art Fair which was super cool! We started the week by briefing Haiyi on our new dimensions for the experimental design. We are now planning a 2x2 factorial wizard of oz study plus a passive bot group and a control group. Haiyi seemed to like our design, so we decided to run another trial on people in the lab.
On Tuesday we tested out the new structure enforced bot on some lab members along with the pre-study procedures that real participants will have to undergo. The study was going fairly well until the group failed to reach a consensus and was forced to go back to brainstorming or discussing. This was problematic because after having seen what others’ opinions were in the final poll, members of the group were willing to re-vote and compromise to make a decision. However, the decision making structure didn’t accommodate for this, which created a large flaw in our bot design. During the second half of the week we ran another pilot study with some of the other Big Data REU students. We ran them through the control environment (just making a decision amongst themselves) and then with the active enforced structure bot. While observing their conversation in the control condition, we realized that because group decision assisting technology has never existed before, humans are already quite satisfied with how they make decisions together. But at the end of our brief existential crisis, we reassured ourselves that we are doing this study to help contribute to contributing to research in Intelligent agents mediating group decisions. In addition, I was hard at work writing an abstract for our project to submit for the Summer Research Symposium. IRB approval is still pending and bot alterations are being made. With each pilot study, we’re reaching our desired functionality for our bot designs. This weekend I made another trip to the Mall America with the other girls in the DREU program. Then a bigger group of us went picnicking and canoeing at Lake Calhoun. It was tons of fun! The fact that there are so many lakes within the city never ceases to amaze me. At Monday’s meeting, Max and Haiyi suggested to keep testing out our chat bot script design on other people for the week and make changes accordingly. Then, we tested our active/non-personalized bot design on a group of people in the GroupLens research lab. Three of us simultaneously worked together to operate the chatbot. The lab participants were testing the limits of what the bot could do, so it was quite difficult to act out the bot, but we still managed well. After they came to a consensus and completed the decision making process, we all met to have a discussion about the chatbot. They provided a lot of insightful feedback that helped us establish what we needed to fix. For example, they felt confused as to whether the voting during the conversation was for the final decision or for discussion purposes. Most of the users felt that the voting was for the final decision, so they felt very pressured to decide quickly. Wednesday, we met briefly with Haiyi and Max to update them on how the trial with the lab members went. They also found the results of our trial to be helpful and determined that using personalization as a variable might not create a significant impact. Because of this we started brainstorming possibilities for a new experimental design. For the second half of the week, we worked hard to finish the IRB draft for submission, and determining a new experimental design. We are moving towards testing variables of active/passive, enforced structure/not enforced, and advice/no advice. Our goal for the next week is to finalize the design of the study so that we can run it within the last weeks of my REU stay. In addition, we hope to have the IRB submitted and approved. This weekend, some of us went to the Minneapolis Institute of Art. This museum was quite different from the Walker. It had a wide variety of exhibits, contemporary, classic, cultural, and historical art. I was extremely impressed and would highly recommend anyone to check it out! On Monday, there was a big deadline for the Computer-Supported Cooperative Work(CSCW) conference, so everyone in the research lab was busy putting the final touches on their papers. Because of all the chaos, Haiyi was not able to join our Monday meeting. However, we conducted our meeting per usual and each briefed the group on what we have accomplished since last week. My group presented the new script design for the active chatbot and Max responded suggested that we test it out to check the flow of our structure. The next day, we tested out the bot design with each other by taking turns acting as the bot and acting as the users. It turned out to be really fun and revealed some details that we needed to change. We determined that we need to be exact in when to intervene the conversation to implement structure into the conversation. For example, we created timed intervals of when we would prompt the users to move on to voting from the discussion phase.
On Wednesday we were finally able to meet with Haiyi. For the bot designs she suggested that we make a script difference for the Personalized/Non-Personalized variables rather than restricting it to the recommendation algorithm. By adding this variable into the dialogue, we will be able to see how user interaction is affected by it. In addition, we have decided to run the study through Amazon Mechanical Turk rather than MovieLens because the process will be much easier and there will be more longevity for future studies. Therefore, my task for the rest of the week is to learn and familiarize myself with the MTurk API and conducting a synchronous study overall. To learn about this I have been using the Amazon Web Services documentation and resources. For the weekend, I explored some spots in Downtown Minneapolis, like the Nicollet Mall (which is under construction for the next Super Bowl), the Central Library, and a very large Target. In addition, a group of us went to the Como Zoo and Conservatory. The entire Como park was peaceful and beautiful. And the Zoo and Conservatory were excellent for the free admission! Due to holidays, we weren’t able to have our usual Monday meeting to get changes approved and to discuss the direction of the project. Luckily Max was able to meet with us on Wednesday to look over our script design and give some advice. Our passive bot received approval, but our active bot design needs some more changes. Previously, we strived to differentiate the 2 designs a little too much and had created some confounding variables. Max suggested that we design the bots to be parallel but to make the Active bot stand out with it’s facilitation of structure and offer of voting polls. So we are working to edit the design for the active bot this week.
In addition, we started writing the user satisfaction survey. At the end of the experiment, the participants will complete a follow up survey to give feedback on their satisfaction with the bot, the movies recommended and the decision making process. The results from the survey will serve as our data to determine which bot design is the best for aiding group decision making. On Thursday, the PhD student asked another undergraduate and I to write a draft for the Institutional Review Board (IRB) proposal. Writing an IRB proposal consists of detailing the objectives and the procedures of a research study. After a proposal is written, it is sent to the IRB to be approved. In order to be approved, the research study must follow the IRB’s guidelines which ensure the beneficence, respect, and justice of the participants. Writing the IRB proposal is an important task, so I was a bit nervous and intimidated. But I’m glad I can contribute towards something so significant. On Friday, I continued to write the draft for the IRB proposal. But in the morning there was a communicating science workshop with the entire Big Data REU group. During this morning session, we went over some strategies for writing abstracts and creating posters in order to prepare us for the symposium at the end of the summer. The workshop was extremely helpful and made me feel more comfortable writing my abstract and prepping my symposium poster. Outside of lab this week, I enjoyed celebrating the 4th of July! For the holiday I went to Stone Arch Bridge to take part in the festivities and to watch the fireworks. Later in the week a group of us took a trip to the Walker Art Center and Sculpture Garden. The Art Center features contemporary art and the Sculpture Garden is home of the famous Spoon and Cherry. On Monday we pitched our experimental design idea to Haiyi and Max. Our PhD student is away for an HCI conference, so it’s up to us undergrads to present. In our presentation, we explained our plans for our 2x2 study by detailing the variables, the participants, the experimental environment, and the outline of the chatbot script. Haiyi and Max liked some elements of what we had so far, but had many helpful suggestions and steered us in the right direction. So the next day (Tuesday) the 3 of us started writing the more detailed script with the new changes in mind. We felt comfortable with the alterations in the bot’s designs, specifically increasing the difference between the passive and the active bot. For the passive bot, we are continuing to make it act upon mention only. But we are also making it solely informative. It will never prompt user action, but merely react and respond to what the members of the group request of it in the chat room. For the active bot, we are having it be more of a discussion facilitator. For example, it will now suggest conversation structure by making statements like “Let’s start the decision making process by brainstorming”. In addition, the active bot will have a voting feature (by polls) to further support the decision making process.
As for the bot with information on MovieLens profiles versus none, we have changed these variables to personalized recommendations versus non-personalized. This made the script writing process easier, because we could just focus on 2 different scripts. One for active and one for passive, rather than 4 separate scripts for each cell. Additional topics that we discussed were which platform to use for the chatroom and where to get the population of participants for the user study(MTurk, Movielens, or Offline). This is more difficult for us to discuss because we do not have a wide knowledge of how these services operate (etc.). Therefore we have made this week’s task to be researching these options in addition to brainstorming more for the script. To accomplish this, I’ve been trying to read more research literature in addition to reviewing past publications I have found to try to find ‘inspiration’ for the script. I have also been informing myself on services that will help us develop the chatbot/chatroom. The tools I have been reading up on are Wit.ai, IBM Watson Conversation and the Slack bot. On Friday, our PhD student came back from his conference so we worked hard to finalize the details on the script. We spent a few hours writing /re-writing everything. We also discussed the chatbot/chatroom set-up. We are leaning towards using Slack for the chatroom, and utilizing the Slackbot feature to create our movie lens bot for the study. As for participants, we are now looking to gain our population from existing MovieLens users. We’ve made many changes this week. Now it’s up to our meeting on Monday with Haiyi and Max to see if these changes are good, or if we need to make more adjustments. For this weekend, we are going to Minnehaha falls! The group has been wanting to go since the start of the program, but the weather forecasts haven’t been in our favor. But this Saturday it will be clear skies! Progress is moving more rapidly for the user study. A team of 4 of us (3 undergraduates, 1 PhD student) are working on this project. We’ve been holding meetings all week to discuss the experimental design and to accomplish our main goal of creating a script for the Wizard of Oz experiment. So, for our first meeting, we discussed which variables we wanted to study. We had a wide selection from last week’s MovieLens group meeting. But we were able to narrow it down to assertiveness of the bot and information available to the bot. After deciding on the variables, we then discussed how we wanted to conduct our experiment.
We decided to conduct an offline study with a sample of multiple small groups of friends (3-5 people). We felt that this would most accurately represent the way that the MovieLens bot will be used. Our alternative options were to conduct the experiment through Amazon Turk and to vary the group sizes of small versus large. However, using Amazon Turk for multiple samples being tested at the same time is quite complicated. As for the variance of group size, we decided that this could potentially be a confounding variable. In addition, it would require us to recruit a larger sample, which is difficult under our limited time. Later, we started to work on the script for the chatbot. We first brainstormed types of interactions that would occur between the bot and the user. Here is an example of what we brainstormed below.
I’m quite proud of what we have accomplished for this week. We met our goal and are heading in the right direction. In addition to the experimental design, I’ve been reading more research literature and continuing the Data Science Bootcamp. For the weekend we had lots of activities! We went to the Science Museum on Friday, celebrated a birthday on Saturday, celebrated another birthday on Sunday and went to the Food Truck Festival in Uptown. The Weekend Outside of bootcamp and lab, I’ve enjoyed making some new friends and exploring my surroundings. I even had my birthday this week! A group of us went to the Mall of America to celebrate. I also got a free cupcake from a place that won “Cupcake Wars”! And today (Sunday) a group of us are going to see Wonder Woman at a nearby theater. I’m really thankful that the other students here are so nice and welcoming. They’ve made my experience here so far very enjoyable!
I landed in the Twin Cities at around 10 am. It's my first time in Minnesota, so I'm excited to experience a new environment! It was very hot today which made for a very tiring exploration of the city/campus. However, I thoroughly enjoyed the environment of the UMN campus. It's a more urban environment than at BGSU which I really like. There's a lot of beautiful buildings and cool places here. I checked into my campus housing at 17th avenue dormitory. It's on the edge of campus, so there are amazing views of the stadium, campus, and the city throughout the building. The room is nice and spacious and there are a multitude of lounges to work and hang out in. I have a roommate named Kate who is very nice. I look forward to getting to know her! Being here feels a bit surreal. Just a few weeks ago I didn't have any plans for the summer, but now I'm in Minnesota doing research! Crazy stuff. But I really look forward to my first day tomorrow and meeting everyone. Week 1
My first day started with orientation. This consisted of meeting everyone in the program and an overview of the summer research experience. We learned about UMN’s mission for the Big Data REU which is to help us better our research schools and to consider graduate school. Then we started our first Big Data lesson. We are utilizing a course from UC Berkeley called Data8. It’s a bit daunting to take on a full semester class in 3 weeks, but I’ll try my best! On Wednesday we had a nice luncheon with our mentors. I got to meet my mentor, Haiyi Zhu. She specializes in HCI(Human Computing Interaction) and does numerous studies on social network platforms like Facebook, Wikipedia etc. I’m really glad I have been paired with her because I find this field of study and her research very interesting. After the luncheon we were introduced to our labs. So I met the other undergraduate students (there are 4 of us) and the other professor whom Haiyi is collaborating with for the summer.They explained the big picture of what the projects they want to work on for the summer. We will be working to develop and improve a site called Movielens. This site is a movie recommendation service for those times you can’t make a decision. In addition, we will be exploring how to best develop a way to do recommendations for groups of people, in order to implement to Movielens. The projects aren’t determined yet, but there will most likely be 3 components. Technical, User study, and Analytics. We each picked 2 of the 3 that we would be interested to work for. I chose the User Study and Analytics project, as those seemed the most interesting to me. Much of the details are unknown this week, but I’m excited for the meeting on Monday to further discuss the project! Later, there was more to do! We had another orientation. And then, it was time for pizza and bowling for all the REU programs. In all honesty, I mostly went for the the pizza, but bowling was really fun! I got to bond with the other students in my program and have lots of fun with them. The last 2 days of the week simply involved data bootcamp in the morning and lab in the afternoon. Since neither the user study or analytic project has been planned, I just read up on related studies and on AI services like the Amazon Alexa for the Echo. |
AuthorHi! My name is Mary Solomon and I'm from Northwest Ohio. I'm a rising Junior studying Data Science with a music minor at Bowling Green State University. I plan to graduate in May 2019.. Archives
August 2017
Categories |