Workshop / User Experience (UX) Scenario
This workshop is designed to offer you experience with executing a scenario-driven, think-aloud protocol in preparation for designing your own user-centered usability study. During the workshop, you will follow a study prompt that I created to help you understand what it means to run a research trial, review and interpret your data, and organize your thoughts in a Summary Report.
Workshop Objectives
- Review materials relevant to designing and executing a usability study.
- Execute one trial of a usability study.
- Interpret data from that study.
- Present your results in a summary report.
Workshop Deliverables
- Post your summary report to the discussion forum in SVSU Canvas that is dedicated to this workshop.
- Post the video of your study trial to your Screencast.com account, and include the SHARE link to it in your report.
Workshop Details
This workshop requires you to conduct a single trial for a usability study that I have designed. It implements the problem-solving scenario method in combination with a think-aloud protocol. This is one of the methods available to you for your work on the Design Assessment project.
Ordinarily, this study would be completed in a team with 3 or 4 participants, each taking on a different role for the study. If you are able to work with one or more classmates to complete the study (or even someone who is not enrolled in the course, but who is willing to assist you), do so. If not, follow the alternative instructions for solo work (designated from now on by the term Solo).
Review each stage of the description for more details.
The workshop has 4 stages.
- Prepare for the study.
- Conduct the study.
- Interpret the results of the study.
- Complete and submit a summary report.
Prepare for the Study
When you prepare to complete this workshop, attend to the following details. Note that the first decision is who will work with you (required during in-person instruction, optional for remote sections).
- Team up with two classmates. Studies are most efficiently managed if you have three people: one to run the study, one to serve as the trial participant, and one to film the process. If the number of people present for the workshop does not allow for an even distribution of three-person groups, some groups can have four members. Decide on your roles before you go any further (coordinator, participant, film assistant).
- Work alone (Solo). If you are unable to partner with another person to complete the study, work alone. Look for the ways that the instructions vary to allow you to do so. In those instances where there is no Solo variation offered, follow the standard instruction for how to proceed.
Once your team organization is decided, continue with your preparations.
- Review Krug's think-aloud protocol. Review Stephen Krug's demonstration video for scenario-driven think-aloud protocols: YouTube.com: Usability Test Demo by Steve Krug. Listen as he explains the basic procedure for this kind of research trial. Note the way he prompts the participant to explain to him what she is doing. You'll want to emulate his actions. (Note that if the link fails for some reason, search YouTube.com for the phrase displayed for this link. It will get you to the content you need.)
Do not skip this step. Krug's demonstration is essential to your knowledge for how to conduct this kind of usability study.
- Retrieve the workshop scenario. Retrieve the scenario document from SVSU Canvas Files in the Workshop Support folder. If you are working with a team, only the study Coordinator should view the scenario. The Coordinator should take a moment to review the document to make sure s/he/they understand the steps involved.
- Retrieve the scenario (Solo). Retrieve the workshop scenario document from SVSU Canvas Files in the Workshop Support folder. Review the scenario to make sure you understand the steps involved.
- Identify your recording device. Recording video and audio of this trial is essential to your success with the workshop.
- If you work in a team, I recommend that your designated film person record on his/her/their smart phone.
- If you work alone, it will be less easy to record yourself. If you have access to an app or service that allows you to record your screen and yourself at the same time (e.g., Techsmith Capture from your Screencast.com account, OBS Studio - visit OBSProject.com), that is definitely the easy option. If not, do the best you can to record yourself and your screen using a smartphone or digital video camera, whichever you have access to. The goal is to capture you and your screen in the video.
-
- Test your recording device. Make sure your video recording device/system is ready to go. If you use a device such as a smartphone, make sure you have plenty of memory free for recording. Make sure the camera and microphone are both functioning properly.
Conduct the Study
The study itself is the core part of this research process. Attention to detail during preparation and the study itself make it more likely that your work will result in meaningful data for later examination. Note that a video recording with clear audio is essential to your ability to gather quality data.
Prior to and during the study, keep in mind the following details.
- Compose the shot. Once the participant and coordinator have settled into their spots, frame the shot you will record. If possible, get both people in frame. If that is not possible, make sure you have a clear shot of the participant and the work space, including the computer screen.
- Designate the work screen as your recording window (Solo). Make sure that the browser window is designated as your record window if you are able to record a screencast with you on screen as well. Otherwise, do your best to capture you and your work screen on your video.
- Monitor the recording as you go. Make sure there are no problems during the recording itself. If any problems arise, ask the participant to pause for a moment while you sort things out.
- Direct the study. The study coordinator introduces the study. Use the script from the task description. Remind the participant that however she or he performs, the study gathers data. In other words, the study does not evaluate the participant, but rather the web site. Keep things moving, if there is a need to do so.
- Keep the protocol on task. Remind the participant to talk about what she or he is doing during the study. If this is necessary, phrase your prompt as if you are encouraging dialog between the participant and yourself: "Explain to me what you are doing." "Tell me what you are thinking."
- Intervene, if necessary. If the participant gets stuck, you are free to assist. Before you intervene, however, ask the participant to think her or his way through the process: "What is giving you trouble? Do you see any way of working out the problem?"
- Work through the scenario to the best of your ability (Solo). You will not be able to monitor or direct the study or intervene if you are working alone. Do your best to explain to the camera what you are doing and why as you work. If you get stuck, talk your way through it.
- Save your video. Export your video from the device you recorded with in MP4 format. Note that this is a standard video format. You will upload this video to your Screencast.com account later.
Interpret the Results of the Study
Review your video after the trial is complete. What do you notice about the participant's/your thinking and doing strategies? Does the process go easily? If not, what problems arise? Were there steps in the process that the study participant struggled with? What caused those issues? Do you notice any problems you have in managing the study? What could have gone better?
Note. Do not skip this step. This is essential to the study.
Complete the Summary Report
The final stage of the workshop requires you to craft and submit a summary report. Whether you worked independently or in a team, prepare and submit your own report. Locate the forum dedicated to this workshop on Canvas Discussions, and follow the guidelines offered there.
Your report should include the following discussion elements.
- Overview. Describe the overall sense of how your study went (e.g., it went well, there were problems), and how easily you/your participant managed the scenario (50 words).
- Observations. What were the key moments from the study? Where did you/the participant seem to have an easy time with the trial? Where did you/the participant struggle? Share direct quotes/statements from the think-aloud protocol (150 to 200 words).
- Reflections. What did you learn about the Object of Study from the trial? What conclusions might you draw from the participant's experience? What did you learn about conducting this kind of study? What are the biggest challenges, and how might you overcome them? (150 words).
- Link to video. Include the SHARE link to your video on Screencast.com. Do not attach your video file to your post. Do not store you video file in your Dropbox account.
Craft your report directly in the forum or in a simple text editor (and then paste it into a response field). Signal sections of your discussion with appropriate headings.
Submission Guidelines
Post your summary report to the forum dedicated to this workshop on Canvas Discussions.
Upload the MP4 file of your video to your Screencast.com account. Include the SHARE link in your summary report (see Complete the Summary Report above).
Evaluation Standards
This section describes the standards by which your work will be evaluated for this workshop. Attend carefully to these details. If you do so, you will earn full credit for the workshop.
Evaluation Summary
There are 50 possible points for this workshop. You will earn points according to this standard.
- 50 points (full credit) for a complete summary report posted to the Discussion forum on SVSU Canvas, including sharing the video from your study by the means described above. The report must meet the specifications defined in this description.
- 40 points (partial credit) for work submitted that does not fulfill all of the posted workshop requirements, or that is submitted late, but within 48 hours of the posted deadline.
- 0 points (no credit) for work that is submitted more than 48 hours late.
A Note to Instructors, Colleagues, and Others
If you are here because of random chance, or because this content came up in a search, then poke about, and read if you see something useful or interesting. If you are a teacher in any context and would like to use any of this content in your courses, feel free to do so. However, if you borrow this material, please do two things:
- Give me credit for my work in your class; and
- Share with me that you are using this material. If you adapt or alter the content, please share with me how you did so. (I appreciate seeing the ways that others apply the ideas I have developed.)