COVID TESTiNG KITS
Conducting qualitative research and usability tests for the Seattle Coronavirus Assessment Network to identify and address pain-points.
BACKGROUND
The Greater Seattle Coronavirus Assessment Network (SCAN) is a joint research study between UW Medicine, Fred Hutch, and the Seattle Flu Study that is investigating the coronavirus (COVID-19) outbreak. SCAN's remote enrollment and at-home self-collection nasal swab kits allow for increased disease surveillance and testing capacity, especially for populations underrepresented in traditional clinical surveillance. SCAN wanted to improve the accessibility and usability of the SCAN testing kit to help increase testing capacity and to reduce participant drop off.
Deliverables
Usability Test Structure and Design Qualitative Analysis of Observations User Journey of SCAN Procedure Report with Findings & Recommendations
Role
UX Researcher, Usability Test Moderator, Data Analyst
Timeline
30 Weeks
Industry
Healthcare, Medical Devices, Informatics, Medical Research
Exploratory Research
We began by conducting a qualitative analysis of the SCAN participant datasets, including a post-study survey (which was sent to all study participants) and the support emails that were sent to SCAN. The massive trove of data that SCAN provided allowed us to narrow our scope and determine which parts of the procedure warranted further investigation.
25,222
TOTAL PARTICIPANTS Signed UP for Study
1,803
TOTAL feedback Surveys Collected
Issues Needing further Validation
After conducting a thorough analysis of the data SCAN provided, the research team identified 3 problem areas and formed corresponding hypotheses to test during the usability study.
Registration
46.5%
of All Drop-Offs Occurred at the Kit Registration Step
HYPOTHESIS
Participants have a difficult time completing the kit registration step and quit at that step due to frustration.
Drop-off / pick-up
38.7%
of Reported Issues Were Related to Kit Pick-Up or Drop-Off Steps
HYPOTHESIS
Participants are not provided with sufficient drop-off and pick-up times and instructions for their kits.
instructions
14.7%
of All Drop-Offs Occurred at the Kit Registration Step
HYPOTHESIS
The Quick Start Guide (pamphlet with step-by-step instructions) is not providing effective guidance.
Usability Testing
10 participants completed our usability test, which were all conducted remotely over Zoom. The first portion of the usability test asked participants to explain their experience up until that point, which included the enrollment process and kit delivery. Then participants were asked to speak their thoughts out-loud as they completed their kits. Finally, the participant was asked follow-up questions about their swab experience and questions about expectations for getting kit results. About 3-4 days after the usability test, participants were called by one member of the research team. During the short (5-10 minute) phone call the participant answered questions providing a retrospective perspective on the test overall and how they found the process of getting their results to be.
Data Analysis & Synthesis
We coded each interview and identified insights and key themes in the data. We used affinity diagramming to group together insights and begin generating recommendations. We based all recommendations on data that came from direct observations of participants.
User Journey
The research team created a User Journey Map to visualize where the pain-points in the procedure were located. We segmented the user journey into 4 stages: recruitment, sign-up, kit completion, and awaiting results. For each action within each stage, we observed the participants' experiences and coded them as being positive, negative, or neutral.
Affinity Diagraming
In addition to the User Journey Map, the team created a digital affinity diagram to help organize all of the information that was collected during the interviews. We first pulled significant and noteworthy observations from out notes and then discussed what insights could be synthesized from them. The final step was to brainstorm data-supported recommendations.
Image 1: Affinity Diagram displayed on MacBook Pro screen.
Key Findings
The affinity diagram and user journey allowed us to synthesize a list of key findings that could be used to generate solutions to improve the experience for future participants. The following findings are not exhaustive and represent findings we observed in the majority of participants (5+). We had many minor findings that influenced our recommendations but were not common across all of the usability tests we conducted.
#1
Participants did not know when to register their kits. Most did not complete when they were "supposed" to.
#2
Participants were satisfied with the speed of the process, from enrollment to receiving their results.
#3
Participants were overwhelmed with the number of communications (texts & emails) they received.
#4
Participants considered the swab collection process to be ”easy,” “straightforward,” and “convenient.”
#5
All but one participant noted that they would watch the educational video, but none knew it existed.
#6
All participants correctly packaged their kits, but they were cautious about reading materials to get it right
Recommendations
In writing our recommendations, we carefully considered all of the data we collected from the usability tests and the datasets we reviewed. We explored many potential solutions and chose to narrow our recommendations to the ones we considered to be the most effective. We also took into account the amount of work necessary for implementation, so as to not burden the SCAN team.
Streamline Content
Most participants did not read all of the Quick Start Guide (picture below). We recommend simplifying the content, reducing the word count, increasing white space, and focusing on images. Renaming the guide "Instructions" would also better communicate the guide's purpose.
Redesign Quick Start Guide
Participants did not follow the steps on the guide as written, but many did read all of the guides. This was especially true of the kit registration step. We recommend reducing the number of steps and moving the kit registration step off the guide. Sending an email (asking participants to register their kits) after enrollment could help participants know they should complete that step before opening their kit. The remainder of the steps should remain on the guide.
Image 2: Inside trifold of “Quick Start Guide” distributed by SCAN in their kits. Half of full guide shown.
Image 3: Simplified “Quick Start Guide.” Full (front & back) of bifold shown.
Clear Notifications At Relevant Times
We suggest using the notifications for confirmations of step completion or reminders for patients suspected of dropping off (reminders). Reducing the number of communication channels would also help reduce complexity, receiving information from multiple emails and phone numbers confuse participants.
Kit Registration Process
Participants do not understand the significance of the kit registration step. 3 participants thought they had already completed the step. Changing the name of the step to "Activation" might help participants understand that the step should be done first. Again, sending the link that is necessary for this step via the participant's preferred method of contact or proving the link via QR code could help reduce registration errors. Below is a diagram showing when participants completed this step vs. when they were supposed to.
Image 4: Timeline of approximately when participants completed the “Register Kit” step.
More Web Resources
Promote the resources available for participants. Create an FAQ page specifically for participants. Further, promote the educational video. These two resources could be accessed by a QR code on the swab kit box, which presently only has the SCAN logo.
Reflection
In writing our recommendations, we carefully considered all of the data we collected from the usability tests and the datasets we reviewed. We explored many potential solutions and chose to narrow our recommendations to the ones we considered to be the most effective. We also took into account the amount of work necessary for implementation work, so as to not burden the SCAN team.
© Alfie Aguilar Vidrio 2023. All rights reserved.