SCAN Testing Kits

My experience creating and conducting a usability study for the Seattle Coronavirus Assessment Network to identify pain-points.

Background

Background

Background

The Greater Seattle Coronavirus Assessment Network (SCAN) is a joint research study between UW Medicine, Fred Hutch, and the Seattle Flu Study that is investigating the coronavirus (COVID-19) outbreak. SCAN's remote enrollment and at-home self-collection nasal swab kits allow for increased disease surveillance and testing capacity, especially for populations underrepresented in traditional clinical surveillance. SCAN wanted to improve the accessibility and usability of the SCAN testing kit to help increase testing capacity and to reduce participant drop off.

Deliverables

Usability Study Design
User Journey of SCAN Procedure
Comprehensive Report

Context

Time Frame: 12 Weeks (Autumn Quarter 2020)
Team: Alfie Vidrio, Ana Rodriguez, Azima Mansuri, & Rachel Feltes
Role: UX Researcher, Usability Test Moderator, Data Analyst

Exploratory Research

We began by conducting a qualitative analysis of the SCAN participant datasets, including a post-study survey (which was sent to all study participants), and the support emails that were sent to SCAN. The massive trove of data that SCAN provided allowed us to narrow our scope and determine which parts of the procedure warranted further attention.

Hypotheses to Test

1 /  Participants have a difficult time completing the kit registration step and quit at that step due to frustration

2 / 
Participants are not provided with sufficient drop-off and pick-up times for their kits

3 / 
The Quick Start Guide (pamphlet) is not providing effective guidance for procedure 

Usability Testing

Ten participants completed our usability test, which were all conducted remotely over Zoom. The first portion of the usability test asked participants to explain their experience up until that point, which included the enrollment process and kit delivery. Then participants were asked to speak their thoughts out-loud as they completed their kits. Finally, the participant was asked follow-up questions about their swab experience and questions about expectations for getting kit results.

About 3-4 days after the usability test, participants were called by one member of the HCDE research team. During the short (5-10 minute) phone call the participant answered questions providing a retrospective perspective on the test overall and how they found the process of getting their results to be.

Affinity Diagram

We coded each interview and identified insights and key themes in the data. We used affinity diagramming to group insights and begin generating recommendations. We charted data themes, insights, and recommendations from interviews in this diagram, and later grouped recommendations to identify key findings.

User Journey

We segmented the journey into four stages: recruitment, sign-up, kit completion, and awaiting results. For each action within each stage, we observed the participants' experiences and analyzed them as positive, negative, or neutral. The steps in the light brown show the parts of the process that were only part of the usability study procedure.

Key Findings

The Affinity Diagram and User Journey allowed us to synthesize a list of key findings that could be used to generate solutions to improve the experience for future participants. The following findings are not exhaustive and represent findings we observed in the majority of participants (5+). We had many minor findings that influenced our recommendations but were not common across all of the usability tests we conducted.

Recommendations

In writing our recommendations, we carefully considered all of the data we collected from the usability tests and the datasets we reviewed. We explored many potential solutions and chose to narrow our recommendations to the ones we considered to be the most effective. We also took into account the amount of work necessary for implementation work, so as to not burden the SCAN team.

The following recommendations are condensed for display purposes. Please send me a message to read the entire report, I would be happy to provide a copy!

Streamline Content

Most participants did not read all of the Quick Start Guide (picture below). We recommend simplifying the content, reducing the word count, increasing white space, and focusing on images. Renaming the guide "Instructions" would also better communicate the guide's purpose.

Organize Instructions
& Step Order

Participants did not follow the steps on the guide as written, but many did read all of the guides. This was especially true of the kit registration step. We recommend reducing the number of steps and moving the kit registration step off the guide. Sending an email (asking participants to register their kits) after enrollment could help participants know they should complete that step before opening their kit. The remainder of the steps should remain on the guide.

Clear Notifications At Relevant Times

We suggest using the notifications for confirmations of step completion or reminders for patients suspected of dropping off (reminders). Reducing the number of communication channels would also help reduce complexity, receiving information from multiple emails and phone numbers confuse participants.

Kit Registration Process

Participants do not understand the significance of the kit registration step. 3 participants thought they had already completed the step. Changing the name of the step to "Activation" might help participants understand that the step should be done first. Again, sending the link that is necessary for this step via the participant's preferred method of contact or proving the link via QR code could help reduce registration errors. Below is a diagram showing when participants completed this step vs. when they were supposed to.

More Web Resources
For Participants 

Promote the resources available for participants. Create an FAQ page specifically for participants. Further, promote the educational video. These two resources could be accessed by a QR code on the swab kit box, which presently only has the SCAN logo.

Limitations

In writing our recommendations, we carefully considered all of the data we collected from the usability tests and the datasets we reviewed. We explored many potential solutions and chose to narrow our recommendations to the ones we considered to be the most effective. We also took into account the amount of work necessary for implementation work, so as to not burden the SCAN team.

The following recommendations are condensed for display purposes. Please send me a message to read the entire report, I would be happy to provide a copy!