Evaluating the Usability of an Online Cannabis Marketplace

Design Brief

Project Overview

Type of Project: Usability Testing

For this project, our team conducted a series of usability evaluations on Leafly.com. We were tasked with performing a heuristic evaluation, a cognitive walkthrough, a usability test, and a comparison study of the Leafly.com platform.

Problem

Identify usability issues through various usability testing methods and provide recommendations for correcting any issues and improving the user experience.

Role

One of three UX designers/researchers who collaboratively evaluated the usability of an existing website using various methods for a graduate-level usability methods course.

Methods

Heuristic Evaluation

First, our team conducted individual heuristic evaluations of the Leafly website. Our intention was to uncover as many issues as possible among the team to combine our findings later.

Following our individual heuristic evaluations, we combined our findings into a combined heuristic evaluation. As a team, we discussed our individual findings and developed a prioritized report.

Cognitive Walkthrough

Each team member identified one core task for analysis and conducted a cognitive walkthrough.

All steps of each task were evaluated using the following questions:

  1. Will the user try to achieve the right effect?
  2. Will the user notice that the correct action is available?
  3. Will the user associate the correct action with the effect to be achieved?
  4. Will the user see that progress is being made toward the solution?

Successes and failures with supporting reasoning were recorded for each question. After completing our walkthroughs, we came together and evaluated each walkthrough as a group. Finally, we participated in several rounds of discussion and analysis to summarize our findings and recommendations.

Usability Testing

Through our heuristic evaluations and cognitive walkthroughs, we determined that the usability issues uncovered may have a negative impact on the experience of users attempting to purchase products. Based on this finding, we focused our usability tests on assessing how easily and efficiently novice Leafly.com users could find and purchase products. Our goals were to identify any usability issues users may encounter while buying products on Leafly.com and provide associated recommendations for improvement. To fit within our time constraints, we focused our testing on the desktop version of Leafly’s website.

Comparative Usability Test

Additionally, our team conducted a comparative usability test of Leafly.com and Weedmaps.com to identify users’ ability to successfully find dispensary hours of operation utilizing the dispensary map on each website. We collected data, including participants’ time on task and participants’ difficulty rating of the task. Our goal was to answer the following test objective through our collected data: Can participants quickly and easily find the hours of operation of a designated dispensary?

Our test plan was designed to compare the learnability and efficiency of the same key task between the Leafly and Weedmaps desktop platforms. The tests were completed using Figma prototypes of the two interfaces connected to the Maze product research platform, allowing for documentation of each tester’s task path, success rate, and time on task.

Techniques Used:

Cognitive walkthrough, comparative usability test, heuristic evaluation, moderated usability testing, statistical tests, unmoderated usability testing

Tools Used:

Excel, Figma, Maze, RStudio, Zoom

Results

Findings

Based on the usability tests conducted, following is an abbreviated list of findings and recommendations to improve the usability of the Leafly website.

Objective: How easily are users able to find dispensaries in a specific location?

Finding one: Some participants did not notice the change location link at the top right of the global navigation due to its relatively small size and location. Participants expected to be able to change their location within the dispensary map.

Recommendation: Make the existing link to change location more prominent. Add the ability to change location within the dispensary map sidebar.

Screenshot showing small size and poor location of the change location link.

Finding two: Two out of four participants encountered an error while trying to change their location that prevented them from continuing to use the website and did not provide instructions regarding how to proceed.

Recommendation: The language in the error message is targeted towards developers, which is not helpful for a typical Leafly user, who may not understand what the error is. The error message should be rewritten so that it is understandable to the average user and assists users in how to proceed.

Screenshot showing ambiguous error message language.

Finding three: Participants expressed confusion and frustration when they searched for a dispensary by location and realized they had to scroll past sponsored results to find the results relevant to their search parameters.

Recommendation: Move sponsored dispensary listings to another location or differentiate them in a more obvious way from search results so that users recognize them as sponsored results more quickly.

Screenshot showing how sponsored dispensaries are currently displayed before search results.

Comparative Usability Test Results

The comparative usability test we conducted to determine whether or not any significant difference existed between the time required to locate a dispensary’s hours of operation on Leafly or Weedmaps resulted in no significant difference between the two platforms. Similarly, the statistical test we conducted to evaluate any difference between perceived user difficulty locating a dispensary’s hours of operation on Leafly or Weedmaps resulted in no significant difference between the two website platforms.

We interpreted these results as additional evidence that the usability issues identified through our heuristic evaluations, cognitive walkthroughs, and usability testing of Leafly.com are of critical importance to fix. Once the usability issues identified have been fixed and the recommendations suggested have been implemented, we predict that Leafly will have a competitive advantage over similar sites, including Weedmaps, in future evaluations.

Lessons Learned

Through this project, I learned how to effectively combine various methods of usability testing and evaluation to validate initial findings and provide a more thorough assessment of usability.

Future Work

If I could conduct this project again, I would find a tool that would allow me to conduct a comparative usability test using live websites, rather than static prototypes. I think this limited the insights we were able to gather during usability testing as users were only able to click to areas that we enabled and could not freely navigate.