customer success by design
articles and links
 

What an actual usability test looks and sounds like...

In this scenario, someone tries to find an archived story about Mark McGwire.

Windows Media Player, 100k streaming
4 min.
9 min.

RealPlayerG2, 80k speed
4 min.
9 min.

Best Playback -- Windows Media AVI file, includes codec install
4 min. 13 Mg download
9 min. 38 Mg download

Contact us if you want files sent to you via CD or other means.

Also available...
usability storyboard --illustrates how usability testing works

stltoday.com Usability Study --
No one could find and buy archived story

Testers encountered their most severe problems when they attempted to find an archived story. The scenario asked them to find an article that came out shortly after Mark McGwire's historic 62nd home run. In the scenario, testers were given the correct spelling of McGwire’s name, the fact it was the 62nd home run that broke the single season home run record, and the fact that it occurred in September 1998.

Analysis from people’s behavior indicates that they had to succeed in seven different steps to find and buy an stltoday.com archived story:

  1. Find the page on the site for doing archive searches.
  2. Enter search parameters that would provide relevant returns.
  3. Enter search parameters that would provide a manageable number of returns.
  4. Review the results.
  5. Choose an article.
  6. Provide personal registration and payment information.
  7. Get the desired story and determine if it is satisfactory.

People had severe difficulties with most of these steps. Everyone eventually gave up. The time before they decided to quit ranged from 4 minutes to 28 minutes. Only one person succeeded in finding an archived article that was relevant. However, that person could not get past the registration process.

Another person who gave up on finding an archived story about McGwire’s 62nd home run was instructed to go ahead and read a different archived story. She succeeded in registering herself, making the purchase, and reading that archived story. However, she felt “cheated” with the story purchased. The story had no real content, only captions to pictures that were not displayed.

Most people had significant difficulty finding the section for doing archives. Many of them started with the assumption that they could use the prominent “Site Search” bar on the home page. Others assumed that they could find an archive story about McGwire if they went to the Sports section or pages dedicated to the St. Louis Cardinals baseball team or Mark McGwire. In fact, the place to start finding archived stories is in the News section, under “P-D Archives”.

From stltoday.com
From stltoday.com

People expected an “Archives” subsection in Sports, and looked for it in the right navigation bar or at the bottom of the page.

The reader is invited to go to stltoday.com and to find an article published shortly after McGwire’s 62nd home run. If you want to see how to find it, the answer is provided here for your convenience, along with examples of searches done by the testers in the study.

back to main page of stltoday.com usability study

Written by Joe Grant
Grant Consulting, Inc.

Posted on August 30, 2001

 

 
Grant Consulting, Inc.
located in metro St. Louis, MO, USA
1013 Bradington Court
Columbia, IL 62236
info@grantconsulting.com
(314) 581-0384
Copyright © 2003 by Grant Consulting, Inc. All Rights Reserved.