stltoday.com Usability Study
One page printer-friendly version
stltoday.com is one of the most popular web sites based in the St. Louis area. This report lists the main findings of the study, with detail on some issues. Please contact us if you are interested in the full report.
Grant Consulting, Inc. sponsored and conducted this study. stltoday.com and Pulitzer Technologies are not associated in any way with the preparation of the study or the reporting of its results.
In general, users were successful and satisfied in browsing the main sections of stltoday.com and in finding current stories or features of interest. Most users had a positive experience in finding a used car that met a specific criterion. And the home page made a positive first impression on people. They not only found the page attractive but also recognized the main purpose for the web site.
The greatest difficulty came from finding an archived article about a major St. Louis story. No one was successful in finding and acquiring the article. Most were very frustrated with the task. Their degree of frustration strongly suggests changing the design to accommodate archived search. These results also warrant further study on why in general people use the stltoday.com "Site Search" and how successful they are when they do so.
A number of other opportunities for improvement are identified by this study.
An independent marketing research firm recruited and compensated all test participants.
The six test participants met the following qualifications:
Three of the participants have little or no previous familiarity with stltoday.com. The other three had moderate or high familiarity with the web site.
This study serves to offer an example of a usability study on a web site from St. Louis and of interest to most St. Louisans. Usability testing offers the best way to directly know what is happening between your customers and your technology.
Many readers of this web site are unfamiliar with usability testing. Of course, if you haven’t been involved with a usability test, you probably wonder whether or not it is worth the time and effort. If that happens to be the case for yourself, the following questions are offered to you:
A future article will discuss design ideas for addressing the usability issues of stltoday.com. If you have any design suggestions for stltoday.com, please send them either to firstname.lastname@example.org, or use the contact page.
Several links on the site were unclear or misled users. For example, some users took the “Desired vehicle is not listed on this page" link to mean that they were not looking at any cars that met their search criteria. In fact, the link simply provides a way to go back to the search page. (Note: This link has been changed since the study was done in early August, and now reads, “Return to Main Search Page".)
From stltoday.com. (The "Desired vehicle..." link has been changed since the study was done.)
Other links that were unclear or misleading include “urban diary,” “Research this vehicle”, and the links that appear at the end of an article to related stories.
Several people were frustrated that they could not select more than one item from a search criteria list. This especially bothered one user who wanted to select all three models of a car at one time. The instructions which indicated that she should be able to select up to five models especially aggravated her.
In fact, she could have selected all three “Bonneville” models for her search if she had known to press the Control key. However, she and other users did not know about or did not think about using this key.
Testers encountered their most severe problems when they attempted to find an archived story. The scenario asked them to find an article that came out shortly after Mark McGwire's historic 62nd home run. In the scenario, testers were given the correct spelling of McGwire’s name, the fact it was the 62nd home run that broke the single season home run record, and the fact that it occurred in September 1998.
Analysis from people’s behavior indicates that they had to succeed in seven different steps to find and buy an stltoday.com archived story:
People had severe difficulties with most of these steps. Everyone eventually gave up. The time before they decided to quit ranged from 4 minutes to 28 minutes. Only one person succeeded in finding an archived article that was relevant. However, that person could not get past the registration process.
Another person who gave up on finding an archived story about McGwire’s 62nd home run was instructed to go ahead and read a different archived story. She succeeded in registering herself, making the purchase, and reading that archived story. However, she felt “cheated” with the story purchased. The story had no real content, only captions to pictures that were not displayed.
Most people had significant difficulty finding the section for doing archives. Many of them started with the assumption that they could use the prominent “Site Search” bar on the home page. Others assumed that they could find an archive story about McGwire if they went to the Sports section or pages dedicated to the St. Louis Cardinals baseball team or Mark McGwire. In fact, the place to start finding archived stories is in the News section, under “P-D Archives”.
People expected an “Archives” subsection in Sports, and looked for it in the right navigation bar or at the bottom of the page.
The reader is invited to go to stltoday.com and to find an article published shortly after McGwire’s 62nd home run. If you want to see how to find it, the answer is provided here for your convenience, along with examples of searches done by the testers in the study.
The following scenario was used during the stltoday.com usability study. Due to difficulties in using the web site, no one was able to completely perform the scenario successfully.
You just spoke on the phone with a New York customer who happens to be a big baseball fan. During this conversation, he wondered if there was any way you could send to him an article that appeared in the St. Louis newspapers the day after the historic 62th home run of Mark McGwire.
You decide to go to stltoday.com to find a good article that covered the event that day. You don't recall the exact day of McGwire's home run, but you do remember that it happened in September 1998, and that it was the 62nd home run that broke the record for home runs in one season.
Since it is an old article, you expect that you might have to pay for it, and you have a credit card handy to do so.
You plan to print the article and mail it to your client.
To go to the archives search page
Below are some examples of the searches done during the study from either “site search” or the Archives search page. These searches did not lead to success for the users.
Grant Consulting, Inc.
located in metro St. Louis, MO, USA
1013 Bradington Court
Columbia, IL 62236
Copyright © 2003 by Grant Consulting, Inc. All Rights Reserved.