The Internet Encyclopedia (Volume 3)

P1: JDW

Zimmerman WL040/Bidgolio-Vol I WL040-Sample.cls June 20, 2003 17:20 Char Count= 0

518 USABILITYTESTING:ANEVALUATIONPROCESS FORINTERNETCOMMUNICATIONS

Web sites. The lessons used from conducting usability

testing of existing Web sites can be used to revise the Web

site and improve it. Whenever and whatever usability test-

ing methods are used, the usability testing plan should

outline the tasks, time line, and responsibilities for the

usability team. Ideally, usability testing will use a team of

two or more usability practitioners or assistants to help

manage the logistics of collecting data.

Develop Clear Objectives

Begin by asking, “What decisions will I make based on the

usability testing?” Specify the questions the site sponsor

wants answered: “What do you want to know about the

site? Will you collect these answers early in site develop-

ment or after the site has been structured and designed?

Are you testing several components or dimensions of the

site, or focusing on just one? What are they?” Specific re-

search questions might include the following: What spe-

cific problems do users encounter when trying to find in-

formation on the Web site? How severe are the problems

that users encounter? How understandable do users find

the information to be?

As discussed earlier, a wide range of evaluation re-

search methodologies are available for conducting usabil-

ity testing. The selection depends on the specific evalua-

tion to be undertaken. For example, user-centered design

can integrate focus groups, surveys, card sorting, protocol

analyses, and follow-up surveys. Focus groups and sur-

veys provide a better understanding of users’ Web expe-

rience, content knowledge, and vocabulary. Card sorting

produces an understanding of how users organize the in-

formation for the Web site and the terminology they use.

Verbal protocol analyses identify user-centered problems

with Web site prototypes.

Analyze Audiences

Participants should have the same characteristics as the

users of the Web site. Thus, a profile is needed to ensure

the recruited participants have the same background as

the intended uses. Profiling the intended users, begin by

asking the following questions:

Who will be the primary users?


Who are the secondary users?

Then ask

What roles do audience members play in their respective

organization?


Why will they be using the Web site?


What information might they need?


How familiar will they be with the content terminology?


What is the level of computer knowledge and skills of

the users?


What Internet technology skills will users have?


What are the demographics of the users?


When will they use the site?


How have they accessed the Internet? By modem? T1

lines? Cable? Other systems?

What are the users’ characteristics relevant to the Web

site design and content?


What are users’ usage patterns?

Although many organizations rely on informal audi-

ence analyses, others conduct formal analyses. Informal

audience analyses consists of members of the Web site

development team identifying the audience and then try-

ing to describe its characteristics based on their prior

experience. Formal analyses include such methods as in-

depth interviews, focus groups, and surveys. Using mul-

tiple methods helps develop a better description of the

intended audience and minimizes the limitations of any

one methodology.

Based on in-depth ethnographic interviews, a Persona

describes the flow of work (i.e., how an individual works

throughout the day) and identifies his or her skills, at-

titudes, working environment, and goals (Cooper 1999;

Goodwin, 2002). Personas guide the design of Web sites

and other communications and products. Other authors

have focused more narrowly on flow for modeling individ-

uals’ behaviors (Csikszentmihalyi, 1997), and others have

used flow to analyze users Web behaviors of e-commerce

sites (Hoffman & Novak, 1996; Novak, Hoffman, & Yung,

2000).

Conduct Task Analysis

With the objectives clearly stated, conduct a task analy-

ses to determine the steps and problems users might en-

counter when using the Web site. Task analyses entails

determining the sequence of steps needed to carry out spe-

cific activities on the Web site. If a potential user wants

to find specific information on a Web site, identify the

needed steps to interact with the Web site design and its

content. Break down the tasks into their basic elements

and isolate each cognitive and physical action required to

complete the task. Identify potential problems users might

encounter. For guidance on task analyses, see Hackos and

Redish (1998), Jonassen, Hannum, and Tessmer (1989),

and Zemke and Kramlinger (1987).

Develop Scenarios for Usability Testing

Next, develop scenarios to guide participants through the

test. For some measurement techniques such as card sort-

ing, the scenario can be short, simple, and straightfor-

ward. For others, such as verbal protocol analyses of a

Web site, the scenario can be lengthy and detailed. Key to

developing scenarios is giving participants the opportu-

nity to navigate the Web site moving through its structure,

using the different interface designs and interacting with

different design elements and different content. Scenarios

need to reflect different situations and need to be written

in language that participants understand.

Select and Develop Measurements

After completing task analyses, select the appropriate us-

ability testing technique. Table 3 identifies the type of

information needed, at what point in Web site develop-

ment the technique is often used, and then the commonly

used usability testing techniques. Formative information