P1: JDW
Zimmerman WL040/Bidgolio-Vol I WL040-Sample.cls June 20, 2003 17:20 Char Count= 0
INTEGRATINGUSABILITYINTO THEDESIGNPROCESS 517usability testing?” If so, consider alternative approaches.
If participants could be at any small risk, usability practi-
tioners must inform them of any potential risks posed by
taking part in the usability testing.
Such procedures have been formalized by the federal
government. Organizations funded by the federal gov-
ernment have institutional review boards (IRBs) that re-
view all research and evaluations involving human parti-
cipants. Although much of their efforts focus on medical
research, IRBs review social science research and usabil-
ity testing evaluations.
The IRB review process usually entails the usability
practitioner preparing a packet of required forms and
supporting materials. The IRB application form details
methodologies, participant requirements, location of the
study, explanations of the known risks, IRB contact infor-
mation, benefits of involvement, and confidentiality assur-
ances. The application packet must also include a copy
of the proposal, consent forms, questionnaires, research
protocols, debriefing materials, investigator resume, and
other required materials. The consent forms spell out
any known risks. For more information on IRBs, see the
National Institutes of Health (n.d.) Office of Human Sub-
jects Research Web page.
When conducting such usability testing and evalua-
tions, most businesses and commercial organizations ask
participants to sign a nondisclosure statement indicating
they will not disclose details of the confidential materials.
Legal departments can provide guidance on developing
non-disclosure statements.AVOIDING USABILITY PITFALLS
Dumas and Reddish (1999) noted a trend toward more
informational usability testing based on the need to do
more usability testing with fewer resources and obtaining
results more quickly and that usability practitioners and
Web site developers no longer need to justify the method-
ology or results of a particular usability testing cycle. They
noted its key value in diagnosing problems. That said, Web
developers and novices to usability testing need to be es-
pecially cognizant of its pitfalls and lack of consistency
across usability testing studies (Gray & Salzman, 1998;
Holleran, 1991).
Such findings call for a systematic approach to usabil-
ity testing to avoid its many pitfalls. A solid understanding
of the fundamentals of social science research methodolo-
gies (Babbie, 1992, 1998), evaluation research method-
ologies (Rossi, Freeman, & Wright, 1979), and specifi-
cally understanding experimental and quasi experimen-
tal design (Campbell & Stanley, 1963) and the threats
to validity and reliability provide a strong background
for developing systematic and careful usability testing
skills.INTEGRATING USABILITY INTO
THE DESIGN PROCESS
Approaching usability testing systematically as a
problem-solving process offers three advantages. First,
a framework guides testing and produces a system-
atic project design. Second, a problem-solving approach- Analyze Audiences
- Develop Scenario
- Develop Objectives
- Conduct Task Analysis
- Select & Develop Measurements
- Recruit Participants
- Analyze Data
- Collect Data
- Interpret Data
- Develop Project Management Plan
- Identify Strengths and Weaknesses
- Make Recommendations
Figure 1: A systematic approach to usability testing.ensures articulating the evaluation objectives to minimize
wasted time collecting useless data. Third, a problem-
solving approach documents the process needed to review
critical steps along the way. To illustrate, the following
discussion focuses on conducting verbal protocol testing
of a Web site for an Intranet (see Figure 1).Develop Project Management Plan
The key to successful usability testing is developing clearly
stated objectives and a plan outlining the steps and time
line for conducting the usability testing. Ideally, usability
testing will be conducted throughout the conceptualiza-
tion and development of the Web site, but usability testing
can be conducted at any time—even for well-established