P1: JDW
Zimmerman WL040/Bidgolio-Vol I WL040-Sample.cls June 20, 2003 17:20 Char Count= 0
522 USABILITYTESTING:ANEVALUATIONPROCESS FORINTERNETCOMMUNICATIONSanswer questions and explore solutions to the identified
problems with the Web site developers.PRODUCING A VIDEOTAPE
In some cases, preparing a short videotape of users inter-
acting with the Web site provides a powerful illustration
of the problems participants encountered and how they
struggled to solve the problems. Such short clips can re-
inforce the points made in the report. With advances in
MS PowerPoint and other presentation software, video
clips can be integrated into the PowerPoint file.CONCLUSION
In this chapter, we have introduced basic concepts of
usability testing and provided resources for readers who
want to consider conducting usability testing or pur-
chasing such services. Usability testing, if handled care-
fully, can provide ways to develop user-centered Web
sites, software, printed publications, and hardware for
diverse purposes.GLOSSARY
Card sorting Used in the iterative Web design to iden-
tify the hierarchical organization and terminology for
a group of concepts based on users’ frame of refer-
ence. Users sort and group items listed on a set of
cards according to how they would collect and label the
concepts.
Experimental design The protocol or plan for conduct-
ing research.
Heuristic evaluation The systematic Web site review
conducted by one or more usability experts to assess
the site’s usability according to generally accepted stan-
dards. The technique identifies major usability prob-
lems and produces a comprehensive site evaluation.
Institution review boards (IRB) Organizational com-
mittees, required by U.S. government regulations, that
review research project designs to minimize partici-
pants’ risk, to ensure participants’ understanding of the
risks of being involved in a research project, and to en-
sure participant’s rights.
Participants Individuals who are tested or observed as
part of a research project and from whom data are
gathered.
Practitioners Individuals conducting usability testing.
They may be software or systems engineers, human
factors engineers, computer programmers, graphical
user interface developers, or Web designers.
Purposive sample A nonrandom sample of partici-
pants intentionally chosen to meet a set of criteria or
characteristics of a larger population.
Qualitative research The analysis of observations or
events in the field that does not rely on numerical mea-
surement and analyses of variables.
Quantitative research Research that measures vari-
ables collected during observations and experiences
that focuses on the frequency and circumstances under
which a variable occurs.Random sample The selection of small group or subset
of a larger population of research participants chosen
by a method ensuring everyone in the population is
equally likely to be selected.
Reliability The degree to which a measurement is con-
sistent in producing the same or nearly same answer
at different times.
Users Any of a group of individuals who employ a piece
of technology to achieve a goal. Over time, users estab-
lish expectations related to the use of the technology.
Validity The degree to which a measurement actually
assesses the concept it is believed to test.
Verbal protocol analysis A technique for investigat-
ing how people solve problems. Applied in usability
testing, verbal or “think aloud” protocol uses a set of
scenarios requiring participants to carry out tasks on a
Web site. As they work, they verbalize their thoughts,
explaining what they are doing and why.CROSS REFERENCES
SeeElectronic Commerce and Electronic Business; Human
Factors and Ergonomics; Universally Accessible Web Re-
sources: Designing for People with Disabilities; Web Search
Fundamentals.REFERENCES
Babbie, E. (1992).The practice of social research.Belmont,
CA: Wadsworth.
Babbie, E. (1998).The practice of social research( 8th ed.).
Belmont, CA: Wadsworth.
Babbie, E. (2001).The practice of social research(9th ed.).
Belmont, CA: Wadsworth.
Baddeley, A. D. (1999).Essentials of human memory.Hove,
England: Psychology Press.
Bailey, R. W. (1996).Human engineering performance.Up-
per Saddle River, NJ: Prentice Hall.
Barasalou, L. W. (1992).Cognitive psychology: An overview
for cognitive scientists.Mahwah, NJ: Erlbaum.
Barum, C. M. (2002).Usability testing and research.New
York: Longman.
Beabes M. S. & Flanders, A. (1995) Experiences with us-
ing contextual inquiry to design information.Technical
Communication, 42,409–420.
Beyer, H. R., & Holtzblatt, K. (1995, May). Apprenticing
with the customer.Communications of the ACM, 38,
45–52.
Bias, R. G., & Mayhew, D. (1994).Cost-justifying usability.
Boston, MA: Academic Press.
Bryman, A. E. (Ed.). (2001).Ethnography(4 Vols.). Thou-
sand Oaks, CA: Sage.
Burton, M. C., & Walther, J. B. (2001). The value of web
log data in use-based design and testing. Journal
of Computer Mediated Communication, 6.Retrieved
March 9, 2003, from http://www.ascusc.org/jcmc/vol6/
issue3/burton.html
Campbell, D. T., & Stanley, J. C. (1963).Experimental and
quasi-experimental designs for research.Chicago: Rand
McNally.
Cooper, A. (1999).The inmates are running the asylum.
Indianapolis, IN: Sams.