CHI’95 MOSAIC OF CREATIVITY ■ May 7-11 1995

Usability Inspection Methods

Jakob Nielsen
Stmsoft
2550 Garcia Avenue
Mountain View, CA 94043-1100
Email: [Link]@[Link] (primary), [Link]@[Link] (backup)

ABSTRACT INSPECTION METHODS

Usability inspection is the generic name for a set of cost- Usability inspection is the generic name for a set of methods
effective ways of evaluating user interfaces to find usability that are all based on having evaluators inspect the interface.
problems. They are fairly informal methods and easy to use. Typically, usability inspection is aimed at finding usability
problems in a design [13], though some methods also
Keywords: Usability engineering, heuristic evaluation,
address issues like the severity of the usability problems and
cognitive walkthroughs, pluralistic walkthroughs, feature
the overall usability of an entire design [28]. Many inspec-
inspection, consistency inspection, standards inspection.
tion methods lend themselves to the inspection of user inter-
face specifications [17] that have not necessarily been
INTRODUCTION
implemented yet, meaning that inspection can be performed
Software inspection [1] [7] has long been used as a method early in the usability engineering lifecycle [19].
for debugging and improving code. Similarly, usability
inspection [25] has seen increasing use since about 1990 as
●
Heuristic evaluation [23][27] is the most informal
a way to evaluate user interfaces. The four basic ways of method and involves having usability specialists judge
evaluating user interfaces are automatically (usability mea- whether each dialogue element follows established
sures computed by running a user interface specification usability principles (the “heuristics” [15] [18] [21] [22]).
through some program), empirically (usability assessedby ●
Cognitive walkthroughs [12] [29][30] use a more explic-
testing the interface with real users), formally (using exact itly detailed procedure to simulate a user’s problem solv-
models and formulas to calculate usability measures), and ing process at each step through the dialogue, checking
informally (based on rules of thumb and the general skill if the simulated user’s goals and memory content can be
and experience of the evaluators). Under the current state of
assumed to lead to the next correct action.
the art, automatic methods do not work and formal methods
are very difficult to apply and do not scale up well to handle
●
Formal usability inspections [9] use a six-step procedure
larger user interfaces. with strictly defined roles to combine heuristic evalua-
tion and a simplified form of cognitive walkthroughs.
Empirical methods are the main way of evaluating user ●
Pluralistic walkthroughs [3] [4] are meetings where
interfaces, with user testing probably being the most com-
users, developers, and human factors people step
monly used method. Often, real users can be difficult or
through a scenario, discussing each dialogue element.
expensive to recruit in sufficient numbers to test all aspects
of all the versions of an evolving design, leading to the use
●
Feature inspection [2] lists sequence of features used to
of inspection as a way to “save users .“ Furthermore, project accomplish typical tasks, checks for long sequences,
schedules or budgets sometimes impose restrictions that cumbersome steps, steps that would not be natural for
make informal methods like inspection desirable as a “dis- users to try, and steps that require extensive knowledge/
count usability engineering” solution [16] [21] since they are experience in order to assess a proposed feature set.
highly cost-effective [8]. Several studies have shown that ●
Consistency inspection [31] has designers representing
usability inspection methods are able to find many usability multiple projects inspect an interface to see whether it
problems that are overlooked by user testing but that user
does things in the same way as their own designs.
testing also finds some problems that are overlooked by
inspection, meaning that the best results can often be
●
Standards inspection [31] has an expert on some inter-
achieved by combining several methods [5] [6] [1 1]. face standard inspect the interface for compliance.

Heuristic evaluation, cognitive walkthroughs, feature

Permission to copy without fee all or part of this material is inspection, and standards inspection normally have the
granted provided that the copies are not made or !istr’ibuted for
direct commercial advantage, the ACM copyright notice and the interface inspected by a single evaluator at a time (though
title of the publication and its date appear, and notice is given heuristic evaluation is based on combining inspection
that copying is by permission of ACM. To copy otherwise, or to reports from a set of independent evaluators to form the list
republish, requires a fee and/or specific permission. of usability problems). In contrast, pluralistic walkthroughs
CHI Companion 95, Denver, Colorado, USA
@ 1995 ACM 0-89791 -755 -3/95/0005 ...$3.50

377
I Tutorials I May 7-11 1995 ■ CHI ’95 MOSAIC OF CREATIVITY

and consistency inspections are group inspection methods. of walk-up-and-use interfaces. Proc. ACM CHI’90 Conf.
FinaJly, formal usability inspections combine individual and (Seattle, WA, April 1-5), 235-242.
group inspections. Many usability inspection methods are 13. Mack, R. L., and Montaniz, F. (1994). Observing, predicting
so easy to apply that it is possible to have regular developers and analyzing usability problems. In Nielsen, J., and Mack,
R. L. (Eds.), Usability Inspection Methods, John Wiley &
serve as evaluators, though better results are normally
Sons, New York, 293–336.
achieved when using usability specialists [20].
14. Mack, R. L., and Nielsen, J. (1993). Usability inspection
methods. ACM SIGCHI Bulletin 25.1 (January), 28-33.
Acknowledgments 15. Molich, R., and Nielsen, J. (1990). Improving a human–
This summary is based on the author’s own work, presentations at computer dialogue. Comm. ACM 33,3 (March), 338–348.
the CHI’92 workshop on usability inspection methods [14], and 16. Nielsen, J. (1989). Usability engineering at a discount. In
chapters in the recent Usabilisy Inspection Methods book [25]. Salvendy, G., and Smith, MJ. (Eds.), Designing and Using
Workshop and book contributors include Brigham Bell, Randolph Human–Computer Interfaces and Knowledge Based Sys-
Bias, Louis A. Blatt, Janice Bradford, Patricia Brooks, George tems. Elsevier Science Publishers, Amsterdam, the Nether-
Casaday, Heather Desurvire, Robin Jeffries, Sandra Jones, lands. 394-401.
Michael J. Kahn, Clare-Marie Karat, James A, Knutson, Thomas 17. Nielsen, J. (1990). Paper versus computer implementations
Lanzetta, Clayton Lewis, Robert L. Mack, AlIan MacLean, Frank as mockup scenarios for heuristic evaluation. Proc. IFIP
Montaniz, Peter Poison, Arnauda Prail, John Rieman, Linda Tse, INTERACT90 Third Intl. Conf. Human-Computer Interac-
Cathleen Wharton, and Dennis Wixon. The present discussion of
tion (Cambridge, U.K., August 27-31), 315–320.
usability inspection is the responsibility of the author and does not
18. Nielsen, J. (1990). Traditional dialogue design applied to
necessarily represent the positions of these other people.
modem user interfaces. Communications of the ACM 33, 10
(October), 109-118.
References 19. Nielsen, J. (1992). The usability engineering life cycle. IEEE
1. Ackerman, A. F., Buchwald, L. S., and Lewski, F. H. (1989). Computer 25,3 (March), 12-22.
Software inspections: An effective verification process. 20. Nielsen, J. (1992). Finding usability problems through heu-
IEEE Software 6,3 @fay), 31–36. ristic evaluation. Proc. CHI’92 Conf. (Monterey, CA, May
2. Bell, B. (1992). Using programming walkthroughs to design 3-7), ACM, New York, 373–380.
a visual language. Technical Report CU-CS-581 -92 (Ph.D. 21. Nielsen, J. (1993). Usabili~ Engineering, Academic Fress,
Thesis), University of Colorado, Boulder, CO. Boston, MA.
3. Bias, R. (1991). Walkthroughs: Efficient collaborative test- 22. Nielsen, J, (1994). Enhancing the explanatory power of
ing. IEEE Software 8,5 (September), 94–95. usability heuristics. Proc. ACM CHI’94 Conf. (Boston, MA,
4. Bias, R. G. (1994). The pluralistic usability walkthrough: April 24-28), 152-158.
Coordinated empathies. In Nielsen, J., and Mack, R. L. 23. Nielsen, J. (1994). Heuristic evaluation. In Nielsen, J., and
(Eds.), Usability Inspection Methods, John Wiley & Sons, Mack, R. L. (Eds.), Usability Inspection Methods, John
New York, 65-78. Wiley & Sons, New York, 25-64.
5, Desurvire, H. W. (1994). Faster, cheaper!! Are usability 24. Nielsen, J. (1994). Scenarios in discount usability engineer-
inspection methods as effective as empirical testing? In ing. In Carroll, J. M. (Ed.), Scenario Based Design: Envi-
Nielsen, J., and Mack, R. L. (Eds.), Usability Inspection sioning Work and Technology. Book under preparation.
Methods, John Wiley & Sons, New York, 173-202. 25. Nielsen, J., and Mack, R. L. @ds.) (1994). Usabili~ inspec-
6. Desurvire, H. W., Kondziela, J. M., and Atwood, M. E. tion Methods, John Wiley & Sons, New York.
(1992). What is gained and lost when using evaluation 26. Nielsen, J., and Landauer, T. K. (1993). A mathematical
methods other than empirical testing. In Monk, A., Diaper, model of the finding of usability problems. Proc. ACM
D., and Harrison, M.D. (Eds.), People and Computers VIZ, INTERCHI’93 Conf. (Amsterdam, the Netherlands, 24–29
Cambridge University Press, Cambridge, U.K. 89-102. April), 206-213.
7. Fagan, ME. (1986). Advances in software inspection. IEEE Nielsen, J. and Molich, R. (1990). Heuristic evaluation of
27.
Transactions on Soflware Engineering 12,7 (July), 744- user interfaces. Proc. ACM CHI’90 Conf. (Seattle, WA, April
751.
1- 5), 249-256.
8. Jeffries, R., Miller, J.R., Wharton, C., and Uyeda, K.M. 28. Nielsen, J., and Phillips, V. L. (1993). Estimating the relative
(1991). User interface evaluation in the real world: A com-
usability of two interfaces: Heuristic, formal, and empirical
parison of four techniques. Proc. ACM CH~91 Conf. (New
methods compared. Proc. ACM INTERCHI’93 Conf.
Orleans, LA, April 28- May 2), 119-124.
(Amsterdam, The Netherlands, April 24-29), 214-221.
9. Kahn, M. J., and Prail, A. (1994). Formal usability inspec- 29. Poison, P., Lewis, C., Rieman, J., and Wharton, C. (1992).
tions. In Nielsen, J., and Mack, R L. (Eds.), Usability Inspec-
Cognitive walkthroughs: A method for theory-based evalua-
tion Methods, John Wiley & Sons, New York, 141–172.
tion of user interfaces. International Journal of Man-
10. Karat, C. (1994). Comparison of user interface evaluation Machine Studies, 36,5,741-773.
methods. In Nielsen, J., and Mack, RL. (Eds.), Usability
30. Wharton, C., Rieman, J., Lewis, C., and Poison, P. (1994).
Inspection Methods, John Wiley & Sons, New York, 203-
The cognitive walkthrough method: A practitioner’s guide.
232.
In Nielsen, J., and Mack, R. L. (Eds .), Usability Inspection
11. Karat, C., Campbell, R. and Fiegel, T. (1992). Comparison Methods, John Wiley & Sons, New York, 105-140.
of empirical testing and walkthrough methods in user inter-
31. Wixon, D., Jones, S., Tse, L., and Casaday, G. (1994).
face evaluation. In Proceedings of cHI’92 (Monterey, Cali-
Inspections and design reviews: Framework, history, and
fornia, May 3-7, 1992), ACM, New York, 397404.
reflection. In Nielsen, J., and Mack, R L. (Eds.), Usability
2. Lewis, C., Poison, P., Wharton, C., and Rieman, J. (1990). Inspection Methoak, John Wiley& Sons, New York, 79-104.
Testing a walkthrough methodology for theory-based design

378