ethod | Stage in Process | Users Needed | Advantages | Disadvantages |
Traditional Testing | Iterative design, formative evlauation, final testing | 3 + | | Time consuming and expensive |
Heuristic evaluation | Competitive analysis, final testing | None | Finds individual usability problems. Can address expert user issues. | Does not involve real users, so does not find "suprises" relating to their needs. |
Performance measures | | 10 + | Hard numbers. Results are easy to compare | Does not find individual usability problems. |
Thinking aloud | Iterative design, formative evaluation | 3-5 | Pinpoints user misconseptions. Inexpensive | Unnatural for users. Hard for expert users to verbalize. |
Observation | Task analysis, follow up studies | 3 + | Ecological validity; reveals users' real tasks. Suggests functions and features. | Appointments hard to set up, No experimenter control |
Surveys | Task analysis, follow-up studies | 30 + | Finds subjective user preferences. Easy to repeat | Pilot work needed (to prevent misunderstandings). |
Interviews | Task analysis | 5 | Flexible, in-depth attitude and experience probing. | Time consuming. Hard to analyze and compare |
Focus groups | Task analysis, user involvement | 6-9 per group | Spontaneous reactions and group dynamics. | Hard to analyze. Low validity |
Logging actual product use | Final testing, follow-up studies | 20 + | Finds highly used (or unused) features. Can run continuously. | Analysis programs needed for huge mass of data. Violation of users' privacy |
User feedback | Follow-up studies | unlimited | Tracks changes in user requirements and views | Organization needed to handle responses |