Participatory Heuristic Evaluations of Jeliot Mobile: End-users evaluating usability of their mlearning application

Abstract

The usability evaluations (testing and inspection) are utmost important in the development of learner-centered educational technology. Poorly designed learning tools put an additional interface learning load on learners' memory and cognition, which may result in significantly decreased attention to the main content learning goal. Nonetheless, discounted inspection (by usability experts) are commonly reported for such tools but expensive user-based testing is a rarity, mostly due to limited budgets and constrict timeframes. In this work, the authors propose to run discounted inspections with special end-user (domain expert) evaluators having a prior experience in usability. The authors postulate that putting discounted inspections into a participatory context shall bring the benefits of both type of usability evaluations to one. It shall also reduce costs typically associated with expensive user-based testing, thus removing hindrances. The argument is put to test via participatory heuristic evaluation experiments on a case tool with a large number of redundant evaluating groups to cross-validate results. Over 164 end-users with some prior usability experience, divided into 45 evaluating groups evaluate the case tool. The outcomes are promising. The dual-personae evaluators find significant amount of domain related problems (missed by usability experts in discounted methods) as well as interface related problems (missed by domain experts in testing), suggesting their usefulness in mixed method. The authors thus advise to evaluate the usability of low-budget educational technology projects with the proposed method.