POnline: An Online Pupil Annotation Tool Employing Crowd-sourcing and Engagement Mechanisms

Files
Self archived version
published versionDate
2019Author(s)
Unique identifier
10.15346/hc.v6i1.9Metadata
Show full item recordMore information
Self-archived article
Citation
Gil de Gómez Pérez, David. Bednarik, Roman. (2019). POnline: An Online Pupil Annotation Tool Employing Crowd-sourcing and Engagement Mechanisms. Human computation, 6 (1) , 176-191. 10.15346/hc.v6i1.9.Rights
Licensed under
Abstract
Pupil center and pupil contour are two of the most important features in the eye-image used for video-based eye-tracking. Well annotated databases are needed in order to allow benchmarking of the available- and new pupil detection and gaze estimation algorithms. Unfortunately, creation of such a data set is costly and requires a lot of efforts, including manual work of the annotators. In addition, reliability of manual annotations is hard to establish with a low number of annotators. In order to facilitate progress of the gaze tracking algorithm research, we created an online pupil annotation tool that engages many users to interact through gamification and allows utilization of the crowd power to create reliable annotations \cite{artstein2005bias}. We describe the tool and the mechanisms employed, and report results on the annotation of a publicly available data set. Finally, we demonstrate an example utilization of the new high-quality annotation on a comparison of two state-of-the-art pupil center algorithms.