Putting Gaze into Context: A Framework for Analyzing Gaze Behavior in Interactive and Dynamic Environments

Conference paper


Thomas Bader
Jürgen Beyerer


IUI Workshop on Eye Gaze in Intelligent Human Machine Interaction, 2010.


International Conference on Intelligent User Interfaces, Workshop on Eye Gaze in Intelligent Human Machine Interaction, Hong Kong, February 7, 2010



Gaze data contains valuable information about user's cognitive processes during execution of a task. In order to use this information, e.g., for studying user's strategies or for designing new gaze-based interaction techniques for HCI, gaze data needs to be aligned with the task executed by the user.

In this paper we propose a novel framework based on the theory of Markov Decision Processes for putting gaze data into context, allowing for automated interpretation of gaze position and movement with respect to the task performed by the user. The model can be used for both, offline and online analysis of gaze data. We evaluate the proposed model with an indirect object manipulation task and demonstrate how it can be used for intention recognition and/or detection of a mismatch of the user's mental model.