Accepted Tutorials

This tutorial addresses the challenges of performing meaningful, interpretable, and fair quantitative assessment of algorithms for eye movement event detection. We discuss the recommendations for evaluation techniques in different scenarios and for different end purposes, such as best algorithm selection, machine learning model training, or comparing patterns of manual annotators or algorithms in detail. The tutorial covers both generic good practices of algorithm evaluation, and aspects of sample- and event-level evaluation specific to the eye movement detection field. A large part of the tutorial discusses approaches to event-level evaluation, which is increasingly gaining momentum in the community in recent years. This type of analysis, while more complex to implement, enables more interpretable and meaningful evaluation outcomes. Through detailed discussions, examples, and experimental evidence, the tutorial provides a comprehensive overview of evaluation techniques for eye movement event detection. We highlight the advantages and drawbacks of various approaches from both theoretical and empirical standpoints. We also provide concrete recommendations for evaluation strategies and metrics, ensuring an interpretable and meaningful evaluation pipeline. On the whole, the tutorial intends to foster well-grounded choices of evaluation strategies and facilitate informed discussions within the research community, thereby improving evaluation and reporting standards in the field.
Presenters:
Mikhail Startsev, Independent researcher, Munich, Germany, mikhail.startsev@gmail.com
Dr. Mikhail Startsev has completed his PhD in 2020 at the Technical University of Munich (TUM), Germany after receiving a Diplom degree in Computational Mathematics and Informatics from the Lomonosov Moscow State University (LMSU), Russia, in 2015. During his PhD, Mikhail’s research centred around the human visual system, with a particular emphasis on eye movements and saliency modelling. He is currently working as an AI developer for medical imaging systems at a Brainlab AG subsidiary while continuing independent research in the eye movement field.
Raimondas Zemblys, Smart Eye AB, Göteborg, Sweden, raimondas.zemblys@gmail.com
Dr. Raimondas Zemblys received his PhD from Kaunas University of Technology, Lithuania in 2013 and worked as a postdoc researcher at Lund University, Sweden in 2013–2015 and Michigan State University, US in 2017–2018. He is currently a Senior Research Scientist at Smart Eye AB, Sweden while also continuing independent research in the field of eye movements. His main research interests are eye-tracking methodology and data quality, event detection and applications of deep learning for behavioral data analysis.
Tentative Schedule:
[13:30-14:15] Introduction; principles of good evaluation procedures (specialized for eye movement event detection evaluation use case)
[14:15-14:45] Overview of the evaluation methods and metrics
[15:30-16:10] Details of event-level evaluation
[16:10-16:40] Empirical effects of the differences between evaluation strategies
[16:40-17:00] Summary of principles and recommendations


This tutorial covers all aspects of an eye-tracking study from experimental design to data analysis. As an example, we present an eye-tracking study that investigates reading code vs. reading text. First, we give an introduction to experimental design and the preparation of an eye-tracking study using PsychoPy Builder. During a hands-on session, participants will collect eye-movement data in teams using Gazepoint eye trackers that will be provided. Finally, the tutorial presents details of a Python-based gaze analytics pipeline used to extract raw eye movement data, detect fixations via velocity-based filtering, collate data for statistical evaluation, analyze and visualize results using R. Attendees of the tutorial will have an opportunity to run the scripts of an analysis of gaze data collected from an example study. The tutorial covers basic eye movement analytics, e.g., fixation count and dwell time within AOIs, as well as advanced analysis using gaze transition entropy.
Presenters:
Nina A. Gehrer, University of Tübingen, Germany, nina.gehrer@uni-tuebingen.de
Nina Gehrer is is a postdoctoral researcher at the Department of Clinical Psychology and Psychotherapy at the University of Tübingen, Germany. She received her MSc in 2015 and completed her PhD in 2020. Her primary research interest focuses on biases in (social) information processing associated with psychological disorders (e.g., antisocial personality disorder, attention deficit hyperactivity disorder, eating disorders, etc.). In this context, she designed and conducted several eye-tracking studies over the last years. She gives courses in clinical psychology at the University of Tübingen and was co-presenter of tutorials on experimental design and gaze analytics at the ACM Symposium on Eye Tracking Research & Applications (in 2018, 2019, 2021, and 2022).
Krzysztof Krejtz, SWPS University, Poland, kkrejtz@swps.edu.pl
Krzysztof Krejtz is a psychologist at SWPS University of Social Sciences and Humanities in Warsaw, Poland, where he is leading the Eye Tracking Research Center. In 2017 he was a guest professor at Ulm University, in Ulm, Germany. He gave several invited talks at e.g., Max-Planck Institute (Germany), Bergen University (Norway), and Lincoln University Nebraska (USA). He has extensive experience in social and cognitive psychology research methods and statistics. In his research, he focuses on the use of eye tracking method and developing a second-order eye data-based metrics that may capture the dynamics of attention and information processing processes (transitions matrices entropy, ambient-focal coefficient K), dynamics of attention process in the context of Human-Computer Interaction, multimedia learning, media user experience, and accessibility. He is a member of the ACM Symposium on Eye Tracking Research and Application (ACM ETRA) Steering Committee.
Andrew T. Duchowski, Clemson University, USA, duchowski@clemson.edu
Andrew Duchowski is a professor of Computer Science at Clemson University. He received his baccalaureate (1990) from Simon Fraser University, Burnaby, Canada, and doctorate (1997) from Texas A&M University, College Station, TX, both in Computer Science. His research and teaching interests include visual attention and perception, eye tracking, computer vision, and computer graphics. He joined the School of Computing faculty at Clemson in January, 1998. He is a noted research leader in the field of eye tracking, having produced a corpus of papers and a textbook related to eye tracking research, and delivered courses and seminars on the subject at international conferences. He developed and maintains the eye tracking laboratory at Clemson University, and teaches a regular course on eye tracking methodology attracting students from a variety of disciplines across campus.
Tentative Schedule:
Session 1: Introduction & Applications (75 min); Duchowski
Session 2a: Experimental Design Principles (60 min); Gehrer
Session 2b: Experimental Design of a Mock Study (40 min); Krejtz
Session 3: Preparations for Using the Gaze Analytics Pipeline (60 min); Duchowski
Session 4a: Traditional Gaze Analytics (60 min); Krejtz
Session 4b: Advanced Gaze Analytics (60 min time permitting); All

Nowadays more and more researchers add additional measurements to their eye-tracking work, especially if you want to look for emotions or activations besides viewing patterns or if you want to detect micro-saccades during your EEG measurements. We have specialised in the integration and synchronisation of these measurements, after all BiSigma stands for Biometric Signal Measurement and Analysis, and with our Hard- and Softwaretools one can compare the different recordings with milliseconds accuracy. We will give participants the opportunity to learn about new ways of eye-tracking analysis together with additional synchronised data channels like GSR, EKG, EEG,NIRS, Facereading, Voiceanalysis, etc. and AI tools available for automatic object detection. Participants will then be able to run test experiments themselves and explore the different possibilities hands-on.
Presenters:
Dr. Achim Hornecker, BiSigma GmbH, ahorn@bisigma.de
Achim has a PhD in Mathematics from University of Freiburg and did develop the Brain Vision Analyser while working at Brain Products. Together with a team of software developers and data analysts, Achim now develops and implements cross-sector and cross-tool methods for the collection, evaluation and visualisation of data. Based on this experience, he also looks at the topic of big data analytics. His focus is not on databases or software manufacturers, but on the questions that need to be answered with the help of collected data. Only when the methods for extracting information from data are known does it make sense to ask which software and tools can be used for this purpose.
Juergen Bluhm, Baden-Wuertemberg Cooperative State University, juergen.bluhm@heilbronn.dhbw.de
Juergen is an academic researcher at DHBW Heilbronn and also a lecturer for various other academic institutions in Marketing, Market Research, Implicit Research, Business Statistics and Business Organisation and Project Planning. He also helps clients with eye-tracking research projects from designing the study, conducting the fieldwork and analysing and reporting the data in the end. Specialized in brand image research, with responsibilities for advertising tracking, brand tracking, advertising pre-testing, new-product research, financial research and advertising post-tests, he has more than 30 years of eye-tracking experience, including virtual shopping research (did develop a truly integrated virtual shopping with eye-tracking methodology) as well as biometric synchronisation of eye tracking with other biometric measurements like EEG, GSR, NIRS, etc., including integration in VR/AR.
Tentative Schedule:
Presentation of methodology (1 hour)
Hands-on Data Collection (3 hours)
Group work to analyse the data (3 hours)
Discussion of the results (1 hour)

This tutorial session will introduce participants to the intricacies of color perception and how it can be analyzed using eye-tracking. Even though eye-tracking systems do not directly measure color vision, color is highly relevant for bottom-up vision and a key determiner of saliency, which has a well-documented impact on eye movements. The multifaceted phenomenon of color perception, shaped by individual predispositions, cultural milieu, and contextual cues, thus finds a powerful non-invasive tool in eye-tracking. In this tutorial, we provide a comprehensive introduction to eye-tracking, its use and limitations for color perception analysis, catering to beginner and intermediate researchers seeking to enhance their understanding. A refresher into color spaces will be provided to envisage how colors are digitally handled; participants will learn about scalable eye-tracking data collection methods such as webcam eye tracking vs. lab-based setups including new extended reality headsets; revisit standard analysis methods gaze plots, heatmaps, and fixation metrics in relation to color perception; explore how computer vision and image processing techniques can enhance the visual experience and accessibility for individuals with color vision deficiency (CVD); and discover how artificial intelligence (AI) can extract color-related reports about the human visual system (HVS), providing deeper insights into color perception mechanisms.
Presenters:
Prof. Dr. Arzu Çöltekin, University of Applied Sciences & Arts Northwestern Switzerland, Switzerland, arzu.coltekin@fhnw.ch
Arzu Çöltekin leads the Institute of Interactive Technologies at the University of Applied Sciences and Arts Northwestern Switzerland. She works at the same institute as a Professor of Human-Computer Interaction, Visualization and Extended Reality. She is also a research affiliate on scientific data analysis and visualization topics at the Seamless Astronomy group in the Harvard-Smithsonian Center for Astrophysics of the Harvard University in Cambridge, USA. Arzu chairs the international Extended Reality and Visual Analytics working group within the ISPRS, co-chairs the Commission on Visual Analytics with the ICA, and serves as a council member of the International Society of Digital Earth. Arzu's interests ineye tracking include using it as an input modality in interactive systems (adaptive, foveated displays) and eye movement analysis for cognitive information processing. She has done various projects on color and cartography and will share some of her experience at the intersection of the two.
Dr Alessandro Bruno, International University of Language and Media, Italy, alessandro.bruno@iulm.it
Alessandro Bruno is a Senior Assistant Professor (Tenure Track) of Computer Science with the Faculty of Communication at IULM University (Milan, Italy). He leads the unit ‘data mining and text analytics’ in the Artificial Intelligence for Business and Society postgraduate programme. He is a computer engineer doctor, covering several postdoc positions in Italy (University of Palermo, INAF - Italian National Institute for Astrophysics) and the UK (Research Associate at NCCA - National Centre for Computer Animation and Bournemouth University’s Department of Computing) since 2012 through 2020. Alessandro was a research visitor at UCL (University College London) in 2019. In 2021, he was a Lecturer in Computing at Bournemouth University, where he served until May 2022. He was also a Co-Investigator of a European Research Project, namely, s4allcities. He now serves as Principal Investigator of an NGI-Search funded project named HeReFaNMi (Health Related Fake News Mitigation).
Tentative Schedule:
TBA