You are here

Home » Groups » Eye Data Quality


The current work of the Standardisation Committee is organised into the following workpackages:
Standardisation of eye data terminology
The terms used to describe data quality are often inadequately defined, or used inconsistently. Agreeing definitions and standardising terminology, both conceptually and mathematically, will allow for consensus on the definition of measurement techniques. This work will provide precise definitions of data quality terminology based on consensus of a large group of experts in the field, so we can refer precisely to what we measure and how. This work is at an advanced stage within the committee and will soon be ready for general review. If you have expertise in eye tracking and eye data quality and would like to take part in reviewing draft versions, please contact us. The committee member leading this work is Kara Latorella, with all members contributing.
Designing a standardised set of artifical eyes
Artificial eyes are important tools for measuring system error or precision. Since real eyes vary according to colour, shape, and pupil size, artificial eyes should represent a range of eye types and work equally well on diverse systems. An important aspect of the design of artificial eyes is that they must work for both bright and dark pupil methods, we need to measure system temporal performance by simulating a change in gaze position, and robotic eyes are considered to measure the quality of the data with regard to eye dynamics. We are currently in negotiation with a manufacturer of artificial eyes for training ophthalmologists which fits the criteria. We aim to modify these eyes to produce a standard set of artificial eyes. Most manufacturers have asked to purchase the same set we will use for data quality measures. The committee member leading this work is Dong Wang, with Fiona Mulvey, Jeff Pelz, Dixon Cleveland and external collaborators.
Designing a standard experimental protocol for the collection of data for data quality measurement
This workpackage defines the actual experimental procedures to investigate data quality, both in our own large scale study and as part of future experiments using provided methods as a control measure in a research project, for example. We defined specifications that the experiment and software should be capable of running on all systems and take account of varying intended usage. There is a high degree of collaboration with manufacturers in this work. The project has produced open source tools for the calculation of data quality using a simple experimental routine and investigate critical issues for reported data quality such as sample selection, variances in calculation results with sample rate and algorithms. This work is at an advanced stage in terms of results of testing human participants, and ongoing in terms of artificial eyes. We are collaborating with vibration and robotics expertise for the design of robustness measures and a means of testing data quality in the eye-in-motion, such as the validity of saccadic measures used in clinical research. We welcome collaboration from students, researchers and manufacturers interested in this work. So far, we have recorded 192 people on 17 eyetrackers and are preparing the first large scale system comparison for publication. ECEM 2015 included a symposium on the EDQ project where 5 of our upcoming papers were presented. This work is led by Fiona Mulvey, with input from all committee members.
Physiological measures of eye data quality
This workpackage focuses on the eye in motion, and uses knowledge from eye physiology to assess the quality of eye data. Methods considered include dual recording of eye movements with coil based systems and DPI systems, reproducing `standard' and realistic eye movements via replay of human movements in robitic artificial eyes, and applying calculus to the detection of error. Other methods including identifying thresholds for maximum velocity, comparing main sequence, peak velocity and skew of saccades. The work is ongoing, we welcome collaboration. This work is led by Fiona Mulvey with input from Mary Hayhoe, Jeff Pelz, Sol Simpson and others.
Implementing commercial eye trackers to the open source experimental software
This work involves manufacturers working with the committee and subcommittee to get as many as possible available eyetrackers integrated with the experimental software. The following systems have been implemented and are part of the large data set recorded in the Humanities Lab, Lund University, Sweden: SMI remote, tower and glasses, SR Research EyeLink systems, Dual Purkinje Imaging from Fourward Technologies, LC Technologies EyeFollower and Eye Gaze systems, EyeTribe eyetracker, tobii eyetrackers, and ASL glasses (some integration work remains for headmounted systems). Several other manufacturers have provided systems or committed time to the work ,and more eyetrackers are being implemented into the experimental software all the time. Manufacturers willing to lend a system and some development time to the project can have their system integrated into the ioHub collaboratively, using the ioHub software developed for the project by Sol Simpson (see 'Tools' menu left). ioHub is not a complete experiment design and runtime API. It's main focus is on device event monitoring, real-time reporting, and persistant storage of input device events on a system wide basis. It is designed to work with psychopy and is now part of the general psychopy release. The ioHub allows us to run eye data quality experiments on any integrated eyetracker, or other integrated sensors or input devices, with extremely precise device event management. This work is led by Sol Simpson, with input from several other committee members and manufacturers/commercial engineers.
Investigating the effects of data quality on eye movement measures
This workpackage uses the data collected for comparison of systems in order to test event detection algorithms and their settings under variant data quality. It is secondary output from the main work - but of critical importance to researchers. We are now preparing publications that investigate the effects of data quality on fixations and saccades in all systems, and on microsaccades in high end VOGs and a dual-purkinje imaging tracker. This work is led by Fiona Mulvey, with input from Jeff Pelz, Dixon Cleveland, and other committee members and subcommittee members. First results were presented at VSS in 2014 and at ECEM 2015.
We are continually looking for funding for this research. Funding is donated either via EMRA (for all aspects of the standardisation work), or COGAIN (for EDQ standardisation work related to gaze interaction) which are both not-for-profit organisations. We have received small donations from SR Research and SMI for committee travel, and several system loans from ASL, tobii, SMI, EyeTribe, and other manufacturers and software providers. Funding covers travel of committee members for meetings and workshops, small equipment and publication costs. No members of the committee are paid for their work.