Navigating uncertain terrains: My understanding of using eye-tracking in assessment research

Written by Sha Liu (PhD student member)

“Do you think that we can use eye-tracking to find out how students respond to automated feedback generated by computer software?”

“I don’t think so…”

This conversation occurred on 8 March 2017 when I attended a research seminar at University of Bedfordshire. At the one-day seminar, all CRELLA researchers reported and reflected on their use of eye-tracking in readi
ng research. Being deeply inspired, I asked this question and was expecting others to applaud my idea of using eye-tracking in research on automated writing evaluation (AWE) systems. Surprisingly, I received the negative response which triggered a heated discussion regarding the potential use of eye-tracking in various areas of assessment research. The take-home m
essage is that researchers need to be clear about their actual goal and motive of using eye-tracking and what they can get from using such technology.

After attsha.jpgending the seminar, I have been questioning myself whether my proposed use of eye-tracking method is necessary. I don’t use this method because of its increasing popularity in my field of research but instead, I plan to use this scientific method because it can tell me about people’s thinking processes which may not be effectively depicted by other data elicitation methods. By tracking the participating
students’ eye-movement, I will get detailed evidence of their physical behaviour and be able to infer the cognitive writing processes underlying such behaviour. This is probably why eye-tracking has been claimed to be by far one of the best ways to understand a participant’s cognitive processes in assessment research.

However, another important message I learned from the seminar is that the use of eye-tracking is not without challenges. First, data generated by eye-tracking method is overwhelmingly large and complex. For example, a change of single word in the revision process can be tracked, which may lead to the overload of data. It may turn out to be highly demanding for me to deal with the very richness of the data generated. Second, eye-tracking evidence appear in visual form and it is challenging for me to map the visual evidence onto my participants’ cognitive processes. One possible solution, as demonstrated in previous studies, is to combine eye-tracking with stimulated recalls and interviews that target certain aspects of visual evidence. For example, what does it indicate when a student fixates more on the feedback on his use of synonyms that on other feedback? To gain insights into the cognitive processes, I can show the eye-tracking data to the participant and ask for the explanation of his physical behaviour.

eye-movement

Before embarking on my doctoral study, I attended the 3rd annual conference of Asian Association of Language Assessment (AALA) at Bali, Indonesia. A very-well established professor I met there suggested that I become a lumberjack who uses both axes and saws when I asked for his advice regarding my professional development. At that time, my take of the suggestion is that I should learn as many research methods (both quantitative and qualitative) as possible. After my several months of being a social science inquirer, I would add another important point into my original interpretation: one important aspect of doctoral study is about learning to use proper methods in an appropriate manner.

barn-images-12223

 

Advertisements

5 thoughts on “Navigating uncertain terrains: My understanding of using eye-tracking in assessment research

  1. Sha,I wish you to have the best luck on the journey of your doctoral research. By reading your research, I had an enjoyable time tracking the way of your logic as if assembling pieces of puzzles.

    How to justify the eye movement in writing assessment sound interesting. This also gives me an inspiring idea: Not just cognitively, adapting eye-movement affectively is likely to function as the indication the level of anxiety so that supportive feedbacks, or praticse could be provided by the teacher to enhance the competence of writing.

    Overall, I enjoyed reading this experince and appreciate Sha for sharing her valuable experience.

    Like

  2. Fantastic! Eye-tracking is new but also fascinates me,hopefully you can discuss more specific and deeply in future.

    Like

  3. Great insight into your reserach journey so far Sha!
    Its great to see you reflecting on what has been learnt and what needs to be learnt. My experiencr so far resonates well with yours 😊
    Good luck with your reserach !

    Like

    1. Hi Shimmi, the feeling of uncertainty may become overwhelming specially when we are now working on our own projects. But I do think that we both will benefit from our constant reflection on what has been learnt and what needs to be learnt. =)

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s