Identifying the response process validity of clinical vignette-type multiple choice questions: an eye-tracking study

Francisco Specian*, Thiago Santos, JOHN SANDARS, Eliana Amaral, Dario Cecilio-Fernandes

*Corresponding author for this work

Research output: Contribution to journalArticle (journal)peer-review

1 Citation (Scopus)
5 Downloads (Pure)

Abstract

Introduction: Clinical vignette- type multiple choice questions (CV-MCQs) are widely used in assessment and identifying the response process validity (RPV) of questions with low and high integration of knowledge is essential. Answering CV-MCQs of different levels of knowledge application and integration can be understood from a cognitive workload perspective and this can be identified by using eye-tracking. The aim of the pilot study was to identify the cognitive workload and RPV of CV-MCQs of different levels of knowledge application and integration by the use eye-tracking. Methods: Fourteen fourth-year medical students answered a test with 40 CV-MCQs, which were equally divided into low-level and high-level complexity (knowledge application and integration). Cognitive workload was measured using screen-based eye tracking, with the number of fixations and revisitations for each area of interest. Results: We found a higher cognitive workload for high-level complexity (M = 121.74) compared with lower-level complexity questions (M = 51.94) and also for participants who answered questions incorrectly (M = 94.31) compared with correctly (M = 79.36). Conclusion: Eye-tracking has the potential to become a useful and practical approach for helping to identify the RPV of CV-MCQs. This approach can be used for improving the design and development of CV-MCQs, and to provide feedback to inform teaching and learning.
Original languageEnglish
JournalMedical Teacher
Early online date25 Feb 2023
DOIs
Publication statusE-pub ahead of print - 25 Feb 2023

Keywords

  • eye tracking
  • cognitive workload
  • medical education
  • assessment
  • multiple choice questions

Fingerprint

Dive into the research topics of 'Identifying the response process validity of clinical vignette-type multiple choice questions: an eye-tracking study'. Together they form a unique fingerprint.

Cite this