Do different measures of assessment performance bias or exaggerate relationships?

Dave Putwain

Research output: Contribution to conferencePaper

Abstract

KS4 assessment performance data is used in at least three ways in educational research. First, an indicator of overall school performance to evaluate policy initiatives/ school practices designed to raise achievement. Second, to establish differential student achievement or mobility across socio-demographic groups (e.g. gender, ethnic background, etc.). Third, to establish how psycho-educational constructs such as achievement motivation, academic self-concept and assessment anxiety may influence students’ attainment. A standard convention for educational research in the UK is convert GCSE grades into an eight-point scale (e.g. Schagen & Schagen, 2005). There is less consistency over the calculation of performance measures and six different conventions are in current usage; total sum of points, total sum of points in the eight highest scoring subjects, grade point average (GPA), the mode of highest grades achieved, the number of five A*-C grades achieved and a binomial categorisation of whether a student achieved 5 A*-C grades or not. There is a possibility that particular measures may bias or exaggerate certain relationships. In order to investigate this possibility, an existing dataset regarding the relationship between GCSE performance and self-reported assessment anxiety was reanalysed using these six measures in a series of simple regression models. This reanalysis suggested that two measures, GPA and whether a student has achieved five A*-C grades or not, produce a stronger effect than the other measures. For example, GPA indicates a highly anxious student would achieve one grade lower across all subjects, whereas the total sum of points suggests a highly anxious student would drop three grades overall. These findings indicate that individual performance measures should be treated with some degree of scepticism and that in order to establish an educationally meaningful effect, multiple measures of performance should be triangulated
Original languageEnglish
Publication statusPublished - 2007
EventBritish Educational Research Association (BERA) Conference - Institute of Education, London, United Kingdom
Duration: 5 Sep 20078 Sep 2007

Conference

ConferenceBritish Educational Research Association (BERA) Conference
CountryUnited Kingdom
CityLondon
Period5/09/078/09/07

Fingerprint

performance assessment
trend
student
performance
self-assessment
educational research
school initiative
anxiety
achievement motivation
self-concept
regression
gender
school
Group

Cite this

Putwain, D. (2007). Do different measures of assessment performance bias or exaggerate relationships?. Paper presented at British Educational Research Association (BERA) Conference, London, United Kingdom.
Putwain, Dave. / Do different measures of assessment performance bias or exaggerate relationships?. Paper presented at British Educational Research Association (BERA) Conference, London, United Kingdom.
@conference{735435b46b2448e985bcdc6d4b0c3182,
title = "Do different measures of assessment performance bias or exaggerate relationships?",
abstract = "KS4 assessment performance data is used in at least three ways in educational research. First, an indicator of overall school performance to evaluate policy initiatives/ school practices designed to raise achievement. Second, to establish differential student achievement or mobility across socio-demographic groups (e.g. gender, ethnic background, etc.). Third, to establish how psycho-educational constructs such as achievement motivation, academic self-concept and assessment anxiety may influence students’ attainment. A standard convention for educational research in the UK is convert GCSE grades into an eight-point scale (e.g. Schagen & Schagen, 2005). There is less consistency over the calculation of performance measures and six different conventions are in current usage; total sum of points, total sum of points in the eight highest scoring subjects, grade point average (GPA), the mode of highest grades achieved, the number of five A*-C grades achieved and a binomial categorisation of whether a student achieved 5 A*-C grades or not. There is a possibility that particular measures may bias or exaggerate certain relationships. In order to investigate this possibility, an existing dataset regarding the relationship between GCSE performance and self-reported assessment anxiety was reanalysed using these six measures in a series of simple regression models. This reanalysis suggested that two measures, GPA and whether a student has achieved five A*-C grades or not, produce a stronger effect than the other measures. For example, GPA indicates a highly anxious student would achieve one grade lower across all subjects, whereas the total sum of points suggests a highly anxious student would drop three grades overall. These findings indicate that individual performance measures should be treated with some degree of scepticism and that in order to establish an educationally meaningful effect, multiple measures of performance should be triangulated",
author = "Dave Putwain",
year = "2007",
language = "English",
note = "British Educational Research Association (BERA) Conference ; Conference date: 05-09-2007 Through 08-09-2007",

}

Putwain, D 2007, 'Do different measures of assessment performance bias or exaggerate relationships?' Paper presented at British Educational Research Association (BERA) Conference, London, United Kingdom, 5/09/07 - 8/09/07, .

Do different measures of assessment performance bias or exaggerate relationships? / Putwain, Dave.

2007. Paper presented at British Educational Research Association (BERA) Conference, London, United Kingdom.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Do different measures of assessment performance bias or exaggerate relationships?

AU - Putwain, Dave

PY - 2007

Y1 - 2007

N2 - KS4 assessment performance data is used in at least three ways in educational research. First, an indicator of overall school performance to evaluate policy initiatives/ school practices designed to raise achievement. Second, to establish differential student achievement or mobility across socio-demographic groups (e.g. gender, ethnic background, etc.). Third, to establish how psycho-educational constructs such as achievement motivation, academic self-concept and assessment anxiety may influence students’ attainment. A standard convention for educational research in the UK is convert GCSE grades into an eight-point scale (e.g. Schagen & Schagen, 2005). There is less consistency over the calculation of performance measures and six different conventions are in current usage; total sum of points, total sum of points in the eight highest scoring subjects, grade point average (GPA), the mode of highest grades achieved, the number of five A*-C grades achieved and a binomial categorisation of whether a student achieved 5 A*-C grades or not. There is a possibility that particular measures may bias or exaggerate certain relationships. In order to investigate this possibility, an existing dataset regarding the relationship between GCSE performance and self-reported assessment anxiety was reanalysed using these six measures in a series of simple regression models. This reanalysis suggested that two measures, GPA and whether a student has achieved five A*-C grades or not, produce a stronger effect than the other measures. For example, GPA indicates a highly anxious student would achieve one grade lower across all subjects, whereas the total sum of points suggests a highly anxious student would drop three grades overall. These findings indicate that individual performance measures should be treated with some degree of scepticism and that in order to establish an educationally meaningful effect, multiple measures of performance should be triangulated

AB - KS4 assessment performance data is used in at least three ways in educational research. First, an indicator of overall school performance to evaluate policy initiatives/ school practices designed to raise achievement. Second, to establish differential student achievement or mobility across socio-demographic groups (e.g. gender, ethnic background, etc.). Third, to establish how psycho-educational constructs such as achievement motivation, academic self-concept and assessment anxiety may influence students’ attainment. A standard convention for educational research in the UK is convert GCSE grades into an eight-point scale (e.g. Schagen & Schagen, 2005). There is less consistency over the calculation of performance measures and six different conventions are in current usage; total sum of points, total sum of points in the eight highest scoring subjects, grade point average (GPA), the mode of highest grades achieved, the number of five A*-C grades achieved and a binomial categorisation of whether a student achieved 5 A*-C grades or not. There is a possibility that particular measures may bias or exaggerate certain relationships. In order to investigate this possibility, an existing dataset regarding the relationship between GCSE performance and self-reported assessment anxiety was reanalysed using these six measures in a series of simple regression models. This reanalysis suggested that two measures, GPA and whether a student has achieved five A*-C grades or not, produce a stronger effect than the other measures. For example, GPA indicates a highly anxious student would achieve one grade lower across all subjects, whereas the total sum of points suggests a highly anxious student would drop three grades overall. These findings indicate that individual performance measures should be treated with some degree of scepticism and that in order to establish an educationally meaningful effect, multiple measures of performance should be triangulated

M3 - Paper

ER -

Putwain D. Do different measures of assessment performance bias or exaggerate relationships?. 2007. Paper presented at British Educational Research Association (BERA) Conference, London, United Kingdom.