Pendidikan editorial
Open AccessOA2023
Comparative Judgement in Education Research
This digest summarizes Jones and Davies' (2023) article on comparative judgement (CJ) as an alternative to traditional methods for constructing measurement scales in education research. It outlines the contexts where CJ is appropriate, the process of applying CJ-based methods, and how to evaluate the reliability and validity of resulting scales.
Ian Jones; Ben Daviesยท International Journal of Research & Method in Educationยท 2023ยท DOI 10.1080/1743727X.2023.2242273
The core problem
Educational researchers frequently need to measure complex and varied representations, such as participants' written work, videoed lesson segments, and policy documents. Traditional approaches to developing precise and reliable measurement scales for such representations can be resource-intensive, time-consuming, and may not always yield reliable outcomes. Jones and Davies (2023) address this challenge by presenting comparative judgement (CJ) as an alternative methodology that has gained popularity in recent years. The article aims to consider the contexts in which CJ-based methods are appropriate, describe in detail what they are and how they can be applied to construct measurement scales across a range of educational research contexts, and provide an overview for evaluating the reliability and validity of the resultant scales.
Innovation
The article is a methodological overview rather than an empirical study. It describes the principles and procedures of comparative judgement (CJ) as a technique for scaling complex representations. In CJ, judges are presented with pairs of representations (e.g., two pieces of written work) and asked to decide which one demonstrates more of a given attribute (e.g., quality). Repeated pairwise comparisons across many judges and representations are used to estimate a scale of relative quality. The authors detail how CJ can be applied in educational research contexts, including the design of comparisons, selection of judges, and the statistical models used to derive scale values. They also discuss considerations for implementing CJ studies, such as the number of comparisons needed and the handling of judge variability.
Introduction
Educational researchers frequently need to measure complex and varied representations, such as participants' written work, videoed lesson segments, and policy documents. Traditional approaches to developing precise and reliable measurement scales for such representations can be resource-intensive, time-consuming, and may not always yield reliable outcomes. Jones and Davies (2023) address this challenge by presenting comparative judgement (CJ) as an alternative methodology that has gained popularity in recent years. The article aims to consider the contexts in which CJ-based methods are appropriate, describe in detail what they are and how they can be applied to construct measurement scales across a range of educational research contexts, and provide an overview for evaluating the reliability and validity of the resultant scales.
Why it matters
The authors discuss the contexts in which CJ is appropriate, such as when researchers need to scale complex and varied representations that are difficult to measure with traditional methods. They compare CJ to traditional scale development approaches, noting that CJ can be more efficient and may yield reliable outcomes. The article also addresses potential limitations and considerations for using CJ, such as the cognitive demands on judges and the need for careful design. Overall, the discussion positions CJ as a valuable tool for educational researchers seeking to construct precise and reliable measurement scales, while cautioning that proper implementation and evaluation of reliability and validity are essential.
Who should read this
Pembaca Research Digest dan praktisi rumpun ini.
Opening member contentโฆ