|
|
|
 |
Search published articles |
 |
|
Showing 2 results for Consistency
Alireza Ahmadi,
Volume 12, Issue 1 (3-2009)
Abstract
This article investigated and compared the consistency of self and peer assessments as alternatives for teacher assessment. Thirty sophomores majoring in TEFL were asked to assess their classmates’ as well as their own speaking ability in a conversation class. They were taught how to do this using a rating scale of speaking. They did the rating twice during the term the first rating was carried out during the 8th and 9th weeks and the second rating at the end of the term (weeks 15 and 16). The results of the study indicated that self and peer assessments were not significantly related at the end of the term and only loosely, though significantly, related in the middle of the term. Both self and peer assessments indicated consistency over time, however peer assessment enjoyed a higher consistency.
Wander Lowie, Houman Bijani, Mohammad Reza Oroji, Zeinab Khalafi, Pouya Abbasi,
Volume 26, Issue 2 (9-2023)
Abstract
Performance testing including the use of rating scales has become highly widespread in the evaluation of second/foreign oral assessment. However, few studies have used a pre-, post-training design investigating the impact of a training program on the reduction of raters’ biases to the rating scale categories resulting in increase in their consistency measures. Besides, no study has used MFRM including the facets of test takers’ ability, raters’ severity, task difficulty, group expertise, scale category, and test version all in a single study. 20 EFL teachers rated the oral performances produced by 200 test takers before and after a training program using an analytic rating scale including fluency, grammar, vocabulary, intelligibility, cohesion and comprehension categories. The outcome of the study indicated that MFRM can be used to investigate raters’ scoring behavior and can result in enhancement in rater training and validating the functionality of the rating scale descriptors. Training can also result in higher levels of interrater consistency and reduced levels of severity/leniency; however, it cannot turn raters into duplicates of one another, but can make them more self-consistent. Training helped raters use the descriptors of the rating scale more efficiently of its various band descriptors resulting in reduced halo effect. Finally, the raters improved consistency and reduced rater-scale category biases after the training program. The remaining differences regarding bias measures could probably be attributed to the result of different ways of interpreting the scoring rubrics which is due to raters’ confusion in the accurate application of the scale.
|
|
