Use of the Rasch and Masters Partial Credit Model in the Assessment of Mathematical Competencies
Keywords:
Partial credit, Rasch, Mathematics, Competencies, Computerized assessment
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Abstract
Recently, the assessment of learning outcomes has improved substantially due to developments in cognitive science, in the use of digital technologies and in the application of new measurement theory, especially Item Response Theory (IRT). Usually, items in a selection exam are designed dichotomously (right or wrong) and are scored accordingly. However, items designed to be solved by constructed answers can be employed and graded with partial responses. The purpose of this study is to examine the operation of the partial credit model (PCM) developed by G. Masters, from the dichotomous Rasch Model, for this type of items (Wright & Masters, 1982). A computer administered constructed answer exam was employed: The Mathematics Basic Competencies Exam -EXCOBA Math. The exam was administered to 547 engineering students in a public Mexican university. Employing the Winsteps© 4.5.5 (Linacre, 2020), a comparison of the students’ scores is made with responses first graded dichotomously and then graded as partial credit. The results show that although scores are not equivalent, these maintain the same order indicated by a correlation near 1. It is concluded that the Rasch & Masters PCM operates efficiently and can be employed even on items with elements not necessarily are concatenated as a sequence of incremental steps, as well as, in areas other than academic performance.
Downloads
References
Andrich, D. (2016). Inference of independent dichotomous responses in the polytomous rasch model. Rasch Measurement Transactions, 30(1), 1566-1569.
Arteaga Martínez, B., Navarro Asencio, E., Fraile Rey, A. y Ramos Alonso, P. (2018). Adaptación de la prueba TIMSS para la evaluación de la competencia matemática en alumnos de magisterio. Bordón. Revista de Pedagogía, 70(3), 95-113.
Boone, W. J, Staver, J. R. y Yale, M. R. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4
Embretson, S. E. (2006). The continued search for nonarbitrary metrics in psychology. American Psychologist, 61(1), 50-55. https://doi.org/10.1037/0003-066X.61.1.50
Embretson, S. E. y Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum.
Freitas, S., Prieto, G., Simões, M. R. y Santana, I. (2015). Scaling cognitive domains of the montreal cognitive assessment: An analysis using the partial credit model. Archives of Clinical Neuropsychology, 30(5), 435-447. https://doi.org/10.1093/arclin/acv027
Henninger, M. (2021). A novel partial credit extension using varying thresholds to account for response tendencies. Journal of Educational Measurement, 58(1), 104-129. http://doi.org/10.1111/jedm.12268
Linacre, J. M. (1997). Kr-20/Cronbach alpha or rasch reliability: Which tells the "truth"? Rasch Measurement Transactions, 11(3), 580-581.
Linacre, J. M. (2020). Winsteps® (Versión 4.5.5). Winsteps.com.
Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149-174. https://doi.org/10.1007/BF02296272
Métrica Educativa, A. C. (2020). Excoba/Matemáticas/ES. https://metrica.edu.mx/examenes/examenes-diagnosticos/exumat/excoba-matematicas-es/
Nakano, T. y Primi, R. (2014). Rasch-master’s partial credit model in the assessment of children’s creativity in drawings. The Spanish Journal of Psychology, 17(35), 1-16. http://doi.org/10.1017/sjp.2014.36
Primi, R. (2004). Avanços na interpretação de escalas com a aplicação da teoria de resposta ao item. Avaliação Psicológica, 3(1), 53-58.
Rasch, G. (1960). Probabilistic models for some attainment and intelligence tests. Danmarks Pædagogiske Institut.
Van der Linden, W. J. y Hambleton, R. K. (1997). Handbook of modern item response theory. Springer. http://doi.org/10.1007/978-1-4757-2691-6
Wesolowski, B., Wind, S. y Engelhard, G. (2016). Examining rater precision in music performance assessment: An analysis of rating scale structure using the multifaceted rasch partial credit model. Music Perception: An Interdisciplinary Journal, 33(5), 662-678. https://doi.org/10.1525/mp.2016.33.5.662
Wright, B. D. y Masters, G. N. (1982). Rating scale analysis. MESA Press.
Wright, B. D. y Stone, M. H. (1979). Best test design. MESA Press.