Núm. 13 (2021): Iniciativas para la evaluación internacional de los resultados de aprendizaje en la Formación Profesional y la Educación Superior
Monográfico

PRUEBAS INTERNACIONALES DE DESEMPEÑO PARA LA EVALUACIÓN DEL PENSAMIENTO CRÍTICO: MARCO PARA LA TRADUCCIÓN Y ADAPTACIÓN

Natalia Ronderos
University of Zürich
Biografía
Richard J. Shavelson
Stanford University
Biografía
Doreen Holtsch
St.Gallen University of Teacher Education
Biografía
Olga Zlatkin-Troitschanskaia
Johannes Gutenberg University
Biografía
Guillermo Solano-Flores
Stanford University
Biografía
Publicado julio 16, 2021

Palabras clave:

Pensamiento crítico, Pruebas de desempeño, Traducción, Adaptación, Pruebas internacionales
Cómo citar
Ronderos, N., Shavelson, R. J., Holtsch, D., Zlatkin-Troitschanskaia, O. ., & Solano-Flores, G. (2021). PRUEBAS INTERNACIONALES DE DESEMPEÑO PARA LA EVALUACIÓN DEL PENSAMIENTO CRÍTICO: MARCO PARA LA TRADUCCIÓN Y ADAPTACIÓN. Journal of Supranational Policies of Education (JoSPoE), (13), 62–87. https://doi.org/10.15366/jospoe2021.13.003

Resumen

En todo el mundo, las instituciones de educación superior afirman que influyen en el aprendizaje de los estudiantes dentro y a través de distintas áreas académicas. El pensamiento crítico destaca entre las áreas de más interés (Braun et al., 2020). En este contexto, existe un interés creciente en pruebas de desempeño de pensamiento crítico que sean ecológicamente válidas y que se pueden utilizar internacionalmente (Zlatkin-Troitschanskaia et al., 2018). Varios estudios internacionales han tenido como objetivo medir y comparar el pensamiento crítico entre países, pero empleando preguntas de opción múltiple. Pocos estudios involucran la traducción y adaptación de pruebas de desempeño y los resultados que han producido apuntan a la necesidad de mejorar el proceso de adaptación (Braun et al., 2020). Basados en una revisión crítica de enfoques y marcos previos relacionados con la adaptación de pruebas, y con el fin de superar las dificultades para logra una mayor sensibilidad cultural, proponemos un nuevo marco conceptual para la adaptación de pruebas de desempeño de pensamiento crítico para estudios internacionales. El marco propuesto distingue dos etapas y tres diseños de adaptación. La primera etapa presenta un enfoque colaborativo para el diseño de las pruebas. La segunda ofrece tres opciones de diseño con diferentes grados de énfasis en aspectos lingüísticos y sensibilidad cultural. Aunque el artículo se enfoca a las pruebas de desempeño de pensamiento crítico para educación superior, se le puede aplicar en contextos preuniversitarios.

Descargas

Los datos de descargas todavía no están disponibles.

Citas

Addey, C., & Sellar, S. (2017) Why do countries participate in PISA? Understanding the role of international large-scale assessments in global education policy. In A. Verger, M. Novelli, H.K. Altinyelken (Eds.), Global Education Policy and International Development: New Agendas, Issues and Policies (pp. 97-117) London: Bloomsbury.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Basterra, M., Trumbull, E., & Solano-Flores, G. (2011). Cultural validity in assessment: Addressing linguistic and cultural diversity. NY: Routledge.

Bennet, R. (2020) Interpreting test-score comparisons. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Berman, A. I., Haertel, E. H., & Pellegrino, J. W. (2020) Introduction: Framing the issues. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Braun, H.I., Shavelson, R.J., Zlatkin-Troitschanskaia, O., & Borowiec, K. (2020) Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation. Frontiers in Education. 5:156. https://doi.org 10.3389/feduc.2020.00156

Brown, G. (2016). Handbook of Human and Social Conditions in Assessment. In G. Brown, & L. Harris (Eds.) Handbook of Human and Social Conditions in Assessment. New York: Routledge. https://doi.org/10.4324/9781315749136

Byrne, B.M., Shavelson, R.J., & Muthen, B.O. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456-466.

DePascale, C., & Gong, B. (2020) Comparability of individual students’ scores on the “same test. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Ercikan, K., & Solano-Flores, G. (2016). Section Discussion: Assessment and Sociocultural Context: A Bidirectional Relationship. (pp. 490–505) In G. Brown, & L. Harris (Eds.) Handbook of Human and Social Conditions in Assessment. New York: Routledge. https://doi.org/10.4324/9781315749136

Ercikan, K., & Por, H.H. (2020) Comparability in multilingual and multicultural assessment contexts. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Hambleton, R. K. (2002). Adapting achievement tests into multiple languages for international assessments. In A. C. Porter & A. Gamoran (Eds.), Methodological advances in cross-national surveys of educational achievement (pp. 58-79). Washington, DC: National Academy Press.

Hambleton, R. K. (2005) Issues, Designs, and Technical Guidelines for Adapting Tests Into Multiple Languages and Cultures. In R.K. Hambleton, P.F. Merenda, C.D. Spielberger (Eds.) Adapting Educational and Psychological Tests for Cross-Cultural Assessment. Psychology Press.

Hambleton, R. K., & Zenisky, A. L. (2010). Translating and adapting tests for cross-cultural assessments. Cross-Cultural Research Methods in Psychology In D. Matsumoto & F. Van de Vijver (Eds.), Cross-Cultural Research Methods in Psychology (pp. 46-70). Cambridge: Cambridge University Press. doi:10.1017/CBO9780511779381.004

Hyytinen, H., & Toom, A. (2019). Developing a performance assessment task in the Finnish higher education context: Conceptual and empirical insights. British Journal of Educational Psychology, 89(3), 551–563. https://doi.org/10.1111/bjep.12283

Holtsch, D., Rohr-Mentele, S., Wenger, E., Eberle, F., & Shavelson, R. J. (2016). Challenges of a cross-national computer-based test adaptation. Empirical research in vocational education and training, 8(18), 1–32. https://doi.org/10.1186/s40461-016-0043-y

International Test Comission. (2017). The ITC Guidelines for Translating and Adapting Tests (Second edition). https://doi.org/10.1111/j.1464-0597.1975.tb00322.x

Keng, L., & Marion, S. (2020) Comparability of aggregated group scores on the “same test”. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing critical thinking in higher education: current state and directions for next-generation assessments. ETS Res. Rep. Ser. 1, 1–23. https://doi.org/10.1002/ets2.12009

McClelland, D. C. (1973). Testing for competence rather than for “intelligence.” American Psychologist, 28(1), 1–14. https://doi.org/10.1037/h0034092

Mihailidis, P., & Thevenin, B. (2013). Media Literacy as a Core Competency for Engaged Citizenship in Participatory Democracy. American Behavioral Scientist, 57(11), 1611–1622. https://doi.org/10.1177/0002764213489015

Mislevy, R. J. (2018). Sociocognitive foundations of educational measurement. New York: Routledge. https://doi.org/10.4324/9781315871691

O’Leary, M., Reynolds, K., Guangming, L.; Ou, L.L., Belton, S., O’Reilly, N., & McKenna, J., (2020) Assessing Critical Thinking in Higher Education: Validity Evidence for the Use of the HEIghten™ Critical Thinking Test in Ireland. Journal of Higher Education Theory & Practice. Vol. 20 Issue 12, p115-130

Oser, F. K., & Biedermann, H. (2020). A three-level model for critical thinking: critical alertness, critical reflection, and critical analysis. In O. Zlatkin-Troitschanskaia (Eds.) Frontiers and Advances in Positive Learning in the Age of Information (PLATO). (pp. 89–106). Springer. https://doi.org/10.1007/978-3-030-26578-6_7

Perie, M. (2020) Comparability across different assessment systems. In A.I. Berman, E.H. Haertel, J.W. Pellegrino (Eds.), Comparability of Large-Scale Educational Assessments: Issues and Recommendations. National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Rogers, W. T., Gierl, M. J., Tardif, C., Lin, J., & Rinaldi, C. (2003). Differential Validity and Utility of Successive and Simultaneous Approaches to the Development of Equivalent Achievement Tests in French and English. Alberta Journal of Educational Research, 49(3), 290–304.

Rogers, W. T., Lin, J., & Rinaldi, C. (2011). Validity of the simultaneous approach to the development of equivalent achievement tests in English and French. Applied Measurement in Education, 24, 39-70.

Schendel, R., & Tolmie, A. (2017). Beyond translation: adapting a performance-task-based assessment of critical thinking ability for use in Rwanda. Assessment and Evaluation in Higher Education, 42(5), 673–689. https://doi.org/10.1080/02602938.2016.1177484.

Shavelson, R.J. (2010). Measuring college learning responsibly: Accountability in a new era. Stanford, CA: Stanford University Press.

Shavelson, R.J., Solano-Flores, G., & Kurpius, A. (2010) GS.4 Conceptual Framework. AHELO Module A: Adaptation and Translation of Performance Tasks. Council for Aid to Education.

Shavelson, R.J., Zlatkin-Troitschanskaia, O., Mariño, J. (2018). International Performance Assessment of Learning in Higher Education (iPAL): Research and Development. In: O. Zlatkin-Troitschanskaia, M. Toepper, H. Pant, C. Lautenbach, C. Kuhn (Eds.) Assessment of Learning Outcomes in Higher Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-74338-7_10

Shavelson, R.J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., Mariño, J. (2019): Assessment of University Students’ Critical Thinking: Next Generation Performance Assessment, International Journal of Testing 19:4, 337-362. https://doi.org/10.1080/15305058.2018.1543309

Siegel, H. (2010). On Thinking Skills. In C. Winch (Eds). Teaching thinking skills (2nd ed.). New York, NY : Continuum International Pub. Group.

Sierci, S., & O’Riordan, M. (2020) Comparability when assessing individuals with disabilities. In A.I. Berman, E.H. Haertel, J.W. Pellegrino (Eds.), Comparability of Large-Scale Educational Assessments: Issues and Recommendations. National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf

Solano-Flores, G. (2011) Assessing the Cultural Validity of Assessment Practices: An Introduction. In M.R. Basterra, E. Trumbull, G. Solano-Flores. Cultural validity in assessment: Addressing linguistic and cultural diversity. NY: Routledge. https://doi.org/10.4324/9780203850954

Solano-Flores, G. (2012). Smarter Balanced Assessment Consortium: Translation accommodations framework for testing English language learners in mathematics.

Solano-Flores, G. (2019a). Examining Cultural Responsiveness in Large-Scale Assessment: The Matrix of Evidence for Validity Argumentation. Frontiers in Education, 4(June), 1–9. https://doi.org/10.3389/feduc.2019.00043 Retrieved from https://www.frontiersin.org/articles/10.3389/feduc.2019.00043/full

Solano-Flores, G. (2019b). The participation of Latin American Countries in International Assessments: Assessment Capacity, Validity, and Fairness. In, L. E. Suter, E. Smith & B. D. Denman, B. D.T (Eds.), Sage Handbook on Comparative Studies in Education: Practices and Experiences in student schooling and learning (pp. 139-161). Thousand Oaks, CA: Sage.

Solano-Flores, G., Javanovic, J., Shavelson, R. J., & Bachman, M. (1999). On the development and evaluation of a shell for generating science performance assessments. International Journal of Science Education, 21(3), 293–315. https://doi.org/10.1080/095006999290714

Solano-Flores, G., Trumbull, E., & Nelson-Barber, S. (2002). Concurrent Development of Dual Language Assessments: An Alternative to Translating Tests for Linguistic Minorities. International Journal of Testing, 2(2), 107–129. https://doi.org/10.1207/s15327574ijt0202_2

Solano-Flores, G., Backhoff, E., & Contreras-Niño, L. Á. (2009). Theory of Test Translation Error. International Journal of Testing, 9(2), 78–91. https://doi.org/10.1080/15305050902880835

Solano-Flores, G., Chía, M., Shavelson, R.J., & Kurpius, A. (2010) GS.36. Translation Guide. AHELO Module A. Council for Aid to Education.

Solano-Flores, G., Shade, C., & Chrzanowski, A. (2014). Smarter Balanced Assessment Consortium: Item Accessibility and Language Variation Conceptual Framework. https://portal.smarterbalanced.org/library/en/item-accessibility-and-language-variation-conceptual-framework.pdf

Survey Research Center, Institute for Social Research, University of Michigan (2016). Guidelines for Best Practice in Cross-Cultural Surveys. http://ccsg.isr.umich.edu/images/PDFs/CCSG_Full_Guidelines_2016_Version.pdf

Stecher, B. M., Klein, S. P., Solano-Flores, G., McCaffrey, D., Robyn, A., Shavelson, R. J., & Haertel, E. (2000). The Effects of Content, Format, and Inquiry Level on Science Performance Assessment Scores. Applied Measurement in Education, 13(2), 139–160. https://doi.org/10.1207/S15324818AME1302_2

Tremblay, K., Lalancette, D., & Roseveare, D. (2012). Assessment of higher education learning outcomes (AHELO): Rationale, challenges and initial insights from the feasibility study. OECD (Vol. 1). https://doi.org/10.1007/978-94-6091-867-4 http://www.oecd.org/education/skills-beyond-school/AHELOFSReportVolume1.pdf

van de Vijver, F. J. R., & Poortinga, Y. H. (2004). Conceptual and methodological issues in adapting tests. Adapting Educational and Psychological Tests for Cross-Cultural Assessment, 39–63. https://doi.org/10.4324/9781410611758

van de Vijver, F. J. R., & Poortinga, Y. H. (2016). On item pools, swimming pools, birds with webbed feet, and the professionalization of multilingual assessment. In C. S. Wells & M. Faulkner-Bond (Eds.), Educational measurement: From foundations to future (pp. 273–290). The Guilford Press.

van de Vijver, F. J. R. (2016). Assessment in education in multicultural populations. In G. T. L. Brown and L. Harris (Eds) Handbook of Human and Social Conditions of Assessment. (pp. 436–453). New York, NY: Routledge

Zhao, X., & Solano-Flores, G. (2020). Testing across languages in international comparisons: cultural adaptation of consensus-based test translation review procedures, Journal of Multilingual and Multicultural Development. https://doi.org/10.1080/01434632.2020.1852242

Zahner, D., & Ciolfi, A. (2018). International Comparison of a Performance-Based Assessment in Higher Education. In: O. Zlatkin-Troitschanskaia, M. Toepper, H.A. Pant, C. Lautenbach and C. Kuhn (Eds.), Assessment of Learning Outcomes in Higher Education Cross-National Comparisons and Perspectives. Springer. https://doi.org/10.1007/978-3-319-74338-7_11.

Zlatkin-Troitschanskaia, O., Toepper, M., Molerov, D., Buske, R., Brückner, S., Pant, H., Hofmann, S., & Hansen-Schirra, S. (2018). Adapting and Validating the Collegiate Learning Assessment to Measure Generic Academic Skills of Students in Germany: Implications for International Assessment Studies in Higher Education. In: O. Zlatkin-Troitschanskaia, M. Toepper, H.A. Pant, C. Lautenbach and C. Kuhn (Eds.), Assessment of Learning Outcomes in Higher Education Cross-National Comparisons and Perspectives. Springer. https://doi.org/10.1007/978-3-319-74338-7_12.