PRUEBAS INTERNACIONALES DE DESEMPEÑO PARA LA EVALUACIÓN DEL PENSAMIENTO CRÍTICO: MARCO PARA LA TRADUCCIÓN Y ADAPTACIÓN
Palabras clave:
Pensamiento crítico, Pruebas de desempeño, Traducción, Adaptación, Pruebas internacionalesResumen
En todo el mundo, las instituciones de educación superior afirman que influyen en el aprendizaje de los estudiantes dentro y a través de distintas áreas académicas. El pensamiento crítico destaca entre las áreas de más interés (Braun et al., 2020). En este contexto, existe un interés creciente en pruebas de desempeño de pensamiento crítico que sean ecológicamente válidas y que se pueden utilizar internacionalmente (Zlatkin-Troitschanskaia et al., 2018). Varios estudios internacionales han tenido como objetivo medir y comparar el pensamiento crítico entre países, pero empleando preguntas de opción múltiple. Pocos estudios involucran la traducción y adaptación de pruebas de desempeño y los resultados que han producido apuntan a la necesidad de mejorar el proceso de adaptación (Braun et al., 2020). Basados en una revisión crítica de enfoques y marcos previos relacionados con la adaptación de pruebas, y con el fin de superar las dificultades para logra una mayor sensibilidad cultural, proponemos un nuevo marco conceptual para la adaptación de pruebas de desempeño de pensamiento crítico para estudios internacionales. El marco propuesto distingue dos etapas y tres diseños de adaptación. La primera etapa presenta un enfoque colaborativo para el diseño de las pruebas. La segunda ofrece tres opciones de diseño con diferentes grados de énfasis en aspectos lingüísticos y sensibilidad cultural. Aunque el artículo se enfoca a las pruebas de desempeño de pensamiento crítico para educación superior, se le puede aplicar en contextos preuniversitarios.
Descargas
Citas
Addey, C., & Sellar, S. (2017) Why do countries participate in PISA? Understanding the role of international large-scale assessments in global education policy. In A. Verger, M. Novelli, H.K. Altinyelken (Eds.), Global Education Policy and International Development: New Agendas, Issues and Policies (pp. 97-117) London: Bloomsbury.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Basterra, M., Trumbull, E., & Solano-Flores, G. (2011). Cultural validity in assessment: Addressing linguistic and cultural diversity. NY: Routledge.
Bennet, R. (2020) Interpreting test-score comparisons. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Berman, A. I., Haertel, E. H., & Pellegrino, J. W. (2020) Introduction: Framing the issues. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Braun, H.I., Shavelson, R.J., Zlatkin-Troitschanskaia, O., & Borowiec, K. (2020) Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation. Frontiers in Education. 5:156. https://doi.org 10.3389/feduc.2020.00156
Brown, G. (2016). Handbook of Human and Social Conditions in Assessment. In G. Brown, & L. Harris (Eds.) Handbook of Human and Social Conditions in Assessment. New York: Routledge. https://doi.org/10.4324/9781315749136
Byrne, B.M., Shavelson, R.J., & Muthen, B.O. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456-466.
DePascale, C., & Gong, B. (2020) Comparability of individual students’ scores on the “same test. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Ercikan, K., & Solano-Flores, G. (2016). Section Discussion: Assessment and Sociocultural Context: A Bidirectional Relationship. (pp. 490–505) In G. Brown, & L. Harris (Eds.) Handbook of Human and Social Conditions in Assessment. New York: Routledge. https://doi.org/10.4324/9781315749136
Ercikan, K., & Por, H.H. (2020) Comparability in multilingual and multicultural assessment contexts. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Hambleton, R. K. (2002). Adapting achievement tests into multiple languages for international assessments. In A. C. Porter & A. Gamoran (Eds.), Methodological advances in cross-national surveys of educational achievement (pp. 58-79). Washington, DC: National Academy Press.
Hambleton, R. K. (2005) Issues, Designs, and Technical Guidelines for Adapting Tests Into Multiple Languages and Cultures. In R.K. Hambleton, P.F. Merenda, C.D. Spielberger (Eds.) Adapting Educational and Psychological Tests for Cross-Cultural Assessment. Psychology Press.
Hambleton, R. K., & Zenisky, A. L. (2010). Translating and adapting tests for cross-cultural assessments. Cross-Cultural Research Methods in Psychology In D. Matsumoto & F. Van de Vijver (Eds.), Cross-Cultural Research Methods in Psychology (pp. 46-70). Cambridge: Cambridge University Press. doi:10.1017/CBO9780511779381.004
Hyytinen, H., & Toom, A. (2019). Developing a performance assessment task in the Finnish higher education context: Conceptual and empirical insights. British Journal of Educational Psychology, 89(3), 551–563. https://doi.org/10.1111/bjep.12283
Holtsch, D., Rohr-Mentele, S., Wenger, E., Eberle, F., & Shavelson, R. J. (2016). Challenges of a cross-national computer-based test adaptation. Empirical research in vocational education and training, 8(18), 1–32. https://doi.org/10.1186/s40461-016-0043-y
International Test Comission. (2017). The ITC Guidelines for Translating and Adapting Tests (Second edition). https://doi.org/10.1111/j.1464-0597.1975.tb00322.x
Keng, L., & Marion, S. (2020) Comparability of aggregated group scores on the “same test”. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205–225). Washington, DC: National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing critical thinking in higher education: current state and directions for next-generation assessments. ETS Res. Rep. Ser. 1, 1–23. https://doi.org/10.1002/ets2.12009
McClelland, D. C. (1973). Testing for competence rather than for “intelligence.” American Psychologist, 28(1), 1–14. https://doi.org/10.1037/h0034092
Mihailidis, P., & Thevenin, B. (2013). Media Literacy as a Core Competency for Engaged Citizenship in Participatory Democracy. American Behavioral Scientist, 57(11), 1611–1622. https://doi.org/10.1177/0002764213489015
Mislevy, R. J. (2018). Sociocognitive foundations of educational measurement. New York: Routledge. https://doi.org/10.4324/9781315871691
O’Leary, M., Reynolds, K., Guangming, L.; Ou, L.L., Belton, S., O’Reilly, N., & McKenna, J., (2020) Assessing Critical Thinking in Higher Education: Validity Evidence for the Use of the HEIghten™ Critical Thinking Test in Ireland. Journal of Higher Education Theory & Practice. Vol. 20 Issue 12, p115-130
Oser, F. K., & Biedermann, H. (2020). A three-level model for critical thinking: critical alertness, critical reflection, and critical analysis. In O. Zlatkin-Troitschanskaia (Eds.) Frontiers and Advances in Positive Learning in the Age of Information (PLATO). (pp. 89–106). Springer. https://doi.org/10.1007/978-3-030-26578-6_7
Perie, M. (2020) Comparability across different assessment systems. In A.I. Berman, E.H. Haertel, J.W. Pellegrino (Eds.), Comparability of Large-Scale Educational Assessments: Issues and Recommendations. National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Rogers, W. T., Gierl, M. J., Tardif, C., Lin, J., & Rinaldi, C. (2003). Differential Validity and Utility of Successive and Simultaneous Approaches to the Development of Equivalent Achievement Tests in French and English. Alberta Journal of Educational Research, 49(3), 290–304.
Rogers, W. T., Lin, J., & Rinaldi, C. (2011). Validity of the simultaneous approach to the development of equivalent achievement tests in English and French. Applied Measurement in Education, 24, 39-70.
Schendel, R., & Tolmie, A. (2017). Beyond translation: adapting a performance-task-based assessment of critical thinking ability for use in Rwanda. Assessment and Evaluation in Higher Education, 42(5), 673–689. https://doi.org/10.1080/02602938.2016.1177484.
Shavelson, R.J. (2010). Measuring college learning responsibly: Accountability in a new era. Stanford, CA: Stanford University Press.
Shavelson, R.J., Solano-Flores, G., & Kurpius, A. (2010) GS.4 Conceptual Framework. AHELO Module A: Adaptation and Translation of Performance Tasks. Council for Aid to Education.
Shavelson, R.J., Zlatkin-Troitschanskaia, O., Mariño, J. (2018). International Performance Assessment of Learning in Higher Education (iPAL): Research and Development. In: O. Zlatkin-Troitschanskaia, M. Toepper, H. Pant, C. Lautenbach, C. Kuhn (Eds.) Assessment of Learning Outcomes in Higher Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-74338-7_10
Shavelson, R.J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., Mariño, J. (2019): Assessment of University Students’ Critical Thinking: Next Generation Performance Assessment, International Journal of Testing 19:4, 337-362. https://doi.org/10.1080/15305058.2018.1543309
Siegel, H. (2010). On Thinking Skills. In C. Winch (Eds). Teaching thinking skills (2nd ed.). New York, NY : Continuum International Pub. Group.
Sierci, S., & O’Riordan, M. (2020) Comparability when assessing individuals with disabilities. In A.I. Berman, E.H. Haertel, J.W. Pellegrino (Eds.), Comparability of Large-Scale Educational Assessments: Issues and Recommendations. National Academy of Education. Retrieved from https://naeducation.org/wp-content/uploads/2020/05/Comparability-of-Large-Scale-Educational-Assessments_final.pdf
Solano-Flores, G. (2011) Assessing the Cultural Validity of Assessment Practices: An Introduction. In M.R. Basterra, E. Trumbull, G. Solano-Flores. Cultural validity in assessment: Addressing linguistic and cultural diversity. NY: Routledge. https://doi.org/10.4324/9780203850954
Solano-Flores, G. (2012). Smarter Balanced Assessment Consortium: Translation accommodations framework for testing English language learners in mathematics.
Solano-Flores, G. (2019a). Examining Cultural Responsiveness in Large-Scale Assessment: The Matrix of Evidence for Validity Argumentation. Frontiers in Education, 4(June), 1–9. https://doi.org/10.3389/feduc.2019.00043 Retrieved from https://www.frontiersin.org/articles/10.3389/feduc.2019.00043/full
Solano-Flores, G. (2019b). The participation of Latin American Countries in International Assessments: Assessment Capacity, Validity, and Fairness. In, L. E. Suter, E. Smith & B. D. Denman, B. D.T (Eds.), Sage Handbook on Comparative Studies in Education: Practices and Experiences in student schooling and learning (pp. 139-161). Thousand Oaks, CA: Sage.
Solano-Flores, G., Javanovic, J., Shavelson, R. J., & Bachman, M. (1999). On the development and evaluation of a shell for generating science performance assessments. International Journal of Science Education, 21(3), 293–315. https://doi.org/10.1080/095006999290714
Solano-Flores, G., Trumbull, E., & Nelson-Barber, S. (2002). Concurrent Development of Dual Language Assessments: An Alternative to Translating Tests for Linguistic Minorities. International Journal of Testing, 2(2), 107–129. https://doi.org/10.1207/s15327574ijt0202_2
Solano-Flores, G., Backhoff, E., & Contreras-Niño, L. Á. (2009). Theory of Test Translation Error. International Journal of Testing, 9(2), 78–91. https://doi.org/10.1080/15305050902880835
Solano-Flores, G., Chía, M., Shavelson, R.J., & Kurpius, A. (2010) GS.36. Translation Guide. AHELO Module A. Council for Aid to Education.
Solano-Flores, G., Shade, C., & Chrzanowski, A. (2014). Smarter Balanced Assessment Consortium: Item Accessibility and Language Variation Conceptual Framework. https://portal.smarterbalanced.org/library/en/item-accessibility-and-language-variation-conceptual-framework.pdf
Survey Research Center, Institute for Social Research, University of Michigan (2016). Guidelines for Best Practice in Cross-Cultural Surveys. http://ccsg.isr.umich.edu/images/PDFs/CCSG_Full_Guidelines_2016_Version.pdf
Stecher, B. M., Klein, S. P., Solano-Flores, G., McCaffrey, D., Robyn, A., Shavelson, R. J., & Haertel, E. (2000). The Effects of Content, Format, and Inquiry Level on Science Performance Assessment Scores. Applied Measurement in Education, 13(2), 139–160. https://doi.org/10.1207/S15324818AME1302_2
Tremblay, K., Lalancette, D., & Roseveare, D. (2012). Assessment of higher education learning outcomes (AHELO): Rationale, challenges and initial insights from the feasibility study. OECD (Vol. 1). https://doi.org/10.1007/978-94-6091-867-4 http://www.oecd.org/education/skills-beyond-school/AHELOFSReportVolume1.pdf
van de Vijver, F. J. R., & Poortinga, Y. H. (2004). Conceptual and methodological issues in adapting tests. Adapting Educational and Psychological Tests for Cross-Cultural Assessment, 39–63. https://doi.org/10.4324/9781410611758
van de Vijver, F. J. R., & Poortinga, Y. H. (2016). On item pools, swimming pools, birds with webbed feet, and the professionalization of multilingual assessment. In C. S. Wells & M. Faulkner-Bond (Eds.), Educational measurement: From foundations to future (pp. 273–290). The Guilford Press.
van de Vijver, F. J. R. (2016). Assessment in education in multicultural populations. In G. T. L. Brown and L. Harris (Eds) Handbook of Human and Social Conditions of Assessment. (pp. 436–453). New York, NY: Routledge
Zhao, X., & Solano-Flores, G. (2020). Testing across languages in international comparisons: cultural adaptation of consensus-based test translation review procedures, Journal of Multilingual and Multicultural Development. https://doi.org/10.1080/01434632.2020.1852242
Zahner, D., & Ciolfi, A. (2018). International Comparison of a Performance-Based Assessment in Higher Education. In: O. Zlatkin-Troitschanskaia, M. Toepper, H.A. Pant, C. Lautenbach and C. Kuhn (Eds.), Assessment of Learning Outcomes in Higher Education Cross-National Comparisons and Perspectives. Springer. https://doi.org/10.1007/978-3-319-74338-7_11.
Zlatkin-Troitschanskaia, O., Toepper, M., Molerov, D., Buske, R., Brückner, S., Pant, H., Hofmann, S., & Hansen-Schirra, S. (2018). Adapting and Validating the Collegiate Learning Assessment to Measure Generic Academic Skills of Students in Germany: Implications for International Assessment Studies in Higher Education. In: O. Zlatkin-Troitschanskaia, M. Toepper, H.A. Pant, C. Lautenbach and C. Kuhn (Eds.), Assessment of Learning Outcomes in Higher Education Cross-National Comparisons and Perspectives. Springer. https://doi.org/10.1007/978-3-319-74338-7_12.