Evaluating Change Processes. Assessing Extent of Implementation (Constructs, Methods and Implications)
Keywords:
Quality, Effectiveness, Change, Improvement, Equity, Innovation.Copyright (c) 2016 REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Abstract
Purpose: In far too many cases the initiatives to change schools by introducing new programs, processes and reforms has not resulted in obtainment of the desired outcomes. A major reason for limited outcomes suggested in this paper is that there has been a failure to learn from and apply constructs and measures related to understanding, facilitating and measuring dimensions of change processes. The aim of this paper is to introduce the three diagnostic dimensions of the Concerns Based Adoption Model (CBAM) along with illustrations of how each can be used to assess extent of implementation. Highlights from the four decades of development and use of each of these constructs are presented. Each of the constructs, Stages of Concern, Levels of Use and Innovation Configurations, is described along with review of the four decade story of its measurement development. Reference is made to selected studies. Implications of each construct for research, program evaluation and facilitating change processes are highlighted. The final section of the paper explores relationships between each construct. The conceptual explorations end with suggested implications for research, evaluation and practice. Throughout the author inserts short more personal anecdotes about the reasoning and experiences related to development and applications of each construct. The paper concludes with acknowledgement that other factors, especially leadership, are key to achieving implementation success. Design/methodology/approach: Introduction of three research-based constructs, Stages of Concern, Levels of Use and Innovation Configurations, their measures. Findings from selected studies are reviewed.
Findings: The three diagnostic dimensions of the Concerns Based Adoption Model have been applied with a wide range of education innovations, different contexts, and across nations and cultures. Research limitations/implications: Implementation needs to be determined through direct measurement. Practical implications: Extent of implementation needs to be determined directly in all treatment and comparison/control groups. Social implications: Without direct assessment of the extent of implementation the outputs and outcomes of new programs and innovations may not be determined.
Originality/value: The three Diagnostic Dimensions of the Concerns Based Adoption Model have been applied widely. The conceptual implications, especially when the three constructs are interconnected two at a time, offer important suggestions for future research and in program evaluations.
Downloads
References
Alquist, A. y Hendrickson, M. (1999). Mapping the configurations of mathematics teaching. Journal of Classroom Interaction, 34(1), 18-26.
Anderson, S.E. (1997). Understanding teacher change: revisiting the concerns based adoption model. Curriculum Inquiry, 27(3), 331-367.
Bailey, D.B. y Palsha, S.A. (1992). Qualities of the stages of concern questionnaire and implications for educational innovations. Journal of Educational Research, 85(4), 226-232.
Banathy, B.H. (1996). Designing Social Systems in a Changing World. Nueva York: Plenum.
Berman, P. y McLaughlin, M.W. (1978). Federal Programs Supporting Educational Change, Vol. VIII: Implementing and Sustaining Innovation. Santa Monica, CA: RAND.
Charalambous, C. y Philippou, G. (2010). Teachers’ concerns and efficacy beliefs about implementing a mathematics curriculum reform: integrating two lines of inquiry. Educational Studies in Mathematics, 75(1), 1-21.
Charters, W.W. y Jones, J.E. (1973). On the risk of appraising non-events in program evaluation. Educational Researcher, 2(11), 5-7.
Cheung, D. y Yip, D. (2004). How science teachers’ concerns about school-based assessment of practical work vary with time: the Hong Kong experience. Research in Science and Technological Education, 22(2), 153-169.
Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S.B., Imm P. y Sheldon, J. (2005). Developing a community science research agenda for building community capacity effective preventive interventions. American Journal of Community Psychology, 35(3/4), 142-157.
Christou, C., Eliophotou-Menon, M. y Philippou, G. (2004). Teachers’ concerns regarding the adoption of a new mathematics curriculum: an application of CBAM. Educational Studies in Mathematics, 57(2), 157-177.
Cuban, L. (1990). Reforming again and again and again. Educational Researcher, 19(1), 3-13.
Cuban, L. (2013). Why so many structural changes in schools and so little reform in teaching?. Journal of Educational Administration, 51(2), 1-32.
Feldman, J., Feighan, K., Kirtcheva, E. y Heereen, E. (2012). Aiming high: exploring the influence of implementation fidelity and cognitive demand levels on struggling readers’ literacy outcomes. Journal of Classroom Interaction, 47(1), 4-13.
Foster, L. y Nixon, M. (1975). The interview reassessed. The Alberta Journal of Educational Research, 21(1), 18-22.
Fuller, F.F. (1969). Concerns of teachers: a developmental conceptualization. American Educational Research Journal, 6(2), 207-226.
Fuller, F.F. y Bown, O.H. (1975). Becoming a Teacher. Teacher Education. Cuadragésimo séptimo anuario de la National Society for the Study of Education (pp. 25-52), Chicago, IL: University of Chicago Press.
Fuller, F.F., Pilgrim, G.H. y Freeland, A.M. (1967), Intensive Individualization of Teacher Preparation. Mental Health in Teacher Education, Sextagésimo cuarto anuario de la National Education Association (pp. 151-187). Washington, DC.
Gallagher, J.J. (1967). Teacher variation in concept presentation in BSCS curriculum programs. BSCS Newsletter, 30(3), 8-18.
George, A.A., Hall, G.E. y Stiegelbauer, S.M. (2006). Measuring Implementation in Schools: The Stages of Concern Questionnaire. Austin, TX: Southwest Educational Development Laboratory.
George, A.A., Hall, G.E. y Uchiyama, K. (2000).Extent of implementation of a standards-based approach to teaching mathematics and student outcomes. Journal of Classroom Interaction, 35(1), 8-25.
Guttman, L. (1954). A new approach to factor analysis: the radiz. En P. F. Larazsfeld (Ed.), Mathematical Thinking in the Social Sciences (pp.258-347). Nueva York: Free Press.
Guttman, L. (1957). Empirical verification of the radix structure of mental abilities and personality traits. Educational and Psychological Measurement, 17(9), 391-407.
Hall, G.E. (1999). Using constructs and techniques from research to facilitate and assess implementation of an innovative mathematics curriculum. Journal of Classroom Interaction, 34(1), 1-8.
Hall, G.E. (2010). Technology’s Achilles heel: achieving high-quality implementation. Journal of Research on Technology Education, 42(3), 231-263.
Hall, G.E. y Hord, S.M. (2011). Implementing Change: Patterns, Principles and Potholes. Upper Saddle River, NJ: Pearson Education.
Hall, G.E. y Loucks, S.F. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14(3), 263-276.
Hall, G.E. y Loucks, S.F. (1981). Program definition and adaptation: implications for Inservice. Journal of Research and Development in Education, 14(2), 46-58.
Hall, G.E., Dirksen, D.J. y George, A.A. (2006). Measuring Implementation in Schools: Levels of Use. Austin, TX: Southwest Educational Development Laboratory.
Hall, G.E., George, A.A. y Rutherford, W.L. (1979). Measuring stages of concern about the innovation. Informe 3032, ERIC, Servicio de reproducción de documetnos nº ED 147 342. Austin, TX: Research and Development Center for Teacher Education, The University of Texas at Austin.
Hall, G.E., Negroni, I.A. y George, A.A. (2013). Examining relationships between urban principal change facilitator style and student learning. International Journal of Leadership and Change, 1(1).
Hall, G.E., Wallace, R.C. y Dossett, W.A. (1973). A developmental conceptualization of the adoption process within educational institutions. Informe 3006, ERIC, Servicio de reproducción de documentos nº Ed 095 126. Austin, TX: Research and Development Center for Teacher Education, The University of Texas at Austin.
Hall, G.E., Loucks, S.F., Rutherford, W.L. y Newlove, B.W. (1975). Levels of use of the innovation: a framework for analyzing innovation adoption. The Journal of Teacher Education, 26(1), 52-56.
Hallinger, P. (2003). Leading educational change: reflections on the practice of instructional and transformational leadership. Cambridge Journal of Education, 33(3), 329-351.
Hallinger, P. y Heck, R.H. (2011). Conceptual and methodological issues in studying school leadership effects as a reciprocal process. School Effectiveness and School Improvement, 22(2), 149-173.
Hallinger, P. y Kantamara, P. (2001). Learning to lead global changes across cultures: designing a computer-based simulation for Thai school leaders. Journal of Educational Administration, 39(3), 197-220.
Hallinger, P. y Lee, M.S. (2011).Assessing a dacade of education reform in Thailand: broken promise or impossible dream?. Cambridge Journal of Education, 41(2), 139-158.
Hill, P.W. y Crevola, C.A. (1999). The role of standards in educational reform for the 21st century. En D.D. Marsh (Ed.), 1999 ASCD Yearbook: Preparing Our Schools for the 21st Century (pp. 117-142). Alexandria, VA: ASCD.
Hord, S.M., Stiegelbauer, S.M., Hall, G.E. y George, A.A. (2006). Measuring Implementation in Schools: Innovation Configuration. Austin, TX: Southwest Educational Development Laboratory.
Khoboli, B. y O’Toole, J. (2012). The concerns-based adoption model: teachers’ participation in action research. Systemic Practice and Action Research, 25(2), 137-148.
Lewis, D. (2011). It’s a matter of principal: examining relationships between leaders’ change facilitator style and students’ academic achievement. Tesis Doctoral. Universidad de Nevada.
Loucks, S.F., Newlove, B.W. y Hall, G.E. (1975). Measuring Levels of Use of the Innovation: A Manual for Trainers, Interviewers, and Raters. Austin, TX: R&D Center for Teacher Education, The University of Texas at Austin.
Liu, Y. y Huang, C. (2005). Concerns of teachers about technology integration in the USA. European Journal of Teacher Education, 28(1), 35-47.
McLaughlin, M. (1990). The Rand change agent study revisited. Educational Researcher, 19(9), 11-16.
Marsh, C.J. (1987). Implementation of a social studies curriculum in an Australian elementary school. Elementary School Journal, 87(4), 475-486.
Park, J.T.R. (2012). Teacher change in Bangladesh: a study of teachers adapting and implementing active learning into their practice. Tesis Doctoral no publicada. Toronto: Universidad de Toronto.
Rietz, D. (2012). IC Map, Common Core State Standards. Las Vegas, NV: Clark County School District.
Rogers, E. (2003). Diffusion of Innovations. Nueva York: The Free Press.
Showanasai, P., Lu, J.F. y Hallinger, P. (2013). Developing tools for research on school leadership development: an illustrative case of a computer simulation. Journal of Educational Administration, 51(1), 72-91.
Slough, S. y Chamblee, G. (2007). Looking beyond short-term implementation and low-level concerns: assessing the full impact of the Concerns Based Adoption Model (CBAM). Conferencia Internacional Proceedings of Society for Information Technology and Teacher Education (pp. 952-957). Chesapeake, VA.
Song, H.D.,Wang,W.T. y Liu, C.Y. (2011). A simulation model that decreases faculty concerns about adopting web-based instruction. Journal of Educational Technology and Society, 14(3), 141-151.
Stewart, S.K. (2012). Principal change facilitator style and student achievement: a student of schools in the middle. Tesis Doctoral. Las Vegas, NV: Universidad de Nevada.
The Network Inc (1997). Making Change Happen! Rowley, MA: The Network Inc.
Thornton, E. y West, C.E. (1999). Extent of teacher use of a mathematics curriculum innovation in one district: years 1 and 2 levels of use (LoU). Journal of Classroom Interaction, 34(1), 9-17.
Van den Berg, R. y Vandenberghe, R. (1981). Onderwijsinnovatie in verschuivend perspectief. Amsterdam: Uitgeverij Zwijsen.
Yan, S.C. y Huang, Y. F. (2008). A study of high school English teachers’ behaviors, concerns and beliefs in integrating information technology into English instruction. Computers in Human Behavior, 24(3), 1085-1103.
Yung, W.Y.A. (2012). Thoughts and practice of a Hong Kong teacher in mathematics alternative assessment via concerns-based adoption model. Tesis doctoral no publicada. Hong Kong: Universidad de Hong Kong.