PROMOTING LIBERAL EDUCATION THROUGH THE LONGITUDINAL STUDY OF CRITICAL THINKING: A RATIONALE AND PLAN
Keywords:
21st century skills, critical thinking, learning outcomes assessment, liberal education, Jesuit higher educationAbstract
In recent years, U.S. higher education has received much criticism for inadequately preparing students for the ‘real world.’ There is substantial empirical evidence (e.g., Arum & Roksa, 2011) that many students graduate with limited proficiency in key 21st century skills such as critical thinking (CT). Despite its importance in achieving personal advancement and professional success, there has been surprisingly little rigorous research on the development of CT during the undergraduate years. We believe this is a missed opportunity for higher education to better understand the various trajectories of CT development and to generate credible evidence to inform policies, programs, and practices – while also strengthening its standing among its various stakeholders. We argue that, despite a number of challenges, it is feasible to design and implement a comprehensive, longitudinal study of the development of CT (and related constructs). Although CT is an important learning objective at all colleges, it has special resonance for schools and programs that aim to provide a liberal education. For various reasons we propose that the initial study be conducted within a particular subset of such institutions; namely, a sample of American Jesuit colleges and universities. We discuss a number of technical issues germane to such a study, as well as the advantages and disadvantages of our preferred choice of an institutional sample.
Downloads
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
Anghel, E., Braun, H. I., Friedman, A. A., & Baez-Cruz, M. (in press). College students’ critical thinking: Assessment and interpretation. Journal of Higher Education Theory and Practice.
Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. University of Chicago Press.
Association of American Colleges and Universities. (2002). Greater expectations: A new vision for learning as a nation goes to college. Washington, DC. Retrieved from https://www.aacu.org/sites/default/files/files/publications/GreaterExpectations.pdf
Association of American Colleges and Universities. (2005). Liberal education outcomes: A preliminary report on student achievement in college. Retrieved from https://www.aacu.org/sites/default/files/files/LEAP/LEAP_Report_2005.pdf
Association of American Colleges and Universities. (2009). Critical thinking VALUE rubric. Retrieved from https://www.aacu.org/value/rubrics/critical-thinking
Association of American Colleges and Universities. (2020). What liberal education looks like: What it is, who it’s for, & where it happens. Retrieved from https://www.aacu.org/publications-research/publications/what-liberal-education-looks-what-it-who-it%E2%80%99s-and-where-it
Banta, T., & Pike, G.R. (2007). Revisiting the blind alley of value-added. Assessment Update, 19(1), 1-2, 14-15.
Bejar, I. I. (2012). Rater cognition: Implications for validity. Educational Measurement: Issues and Practice, 31 (3), 2-9.
Blaich, C., Bost, A., Chan, E., & Lynch, R. (2004). Defining liberal arts education (Unpublished manuscript). Center of Inquiry in the Liberal Arts at Wabash College.
Bok, D. (2006). Our underachieving colleges: A candid look at how much students learn and why they should be learning more. Princeton University Press.
Boston College. (2007). The journey into adulthood: Understanding student formation. Retrieved from https://www.bc.edu/content/dam/files/offices/mission/pdf1/umm1.pdf
Boyle, M. E. (2019). Global liberal education: Theorizing emergence and variability. Research in Comparative and International Education, 14(2), 231-248.
Braun, H. (2019). Performance assessment and standardization in higher education: A problematic conjunction? British Journal of Educational Psychology, 89(3), 429–440. doi: 10.1111/bjep.12274
Braun, H. I., Kirsch, I., & Yamoto, K. (2011). An experimental study of the effects of monetary incentives on performance on the 12th grade NAEP reading assessment. Teachers College Record, 113(11), 2309–2344.
Braun, H. I., Shavelson, R.J., Zlatkin-Troitschanskaia, O., & Borowiec, K. (2020). Performance assessment of critical thinking: Conceptualization, design, and implementation. Frontiers in Education, 5, 156. doi: 10.3389/feduc.2020.00156
Center of Inquiry at Wabash College. (n.d.). Wabash National Study participating institutions. Retrieved from https://centerofinquiry.org/wabash-national-study-participants/
Cheng, B., & Zhang, D. (2020). Cultivating citizens with Confucian cosmopolitanism: Defining the purpose of liberal arts education in the Asian context. Frontiers of Education in China, 15(4), 564-587.
Daley, B. (1988). “Splendor and wonder”: Ignatian mysticism and the ideals of liberal education. In W. J. O’Brien (Ed.), Splendor and wonder: Jesuit character, Georgetown spirit, and liberal education (pp. 1-22). Georgetown University Press.
Davey, T., Ferrara, S., Shavelson, R., Holland, P., Webb, N., & Wise, L. (2015). Psychometric considerations for the next generation of performance assessment. Washington, DC: Center for K-12 Assessment & Performance Management, Educational Testing Service.
Delbanco, A. (2012). College: What it was, is, and should be. Princeton University Press.
Finley, A. (2021). How college contributes to workforce success: Employer views on what matters most. Washington, DC: Association of American Colleges and Universities. Retrieved from https://www.aacu.org/sites/default/files/files/research/AACUEmployerReport2021.pdf
Godwin, K. A. (2013). The global emergence of liberal education: A comparative and exploratory study [Doctoral dissertation, Boston College].
Godwin, K. A., & Altbach, P. G. (2016). A historical and global perspective on liberal arts education: What was, what is, and what will be. International Journal of Chinese Education, 5(1), 5-22.
Grinnell College. (n.d.). Our mission. Retrieved from https://www.grinnell.edu/about/at-a-glance/mission
Hacker, J. S. (2019). The great risk shift: The new economic insecurity and the decline of the American dream. Oxford University Press.
Hatch, N. (2012, Nov. 8). Hope and challenge in the middle ground [Speech transcript]. In E. Owens (Chair), Boston College Symposium on Religion and the Liberal Aims of Higher Education [Symposium]. Boisi Center for Religion and American Public Life, Boston College, Chestnut Hill, MA. Retrieved from
https://www.bc.edu/content/dam/files/centers/boisi/pdf/f12/RLE%20Hatch%20Keynote.pdf
Indiana University (2019). FSSE 2019 frequencies: FSSE 2019 aggregate. Retrieved from http://fsse.indiana.edu/pdf/FSSE_IR_2019/summary_tables/FSSE19_Frequencies_(FSSE_2019).pdf
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73.
Katz, S. N. (2008, May 23). Taking the true measure of a liberal education. The Chronicle of Higher Education. Retrieved from http://chronicle.com/weekly/v54/i37/37a03201.htm
King, P. M., & Kitchener, K. S. (2002). The reflective judgment model: Twenty years of epistemic cognition. In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 37–61). Lawrence Erlbaum Associates, Inc.
Kitchener, K. S., & King, P. M. (1985). The reflective judgment scoring manual (Unpublished manuscript).
Klein, S., Benjamin, R., Shavelson, R., & Bolus, R. (2007). The Collegiate Learning Assessment: Facts and fantasies. Evaluation Review, 31(5), 415–439.
Klein, S. C., Liu, O. L. E., Sconing, J. A., Bolus, R. C., Bridgeman, B. E., Kugelmass, H. C., ... & Steedle, J. C. (2009). Test validity study (TVS) report. Retrieved from https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.3647&rep=rep1&type=pdf
Kronman, A. (2008). Education's end: Why our colleges and universities have given up on the meaning of life. Yale University Press.
Lane, S., & Stone, C. A. (2006). Performance assessment. In R. L. Brennan (Ed.), Educational measurement (4th ed.) (pp. 387-432). Rowman & Littlefield Publishers.
Liu, O. L., Frankel, L., and Roohr, K. C. (2014). Assessing critical thinking in higher education: current state and directions for next-generation assessments (ETS RR–14-10). Educational Testing Service. doi:10.1002/ets2.12009
McClellan, C. A. (2010, Feb.). Constructed response scoring doing it right. R&D Connections, 13.
Mejía, A., Mariño, J. P., & Molina, A. (2019). Incorporating perspective analysis into critical thinking performance assessments. British Journal of Educational Psychology, 89(3), 456-467.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23. doi: 10.3102/0013189x023002013
Minnameier, G., & Hermkes, R. (2020). Learning to fly through informational turbulence: Critical thinking and the case of the minimum wage. Frontiers in Education, 5, 573020. doi: 10.3389/feduc.2020.573020
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence?centered design. ETS Research Report Series, 2003(1), i-29.
Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice, 25(4), 6–20. doi: 10.1111/j.1745-3992.2006.00075.x
Morrill, R. (2012, Nov. 9). Religion and a larger vision for liberal education [Speech transcript]. In E. Owens (Chair), Boston College Symposium on Religion and the Liberal Aims of Higher Education [Symposium]. Boisi Center for Religion and American Public Life, Boston College, Chestnut Hill, MA. Retrieved from https://www.bc.edu/content/dam/files/centers/boisi/pdf/f12/RLE%20Morrill%20Keynote.pdf
O'Malley, J. W. (2015). Jesuit schools and the humanities yesterday and today. Studies in the Spirituality of Jesuits, 47(1), 1-34.
Oser, F. K., & Biedermann, H. (2020). A three-level model for critical thinking: Critical alertness, critical reflection, and critical analysis. In O. Zlatkin-Troitschanskaia (Ed.), Frontiers and Advances in Positive Learning in the Age of Information (PLATO) (pp. 89-106). Springer, Cham. doi: 10.1007/978-3-030-26578-6_7
Nagel, M.-T., Zlatkin-Troitschanskaia, O., Schmidt, S., & Beck, K. (2020). Performance assessment of generic and domain-specific skills in higher education economics. In O. Zlatkin-Troitschanskaia, H. A. Pant, M. Toepper, & C. Lautenbach (Eds.), Student learning in German higher education (pp. 281–299). doi: 10.1007/978-3-658-27886-1_14
Pascarella, E. T. (2007). Methodological report for Wabash National Study of Liberal Arts Education. Iowa City, IA: Center for Research on Undergraduate Education. Retrieved from https://centerofinquiry.org/wp-content/uploads/2017/04/WNSLAE_Research_Methods_March_2008.pdf
Pascarella, E. T., & Blaich, C. (2013). Lessons from the Wabash National Study of Liberal Arts Educaton. Change: The Magazine of Higher Learning, 45(2), 6–15. https://doi.org/10.1080/00091383.2013.764257
Pascarella, E. T., Wang, J. S., Trolian, T. L., & Blaich, C. (2013). How the instructional and learning environments of liberal arts colleges enhance cognitive development. Higher Education, 66(5), 569–583. https://doi.org/10.1007/s10734-013-9622-z
Pratt, M. (2016). AUC 21: Our community’s vision for excellence, diversity and global citizenship. Retrieved from https://www.auc.nl/about-auc/mission-and-values/auc21/auc21.html
Shavelson, R. J. (2013). On an approach to testing and modeling competence. Educational Psychologist, 48(2), 73-86.
Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30(3), 215-232.
Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., & Marino, J. P. (2019). Assessment of university students’ critical thinking: Next generation performance assessment. International Journal of Testing, 19(4), 337-362.
Spellings, M. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, D.C.: U.S. Department of Education. Retrieved from https://www2.ed.gov/about/bdscomm/list/hiedfuture/reports/pre-pub-report.pdf
Stassen, M. L., Herrington, A., & Henderson, L. (2011). Defining critical thinking in higher education: Determining assessment fit. In J. E. Miller & J. E. Groccia (Eds.), To improve the academy: Resources for faculty, instructional, and organizational development (Vol. 30) (pp. 126-141). John Wiley & Sons.
Van der Wende, M. (2011). The emergence of liberal arts and sciences education in Europe: A comparative perspective. Higher Education Policy, 24(2), 233-253.
Van der Wende, M. (2013). Trends towards global excellence in undergraduate education: Taking the liberal arts experience into the 21st century. International Journal of Chinese Education, 2(2), 289-307.
Wheeler, P., & Haertel, G. D. (1993). Resource handbook on performance assessment and measurement: A tool for students, practitioners, and policymakers. Owl Press.
Wolfe, E.W. & Song, T. (2016). Methods for monitoring and document rating quality. In H. Jiao & R.W. Lissitz (Eds.), The next generation of testing (pp. 107-142). Information Age Publishing.
Wortham, S., Love-Jones, R., Peters, W., Morris, S., & García-Huidobro, J. C. (2020). Educating for comprehensive well-being. ECNU Review of Education, 3(3), 406-436.
Zahner, D. (2013). Reliability and validity–CLA+. New York: Council for Aid to Education.
Zhang, M. (2013). Contrasting automated and human scoring of essays. R & D Connections, 21, 1-11.
Zlatkin?Troitschanskaia, O., Shavelson, R. J., Schmidt, S., & Beck, K. (2019). On the complementarity of holistic and analytic approaches to performance assessment scoring. British Journal of Educational Psychology, 89(3), 468-484.